journals share criteria used in evaluating the merit of articles, including significance levels and the intellectual contribution of the article. Such criteria may be shared across journals, but journals obviously differ in the content of articles they publish; even within a given journal, editors in chief (who usually serve a specific length of time) may be interested in promoting a particular focus within the discipline. Furthermore, specialized journals are being added to address emerging fields, especially in technology.
Significance Levels
Null hypothesis: Hypothesis that there are no group differences or relationships between variables.
Null hypothesis significance testing procedure (NHSTP): Using statistical inference, a procedure for evaluating whether the null hypothesis should be rejected.
Alpha level: Probability (usually set at .05) of incorrectly rejecting the null hypothesis.
p value: Probability value based on the characteristics of the observed data used for hypothesis testing.
Confidence interval: Shows range of values that you can be sure contains the population mean a certain percentage of the time (e.g., 95%).
In most research that uses inferential statistics (and most research in the social and behavioral sciences does), the aim of the project is to be able to reject the null hypothesis of no difference between sampled populations. This process is sometimes called the null hypothesis significance testing procedure (NHSTP). As a scientific community, researchers therefore need a decision rule about when differences exist (that is, when to reject the null hypothesis represented by the data). Most members of the scientific community accept the standard of a statistical outcome with a probability value of .05 (5 / 100) or less frequent. The value .05, called the alpha level, represents the likelihood that in only 5 of 100 cases would the outcome occur by chance. This number is stated (acknowledged) before the analyses are conducted. This p value (or probability value) of .05 is the standard used by most journals for having significant results. In addition to the p value, it is common for journals to request that confidence intervals be provided. For example, a 95% confidence interval shows the range of values within which you can be sure the population mean is contained 95 out of 100 times. Current standards for APA journals are “estimates of appropriate effect sizes and confidence intervals” as a minimum, according to the APA manual (2020, p. 87).
It should be noted that at least one journal (Basic and Applied Social Psychology) has banned the use of NHSTP in papers submitted to the journal. In its place, the journal editors call for:
Strong descriptive statistics, including effect sizes. We also encourage the presentation of frequency or distributional data when this is feasible. Finally, we encourage the use of larger sample sizes than is typical in much psychology research, because as the sample size increases, descriptive statistics become increasingly stable and sampling error is less of a problem. (Trafimow & Marks, 2015, p. 1)
Most journals are interested in publishing results that advance the literature in some way. For that reason, research in which the null hypothesis is accepted (i.e., not rejected) is unlikely to be published.
In the most recent edition of the Publication Manual of the American Psychological Association (2020), an entire chapter is devoted to journal article reporting standards (JARS), covering standards for quantitative, qualitative, and mixed methods (i.e., containing both quantitative and qualitative) research. Additional resources beyond that chapter are provided by the APA Style JARS website (https://apastyle.apa.org/jars).
Reviewer Selection
When a journal states that it is a peer-reviewed journal, what that means is that people knowledgeable about the topic in question have been invited by the journal editor to evaluate the quality of the paper. Usually a journal specifies particular categories for comment, including the currency and coverage of the literature, as represented in the introduction, the theoretical grounding, the scope of the research question, methodological issues (e.g., sample size and adequacy of measures), appropriateness of statistical approach, significance of findings, and contribution of the study to the literature (answering the “what’s new and noteworthy here?” question).
Journal editors are increasingly aware that fraud occurs in the reviewer process (e.g., authors create false e-mail addresses for reviewers and suggest these fictitious reviewers provide a review—essentially generating the reviews for their own manuscripts). In one instance, a reputable journal retracted 60 published articles after it was determined that there was author and reviewer fraud (https://www.sagepub.com/en-us/nam/press/sage-statement-on-journal-of-vibration-and-control). Many journals subscribe to the Committee on Publication Ethics (COPE) for reviewers. The guidelines are available on the COPE website: http://publicationethics.org/resources/guidelines.
Committee on Publication Ethics (COPE): Sets standards for editors and reviewers who evaluate research.
In addition to what one would normally expect of a reviewer (e.g., expertise in the subject area and agreement to keep the manuscript and review details confidential), providing fair reviews involves degrees of separation. The reviewer should not have collaborated with the author (usually there is a time frame, for example, 3 years), be at the author’s institution, or in other ways be familiar with the manuscript that is being reviewed.
Intellectual Contribution of the Article
Large numbers of manuscripts are submitted for publication review, and few can be accepted for publication (e.g., the rejection rate for the Journal of Personality and Social Psychology was 87% in 2017). The statistics provided by the APA indicated that 1,051 manuscripts were submitted to that journal in 2017. Given the number of submissions, research must be more than methodologically sound and well written; research must make a contribution and advance the state of knowledge in a meaningful way. The extent of the contribution and whether the research is of sufficient scope or merit is a subjective decision, which is why the expertise and fair-mindedness of the peer reviewers is important.
With the recent work of the Reproducibility Project (see Chapter 1), which is determining whether the published findings of 100 correlational and experimental studies in psychology are reproducible, journals may be more welcoming of replications than in the past. Because replications (for the most part) have not been judged to advance the field, there has been a bias against publishing replications to this point.
Editorial Policy
As suggested in their titles, journals are interested in publishing work about a specific area of interest (e.g., Health Environments Research & Design Journal, Cultural Sociology, and Attachment and Human Development). Editors generally serve for a particular period of time (e.g., 3 years) and may have a specific focus they want to emphasize during their tenure. For example, for Ronald Brown, the outgoing editor of Professional Psychology: Research and Practice, the focus of the journal was on “publishing research that could increase access to mental health treatments” (http://www.apa.org/pubs/journals/pro/).
Specialized Journals
As knowledge expands, new journals are introduced to provide a home for those topics, whether related to technology (e.g., Digital Applications in Archaeology and Cultural Heritage, begun in 2014); mental health (Spirituality in Clinical Practice, begun in 2013); or demographic groups that have been underrepresented in the literature (e.g., Asian American