Reviewing instructions for the journal Psychological Science which probably started last year. More Methods reporting, part of effort to improve reproducibility.
(1) Manuscript Evaluation Criteria. Editors and reviewers alike are now asked to evaluate submissions with three questions in mind: (a) What will the reader of this paper learn about psychology that she or he did not know (or could not have known) before? (b) Why is that knowledge important for the field? (c) How are the claims made in the article justified by the methods used? Manuscripts that provide clear and compelling answers to these “What,” “Why,” and “How” questions will have the best prospects of being accepted for publication. For background on the rationale for these questions, please see this editorial: http://pss.sagepub.com/content/early/2013/11/25/0956797613512465.full/
2) Enhanced Reporting of Methods. Another new feature of the manuscript submission process for 2014 is a section containing checkboxes for four Research Disclosure Statement items (very similar to existing items confirming that research meets ethical guidelines, etc.). Submitting authors must check each item and, in doing so, actively declare that:
(a) the total number of excluded observations, and the reasons for making these exclusions, have been reported in the Method section(s),
(b) all independent variables or manipulations, whether successful or failed, have been reported in the Method section(s) [Authors skip this step if there were no independent variables or manipulations, as in the case of correlational research.],
(c) all dependent variables or measures that were analyzed for this article’s target research question have been reported in the Methods section(s), and
(d) the Method section(s) describe how sample size was determined and the rule for stopping data collection
As a reviewer, you don’t need to worry about the checkboxes or anything else to do with the manuscript submission process. The key point here is that the manuscript you’ll be reading will contain four categories of methodological details—Exclusions, Manipulations (if any), Measures, and Sample Size—that have not required disclosure under past reporting standards (of Psychological Science in particular or psychology journals in general) but that are important for interpreting research findings. I hope you will find these methodological details helpful to you in evaluating PSCI-15-0379.
(3) New Word Limits. For manuscripts submitted after January 1, 2014, the Method and Results sections of a manuscript will be excluded from the word limits on Research Articles and Research Reports. The new limits on Research Articles and Research Reports will be 2,000 and 1,000 words, respectively, and will include introductory and Discussion sections, as well as notes, acknowledgements, and appendices. The purpose of eliminating strict limits on the Method and Results sections is to afford authors the opportunity to report what they did, and what they found, in a manner that is clear, concise, and complete (see more details here: http://www.psychologicalscience.org/index.php/publications/observer/2013/november-13/whats-new-at-psychological-science.html
). I welcome any ideas you may have for improving the exposition of the paper you’re reviewing.
(4) Embracing the “New Statistics.” Psychological Science now recommends the use of the “new statistics”—effect sizes, confidence intervals, and meta-analysis—to avoid problems associated with null-hypothesis significance testing (NHST). The journal sought to aid researchers in shifting from reliance on NHST to estimation and other preferred techniques by publishing a tutorial by Geoff Cumming, a leader in the new-statistics movement, that includes examples and references to books, articles, software, and online calculators that will aid authors in understanding and implementing estimation techniques in a wide range of research settings. The tutorial is available here: http://pss.sagepub.com/content/early/2013/11/07/0956797613504966.full
Please note that the journal recommends, but does not require, adoption of the new-statistics approach. In certain areas of research—for instance, function magnetic resonance imaging (fMRI) when a brain region of interest hasn’t been pre-identified—NHST methods of analysis may be more appropriate.
(5) Supplemental Online Material. Since January 2012, Psychological Science has allowed the online publication of two types of supplemental material. One type, referred to as SOM-R, includes material that has undergone both an initial review (by two members of the editorial team) and an extended review (by two or more external referees). The other type, SOM-U, includes unreviewed material, or information that has not been vetted by either the editors or the external referees.
The distinction between the two types of supplemental material is discussed in greater detail here (http://www.psychologicalscience.org/index.php/publications/journals/psychological_science/ps-submissions#SM
). Here let me emphasize two key points. First, feel free to skip over any SOM-U material that the author may have uploaded, but please look carefully at any SOM-R material. Second, in the Submission Guidelines, authors are advised that the editors take the adjective “supplemental” seriously: both SOM-R and SOM-U should include the sort of material that enhances the reader’s understanding of an article but is not essential for understanding the article. Thus, should you come across any SOM-R material you consider essential, please mention that in your review, so that the authors will know to move the material into the main article.