Journal of Personality Assessment: Reviewer Guidelines

A review is most helpful when it is detailed and when its suggestions are well documented. We encourage reviewers to focus on the following components of a manuscript: strengths and weaknesses of the methods, integrity of the data collection, reliability of the predictor and criterion variables, ability of the statistical analyses to answer the question(s) posed, attention to prior relevant literature, use of a sample that is representative of the population to which generalizations are made, reasonableness of the inferences drawn from the data, and clarity of the presentation. In combination, these factors help determine whether the manuscript makes a valuable contribution to the personality assessment literature.

When writing a review, it is easy to focus on problems or limitations. These are very important, but be sure to recognize strengths in the manuscript. Write in the tone you would use when communicating with a colleague. Appreciate the hundreds of hours that were spent conducting the research and preparing the manuscript. At the same time, be frank about problems and limitations you see in the hypotheses, design, analyses, or writing. Overall, use positively toned, constructive, and specific feedback to encourage authors to strengthen their work. We want authors to feel they have gained from the JPA review process. And feel free to sign your reviews.

Please use JPA's statement of scope to ground your thinking about the overall suitability of a manuscript. In addition, our submission instructions outline several specific requirements and recommendations that we would like you to keep in mind as you review a manuscript. Our statement of scope and submission instructions can be found here.

In particular, we have several guidelines for traditional research submissions. Authors are asked to follow the recommendations in "Statistical Methods in Psychology Journals: Guidelines and Explanations" by Wilkinson and the APA Task Force on Statistical Inference (1999, American Psychologist, 54, 594-604; or http://content.apa.org/journals/amp/54/8/594.pdf), so please familiarize yourself with this article.

Consistent with Wilkinson et al. (1999), authors are required to publish measures of effect size (e.g., r, Cohen's d), along with traditional indices of statistical significance. We require effect sizes to be reported for all analyses, regardless of their statistical significance. Because personality tests rarely use inherently meaningful units of measurement, virtually all studies should report findings in terms of r or d, though authors may also report an odds ratio for a 2 × 2 table. In general, we discourage exploratory, shot-in-the-dark analyses. In addition, authors should not report effect sizes derived from unfocused statistical analyses (e.g., F with numerator df > 1, chi-square with df > 1, stepwise regression). Rather they should rely on specific contrasts that directly test a directional hypothesis. To help authors with effect sizes, we developed a guide to compute them using SPSS and SAS syntax, which is available here.

Although it is not a JPA requirement, when authors describe findings from other studies (e.g., in the Introduction or Discussion section), we encourage them to compute and report effect sizes to characterize those findings. For example, rather than saying results were "significant" or the study found "an association," we would like authors to report the findings using r or Cohen's d. When authors begin to attend more systematically to the size of effects found in prior research, hypotheses will become more fully grounded and the scientific literature will become more cumulative and directed. Knowing the magnitude of prior findings makes it easier for new research to estimate statistical power and craft hypotheses that build on and incrementally extend the earlier findings.

For studies that examine the diagnostic accuracy of tests, we strongly encourage authors to follow the Standards for Reporting Diagnostic Accuracy (STARD) guidelines. We provide several resources related to the STARD guidelines here. These include an editorial explaining why JPA endorses the STARD initiative and a link to the actual guidelines. In addition, we provide downloadable files for the STARD checklist and flowchart. Authors are encouraged to submit the checklist and flowchart with their manuscript. The web page also provides a primer written by David Streiner explaining how to compute a wide range of test statistics from 2 × 2 tables, as well as a corresponding Excel program that computes all of these statistics.

It is most helpful when your review lists points in a sequentially numbered format because the Action Editor can then easily reference specific comments when communicating with the author. Please do not indicate whether you think the paper should be accepted or rejected in the text of your review. This information will be indicated on the Reviewer Feedback Form.

If you believe the manuscript focuses on issues that fall outside your area of expertise, you may return it. Alternatively, you may provide feedback on the elements you feel comfortable reviewing while noting the areas you did not feel prepared to address.

The manuscript is a privileged document and you should not cite or refer to the work before it is published. You also should not forward the manuscript to others or discuss it with the author. Finally, you should respect the confidentiality of the unpublished work and the author's proprietary rights to the ideas and material contained in the manuscript. Reviewers may not use this material to advance their own work or the work of others without the author's consent.

If reviewing a manuscript poses a conflict of interest that may make it difficult to offer a balanced and fair judgment, please notify the Action Editor and return it. Possible instances when this might occur include reviewing the manuscript of a friend, former student, or co-worker; or reviewing research on an instrument in which you have a financial stake.