Best Practices for Peer Review

Section 3

Working with Peer Reviewers

 

Guidelines for reviewers

Presses often provide reviewers a short list of questions to guide their evaluation of a project in order to improve the chances that the review will address the points most pertinent to a press’s publication decision. This list should ask reviewers to focus on key areas such as the quality of argument, evidence, and writing in the context of subject-specific and manuscript-specific issues. Just as different reader criteria are brought to different projects, so too is it useful to have a range of reviewer questions tailored to particular kinds of projects, such as scholarly monographs, course books, trade nonfiction, fiction, or poetry. (See Do different types of books require different types of peer reviews? above.) The list may end by asking reviewers to recommend whether a project should be (1) rejected, (2) revised and resubmitted, or (3) accepted for publication. Although very important, such opinions should not outweigh the AE's own judgment of the manuscript’s potential and his or her assessment of the reviews. It is not uncommon for two reviews to offer similar feedback and yet make different recommendations about publication.

AEs should explain to reviewers, either in the initial query or when sending the materials provided for the review process, that their reports will be confidential and their identities concealed from the author. The query or the review guidelines should specify who will see the reports (AEs and their assistants, the author, faculty board members) and who will know the reviewers' identities (AEs and their assistants, other press staff, faculty board members). (See Confidentiality and anonymity in the peer review process above.)

 

How should readers be remunerated for reports?

Presses generally offer readers an honorarium in return for their evaluations of projects. That the compensation is an honorarium, not a fee, is important. First, using the term "honorarium" highlights the fact that peer review is a responsibility academics bear as members of the scholarly community. Second, the term points to the fact that a press is not buying an expert opinion in the way that, say, a defense attorney may pay an expert to offer a particular reading of evidence. A peer reviewer is expected to provide an unbiased, candid, well-supported evaluation of a project’s merits.

An honorarium generally takes one of two forms. A reviewer may be offered a cash payment or a selection of books from a press’s catalog up to a certain dollar amount (usually larger than the amount of the cash payment, as the unit cost of books is significantly lower for publishers than for retail buyers). Some presses offer a combination of cash and books. AEs should tell a potential reader what the honorarium is in their initial queries, before the review begins. If certain categories of books are ineligible for selection, such as distributed books from other publishers, this should be noted on the honorarium form.

Honoraria amounts vary widely by presses, and AEs should be familiar with their own press’s conventions. The amounts should reflect the scope of the work the reviewer is being asked to do; honoraria are typically larger for full manuscripts than for proposals. In addition, asking a peer reviewer to evaluate a particularly long manuscript or to provide a report in an unusually short amount of time often warrants increasing the amount of an honorarium. (See What is a reasonable amount of time to allow a peer reviewer to read and report on a project? below.) Honoraria are paid on receipt of reports. Also, if the press ultimately publishes the work in question, the reviewer should receive a gratis copy.

 

What is a reasonable amount of time to allow a peer reviewer to read and report on a project?

While it is generally in both an author’s and a press’s interests to receive reports as quickly as possible, AEs should be aware that properly reviewing a manuscript is both time- and labor-intensive. It is customary to give peer reviewers at least six to eight weeks to review a full manuscript and three to four weeks to review a proposal, though in competitive situations an AE may request a faster turnaround. It may also be necessary to allow more time for particularly long or complex projects. AEs and reviewers should agree on a deadline before the process starts, and it is generally recommended that an AE or assistant check in with reviewers as the deadline approaches. AEs or their assistants should track due dates for reviews in some kind of database—an essential tool, given the volume of projects an AE may have out for review at any given time.

 

What should an AE do when a peer reviewer fails to produce a report within an acceptable period of time? Can compensation be withheld in such cases?

Given the time it can take to secure appropriate readers for a project, AEs should accommodate modest delays (one to two weeks). However, a reviewer who misses an initial deadline is likely to miss another one, and AEs should exercise caution in granting longer extensions (a month or more). If a second deadline passes without a review, the AE should take steps to line up an additional reader rather than risk longer delays for the author. A new reader should also be found if a reader does not respond to follow-up queries. In such cases, the AE should notify the original reader that the press no longer expects a report and will not compensate him or her. There is always the possibility, however, that a late review will surface, and an AE will need to decide whether to provide the normal honorarium in such cases.

As challenging as the lack of review can be, AEs also face situations in which a review is unsatisfactory: either it fails to address the questions posed, it does so without sufficient detail, or its assessment is unclear. AEs should first try to encourage the reviewer to flesh out the report, but if a full review does not materialize, the honorarium may be prorated. Similarly, if a reviewer fails to submit a review, the press is not obliged to pay the honorarium. If, however, the press decides it no longer needs a commissioned report (for example, if a project is lost to another press in competition), the reviewer should still be offered the honorarium, even if the report has not yet arrived.

 

What should an AE do about an obviously incompetent, biased, or ad hominem report? Can a commissioned report be disregarded? What is the best way to communicate such concerns to a peer reviewer?

Peer review is meant to provide an honest and rigorous assessment of the merits of a project, and archetypical reports can be as much an art form as the manuscripts under consideration. The ideal report offers sound advice for helping a project realize its fullest potential. It is the AE’s responsibility, in turn, to assess the reviews to ensure that reviewers have met expectations. Reports that do not engage with the content of a work, that offer insufficient support for a reviewer’s criticisms, or that evince animus toward authors or their ideas do not provide useful guidance to AEs, authors, or faculty boards.

Upon receipt of an opaque or problematic review, the AE should request amplification or clarification for the sake of the author and the press. Specificity is important in such situations. The ultimate goal is to secure a suitable review, and so giving the reviewer an opportunity to revisit the report is in most cases worthwhile. On the other hand, if a report is flawed because the reader is clearly biased against an author or his approach to a subject, there is little to be gained in returning to that reviewer. The decision to address flawed reviews directly can be a vexed one for AEs, who should discuss such reports with their supervisors before proceeding.

If the report is biased against the author's approach, the AE should consider it in the context of the scholarly discipline in question. If the field is deeply divided and the author and reviewer are on opposing sides of that divide, then the review may help the author anticipate and address criticisms. Ideally, the AE will be aware of such disciplinary politics and will take them into consideration in selecting peer reviewers. If the bias is against the author personally, the review should be disregarded because it does not assess the manuscript itself. For the sake of expediency, it is often best to extend the usual courtesy to such a reviewer and process his or her honorarium, even if the report is disregarded.

The AE need not share an unfairly prejudiced or hostile report with an author; instead, the AE should seek an alternate peer reviewer. Presses differ in whether they include biased reports in packets for the faculty board. If such a report is included, the AE‘s statement should take care to contextualize the review and its criticisms and explain that it has not been shared with the author.

 

If a report is delayed or otherwise unacceptable, what should the AE say to the author? Should the author be told the reviewer is at fault, or is it best to simply cite unavoidable delays?

In general, transparency in the author-editor relationship is paramount, and the AE should tell the author about any delays in the review process promptly. However, AEs need not always reveal the source of the delay. In deciding whether to inform an author that a delay is due to a reviewer’s tardiness, the AE should avoid giving the impression that the report is hastily or haphazardly prepared. Peer reviews need to carry authority with an author because they form, at least in part, the basis of a press’s judgment about whether to accept or reject a project. If a reviewer submits a well-constructed but delayed review, its tardiness should not undermine its force. If a reader fails to submit a review, an AE should alert the author of the reader’s unresponsiveness, though ultimately it is the role of an AE to manage the peer review process as efficiently as possible.

 

What if a reviewer jeopardizes a project by revealing his or her role to others in the field?

In spite of the press’s best intentions in assuring the confidentiality of peer reviews (see Confidentiality and anonymity in the peer review process above), in some cases a reviewer may discuss the project with interested parties other than the author. This discussion may jeopardize a book that is, for example, based on confidential interviews or takes a stand on a controversial issue. In such cases, AEs must weigh the likely impact of the revelation in deciding whether to disregard the report. Will public knowledge of the reviewer's identity undermine the legitimacy of the report with the author or other scholars in the field? Has the revelation reshaped the readership for the work or its public reception? Does it potentially poison the author's relationship with the subjects of his or her research or employer? Where the revelation has had a significant impact on the likely success of the work, the press may need to reconsider its decision to publish it.

 

Continue to Section 4: Sharing Peer Reviews with Authors >
< Back to Section 2: Selecting Peer Reviewers
< Return to Table of Contents

 

by-nc-sa

(c) 2016 by the Association of American University Presses. Best Practices for Peer Review is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-sa/4.0/.