Democracy has been called the least worst system of government. Peer review is the least worst system for assessing the merit of scientific work.
Peer review is the written evaluation of an article by other experts in the field. Although it looks like an evaluation by equals, the power imbalance created by the roles of reviewer and reviewer distorts the relationship and affects the tone of the review. Reviews can be condescending, demanding and mean.
It is painful to read harshly worded criticism of work that took hundreds or thousands of team hours and was submitted with hope and good faith. From our experience, we know that reviews can be accurate, strong, and make every scientific point while using helpful and encouraging language and tone.
We are a team of editors of a Canadian open access renal journal, the Canadian Journal of Kidney Health and Disease. When we founded our journal in 2014, the supportive journal was the first of our guiding principles. Since then, we have written as editors, selected reviewers who write in support, and helped train the next generation of Canadian kidney scientists to perform comprehensive, rigorous, and caring reviews.
Supported by a broader group of like-minded people from multiple disciplines, we recently published an editorial outlining these principles. A dozen other kidney journals have expressed support for the idea, with Opinions on nature Nephrology, NDT and Pediatric nephrology publish coordinated editorials reaffirming the principles of constructive criticism.
The long research process
Scientific articles condense a large amount of work into a structured format, usually no more than four to eight times the length of that article. The work of a paper begins with an idea that may be developed by the team for a year or more before crystallizing into a funding request, which may go through rounds of revisions.
Once funded, people and budgets are assigned to the project and the work continues. The work can involve the time of several team members for months or even years.
When the job is done, they write an article, detailing what they did, how and why, what they found, and what they think it means. This article itself is often the product of hundreds of hours of work, with multiple authors contributing their specific expertise and working on the overall message.
The journal receives the manuscript and appoints an editor, who appoints peer reviewers. Peer reviewers are other scientists working on similar topics. They must be completely independent of the people who write the journal. With a few notable exceptions, most journals use single-mask peer review: the reviewer sees the authorship of the article, but the authors of the article do not see who wrote the review.
Peer reviewers are not paid or rewarded for their review of the manuscript – they assume it as part of the work of academic life. It is essentially an unpaid activity carried out by people who are themselves authors. It varies by discipline, but in biomedicine they can spend three to six hours on an exam.
How does this altruistic activity, undertaken by a critic who knows the authorship very well, lead to so much pain and frustration for other authors?
We believe that scientists sometimes confuse harshness with intellectual rigor and that a reviewer’s experience of harshness in critiques of their own work, amplified by the power imbalance between reviewer and reviewer, leads to the perpetuation of harsh and pointless criticism. Other reviewers and editors avoid these pitfalls altogether.
“This seems to me to be one of your first attempts at scientific publishing, and I can understand that you were also writing in a non-native language” wrote an anonymous reviewer to a mid-career woman scientist with 13 peer-reviewed publications. “I just wanna quit today,” she wrote.
But she won’t. Scientists are ready to receive these kinds of comments and be hurt again and again in the name of science. As editors, we believe there is a better way – that comments should be rigorous, but will be more easily incorporated if kindly given, to the advancement of science.
These are not new ideas. In 2006, Professor Mohan Dutta suggested 10 commandments for examiners, all of which focus on the collaborative nature of the relationship between examiner and examinee. Advice to reviewers often includes a recommendation to write constructively, although this is sometimes phrased as something like “write constructively, then turn to criticism”, as if these are mutually exclusive.
We can take this principle one step further, and through our community of kidney medicine reviewers, we and other kidney journals are committed to reviewing kindness. Dutta’s 10th commandment is “do unto others as you would have them do unto you”. Every branch of science would be improved by the implementation of this idea.