Reviewing the reviewers

A new online award that recognises the contributions of highly-productive peer reviewers also highlights biases in the system that underscores scholarly publishing.

7 November 2016

age photostock

The inaugural Sentinels of Science Awards in September named three researchers who had made the greatest contribution to peer review in the previous 12 months, cumulatively reviewing almost 1500 papers.

The awards, announced by the website Publons, recognise researchers for peer reviewing by encouraging them to keep a record of their reviews on the site. The site also listed the most prolific reviewers in a range of scientific fields.

Microbiology researcher Elisabeth Bik, from Stanford University, was recognised for her contributions to the field of immunology and microbiology and was ranked the 14th most prolific reviewer in that field. She reviews about 40 papers a year, spending about five to six hours on each article.

Bik believes reviewing, which is unpaid and undertaken by scientists in addition to their research is part of their role. “I see peer review as part of my academic job, because it ensures that published science is of a good quality,” Bik told Nature Index. She says reviewing also helps her keep up-to-date with her field and improve her own writing.

Gender imbalance

But, while Bik welcomes the recognition of peer reviewers, she has also observed a clear gender disparity in Publons' results. The top three ‘Sentinels of Science’ were male, and significantly more male reviewers were listed in the top 10% of each field than women. While Publons’ lists of reviewers represent only a small portion of the total number academic reviewers, the trend is also evidence of the broader gender imbalance in tiers of the scientific workforce.

“There are just fewer women in senior positions as scientists,” says Bik. And because professors do more peer reviews, fewer women in those positions means fewer female reviewers, she says.

The site’s marketing director Tom Culley said while the site doesn’t collect gender information on users, an analysis of names suggested about half as many women than men signed up to the site.

Of the reviewers on the site, three out of four reviewers in the top 10% of each field were male.

Culley said Publons is sensitive to gender disparity in peer review, and research more generally. “We hope to bring more attention to the issue as we build a more complete picture of the world's peer review behaviour.”

Bik has since been invited to become one of the site’s ‘academic advisors’, someone who shares advice and insights for early-career reviewers, especially female.

Researchers are usually inducted into the peer review circuit by their mentors. Many of the most productive reviewers are in senior research positions, and so approached more frequently by editors. Referring editors to early-career scientists is crucial to help them get their foot in the door.

Science needs diversity

Suzana Camargo, a hurricane-climate researcher at Columbia University and 7th on Publons list of Earth and planetary science reviewers , says addressing biases in peer review is important because the system should reflect the diversity of the scientific workforce. She says gender isn’t the only imbalance that needs attention.

When Camargo moved as a researcher from Germany to Brazil, her experiences of the peer review process changed dramatically. “I could see that the peer review was clearly dependent on the country you are sending your paper from. I had much harder reviews when I was in a developing country compared to when I sent similar papers from a developed country.’”