Italian scientists increase self-citations in response to promotion policy

Study reveals how research evaluations can lead to self-serving behaviour.

4 June 2018

Topp_Yimgrimm/iStock/Getty

Italian scientists have been citing themselves more often since a controversial law came into effect in 2010 demanding academics meet productivity thresholds to gain promotion. Economic and managerial engineering showed the greatest leap in self-citations from 2010 to 2014 of the four fields examined.

The study is the first to quantify the extent to which scientists engage in opportunistic behaviour in response to evaluation metrics, says study co-author, Marco Seeber, a sociologist at Ghent University in Belgium. Evidence until now has been anecdotal, he says, focussing “on rare types of misconduct such as manipulation of data or plagiarism, and without showing a clear link between the introduction of metrics and behaviours”.

The 2010 legislation regulates access to professor positions using a set of quantitative metrics. It was introduced in response to criticisms that the principles of meritocracy were lacking in Italian academia, with allegedly significant levels of localism, cronyism and nepotism, says Seeber.

Under the new policy, university-selected disciplinary committees are required to make hiring decisions based on one of two approaches. The bibliometric approach, largely enforced in medicine and natural science disciplines, requires candidates for professor positions to outperform the median productivity of current faculty in at least two of three criteria: number of published papers, citations counts or h-index — a common measure of a researcher’s productivity and the impact of their studies.

The non-bibliometric approach, followed by some disciplines in the social sciences, arts and humanities, reviews candidates based on the number of papers published in what the committee considers first- and second-class journals, as well as the number of published books.

Seeber and his co-authors wanted to assess the impact of the legislation on academic behaviour. Their study, published in Research Policy in December 2017, analysed the self-citation practices of 886 Italian academics between 2002 and 2014. They observed a significant increase in self-citations after 2010 in fields that adopted the citation metrics: economic and managerial engineering, psychiatry and genetics.

The largest increase was seen among assistant and associate professors in economic and managerial engineering, who inflated their self-citations by 179% and 91%, respectively. Even full professors in the field increased their self-citations by 43%. In genetics, assistant professors cited themselves more, but not associate professors.

Applied economics, the only discipline the authors examined that adopted the non-bibliometric approach, showed no corresponding rise in self-citations.

“Self-citations might be the tip of the iceberg,” Seeber says, suggesting that researchers could have adopted other problematic behaviour in response to the policy, including abandoning subjects that attract, on average, fewer citations, simply to boost their metrics.

“The use of metrics for research evaluations can have several detrimental effects, like strategic game-playing”, concludes Franca Agnoli, a psychometrician at the University of Padua in Italy, who has previously reported on questionable research practices among psychologists in Italy.

Agnoli says that the average number of co-authors per paper might have also increased in response to the 2010 policy, as researchers pad each other’s publication lists by adding authors to their papers. Metrics used under the new policy give equal credit to all study co-authors, even those who contribute minimally. “This way of assigning credit can have a detrimental ethical effect by motivating researchers to barter authorship without regard to their contribution,” Agnoli says.