Scientists get more bang for their buck if given more freedom

Multi-national study challenges long-held assumptions about efficiency.

23 May 2018

erhui1979/Getty

Scientists are more efficient at producing high-quality research when they have more academic freedom, according to a recent study of 18 economically advanced countries. Researchers in the Netherlands are the most efficient of all.

The existence of a national evaluation system that is not tied to funding was also associated with efficiency, the study finds.

The analysis challenges the prevailing view that competition for funding and strong university management drive efficiency. “For both variables we find the opposite,” says Peter Van den Besselaar, an informatics researcher at VU University Amsterdam, the Netherlands, who co-authored the paper with Ulf Sandström at KTH Royal Institute of Technology, Sweden.

Pros and cons of competition

Until recently, studies into research efficiency usually looked at the attributes of countries that scored highly when measuring their inputs (funding) against their outputs (publications) in absolute terms. Van den Besselaar and Sandström looked instead at relative changes, to gauge a system’s efficiency at converting additional funding to additional publications.

Specifically, they assessed how a change in spending on higher education between 2000 and 2009 contributed to a change in authorship in the top 10% of highly cited papers in the Web of Science database. In this assessment, the Netherlands scored the highest, with a funding-to-publication ratio of 2.67, followed by Belgium.

Japan had the worst outcome of -2.46, with the United Kingdom and Israel tied for second-last.

Individual autonomy

Van den Besselaar and Sandström investigated several indicators to see which were most influential in making research more efficient. These were broadly related to the level of competition, university autonomy, academic freedom and stratification in a country.

Research systems in less efficient countries, such as the UK, are more prescriptive, with competitive project funding schemes, institutional evaluations tied to funding, and strong university managements determining the focus of research. “The more power university leadership has, the lower the power of the traditional academic,” says Van den Besselaar.

More efficient countries, including the Netherlands, give researchers a high degree of academic freedom, and have established systems for evaluating institutions not linked to the funds they receive. Institutions in these countries also receive relatively high amounts of funding.

“The finding on autonomy intrigues me, as it may counter some of the current policy rationales,” says Koen Jonkers at the European Commission’s Joint Research Centre.

He also suggests another factor not discussed in the study that might affect efficiency. “The relatively high degree of openness can also play a role.” In 2017, Jonkers co-authored a piece which found a strong correlation between a country’s openness to international collaboration and cross-border movement, and its production of highly cited papers. Both the Netherlands and the UK scored highly on openness.

Role reversal

The UK often comes out on top in absolute assessments of efficiency, including in an analysis of the leading countries in the Nature Index, so its poor performance in the recent study is unexpected.

In a 2010 paper by researchers in Finland, the UK was placed in the more efficient group of university systems and the Netherlands in the less efficient group, based on the number of papers published relative to funding. The efficient systems were characterized by a high level of competition for money.

However, the 2010 study points out that the supposedly efficient systems had not increased their efficiency in publishing papers over the two decades of analysis, from the early 1990s to the mid-2000s. “The idea of output and competition-based incentives promoting productivity in science is more complex than policy-makers seem to believe,” the researchers noted.

The UK’s edge in assessments based on absolute output could be an accounting fluke, suggests Van den Besselaar. In the UK, PhD candidates are considered students and are therefore not counted as a cost to the research system. In the Netherlands, on the other hand, PhD students are paid a wage — an additional cost that shouldn’t be conflated with inefficiency, he says.