How should governments measure innovation?

There is a need to account for all types of innovation, from big to small

11 October 2016

Ikon Images/Alamy Stock Photos

Boosting jobs and revenue through innovation is the goal of many governments and businesses. Innovation, however, is difficult to measure, partly because it means different things to different groups. But overcoming this challenge is essential to understanding innovation’s real drivers.

Two major indices assess innovation at a country level. The World Economic Forum Global Competitiveness Index (GCI), which considers factors that contribute to long-term economic growth, uses mostly qualitative data from a survey of business executives. The Global Innovation Index (GII) – published jointly by Cornell University, INSEAD and the World Intellectual Property Organization – has 82 input and output indicators using data from more than 30 sources. Inputs include gross expenditure on R&D, the number of venture capital deals and intellectual property payments. Outputs incorporate the number of trademarks, new businesses and high tech exports.

In the case of Australia, Mary O’Kane, chief scientist and engineer in the state of New South Wales, believes the metrics from both indices are useful because they give a detailed picture of the country’s declining productivity growth during more than a decade. In the latest GII, Australia dropped two places to 19th out of 128 ranked economies and countries. But more telling is the country’s GII efficiency ratio ranking of 73, which is based on a conversion of inputs to outputs. In comparison, New Zealand sits in 17th place overall, just two ahead of Australia, but has a GII efficiency ratio ranking of 40, up from 90 in 2013. This means New Zealand’s researchers are gaining a lot more innovation outputs for their inputs.

“A country’s capacity to produce innovation outputs in the form of new goods and services is needed for growth and prosperity,” says O’Kane, who is on GII’s advisory board. "[Australians] are great at inputs but terrible converters to outputs,” she says. The country’s input strengths include its regulatory, education and knowledge creation systems. But Australia struggles to diffuse and absorb the knowledge it creates, O’Kane says. This result is partly due to shortfalls in business sophistication and leading-edge technological innovation, both of which are outputs.

Defining innovation

New Zealand economist James Turner leads a government and industry backed project to increase innovation in the agricultural sector. In deciding which innovation metrics to use, Turner, from the government-owned AgResearch, says it was essential to know how different groups defined innovation. “Scientists were interested in measuring the impact of a transformational new idea, whereas industry was more interested in incremental improvements that increased efficiency and profitability,” he says. “Business and government were focused on income from new commercial opportunities.”

At Data61, the digital research unit of Australia’s CSIRO, innovation is defined as an idea or invention multiplied by its impact. “The impact can be societal or environmental or economic,” Data61 CEO, Adrian Turner, explained at the AFR Innovation Summit in Sydney in August.

But Turner is concerned that Australia is focusing only on invention-based innovation. “There’s also business model innovation and value-based innovation,” he says.

Amantha Imber, founder of Australian consultancy Inventium, says while benchmarking innovation at the national level is important, it is largely created at the local level. “It’s hard to have a blanket approach to measuring innovation – every project requires different metrics, even within the same industry,” she says. And inputs on their own don’t guarantee outputs, she says. People using metrics need to be mindful of other factors that may be at play. For example, an organization’s culture can determine if innovation thrives or dies, she says.

Data61’s Turner believes innovation measurements focus too much on inputs. “There are not enough output measures, which at the end of the day are economic growth and jobs,” he says.

More than metrics

Although impact can directly reflect innovation, innovation is also a measure of the process. “Measuring it gives you confidence that your impact was created by the innovation and not external factors,” Turner says. For example, better water quality in catchments near areas of intense farming could be due to destocking by farmers. This is not the same as the impact derived from new practices and technologies that have been designed to achieve the same goal, Turner says.

Imber believes examples help the public connect with innovation discourse. “There needs to be more stories to help the average person understand the value of innovation and how it is improving lives.”