This report presents the content analysis of semi-structured interviews, conducted with representatives from major Nordic research councils and funders. The analysis considers the methodologies and philosophies adopted by Nordic research councils and funds in assessing research impact. It provides insight into the phenomenology of research impact measurement, analysis and data utilisation within the Nordics.
A central theme revolves around the challenges in defining, measuring, and reporting research impact, as research impact extends beyond academic circles into broader societal benefits. Each Nordic country showcases its unique approach, reflecting their cultural and institutional nuances. However, a common thread among them is the growing emphasis on reliable and practical digital infrastructure, valid operationalisation and definition of research impact itself, and qualitative assessments to complement the shortcomings of the traditional scientometrics.
One notable challenge, highlighted in our findings, is the researchers’ reporting burden. Our results further underscore complexities faced by researchers and funders alike in documenting their work’s impact, balancing between the need for comprehensive assessment and the constraints of time and resources. This aspect raises concerns about the practicality and efficiency of time consuming but comprehensive of reporting practices, and highlights the importance of meaningful operationalisation when quantifying impact.
Our results also underscore the importance of data quality in research impact assessment. Ensuring accurate, relevant, and comprehensive data collection is difficult, especially when dealing with qualitative impacts or long-term effects. The variability in methodologies across different fields adds another layer of complexity, often resulting in inconsistent and partially incomparable data sets.
In addressing these challenges, our findings point to the potential of qualitative assessments and case study approaches to complement quantitative metrics. While these methods offer a richer, more nuanced understanding of research impact, they are not without their own shortcomings. The risk of subjective interpretations and the labour-intensive nature of detailed case studies are notable concerns.
Furthermore, our results stress the importance of a robust workflow for research impact assessment. An efficient digital infrastructure, a well-organised database system, and competent software tools are essential for effective data management and analysis. Equally important is the expertise in collating, analysing, and reporting data, which plays a central role in drawing accurate and meaningful conclusions from the research findings and their impact. As such, our hope is that this report will serve as a contribution to the ongoing discourse on how best to measure and understand the broader implications of research, not just within academia but in society at large.
Simon Jernroth, Analytics Officer