The NordForsk model is included in this report to provide a further perspective to the discussion. Like the national organisations, NordForsk has a shared interest in ensuring accountability and transparency, as well as demonstrating the value of research endeavours. Measuring and reporting on research impact is a central part of this work.
5.2 Methodology
NordForsk has a standard infrastructure for gathering data, which is being continually developed. For one, applications are expected to write on expected outcomes and impacts, not least on Nordic added value, a measure of strategic benefits of specific importance to Nordic research cooperation. Further, qualitative, and descriptive project data is reported by the research groups in our project management system, Insights. This data contains abstracts, popular summaries, final reports, funding, and rudimentary demographics.
Annual and final quantitative reports are also submitted to an online database, ResearchFish. Where projects self-report on activities, outcomes, results, and outputs, such as publications, policy influence and dissemination activities. Some of these are automatically validated through the systems data harvesting feature. Qualitative descriptions may be added to all datapoints by the researchers.
In addition to these, trials with impact narratives and case-studies have been made, to serve as a supplement to the quantitative reports.
5.3 Reporting
The main output for reporting is annual reports on the quantifiable metrics reported in ResearchFish. Further, the relatively robust collection and storage of data has allowed for the construction of data dashboards, where anyone can analyse the quantitative data and its relationships.
In specific cases, thematic reports are produced, where quantitative and qualitative data supplement each other for to produce a more comprehensive glimpse of a certain field of study.
5.4 Challenges
Aside from the philosophic dilemmas regarding causality, attribution and defining research impact, the harmonisation of data as well as producing reliably automated processes for analysis have been the foremost challenges. These stem mainly from the reliance on self-reporting, which affects the data quality.
Furthermore, NordForsk has a keen interest in analysing and benchmarking Nordic research endeavours, which is complicated by the difficulty of gathering comparable and standardised data on national, Nordic, European, and global levels.
These may be addressed by more robust and standardised reporting systems and collaborative databases.