Encouraging translation and assessing impact of the Centre for Research Excellence in Integrated Quality Improvement: rationale and protocol for a research impact assessment

Abstract

Introduction

There is growing recognition among health researchers and funders that the wider benefits of research such as economic, social and health impacts ought to be assessed and valued alongside academic outputs such as peer-reviewed papers. Research translation needs to increase and the pathways to impact ought to be more transparent. These processes are particularly pertinent to the Indigenous health sector given continued concerns that Indigenous communities are over-researched with little corresponding improvement in health outcomes. This paper describes the research protocol of a mixed methods study to apply FAIT (Framework to Assess the Impact from Translational health research) to the Centre for Research Excellence in Integrated Quality Improvement (CRE-IQI). FAIT will be applied to five selected CRE-IQI Flagship projects to encourage research translation and assess the wider impact of that research.

Methods and analysis

Phase I will develop a modified programme logic model for each Flagship project including identifying process, output and impact metrics so progress can be monitored. A scoping review will inform potential benefits. In phase II, programme logic models will be updated to account for changes in the research pathways over time. Audit and feedback will be used to encourage research translation and collect evidence of achievement of any process, output and interim impacts. In phase III, three proven methodologies for measuring research impact—Payback, economic assessment and narratives—will be applied. Data on the application of FAIT will be collected and analysed to inform and improve FAIT’s performance.

Ethics and dissemination

This study is funded by a nationally competitive grant (ID 1078927) from the Australian National Health and Medical Research Council. Ethics approval was obtained from the University of Newcastle’s Human Research Ethics Committee (ID: H-2017–0026). The results from the study will be presented in several peer-reviewed publications, through conference presentations and via social media.

Strengths and limitations of this study.

• The proposed study uses a comprehensive mixed method four-phase design to validate a framework to encourage research translation and measure research impact.

• The study incorporates a process evaluation to understand users’ experience of the framework.

• Measurement of impact uses three proven methods for impact assessment—Payback (modified), economic evaluation and narratives.

• The time lag between translation and impact means impacts may not have been realised at the point of assessment. Impact assessment in this study is limited to five research projects rather than the Centre for Research Excellence in Integrated Quality Improvement as a whole programme.

Introduction

A substantial amount of health and medical research does not translate, is not implemented by healthcare systems, is not used by end users in policy and practice, and does not create impact.¹ Research translation is the process of knowledge generation and transfer that enables those utilising the developed knowledge to apply it.² The definition for ‘research impact’ modified for the health and medical research context and used in this protocol is the demonstrable effect from basic, health systems, patient and population-orientated research, and clinical trials that ultimately improves healthcare delivery, human health and quality of life, and generates benefits for the economy, society, culture, public policy or the environment.³ Any level of suboptimal translation means the returns earned from research investments do not achieve their potential. Further, in terms of the broader fiscal environment, there is a growing demand for more accountability in public spending across all sectors, including health.⁴

A contributor to suboptimal translation and impact is that research translation has not been systematically encouraged, and impact measurement beyond academic outputs such as peer-reviewed publications while becoming more common in countries like the UK⁵ is still not standard practice in most other countries.⁶ There are a plethora of impact measurement frameworks available and several studies including two recent systematic reviews of these frameworks, models and applications.^7–9 However, there is a lack of evidence to suggest that the availability of these frameworks and models has actually increased the proportion of health and medical research projects that actually measure and report on impact, rather than just outputs. There is growing recognition that translation of research into policy and practice needs to increase and that the pathways to realising impact ought to be more transparent.¹⁰ There have been several studies trialling the use of impact measurement applications in Australia,^11–15 but a national framework for measurement of research impact has not yet been implemented. However, there have been major developments in this space. Key initiatives include the following:

• The Excellence in Research for Australia (ERA), a national framework to examine how universities are translating their research into economic, social and other benefits¹³

• Australia’s National Health and Medical Research Council’s (NHMRC’s) Advanced Health Research and Translation Centres programme¹⁶

• Medical Research Futures Fund to support translational medical research¹⁷

• Australian Research Council’s (ARC’s) development and piloting of a national engagement and impact assessment framework to sit alongside the current ERA¹²

• NHMRC’s Centres for Innovation in Regional Health¹⁸

• Australia’s National Innovation and Science Agenda, which has dedicated $9 million to assess and report on the engagement and impact of university research.¹⁹

These initiatives confirm that research translation and impact assessment are high on Australia’s research agenda, and attempts to close the gap between research outputs and impacts will be highly regarded.

In Australia, this need for greater accountability is particularly evident in the area of Aboriginal and Torres Strait Islander (thereafter respectfully referred to as Indigenous Australians) health, where health disparities continue between Indigenous and non-Indigenous Australians.^{20 21} There remain serious concerns that Indigenous Australians have been over-researched without corresponding improvements in health outcomes.²² One reason for this has been the overabundance of descriptive studies in Indigenous health that, of themselves, rarely translate to changes in policy and practice. A recent systematic review of reviews of Indigenous health and well-being research identified knowledge and methodological gaps in documenting Indigenous health research impact and found that not one of the reviews assessed the impact of research nor explicitly referred to research impact.²³ More needs to be done to ensure that Indigenous health research improves the quality of health and health-related services delivered to Indigenous communities, and ultimately improves the health and well-being of the Indigenous community. Consideration of ‘value for money’ must be a component of determining the success or failure of health and well-being programmes. Currently, government agency reporting is based largely on inputs and outputs rather than impacts, and the absence of robust and available outcome data has been a significant barrier to assessing impacts. In a debate-style article on the benefits of Indigenous health research, the authors concluded that adopting a benefit-led approach and embedding the assessment of benefit from the outset of the research is a vital prerequisite to maximising research impact.²²

Recognising a need to enhance continuous quality improvement (CQI) initiatives in Indigenous primary healthcare (PHC), the NHMRC funded the Centre for Research Excellence in Integrated Quality Improvement in Indigenous PHC (CRE-IQI) from 2015 to 2019. The vision for CRE-IQI is to improve Aboriginal and Torres Strait Islander health outcomes by accelerating and strengthening large-scale PHC quality improvement efforts. Building on the partnership learning model²⁴ developed through earlier research, and on innovation platform concepts, the CRE-IQI brings together stakeholders from across different levels of the health system (clinical, policy, service coordination and support) to share and exchange their experiences, knowledge, skills, ideas and resources to accelerate and strengthen systems-level PHC improvement efforts. A significant portion of CRE-IQI resources are dedicated to the functioning and improvement of the innovation platform and cross-cutting programmes including research capacity-strengthening, strengthening collaboration and research translation. Specifically, a portion of its funding has been allocated to adopting a framework to encourage research translation and assess impact of its research programme. The selected Framework to Assess the Impact from Translational health research (FAIT) was developed by a team of health economists and health and medical researchers based at the Hunter Medical Research Institute with the specific aim of encouraging and measuring research translation and impact. The framework was based on a mixed methods study involving (1) a scoping review of existing research impact frameworks and techniques to inform the development of FAIT; (2) a development stage to design the prototype; and (3) a feedback stage where iterations of the prototype were presented to selected researchers for discussion and refinement.²⁵ The adoption of FAIT by CRE-IQI presents an opportunity to pilot the framework’s implementation and trial its research impact assessment methodology. The framework will be applied to five selected Flagship projects to present transparency to the translation process, provide capacity to improve the speed of translation (when applied prospectively), and ultimately to assess the impact of these research projects (see table 1 for details of each project).

Table 1.

CRE-IQI Flagship projects selected for implementation of FAIT

Project title	Project synopsis	Years in progress	Type of FAIT implementation
Engaging stakeholders in identifying priority evidence–practice gaps and strategies for improvement in primary healthcare (ESP Project)	The ESP Project brings together the concept of knowledge cocreation and evidence on how to achieve large-scale change in quality of care. It engages a wide range of stakeholders in using aggregated continuous quality improvement data to identify priority gaps in care, barriers and enablers and strategies for improvement.33	2014–current	Retrospective and prospective
Ongoing collaborative analysis and reporting of data from the Audit and Best Practice in Chronic Disease National Research Partnership (ABCD Partnership)	The ABCD Partnership investigates the variation in quality of care in Indigenous primary healthcare centres. It also explores the underlying factors associated with variation at the health centre and regional level, and examines specific strategies that have been effective in improving primary care clinical performance. This information will be used to work with health service staff, management and policy makers to enhance the effective implementation of successful strategies.34	2010-current	Retrospective and prospective
Quality improvement in Aboriginal primary healthcare: lessons from the best to better the rest	This project examines six ‘high improving’ PHC services within the ABCD cohort to enhance understanding of how contextual factors interact to facilitate the success of continuous quality improvement (CQI) initiatives within a service. The findings will be used to assist striving services to increase their success in implementing CQI initiatives.35	2014–2017	Retrospective and prospective
Strategies for improving provision of maternal care for Aboriginal and Torres Strait Islander women	This project aims to use the ABCD data to ascertain which combination of components of pregnancy care has the largest positive impact on birth outcomes. This information will be used to develop a list of essential pregnancy care items, a tool and accompanying resources for health services to implement these essential care items. The project will also work with stakeholders to further develop strategies to improve maternal health outcomes.36	2017–2019	Prospective
Sustainable Family Well-Being (FWB) implementation and evaluation using CQI approaches	This project will define and develop implementation mechanisms that support FWB empowerment programme integration and implementation within family support programmes. This includes the development of evidence-informed funding models, mechanisms and sustainable ways of embedding FWB and upscaling proven family support programmes and services.36	2015–2019	Retrospective and prospective

CRE-IQI, Centre for Research Excellence in Integrated Quality Improvement; FAIT, Framework to Assess the Impact from Translational health research; PHC, primary healthcare

The remainder of this paper focuses on the research protocol of a mixed methods study to document the pathway to translation (including engagement with stakeholders and barriers and enablers of translation) and measure the impact of the five aforementioned projects—all of which are at different stages of the research pipeline. There are four objectives for this study:

1. provide transparency about the pathway to generating research impact

2. examine process issues associated with the implementation of FAIT

3. test the feasibility of using FAIT’s package of validated impact assessment methodologies

4. assess the impact of the five Flagship projects.

The anticipated outcome of this study will be greater translation of research among CRE-IQI associates and others working in this field. It is also anticipated that the study will provide an evidence-based report of the impact of the five CRE-IQI projects and evidence to other health services researchers wishing to implement a framework to encourage greater translation and optimise and measure their research impact.

Methods and analysis

This study involves the application of a specific framework (FAIT) to encourage research translation and measure research impact.²⁵ The setting will be the CRE-IQI. While the CRE has a project coordinating centre in Lismore, New South Wales, its work is largely carried out through collaborative teams from multiple organisations including community-controlled health services, government health services, policy organisations, universities and research institutions across New South Wales, Queensland, the Northern Territory, South Australia and Western Australia. Participants in data collection will be a mix of researchers and students who are associated with CRE-IQI and collaborators in related fields such as health service staff, clinicians, policy makers, representatives of peak Indigenous health organisations and Indigenous community members.

The study design will involve a four-stage sequential mixed method design, summarised below:

Phase I: A modified programme logic model of CRE-IQI’s five selected Flagship research programmes will be developed with input from key stakeholders. The models will be developed retrospectively for ongoing research projects and prospectively for newly instigated initiatives. Part of this process will include a scoping review to identify categories of impact that will inform the type of benefits that may be expected to result from research into integrated quality improvement in Indigenous PHC and to identify potential values or sources of value associated with those benefits.

Phase II: The implementation of FAIT focusing on data collection (process, outcome and impact metrics). This stage will also incorporate a process evaluation to collect participants’ perceptions of FAIT and its implementation.

Phase III: The impact of the five Flagship projects will be measured and evaluated using the package of FAIT methodologies for impact assessment, namely ayback,²⁶ economic assessment and case studies. The results will be summarised and presented by way of a scorecard, including narratives describing the process by which the research translated and generated impact.

Phase IV: The outcomes of both the implementation of FAIT and the results of the assessment of the five Flagship projects will be compiled. This report will include recommendations for the future implementation of FAIT in Indigenous health research settings.

The approximate timelines for the various activities and key dates are summarised in figure 1.

Figure 1.

CRE-IQI timeline for implementation of FAIT. CRE-IQI, Centre for Research Excellence in Integrated Quality Improvement in Indigenous Primary Healthcare; FAIT, Framework to Assess the Impact of Translational health research.

The following sections provide details about the methods for each of these four phases of the study.

Phase I

A modified programme logic model

The first phase will be the creation of five modified programme logic models.²⁵ One of the modifications to the logic model as used in FAIT relates to the insertion of ‘end users’, which has the advantage for impact assessment purposes of identifying who will use the research outputs. However, in the context of CRE-IQI, end users are defined as collaborators along the pathway to impact that are both cocreators and co-users of the research outputs, including CQI coordinators, other health service staff, clinicians, policy makers, representatives of peak Indigenous health organisations and Indigenous community members. This definition includes both interim and final users. A further modification is the introduction of process and output metrics in addition to impact metrics to provide greater transparency between the aims and intended impacts of the research. The purpose of the logic models will be to provide a strategic map of how each of the five Flagship projects plans to generate impact. The logic models link community and other needs to the research priorities and activities. These activities should produce an output that when used by an end user creates an opportunity for the generation of impact. While recognising that translation is a multidirectional phenomenon, this approach provides ‘line of sight’ from need to research to impact (see figure 2).

Figure 2.

Modified programme logic model.

The value in articulating these processes in a programme logic model gives transparency to how the research producers believe their project will generate impact. The programme logic model provides insights about the planned activities, expected outputs and intended impacts. This information is used to determine a series of metrics to measure the project’s progress against plans. Process metrics not only allow researchers to determine if the research is going to plan, they are an opportunity to include activities that have, in the literature, been associated with successful translation and the generation of impact. Output metrics help identify when key outputs or products of the research activity have been generated. Impact metrics are measures that reflect the consequence of the research output being used by end users. For example, a new clinical guideline might be the product of a Flagship project, but it will need to be used or implemented by clinicians before it can generate impact such as improved patient outcomes.

Data for the given programme logic models will be obtained through a series of semistructured individual and group telephone interviews with key stakeholders from each project and group feedback sessions to ensure all perspectives are covered. For projects that are further along the research pipeline, information obtained from researchers and collaborators will be triangulated against existing documents such as published papers, and other project documents including meeting minutes and progress reports.

With the prospective application of FAIT, the modified programme logic model will be used to discern the relevant research outputs and to describe the expected impacts when used by the end users. In a retrospective application, it will be used to give clarity to the extent to which research translation and impact were given consideration at the programme outset. While the programme logic model appears linear within this diagrammatic representation (necessary for the development of a logic model), its application, including project development, stakeholder engagement and project refinement, is in most part non-linear and iterative in nature. Hence the programme logic models will be living documents open to change at all stages of the research to ensure they capture the actual translational pathways to impact.

Scoping review

The development of the programme logic models will be informed by a scoping review. The objective of the review will be to identify categories of impact that will inform the benefits that may be expected from research about integrated quality improvement in Indigenous PHC. It will also be used to identify potential values or sources of value associated with those anticipated benefits.

The review process will follow the Joanna Briggs Institute (JBI) guideline for scoping reviews.²⁷ While still methodical, scoping reviews are typically broader in their focus with less restrictive inclusion criteria than systematic reviews.²⁸ The review will be used to map the key concepts underpinning the measurement of impact on the delivery of health services to Indigenous populations. As outlined in the JBI guideline, a three-step search strategy will be used and a provisional search strategy is described here. Step 1 of the review will be conducted using combinations of the following free text and medical subject heading (MeSH) terms in the titles and abstracts of articles from two online databases: Indigenous health research, health services research, CQI, integrated quality improvement, research impacts, knowledge generation, health impacts, health outcomes and economic impacts. Step 2 will be an analysis of the text words contained in the title and abstract of any retrieved papers and of the index terms used to describe the articles. A second search will then be undertaken using all identified keywords and index terms across all included databases. Third, the reference list of all identified reports and articles will be hand-searched for additional studies. In this review, literature will be drawn from both economic (ie, EconLit and JSTOR) and general health and medical academic databases (ie, Medline, Embase, The Cumulative Index to Nursing and Allied Health Literature, Cochrane Database of Systematic Reviews). The searches will also extend to Google Scholar and Google to identify grey literature from government departments, international organisations and research funders, including information of potential grey and published literature identified by CRE-IQI members and affiliates during the programme logic development phase. The searches will be limited to articles published in English between 1995 and 2017. This time frame is considered to be appropriate because knowledge translation, a precursor to impact assessment, first gained prominence in the late 1990s.

The data from the review will be charted to record the key information. In line with recommended scoping review guidelines, the charting of results will be iterative.^{27 28} The tabulated results will be accompanied by a narrative aligned to the review objective. The findings will be used to inform the domains of benefit and valuations for inclusion in the Payback and economic assessment of Flagship projects. No formal assessment of the quality of the studies will be undertaken and the results will not be published.

Phase II

Implementation of the FAIT framework

Phase II of the study will be the implementation of the FAIT framework over the remaining 24 months of CRE-IQI operations until the end of July 2019. This will entail sharing the programme logic models with all CRE-IQI associates, allowing for feedback and modifications to the five models and 6-monthly updating of the models including any modifications to expectations and predefined activities. Through a process of audit and feedback, Flagship project teams will have the opportunity to assess how they are tracking against their output and impact goals and to refine their research translation and engagement activities to maximise impact. In addition, CRE-IQI associates will be exposed to current thinking around research translation, implementation and impact through CRE-IQI’s research capacity strengthening programme.

Data collection for this stage of the study will involve a series of online and telephone surveys of CRE-IQI management and associates to elicit their perceptions of FAIT, determine if the framework encourages translational behaviours and how the implementation of the framework can be improved. Participants will also be asked to articulate which aspects of the framework work well and which aspects need refinement.

Phase III

Research impact measurement and valuation

Currently, there is no single measurement method capable of capturing the impacts stemming from health and medical research. For this reason, FAIT employs a combination of three integrated but separate proven impact assessment methods: quantified metrics,²⁹ economic assessment³⁰ and narratives of the process by which the research in question translates and generates impact. Using qualitative project examples, the case studies will be triangulated against the Payback and economic assessment to validate the impact of the research in question.

Metrics—modified Payback

The metrics referred to in FAIT are a variation of the methods used in the Payback Framework.²⁶ Metrics will be organised under broad domains of benefit such as knowledge impacts, impacts on practice, economic impacts, policy impacts and community impacts. Semi-structured interviews and group discussions with each Flagship project team will be used to generate process, output and impact metrics that will be used to populate the domains of benefit within Payback. These metrics will be structured to support the planned economic assessment. Robust metrics that are contextually relevant to Indigenous health research will be selected with consideration to objectivity, administrative efficiency, transparency and comparability, as well as their ability to be verified.

In cases that involve the retrospective application of FAIT, examples of process metrics will include the historical level of engagement with key stakeholders as well as activities that could assist the translation of research outputs through to others in the research pipeline. With retrospective analysis, the metrics may necessarily be constrained to outcome measures selected at the research programme outset to measure the efficacy of the research components—for example, changes in organisational systems that support the adoption of CQI within those health services that are involved in translation of findings from Lessons from the Best.

Economic assessment

The economic assessment component will entail a comparison of the costs associated with developing and implementing the five Flagship projects versus (where feasible) a calculated value for the expected impact or consequence of the funded research. The descriptive nature of much of CRE-IQI work will impact on the type of economic assessment that is feasible and useful. The planned assessment will collect, on a case-by-case basis, the resources used to fund the research, including non-CRE funding expended on each project prior to the commencement of CRE-IQI. The cost of running the CRE-IQI programme will be appropriately apportioned across the five Flagship projects (these projects represent the major investment of the CRE), keeping in mind that programmes such as the capacity strengthening programme benefit a range of other CRE-IQI-affiliated projects and partners. Additional costs in utilising the research outputs of each project will also be included. For example, research that develops a minimum package of pregnancy care content for implementation will have used resources to develop and evaluate the package. Implementation of that package might increase the number of maternal prenatal care consults or diagnostic tests. These consults and tests are additional costs to the health system and can be modelled on the Cooperative Research Council-endorsed evaluation framework—the Impact Tool—which uses cost benefit analysis as its foundation. Implementation of that minimum package might also have positive impacts on preterm birth, low birth weight and small-for-gestational age, which can be reported as downstream savings to the health system. The appeal in using the tool to guide the economic assessment stems from the emphasis on the logic underpinning the research activity–output–usage–impact chain to give transparency and clarity to the research, which is also at the heart of FAIT. The programme logic model will assist in articulating programme inputs, expected outputs, uptake and ultimate impact. The total calculated expected costs and benefits will be combined by way of an impact map. Depending on the focus and stage of each Flagship project, three broad steps will be involved in the economic assessment: (1) identification and measurement of resource use; (2) measurement and valuation of the expected impact, where possible; and (3) comparison of the costs and expected impacts, where possible, in a single metric. Where practical, the analysis will assume a societal perspective to ensure all possible costs and benefits are accounted for. Expected costs and impacts will be reported in net present value terms and streams of projected future costs and benefits will be discounted at a rate of 3%.

Identification, measurement and valuation of resource use

Guided by the programme logic model, resource use pertaining to (1) the development of the research, (2) delivery of any research outputs or interventions, (3) uptake of outputs by end users and (4) health service changes will be identified, measured and valued. The retrospective nature of the application of FAIT to a majority of the Flagship projects will hamper the collection of data to inform many of the costs and benefits. This is especially true for costs incurred as a result of adopting or using the research outputs of each CRE-IQI project.

Resource use associated with development and delivery of the various projects will be costed using financial and administrative records from the respective research teams. Where appropriate, the costs associated with translation of the project findings and outputs will include any costs (including opportunity costs) incurred by the various health service organisations, such as costs related to practice change. As stated above, it will be problematic to collect data to inform these costs retrospectively. However, some attempt will be made to model these costs using administration records and detailed descriptions of uptake obtained from programme managers and CQI facilitators to inform the modelling.

Unit costs for health service resource use, where appropriate and available, will be based on the best available data at the time, including the Medicare Benefits Schedule.³¹ Resource use of marketed goods and services outside the health sector will be valued at current market prices. Unmarketed goods and services such as travel time and the time of volunteers will be costed using opportunity cost prices.

Measurement and valuation of the expected impact

Impact will be calculated for selected domains from each of the five programme logic models. The calculations will be adjusted for risk to give the expected value of the impact. Attribution will be assigned at a conservative rate, the value of which will be informed by administrative and evaluation records and qualified during the researcher and health service staff interviews. Projected valuations will include a ‘drop-off’ factor to account for waning benefit over time. The sensitivity of the impact calculation to changes in attribution and drop-off will be extensively tested. Any and all assumptions underpinning the analysis will be made explicit in the reporting of the results. The economic assessment will be limited to assessments for which there is existing evidence or for which evidence can be collected.

Narratives (case studies)

The FAIT approach also incorporates the use of illustrative examples or narratives which will be compiled for each Flagship project to describe in more qualitative terms how translation occurred and how research impact was generated for each project.²⁵ Narratives can be powerful tools for communicating the nature and extent of research translation and, ultimately, research impact. They also enable quantitative findings to be placed in context and are an opportunity to explain variances in research costs, outputs and impacts. Feedback received by the ARC as part of the development of its Engagement and Impact Assessment Framework indicates that a narrative approach is the most appropriate method to convey information and data on Indigenous research particularly regarding engagement and impact.

Case studies have been the basis of the research evaluation system currently used in the UK.⁵ Within the Payback Framework, case studies are developed retrospectively for selected research projects and used to validate impact data and to provide illustrations of high impact for different Payback categories.¹⁵ In FAIT, narratives are developed progressively along the research pathway and used for describing the often complex pathways for research translation.

In other applications of FAIT, these narratives have also been important vehicles for verifying the consistency of the impact findings generated from the economic assessment and Payback quantified metrics. In this application, it is expected that the narratives are supported with evidence extracted from the quantified metrics and economic assessments, and will be informed by interviews at different points along the research pathway with key CRE-IQI researchers and key stakeholders including end users of the research such as health service staff, representatives of peak bodies, government representatives and Indigenous community leaders. It is hoped that the collaborative and prospective approach to the development of the narratives will render them less likely to be impacted by the biases that often characterise case studies based only on self-reports such as selective memory.³²

Limitations

This study is being conducted in a real-world setting, which presents some obvious limitations. Impact assessments are resource-intensive, and although the prospective collection of evidence is more cost-effective, not all the CRE-IQI Flagship projects will allow prospective collection of the required data. Final metrics for the Payback assessment and data for the narratives and economic assessments for each project will be based on what can feasibly be collected versus the ideal list of impact metrics and evidence. The lag between research translation and impact means that valuations may need to be undertaken with reference to interim rather than final impacts. For CRE-IQI Flagship projects that are further along the research pipeline, this constraint will be less problematic compared with projects that have commenced more recently. Conduct of the study in a real-world setting means there are no controls (counterfactuals); thus, attribution of impact for all five projects will be necessarily conservative. And finally, the FAIT framework is project-based and is being applied (as intended) to a select number of CRE-IQI projects that represent a major investment of the CRE. A limitation, therefore, is that this study will not assess the impact of CRE-IQI as a whole.

Phase IV

Reporting and recommendations around the implementation of FAIT

The results, including the narratives, will be summarised and reported by way of a scorecard (see figure 3 for hypothetical scorecard). This scorecard will form the basis of CRE-IQI reporting of the translation and impact of its five Flagship projects, as well as feed into a more comprehensive evaluation of the CRE as an innovation platform (the details of which are not covered in this protocol).

Figure 3.

Hypothetical scorecard for a research project looking at increasing the delivery of cardiovascular risk assessments and follow-up for Aboriginal and Torres Strait Islander people.

The findings from the implementation of the FAIT Framework within CRE-IQI and specifically about its applicability within the Indigenous health research context will be compiled, and a workshop with key CRE-IQI researchers and stakeholders will be employed to discuss the findings and to obtain feedback with a view to the final refining of the framework for future use.

Ethics and dissemination

The implementation of FAIT within CRE-IQI is funded as part of a nationally competitive grant (Grant ID 1078927) through the Australian NHMRC. The study, as described in this protocol, has received ethics approval from the University of Newcastle’s Human Research Ethics Committee (Ethics ID: H-2017–0026). While no participant details will be collected as part of the study, consent will be sought and recorded for each participant and associated organisation.

It is anticipated that the results from the study described in this protocol will be presented in several related publications. The first will focus on the implementation of the framework (development of the programme logic) and its evaluation (did it work?). The second will summarise the learnings from the study and present recommendations for improving FAIT. The research impact assessment findings will be presented in a series of publications.