Impact of clinical registries on quality of patient care and health outcomes: protocol for a systematic review

Abstract

Introduction

Many developed countries have regional and national clinical registries aimed at improving health outcomes of patients diagnosed with particular diseases or cared for in particular healthcare settings. Clinical quality registries (CQRs) are clinical registries established with the purpose of monitoring quality of care and providing feedback to improve health outcomes. The aim of this systematic review is to understand the impact of CQRs on (1) mortality/survival; (2) measures of outcome that reflect a process or outcome of healthcare; (3) healthcare utilisation and (4) costs.

Methods and analysis

The PRISMA-P methodology, checklist and standard strategy using predefined inclusion and exclusion criteria and structured data abstraction tools will be followed. A search of the electronic databases MEDLINE, EMBASE, Cochrane Central Register of Controlled Trials (CENTRAL) and CINAHL will be undertaken, in addition to Google Scholar and grey literature, to identify studies in English covering the period January 1980 to December 2014. Case–control, cohort, randomised controlled trials and controlled clinical trials which describe the registry as an intervention will be eligible for inclusion. Narrative synthesis of study findings will be conducted, guided by a conceptual framework developed to analyse the outcome measure of the registry using defined criteria. If sufficient studies are identified with a similar outcome of interest and measure using the same comparator and time of interval, results will be pooled for random-effects meta-analysis. Test for heterogeneity and sensitivity analysis will be conducted. To identify reporting bias, forest plots and funnel plots will be created and, if required, Egger's test will be conducted.

Ethics and dissemination

Ethical approval is not required as primary data will not be collected. Review results will be published as a part of thesis, peer-reviewed journal and conferences.

Trial registration number

CRD42015017319.

Strengths and limitations of this study.

• The strengths of this study are that it will generate best evidence on the impact of clinical quality registries (CQR) on mortality/survival, measures of outcome that reflect a process or outcome of healthcare, healthcare utilisation and costs.

• Results will inform policy development and rational investment in registries. The use of a broad term ‘registries’ enhances the likelihood of capturing all relevant articles.

• The limitations of this study are that CQRs are established for different diseases and collect different outcomes of interest for which it will be difficult to do a pooled analysis with similar outcomes. We are using broad search terms which have yielded 22 235 abstracts for review. The process of undertaking this review will be laborious and time-consuming.

Introduction

Healthcare systems in developed countries are struggling with rising healthcare costs, creating an urgent need for new cost-effective systems to improve the quality of healthcare while also improving efficiency. Several strategies are being implemented across different healthcare systems around the world, including the use of clinical registries, which have been identified as a means of improving quality and cost efficiency in health and medical care.¹ Their growth is thought to be a result of advances in information technology, including the increasing use of electronic health records and growing demand for accountability in quality of care.²

In epidemiology, the term register is applied to the file of data concerning all cases of a particular disease or other health-relevant condition in a defined population such that the cases can be related to a population base. The registry is the system of ongoing registration to the register.³ Clinical registries collect a defined minimum data set from patients undergoing a particular procedure, diagnosed with a disease or using a healthcare resource.⁴ They capture data systematically from existing administrative systems and databases, from medical records or directly from clinical staff using data collection forms. Clinical registries generally fall into two categories: those collecting data on patients who are exposed to particular health services for a relatively short period of time, and those tracking diseases or conditions over time or across multiple providers and/or health services. Importantly, both capture exposures and outcomes of interest to healthcare providers or healthcare systems.⁵

Clinical registries have been demonstrated to serve a number of purposes. They can assist in case identification for prospective and retrospective studies and clinical trials;⁶ improve patient management and record-keeping;⁷ assist in research, evaluation and planning of healthcare services;⁸ document the effectiveness of therapies in real-world settings; generate insights into risk factors for adverse outcomes;⁹ provide insight into the nature of disease and the benefit of treatments in subgroups of patients, particularly in poorly resourced environments that are not well represented in randomised clinical trials;¹⁰ and provide comparative benchmark reports to stakeholders, including hospitals, clinical staff, patients, consumers, funders and insurers.¹¹

A recent review conducted by Stey et al¹² in 2015 on clinical registries and quality measurement in surgery concluded that clinical registries have made a significant contribution to the quality of surgical care by evolving the definition of surgical quality of care, measurement process and risk modelling and by aiding as a platform for local interventions to address improvement of quality of care.

The term ‘clinical quality registry’ (CQR) has been used to describe clinical registries whose principal focus is to improve quality and safety of care. Originally coined in Sweden, CQRs are described in their Health Act as ‘An automated and structured collection of personal data that were initiated with the purpose to systematically and continuously develop and safeguard quality of care. A national or regional Quality Registry refers to one in which personal data have been collected from several caregivers and which allows for comparisons within healthcare at a national or regional level’.¹³ CQRs identify variation in ‘best practice’ treatments and outcomes across centres¹⁴ and provide feedback on performance in an effort to stimulate quality improvement processes to motivate change.^{15 16} Feedback may take various forms and be directed to hospital administrators, policy developers, health practitioners, government or a combination of each. An example of a Swedish CQR is the Registry for Ulcer Treatment (RUT), which through monitoring ulcer management and providing feedback to participating units has achieved a reduction in inappropriate antibiotic treatment for patients with hard-to-heal ulcers from 71% to 29% (p=0.001).¹⁷ In Australia, the number of known clinical registries has risen from 28 in 2006–2007 to 37 in 2012, few of which have national coverage.¹⁸ It is unclear how many of these provide feedback on quality of care and would be regarded as CQRs.

Despite some evidence of the capacity of registries to improve health outcomes, several challenges have been identified which have impacted the development of registries. First, establishing a new registry requires a considerable long-term commitment of resources, which is often difficult to garner.¹⁹ Second, information required by registries is often not routinely collected in a standardised manner at a population level. This is particularly the case in developing countries, where even establishing cancer incidence registries is challenged because of low priority in resource allocation. Poor transportation network, postal services and telephone communication cause problems for patients quickly seeking healthcare and for registry personnel for collecting follow-up data on health and vital status as well as becoming costly and time-consuming.²⁰ Finally, it is not possible to evaluate the direct impact of a CQR on the cost or quality of healthcare until it achieves a sustainable operation, which can take considerable time.

Only a few studies have quantified the value of establishing clinical registries in terms of economic and clinical impact; hence, supporting evidence for costly investment in the registry is minimal.²¹ One such study reviewed 13 registries in five countries and identified that Sweden could reduce its annual growth in healthcare spending from an estimated 4.7% to 4.1% by investing $70 million annually in disease registries, resources for data analysis and information technology infrastructure.²²

Rationale and objectives

It is evident that clinical registries, including CQRs, are being developed as tools to monitor and improve quality of care or as platforms for epidemiological research⁴ with benefits described by several authors.^21–23 However, to date, no systematic reviews have been undertaken to examine the impact of clinical registries on survival/mortality and on improving health outcomes.

We will undertake a systematic review to identify studies which have reported the impact of clinical registries, including CQRs on mortality/survival in the population under investigation and process of healthcare and other outcome measures other than survival (quality of care); utilisation of health services; and costs.

Methods and design

A description of the population, intervention, comparison and outcome (PICO)²⁴ of the systematic review are outlined in table 1 and described below:

Table 1.

Description of the PICO of the systematic review

Serial number	PICO	Descriptions
1	Population	Studies conducted in clinical environments which may include: Acute care (inpatient and outpatient); Subacute care (rehabilitation centre); Community (general practice and aged care).
2	Intervention	Registry as an intervention with the following inclusion and exclusion criteria Inclusion criteria Studies will be included from the systematic review if they: Describe either a clinical registry or a CQR which collects data on a procedure, disease or healthcare resource; Collect data systematically and on an ongoing basis from the population being investigated; Provide feedback on the performance of a health system on an ongoing basis; Collect data from more than one hospital. Exclusion criteria Studies will be excluded from the systematic review if they: Collect and report on data from only one hospital; Do not provide feedback on an ongoing basis (such as an audit or point prevalence study); Are written in a language other than English; Were published prior to the start date on 1 January 1980 or after the end date of 31 December 2014 or were review articles.
3	Comparison	Comparators: Data collecting tools other than registry (population-based data, administrative data and medical record) to monitor health outcomes; Studies without a comparator will be included.
4	Outcome	The primary outcome measure is impact on survival/mortality. Secondary outcome measures reflect a process or outcome of healthcare, healthcare utilisation and costs.

CQR, clinical quality registry; PICO, population, intervention, comparison and outcome.

Inclusion criteria

We will include the studies in the systematic review if they meet the following criteria:

• Describe either a clinical registry or a CQR which collects data on a procedure, disease or healthcare resource;

• Collect data systematically and on an ongoing basis from the population being investigated;

• Provide feedback on the performance of a health system on an ongoing basis;

• Collect data from more than one hospital.

Exclusion criteria

Studies will be excluded from the systematic review if they:

• Collect and report on data from only one hospital;

• Do not provide feedback on an ongoing basis (such as an audit or point prevalence study);

• Are written in a language other than English;

• Were published prior to the start date on 1 January 1980 or after the end date of 31 December 2014 or were review articles.

Study design

On the basis of a pilot search, it is evident that there are likely to be insufficient randomised controlled trial (RCT) studies assessing registries as an intervention to enable a meta-analysis to be undertaken. Therefore, a decision was made to include case–control, cohort and clinical controlled trial study designs in addition to RCTs.

Comparator(s)/control

Studies with and without comparators will be eligible for inclusion. Comparators may include contemporaneous data sources such as hospital administrative databases, insurance databases and clinical information systems measuring mortality/survival; processes of care; outcomes other than mortality/survival; healthcare utilisation and costs.

Context

Studies conducted in the clinical environments which may include acute care (inpatient and outpatient), subacute care (rehabilitation centre) and community (general practice and aged care).

Outcome measures/outcome of interest

The primary outcome measure is impact of clinical registries, including CQRs, on survival/mortality. Secondary outcome measures are those that reflect a process or outcome of healthcare (quality of care), healthcare utilisation and costs. Quality of care measures may include those assessing the impact of the registry on patient safety (complications), timeliness (reduce waits and harmful delays), effectiveness (adherence to clinical guidelines—provide services based on scientific knowledge), efficiency (avoid waste), equitable access to the services (provide services that do not vary in quality), patient centredness (provide care that is respectful of and responsive to individual patient preference, needs and values).²⁵

Search methods

Before finalising the search strategy, it will be tested and amended as necessary across the different databases. A database record will be maintained at each stage of the review process detailing how the search was undertaken and results of the search strategy. A senior medical librarian (LR) will assist in the final draft of the search strategy.

The following electronic databases will be searched: MEDLINE, EMBASE, Cochrane Central Register of Controlled Trials (CENTRAL), CINAHL and Google Scholar to identify the studies in English covering the period January 1980 to December 2014. The search strategy will include keywords describing studies involving established registries as the intervention. The Medical Subject Headings (MeSH) terms relating to registry or registries combined with the MeSH terms referring to mortality, morbidity, patient-reported outcome, healthcare utilisation, cost/economic evaluation and clinical competence will be included in the search. The search strategy for MEDLINE is shown as online supplementary document 1.

Additional searches will be conducted in grey literature resources such as conference websites and government websites. Handsearching and reference checking of citations and reference lists will also be undertaken.

Study screening and selection

Titles and/or abstracts of studies identified using the search strategy and those from additional sources will be distributed among four review authors. Two review team members will independently retrieve and assess the eligibility of the full text articles of potentially suitable studies. Any disagreement between reviewers will be resolved through discussion with a third review author on the study team.

Data extraction

A standardised form will be developed and piloted, based on the template of the Cochrane data abstraction form.²⁶ It will be used to extract data from the selected studies for assessment of study quality and synthesis of evidence. Extracted information will include study design and methods, country setting, participant characteristics, intervention characteristics including the feedback provided, study outcomes, discussion points, recommendations and study funding sources. Attributes of registries will be assessed using the criteria described by Black et al,²⁷ which assess coverage, organisation and management, security and confidentiality of data, uses, and quality of data.

Two review authors will extract data independently; inconsistencies will be identified and resolved through discussion with a third author where necessary. If essential data are missing, we will contact the authors for further information.

The data extraction tool has been shown as online supplementary document 2.

Data management

Literature search results of Endnote will be uploaded to Monash University faculty-allocated network storage, which will be password protected and only accessible to the reviewers. This network drive will allow us to create the data extraction process and entry of data and keep a record of all review-related documents.

Study quality and assessing risk of bias

Study quality will be appraised using the Scottish Intercollegiate Guidelines Network checklists developed for RCT, cohort and case–control studies.²⁸ The criteria we will use to assess the quality of studies are outlined in table 2. All questions answered will be categorised as yes, no or cannot say. An overall assessment will categorise the study as being of high quality (++), acceptable (+) or unacceptable–reject (0).

Table 2.

Criteria for assessing the quality of the study adapted from the SIGN checklist

Section 1: internal validity	Cohort study	Case–control study	RCT/CCT
In a well-designed study	The study has addressed an appropriate and clearly well-defined research question.	The study has addressed an appropriate and clearly well-defined research question.	The study has addressed an appropriate and clearly well-defined research question.
Selection of participant	Two groups being studied are selected from source population and that are comparable	Cases and controls are taken from comparable populations	The assignment of participants to intervention/treatment groups is randomised
	The study indicates how many of the people asked to take part did so in each of the groups being studied	The same inclusion/exclusion criteria are used for cases and controls	An adequate concealment method is used
	The likelihood that some eligible participants might have the outcome at the time of enrolment is assessed and taken into account in the analysis	What percentage of each group (cases and controls) participated in the study?	The design keeps participants and investigators ‘blind’ about treatment allocation
	What percentage of individuals or clusters recruited into each arm of the study dropped out before the study was completed?	Comparison is made between participants and non-participants to establish their similarities or differences	The treatment and control groups are similar at the start of the trial
	Comparison is made between full participants and those lost to follow-up by exposure status	Cases are clearly defined and differentiated from controls	The only difference between groups is the treatment under investigation
Assessment	The outcomes are clearly defined	Measures have been taken to prevent knowledge of primary exposure influencing case ascertainment	All relevant outcomes are measured in a standard, valid and reliable way
	The assessment of outcome is made blind to exposure status. If the study is retrospective, this may not be applicable	Exposure status is measured in a standard, valid and reliable way	All the participants are analysed in the groups to which they were randomly allocated
Confounding	The main potential confounders are identified and taken into account in the design and analysis	The main potential confounders are identified and taken into account in the design and analysis
Statistical analysis	CIs are provided	CIs are provided	CIs are provided
Section 2: overall assessment of the study	How well was the study conducted to minimise the risk of bias or confounding?	How well was the study conducted to minimise the risk of bias or confounding?	How well was the study conducted to minimise bias?
	Methodology used for evaluation and the statistical power of the study, there is clear evidence of an association between exposure and outcome	Methodology used for evaluation and the statistical power of the study, there is clear evidence of an association between exposure and outcome	Methodology used for evaluation and the statistical power of the study, there is clear evidence of an association between exposure and outcome
	The results of this study are directly applicable to the patient/intervention groups targeted by this study	The results of this study are directly applicable to the patient/intervention groups targeted by this study	The results of this study are directly applicable to the patient/intervention groups targeted by this study

CCT, controlled clinical trial; CI, confidence interval; RCT, randomised controlled trial; SIGN, Scottish Intercollegiate Guidelines Network.

Two review authors will independently assess the risk of bias in included studies by developing a checklist. If a sufficient number of randomised studies are found, the Cochrane risk of bias tool recommended by the International Cochrane Collaboration²⁹ will be used (table 3).³⁰

Table 3.

Major criteria for assessing risk of bias adapted from the Cochrane risk of bias tool

Criteria	Questions
Randomisation sequence generation	Was the allocation used to assign participants to the treatment and control groups adequately generated for randomised controlled trials?
Treatment allocation concealment	Was the allocated treatment sufficiently concealed from study participants and clinicians and other healthcare or research staff at the enrolment stage?
Blinding	Were the personnel assessing outcomes and analysing data sufficiently blinded to the allocation of intervention throughout the trial?
Completeness of outcome data	In the published report participant exclusions, lost to follow-up and incomplete outcome of data were adequately addressed?
Selective outcome reporting	Is there any evidence of selective outcome reporting that might have affected the study results?
Other sources of bias	Identify the other possible problems that can create risk or bias.

Analysis

Descriptive analysis

A descriptive synthesis of the included studies will be undertaken, for which we have developed a conceptual framework for analysis (figure 1). We will construct the descriptive synthesis of details outlined in the online supplementary Data Extraction Form.

Figure 1.

Conceptual framework for analysis of outcome measure of registry.

Statistical analysis

It is anticipated that there will be limited scope for meta-analysis because of the range of different outcomes from the published articles. However, should sufficient numbers of studies be identified in which the outcomes of interest are similar (eg, mortality/survival) and which are measured using the same comparator and across the same time interval, the results will be pooled for random-effects meta-analysis. We will consider random-effects meta-analysis because it is unlikely that all studies will be functionally equivalent. We will use Stata V.13 (StataCorp. 2013. Stata Statistical Software: Release 13. College Station, TX: StataCorp LP.) to pool the results of RCTs using a random-effect meta-analysis for continuous outcome using weighted mean differences and for dichotomous outcomes using the risk ratio. In the analysis, 95% CIs and two-sided p values for each outcome will be considered and a forest plot will be created. To measure the heterogeneity of the study, we will compute both the χ² and the I² statistics. We will contemplate an I² value greater than 50% as indicative of considerable heterogeneity. We will conduct sensitivity analysis based on the risk of bias and other factors in order to investigate possible sources of heterogeneity. If the number of studies is very small, we will present the result as separate effects rather than presenting the summary effect. If we do not find enough studies for meta-analysis we will present descriptive analysis for the selected studies.

To identify potential reporting bias, we will initially review the forest plot. If >10 studies are available, funnel plots will be created for further exploration. If required, Egger's linear regression method will be conducted.

Discussion

This review will systematically assess the best evidence on the impact of clinical registries and CQRs on disease outcomes and/or care delivery and will describe economic benefits associated with such registries. It will provide a description on where clinical registries and CQRs have made an impact on patient safety and quality of care.

However, there will be challenges in the review process and in interpreting findings. As discussed above, registries are established for different diseases and conditions and collect different outcomes of interest. Even the capacity of the registry to impact survival will be dependent on the diseases and population being captured. For outcomes other than survival, it will also be difficult to do a pooled analysis. For this reason, we have broadly categorised secondary outcomes into measures of process or outcomes of healthcare (quality of care), healthcare utilisation and costs. It is doubtful that this systematic review will identify RCTs (level II evidence); it is likely that findings from this study will be based on lower level III interventions such as comparative studies with or without concurrent controls. We think it is unlikely that hospitals would be willing to be randomised to developing a registry when it requires such a heavy investment in infrastructure, resources and organisational and cultural change.³¹ This will be discussed when interpreting the strength of the evidence and generalisability of findings.

Challenges in interpreting findings are numerous and will be considered in the analysis of the systematic review. First, the outcomes of interest will be evaluating depends on a complex set of interventions such as institutional structure, institutional processes (human resource and policies) intervening variables (morale, motivation, culture and ability of staff) and clinical processes (review of feedback) making it difficult to quantify the impact of the registry in isolation from other factors.³² Second, data elements and definitions collected by the registry may change over time, making it difficult to assess improvement in outcomes as a continuum. Third, since the value of registries is most likely realised over time, any economic evaluation should include more than the initial establishment costs.

Conclusion

This systematic review of clinical registries will provide a comprehensive summary of the proof of benefits of establishing registries, including CQRs on mortality/survival; and on the process or outcome of healthcare, healthcare utilisation and costs.