Skip to main content
Elsevier - PMC COVID-19 Collection logoLink to Elsevier - PMC COVID-19 Collection
. 2022 May 17;118:106794. doi: 10.1016/j.cct.2022.106794

Data-driven quality improvement program to prevent hospitalisation and improve care of people living with coronary heart disease: Protocol for a process evaluation

Nashid Hafiz a,, Karice Hyun a,b, Qiang Tu a, Andrew Knight c,d, Charlotte Hespe e, Clara K Chow f,g, Tom Briffa h, Robyn Gallagher i, Christopher M Reid j,k, David L Hare l, Nicholas Zwar c,m, Mark Woodward n,o, Stephen Jan n, Emily R Atkins n, Tracey-Lea Laba p, Elizabeth Halcomb q, Tracey Johnson r, Timothy Usherwood n,s, Julie Redfern a,n
PMCID: PMC9110058  PMID: 35589026

Abstract

Background

Practice-level quality improvement initiatives using rapidly advancing technology offers a multidimensional approach to reduce cardiovascular disease burden. For the “QUality improvement in primary care to prevent hospitalisations and improve Effectiveness and efficiency of care for people Living with heart disease” (QUEL) cluster randomised controlled trial, a 12-month quality improvement intervention was designed for primary care practices to use data and implement progressive changes using “Plan, Do, Study, Act” cycles within their practices with training in a series of interactive workshops. This protocol aims to describe the systematic methods to conduct a process evaluation of the data-driven intervention within the QUEL study.

Methods

A mixed-method approach will be used to conduct the evaluation. Quantitative data collected throughout the intervention period, via surveys and intervention materials, will be used to (1) identify the key elements of the intervention and how, for whom and in what context it was effective; (2) determine if the intervention is delivered as intended; and (3) describe practice engagement, commitment and capacity associated with various intervention components. Qualitative data, collected via semi-structured interviews and open-ended questions, will be used to gather in-depth understanding of the (1) satisfaction, utility, barriers and enablers; (2) acceptability, uptake and feasibility, and (3) effect of the COVID-19 pandemic on the implementation of the intervention.

Conclusion

Findings from the evaluation will provide new knowledge on the implementation of a complex, multi-component intervention at practice-level using their own electronic patient data to enhance secondary prevention of cardiovascular disease.

Trial registration: Australian New Zealand Clinical Trials Registry (ANZCTR) number ACTRN12619001790134.

Keywords: Quality improvement, Data, Primary care, Secondary prevention, Cardiovascular disease, Process evaluation, Mixed-method research

List of abbreviations

CVD Cardiovascular Disease
CHD Coronary Heart Disease
PHN Primary Health Networks
QI-PIP Quality Improvement Practice Incentive Program
QI Quality Improvement
QUEL QUality improvement in primary care to prevent hospitalisations and improve Effectiveness and efficiency of care for people Living with coronary heart disease
cRCT Cluster randomised controlled trial
RDS Research Data Storage
PDSA Plan-Do-Study-Act
GP General Practitioner
EPOC Effective Practice and Organisation of Care Review Group

1. Background

Cardiovascular disease (CVD) including coronary heart disease (CHD) and stroke remains the leading cause of death and disease burden worldwide despite decades of significant advances in the prevention and management of CVD [1,2]. .Globally an estimated 17.8 million people die every year from CVD constituting approximately one-third of global deaths [2,3]. The burden of CVD continues to contribute heavily towards the global economic burden due to the associated direct and indirect effects including hospitalisations, medications, post-discharge primary care management, rehabilitation services, disability, and unemployment [4]. As a result, the global cost of CVD is predicted to rise from US$863 billion in 2010 to US$1044 billion by 2030 [5,6]. With the aging population and more people surviving initial cardiac events, the prevalence of CVD is increasing along with the economic cost [7]. To reduce the risk of future cardiovascular events in those with established disease, secondary prevention strategies have become an international priority [8,9] and include the use of guideline-indicated medications, adopting a healthy lifestyle, implementation of chronic disease management plans and participation in a cardiac rehabilitation program following an acute event [8]. Primary care plays an integral role in implementing successful secondary prevention strategies as the majority of people hospitalised for CVD regularly visit their primary care practitioners and use government-funded health services at least once a year following their acute CHD diagnosis [[10], [11], [12], [13]].

Funded by the federal government, primary care is the first point of contact for all Australians to access care. Under the primary care system, individuals can receive services that includes treatment of acute conditions, chronic disease management, health promotion, prevention and early intervention [14]. These services are provided via the general practices, community health centres and allied health practices with the help of Primary Health Networks (PHNs) [14,15]. The Australian government has recently launched the Quality Improvement Practice Incentive Program (QI-PIP), which encourages primary care practices to collaborate with their PHNs and undertake quality improvement activities within their practices to provide high-quality patient care for better health outcomes [16]. As a result, many primary care practices worldwide are rapidly adopting the use of quality improvement (QI) initiatives [[17], [18], [19]]. QI initiatives offer an innovative, multidimensional approach to healthcare and have excellent potential to improve patient outcomes in primary care [20]. Also, current technology has enabled the integration of automated data extraction leading primary care practices to consider data-driven QI programs to provide high-quality patient care [21,22]. Primary care practices have been successfully implementing QI programs in several health conditions including asthma [23], diabetes, neonatal health [24,25]. However, there is a paucity of research focused on evaluating the effectiveness of such QI interventions only at individual patient level rather than community or clinic level in CVD management [26,27]. .The “QUality improvement in primary care to prevent hospitalisations and improve Effectiveness and efficiency of care for people Living with heart disease” (QUEL) study aimed to determine the effectiveness of a quality improvement program for improving CVD management [28].

For the QUEL cluster randomised controlled trial (cRCT), a structured QI program is delivered within Australian primary care practices to reduce CVD hospitalisations, improve CVD risk factors and medication adherence in patients with CHD over 24 months [28]. The intervention practices are supported by the study team or their relevant PHNs to enhance efficiency in management and outcomes of CVD patients by better using their routinely collected data. The trial is ongoing involving 52 (27 intervention and 25 control) Australian primary care practices with approximately 15,000 CVD patients with 12- and 24-month follow-up with data collection scheduled for completion in mid-2022. The primary outcome is CVD hospitalisations, collected via linkage with state-based administrative data linkage centres that collect data on all hospitalisations in Australian hospitals and as such will not be adjudicated, and secondary outcomes are cardiovascular risk factors recorded electronically by the GPs in real time which will be collected routinely across all participating primary care practices using a standardised data extraction software; medication prescriptions and use collected by data linkage of the QUEL cohort with federal level Medicare Benefits Schedule (MBS) and Pharmaceutical Benefits Scheme (PBS) data; and deaths collected via linkage of the same cohort with the federal level National Death Index. Specific details on the cRCT including trial aims, design, sample size, and outcome measures are described elsewhere [28].

The QUEL intervention is based on the Collaborative Framework [29] and consists of (1) a virtual orientation session, (2) electronic data collection at baseline, and thereafter monthly, from the intervention practices via a practice-level software system that enables automated data extraction [30], (3) monthly data reporting, (4) completion of Plan, Do, Study, Act (PDSA) cycles that summarise practice-level progress towards pre-determined CVD indicators for the QUEL study, (5) a series of interactive learning and benchmarking workshops (1 in-person and 5 virtual sessions due to the COVID-19 pandemic) and (6) provision of support from PHNs and the study team. The intervention is delivered over a period of 12 months.

The study team is collaborating with five PHNs to ensure optimal delivery of the collaborative intervention. PHNs are independent organisations funded by the Australian government aimed to coordinate health services for the communities in a specific region [15]. PHNs also work closely with the primary care practices and other health care professionals within the region to identify gaps and build capacity to ensure optimal service delivery [15]. There are thirty-one PHNs operating in Australia including in remote and Aboriginal Torres Strait Islanders communities to encourage use of available health resources and access health care [15]. All PHNs were invited to participate through a variety of communication channels, including a mailing list direct to PHN CEOs, University of Sydney's as well as research partners' networks. Five out of the thirty-one PHNs agreed to collaborate on the study based on their previous experience in QI collaborative and existing collaboration with the University of Sydney. For the QUEL study, each collaborating PHN nominates a primary contact to provide liaison, leadership and coordination to the participating practices within the PHN's jurisdiction. During the intervention period, PHN representatives play a key role in ensuring successful implementation of the intervention. The role of the PHN involves, but is not limited to, supporting practices to achieve pre-defined key performance measures to optimise outcomes, participating in program activity including training and learning workshops, encouraging practice level engagement in these activities and using PDSA cycles between activity periods, sharing practice achievement and providing additional support as required.

Process evaluation is particularly important in complex intervention trials as it provides in-depth information required to evaluate the intervention's effectiveness and investigate the implementation process. It provides valuable insights into describing the various intervention components [31] and identifying factors associated with successes and challenges of the programs in various healthcare settings [32]. Use of process evaluation alongside complex interventions is increasing given because of the associated multisite, multicomponent features [33,34]. However, little research has reported the mechanisms of impact, context and what constitutes effective QI interventions aimed at improving CVD management in primary care settings.

The QUEL QI program is a complex intervention with multiple interactive components, as such, process evaluation can accurately describe the intervention implementation, exposure of the intended intervention and real-time experiences of those involved [35]. We hypothesise that evaluating the implementation of the multi-component QI intervention within the QUEL trial will help primary care practices to undertake further QI activities to improve care of CVD within their practices. The earlier protocol describes the cluster RCT itself [28], while this current protocol details the evaluation plan for the data-driven QI intervention program within the QUEL cRCT and its effects. The process evaluation aims to:

  • 1.

    Explore to what extent the intervention is delivered as intended, identify key elements of the intervention associated with positive study outcomes, and how, for whom and in what context it was effective.

  • 2.

    Describe and analyse practice engagement, attendance, time commitment, software capability, skills and capacity of the practice team members associated with attending learning workshops.

  • 3.

    Understand acceptability, satisfaction, uptake, utility and feasibility of the QI program.

  • 4.

    Identify and describe barriers and enablers of the QI program.

  • 5.

    Evaluate the effect of COVID – 19 on the implementation of the QI program.

2. Methods

2.1. Study design

A mixed-methods approach will be undertaken using data from 27 intervention practices (out of 52 participating practices) from the QUEL cRCT [36,37]. For this study, data will be collected only from the intervention practices as it aims to evaluate the effect of the QI intervention program. Qualitative and quantitative data will be collected both during and at the end of the trial intervention period. Semi-structured interviews and open-ended questions will be used to collect qualitative data. Quantitative data will be collected from the intervention practices via multiple data sources throughout the intervention period.

A program logic model was developed to describe how, why and among whom the collaborative intervention works in practices within the QUEL cRCT (Fig. 1 ). This logic model is a visual representation of the intervention design and its intended implementation. The Cochrane Effective Practice and Organisation of Care Review Group (EPOC) checklist was used as a guidance to develop the logic model [38] to identify the key features of the intervention, check the fidelity of the implementation and assess participant's experience [31]. The model includes 5 domains of the intervention: (1) input, (2) activities, (3) outputs, (4) outcome and (5) impact specific to the data-driven QI program that will be used to describe the study objectives [39]. Inputs refer to various resources that are required to ensure program operation, activities refer to the planned actions, such as delivery of workshops, data collection that are an essential part of the implementation [39]. Resources, inputs, and activities together form the program design. Outputs include the changes in the participant's behaviour, knowledge, skills, and awareness resulting from the activities and impact describes the fundamental changes occurring in the health services over a longer period as a result of the program activities [40].

Fig. 1.

Fig. 1

Logic model for data-driven Quality improvement (QUEL) intervention process evaluation.

CVD: Cardiovascular Disease, CHD: Coronary Heart Disease, PEN CS: Pen Computer Systems, GP: General Practitioner, PDSA: Plan, Do, Study. Act, PHC: Primary Health Care, IT; Information Technology, PHN: Primary Health Network, QI: Quality Improvement.

2.2. Participants

Participants in the process evaluation will include practice team members (including general practitioners, nurses and practice managers) from primary care practices allocated to the intervention arm and PHN staff who are providing direct support to the intervention practices under their jurisdiction. At least two practice team members from all 27 intervention practices who were actively involved in QI activities in their practices; such as participated in QI workshops, submitted and carried out PDSA cycles and regularly communicated and shared reports with the study team on their activities during the intervention period will be approached to complete the surveys and participate in semi-structured interviews. These participants will be able to understand sufficient English to provide written and informed consent. Practice team members from primary care practices allocated to the control group and any PHN staff not involved in the QUEL project will not be included in the process evaluation. All practice team members who are part of the intervention practices will be approached for recruitment to provide feedback and participate in interviews during and at the end of the intervention.

2.3. Data sources

Multiple data sources, collected throughout the cRCT, will be used in addition to surveys and interview data, to evaluate whether the complex intervention was delivered as planned. Combining these data sources will help to identify the key intervention elements, identify the dose, frequency and activities delivered to the intervention practices as well as describe barriers and enablers associated with the program implementation. To maintain balanced quality of information across the multiple data sources the research team will ensure a) close communications and interaction with practices and PHNs (e.g. workshops; practice visits, regular contact via email or phone calls) to promote quality of data collection; b) an experienced research officer is responsible for data collection throughout the study; c) all the practices are well informed on the study procedures before they are enrolled; d) the participating practices receive appropriate research support when required; e) routine extraction, monitor and check data for quality assurance and help practices solve issues if data is not returned. The data sources will include: 1) practice-level enrolment data, 2) attendance record, 3) SharePoint data, 4) practice correspondence record, 5) data collection record, 6) PDSA cycles, 7) learning workshop surveys, 8) end of program survey and 9) semi-structured interviews of practice team members and PHN representatives. These data sources will be used as credible evidence collected at different time point during the intervention period (Fig. 1).

2.3.1. Practice-level enrolment data

Practice-level enrolment data will be created at the time of recruitment and will be recorded in a Microsoft Excel (2016) spreadsheet. Information collected will include practice location (urban and rural), practice team members information, software compatibility, and randomisation group. Urban and rural primary care practices were defined using the Australian department of Health's Health workforce classification guideline [41]. This spreadsheet will be used throughout the main trial period and updated regularly with current dates and version numbers. We will use these data to identify intervention practices, software eligibility and installation requirement of the eligible software, practice support, describe practice type and gather details of the practice team members involved in the delivery of the intervention.

2.3.2. Attendance records

Participation of the practice team members in any events related to the intervention including orientation and learning workshops (both face-to-face and virtual) will be recorded in another Microsoft Excel (2016) spreadsheet and updated regularly throughout the intervention period. These data will provide information on the frequency of the workshops attended by the intervention practices, the number of staff from each practice attending the orientation and workshops.

2.3.3. SharePoint data

Microsoft SharePoint [42] is an online platform where a unique account is created for individual practices in both intervention and control arms. This platform is created for the practices to submit their PDSA cycle records and track improvements via monthly graphs which are uploaded in their respective accounts by the study team. From the SharePoint data, we will identify whether each practice had access to their account and all intervention materials including workshop recordings and lectures, monthly feedback reports, frequency and number of PDSA cycles submitted by each practice.

2.3.4. Practice correspondence record

The study team will be communicating with the intervention practices during the trial period and practices will also be encouraged to directly communicate with the study team as required. These communications will be undertaken via phone call, email or in-person site visits. Any communication will be saved and used to identify the reason, mode of contact (email, site visit or phone call), time spent on the contact, person contacted and solution provided in a Microsoft Excel (2016) spreadsheet. This document will be updated throughout the study period with current dates and version numbers.

2.3.5. Data collection record

The intervention practices will submit clinical data electronically in an aggregated and de-identified form monthly via the automated data extraction software [30]. All aggregated data will be stored in the University's Research Data Storage (RDS). These data will be used to create practice level reports and will be uploaded to practices' SharePoint sites monthly as graphs for benchmarking their improvement for the pre-defined QUEL study performance measures. A Microsoft Excel (2016) spreadsheet will be used to record monthly data collection and reporting for each intervention practice. This spreadsheet will also be updated regularly throughout the intervention period with current dates and version numbers.

2.3.6. Plan, Do, Study, Act cycles

The PDSA cycle is a simple but powerful tool to measure improvements and increasingly used in many QI collaborative to boost quality of healthcare [43,44]. It guides users to explicitly plan, implement, reflect on, and then repeat, incremental improvements as they make system changes to achieve the aim [45]. Practices participating in the QUEL QI intervention are required to document and upload their PDSA cycles using a template. Training will be provided to the practices during the learning workshops on the process of completing PDSAs. Submitted PDSA cycles will be saved in their respective SharePoint accounts and the study team will be able to download a copy of the cycle when required. We will use all the PDSA cycles submitted by primary care practices during the intervention period to gather information on practice engagement, number of PDSAs submitted by each practice, identify key areas practices focused on improving and identify barriers and enablers to make improvement changes within the practices.

2.3.7. Learning workshop surveys

At least two practice team members (one clinical and one administrative) will be invited to participate in a series of six learning workshops that will be delivered during the intervention period. Six surveys corresponding to six workshops will be administered at the end of each workshop (paper-based for in-person workshops and online for the virtual workshops) to the workshop attendees. Each survey will contain questions that are specific to the workshop content and a set of common questions that will be asked at every workshop. The common questions include feedback on the workshop evaluation, learnings, satisfaction and suggestions for improvement collected as Likert scale and free-text response The surveys aim to evaluate practice engagement, workshop attendance, time commitment, staff skills and capacity involved in implementing QI changes in their practice. The survey will also be used to evaluate the appropriateness of content and the effectiveness in terms of practice-level implementation of QI.

2.3.8. End of program surveys

Practice team members including general practitioners, practice managers or nurses from QUEL study intervention practices who are actively involved in implementing QI changes within their practices will be invited to complete a comprehensive survey on the overall program at the end of the intervention. To ensure as many responses as possible, the survey will be sent by post, with a return address envelope, email, online or by direct contact. The survey aims to evaluate the whole intervention and examine acceptability, satisfaction, uptake, utility and feasibility among users. The survey will include fifty-five questions, of which forty questions require Likert scale responses focused on overall workshop content, design, facilitators, results and outcome; practice software usability, use of electronic data for the management of CVD patients, quality and satisfaction of care provided by the primary care team, impact of the intervention on the quality of care provided, leadership involvement and staff capacity. Six questions will require yes or no responses with possible further explanation which focuses on QI-PIP [16], SharePoint use and access. Nine questions will allow free text responses focused on sharing experience implementing changes within the practice, change in staff role, sharing feedback on different intervention components. These free text questions will also include questions and discussion points on the effect of COVID-19, which will provide detailed information to help us evaluate its effect on the intervention implementation.

2.3.9. Interviews with practice team members and PHN representatives

Practice team members participating in the QUEL study will be invited by email, telephone, or post to take part in a confidential one-on-one interview at the end of the intervention. Practices will be selected based on their performance (high, low and medium); which will be defined by the practices' interaction during the intervention period such as participation on the learning workshops, submission of PDSA cycles. We will invite practice team members from at least three practices from each high, medium and low performing tier to participate in the interviews to ensure minimum bias. The purpose of these interviews is to evaluate workforce capability, describe perceived benefits, barriers or successes to implementation, uptake, and acceptability of the program. We will also be able to explore the differences in the dose, frequency and the activities implemented by the practices which were supported by the PHNs vs the study team. The interviews will also expand on themes within the surveys to triangulate these data; explore their experiences with QI strategies, gain detailed insight into the staff involvement, changes that occurred in the staff role due to the program. Interviews will be also used to explore the capacity of the practice software and the use of electronic data extracted from the software for QI program implementation. The semi-structured interviews will also explore how the COVID-19 affected the implementation of the intervention and will provide us with information on different approaches taken by the practices to overcome the challenges.

Interviewing the PHN representatives will enable us to describe the role of PHN in implementing QI intervention in primary care. With the interviews, we will obtain in detail PHN's perspective on the program, barriers and enablers to implementation, practice engagement, time commitment and efforts required by PHN representatives.

Semi-structured interviews are widely used in healthcare research to collect open-ended, qualitative data and to explore in-depth understanding of a specific topic [46,47]. For the evaluation, we estimate a sample of approximately 10 interviews from different suburban locations, including a variety of practice team members such as GPs, nurses and practice managers and PHN representatives reflecting diverse participant demography. However, the final number of interviews will be dependent on the thematic saturation. A trained researcher will conduct and audio record the interviews of approximately 45 min duration at the practice or health service, or via telephone, as convenient for the participants [48]. A topic-centred discussion guide will be used by the interviewer to conduct the semi-structured interviews to ensure the topics are systematically explored [48]. The researcher may take notes during the interview to document relevant information.

2.4. Data analysis

Descriptive statistical analysis will be used to analyse quantitative data. Responses and measurements from all data sources will be presented as numbers and percentages for categorical variables and mean and standard deviation or median and interquartile intervals for continuous variables. This will help to understand the level of satisfaction and perceived utility associated with program implementation and explain the extent to which the intervention is delivered as intended. The quantitative data will be compared between the following subgroups: rural vs urban and small (≤2 GPs) and large primary (>2 GPs) care practices. Chi-squared test or Fisher's exact test will be used to compare categorical variables between the subgroups and independent t-test or Wilcoxon rank-sum test will be used to compare continuous variables.

Qualitative data, including semi-structured interviews and free-text responses from the surveys and PDSA cycles, will undergo thematic analysis [49]. The thematic analysis will include preparing and transcribing the data, familiarising and coding, generating, reviewing and defining the themes and writing up the interpretation of the data [49,50]. Two independent researchers will thematically analyse interview transcripts. All data collected will be converted into electronic format and stored in one location. Interviews will be recorded with consent and verbatim method will be used for transcription of the interviews. Interview transcript, any free text and notes from the interviews as well as the surveys and PDSAs will be coded and managed in NVivo Software.

2.5. Data storage, retention and disposal

All data collected for the process evaluation including personal information will be securely stored in the University's RDS database. Access to the RDS will require an employee unikey and password and only a limited number of people will have access to it. All data will be stored on The University's RDS for the duration necessary to comply with regulatory requirements; thereafter database will be destroyed in accordance with University's IT recommendations. Completed surveys and interview data will be stored securely for 5 years after publications, after which time, they will be destroyed securely. No personal information will be published. At the end of this data retention period, all files will be physically destroyed.

3. Discussion

This protocol outlines the systematic methods of a process evaluation of a complex QI intervention embedded within a cRCT to improve secondary prevention of CVD in primary care. The evaluation logic model is described along with methods for understanding the impact of the intervention and the context in which the impact occurs. It will assess successes and failures related to the program implementation in addition to determining factors associated with program scale-up and adaptation for other primary care settings [32]. The EPOC framework does not provide information on describing the actual QI intervention, therefore this evaluation also includes interviewing of key participants at completion of the intervention. The study will contribute to stronger evidence around the use of QI in primary care to improve CVD outcomes as well as to literature through encouraging the development of process evaluation methodology in the design and promoting transparency in the reporting of the findings.

A strength of our study is the use of a mixed-methods approach. Mixed-methods research can strengthen data quality, improve interpretations of findings, and offer a more comprehensive understanding of the program implementation, and hence, it has become a very useful tool to evaluate complex interventions [51]. Quantitative data will provide key information on what was effective and qualitative data will provide deeper understanding of why and in what context the intervention was effective. Therefore, combination of both will provide a more holistic understanding of the complex intervention than either method alone [52]. Furthermore, this approach enables a richer perspective from a range of participants (GPs, nurses, practice managers, and PHN representatives) using various surveys and semi-structured interviews integrated within the main QUEL cRCT. Combining interview and survey data will enable in-depth knowledge on program utility, barriers, and likelihood of adoption. Findings from the process evaluation will also inform other primary care practices to implement data-driven QI programs and provide valuable insights to policymakers on wider adoption and scaling-up of such strategies.

While this process evaluation will enable evaluation of a complex QI intervention and barriers and enablers to its implementation, there are several limitations. One of the limitations is the data collected from the interviews may be subjected to recall bias as interviews will take place after the intervention. Another limitations is the cRCT is designed to be delivered in Australian primary care environment, hence it may only be relevant to health systems with similar contexts, funding, and infrastructures.

4. Conclusion

At the completion of the evaluation, we will gather rich data about collaborative implementation in terms of key features, impact, barriers and enablers and other factors including collaborative teams, staff resistance, use of experienced resources within the team to train staff specific to improving care of CVD management in primary care. Results from the evaluation will also contribute to further high-quality evidence regarding the implementation of quality improvement programs in primary care. This process evaluation will therefore help identify gaps in implementation and influence practice-level decision making in adopting data-driven quality improvement strategies and to improve CVD management.

Ethics and dissemination

The study is approved by the New South Wales Population & Health Services Research Ethics Committee (HREC/18/CIPHS/44). The ethics committee provides approval for all four participating states including New South Wales, Australian Capital Territory, Victoria, Queensland, and South Australia under the National Mutual Scheme. Participants who are participating in the process evaluation will be provided with a Participant Information Sheet and Consent Form and written consent will be obtained from each of the participants. Written informed consent will be obtained from participants and only de-identified data will be analysed and report. Results of this process evaluation as well as the cRCT, will be communicated through peer-reviewed publications and presentations at scientific forums including national and international conferences. Published papers, reports and any barriers, enablers, and key outcome identified through the results will be shared among national stakeholder organisations, participating practices and clinical networks.

Disclaimer

The funding body and industry partners was not involved in the design of the study; and will not have any role during its execution, analyses, interpretation of the data, or decision to submit results.

Data statement

Not applicable.

Funding

Funding for this study was provided by a National Health and Medical Research Council (NHMRC) Partnership Project Grant (Award Grant Number: GNT1140807). Additional in-kind and cash support from the following partner organisations; Amgen (cash support), Austin Health, Australian Cardiovascular Health and Rehabilitation Association, Australian Commission on Safety and Quality in Health Care, Australian Primary Health Care Nurses Association, Brisbane South PHN, Fairfield General Practice Unit, Heart Support Australia, Improvement Foundation, Inala Primary Care, National Heart Foundation of Australia, Nepean Blue Mountains PHN (cash support), Royal Australian College of General Practitioners, Sanofi (provided cash support via the Externally Sponsored Collaboration pathway), South Western Sydney PHN, The George Institute for Global Health (cash support) and University of Melbourne. JR is supported by a NHMRC Career Development Fellowship (APP1143538). KH is supported by the NHMRC Investigator Grant (Emerging leadership 1) (APP1196724). MW is supported by the NHMRC grants (1080206 and 1149987). CR is supported by a NHMRC Principal Research Fellowship (APP1136372). TL is funded by a NHMRC Early Career Fellowship (APP110230). EA is supported by a National Heart Foundation Australia postdoctoral fellowship (101884). CC's salary is funded by a Career Development Fellowship level 2 co-funded by the NHMRC and National Heart Foundation Future Leader Award (APP1105447), which supports 0.05FTE for trial meetings.

Authors' contributions

NH, JR and KH drafted the protocol. JR, NH, KH, AK, CH, TU helped with intervention design. JR, NH, KH and QT managed ethics and legal approvals. NH, JR, KH and QT involved in data collection and management. All authors reviewed and approved the final manuscript.

Declaration of Competing Interest

Amgen and Sanofi Australia has provided cash support to the main cRCT. MW is a consultant to Amgen, Freeline and Kyowa Kirin. Other authors has nothing to disclose.

Acknowledgements

The authors acknowledge the support of all the PHN and primary care practices who continue to support the QUEL project. Also, Pen CS for providing the services and eHealth data platform for the study; and the Improvement Foundation for their continuous support in the delivery of the QI program and other study partners including; Inala Primary Care, Fairfield Hospital General Practice Unit, Australian Primary Health Care Nurses Association, Royal Australian College of General Practitioners, Australian Commission on Safety and Quality in Health Care, Heart Support Australia Ltd., Austin Health, Australian Cardiovascular Health and Rehabilitation Association, National Heart Foundation, Sanofi, and Amgen. The authors would also like to acknowledge the ongoing contribution of Kane Williams in the legal arrangement and Caroline Wu in the research management of the trial.

Footnotes

Appendix A

Supplementary data to this article can be found online at https://doi.org/10.1016/j.cct.2022.106794.

Appendix A. Supplementary data

Peer-review application assessment

mmc1.docx (20KB, docx)

Peer-review application assessment Outcome

mmc2.docx (142.4KB, docx)

HREC Ethics approval

mmc3.pdf (321.6KB, pdf)

HREC Ethics amendment approval

mmc4.pdf (134.5KB, pdf)

References

  • 1.Bansilal S., Castellano J.M., Fuster V. Global burden of CVD: focus on secondary prevention of cardiovascular disease. Int. J. Cardiol. 2015;201:S1–S7. doi: 10.1016/S0167-5273(15)31026-3. [DOI] [PubMed] [Google Scholar]
  • 2.World Health Organization Cardiovascular diseases CVDs Fact Sheet. 2017. https://www.who.int/news-room/fact-sheets/detail/cardiovascular-diseases-(cvds)
  • 3.World Heart Federation Cardiovascular Disease: The World's Number 1 killer. Infographics. 2021. https://world-heart-federation.org/wp-content/uploads/2021/04/WHF-CVD-Infographic.pdf
  • 4.Mela A., Rdzanek E., Poniatowski Ł.A., Jaroszyński J., Furtak-Niczyporuk M., Gałązka-Sobotka M., et al. Economic costs of cardiovascular diseases in Poland estimates for 2015–2017 years. Front. Pharmacol. 2020;11(1231) doi: 10.3389/fphar.2020.01231. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.World Heart Federation Champion Advocates Programme: The costs of CVD. 2021. http://www.championadvocates.org/en/champion-advocates-programme/the-costs-of-cvd
  • 6.Kuehn B.M. Costs of cardiac care likely to increase, despite advances in prevention, care. JAMA. 2013;310(19):2029. doi: 10.1001/jama.2013.282805. [DOI] [PubMed] [Google Scholar]
  • 7.Redfern J., Chow C.K. Secondary prevention of coronary heart disease in Australia: a blueprint for reform. Med. J. Aust. 2013;198(2):70–71. doi: 10.5694/mja12.11080c. [DOI] [PubMed] [Google Scholar]
  • 8.Chew D.P., Scott I.A., Cullen L., French J.K., Briffa T.G., Tideman P.A., et al. National Heart Foundation of Australia and Cardiac Society of Australia and New Zealand: Australian clinical guidelines for the management of acute coronary syndromes 2016. Med. J. Aust. 2016;205(3):128–133. doi: 10.1016/j.hlc.2016.06.789. [DOI] [PubMed] [Google Scholar]
  • 9.Ibanez B., James S., Agewall S., Antunes M.J., Bucciarelli-Ducci C., Bueno H., et al. 2017 ESC Guidelines for the management of acute myocardial infarction in patients presenting with ST-segment elevation: The Task Force for the management of acute myocardial infarction in patients presenting with ST-segment elevation of the European Society of Cardiology (ESC) Eur. Heart J. 2017;39(2):119–177. doi: 10.1093/eurheartj/ehx393. [DOI] [PubMed] [Google Scholar]
  • 10.Guo S., Oberst C., Mathur S. Australian Institute of Health and Welfare; 2018. Transition between hospital and community care for patients with coronary heart disease: New South Wales and Victoria 2012–2015. [Google Scholar]
  • 11.Einarsdóttir K., Preen D.B., Emery J.D., Holman C.D.A.J. Regular primary care plays a significant role in secondary prevention of ischemic heart disease in a Western Australian cohort. J. Gen. Intern. Med. 2011;26(10):1092–1097. doi: 10.1007/s11606-011-1665-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Tomasik T., Windak A., Seifert B., Kersnik J., Palka M., Margas G., et al. The self-perceived role of general practitioners in care of patients with cardiovascular diseases. A survey in central and eastern European countries following health care reforms. Int. J. Cardiol. 2013;164(3):327–333. doi: 10.1016/j.ijcard.2011.07.007. [DOI] [PubMed] [Google Scholar]
  • 13.Fors A., Ekman I., Taft C., Björkelund C., Frid K., Larsson M.E., et al. Person-centred care after acute coronary syndrome, from hospital to primary care—a randomised controlled trial. Int. J. Cardiol. 2015;187:693–699. doi: 10.1016/j.ijcard.2015.03.336. [DOI] [PubMed] [Google Scholar]
  • 14.Department of Health Primary Care. Canberra, ACT, Australia. 2021. https://www1.health.gov.au/internet/main/publishing.nsf/Content/primarycare
  • 15.Department of Health . ACT, Australia; Canberra: 2021. What Primary Health Networks Are.https://www.health.gov.au/initiatives-and-programs/phn/what-phns-are [Google Scholar]
  • 16.Department of Health PIP QI Incentive guidance Canberra, ACT, Australia. 2021. https://www1.health.gov.au/internet/main/publishing.nsf/Content/PIP-QI_Incentive_guidance
  • 17.Balasubramanian B.A., Marino M., Cohen D.J., Ward R.L., Preston A., Springer R.J., et al. Use of quality improvement strategies among small to medium-size us primary care practices. Ann. Fam. Med. 2018;16:S35–S43. doi: 10.1370/afm.2172. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Knight A.W., Dhillon M., Smith C., Johnson J. A quality improvement collaborative to build improvement capacity in regional primary care support organisations. BMJ Open Qual. 2019;8(3) doi: 10.1136/bmjoq-2019-000684. e000684-e. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Øvretveit J., Bate P., Cleary P., et al. Quality collaboratives: lessons from research. Quality and safety in health care. 2002;11(4):345–351. doi: 10.1136/qhc.11.4.345. (Batalden PB, Davidoff F. What is “quality improvement” and how can it transform healthcare? BMJ Publishing Group Ltd; 2002;11:345-351. doi:10.1136/qhc.11.4.345) [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Batalden P.B., Davidoff F. What is “quality improvement” and how can it transform healthcare? Qual Saf Health Care. 2007;16(1):2–3. doi: 10.1136/qshc.2006.022046. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Wells S., Tamir O., Gray J., et al. Are quality improvement collaboratives effective? A systematic review. BMJ quality & safety. 2018;27:226–240. doi: 10.1136/bmjqs-2017-006926. [DOI] [PubMed] [Google Scholar]
  • 22.Langley G.J., Nolan K.M., Nolan T.W. The foundation of improvement. Qual. Prog. 1994;27(6):81–86. [Google Scholar]
  • 23.Homer C.J., Szilagyi P., Rodewald L., Bloom S.R., Greenspan P., Yazdgerdi S., et al. Does quality of care affect rates of hospitalization for childhood asthma? Pediatrics. 1996;98(1):18–23. [PubMed] [Google Scholar]
  • 24.Tricco A.C., Ivers N.M., Grimshaw J.M., Moher D., Turner L., Galipeau J., et al. Effectiveness of quality improvement strategies on the management of diabetes: a systematic review and meta-analysis. Lancet. 2012;379(9833):2252–2261. doi: 10.1016/S0140-6736(12)60480-2. [DOI] [PubMed] [Google Scholar]
  • 25.Schouten L.M., Hulscher M.E., van Everdingen J.J., Huijsman R., Grol R.P. Evidence for the impact of quality improvement collaboratives: systematic review. BMJ. 2008;336(7659):1491. doi: 10.1136/bmj.39570.749884.BE. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Coorey G., Peiris D., Neubeck L., et al. A realist evaluation approach to explaining the role of context in the impact of a complex eHealth intervention for improving prevention of cardiovascular disease. BMC Health Serv. Res. 2020;20(1):764. doi: 10.1186/s12913-020-05597-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Patel B., Usherwood T., Harris M., Patel A., Panaretto K., Zwar N., et al. What drives adoption of a computerised, multifaceted quality improvement intervention for cardiovascular disease management in primary healthcare settings? A mixed methods analysis using normalisation process theory. Implement. Sci. 2018;13(1):140. doi: 10.1186/s13012-018-0830-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Redfern J., Hafiz N., Hyun K., et al. QUality improvement in primary care to prevent hospitalisations and improve effectiveness and efficiency of care for people living with coronary heart disease (QUEL): protocol for a 24-month cluster randomised controlled trial in primary care. BMC Fam. Pract. 2020;21(1):36. doi: 10.1186/s12875-020-01105-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Langley G.J., Moen R.D., Nolan K.M., Nolan T.W., Norman C.L., P. L. 2nd ed. Jossey Bass Wiley; Chichester, England: 2009. The Improvement Guide: A Practical Approach to Enhancing Organizational Performance. [Google Scholar]
  • 30.PEN Computer Systems CAT4 Overview. https://www.pencs.com.au/products/cat4/
  • 31.Grant A., Dreischulte T., Guthrie B. Process evaluation of the data-driven quality improvement in primary care (DQIP) trial: active and less active ingredients of a multi-component complex intervention to reduce high-risk primary care prescribing. Implement. Sci. 2017;12(1):4. doi: 10.1186/s13012-016-0531-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Stephens T.J., Peden C.J., Pearse R.M., Shaw S.E., Abbott T.E.F., Jones E.L., et al. Improving care at scale: process evaluation of a multi-component quality improvement intervention to reduce mortality after emergency abdominal surgery (EPOCH trial) Implement. Sci. 2018;13(1):142. doi: 10.1186/s13012-018-0823-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Fisher J., Nguyen T., Tran T.D., Tran H., Tran T., Luchters S., et al. Protocol for a process evaluation of a cluster randomized controlled trial of the learning Club intervention for women’s health, and infant’s health and development in rural Vietnam. BMC Health Serv. Res. 2019;19(1):511. doi: 10.1186/s12913-019-4325-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Mann C., Shaw A., Guthrie B., Wye L., Man M.-S., Hollinghurst S., et al. Protocol for a process evaluation of a cluster randomised controlled trial to improve management of multimorbidity in general practice: the 3D study. BMJ Open. 2016;6(5) doi: 10.1136/bmjopen-2016-011260. e011260-e. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Hulscher M.E., Laurant M.G., Grol R.P. Process evaluation on quality improvement interventions. BMJ Quality & Safety. 2003;12(1):40–46. doi: 10.1136/qhc.12.1.40. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Creswell J.W. 5th ed. SAGE Publications; Thousand Oaks, CA: 2017. Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. [Google Scholar]
  • 37.Creswell J.W., Plano Clark V.L. SAGE Publications; Thousand Oaks, CA: 2017. Designing and Conducting Mixed Methods Research. [Google Scholar]
  • 38.Cochrane Collaboration . Data collection checklist; 2009. Cochrane Effective Practice and Organisation of Care Review Group (EPOC) [Google Scholar]
  • 39.Knowlton L.W., Phillips C.C. The logic model guidebook: better strategies for great results. Sage. 2012:131–133. [Google Scholar]
  • 40.WK Kellogg Foundation . WK Kellogg Foundation; 2004. WK Kellogg Foundation Logic Model Development Guide. [Google Scholar]
  • 41.Department of Health . ACT; Australia: 2021. Rural, Remote and Metropolitan Area Canberra.https://www.health.gov.au/health-topics/rural-health-workforce/classifications/rrma Available from. [Google Scholar]
  • 42.Corporation M What is SharePoint? https://support.microsoft.com/en-us/office/what-is-sharepoint-97b915e6-651b-43b2-827d-fb25777f446f
  • 43.Donnelly P., Kirk P. Use the PDSA model for effective change management. Education for Primary Care. 2015;26(4):279–281. doi: 10.1080/14739879.2015.11494356. [DOI] [PubMed] [Google Scholar]
  • 44.Leis J.A., Shojania K.G. A primer on PDSA: executing plan–do–study–act cycles in practice, not just in name. BMJ quality & safety. 2017;26(7):572–577. doi: 10.1136/bmjqs-2016-006245. [DOI] [PubMed] [Google Scholar]
  • 45.Harrington J.T., Newman E.D. Redesigning the care of rheumatic diseases at the practice and system levels. Part 1: practice level process improvement (redesign 101) Clin. Exp. Rheumatol. 2007;25(6 Suppl 47):55–63. [PubMed] [Google Scholar]
  • 46.Jamshed S. Qualitative research method-interviewing and observation. Journal of basic and clinical pharmacy. 2014;5(4):87–88. doi: 10.4103/0976-0105.141942. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Pope C., Van Royen P., Baker R. Qualitative methods in research on healthcare quality. BMJ Quality & Safety. 2002;11(2):148–152. doi: 10.1136/qhc.11.2.148. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.DiCicco-Bloom B., F-Crabtree B.F. The qualitative research interview. Med Educ. 2006;40(4):314–321. doi: 10.1111/j.1365-2929.2006.02418.x. [DOI] [PubMed] [Google Scholar]
  • 49.Maguire M., Delahunt B. Doing a thematic analysis: a practical, step-by-step guide for learning and teaching scholars. All Ireland. J. High. Educ. 2017;9(3) http://ojs.aishe.org/index.php/aishe-j/article/view/3354 [Google Scholar]
  • 50.Lester J.N., Cho Y., Lochmiller C.R. Learning to do qualitative data analysis: a starting point. Hum. Resour. Dev. Rev. 2020;19(1):94–106. doi: 10.1177/1534484320903890. [DOI] [Google Scholar]
  • 51.Bamberger M. Introduction to mixed methods in impact evaluation. Impact Evaluation Notes. 2012;3(3):1–38. [Google Scholar]
  • 52.Malina M.A., Nørreklit H.S., Selto F.H. Lessons learned: advantages and disadvantages of mixed method research. Qual. Res. Account. Manag. 2011;Vol. 8 No. 1:59–71. [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Peer-review application assessment

mmc1.docx (20KB, docx)

Peer-review application assessment Outcome

mmc2.docx (142.4KB, docx)

HREC Ethics approval

mmc3.pdf (321.6KB, pdf)

HREC Ethics amendment approval

mmc4.pdf (134.5KB, pdf)

Articles from Contemporary Clinical Trials are provided here courtesy of Elsevier

RESOURCES