Development and evaluation of an intervention based on the provision of patient feedback to improve patient safety in Spanish primary healthcare centres: study protocol

Abstract

Introduction

Despite the enormous potential for adverse events in primary healthcare (PHC), the knowledge about how to improve patient safety in this context is still sparse. We describe the methods for the development and evaluation of an intervention targeted at PHC professionals to improve patient safety in Spanish PHC centres.

Methods and analysis

The intervention will consist in using the patient reported experiences and outcomes of safety in primary care (PREOS-PC) survey to gather patient-reported experiences and outcomes concerning the safety of the healthcare patients receive in their PHC centres, and feed that information back to the PHC professionals to help them identify opportunities for safer healthcare provision. The study will involve three stages. Stage 1 (developing the intervention) will involve: (i) qualitative study with 40 PHC providers to optimise the acceptability and perceived utility of the proposed intervention; (ii) Spanish translation, cross-cultural adaptation and validation of the PREOS-PC survey; (iii) developing the intervention components; and (iv) developing an online tool to electronically administrate PREOS-PC and automatically generate feedback reports to PHC centres. Stage 2 (piloting the intervention) will involve a 3-month feasibility (one group pre-post) study in 10 PHC centres (500 patients, 260 providers). Stage 3 (evaluating the intervention) will involve: (i) a 12-month, two-arm, two-level cluster randomised controlled trial (1248 PHC professionals within 48 PHC centres; with randomisation at the centre level in a 1:1 ratio) to evaluate the impact of the intervention on patient safety culture (primary outcome), patient-reported safety experiences and outcomes (using the PREOS-PC survey), and avoidable hospitalisations; (ii) qualitative study with 20 PHC providers to evaluate the acceptability and perceived utility of the intervention and identify implementation barriers.

Ethics and dissemination

The study was approved by the Ethics Committee of the Balearic Islands (CEI IB: 3686/18) with the 1964 Helsinki Declaration and its later amendments. The results will be disseminated in peer-reviewed publications and national and international conferences.

Trial registration number

NCT03837912; pre-results.

Strengths and limitations of this study.

• We propose the use of a theory-based intervention.

• Both patients’ and providers’ views have been taken into account in the design of the intervention.

• The intervention has the potential to be highly scalable and sustainable for the Spanish National Health Service.

• A high proportion of missing primary healthcare professionals outcome data may compromise the validity of our findings.

Introduction

Patient safety has been defined as ‘the avoidance, prevention, and amelioration of adverse outcomes or injuries stemming from the processes of healthcare’,¹ and has been on the research agenda since the publication of the report ‘To Err is Human’² in 2000. A recent meta-analysis estimated that around 1 in 20 patients are exposed to preventable harm in medical care.³ Over the last two decades a substantial body of work has been undertaken to understand the reasons for patient safety incidents to occur in the hospital setting, but far less is known about the nature, causes or consequences of incidents in the primary care setting—which is where the majority of medical consultations take place.⁴ This may be due to the assumption that primary care is a low technology environment where safety would not be a major problem. However, a recent systematic review including studies from 21 different countries⁵ estimated that two to three patient safety incidents occur per 100 primary care consultations, and 4% of them result in severe harm (long‐term physical or psychological problems or death). Most common causes of harm are related with diagnosis (either delayed or missed) or to treatment (delayed or inappropriate)-related incidents.⁶ A number of factors contribute to these incidents such as the working environment, information transfer at the primary–secondary interface,⁷ doctor–patient relationship⁸ or continuing education.⁹ The direct costs of harm (additional tests, treatments and healthcare) are around 2.5% of total health expenditure.¹⁰

In Spain (country with the highest primary healthcare (PHC) frequentation figures in Europe), the PHC is organised into 2700 PC centres, where the professionals work in teams. Each team includes on average 10 doctors, 2 paediatricians, 12 nurses, midwife, social worker and admin staff.^{11 12} During the last decade we have witnessed an increasing interest around patient safety in the Spanish PHC centres. The Prevalence of Adverse Events in Primary healthcare in Spain (APEAS) study,¹³ which involved 48 PHC centres from 16 regions, estimated that each year 3 million adverse events occur in the Spanish PHC centres, of which around two thirds are preventable.

Improving safety culture (defined as the product of individual and group values, attitudes, perceptions, competencies and patterns of behaviour that determine the commitment to, and the style and proficiency of, an organisation’s health and safety management¹⁴) is ‘the biggest challenge to moving toward a safer health system’ according to the Institute of Medicine.¹⁵ Notwithstanding the increasing efforts to develop effective strategies to improve patient safety in PHC centres through enhancing patient safety culture and reducing preventable adverse events and harm,^{16 17} the available evidence base concerning the effectiveness of the different strategies proposed up until now is still limited.^{18 19} To tackle this important problem, international organisations such as the WHO,²⁰ the Organisation for Economic Co-operation and Development (OECD)¹⁰ or the US Agency for Healthcare Research and Quality²¹ urge for the development of strategies focused on promoting patient engagement in patient safety—a largely unexplored area until recently.^{22 23} A number of different approaches have been proposed to engage patients in their own safety.²⁴ One of them is based on gathering patient-reported safety experiences and outcomes, and feeding the data back to healthcare providers.²⁵ This approach has been tested in the hospital setting with mixed results,^26–28 but no previous studies in the PHC setting are available.²⁹ This is mainly due to the absence of valid and reliable tools to obtain patient safety feedback in PHC.³⁰ To address this gap, we developed and validated the ‘Patient Reported Experiences and Outcomes of Safety in Primary Care’ (PREOS-PC) questionnaire.^{31 32}

In this protocol paper we describe a study that aims at developing and evaluating an intervention to improve patient safety in PHC centres by providing them with patient feedback obtained through the administration of the PREOS-PC questionnaire.

Methods and analysis

Description of the intervention

The intervention will consist in gathering patient-reported experiences and outcomes concerning the safety of the healthcare they have received in their PHC centres during the previous 12 months. This information will be processed and fed back to their PHC professionals to help them identify potential problems, and then target improvements about problematic areas. The three key stages of the intervention are as follows:

• Measurement: patients will be approached in the waiting room, the study explained, and informed consent taken. The PREOS-PC questionnaire will be self-completed using a tablet computer. Patients will be given a choice of whether they would prefer to self-complete the questionnaire or have it facilitated by the researcher.

• Feedback: using a bespoke online tool, the information for each PHC centre will be collated and presented to the centres. They will receive an automatically generated ‘Feedback Report’, which will offer comparisons with other centres and include a set of recommendations about how the safety issues identified could be addressed.

• Action planning and change: participating PHC centres will form an action planning team. Each team will comprise around four people working in the centres. The team will be responsible for receiving the feedback report, considering which area(s) should be targeted, and developing, implementing and monitoring an action plan for safety improvement.

This intervention is based on the clinical performance feedback intervention theory (FIT), which states that behaviour is regulated through comparison with standards or goals, and that feedback can draw attention to existing gaps.³³ FIT further postulates that once the gap has been identified, different methods can be followed in order to decrease it and attain the standard, including increasing the effort currently done,³³ and implementing new strategies to address the problems (figure 1). This could result in improving proximal outcomes (such as safety climate), and potentially impact more distal outcomes (eg, safety events or avoidable hospitalisations).

Figure 1.

Logic model of the proposed intervention. *Intervention logic model based on feedback intervention theory and the capability, opportunity and motivation-behaviour system. PREOS-PC, patient reported experiences and outcomes of safety in primary care.

Development and evaluation of the intervention

The methods described below are based on the Medical Research Council guidance for the development and evaluation of complex interventions.³⁴ This study includes three stages (see in figure 2).

Figure 2.

Stages for the development and evaluation of the proposed intervention. PHC, primary healthcare; PREOS-PC, patient reported experiences and outcomes of safety in primary care.

Stage 1: intervention development

This stage involves the following:

Qualitative study with PHC providers: we will conduct three semi-structured interviews and four focus groups with PHC doctors, nurses and admin staff (n=40) to examine the acceptability and perceived utility of the intervention, and to identify potential barriers towards wider implementation.

Translation and cross-cultural adaptation of the PREOS-PC questionnaire ³¹ into the Spanish context: the translation process, based on ‘state of the art’ methods,³⁵ will consist in forward and back translation by four independent translators, followed by cognitive interviews with eight to ten participants (diverse in terms of age, sex and educational attainment) using the ‘think aloud’ method³⁶ to ensure the translated version of the questionnaire is easy to understand and complete. The cross-cultural adaptation will be carried out using an expert consultation process involving about five national experts in patient safety. The original version of the PREOS-PC questionnaire that will be adapted and translated into the Spanish context is available in online supplementary appendix 1.

Development of the intervention components: we will design the feedback report based on evidence from previous studies^{26 37 38} and from the qualitative study with PHC providers described earlier. The feedback report will show the results of the Spanish PREOS-PC questionnaire specific for each PHC centre. It will provide benchmarking data—that is, practices will be able to see their individual scores compared with the average scores of the rest of participating. To facilitate the design of action plans to address the potential safety issues identified in the feedback report, we will also produce a guidance document with recommendations, good practices and materials to improve patient safety in PHC, which will be identified as a result of a literature review, including the WHO,³⁹ the European Union Network for Patient Safety and Quality of Care (PaSQ Joint Action),¹⁶ the Agency for Healthcare Research and Quality⁴⁰ and the LINNEAUS EuroPC collaboration⁴¹ among others. We will also produce a registry form to help PHC centres register and monitor progress of the planned actions to address the safety problems identified. The intervention materials will also include information to increase PHC providers’ awareness of the usefulness of patient elicited information as a strategy to identify potential safety problems and design strategies to address them.

Development of an online tool: we will develop a bespoke online tool to allow the electronic administration of the PREOS-PC to patients using tablet computers. The data collected will be transferred to a database stored in a virtual server. Once all patient data have been collected from in each PHC centre, the tool will automatically generate and send the feedback reports to each centre. The tool will also be used to collect data from the healthcare professionals participating in the trial, which will be stored in a separate database with a protected authentication password to access to the provider questionnaire and to access to the feedback report.

Stage 2: piloting and refining the intervention

We will pilot the intervention in a 3-month, one-arm (pre-post) feasibility trial. This will allow us to estimate the follow-up rate for the main trial; test the collection of the planned outcome data; the willingness of PHC centres, providers and patients to participate; and the trial procedures. It will also allow us to examine the psychometric properties of the Spanish PREOS-PC, and introduce final changes in the instrument if needed. Participants will include PHC centres, providers and patients, with the following eligibility criteria: (i) centres: PHC centres from the Balearic Islands Health Service; (ii) providers: all healthcare professionals working in the centre, including administrative staff; (iii) patients: we will invite patients who have visited their PHC centre at least once in the previous 12 months. They will have to be able to speak Spanish. Patients aged <18 will be included only if their parents or guardians agree to complete the questionnaire on their behalf. We will exclude overt psychosis/critically ill/altered mental status, and inability to provide written informed consent.

Sample size: assuming an average of 26 healthcare professionals per centre,¹² recruiting ten centres will result in approximately 260 professionals taking part in the feasibility trial. A sample of 260 professionals would allow to detect a 80% follow-up rate within 95% CI of 75.1%–84.8%. With 500 patients (50 per centre), the study is powered to detect a patient response rate to the questionnaire of 75% within 95% CI of 71.2%–78.8%. 500 participants are sufficient to perform factor analyses and the rest of analyses planned for the evaluation of the psychometric properties of the Spanish PREOS-PC.

Recruitment: we will recruit 10 PHC centres from the Balearic Islands diverse in terms of list size, deprivation and rurality. 500 patients will be approached and recruited in the waiting room by a research assistant and invited to complete the Spanish PREOS-PC.

Outcome measures will include: (i) healthcare professionals’ follow-up rate, which will be measured as the proportion of PHC professionals who successfully complete the validated Spanish version of the Medical Office Survey on Patient Safety Culture (MOSPSC)^{42 43} at baseline and post-intervention, and (ii) patient response rate to the PREOS-PC.

Statistical analysis: we will calculate the proportion of healthcare providers that complete the Spanish MOSPSC at baseline and at 3-month post-intervention. We will also calculate the follow-up rate by type of healthcare provider (nurse, doctor, social worker, administrative, etc.). Response rate to the PREOS-PC will also be calculated (overall and by centre and patient characteristics). The evaluation of the psychometric properties of the Spanish PREOS-PC will involve the examination of floor and ceiling effects, internal consistency (inter-item correlations,⁴⁴ Cronbach’s α⁴⁵) and construct validity (confirmatory factor analysis). We will also examine potential differences in mean scores between patients who have and have not received help completing the questionnaire.

Embedded qualitative study: after the feasibility study we will conduct semi-structured qualitative interviews with 20 healthcare professionals. They will be purposefully selected to ensure variation in terms of professional roles. They will be conducted by a researcher either face to face or telephonically, and will be audio-recorded after informed consent. The audio recordings will be transcribed and imported to the qualitative analysis software NVivo11. Thematic analysis⁴⁶ will be used to explore the acceptability and perceived utility of the intervention, as well as possible suggestions to improve the intervention delivery or content. After an in-depth reading of the transcriptions, codes will be assigned to sentences or paragraphs that had the same meanings, and then, by grouping codes, we will create and refine categories in an iterative process. The analysis will be conducted by two researchers independently. A third researcher will be involved to solve potential discrepancies.

Results from the feasibility trial will be used to inform the potential refinements about the intervention as well as the trials procedures (eg, methods for data collection), with an explicit process to decide the final intervention content, including a systematic appraisal of the trial processes (both quantitative and qualitative data) and proposals for solutions to identified problems.

Stage 3: evaluating the acceptability, perceived utility and effectiveness of the intervention

The evaluation of the intervention will involve a 12-month, two-arm, two-level cluster randomised controlled trial (1248 PHC professionals within 48 PHC centres; with randomisation at the centre level in a 1:1 ratio). The trial timeline and CONSORT flow chart are available in figures 3 and 4. A cluster randomised trial is proposed to avoid the risk of contamination across professionals working in the same centre. 24 PHC centres in the intervention group will receive the intervention described earlier. 24 centres in the control group will receive the feedback reports at the end of the study.

Figure 3.

Trial timeline. *PREOS-PC, Patient Reported Experiences and Outcomes of Safety in Primary Care. **MOSPSC, Medical Office Survey on Patient Safety Culture. Provider reported patient safety culture.

Figure 4.

Consort flow chart. ITT, intention to treat; PHC, primary healthcare.

Randomisation will be done using a fully validated randomisation algorithm. Allocation will be carried out using a non-deterministic minimisation algorithm to ensure PHC centres are balanced for important characteristics (including region, deprivation and list size) and baseline measures. Participants: staff working and patients registered in the PHC centres. Eligibility criteria will be the same than in the feasibility trial described earlier.

The main outcome will be the patient safety culture, measured with the Spanish MOSPSC⁴³ at the PHC professional level. The MOSPSC is a recognised instrument in Spanish PHC and it is supported by the Ministry of Health and the main PHC society (http://www.mscbs.gob.es/organizacion/sns/planCalidadSNS/docs/MOSPS.pdf). The full questionnaire is available in online supplementary appendix 2. This validated instrument includes 67 items grouped in 13 dimensions. Patient safety culture will be computed either as a global score (synthetic index calculated at the healthcare professional level based on the mean score of the 67 items in the questionnaire⁴²) or at the individual domain level. This decision will be made based on the results from the feasibility study in terms of the performance (in terms of internal consistency and sensitivity to change) of both measurement methods.

Secondary outcomes will be evaluated at the PHC centre level, and will include (i) the five scales in the PREOS-PC questionnaire (measuring PHC centre activation; patient activation; experiences of safety problems; harm; and overall rating of patient safety), and; (ii) rate of avoidable hospitalisations, based on data extracted from electronic medical records using available codes from the International Classification of Diseases(ICD-9),^{47 48} calculated as the number of avoidable hospitalisations per 1000 patients in the last 12 months.

The sample size calculation is based on the trial’s main outcome measure—the Spanish MOSPSC, which produces a score ranging from 1 to 5. Assuming an average of 26 professionals per centre, approximately 1248 professionals, will take part in the study. Assuming a follow-up rate of 80%, we will have complete data from approximately 998 professionals. Taking into account the cluster design, and using a conservative estimation of intra-class correlation of 0.1, this sample size will allow us to detect at least a 0.3 difference in effect size (with 80% power and a significance level of 5%). This would approximately correspond to a difference of 0.8 points in the index (assuming SD of 2.3 from a previous study).⁴⁹ We will recruit 75 patients per centre (3600 in total), which is the minimum number to achieve a 0.7 reliability of scale scores at the centre level.³¹

Recruitment and training of PHC centres: we will recruit 48 PHC centres from Balearic Islands and other regions in Spain, through scientific societies and key informants and purposefully selected in order to ensure variation in terms of list size, rurality and levels of deprivation. Centres will be asked to consent as a unit, with all professionals being willing to participate. Consent will also be taken from patients invited to complete the patient survey. The intervention will be standardised across all sites and regions.

Data collection: data will be collected at baseline and 12 months post-intervention (ie, 12 months after the feedback reports are sent to the centres). We will monitor the progress of the intervention in all the centres. Data from patients will include patient reported experiences and outcomes of patient safety in PHC (measured with PREOS-PC) and patient sociodemographic characteristics. Data from healthcare professionals will be collected through online questionnaires and will include the perceived safety climate (with the Spanish MOSPSC), and sociodemographic and occupational characteristics. Data from centres will include rate of avoidable hospitalisations in the previous 12 months (extracted from electronic medical records), and centre characteristics (rurality, list size, number of healthcare professionals and "Mortalidad en áreas pequeñas Españolas y Desigualdades Socioeconómicas y Ambientales"- MEDEA deprivation index).⁵⁰

Statistical analyses: baseline characteristics will be examined by group using frequencies (with percentages) for binary and categorical variables and means (and SD) or medians (IQR) for continuous variables. The results from the trial will be presented as comparative summary statistics (difference in proportions or means) with 95% CI. The primary outcome will be analysed using a hierarchical model, with individuals (PHC professionals) nested within PHC centres in an analysis of covariance adjusted for minimisation factors. All analyses will be carried out on the basis of intention to treat.

Strategies to monitor and improve PHC adherence to intervention protocols: through our online tool we will monitor the competition of the providers’ questionnaire at baseline and 12-month follow-up, and also whether or not they record action plans for safer healthcare. Up to three email reminders will be send to healthcare professionals if they don’t complete the requested tasks as part of the intervention. In addition, during the trial all PHC centres will be contacted telephonically to ensure they have received the feedback report and have no problems accessing and understanding the information.

Qualitative study with PHC providers: after post-intervention follow-up we will carry out an qualitative study with 30 PHC professionals (intervention group) to understand the way the intervention is perceived among PHC professionals in terms of acceptability, perceived utility and implementation barriers (including any unintended consequences). We will use purposeful sampling to ensure variation in terms of type of PHC professionals (doctors, nurses, administrative staff, etc.) and of centres (region, rurality, deprivation, list size). Interviews will take place in the centres or telephonically. Thematic analysis⁴⁶ will be used to identify recurrent themes and subthemes common to interviewees working in centres.

Patient and public involvement

In this study, the intervention design will be determined based on group discussions with PHC providers. Four group discussions with researchers took place in May–June 2018 to review and comment on the intervention design and materials based on their priorities, experiences and preferences. A meeting with four patient representatives was also held in September 2018, where the study was presented and discussed with them, providing useful feedback that helped us to refine the methods for administering the patient reported questionnaire.

Trial status

The cluster randomised control trial will start around July 2019 and will continue until July 2020.

Discussion

The prevention and amelioration of avoidable harm is a major priority for most PHC systems. Patients are the common element across the various settings, organisations and health professionals usually involved in their healthcare, and therefore, they are ideally suited to reflect on the healthcare they receive.⁵¹ As recently highlighted both by WHO²⁵ and the OECD,¹⁰ tapping into such a rich resource could contribute significantly to improving safety in PHC.

Some, but not all, of the studies evaluating the use of patient feedback interventions in the hospital setting support the effectiveness of this type of intervention to achieve safer healthcare. For example, a study in a hospital in England observed that obtaining feedback from patients and promoting staff ownership of safety improvement processes helped to raise standards of care.²⁷ In Japan and Denmark, patient feedback contributed to increase awareness among professionals and develop new safety protocols about minimising risk.²⁸ However, a recent study in 33 UK hospital wards found that patient feedback did not reduce harm and patient reported safety problems.²⁶ The authors attributed this lack of effect to poor staff adherence to the intervention, due to a lack of normative legitimacy (ie, staff not believing that listening to patients was a worthwhile exercise) and of structural legitimacy (ie, staff not having adequate autonomy, ownership and resource to enact change).⁵² Learning from these experiences is key to achieve progress in this area. In order to address these potential barriers in our study, our training materials will aim to raise awareness about the importance of patient reported information, as a way of increasing normative legitimacy. We will also provide practices with specific recommendations and educational material to help them design and implement actions for safer care—with the ultimate objective of increasing structural legitimacy.

The methods for the development and evaluation of the intervention are in line with the Medical Research Council guidance for the development and evaluation of complex interventions,³⁴ including (i) identification of the relevant evidence base, (ii) formative work (primary qualitative research) and use of theory to develop a theoretical understanding of the likely process of change; (iii) a feasibility study to test trial procedures, estimate recruitment and retention, and determine sample size and (iv) a full-scale randomised controlled trial to assess the effectiveness of the intervention, with an embedded quality study to help understand the mechanisms and contexts by which this intervention does (or does not) work, and identify potential barriers to implementation and wider roll-out. Findings from this study will provide useful information to confirm or revise the theoretical frameworks in which the proposed intervention is grounded.

Strengths and limitations

Our study has a number of strengths. First, the intervention has been designed to minimise costs and maximise scalability and sustainability. Since it would be delivered with a bespoke online tool to collect patients’ feedback and automatically generate and send tailored feedback reports to PHC centres, it could be rolled out in Spanish centres with minimal external input and at a low cost. We have taken into account in the design of the intervention both patients’ and providers’ views about the intervention in order to maximise its acceptability. The PREOS-PC questionnaire is a patient-centred instrument which was developed with strong patient input, including patient focus groups⁵³ and a meta-synthesis⁵⁴ of patient experiences of patient safety in PHC. However, this study also has some limitations. First, it is not possible to blind centres to the condition they have been allocated to (intervention or control). Also, there is a risk that a high proportion of missing outcome data could compromise the validity of our findings in case we experience low response rates by PHC professionals in the MOSPSC questionnaire.

In conclusion, the proposed intervention based on the provision of patient feedback to PHC centres has the potential to be an acceptable, cost-effective, feasible and sustainable strategy to achieve safer healthcare provision in PHC centres. A large pragmatic cluster randomised trial in 48 PHC centres will provide solid evidence about its potential effectiveness in improving patient safety culture, patient reported safety experiences and outcomes, and avoidable hospitalisations.

Ethics and dissemination

Ethical approval

All participants will sign an informed consent before participating in this study. All the information from patients and PHC professionals will be anonymised. Patients and providers will be able to withdraw from the study at any time and without having to provide any reason for withdrawing. Any important protocol modifications will be submitted to the ethics committee for approval.

Dissemination

The main findings of this study will be disseminated via publications in peer-reviewed international journals. Presentations of study findings will also be offered at relevant research conferences, and national and international academic symposia and seminars.