Evaluation of outcome relevance of quality indicators in the emergency department (ENQuIRE): study protocol for a prospective multicentre cohort study

Abstract

Introduction

Quality of emergency department (ED) care affects patient outcomes substantially. Quality indicators (QIs) for ED care are a major challenge due to the heterogeneity of patient populations, health care structures and processes in Germany. Although a number of quality measures are already in use, there is a paucity of data on the importance of these QIs on medium-term and long-term outcomes. The evaluation of outcome relevance of quality indicators in the emergency department study (ENQuIRE) aims to identify and investigate the relevance of QIs in the ED on patient outcomes in a 12-month follow-up.

Methods and analysis

The study is a prospective non-interventional multicentre cohort study conducted in 15 EDs throughout Germany. Included are all patients in 2019, who were ≥18 years of age, insured at the Techniker Krankenkasse (statutory health insurance (SHI)) and gave their written informed consent to the study.

The primary objective of the study is to assess the effect of selected quality measures on patient outcome. The data collected for this purpose comprise medical records from the ED treatment, discharge (claims) data from hospitalised patients, a patient questionnaire to be answered 6–8 weeks after emergency admission, and outcome measures in a 12-month follow-up obtained as claims data from the SHI.

Descriptive and analytical statistics will be applied to provide summaries about the characteristics of QIs and associations between quality measures and patient outcomes.

Ethics and dissemination

Approval of the leading ethics committee at the Medical Faculty of the University of Magdeburg (reference number 163/18 from 19 November 2018) has been obtained and adapted by responsible local ethics committees.

The findings of this work will be disseminated by publication of peer-reviewed manuscripts and presentations as conference contributions (abstracts, poster or oral presentations).

Moreover, results will be discussed with clinical experts and medical associations before being proposed for implementation into the quality management of EDs.

Trial registration number

German Clinical Trials Registry (DRKS00015203); Pre-results.

Strengths and limitations of this study.

• The strength of the present study is the linkage of treatment and outcome data on an individual level to evaluate the association of quality indicators in the emergency department (ED) with medium-term and long-term outcomes.

• Another strength is a sophisticated data protection policy that allows data linkage from different sources at an individual level in conjunction with the protection of the personal privacy of individuals who participate.

• Strength and limitation in one: the study is conducted in the real-life setting of the ED.

• Limitations result from the quality of routine data in terms of completeness and accuracy.

• Limitations of the study result from the necessity of a written consent, which results in a limited access to some patients and from the inclusion of only the insured persons of one statutory health insurance (Techniker Krankenkasse) that may not represent the total cohort of all ED patients.

Background

Emergency departments (EDs) play a crucial role within the healthcare system as gateway into clinical care for unscheduled arrivals of patients with acute, potentially life-threatening conditions. The patient cohort in the ED is characterised by a great heterogeneity with regards to presenting complaints,^{1 2} severity of illness or injury, acuity and others. Additionally, the complexity of emergency admissions increases with the ageing population and the increase in the number of patients suffering from frailty and multiple comorbidities.³

There is strong evidence that assessment of quality in the ED can improve quality of care and patient outcomes. This makes implementation of quality improvement programmes necessary to deliver high quality of care to patients.⁴

Over the past two decades, several systematic literature reviews have been published summarising relevant quality indicators (QIs) proposed for implementation in the EDs. Well-described and validated QIs are available for patients with trauma,^{5 6} and musculoskeletal injuries⁷ as well as geriatric patients.⁸

In 2013, Sørup and colleagues summarised relevant quality measures to assess overall ED performance. They divided them into three groups related to patients (eg, left without being seen), staffing (eg, occupation profile) and operational performance (eg, timelines).⁹

Regarding large patient numbers in EDs, routinely recorded data stored in information systems are an optimal data source for QI calculation. The use of an ED information system (EDIS) to collect structured medical information is a necessary prerequisite.¹⁰ A national documentation standard for EDs according to the German Emergency Department Medical Record (GEDMR) was established in 2010 by the German Interdisciplinary Association for Intensive Care and Emergency Medicine.¹¹ In a systematic literature search, Hörster et al identified 25 QIs that were mentioned at least two times in independent sources. A total of 10 QIs were identified that can be derived from this GEDMR data set.¹² These findings are in agreement with the findings from Kulla et al¹³ who rated and evaluated these QIs for their scientific relevance and practicability in German EDs using a modified QUALIFY approach.¹⁴

While there are a variety of data on QIs in EDs, systematic studies and reliable information on the effect of these QIs on medium-term and long-term outcomes are lacking. For patients with trauma, who represent a small group of ED patients, Boyd and coworkers found no association between intensity and nature of QIs in trauma centres with patient outcomes.¹⁵ For severely injured patients, a set of 40 QIs has recently been rated by experts and validated in a national registry using mortality as outcome.¹⁶ Conflicting results have been reported for other established QIs in emergency care. In some countries, length of stay in EDs (ED-LOS) was implemented as a main QI and policy measures were taken to achieve time targets under 4 or 6 hours, respectively.^17–19 ED-LOS is a well-characterised QI with regards to patients outcome. However, results are conflicting. Findings from Jones et al suggest that with introduction of the 6-hour time target policy, most investigated outcomes, such as mortality and representation to the ED, remained unchanged; while others improved, such as hospital LOS.²⁰ In contrast, Rose et al found that a prolonged ED stay may worsen outcomes for the subgroup of patients admitted to intensive care units.²¹ Assessing quality in the ED by using ‘LOS’ as a single key indicator remains controversial. Improvement of the ED-LOS did not result in an improvement of other ED QIs over the same time period.²² Suriyawongpaisal and coworkers conclude that a set of indicators and different outcome measures should be considered instead.²³

Outcome measures in the ED such as mortality, or unplanned re-admission within 24 hours, 72 hours or 7 days for the same or similar complaint already have been analysed in detail,9 20 24–26 however, they are limited to events within the same institution. It is difficult to obtain outcome measures in a cross-sectoral setting and in a medium-term and long-term follow-up for analysis.

This manuscript describes the study design for the evaluation of outcome relevance of QIs for the EDs (ENQuIRE) in Germany using medical data from EDs, claims data from one statutory health insurance (SHI) fund and survey data. A set of QIs for application on routine ED documentation with a focus on long-term outcomes will be evaluated, established and possibly implemented into the ED quality management.

Methods/design

The study aims to evaluate QIs in the ED in terms of their relevance for the treatment outcomes including patient-reported outcomes. This is enabled by linking clinical data from the electronic documentation in the ED with quantitative and consistent outcome results in a 12-month follow-up using claims data of the SHI at an individual level.

A scientific advisory board consisting of representatives of medical associations (German Interdisciplinary Association for Intensive Care and Emergency Medicine and the German Society for Interdisciplinary Emergency and Acute Medicine), clinical experts from the field of emergency medicine and a patient representative (German Coalition for Patient Safety, Aktionsbündnis Patientensicherheit e.V., Berlin) has been established to provide scientific advice on the project, to represent the interests of patients and support the implementation of the results.

Setting

The ENQuIRE study is a prospective non-interventional multicentre cohort study. Fifteen EDs throughout Germany including university hospitals as well as municipal hospitals, clinics of different care levels and located in cities with different population numbers are participating in this study (table 1). The study includes all eligible patients throughout 2019.

Table 1.

Characteristics of participating emergency departments

Patients per year	Emergency departments (n=15)
<20 000	2
20 000–50 000	10
>50 000	3
Population/city of
<100 000	3
>100 000 <200 000	9
>500 000	3
Level of emergency care
basic	1
specialised	6
maximum	8
Academic status
University hospital	5
Academic teaching hospital	8
Non teaching	2

The Techniker Krankenkasse (TK) is Germany’s largest health insurance company with approximately 10.6 million persons insured. According to Hoffmann and Icks²⁷, persons insured at the TK are more often men, have a higher education level and are in a better state of health compared with the total of all insured persons in Germany.

The study protocol was reviewed and approved by the leading ethics committee at the Medical Faculty of the University of Magdeburg (163/18 from 19 November 2018) and by relevant institutional ethics committees: Carl von Ossietzky University Oldenburg (2018-142), University Hospital Jena (2018-1259-Daten), University Medical Center Göttingen (26/12/18Ü) and by local ethics committees of the State Medical Associations: Sächsische Landesärztekammer, Dresden (EK-BR-2/19-1), Ärztekammer Schleswig Holstein, Bad Segeberg (001/19 m), Ärztekammer Bremen (654) and Landesärztekammer Rheinland-Pfalz, Mainz (2019-14014).

Data privacy is in agreement with the General Data Protection Regulation. A data protection policy has been developed involving a trusted third party (TTP). This TTP maintains the patient lists, mails patient questionnaires and queries ED treatment data as well as claims data from the SHI. For data linkage, the pseudonymisation is carried out in two stages. First, a subject identification code is generated within the TTP. Second, a data delivery web service is used to generate random 64-digit, second-level alphanumeric pseudonyms. Pseudonymised data are then routed to the project’s evaluation unit for analysis.

An application had been submitted to the German Federal Insurance Office (in German Bundesversicherungsamt) for permission to use the claims data of the SHI while guaranteeing all data protection aspects according to §75 Sozialgesetzbuch, that is, Social Security Code (SGB) X.

Inclusion and exclusion criteria

All patients attending a participating ED in 2019 were screened for eligibility. The inclusion criteria were defined as follows: 18 years of age or older and insured at the TK. Exclusion criteria were insufficient German language skills and no residence in Germany. All eligible patients should be included, independently of presenting complaints, severity of illness or injury, mode of arrival, acuity, hospital admission or outpatient treatment, weekday and daytime.

If possible, patients were asked for written consent in the ED by a study nurse or by trained nursing staff. Additionally, admitted patients could be contacted during their inpatient stay if informed consent was not possible in the ED. Non-admitted patients who did not have the opportunity to provide consent during their stay in the ED were contacted by post.

The target number of participants to be recruited was calculated according to the mean number of ED attendees in the participating EDs in the last 2 years and an average percentage of TK insured patients of about 12.9%. After reduction by an expected amount of about 10% of patients aged <18 years, the number of patients to be recruited was estimated at about 25 900. A subgroup of 5000 patients who have given consent to be contacted again received a questionnaire 6–8 weeks after emergency admission to evaluate health-related quality of life (HrQoL). The self-reported questionnaire includes items adapted from the short form 12 version 2 (SF12v2) survey of the German Socio-Economic Panel²⁸ as well as socioeconomic items and questions on patient satisfaction. To test the questionnaire, a pretest had been conducted in two study centres for evaluation in terms of readability and ease of understanding.

Selection of QIs

For the selection of QI, a set of QIs was collected based on systematic literature search in MEDLINE (PubMed) and screening of publications of health organisations (National Health Service, Agency for Healthcare Research and Quality, National Quality Forum, Canadian Institute for Health Information, German Institute for Quality assurance and Transparency in Healthcare). Extracted QIs were discussed with clinical and IT experts with regards to practicability, operational feasibility for routine collection and data validity. A core set of selected QIs including mainly patient-centred parameters and process times as well as selected structural data are shown in table 2.

Table 2.

Core set of quality indicators for analysis

Quality indicator	Reference
Time from arrival to CT	12
Length of stay (LOS) of admitted patients	12 41–49
LOS of non-admitted patients	12
Left before/without being seen	12 46–48
Time from arrival to initial triage	12 46
Brain imaging in stroke suspicious patients	50
Time from arrival to pain management	7 47
Emergency department staffing: nurses (full-time equivalent) per patients	41 46 48
Left before treatment completion	41 46 48 49
Time from arrival to provider	41 42 46 48
Left against medical advice	48
Time from arrival to first ECG in suspected cardiac chest pain or acute myocardial infarction	49 51
Time from arrival to brain CT for patients presenting within 4 hours of onset of symptoms consistent with a stroke	42 49
ECG within 10 min of arrival for patients presenting with chest pain	51
ECG for patients with non-traumatic chest pain	51
Time from arrival to intravenous tissue plasminogen activator within 4.5 hours of symptom onset in patients with acute ischaemic stroke	42 52
ECG performed for syncope	53
Time from arrival to chest radiography for admitted patients	54
Time from arrival to chest radiography for non-admitted patients	54
Determination of the respiratory rate at admission for patients with outpatient-acquired pneumonia	55
Time from arrival to reperfusion for patients with acute myocardial infarction	56

Outcome

The primary aims of the study are:

• Identification of strengths and weaknesses of the QI.

• Checking operationalisability and feasibility of QIs in real life.

• Evaluation of the outcome relevance of QIs in the ED by determination of the influence of QI performance on mortality, morbidity, incapacity to work and the use of healthcare facilities after discharge from the ED.

The long-term goal of the investigation is a set of evaluated QIs ready for implementation into quality assessment programmes in EDs.

Data linkage

After completion of the recruitment phase, clinical records will be extracted from the routine documentation (clinical dataset is shown in table 3) using, for example, the infrastructure of the German Emergency Department Data Registry (GEDD-registry, AKTIN) and linked with outcome variables from SHI collected in a follow-up over 12 months. This includes variables such as mortality, morbidity, incapacity to work, levels of care, prescriptions of drugs, therapeutic remedies and aids as well as repeated outpatient and inpatient treatments (table 4). For adjustment of patients’ health status before ED admission, the claims data of the SHI over 12 months before the ED contact will be used (figure 1).

Table 3.

Patient and treatment data derived from the basic module of the German Emergency Department Medical Record

Patient data

Age (years)

Gender

Referral

Transport

Anamnesis

Presenting complaints (CEDIS)

Duration of symptoms

Triage system (MTS or ESI)

Triage level

Vital signs

Respiratory frequency

Oxygen saturation

Systolic blood pressure

Heart rate

Core temperature

VAS pain (0–10)

Glasgow coma scale

Rankin

Pupillary reflex

Diagnosis

Leading diagnosis

Number of diagnoses

Discharge (outpatient, inpatient)

Examinations

ECG

Sonography

Echocardiography

CT/cCT

Traumascan

X-ray (spine, thorax, pelvis, limbs)

MRI

Laboratory (BGA, urine)

Time stamps

Admission

Triage

Start of therapy

First contact with a physician

ECG

Sonography

Echocardiography

CT/cCT

Traumascan

X-ray (spine, thorax, pelvis, limbs)

MRI

Laboratory (BGA, urine)

Discharge from ED

BGA, blood gas analysis; (c)CT, (cranial) computer tomography; CEDIS, Canadian emergency department information system; ECG, electrocardiogram; ED, emergency department; ESI, Emergency Severity Index; MRI, magnetic resonance imaging; MTS, Manchester Triage System; VAS, Visual Analogue Scale.

Table 4.

Contents of SHI claims data for deriving patients’ outcome

Patient data

Year of birth

Postcode

Type of employment

Claim to sick pay

Duration of occupational disability

Nursing care level

Duration of need for nursing care

Termination (reason and date)

Periods of insurance (duration pre and post)

Cases of incapacity for work

Diagnosis (ICD)

Date of beginning and end of incapacity

Physician’s specialty (code)

Quarter of diagnosis identification

Therapeutic products and aids

Physician’s specialty (code)

Type and amount of therapeutical sessions/aid

Date of prescription

Service provider (code and date)

Outpatient treatment

Service provider (code)

Date of beginning and end of treatment

Physician’s specialty (code)

Diagnosis (ICD)

Services provided (code and date)

Procedures (OPS-code and localisation)

Inpatient treatment

Department’s specialty (code)

Admission (reason and date)

Discharge (reason and date)

Diagnosis (ICD)

Procedure (OPS-code, date, localisation)

Outpatient pharmaceutical supply

Physician’s specialty (code)

Date of prescription and date of prescription filled

Amount of prescripted unit dose (daily defined dose)

Anatomical therapeutic chemical code

ICD, International Classification of diseases and related health problems; OPS, German classification of operations and procedures; SHI, statutory health insurance.

Figure 1.

Flowchart of the study design. ED, emergency department; EDIS, ED information system; SHI, statutory health insurance; TK, Techniker Krankenkasse.

Analyses

Descriptive and analytical statistics will be applied to provide an overview about the characteristics of the QI and the associations between quality measures and patient outcome. Quality measures will be analysed as either dichotomous or graded variables and associated with appropriate outcome measures. Analyses will be performed for the entire cohort and for subgroups of selected presenting complaints (Canadian Emergency Department Information System (CEDIS) presenting complaint list).^{29 30}

In order to assess representativeness of the cohort of patients, who have given consent (a subgroup of the TK insured), the distribution of variables (demographic data, eg, age, gender and ED data, eg, mode of arrival, presenting complaints, triage, admission), will be compared with anonymised corresponding data from the total cohort of all ED patients of participating hospitals in the observation period.

Furthermore, the validity of retrospective reporting of HrQoL when arriving in the ED will be investigated in a subgroup of study participants who receive a questionnaire to report their HrQoL during the initial ED visit and 6 weeks after. The retrospective recording of HrQoL in the setting of EDs is neither validated nor has the extent and direction of a possible systematic bias due to retrospective survey or the current emergency situation been investigated sufficiently.^31–33

Patient and public involvement

There was no patient or public involvement in the design of the study.

Discussion

Care structures and processes in EDs as well as the healthcare service provided can directly influence the outcome of patients in terms of quality of health and HrQoL. The ultimate goal for quality improvement is a better treatment outcome. Quality management is required from the healthcare providers by Federal law (§136 SGB V). The identification of outcome-related QIs is of considerable interest for the patient-relevant quality management in EDs. These QIs may support improvements in patient-relevant processes in the ED. The ENQuIRE study will help to establish a set of meaningful QIs evaluated for their relation to patients’ outcome. For the first time, ENQuIRE will link clinical routine data with SHI data to analyse the influence of patient-centred parameters, process times and treatment procedures on patient medium-term and long-term outcomes. Regarding patient morbidity and utilisation of health services, routine data of the SHI system are an important source of information. These data are usually characterised by high external validity.³⁴ However, since these data are secondary data, they must be critically examined for use in scientific questions.³⁵ While the linkage of patient data from different sources as applied in our study is a relatively new approach in medical research,³⁶ several scientific standards for using claims data in research actually have been established.³⁷

The use of quality measures should ideally result in an improvement of outcome measures without increasing the risk for unintended dysfunctional consequences,³⁸ misplaced incentives or side effects.³⁹ Focusing on a single QI without considering other factors that affect quality of care can lead to poor clinical outcomes. These findings support the approach of this study to define not a single but a set of outcome-relevant QIs.

Results of this study can be implemented into the current activities for the planned reorganisation of emergency care in Germany.⁴⁰ Considering the patient-related quality of care, optimised organisational models can be developed from healthcare providers. The results of the ENQuIRE project will constitute the basis for a future standardised and comprehensive quality management in EDs.

Preliminary results will be submitted for publication in 2021. The discussion of results will be performed with clinical experts and medical associations before being proposed for implementation into the quality management of the ED. Thus, the results of ENQuIRE could be directly implemented into the quality management and could be relevant to health policy decisions.

Ethics and dissemination

This study was approved by the leading ethics committee at the Medical Faculty of the University of Magdeburg (163/18 from 19 November 2018) and by relevant institutional ethics committees: Carl von Ossietzky University Oldenburg (2018-142), University Hospital Jena (2018-1259-Daten), University Medical Center Göttingen (26/12/18Ü) and by local ethics committees of the State Medical Associations: Sächsische Landesärztekammer, Dresden (EK-BR-2/19-1), Ärztekammer Schleswig Holstein, Bad Segeberg (001/19 m), Ärztekammer Bremen (654) and Landesärztekammer Rheinland-Pfalz, Mainz (2019-14014). Written informed consent was obtained from all participants who were informed about the purpose and risks of the study. Findings of the study will be made available as oral or poster presentations at scientific conferences and through peer-reviewed journals.