Patient-Reported Outcome Measurements (PROMs) After Discharge From the Emergency Department: A Cross-Sectional Study

Abstract

Background:

The purpose of a patient-reported outcome (PRO) is to elicit the perspectives of patients and translate them into a reliable measurement questionnaire.

Objectives:

The objective of this cross-sectional study was to detect a set of PROs and PRO measurements (PROMs) about patients with isolated trauma of the limbs receiving emergency department (ED) care.

Methods:

A survey was performed in the ED using a questionnaire among the enrolled patients to identify which proposed outcomes were perceived as important by the patients according to their expectations.

Results:

Ninety-six consecutive patients were conveniently enrolled. For each item of the questionnaire, the percentage of patients who agreed to perceive it important were calculated. Three items were perceived important by almost 85% of the patients: getting an x-ray (91%; 95% CI: 88%-98%), obtaining a written therapy (94%; 95% CI: 87%-97%), and feeling the physicians’ and nurses’ empathy (97%; 95% CI: 91%-99%). The ED system was able to satisfy 2 of the 3 agreed PROs in at least 85% of the cases: getting an x-ray (97%; 95% CI: 91%-99%) and obtaining a written therapy (97%; 95% CI: 91%-99%). Moreover, in 30/96 patients (31%; 95% CI: 22%-41%), all the PROs were satisfied, and in 75/96 patients (78%; 95% CI: 69%-85%), all agreed PROs were satisfied.

Conclusions:

Our study shows an example of core of PROs proposed by the ED physicians and agreed by the patients. Moreover, we presented a set of PROMs which could be used to measure the quality of an ED.

Introduction

Patient-reported outcomes (PROs) are quantitative measurements of functional outcomes. ¹ Outcome dimensions include health status achieved (eg, functional status, quality of life), the time and suffering involved in getting care, and the sustainability of benefits achieved. The purpose of PROs is to elicit the perspectives of patients. Measuring PROs with standardized questionnaires is 1 way of getting this information (1,2).

Patient-reported outcome measurements (PROMs) are measurement instruments that translate PRO in values. ² Patient-reported outcome measurement could be useful not only for patients but also for clinical decision-making and for population health managers—to compare and improve the quality and the accountability of health care services.

Patient-reported outcome collection has proliferated in oncology, urology, orthopedics, psychiatry, and primary care (1,2). However, research has revealed many technical barriers to successful adoption of PROs into routine practice (3,4). The use of PROs in the context of emergency department (ED) may be promising, since 80% of patients are discharged without any other way to track outcome (5), but their development has not been widely explored. Thus, a better definition of PROs and PROMs following an ED visit may highlight currently unrecognized areas for measurement and quality improvement (5 –10).

The objective of this cross-sectional study was to detect a set of PROs and PROMs about patients with isolated trauma of the limbs receiving ED care.

Materials and Methods

Study Design and Setting

A nonrandom sample technique was used for this study. A cross-sectional study was performed in a Level II trauma center with an annual ED census of 60 000 affiliated to the Ancona Teaching Hospital, Italy. A survey was performed in the ED using a questionnaire among the enrolled patients. During the study, the physicians and the nurses were blind to the research. The local ethics committee exempted our observational study from formal review.

Selection of Participants, Data Collection

From September 2019 to November 2019, between 8 am and 8 pm, consecutive patients aged 14 years or older with isolated trauma of the limbs, defined as any trauma to the superior and/or inferior limbs and no other trauma or acute disease, were conveniently enrolled for this study. Additionally, to be included, patients had to present to the ED within 24 hours of their trauma and have an Injury Severity Score less than 15. Patients whose mental status appeared altered (eg, because of psychotropic drugs or alcohol or an acute neurological or psychiatric condition) were excluded from the present study. Each patient received the questionnaire in 2 moments: at triage (item n = 1-10) and at discharge from ED (item n = 11-16).

The authors, an interdisciplinary team of physicians with training in Emergency Medicine, Internal Medicine, Public Health, as well as experts in Management, developed the 17-dicothomic (yes/no) question questionnaire (see Appendix A). Socioeconomic variables included age, sex, and level of education (low/elementary, intermediate, or high school or higher degree). Themes explored in the questionnaire were based on a review of existing PROs for ED care published in the scientific and gray literature (11,12), in particular by Cooke (11), who reported what the interviewed patients placed their high importance on and why these specific expectations were important for them. The questionnaire was specifically designed to gather data to inform the development of a conceptual model: patients’ expectations on their care and perspectives on the outcome of their care after their ED visit (see Appendix A). The initial questionnaire was validated during a pilot phase before administration. Experts’ opinion was used to estimate content validity at the item level. Internal consistency was assessed by focusing on the inter-item correlations within the questionnaire, by means of Cronbach’s α testing; an α between 0.70 and 0.95 was considered acceptable (13).

Outcomes Measures

The patients’ expectations on their care have been evaluated after the triage and before the visit as follows: achieving pain relief and achieving reassurance by health care professionals as patients’ expectations on their needs that would be addressed (item 2 and 8 of the questionnaire, see Appendix A) and time of staying in ED <4 hours, getting an x-ray, being visited by a specialist, obtaining a written therapy, and prognosis as patients’ expectations on the process of care (item 3-7). For each item investigating the patients’ expectations, the percentage of patients who agreed to perceive it important was calculated. Those outcomes for which the grade of agreement by the patients was ≥85% were defined core-agreed outcomes (Figure 1). We thought that this value was high enough because it was even higher than 75%, which resulted the median threshold to define consensus in a recent systematic review about Delphi studies (14). The analogic pain scale at the moment of the triage and at discharge were calculated too. Moreover, we asked each patient (a) how many minutes for his/her medical problem were perceived tolerable for the waiting time and for the whole time to be spent in ED (item 9-10); (b) which priority code they would have assigned for themselves as triage (item 1). The grade of concordance between the patient-assigned triage codes and the “real” ones was calculated. We asked each patient if he/she wanted to express one outcome that he/she perceived important.

Figure 1.

Flow chart for the study’s outcomes. PRO indicates patient-related outcomes; PROMs, patient-related outcomes measurements.

About perspectives on the outcome of their care after their ED visit, we evaluated at the discharge: a dichotomic statement about the well-being communication with health care professionals and about their global well-being (item n = 16-17). A contextual information in order to examine the capacity of the ED system to satisfy the outcomes was also collected: time spent in ED within the limit indicated by the patient and/or <4 hours; getting the pain control (defined as a reduction of pain scale ≥3), if perceived important by the patient; getting a x-ray, if perceived important by the patient; getting a visit by the specialist, if perceived important by the patient; getting a written therapy and prognosis, if perceived important by the patient (item 12-15). The percentages of agreed outcomes that were satisfied were calculated: For each outcome, a value ≥85% defined the ability of satisfaction by the ED system (Figure 1). Finally, the percentages of patients for whom the ED system was able to satisfy all the proposed outcomes, and at least the core-agreed outcomes were calculated.

Statistical Analysis

Association between global well-being/satisfaction with the visit and selected explanatory variables (age, sex, and level of education) was studied by χ² test; moreover, multivariable logistic regression model was built to evaluate factors independently associated to the global well-being on discharge; a step-forward approach was used including variables with P < .40 at bivariate analysis. Our data were descriptively analyzed with SPSS (version 18; SPSS Inc).

Results

One hundred patients were recruited according to our protocol. Four of these patients declined to participate in the survey; thus, 96 patients completed surveys, and their characteristics are shown in Table 1.

Table 1.

Demographic Characteristics of the Study Population.

Characteristic	Value
Total population	96
Mean age (range; SD), years	49 (14-90; 22)
Male (%)	59 (56)
Education: primary/middle school (%)	45 (47)
Code: green/white/yellow (%)	75/11/2 (86/11/3)
Median pain scale (range; SD),	6 (1–10; 2)
Time before visit (range; SD) minutes	28 (1-360; 73)
Time spent in ED (range; SD), minutes	210 (50-480; 96)

Abbreviation: ED, emergency department.

Among the proposed outcomes, 3 items were perceived as important by almost 85% of the patients according to their expectations (Table 2): getting an x-ray (91%; 95% CI: 88%-98%), obtaining a written therapy (94%; 95% CI: 87%-97%), and feeling the physicians’ and nurses’ empathy (97%; 95% CI: 91%-99%). These outcomes were the core-agreed outcomes. There was only a trend for consensus for obtaining a written prognosis (86%; 95% CI: 78%-92%).

Table 2.

Concordance of the Patients With the Proposed Outcomes.

	Expectation on
	Pain relief	Time spent in ED <4 hours	X-ray	Specialist’s visit	Written therapy	Written prognosis	Physicians’ and nurses’ empathy
Patients, n (%)	70 (73)	80 (83)	91 (95a)	76 (79)	90 (94a)	83 (86a)	93 (97a)

Abbreviation: ED, emergency department.

^a Value ≥85%.

The median time perceived as tolerable by the patients for their medical problem were: 120 minutes (range 10-600; SD 100) for the waiting time and 180 (range 10-600; SD 102) minutes for the whole time to be spent in ED. The concordance between the patient own-assigned priority codes and the “real” codes assigned by triage in ED was 75% (72/96), while in 18 cases (19%), the patients perceived to have been undertriaged, and in 6 cases (6%), the patients perceived to have been overtriaged. None of the participants detailed a specifically important outcome from a personal point of view.

Table 3 shows the percentage of the proposed outcomes that were satisfied by the ED system. The waiting time during ED stay and the whole time spent in ED were within the limits perceived tolerable by each patient in 78 (81%) cases and in 51 (53%) cases, respectively. Twenty-one patients preferred not to answer to the last 2 items (n = 16 and 17): 81% of the remaining patients (61/75) reported a good communication with physicians and nurses, and 59% (44/75) reported a global well-being after discharge from ED. The ED system was able to satisfy 2 of the 3 core-agreed outcomes in at least 85% of the cases: getting an x-ray (97%; 95% CI: 91%-99%) and obtaining a written therapy (97%; 95% CI: 91%-99%). There was a trend to a satisfaction by the ED system for being visited by a specialist (90%; 95% CI: 82%-94%) and obtaining a written prognosis (86%; 95% CI: 78%-92%). Moreover, in 30/96 patients (31%; 95% CI: 22%-41%), all the outcomes were satisfied, and in 75/96 patients (78%; 95% CI: 69%-85%), all core-agreed outcomes were satisfied.

Table 3.

Ability of the ED System to Satisfy the Proposed Outcomes.

	Pain reliefa	Time spent in ED <4 hours	X-ray	Specialist’s visit	Written therapy	Written prognosis	Physicians’ and nurses’ empathy	Well-being after discharge from ED
Satisfied patients (%)	20 (21)	51 (53)	93 (97b)	86 (90b)	93 (97b)	83 (86b)	61/75 (81)	44/75 (59)

Abbreviation: ED, emergency department.

^a Pain relief was achieved if there was a reduction of the Analogue Scale Pain ≥3 points from the triage’s value to the discharge’s value.

^b Value ≥85%.

Empathy during the visit was the only variable independently associated to self-perceived benefit of the visit at χ² test, and this association was not modified by age, sex, education level, waiting time, or pain before and after the visit at multivariable analysis (P < .05).

Limitations

First, our study was conducted in a single hospital, which may not be representative of all patient populations. It is possible that many indicators are only useful at a local level and do not lend themselves to national benchmarking. Second, only patients with isolated trauma of 1 or more limbs were analyzed because they are not critical, often young (even if one quarter of them were >75 years old) and have active life so they have time and ability to answer the questionnaire in ED. We know that our survey was applied to specific subgroup of ED patients, and not to all patients in the ED at that time; nevertheless, the number of these patients is high in our “real life,” being about 10% to 15% of all of the patients seen in ED, so the use of the PROs and PROMs we proposed might already be useful. Moreover, their potential application is definitely broader. Third, a written questionnaire was read to each patient in 2 different moments during their stay in ED and filled in by nurses or resident physicians who were blind to the scope of the study, but it could be possible that both these factors may have had some influences on the patients anyway. Fourth, because of the limited size of the study population, it is possible that we lacked statistical power to analyze possible correlations between age, education type of trauma, scale of pain and outcomes, even if an association between empathy during the visit, and a global well-being was found.

Discussion

There is a growing emphasis on including patients’ perspectives from medical research toward applications in the clinical setting, quality measurement, and system accountability (12,15,16,3,17). To date, this has been challenging in the ED setting (9,10,5). Our study shows an example of core of outcomes proposed by the ED physicians and agreed by the patients: We identified 3 outcomes as the core-agreed outcomes in a specific subset of patient with isolated trauma of the limbs: getting an x-ray, obtaining a written therapy, and feeling the physicians’ and nurses’ empathy. The latter was also significantly associated to a self-perceived benefit of the visit by patients at multivariate analysis. Therefore, our data seem to confirm empathy as one of the main drivers of patients’ satisfaction in the ED (18). Vaillancourt and coworkers (19) stated that 46 patients with diverse reasons for seeking care in ED were interviewed by phone within 10 days of the ED visit. Questions were designed to elicit patients’ perspectives on the outcome of their care after their ED visit and let the authors to identify common outcomes from ED care that centered around 4 themes: understanding the cause and expected trajectory of patients’ symptoms; reassurance; symptom relief; having a plan to manage symptoms, resolve issue, or pursue further medical care. These 4 domains constituted a proposed conceptual model. Since all the patients in our study were enrolled because of trauma, understanding the cause of their symptoms was less significant in our study. In our survey, reassurance and having a plan to manage symptoms were among the core-agreed outcomes. Interestingly, pain relief received the lowest percentage of concordance among the interviewed patients (70%). However, our data were too limited to make a correlation between the scale of pain and the pain relief’s expectation for each patient.

Notably, no patient even suggested an outcome that was perceived important for him/her. In our opinion, it is very interesting because in real life physicians hear many complaints by the patients, especially during their stay in ED, every day. Once our patients had the possibility to express their opinion than they didn’t. One explanation would be that the suggestions of the questionnaire were exhaustive, and another one would be that it is no simple for a patient to formulate expectations for him/herself especially in an emergency setting.

In emergency medicine, there is a lack of consensus regarding what an appropriate framework for measuring quality should look like (20,21). Bos et al (22) presented the Consumer Quality Index for the Accident and Emergency department as a validated survey to measure health care performance in the EDs from patients’ perspective. The “global quality rating” and the following 5 domains had the capacity to discriminate among EDs: timeliness, attitude of health care professionals, technical quality of received care, information during treatment, environment, and facilities. Raleigh et al (23) studied 6 domains of patients’ experiences and found that 1 of 21 analyzed A&Es performed better on all 5 discriminative domains. More recently, Vaillancourt et al developed and validated the Patient-Reported Outcome Measure for Emergency Department Care (PROM-ED) for use with a general patient population following discharge home after ED care (24). In this study, a core-agreed outcomes as framework for measuring quality was presented. It could be used as PROM to measure the ability of an ED to ensure each outcome in at least 85% of the patients or to ensure all these 3 outcomes in each patient: In our ED system, the latter performance measure was 78%. We think that similar values could let the identification of a best practice ED, which potentially would be a role model for other EDs, and might have a general positive effect on quality of care. Based on the involvement of a great number of patients, our questionnaire and other similar can be used to capture the perspective of patients routinely, focusing the work of clinicians on improving outcomes for patients. For example, we found that health worker’s empathy was perceived as important for the majority of the ED (97%) patients, so it could be invested to improve this competence among ED physicians and nurses. The result of such a process could be to increase the rate of satisfied ED patients, at least for this specific area, above the reported value (81%), helping to raise the quality of the ED system.

In conclusion, in this article, a set of PROs and PROMs about patients with isolated trauma of the limbs receiving ED care were reported. This could be thought as a conceptual model to assess the patients’ views of their outcomes and potentially to develop a quality measurement program in the practice, administration, and research of the ED care.

Author Biographies

Vincenzo G Menditto, MD and PhD, is an emergency and internal medicine attending physician in the Ospedali Riuniti di Ancona Hospital Department of Emergency Medicine. He has academic interests clinical research, emergency medicine and patient experience.

Antonello Maraldo is the administrative director of the Ospedali Riuniti di Ancona Hospital.

Pamela Barbadoro is an associate professor of Public Health at Università Politecnica delle Marche.

Roberto Maccaroni is a nurse and works in the Ospedali Riuniti di Ancona Hospital Department of Emergency Medicine.

Aldo Salvi, MD, is the Director of Ospedali Riuniti di Ancona Hospital Department of Emergency Medicine.

Marcello M D'Errico is a professor of Public Health at Università Politecnica delle Marche.

Stefano Marasca is a professor of Management at Università Politecnica delle Marche. His academic and fellowship work has focused on systems improvement, with an emphasis on outcomes.

Appendix A: Questionnaire

At the Moment of Triage

1. Which priority code would you assign to yourself? (a) White (b) Green (c) Yellow (d) Red

2. Is “pain relief” one of your expectations? yes/no

3. Is “time to be spent in ED <4 hours” one of your expectations? yes/no

4. Is “getting an x-ray” one of your expectations? yes/no

5. Is “being visited by a specialist” one of your expectations? yes/no

6. Is “obtaining a written therapy” one of your expectations? yes/no

7. Is “obtaining a written prognosis” one of your expectations? yes/no

8. Is “achieving reassurance by with health care professionals” one of your expectations? yes/no

9. Do you want to add an outcome you perceive as important for you? What is it?

10. How many minutes you perceive as tolerable for waiting before the visit in ED for your actual problem?…minutes

11. How many minutes you perceive as tolerable to be spent in ED for your actual problem?

Pain score: /10

At the Moment of Discharge From ED:

• 12. Have you got an x-ray? yes/no

• 13. Have you been visited by a specialist? yes/no

• 14. Have you obtained a written therapy? yes/no

• 15. Have you obtained a written prognosis? yes/no

• 16. Have you achieved reassurance by physicians and nurses? yes/no

• 17. Do you feel better now than when you came to the ED? yes/no

Pain score: /10