Engaging patients to develop a patient-reported outcome measure for the emergency department

KEY POINTS

• We developed and validated the Patient-Reported Outcome Measure for Emergency Department Care (PROM-ED), the first PROM for use with a general patient population following discharge home after emergency department care.

• Patient partners were involved throughout the study to help guide the development of the questionnaire.

• We differentiated the perspectives of patient partners from those of study participants, who were of primary interest.

• Partnering with patients in research requires substantial time and resources, and involves major challenges with regard to institutional registration, research ethics board credentialling and authorship guidelines.

Emergency departments are a major point of access to timely care for Canadians, and yet little is known on the outcome of the care provided. Patients’ perspectives on the outcomes of their care could provide an untapped source of information for improvement. Past attempts at questionnaires for patients seen in the emergency department have focused on the experience or satisfaction with the process of receiving care and have had limited impact on quality improvement.^1,2 Since the ultimate purpose of medical care is to improve health outcomes, many questionnaires have been developed for a range of conditions to gather this information reliably from patients’ perspectives; these questionnaires are called patient-reported outcome measures (PROMs). No such tool exists to gather the perspectives of patients seen in the emergency department on outcomes after they go home. An example of the change in scope can be seen in the difference between asking patients, days after their care, “How are your symptoms now?” (outcome) instead of “Did you receive medication for your pain during your time in the emergency department?” (experience). In emergency department care, there is evidence that unmet patient needs is a problem on its own, which increases repeat visits.^3,4 Measuring such outcomes could inform care improvement at the clinician and system level. We designed a study to develop and validate a PROM to measure outcomes of importance to patients seen in the emergency department in the days after they are discharged home.

We characterized outcomes of importance to patients after emergency department care through a qualitative study in which we conducted in-depth interviews with patients 2 to 10 days following their care in the emergency department. Interviews with patients in this time frame after their care allowed us to characterize four core outcome concepts to measure with our PROM. These were symptom relief, understanding of their health concern, reassurance and having a plan.⁵ Based on this foundational work, we went through a comprehensive development and validation process, which resulted in the Patient-Reported Outcome Measure for Emergency Department Care (PROM-ED) questionnaire (Appendix 1, available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.180353/-/DC1).

We started with a formal review of existing questionnaires for patients seen in the emergency department. Most of such questionnaires were patient-reported experience measures (PREMs), and we found no adequate fit for the concepts identified. Based on relevant questions identified, we built a group of questions for each outcome concept, supplemented by other questions based on the qualitative interviews. Questions were then tested with a group of eight patients days after their emergency department care. They were asked to think out loud as they answered the questions one-on-one with a trained research assistant, a process called cognitive testing. Based on these interviews (which were recorded, transcribed and analyzed), we removed or changed questions to improve accuracy and comprehension. A refined version of the questionnaire was then tested with 444 participants who had recently received emergency department care. We conducted statistical analyses for field and validity testing to identify questions that may not add pertinent information. The results of these analyses were then presented to a group of 21 experts (including five patients), who made final decisions on the questions to be included in PROM-ED. The final questionnaire includes 21 questions covering four outcome domains (www.prom-ed.org). The perspective of the patient partners was critical in each phase of development.

Image courtesy of doranjclark/iStock

The purpose of a PROM is to elicit the perspectives of patients and translate them as faithfully as possible into a reliable measurement questionnaire. Three patient partners at different times acted as brokers between the perspectives of the research team and the patient participants. Early on, the principal investigator (S.V.) fortuitously met with coauthor A.M., who had been involved in creating recommendations for patient partners, and they began a collaboration on the design and involvement of other patient partners in developing the methodology for PROMED. A patient partner (A.C.F.) was approached by a research assistant because she was involved with the volunteers’ association of a partner hospital; another patient partner (C.H.) was recruited because she had prior experience working with coauthor D.B. on the development of other outcome instruments in rheumatology.⁶ Two other patients (B.C. and N.E.) also collaborated with the research team for shorter periods.

In the study design, we differentiated between patient partners, who were members or advisors to the research team, and patient participants, who were conceptualized as the focus of the study (Appendix 1). Patient partners were recruited organically, whereas we made purposeful efforts to recruit a diversity of participants through the use of a sampling matrix (identifying sex, age, reason for seeking care, income and ethnicity) for the qualitative study and cognitive testing, in addition to using a national survey panel for the field and validity testing.

The role of the patient partners was particularly affected by two features of developing a PROM for emergency department care. First, there was an early recognition that the perspectives of patient partners could not be a substitute for those of the patient participants, who were intended to be much more diverse and elicited through rigorous methodology. The patient partners (A.M., A.C.F. and C.H.) were experienced and insightful, having worked on guidelines for recruitment and roles of patient partners. This contribution proved very important as initial concerns gave way to the realization that insightful patient partners could maintain a perspective broader than their own and make the research team more empathetic to the perspectives of patients broadly.

A second feature became apparent as the study unfolded. Most of the investigators became emergency department patients at some point during the project. One had a broken wrist, one suffered a bad sore throat and was concerned about mumps, and another accompanied a close relation to the emergency department for assessment. All now had momentary experiences as patients seen in the emergency department. Instead of taking away from the unique role of patient partners, these circumstances added to the validity of their voices, reminding us of our own lived expertise as emergency department patients. This allowed us to supplement the meaning of the conceptual framework and questions with our own experiences.

Over the course of the project, we sought to meet each other’s evolving expectations. It proved difficult, given the relatively small budget of the PROM-ED project, to set up systems for onboarding, orientation and support of patients as partners. We managed challenges on a case-by-case basis as the project unfolded, and at times we fell short of our intent in providing constant communication and opportunities for contribution to patient partners and all investigators. During periods of data analysis, there were sometimes longer periods without news updates, which made it difficult for patient partners to feel engaged. As a remedy, our part-time research assistant set up monthly news updates.

During our study, we faced logistical hurdles at the institution and ethics levels to the involvement of patient partners. Access to the data is an issue. Funding agencies have recently welcomed the addition of patients as partners. However, our research ethics board did not have a tailored process for training or credentialling patient researchers. This means patient partners must make a large commitment if they are to have any access to primary data. For example, we used the primary data from the qualitative study to create some of the questions. Some of this information had to be restricted from the patient partners, who were not registered with the institution and did not have adequate ethics training. Much work has been done to facilitate patient involvement, but institutional registration and specially tailored ethics training are still lacking.⁷ This is even more of an issue as attempts are made to improve the diversity of backgrounds of patient partners.

Publications are the usual way to acknowledge people’s contribution to new knowledge. However, patient partners’ contributions to a research project often fall short of a strict interpretation of the authorship guidelines outlined by the International Committee of Medical Journal Editors.^8–10 This is due to patient partners’ volunteer position, lack of access to funding and sometimes inability to sustain years of engagement. More emphasis on recognizing lived-experience contribution in authorship guidelines would address this issue.

Today, the idea of patient partners as part of a research team undertaking the development of a patient-reported outcome measure would appear uncontroversial and that, in itself, is great progress. Based on the involvement of a great number of patients, the PROM-ED questionnaire can now be used to capture the perspective of patients routinely on the result of their care, focusing the work of clinicians on improving outcomes for patients.