Development of a Patient‐Reported Outcome Measure (PROM) in Bony Facial Trauma Patients

ABSTRACT

Objectives

There are currently no validated patient‐reported outcome measures (PROMs) to specifically evaluate quality‐of‐life outcomes for patients who have suffered bony facial trauma. Our goals are the following: (1) Investigate functional and psychosocial concerns that are self‐identified as important among patients following bony facial trauma. (2) Formalize these factors into an appropriate patient‐reported outcome measurement tool that can be utilized clinically.

Methods

We performed semi‐structured interviews with 10 providers at the University of Virginia and other academic institutions who are experts in the field of facial trauma. Providers were asked about functional and psychosocial concerns that patients with bony facial trauma may experience. Responses were then used to create a preliminary PROM instrument, which was programmed for self‐administration through the Qualtrics online survey platform. Subsequent semi‐structured interviews with 15 patients affected by bony facial trauma were conducted during the cognitive debriefing portion. Patients completed the questionnaire and their responses provided focus to the finalized PROM survey.

Results

The final PROM instrument included functional concerns such as nasal obstruction, persistent facial numbness, vision issues, and limited jaw mobility. Psychosocial concerns included difficulty completing personal tasks, feeling depressed or anxious, and not wanting to spend time with family or friends.

Conclusion

Using a standardized approach, we developed a patient‐reported outcome measure specific to bony facial trauma patients by performing semi‐structured interviews with content experts as well as cognitive debriefing interviews with patients. The final PROM included functional and psychosocial concerns relevant to the bony facial trauma population, such as vision issues and difficulty completing personal tasks. Future directions of the study will include a multi‐institution validation phase with a larger volume of patients.

Level of Evidence

4.

We utilized structured survey development methodology to develop a PROM specific to patients affected by bony facial trauma. The final PROM included functional concerns such as vision issues, nasal obstruction, limited jaw movement, facial numbness, facial asymmetry, and persistent pain. Psychosocial factors such as difficulty completing personal tasks, not wanting to spend time with family or friends, and depression or anxiety related to the facial injury were included.

1. Introduction

Over the last few decades, health care systems have increasingly recognized patients' perspectives as fundamental to ensuring that medical services are of high quality and delivered in an equitable and safe manner. The expanded use of PROMs has become integral to this shift in healthcare [1, 2, 3]. PROMs are standardized questionnaires that collect information on health outcomes directly from patients, including symptoms, health‐related quality of life, and functional status [4, 5]. They also include psychosocial factors such as how patients perceive themselves and their appearance [2, 4, 5, 6, 7]. PROMs were first developed in clinical trials to assess the effectiveness of treatments [8, 9, 10]. More recently, their use has expanded to include supporting clinical decision making, comparing outcomes among health care providers, stimulating quality improvement, and evaluating practices and policies [6, 9, 10, 11, 12].

PROMs are extremely important in understanding and improving patient care at multiple levels of the health care system [4, 5, 6]. PROMs have been shown to dramatically improve patient‐provider communication and patient satisfaction [13, 14, 15]. Patient satisfaction has become an important component of reimbursement models implemented by the Centers for Medicare and Medicaid Services (CMS) [13]. Ideally, PROMs undergo psychometric validation to ensure that they accurately reflect outcomes they hope to measure and that they can reliably assess changes over time [6, 7, 8].

Facial trauma is considered a serious public health problem in both developed and developing countries. On average, there are 3,000,000 bony facial traumas in the United States and 10.7 million cases worldwide each year [9]. Facial fractures are often associated with a loss of function, disfiguration, and psychological problems [9, 10, 11, 12]. Therefore, it is essential that this patient population is further studied. We must determine how these injuries have influenced the physical and mental health of these patients to optimize their healthcare and overall quality of life [3, 4, 5, 9].

While there are reports of utilizing several PROM instruments in patients who have suffered a facial injury, there are no PROM instruments that have been specifically developed and validated in the bony facial trauma population [7, 8, 9, 10]. Hence, our objectives in this study are as follows: (1) Investigate functional and quality‐of‐life concerns that are self‐identified as important among patients following facial trauma and (2) formalize these factors into an appropriate patient‐reported outcome measurement tool that can be utilized clinically. Ultimately, this will help us better understand the patient's perspective following facial trauma, guide surgical care toward improving these outcomes, and ultimately optimize patient health outcomes.

2. Materials and Methods

Determining relevant items to include in a PROM involves a standardized protocol that adheres to internationally recognized and accepted guidelines for PROM development [5, 6, 7, 8]. We consulted with the Center for Survey Research at the University of Virginia to ensure that we strictly adhered to PROM development guidelines [3, 6, 8]. We began the study with concept elicitation interviews, which included semi‐structured interviews with 10 physician experts within the field of Facial Trauma at the University of Virginia and other academic institutions including the University of California‐Los Angeles, Penn State University, Medical University of South Carolina, University of Maryland, Tripler Army Medical Center, and Eastern Virginia Medical Center. This included providers within the fields of Otolaryngology, Oral Maxillofacial Surgery, and Plastic Surgery. Interviews were performed virtually and the conversations were recorded with their consent. Interviews lasted 20–30 min and followed a semi‐structured pattern. The providers were asked about their experiences in treating facial trauma patients, including whether they currently track any outcomes in this patient population. We then discussed a range of patients with upper, midface, and lower facial fractures. They were asked about functional, psychological, and other quality‐of‐life issues that these patients may experience. The physician responses were then utilized to create a preliminary patient‐reported outcome measure survey.

Qualtrics was utilized to program a preliminary set of 10 PROM items in conjunction with the Center for Survey Research based on the concept elicitation interviews. This was achieved by recording the most frequently cited functional and psychosocial concerns deemed relevant by physician experts in the facial trauma population. Simple ordinal scales were used to gauge severity for each area where the respondent voiced concern. We adjusted the survey to a middle‐school reading level. We then proceeded with the cognitive debriefing interviews, which were useful to determine the clarity and importance of the included items to the target population.

After obtaining IRB approval at the University of Virginia (HRS #230226), a list of patients treated for bony facial trauma over the previous 2 years was created through the electronic medical record. This included 78 patients. Criteria for inclusion in the study were: age ≥ 18 years old with the ability to read and speak English. Patients were excluded for the following reasons: major psychiatric illnesses (defined as schizophrenia, schizoaffective disorder, bipolar disorder, major depressive disorder), cognitive impairment, known traumatic brain injury (such as subdural or epidural hematoma, longstanding neurological deficits), or disabling chronic medical illnesses. Post‐traumatic stress disorder did not disqualify a patient from being included in the study. Concussion was not officially diagnosed in the majority of patients we reviewed, but patients were included in the study if they did not have any cognitive or neurological deficits from the injury. Patients who did not have any available contact information in their chart were also excluded. The final list included 44 patients. Patients were selected for enrollment to include a wide range of injury types (all facial zones), time frame from injury, and treatment modalities (operative vs. nonoperative). This would allow the PROM to apply to the full spectrum of bony facial trauma patients.

We then created an interview script that was in accordance with PROM guidelines [8, 11]. The questions and answers were programmed into Qualtrics, a high‐quality online survey platform. Patients were initially contacted over the phone. If they agreed to participate, we explained the study process including a brief virtual interview (10–15 min) and that they would receive a study incentive ($25 gift card) for their participation. The preliminary PROM instrument was administered to 15 patients who had suffered facial trauma, including a range of upper, midface, and lower facial trauma (see Table 1 for further details). Given that several patients suffered their injury over a year ago, all patients were asked to fill out the survey as if they were reporting from the time of the initial injury, rather than in the last 2 weeks as stated in the PROM. The interviews allowed the patients to freely discuss their experience after the facial trauma while also trying to prompt them when appropriate. The sample size of the study was determined by the saturation point of data, so an exact sample size for the cognitive debriefing phase of PROM development was not determined prior to patient interviews. Saturation is a term utilized in PROM survey development where no new additional emergent themes were identified during the structured interviews [16]. Patients then completed the PROM survey followed by an open discussion regarding their interpretation of the items and their thought process in response to the selection. Item revisions were then implemented into the final PROM with consideration of the suggested improvements from the patients. This included avoiding medical jargon and clarifying that all questions were related to the facial trauma itself rather than possible confounding factors such as other injuries or comorbid conditions. The final version of the PROM had a Flesch–Kincaid grade reading level of 5.8.

TABLE 1.

Patient characteristics: cognitive debriefing interviews.

Variable	Value (N = 15)
Age, years
Mean (SD) [range]	38.2 (18.2) [21–68]
Gender, n (%)
Male	8 (53.3%)
Female	7 (46.7%)
Location of fracture
Orbital floor	4 (26.7%)
Midface	3 (20%)
Mandible	5 (33.3%)
Combination	3 (20%)
Surgical status
Operative	9 (60%)
Nonoperative	6 (40%)

3. Results

The mean age of the 15 patients included in the cognitive debriefing interviews was 38.2. The standard deviation was 18.2 with a range of 21–68 years old. See Table 1 for additional demographic information. Most patients who were interviewed had their initial injury within 9 months from the time of the interview, but there were four patients whose injury was over a year ago. All patients suffered their facial injury within the past year and a half. The average time between the trauma and the interview was 6.2 months (SD ±4 months, range: 2–18 months).

The preliminary and final PROM instrument had the same functional and psychosocial factors. However, the final PROM instrument clarified that the variables were strictly related to the facial injury itself and not to other compounding variables such as other injuries/fractures or comorbid conditions. The final survey also eliminated any medical terminology or wording that could be confusing for patients with low health literacy. The final PROM included a time frame of “in the last 2 weeks” to capture the patient's most recent symptoms for when it is administered clinically. The survey included a combination of functional and psychosocial variables, including vision changes, nasal obstruction, limited jaw movement, facial numbness, facial asymmetry, and persistent pain. We utilized the Numeric Pain Rating Scale (NPRS), which has been extensively studied for its validity and reliability [17]. Psychosocial factors included not wanting to spend time with family or friends as well as difficulties completing household or personal tasks because of the facial injury. Depression and anxiety related to the facial injury were also deemed important (Figure 1).

FIGURE 1.

Patient‐reported outcome measure in facial trauma patients.

See Table 2 for a description of the various functional and psychological symptoms experienced by patients according to the fracture location. Anxiety was common around the time of injury in patients with orbital floor fractures. In the cognitive debriefing interviews, they attributed their anxiety to uncertainty about long‐term vision. Three out of the five patients with mandibular fractures reported not wanting to spend as much time with family and friends around the time of their injury given pain and difficulty eating. Two patients with multisite fractures reported depression in the time surrounding their injury due to feeling socially isolated and their inability to exercise; both of those factors were important for maintaining their mental health.

TABLE 2.

Symptoms based on fracture pattern.

Fracture pattern	Functional symptoms	Psychosocial symptoms
Orbital floor	4/4: vision changes	4/4: difficulty in completing personal tasks
	2/4: temporary facial numbness	3/4: anxiety around time of injury
Mandible	4/5: limited jaw movement a	3/5: not seeing family and friends
Midface	2/3: facial asymmetry b
	1/3: nasal obstruction
Combination	3/3: nasal obstruction, facial numbness, facial asymmetry, persistent pain	2/3: depression around time of injury

^a 3/5 patients required open reduction and internal fixation of the fractures.

^b Prior to surgical repair.

Lastly, Table 3 demonstrates the correlation between the patients' self‐reports of their physical symptoms and their reports of pain and psychosocial distress. The SymptomAreas measure counts how many of the first five items from the PROM received a “yes” response. These included vision, nasal obstruction, limited jaw movement, numbness, and facial asymmetry. The Experience measure is the sum of the patients' responses to the four psychosocial items plus their pain score divided by two. Since pain was reported on a 0–10 scale, it was reduced by half so that its variance would be more comparable to that of the psychosocial items, which are measured on a five‐point (0–4) scale.

TABLE 3.

Correlating SymptomAreas and mean experience score.

SymptomAreas: number of physical symptom areas	N	Mean experience score	Std. deviation
0	4	0.50	0.58
1	3	2.83	2.02
2	5	2.90	1.34
3	3	5.83	1.61
Total	15	2.83	2.24

4. Discussion

There are currently no validated patient‐reported outcome measure tools within the facial trauma population [1, 2, 3, 4, 5, 6, 7, 8]. One study by Elegbede et al. investigated patients following primary repair of traumatic facial fractures at a Level I trauma center from 2016 to 2018 [4]. They utilized FACE‐Q, a set of patient‐reported outcome instruments designed for facial esthetic surgery, to measure satisfaction and health‐related quality of life following the repair of traumatic facial fractures [4, 7]. They found that predictors for lower satisfaction were tobacco smoking status, maxillomandibular fixation, and Le Fort pattern fractures. On average, patients reported poor health‐related quality of life in the early postoperative period [4]. The FACE‐Q questionnaire has only been validated in facial esthetic patients [7]. It has not been studied specifically in patients who have suffered facial trauma.

Facial trauma is an ongoing public health concern due to its significant burden on the patient, his or her families, as well as the healthcare system. Given the magnitude of its functional and psychological impact, it is essential that treatment‐specific tools are developed that appropriately measure outcomes in this patient population. Functional variables such as malocclusion and vision changes have been commonly identified as physical issues in facial trauma patients. However, there are no studies that have analyzed psychosocial variables in facial trauma patients [4, 7]. Quality‐of‐life assessments are becoming essential in not only ensuring optimized patient outcomes, but they are also becoming more prevalent as a key indicator in various reimbursement models [3, 5, 6, 7, 8].

We found that there was significant overlap between what physician experts and patients identified as important regarding function and psychosocial variables. The items included in the PROM survey represented a range of factors relevant to patients after their facial trauma. Of note, there were several additional variables that could have been included in the PROM, including malocclusion, point tenderness, specific eating or diet restrictions, or persistent cognitive or concussion symptoms. However, symptoms such as these are very specific and are frequently documented in clinical encounters. The goal of our PROM was to identify factors that we do not currently measure in practice regularly but they are significant to the patient experience after suffering facial trauma. In addition, there are many variables that should be captured by the psychosocial portion of the questionnaire. Lastly, patients were given the opportunity to review and respond to the items within the PROM during the cognitive debriefing phase and they did not suggest any additional items to be included beyond the saturation point of the themes that were identified in the current PROM. We therefore elected to keep the PROM focused on the most common variables identified by surgeons and patients during the semi‐structured interviews. We hoped to keep the survey as user‐friendly as possible to encourage widespread clinical use.

Patients were more prone to developing certain symptoms depending on the location of the fracture, including vision changes in the setting of orbital floor fractures and limited jaw mobility in mandibular fractures. A benefit of the PROM is that it is able to measure quality of life for complex, multisite fractures. Several patients commented on the impact of the injury on their mental health and social life, including anxiety about the uncertainty of their long‐term vision and depression given their inability to exercise regularly. While functional concerns are paramount in optimizing surgical outcomes, we must also appreciate the psychosocial impact of facial injuries in this patient population. We found that patients who responded “yes” more commonly to the first five questions of the PROM (SymptomAreas score) were more likely to have a higher Experience score, which included the adjusted pain score and the four psychosocial components. This could indicate that the more symptomatic a patient is after their facial trauma, the more likely they are to suffer from other comorbid conditions such as not wanting to spend time with family/friends or anxiety. This preliminary analysis suggests the PROM instrument is able to differentiate the number of functional symptoms with an increased severity in reported quality of life, but the current sample size of this study is too small to draw significant conclusions at this stage. This concept will be further analyzed in the validation portion of the study.

Despite following rigorous methodology important in PROM generation, this study has several limitations. Although we approached 44 eligible patients, the 15 who were ultimately enrolled did so based on their willingness to participate and each participant received a $25 gift card for the time required for the extended cognitive debriefing interviews. This may introduce some bias into which patients chose to participate in this study. While the concept elicitation stage included experts from multiple surgical specialties and institutions, patients included in the cognitive debriefing portion were limited to a single institution. This potentially limits the generalizability of this PROM to patients in different geographic and sociodemographic locations. There was a component of recall bias given that a few of the patients had experienced bony facial trauma over a year from the time of the interview and they may not have accurately remembered their symptoms around the time of the trauma. Our inclusion of patients with both recent and remote traumas allowed for elucidation of QOL themes from a more diverse timeframe and these patients identified similar QOL. This may improve the generalizability of this PROM in the longitudinal tracking of patient outcomes, so these patients were included to enhance the relevance of this instrument to patients outside the immediate trauma timeframe.

There was a relatively small final sample size (n = 15) for the cognitive debriefing survey portion, but this was determined using standardized PROM development protocols including the saturation point, indicating that additional patients were not necessary given that themes and key points had already been discussed at length and undergone rigorous analysis [16]. According to this theory, additional patients beyond the 15 that were interviewed would not have significantly impacted the variables included in the survey. Additionally, we sought to develop a PROM that could encompass quality‐of‐life and psychosocial factors for all zones of facial injury, rather than restricting its relevance to an isolated zone. This will mean that some symptoms (e.g., vision disturbance) will not apply to every patient (e.g., mandible fractures), but our aim was to capture a broad spectrum of patient experiences to allow more widespread application of this instrument in the future. Next steps include a validation phase among a larger number of patients affected by facial trauma from multiple institutions in order to perform psychometric testing of this instrument to ensure internal and external validity prior to adopting this PROM for routine clinical use. This includes comparing the survey with other validated PROMs with overlapping measurement areas to evaluate the external validity of this new instrument. Planning for this phase is currently underway in collaboration with facial trauma experts from several institutions.

Once validated, this PROM will hopefully facilitate and standardize information gathering regarding crucial factors in the preoperative and postoperative setting. Input from patients who were treated nonoperatively will also be essential in better tailoring the care we provide to these patients. Administering the PROM could help physicians better address perioperative counseling with the facial trauma population, including what to expect in the recovery period. Specialists in ophthalmology/oculoplastic surgery, dental/prosthodontics, and physical therapy could also be included in future validation of the instrument to broaden the generalizability. Utilizing this PROM longitudinally could also help improve our ability to monitor outcomes of importance to our patients and better assess and compare efficacies of our facial trauma treatments. Early identification of any psychosocial concerns related to the injury could allow for expedited referrals to mental health or other ancillary providers if necessary to aid in the healing process for this vulnerable population.

5. Conclusion

We utilized structured survey development methodology to develop a PROM specific to patients affected by bony facial trauma. The final PROM included functional concerns such as vision issues, nasal obstruction, limited jaw movement, facial numbness, facial asymmetry, and persistent pain. Psychosocial factors such as difficulty completing personal tasks, not wanting to spend time with family or friends, and depression or anxiety related to the facial injury were included. Cognitive debriefing interviews confirmed that the survey can be efficiently administered in the clinical setting and contains elements of importance to patients. Future directions of the study will include a multi‐institution validation phase with a larger number of patients.

Funding

This work was supported by the Paul A. Levine Research Grant, Department of Otolaryngology, University of Virginia.

Disclosure

The authors have nothing to report.

Conflicts of Interest

The authors declare no conflicts of interest.