Assessing the Use of Patient-Reported Outcome Measures in the Routine Clinical Care of Chronic Rhinosinusitis Patients: A Canadian Perspective

Abstract

Importance

Chronic rhinosinusitis (CRS) is a common inflammatory disease of the paranasal sinuses with significant quality of life impairments. There is a need to implement outcome-based metrics to evaluate the outcomes of CRS treatment with endoscopic sinus surgery or biologics.

Objective

We aimed to understand Canadian otolaryngologists’ opinions on patient-related outcome measures (PROM) for CRS and identify potential barriers to implementation.

Design

Qualitative research.

Setting and Participants

A cross-sectional survey was distributed via the Canadian Society of Otolaryngology-Head and Neck Surgery and direct emailing.

Measures

Participants’ demographics, practice information, and opinions on PROM were collected.

Results

Of 346 (23%) Canadian otolaryngologists, 78 responded to the survey (26 rhinology fellowship–trained, 51 non-fellowship-trained, and 1 missing data). Thirty-eight responded that they collect PROM (69% with fellowship-trained, 39% non-fellowship-trained, P = .029). Regarding opinions on PROM, 74% of respondents agreed that it helps patients report their symptoms, 42% agreed that it improves the efficiency of the patient encounter, 54% agreed that it is easy for patients to understand, 62% agreed that it improves management and monitoring of clinical outcomes, and 71% disagreed that PROM is not helpful. Fellowship-trained otolaryngologists were 4 times more likely to agree that PROM improves management and monitoring of clinical outcomes (P = .014), and no other differences in opinions were significant. The most-frequently-identified barriers to PROM usage were lack of time for 67% of respondents, difficulty integrating into clinical workflow for 64%, and lack of integration into the electronic medical record for 47%. If these barriers were addressed, 86% of respondents said they would use PROM in their practice.

Conclusions and Relevance

Despite the low uptake of PROM among otolaryngologists without rhinology fellowship, opinions were generally favorable. We identified barriers that, if addressed, may increase their use in clinical practice. As resource-limited therapies such as biologics become more prevalent in CRS management, PROM may find more applications in shared clinical decision making.

Graphical abstract.

Introduction

Chronic rhinosinusitis (CRS) is a common chronic inflammatory disease of the paranasal sinuses with approximately 2.5% to 5% of prevalence in the Canadian population.^1,2 It is associated with numerous negative health effects, including reduced quality of life, impaired sleep, fatigue, and emotional distress.^3-7 There are significant direct and indirect costs associated with CRS, estimated in the United States to be in excess of US$9.9 billion and $13 billion per year, respectively.^8,9 In the context of a publicly-funded healthcare system in Canada, the out-of-pocket expenses per patient per year has been estimated to be CA$607.10. ¹⁰

Endoscopic sinus surgery (ESS) has been shown to be effective in improving quality of life and reducing healthcare spending in appropriately-selected patients.^8,9,11 ESS is a commonly-performed procedure in North America with an estimated prevalence of 10,000 to 15,000 cases of ESS per year in Canada and 250,000 cases of ESS in the United States.^12,13 However, there is a large geographic variation in resource utilization for ESS, which may hinder the provision of quality care for patients with CRS.^12,14 Therefore, there is a need to implement outcome-based metrics to evaluate the outcomes of CRS treatment. The Quality Improvement Committee of the American Rhinologic Society devised a framework for quality measurement in the care of patients with CRS, including assessing for the appropriateness of ESS.^15,16 One of the appropriateness criteria involves the use of Sinonasal Outcome Test 22 (SNOT-22), which is a CRS-specific validated patient-reported outcome measure (PROM). ¹⁶ Other available PROMs include the Rhinosinusitis Disability Index (RSDI), Chronic Sinusitis Survey (CSS), and CRS-PRO. The introduction of biologics in the treatment of CRS also highlights the need to use appropriate validated PROMs in determining which patients should receive biologics.^17-19 In fact, the use of PROM in clinical practice is recommended by international consensus statements such as the International Consensus Statement on Allergy and Rhinology: Rhinology 2021, the European Position Paper on Rhinosinusitis and Nasal Polyps 2020, and the Canadian Rhinology Working Group consensus statement.^17-19

Despite the demonstrated importance of the use of PROM, it is unclear whether PROM is used on a consistent basis clinically and whether there are any barriers in implementing PROM in daily clinical practice. Therefore, our objectives were as follows: (1) to determine the frequency of use of PROM such as SNOT-22 in clinical practice among Canadian otolaryngologists and (2) to evaluate the potential barriers in adopting PROM in otolaryngology clinics.

Methods

Study Design

We conducted a cross-sectional survey investigating opinions on the use of PROM for CRS among Canadian otolaryngologists. This study was approved by the University of Toronto Research Ethics Board (RIS Protocol #43494) and distributed on November 7, 2022, by the Canadian Society of Otolaryngology-Head and Neck Surgery (CSOHNS). A second round of surveys was then sent out via directed email on September 5, 2023. Responses were collected until October 5, 2023. Potential participants were informed that participation was completely voluntary, and completion was considered implied consent.

Study Population

We manually compiled, to the best of our knowledge, a list of otolaryngologists currently practicing in Canada through the Canadian Society of Otolaryngology and previously-collected databases by the senior author, which may not capture the full list of otolaryngologists practicing in the country. In order to minimize the risk of bias, we included all generalist and subspecialty otolaryngologists except those with pediatric fellowship training, given that there are no validated PROMs for CRS in children. We also excluded otolaryngologists whose email addresses could not be found. Specialists from other subspecialties such as head and neck oncology, otology, facial plastic surgery, and laryngology who were contacted and responded that they do not use PROMs for CRS were excluded from the total number count as well.

Survey Data

We constructed a 32-item survey to evaluate respondents’ opinions on PROM. This included 9 demographic questions and questions about the clinician’s practice. Respondents were also asked for their perceptions and experiences using PROMs. Items included whether PROMs were collected, how frequently they were collected, and when. Five questions assessed perceptions of whether PROM helps patients report their symptoms, PROM improves the efficiency of the patient encounter, PROM is easy to understand, PROM improves management and monitoring clinical outcomes, or PROM is not helpful. Responses were on a 5-point rating scale (strongly disagree, disagree, neutral, agree, strongly agree). The complete survey used can be found in Appendix 1. Responses were collected via REDCap and analyzed using frequencies and chi-squared tests in the R statistical software (v4.2.3).

Results

Seventy-eight out of 346 (23%) Canadian otolaryngologists we contacted responded to the survey (26 with rhinology fellowship, 51 without rhinology fellowship, 1 missing data). Two of these survey responses were incomplete. Gender distribution was predominantly male and not significantly different by fellowship status (79% male with fellowship, 71% male without fellowship). Seventy-seven percent of fellowship-trained otolaryngologists saw rhinology patients as more than half of their clinical practice compared with only 12% of non-fellowship-trained otolaryngologists (P = .003). The percentage of otolaryngologists with over 15 years of practice was 31% in those with fellowships and 55% in those without (P = .06, Table 1).

Table 1.

Characteristics of Respondents With and Without Fellowship Training in Rhinology.

	Fellowship-trained (N = 26)	Non-fellowship-trained (N = 51)
Gender, male (%)	19 (79)	34 (71)
Percentage of practice seeing rhinology patients (%)
0%-25%	2 (8)	29 (57)
26%-50%	4 (15)	16 (31)
51%-75%	7 (27)	6 (12)
76%-100%	13 (50)	0 (0)
Years in practice (%)
Resident/fellow	0 (0)	2 (4)
0-5	5 (19)	8 (16)
6-10	4 (15)	9 (18)
11-15	9 (35)	4 (8)
16+	8 (31)	28 (55)
Collect PROM in practice, Yes (%)	18 (69%)	20 (39%)

Abbreviation: PROM, patient-related outcome measures.

Findings for the use of PROMs was mixed and varied by training and practice type. Specifically, only about half of otolaryngologists responded that they collect PROM from their patients (49%, n = 38). The frequency of PROM collecting was 69% in fellowship-trained and dropped to 39% in non-fellowship-trained otolaryngologists (P = .03, Table 1). This frequency was 77% in otolaryngologists who see rhinology patients as more than half of their practice, and only 35% in those who see rhinology patients as less than half of their practice (P < .001). Among the respondents who collected PROM, fewer than half reported they collected PROM at all clinical visits (43%), 41% collected PROM at pre- and postoperative visits, 11% at pre-operative visit only, and 5% at postoperative visit only. Forty-seven percent said they collected PROM at more than 75% of clinic visits. Fifty-three percent said they incorporated PROM into the care plan of their patients with CRS more than half of the time. Sixty-eight percent said patients completed the PROM given to them over 75% of the time (Figure 1).

Figure 1.

Frequency of PROM collecting by study respondents. PROM, patient-related outcome measures.

Given the finding that many respondents did not always collect PROM data, it was not surprising that perceptions about the value and helpfulness of PROM data were variable. Specifically, 74% of respondents agreed or strongly agreed it helps patients report their symptoms, while only 2 respondents (3%) disagreed (no one strongly disagreed). Fewer than half of respondents (42%) agreed or strongly agreed it improved the efficiency of the patient encounter [1 (1%) strongly disagreed], 54% agreed or strongly agreed it was easy for patients to understand [1 (1%) strongly disagreed], and 62% agreed or strongly agreed that it improved the management of patients and monitoring of clinical outcomes. Twenty-nine percent reported that PROM was not helpful, while 51% disagreed or strongly disagreed (Figure 2). When stratified by whether the respondents collected PROM from their patients, those who collected PROM were more likely to agree or strongly agree that PROM helped patients report their symptoms (OR 4.5, P = .01), improved the efficiency of the encounter (OR 4.5, P = .003), and helped monitor their patients’ symptoms (OR = 10.2, P < 10⁻⁴), and were more likely to disagree or strongly disagree that PROM was not helpful (OR = 3.8, P = .006). There were no significant differences in reports that PROM was easy to understand for patients (Table 2). Fellowship-trained otolaryngologists were 4 times more likely to say PROM helps them manage their patients and monitor outcomes (P = .01), and no other differences in other perceptions were significant.

Figure 2.

Opinions regarding PROM among all respondents. PROM, patient-related outcome measures.

Table 2.

Opinions Regarding PROM Depending on Whether Respondents Collect PROM.

	Collect PROM (N = 38)	Do not collect PROM (N = 39)	P value
PROM helps patients report their symptoms = Agree/Strongly agree (%)	33 (87)	23 (59)	.01
PROM improves the efficiency of the patient encounter = Agree/Strongly agree (%)	23 (61)	10 (26)	.003
PROM is easy for patients to understand = Agree/Strongly agree (%)	25 (66)	17 (44)	.07
PROM helps me manage patients and monitor outcomes = Agree/Strongly agree (%)	33 (87)	15 (38)	<.0001
PROM is not helpful = Disagree/Strongly disagree (%)	26 (68)	14 (36)	.006

Abbreviation: PROM, patient-related outcome measures.

The most frequently-identified barriers to PROM usage were lack of time for 67% of respondents, difficulty of integration into clinical workflow for 64% of respondents, and lack of integration into the electronic medical record for 47% of respondents. Other barriers, in decreasing order of importance, included length of PROM questionnaires, language barriers among patients, cost of the PROM questionnaires, and uncertainty of PROM’s role in making clinical decisions. Ten respondents (13%) identified additional barriers including lack of familiarity with PROM or its validity, a lack of understanding of PROM items among their patients, and patient preference to not use PROMs (Figure 3). If these barriers were addressed, 86% of respondents said they would use PROM in their practice. Five of the 10 respondents who said they would not use PROM regardless believed it was rarely helpful clinically, and one said it was more valuable as a research tool.

Figure 3.

Barriers to PROM usage identified by all respondents. PROM, patient-related outcome measures.

Discussion

The present study is the first to survey Canadian otolaryngologists’ opinions on PROM in the management of CRS. Survey respondents were diverse in gender, rhinology fellowship status, percentage of practice dedicated to rhinology, and overall years in practice. Overall, the findings were mixed in terms of usage of PROMs and perceptions of their value, with a number of study participants reporting numerous barriers to their usage in regular clinical practice. These findings are important to help identify potential changes that need to be made to the PROMs measures themselves, as well as how they might be better integrated into practice, especially the need to provide clinicians with data that show the value in using PROMs to improve clinical outcomes.

Our investigation revealed insights into the utility of PROM dependent on whether the respondent was rhinology fellowship–trained or used PROM in their practice. PROM was used by half of respondents with preference by those who had rhinology fellowship training or whose practice comprised mostly of rhinology patients. Unsurprisingly, perceptions about PROM were also more positive by respondents who used PROM in their practice, with regard to all items except that it is easy for patients to understand. This suggests that a physician’s decision to use PROM is less influenced by perceived barriers in patient understanding and more by how the physician can utilize it productively. Interestingly, the only difference in perceptions between otolaryngologists with or without fellowship training was that the former were 4 times more likely to say PROM helps them manage patients and monitor outcomes. This suggests that the utility of PROM may be dependent on the clinician’s experiences with managing patients using PROM during their training, or PROM may be more helpful in managing the higher complexity of patients with CRS referred to fellowship–trained rhinologists.

The role of PROM in CRS management in the tertiary setting has been well described by Gallo and colleagues in a 2020 study that emphasized the value of quantifying the severity of patients’ symptoms pre- and postoperatively to assess whether a specific treatment has been useful, using the minimal clinically-important difference of 8.9 for the SNOT-22. ²⁰ Data from their study as well as Rudmik and colleagues’ 2016 appropriateness criteria study demonstrate that sinus surgery should only be considered in patients with a SNOT-22 score greater than 20. ¹⁶ In addition, one should be cautious in performing surgery on patients who have a disconnect between SNOT-22 scores and lack of objective disease on CT or endoscopy. As most studies of PROM to date have been based in tertiary centers, studying its use in community settings may reveal objective data that challenge the current perceptions held by community otolaryngologists. Going forward, continuing medical education catered toward both practicing otolaryngologists and trainees with evidence-based recommendations and examples on how to efficiently integrate PROM into practice may continue to encourage its adoption.

Our study also revealed insights into Canadian otolaryngologists’ perceptions of the weaknesses and barriers to implementing PROM in practice. Previously-identified barriers to PROM implementation in practice include efficiency and convenience, and these were similarly identified by our respondents. ²¹ The biggest barrier identified by most respondents was a lack of time. To help address this issue, future research could identify the key visits at which PROM is most helpful to administer, for example, pre-op and post-op, to minimize the overall time spent in administration. The second biggest barrier was challenges in integrating PROM into clinical practice. This may be addressed by future work focused on simplifying the tools for online administration and programing PROM into electronic medical records. Other issues including unfamiliarity on the patients’ or physicians’ end of PROM, and its results may be topics to address in the future with continuing professional development initiatives. A few respondents believed PROM was more useful as a research tool than a clinical practice tool, and others identified the cost of using PROMs in clinical practice as a barrier. SNOT-22 is free to use for academic purposes, but costs $25 to $60 per use in clinical trials and $500/year per clinical group of 20 for clinical use. In the research setting, SNOT-22 has recently been used as a reliable outcome measure in investigating the efficacy of biologic therapies for CRS.^22-25 Given the high cost of biologic therapies, there is a growing need for reliable tools clinicians can use to select patients who are most likely to benefit.^26,27 More studies are needed to determine whether PROMs can fill this need as a cost-effective solution for patients who have failed maximal medical therapy and ESS.

Although SNOT-22 is currently the most-widely-used PROM in CRS, other PROMs have their own distinct advantages and disadvantages. The RSDI is another standardized, well-validated, CRS-specific tool with high correlation to SNOT-22.^28,29 It captures more sophisticated aspects of the patient’s physical, functional, and emotional experience, at the cost of increased survey length with 30 questions.^28,29 The CSS, using only 6 questions, captures CRS-related symptoms and data regarding recent medication use, but lacks information on taste or smell dysfunction. ²⁹ It complements data collected from the RSDI and is used in diagnostic criteria for CRS in the Academy of Otolaryngology-Head and Neck Surgery Task Force on Rhinosinusitis (RSTF). ²⁹ The CRS-PRO is another brief questionnaire with 12 items comprising “rhino-psychologic,” “facial discomfort,” and “cough” domains that were developed with extensive patient input.^30,31 It has demonstrated high validity in measuring clinical and biochemical treatment response to ESS in chronic rhinosinusitis with nasal polyposis (CRSwNP) and chronic rhinosinusitis without nasal polyposis (CRSsNP), and high correlation with other PROMs including SNOT-22.^30-32 Future studies can potentially investigate the efficiency of each PROM in clinical practice to encourage adaptation and optimization of PROMs in clinical practice.

Our study’s limitations include our low response rate of 23%. This is partially because, to reduce the risk of bias, we included all general and subspecialty otolaryngologists except those with subspecialty training in pediatrics, as there are no validated PROMs for use in children. ³³ However, this broad inclusion criteria included many subspecialty otolaryngologists who never see patients for CRS, and we received replies from 3 otolaryngologists that they would not be filling out the survey for this reason. On the other hand, because the surveys were filled out on a voluntary basis after mass emailing, sampling bias may have selected for individuals who use PROM already or alternatively, and who were not interested in PROMs and wanted to convey their barriers in the study. It is also important to remember that the survey represents the opinions of Canadian otolaryngologists and is not necessarily generalizable to other countries with different practice patterns, policies, or health service models. Unfortunately, we also were unable to explore our findings in detail to examine potential similarities and differences related to variables such as years of practice because of the small sample size.

Future surveys that include larger diverse sampling of otolaryngologists as well as other health service models, practices, and countries would be valuable to elucidate similarities and differences in PROM usage and perceptions compared with our study. Lastly, surveying the patients’ perceptions on the use of PROMs would be an important perspective to have for incorporating PROM in practice.

Conclusion

In conclusion, there was considerable variability in the uptake of PROM among otolaryngologists and in perceptions, with less use and poorer perceptions more common among those without rhinology fellowship. We identified barriers such as time and the need for greater understanding and recognition of the value of PROMs in clinical practice that, if addressed, may increase their use and applicability in clinical practice. As new resource-limited therapies such as biologics become more prevalent in CRS management, PROM may find more applications in shared clinical decision making.

Appendix 1

Survey Distributed to Canadian Otolaryngologists on REDCap

Assessing the use of patient-reported outcome measures in the routine clinical care of chronic rhinosinusitis patients

We need your help to improve the care of patients with Chronic rhinosinusitis! This survey will take less than 5 minutes.

Chronic rhinosinusitis (CRS) is a chronic inflammatory disease of the sinonasal disease and a common reason for referral to otolaryngologists. While many CRS-related consensus statements and CRS literature endorse the use of patient-reported outcome measures (PROM) such as SNOT-22, it is unclear whether clinicians perceive limitations or barriers in implementation.

Regardless of whether you are using PROM or not in your practice, we would like to hear from you. This survey will take less than 5 minutes.

We would like to understand (1) how often PROM is used in your practice for the management of CRS, (2) what the limitations/barriers of PROM would be in your practice.

Your participation is completely voluntary and declining to take part will have no professional impact on you, your practice, or your training. Completing the survey is considered implied consent. You may stop answering the survey at any time without any penalty.

Section 1—Use of PROM

Please indicate the degree to which you agree with each of the following statements related to the clinical use of standardized PROM (such as SNOT-22, RSDI).

PROM helps patient report their symptoms

• • Strongly Disagree

• • Disagree

• • Neutral

• • Agree

• • Strongly Agree

PROM improves the efficiency of the patient encounter

• • Strongly Disagree

• • Disagree

• • Neutral

• • Agree

• • Strongly Agree

PROM is easy to understand for patients

• • Strongly Disagree

• • Disagree

• • Neutral

• • Agree

• • Strongly Agree

PROM enables me to better manage patients and monitor my clinical outcomes

• • Strongly Disagree

• • Disagree

• • Neutral

• • Agree

• • Strongly Agree

I believe PROM is not helpful in clinical decision (i.e. History and physical examination alone is sufficient for my clinical assessment)

• • Strongly Disagree

• • Disagree

• • Neutral

• • Agree

• • Strongly Agree

In your clinical practice, do you collect patient-reported outcome measures (PROM) for patients with chronic rhinosinusitis? (e.g. SNOT-22, RSDI)

• • Never

• • Yes

What are the barriers in adopting CRS-related PROM (e.g. SNOT-22) in your practice? (Please check all that apply)

• • Uncertainty of whether PROM affects clinical decision/outcomes

• • Price of the Survey (e.g. $500/year for SNOT-22)

• • Lack of time

• • Difficulty with integration into the clinical flow

• • Lack of integration into Electronic Medical Record

• • Length of the survey

• • Patient language barrier

• • Other (specify):

If the barriers were addressed, would you start using PROM in your practice regularly?

• • Yes

• • No

Is there anything we did not ask about PROM?

Thank you for all your responses!

We want to remind you that your responses are confidential.

Now, we would like to collect background information for all participants to better understand the nature of this.

Section 2—Demographic information

How long have you been practising Otolaryngology-Head & Neck Surgery (OHNS)?

• • 0-5 years

• • 6-10 years

• • 11-15 years

• • 16 or more years

• • I am a resident physician or fellow

• • I would prefer not to answer

Please describe the primary appointment of your practice

• • Academic Centre

• • Community Centre

• • Combined practice between Academic and Community

• • Military

• • Other (please explain)

Please describe the nature of your practice

• • Solo office practice

• • Group office practice

• • Hospital-based practice

• • Other (describe):

How do you self-identify in terms of gender?

• • Man

• • Woman

• • I do not identify within the gender binary

• • I prefer not to answer

Do you have subspecialty training/fellowship in rhinology & skull base surgery?

• • Yes

• • No

Please estimate the percentage of your practice dedicated to rhinology.

• • 0%-25%

• • 26%-50%

• • 51%-75%

• • 76%-100%

Thank for taking the time to complete this survey

Any last comments?

Email Address