Implementing paediatric patient-reported outcome measures in outpatient asthma clinics: a feasibility assessment study

Abstract

Objective

Implementation of patient-reported outcome measures (PROMs) is limited in paediatric routine clinical care. The KidsPRO programme has been codesigned to facilitate the implementation of PROMs in paediatric healthcare settings. Therefore, this study (1) describes the development of innovative KidsPRO programme and (2) reports on the feasibility of implementing PedsQL (Pediatric Quality of Life Inventory) PROM in asthma clinics using the KidsPRO programme.

Design

Feasibility assessment study.

Setting

Outpatient paediatric asthma clinics in the city of Calgary, Canada.

Participants

Five paediatric patients, four family caregivers and three healthcare providers were recruited to pilot the implementation of PedsQL PROM using KidsPRO. Then, a survey was used to assess its feasibility among these study participants.

Main outcome measures

Participants’ understanding of using PROMs, the adequacy of support provided to them, the utility of using PROMs as part of their appointment, and their satisfaction with using PROMs.

Analyses

The quantitative data generated through closed-ended questions was analysed and represented in the form of bar charts for each category of study participants (ie, patients, their family caregivers and healthcare providers). The qualitative data generated through the open-ended questions were content analysed and categorised into themes.

Results

The experience of using PROMs was overwhelmingly positive among patients and their family caregivers, results were mixed among healthcare providers. Qualitative data collected through open-ended questions also complemented the quantitative findings.

Conclusion

The evidence from this study reveals that the implementation of PROMs in routine paediatric clinical care asthma clinics in Alberta is seems to be feasible.

STRENGTHS AND LIMITATIONS OF THIS STUDY

• This study aimed at assessing feasibility rather than establishing causation or correlation, thus despite the small sample size, this study generated evidence to support scale-up and spread of patient-reported outcome measures (PROMs) in routine clinical care.

• Engaging paediatric patients and their family caregivers as patient partners throughout the project is a fairly novel approach paediatric health services research, so it is a major strength of this study.

• Due to COVID-19 pandemic, most appointments were done virtually, and research activities were halted at the asthma clinics, which negatively impacted the implementation of PROMs within the asthma clinics and subsequently the conduct of this study.

• Since this study was conducted during the pandemic, some feasibility aspects might differ during non-pandemic times.

Introduction

Patient-reported outcome measures (PROMs) are increasingly being used to provide patient- and family-centred care.^{1 2} The use of PROMs in routine clinical care has shown to enhance communication between patients, family caregivers and healthcare providers, facilitating patient engagement and discussion on patient- and family-identified concerns, resulting in shared-decision making and attaining better health outcomes for patients.^3–6 Despite the growing use of PROMs in the adult population, their implementation in routine paediatric clinical care is still limited.^{7 8}

In Canada, PROMs are sporadically implemented in some paediatric healthcare settings.⁹ But their use is inconsistent and lacks system-wide implementation.¹⁰ Thus, this project aims to generate evidence on the feasibility of facilitating Alberta’s province-wide implementation of paediatric PROMs and patient-reported experience measures (PREMs) through electronic platforms. PROMs and PREMs are questionnaires that have been standardised and validated to help patients report on their health status and experience of care, respectively.¹¹ In general, PROMs can be categorised into generic and condition-specific measures. Generic PROMs measure health-related quality of life (HRQOL) and capture common aspects of health and impact of interventions or treatments on patients’ general quality of life.^{11 12} On the other hand, disease-specific PROMs serve the purpose of addressing particular disease symptoms that may impact health conditions and outcomes.¹² With our industry partners and patient partners, we codesigned KidsPRO, an innovative eHealth programme available to patients and family caregivers on mobile devices, tablets and desktop computers. Using unique login credentials, paediatric patients (8–18 years old) can complete self-reported PROMs, while their caregivers can complete proxy PROMs from home before visiting the healthcare facilities. PROM results are graphically presented and available to patients, family caregivers and their healthcare providers during the clinic consultation. Displayed results highlight the area requiring attention from healthcare providers. For patients coming for multiple visits, results could be presented in the form of a ‘trajectory’ of history of symptoms and care, allowing healthcare providers to track their patients’ progress using longitudinal data. Figure 1 shows the KidsPRO programme’s ability to administer PROMs, analyse their results and support patient–provider communication.

Figure 1.

The functioning of the KidsPRO programme.

Before codeveloping the KidsPRO programme, our team conducted a systematic review of paediatric PROMs used and evaluated worldwide.¹³ This systematic review identified Pediatric Quality of Life Inventory (PedsQL)¹⁴ as the most commonly used and evaluated PROM in paediatric clinical care around the world. Thus, PedsQL and the accompanying asthma-specific module were built into the KidsPRO programme. Before scaling up and spreading the implementation of PROMs in routine paediatric clinical care through the KidsPRO programme, it is important to evaluate the feasibility of such an intervention among its users to ensure research is translated into practice. Asthma is the most common chronic condition among the paediatric population,¹⁵ and up to 20% children, youth in Canada have asthma.¹⁶ Thus, this study aimed to investigate whether the implementation of PROMs is feasible in routine asthma clinics.

Methods

Study design and ethics

This feasibility assessment was conducted between January 2021 and May 2022. The Conjoint Health Research Ethics Board approved this study at the University of Calgary (REB18-0564). Alberta Health Services also provided the administrative approval for this study.

Study setting

This study was conducted at the outpatient paediatric asthma clinics in the community and the Alberta Children’s Hospital (ACH). ACH, located in Calgary, is a teaching hospital affiliated with the University of Calgary and is the only tertiary-level paediatric hospital in Calgary. Typically, asthma clinics run 3 days per week, and asthma care is provided by respirologists, paediatricians, nurses and allied health professionals, including social workers, medical psychologists, and technicians at the pulmonary function test laboratories.

Participant recruitment

Outpatient asthma clinics regularly mail out appointment reminders to the patients and their family caregivers, so a letter and a card about this study were added to the mailing envelopes. The letter and card included more information about the current study and had a link asking potential participants to fill out an online ‘consent to contact’ questionnaire with their relevant contact information. The study coordinator (SB) then called these participants (ie, family caregivers) to verify and enrol those who were eligible to participate in the study. Eligibility criteria included a child between 8 and 18 years old with a confirmed diagnosis of asthma and who had an upcoming appointment at the outpatient asthma clinics. After confirming the eligibility, a virtual meeting was set up with family caregivers and patients to complete consent forms and explain the steps involved with participating in this study. Healthcare providers of these enrolled study participants were contacted by email to recruit them in the study.

Intervention

PedsQL¹⁴ and its asthma module were chosen as the PROMs for this feasibility study. PedsQL uses a modular approach to measuring HRQOL in healthy children and adolescents, as well as those with acute and chronic health conditions. Table 1 provides more description of the PedsQL questionnaire. A user guide for the users of PROMs was developed in three versions—patient, family caregiver and healthcare providers. The patient and their family caregiver’s version of the user guide contained information about PROMs, why they are used in healthcare, steps to complete PROMs through the KidsPRO programme, and what the PROMs result means for the patient. Similarly, the healthcare provider’s version of the user guide contained information about PROMs, why PROMs are used in healthcare, how to interpret the PROM results, and case studies on how PROM results can help deliver patient- and family-centred care.

Table 1.

Description of PedsQL questionnaires

Measure	Description
Pediatric Quality of Life Inventory (PedsQL) 4.0 Generic Measure23	A 23-item generic score scale measures HRQOL in healthy children and adolescents and those with acute and chronic health conditions. It consists of four domains: Physical functioning Emotional functioning Social functioning School functioning
Pediatric Quality of Life Inventory (PedsQL) 3.0 Asthma Module24	Asthma specific 28 items score scale to complement the generic core scale. It consists of four domains: Asthma Treatment Worry Communication

HRQOL, health-related quality of life.

Survey instrument

Three versions of a quantitative survey were developed to assess the feasibility of using PROMs among (1) patients, (2) their family caregivers and (3) healthcare providers. While the patient and family caregiver versions included eight close-ended questions with five Likert Scale response options and two open-ended text questions, the healthcare provider version had nine close-ended questions with five Likert Scale response options and two open-ended text questions. The options for the 5-point Likert Scale questions were ‘strongly agree’, ‘agree’, ‘neither agree nor disagree’, ‘disagree’ and ‘strongly disagree.’ The survey was developed and administered through Qualtrics (Qualtrics, Provo, Utah, USA). It included questions on participants’ understanding of using PROMs, the adequacy of support provided to them, the utility of using PROMs as part of their appointment, and their satisfaction with using PROMs. All the versions of the survey are provided in online supplemental appendix 1.

Patient and public involvement

As a patient-oriented research study, involving patients and their family caregivers throughout the study is important to understand the health service user’s perspective. Therefore, we engaged five patient-caregiver pairs to act as ‘patient partners’ for our project. Patient partners were paediatric patients aged 8–18 who receive routine clinic care for asthma at the ACH. They were consulted while developing study participant recruitment material, codesigning and testing the KidsPRO programme including choosing appropriate PROM to be included in KidsPRO programme, and developing patients and family caregivers’ versions of the survey instrument.

Study procedure

First, all the enrolled patients and their family caregivers received an email from the KidsPRO programme a few days before their appointment. This email contained a link to a web page, which included their individually assigned PROM questionnaires. User guides for patients and their family caregivers were also sent by email. Each patient and their family caregivers were asked to complete one generic PedsQL questionnaire and one asthma-specific module of the PedsQL questionnaire. Once participants completed these surveys, the KidsPRO system automatically generated results in the form of graphs showing the scores in each domain of the PedsQL questionnaire. These results were then shared with the patients’ healthcare providers before the patient appointments. The healthcare providers were also provided with their version of the user guide. Following each appointment, all the study participants were sent an email with the link to complete the feasibility surveys on Qualtrics.

Data analysis

All the raw and summary data were exported into MS Excel for analysis. The closed-ended survey questions generated the quantitative data. These quantitative data were analysed and represented in the form of bar charts for each category of study participants (ie, patients, their family caregivers and healthcare providers). The qualitative data generated through the open-ended questions were content analysed and categorised into themes.

Results

A total of 150 letters and cards were sent via postage mail-outs to recruit patients and their family caregivers, with 19 family caregivers completing the consent to contact form, resulting in a response rate of 12.7%. Two respondents did not provide any contact information. The remaining 17 respondents were first sent an email asking them about their eligibility; a follow-up phone call was then made to respondents who did not reply to the emails. Four respondents did not respond to emails or calls. Three respondents had already passed their appointment times, rendering their participation ineligible, and six did not meet the eligibility criteria. In total, five patients, four family caregivers and their three healthcare providers were recruited in the study. Figure 2 shows the participant recruitment chart.

Figure 2.

Participant recruitment flow chart.

Quantitative data

Figure 3 demonstrates the views on the use of PROMs in outpatient asthma clinics among the patients. Sixty per cent (n=3) of the patients who completed PROMs expressed strong agreement with the statements about having an understanding of why they were completing PROM questionnaires. Similarly, 40% (n=2) strongly agreed with statements about the usefulness of the support they were provided to complete the PROMs, including the user guide. Sixty per cent (n=3) of the patients strongly agreed with the statement that PROMs helped them relay their health concerns better to their healthcare professionals. While there were mixed responses to the question about discussing results with their family caregivers, 60% (n=3) of the patients agreed that PROMs facilitated discussions with their healthcare providers during their appointment. Most patients (80%, n=4) felt that PROMs helped them discuss new concerns and would like to use PROMs in the future. Overall, the patient experience was positive, with 60% (n=3) of them strongly agreeing and 40% (n=2) somewhat agreeing with the statement on overall satisfaction in using PROMs.

Figure 3.

Patients’ views on the use of PROMs in outpatient paediatric asthma clinic (n = 5). PROM, patient-reported outcome measure.

Figure 4 shows family caregivers’ views on using PROMs in paediatric outpatient asthma care. Their perception was highly positive, with all of them either equally agreeing or strongly agreeing with most of the statements. The response to the question on willingness to use PROMs in the future was mixed, with 50% (n=2) of the family caregivers strongly agreeing, 25% (n=1) agreeing and 25% (n=1) neither agreeing nor disagreeing with that statement.

Figure 4.

Family caregiver’s views on the use of PROMs in outpatient paediatric asthma clinic (n = 4). PROM, patient-reported outcome measure.

Figure 5 depicts healthcare providers’ views on PROMs as part of their clinical practice in the outpatient paediatric asthma clinics. Overall, healthcare providers’ responses regarding the use of PROMs were not as positive as patients or family caregivers. The majority (66%, n=2) of the healthcare providers understood why PROMs are completed and why they are provided with the results, but their responses were mixed for most of the other questions. Sixty-six per cent (n=2) neither agreed nor disagreed with the statements on PROMs facilitating discussion in the appointment, and overall satisfaction with using PROMs. Thirty-three per cent (n=1) of the respondents did not get the opportunity to see the PROM results before the appointment with their patients.

Figure 5.

Healthcare provider’s views on the use of PROMs in outpatient paediatric asthma clinic (n = 3). PROM, patient-reported outcome measure.

Qualitative data

While answering the question on what participants liked about the use of PROMs, patients responded that the PROM questionnaires were easy to understand, helped them make self-assessments and assisted in noticing any changes.

I was able to personally assess myself and notice changes. (Patient—01)

In response to a similar question, family caregivers stressed the importance of using PROMs to understand their child’s situation better and help them manage their asthma.

It brought up questions I could ask my child. It also brought up things that I should watch for. It made me more aware of what my child was going through. (Family Caregiver—04)

According to the healthcare providers, the most helpful aspects of using PROM results in their practice included having additional information about the family and their understanding of self-management. One respondent highlighted that PROMs helped their patients think about their asthma and express what mattered to them.

So helpful to have the child think about their asthma prior to the visit and tell me what matters to them. (Healthcare Provider—03)

The last open text question asked participants for suggestions on how to improve the use of PROMs in asthma clinical care. Most of the patients did not provide any specific suggestions, but one respondent requested more videos for supplemental information.

I would like to watch videos for extra information. (Patient—02)

While most family caregivers did not provide any specific suggestions on improving the use of PROMs, similar to the one patient who responded, one family caregiver shared that providing more videos would be helpful for additional learning and information.

Provide videos for extra learning information. (Family Caregiver—02)

Lastly, healthcare providers suggested that making the process more timely and user-friendly could improve the use of PROMs in asthma clinical care. Additionally, one respondent suggested providing an overview of the PROM results on one page.

A simple overview in point form or diagrams, preferably a single page. (Healthcare provider—01)

In summary, the quantitative and qualitative data collected through the survey questionnaire complement each other. Overall, our study participants responded positively to the use of PROMs in the outpatient paediatric asthma clinics. None of the participants showed strong disagreement with any of the statements. These findings show that the implementation of PROMs is acceptable and feasible in the outpatient paediatric asthma clinics in Calgary.

Discussion

The broader implementation of PROMs in routine paediatric clinical care first warrants a feasibility assessment on a small scale. The current study conducted a feasibility assessment of implementing PROMs at Calgary’s outpatient paediatric asthma clinics. This study recruited 12 participants who used PROMs in the roles of asthma clinic patients, family caregivers and healthcare providers. While the experience of using PROMs was overwhelmingly positive among patients and their family caregivers, quantitative results were mixed among healthcare providers. Overall, the qualitative findings captured through the open-ended survey questions were positive and supportive of the continued use of PROMs by all three types of study participants.

Overall, higher levels of agreement among patients, family caregivers and healthcare providers on why they completed the PROMs underline the importance of educating these three groups about patient-reported measures like PROM. Some recent studies have shown that using PROMs have the potential to transform current model of clinically focused and utilitarian model of patient care.^{6 17} Similarly, higher levels of agreement on the helpfulness of the user guides show that health systems need to create resources like user guides for patients, family caregivers and healthcare providers. Creating training and information resources are also considered as one of the important strategies under implementation science approach.¹⁸

Although there were no negative responses or comments, it was interesting to observe the difference between patient and family caregivers’ views on using PROMs compared with those of healthcare providers. It is evident that patients and family caregivers are keen on using PROMs, while healthcare providers expressed some logistical concerns. One of the potential reasons for this difference could be the lack of clear processes and resources for healthcare providers to use PROMs as part of their regular practice. Another mixed-methods study conducted by our team has also identified similar challenges faced by the healthcare providers implementing PROMs and PREMs in Alberta.¹⁰ Moreover, our study was conducted during the COVID-19 pandemic, which may have contributed to already increased levels of stress and burden among healthcare providers, thereby influencing their responses in our study. The concerns raised by healthcare providers in our study might also be felt by other healthcare providers. Therefore, healthcare organisations like Alberta Health Services (AHS) should ensure that all healthcare providers are adequately supported before implementing PROMs.

Patients prefer completion of PROMs at home ahead of the appointments because completing PROMs from home helps increase privacy and reduce time spent at the clinic.^19–21 In this feasibility study, patients and family members were asked to complete the PROMs at home, which might have contributed to the better use and positive views on the use of PROMs. One of the major barriers to administering at-home PROMs is the lack of electronic platforms to collect such data in advance.²¹ Historically, PROMs have been administered in paper format, but the use of electronic programmes to implement PROMs is on the rise.²² One of the studies assessing the feasibility of integrating electronic PROMs shows that it is feasible, but it depends on close collaborations with clinicians and Information Technology (IT) support systems to create an efficient and easy-to-use platform.²¹ In the province of Alberta, AHS has recently implemented Connect Care, a province-wide electronic medical records system. Connect Care has the capability to support the integration of PROMs, but currently, it does not facilitate the integration of PROMs in routine paediatric clinical care in Alberta. Thus, the findings from this study demonstrating the feasibility of administering paediatric PROMs through the KidsPRO programme could inform the implementation of PROMs in other paediatric health settings in Alberta.

Strengths and limitations

The original recruitment strategy for this study was to recruit participants during their in-person clinic appointments, which would have helped researchers to better explain this study to the potential study participants, resulting in higher enrolment rates. However, similar to most healthcare settings, the COVID-19 pandemic negatively impacted implementation of PROMs within the asthma clinic and subsequently the conduct of this study. Due to the pandemic, most appointments were made virtually, and in-person research activities were halted at the asthma clinics. In such situation, the only option available was to include the recruitment invitation cards inside the regular appointment reminder and health promotion material they receive from local health system. This means, we were not able to explain the study to potential participants. It could also be possible that many patients have other mechanisms to remember their appointment, hence they do not even check mailed material thus missing the opportunity to participate in this study. Moreover, the recruitment rates for mail-based recruitment strategies are typically lower. We believe that these factors contributed to the lower response and recruitment rates for this study. Therefore, the results of this feasibility study should be interpreted with caution for several reasons. First, this study was conducted during the pandemic so that some feasibility aspects might differ during ‘normal’ times. Second, the PROMs were only used for one appointment; more frequent use of PROMs might change the views among its users. Lastly, the sample size of this study might have been too small to comprehensively capture views of a more representative population of patients, family caregivers and healthcare providers.

Despite the small sample size, the objective of this study was to determine the feasibility of the use of PROMs, and there was no intent to establish causation or correlation between the use of PROMs and other variables. Thus, the findings of this study remain helpful to healthcare systems and other stakeholders in understanding the implementation of interventions like paediatric PROMs in outpatient clinics. Ideally, future studies should be conducted during ‘normal’ times (eg, non-pandemic type situations) include a more diverse and representative sample of participants, and use PROMs in multiple clinical encounters to elucidate differences in experience over more extended periods.

Conclusion

This study has demonstrated that using PROMs in routine paediatric clinical care in asthma clinics is feasible. The study highlights a higher degree of agreement with the statements about the use of PROMs and acceptance of PROMs among patients and their family caregivers compared with the healthcare providers. The strategies used in this feasibility study, such as electronic administration of PROMs and providing user guides to patients, family caregivers and healthcare providers, can be leveraged by AHS. These findings will help AHS scale up the use of PROMs in paediatric healthcare settings in Alberta.

Data availability statement

Data are available upon reasonable request. The data that support the findings of this study are available from the corresponding author, SB, upon reasonable request.

Ethics statements

Patient consent for publication

Not applicable.

Ethics approval

This study was performed in line with the principles of the Declaration of Helsinki. Approval was granted by the Conjoint Health Research Ethics Board of the University of Calgary (REB18-0564). Participants gave informed consent to participate in the study before taking part.