Does the Patient-Reported Apnea Questionnaire (PRAQ) increase patient-centredness in the daily practice of sleep centres? a mixed-methods study

Abstract

Objectives

The objective of this exploratory study was to see how the Patient-Reported Apnea Questionnaire (PRAQ) may impact the daily clinical practice of sleep centres, and why it may or may not work as expected. The hypotheses were tested that this patient-reported outcome measure makes patients more aware of which of their health complaints may be related to obstructive sleep apnoea (OSA), and that it improves patient-centredness of care by shifting the focus of care away from (only) medical problems towards the individual burden of disease and quality of life.

Design

Mixed methods. The quantitative study (surveys, patient records) was a before-and-after study.

Setting

Three sleep centres in The Netherlands (secondary care).

Participants

27 patients and 14 healthcare professionals were interviewed. 487 patients completed surveys pre-implementation, and 377 patients completed surveys post-implementation of the PRAQ. For the health records, 125 patients were included in the pre-implementation group, and 124 other patients in the post-implementation group.

Interventions

The PRAQ was used in clinical practice for six successive months.

Outcome measures

Scores on individual survey items, number of patients receiving non-medical treatment, adjustment of treatment at first follow-up, compliance with treatment.

Results

Patients were generally positive about the usefulness of the PRAQ before and during the consultation, as they felt more informed. Healthcare providers did not consider the PRAQ very useful, and they reported minor impact on their consultations. The surveys and health record study did not show an impact of the PRAQ on clinical practice.

Conclusions

Implementing the PRAQ may be beneficial to patients, but this study does not show much impact with regard to patient-centredness of care. New Dutch guidelines for OSA care may lead to a greater emphasis on quality of life and value of care for patients, making its integration in clinical care potentially more useful.

Strengths and limitations of this study.

• The mixed methods approach of this exploratory study is its major strength: the study provides insight into the reasons why the Patient-Reported Apnea Questionnaire (PRAQ) does not work as intended.

• The patient survey may not have been discriminative enough to show differences between the groups pre-implementation and post-implementation of the PRAQ.

• Electronic health records were only studied in one of the included sleep centres – however taking into account the interview results we do not expect different results in the other centres.

• The PRAQ was in practice not used for follow-up consultations as often as intended, making evaluation of its use in this setting less robust.

Introduction

The integration of patient-reported outcome measures (PROMs) in clinical practice has been gaining popularity in the past decade.^1–3 PROM data collected in clinical practice can be aggregated and used for quality improvement purposes, or individual scores can be used in daily clinical practice to improve patient care. In this latter function PROMs can be used in different ways, for example, as a screening tool, a monitoring or evaluation tool, a tool to inform and empower patients and/or to increase the patient-centredness of care by shifting the focus of care away from (only) medical problems towards the problems patients experience in their daily life.⁴ When using PROMs in daily clinical practice, it may be sensible to combine the use of a PROM on an individual patient level with application on an aggregate level.⁵ There have been a number of studies that aimed to evaluate the usefulness of PROMs in clinical practice in a variety of settings, of which the results are mixed.^6–8 Though qualitative research on this topic has been synthesised in a recent review^{4 9} including a list of hypotheses on how PROMs might work, there are still many questions regarding which PROMs can be potentially useful in which settings.

This study is focused on the application of individual PROM scores in sleep centres which diagnose and treat patients with obstructive sleep apnoea (OSA), a condition for which a PROM could be a useful tool to improve patient-centredness of care. OSA is a highly prevalent but often unrecognised condition in which frequent collapse of the upper airway causes breathing stops while asleep. The subsequent arousals can result in severe sleepiness and fatigue during the day, often affecting a patient’s cognitive function, psychological well-being, relationships and ability to work.^10–12 OSA has also been shown to be an independent risk factor for hypertension, heart failure and diabetes.^13–15 The prevalence of OSA has been reported to be 6% to 38%, depending on the exact definition of OSA and the population studied, and is higher in men.¹⁶

Severity of OSA and necessity for treatment has historically been based on the number of (partial) breathing stops per hour: the Apnoea-Hypopnea Index (AHI).^{17 18} However, there is no linear association between AHI and severity of symptoms or the presence of comorbidities.^19–23 There is also little evidence that treating patients with mild OSA (based on AHI) or patients with low sleepiness is useful in preventing cardiovascular disease or incidents.^24–27 In the past few years there has therefore been international discussion regarding new approaches to diagnose ‘clinically relevant’ OSA.^{28 29} This discussion has also made its way into recent Dutch guidelines for OSA, in which it is recommended that there should be a greater focus on the presence of potentially related comorbidities, as well as the experienced burden of disease for individual patients. The goal of treatment is the improvement of these aspects of OSA.³⁰

We have developed and validated a PROM for use in clinical practice which may aid this new focus of care for patients with OSA: the Patient-Reported Apnea Questionnaire (PRAQ),^{31 32} which measures OSA-related quality of life. The goal of this PROM is to improve patient-centredness of care on an individual level by shifting the conversation away from the medical problems and towards and individual’s burden of disease/quality of life, and also to measure quality of care on an aggregate level. To develop the PRAQ, the input from patients and healthcare professionals was used to select the topics that were considered most important to discuss in clinical practice.³¹ The individual PRAQ scores of each patient with (suspected) OSA are captured in the ‘PRAQ-report’, which was designed together with patients and uses coloured smileys to show the results for the 10 domains of the PRAQ. The advantage of the PRAQ compared with other commonly used PROMs in the care for patients with OSA (such as the Epworth Sleepiness Scale, Functional Outcomes of Sleep Questionnaire, etc) is that it provides a comprehensive overview of the possibly impacted aspects of quality of life that patients with OSA may experience. It is therefore potentially suitable for shifting the focus of care away from (only) medical problems towards the problems patients experience in their daily life.

This explorative study aims to study the impact of the PRAQ and PRAQ-report on the clinical practice of OSA, and explore why the PRAQ did or did not have an impact. A combination of both qualitative and quantitative methods is used that will add to the general knowledge on the circumstances under which PROMs do or do not work in clinical practice.

Methods

This article describes an exploratory mixed methods study in which the PRAQ is implemented in the clinical practice of three sleep centres. Qualitative interviews and a patient survey were used to explore patients’ and healthcare providers’ experiences with the PRAQ, and to identify potential barriers and facilitators to its use. Additionally, data were collected from electronic health records to study whether the hypotheses about the potential impact of the PRAQ mentioned in the introduction are correct. For the patient survey and the patient record study we conducted a before-and-after study. The different methods are described in more detail in the next sections.

Hypotheses

We have several hypotheses regarding how the PRAQ may influence patients and healthcare professionals, and how this could impact clinical practice. First of all, completing the PRAQ could:

• Encourage patients to consider which problems they experience that might be related to OSA and that they might want to discuss.

• Aid healthcare professionals in opening a conversation about an individual patient’s burden of disease (apnoea-related quality of life).

• Aid healthcare professionals to evaluate treatment and identify problems that are still present.

We think that this may potentially lead to:

• Higher patient compliance with treatment.

• More explicit choices regarding whether clinical treatment for OSA is (potentially) beneficial to the patient.

• An increase in referrals to other healthcare providers, such as psychologists.

• More ‘holistic’ care, in which there is increased attention for the well-being of patients, including the psychological and social effects of OSA and its comorbidities.

The PRAQ and its implementation

The PRAQ and its complementary PRAQ-report were designed with the input of patients with OSA and healthcare professionals.³¹ The questions of the PRAQ can be found in online supplementary file 1. The PRAQ takes approximately 15 min to complete.³¹ More information about the PRAQ-report and how the PRAQ was implemented in clinical practice can be found in online supplementary file 2.

Setting and subjects

Sleep centres of three Dutch hospitals took part in the study. The PRAQ was part of the clinical practice routine of these centres for six successive months. The PRAQ was distributed to patients attending an intake consultation for possible OSA (which takes place after a patient’s diagnostic sleep study), and subsequently to the subselection of these intake patients diagnosed with OSA who returned for a follow-up consultation after starting treatment.

Interviews

In-depth, semi-structured interviews were conducted with patients and healthcare professionals in the last 2 months of the study. The interview guides contained broad, open questions as well as more specific questions informed by topics previously identified in the literature.⁴ For patients the main goal was to assess whether completing the PRAQ was acceptable to them, and to find out the impact that the PRAQ and the PRAQ-report had for them on the (preparation for) the consultation. For healthcare providers, questions were mostly focused on how they used the PRAQ and why they used it this way, and the impact the use of the PRAQ has on their practice. This information can provide the basis for interpreting the results of the electronic health record study.

Patients were invited via email by the sleep centre before their scheduled consultation, or by their healthcare professional directly after their consultation (for more information see online supplementary file 3). Only patients who had completed the PRAQ were invited. We interviewed 27 patients. Data saturation was reached. Characteristics of the interviewed patients and of the interviews can be found in table 1.

Table 1.

Characteristics of the interviewed patients and the interviews

Patient characteristics (n=27)
Age (mean, range)	59 (31–82)
Gender (male)	18
Highest education level (range)	Primary school - PhD

Interview characteristics (n=27)
Interview after intake consultation (n)	18
Interview after follow-up consultation (n)	9
Interview together with partner or other relative that attended the consultation	4
Patients who had not seen the PRAQ-report at the time of the interview*	5

*Viewing the PRAQ-report before the consultation was optional, and not all healthcare providers showed the report to the patient during the consultation. PRAQ, Patient-Reported Apnea Questionnaire.

All healthcare professionals of the three participating sleep centres that had had the option to work with the PRAQ were invited to participate. This resulted in interviews with 14 healthcare professionals: six pulmonologists, six physician assistants (PAs) and two nurses. Two pulmonologists refused an interview because they had not seen many patients for OSA, two others because they had not used the PRAQ at all and one PA refused for personal reasons. At least four healthcare professionals were interviewed at each of the three sleep centres.

More information on the (analysis of) the interviews can be found in online supplementary file 3.

Surveys

The patient survey was designed for this study to study potential differences in patient empowerment and patient-centredness of care before and after the implementation of the PRAQ. The items of the survey covered how prepared patients felt for their consultation, whether there was discussion of the health problems that patients consider relevant during the consultation and whether patients were motivated to start their treatment. Patients could indicate their agreement on several statements on these topics with the statement on a 7-point Likert scale. The survey was checked by the members of the research team, which included a patient, but was not pilot tested. A translated version of the survey can be found in online supplementary file 4.

Surveys were distributed by healthcare professionals to all of their patients attending either an intake or first follow-up consultation for (suspected) OSA. Distribution of the surveys took place in the 2 months before implementation of the intervention (control group), and in the last 2 months of the 6 months that the intervention was part of daily clinical practice (intervention group). For the intervention group, the survey also contained additional questions about the patient’s opinion on the usefulness of the PRAQ. Participation was voluntary and anonymous.

Electronic health records

Electronic health records from one of the included sleep centres were studied to explore potential changes in treatment and compliance with treatment resulting from the use of the PRAQ. Data were collected from patients with an AHI ≥5 attending an intake consultation during the final 2 months of the study period and during the same time period the previous year. Information was collected about treatment choice at intake, treatment adaptations and compliance with treatment at the first follow-up consultation and patient characteristics. Compliance data is only available for patients who receive continuous positive airway pressure (CPAP), the most commonly prescribed treatment for patients with OSA. As part of standard care, hours of use are registered by the CPAP device and entered into the health record at follow-up consultations. CPAP compliance is expressed as average hours of CPAP use/night in the month before the follow-up consultation, with an average of 4 hours/night generally being the minimum to be considered compliant.³³

No identifying information was collected from the health records. The data collection procedure guaranteed that the records would at all times remain anonymous to the researchers.

Statistical analyses

Mann-Whitney U tests were conducted for each of the survey items that patients were asked to complete both pre-implementation and post-implementation of the PRAQ. For the electronic health record study, treatment choice at intake was studied by aggregating the choice into two variables: medical treatment of OSA (eg, CPAP, mandibular repositioning device, referral for surgery) and no or non-medical treatment (eg, lifestyle advice), as these are the variables which we potentially expected the PRAQ to influence. A χ² test was used to test for statistical significance. For the follow-up variables of the patient record study, χ² tests (for dichotomous variables) and an independent samples t-test (for CPAP compliance in minutes) were conducted.

No correction for multiple testing was performed because this is an exploratory study. A p value of <0.05 was therefore taken as a significant difference, which can be interpreted as an indication that this is a potentially interesting variable for a possible future study.

Patient and public involvement

A board member (author MI) of the Dutch patient organisation for OSA (ApneuVereniging) was involved with this study from its inception, including the research question and outcome measures and interpretation of the results. This author was also closely involved in the development of the intervention itself (the PRAQ and its complementary PRAQ-report), as were other members of the patient organisation.³¹ They also approved of the burden and time required for the intervention. Patients were not involved in the recruitment for the study.

Results

Interviews

Patient perspective

Patients were generally willing to complete the PRAQ before their consultation, and patient response as reported by the healthcare professionals was high. About half of the interviewed patients indicated that completing the PRAQ helped them prepare for their intake consultation by giving them more insight into their complaints and functioning and how this might relate to OSA, and/or made them consider what they wanted to discuss with the healthcare professional. Many patients completed the PRAQ with a family member which instigated discussions patients often considered useful. A great majority of interviewed patients indicated that they did not mind taking the time to complete the PRAQ, and many also considered the smileys of the PRAQ-report a clear and easy way of communicating the results. Box 1 contains quotes illustrating the statements in this paragraph.

Box 1.

‘Look, it’s just very insightful. You can see instantly where the problems are and on this other (page) you can see what the improvements are. Yes, it’s kinda nice.’ (Centre 3, patient 10)

‘Yes, you know I do find it useful, because you have so many… so many things that bother you, that you forget what it is that bothers you. Or because it has become part of you, so to say. So yeah in order (not) to forget things, a questionnaire like this comes in handy.’ (Centre 2, patient 1)

‘But there were quite a lot of questions where I was like, oh, sometimes I’m like, how does that fit with (apnoea)? But most did, but there were questions where I was like, is that related to sleep apnoea? So. Yes. Apparently.’ (Centre 3, patient 7)

‘Actually I liked (seeing it beforehand), because this way I can by myself… otherwise I would have gone into it timidly like, tell me, what did you see? And now I could ask specific questions.’ (Centre 3, patient 2)

The interviews also revealed some unintended effects of the PRAQ. A majority of patients assumed that the main purpose of the PRAQ was to aid their healthcare professional in setting a diagnosis, by providing information about symptoms ahead of time. A few patients believed that discussion of patient complaints during the consultation was therefore no longer necessary after completing the PRAQ (box 2), while healthcare professionals consider this discussion very important (see next section). What may have played a role here is that several interviewed patients seemed eager to hear their sleep study results, rather than (first) spend much time talking about their symptoms or problems.

Box 2.

‘I think it’s very good, because you can from the beginning very clearly indicate your problems. So it doesn’t need to all be done during the short conversation you have with the specialist. (…) It’s clear it doesn’t need to be mentioned again, because it’s clear to her as well what the problems are.’ (Centre 1, patient 4)

‘Just that when you complete a questionnaire aimed at establishing something, then it’s useful that you also get a sort of result. So a preliminary… not that you should instantly think like nothing is wrong, nothing needs to be done, let’s get out of here. But, I did like it, yeah.’ (Centre 1, patient 3)

‘Well, because there were only two orange (smileys), and the others were all green and then you think, well…. And then when you look at it again then I’m like, ‘I can live with that’.’ (Centre 2, patient 7)

Additionally, there were some issues around the interpretation of the smileys in the PRAQ-report. Several of the interviewed patients did not seem to view the PRAQ-report as merely a visualisation of the answers they had given, but rather as a ‘test result’. Some considered the number of ‘unhappy’ smileys as an indication of whether they were doing well or not, which made some patients reconsider the severity of their complaints (box 2).

Healthcare professional perspective

Most healthcare professionals used the PRAQ during consultations (table 2), but usually briefly. Several professionals mentioned that, especially during intake consultations, they used it for the sake of the study. Only a few tried to provide more holistic care with the PRAQ. Some professionals stated that their minimal use of the PRAQ was due to unwillingness to change their practice, while others mentioned a general aversion to questionnaires, and/or not being convinced that the PRAQ would offer new or useful information considering what was already discussed during a regular consultation. There were also practical issues that to some extent hindered the uptake of the PRAQ: most notably the (limited) time available for consultations, and the fact that the PRAQ was not embedded in the electronic health records which hindered the regular workflow. There were no notable differences in attitude towards the PRAQ between physicians, PAs and nurses.

Table 2.

Use of PRAQ-report by interviewed healthcare professionals

Use of the PRAQ-report during intake consultations
Discussed it with patients	Only looked it up	Did not look at it	N/A*
8	1	3	2

Use of the PRAQ-report during follow-up consultations
Discussed it with patients	Only looked it up	Did not look at it	Want to use it†	N/A*
3	1	3	6	1

*Not all healthcare professionals held both intake and follow-up consultations.

†Did not see (many) patients with follow-up PRAQ but are interested in using it in this setting.

N/A, not applicable; PRAQ, Patient-Reported Apnea Questionnaire.

Most of the professionals that used the PRAQ did so at the end of their usual discussion of symptoms, to check whether all topics that were problematic had been discussed and potentially address more topics. As such they could still start the conversation in their usual way, allowing patients to explain their problems in their own words and allowing the healthcare professionals to ask their standard diagnostic questions. Professionals indicated that most ‘symptoms’ that are part of the PRAQ were already part of the standard diagnostic questions during an intake consultation (sleepiness, problems at night), and also overlapped with their usual (diagnostic) intake questionnaire. However, several professionals mentioned that the PRAQ-report increased discussion of the topic ‘health concerns’, which was considered valuable. Furthermore, the few professionals that indicated that they valued offering more holistic care noticed that the PRAQ was useful in drawing the conversation away from medical facts and more towards the underlying emotions related to a patient’s problems. However, many other professionals did not see much added value in actively bringing up topics like emotions and social interactions. They were potentially willing to discuss these issues but considered it up to the patient to raise them. If the PRAQ was used to identify problems, it was more common for the professional to mention very briefly that these problems were likely to improve with treatment of OSA, without further discussing these problems. Professionals reported that they did not notice any increase in OSA-related knowledge in their patients, or a difference in whether or how patients raised health complaints or quality of life issues of their own accord. Box 3 contains quotes illustrating the statements in this paragraph.

Box 3.

‘Well I myself don’t ask ‘are you worried about your (health)’? I won’t ask that, but that is what it shows. So then… then it’s like ‘hey, I would otherwise not have discussed that’.’ (Centre 3, healthcare provider 4)

‘Yes, but then in a solution-oriented way - then you will see someone with 30 apnoeas an hour and you see that and you say I hope that (your problem with emotions) will get a lot better with the therapy I will start for you.’ (Centre 1, healthcare provider 2)

‘Especially I thought people were, uhm… that lack of initiative, not going out, right? So they don’t do things because of their sleep problem, that was what (the Patient-Reported Apnea Questionnaire) often showed. And I didn’t always get that from taking the patient history. So people maybe find that hard to tell me, or they have trouble indicating that it really does have an impact on them. And then they try to focus more on the fact than on the underlying emotion. And that would sometimes give added value.’ (Centre 1, healthcare provider 1)

With regard to treatment choice, the professionals mentioned that the severity of symptoms generally only plays a role in patients with an AHI <15, for which shared-decision making could potentially lead to a decision not to start clinical treatment for OSA. If the AHI is ≥15, professionals generally wish to treat a patient for health reasons irrespective of symptoms. Many patients also have a reason to opt for treatment: there is a motor vehicle driving ban for untreated patients with AHI ≥15.

Use of the PRAQ during follow-up consultations could not be fully evaluated, because a limited number of patients had completed the PRAQ at follow-up at the time of the interviews. This was due to practical implementation issues in combination with the relatively short duration of the study. However, several healthcare professionals mentioned that they thought the PRAQ would be more useful during follow-up consultations than intake conversations, as it would be interesting to see which problems remained after starting treatment (table 2). Those that had the opportunity to use the PRAQ in this setting mentioned that it was nice to show patients how their problems had improved, with the improvement sometimes being greater than the patients had realised. This could be used as encouragement to continue with treatment.

Survey results

A total of 487 patients completed surveys pre-implementation, and 377 patients completed surveys post-implementation of the PRAQ. Characteristics of the survey populations pre-implementation and post-implementation can be found in table 3.

Table 3.

Characteristics of the survey population

	Pre-implementation (n=239)	Post-implementation (n=197)
Intake consultations
Age (yrs)	53.9	55.4
Gender (% male)	68.4	69.5
Severity of symptoms*	6.50	6.44
Diagnosed with OSA (%)	82.8	83.2
CPAP† (%)	71.0	70.7
MRA‡ (%)	13.7	19.5
Other treatment† (%)	10.7	7.4
No treatment† (%)	1.0	2.4
Missing† (%)	3.6	0.0

	Pre-implementation (n=248)	Post-implementation (n=180)
Follow-up consultations
Age (yrs)	57.33	58.54
Gender (% male)	75.3	69.7
Severity of remaining symptoms or problems with treatment*	4.25	5.03
CPAP (%)	89.1	89.4
MRA‡ (%)	3.6	3.9
Other§ or missing (%)	7	5.6

*Scale 1–10, higher is more problems.

†Percentage of patients with this treatment of the total of patients diagnosed with OSA.

‡Device worn over the teeth that pushes tongue and jaw forward to hold the airway open.

§Other possible treatments are surgery of the jaw or throat, and methods that will help a patient with positional OSA (who experiences breathing stops mainly when they lie on their backs) sleep on their side.

CPAP, continuous positive airway pressure; MRA, mandibular repositioning device; OSA, obstructive sleep apnoea; yrs, years.

Patients generally showed high agreement with the statements of the survey: 73.3% to 97.3% of patients indicated ‘agree’ or ‘completely agree’ per statement about the intake and follow-up consultations (table 4). Follow-up patients post-implementation showed significantly less agreement with the statement ‘In my opinion, my treatment is worth it for me’ (p=0.005). The main difference between pre-implementation and post-implementation scores lies in distribution between scores 6 and 7 (‘agree’ and ‘completely agree’), with 68.2% of pre-implementation patients giving a score of 7, and 54.3% of post-implementation patients giving a score of 7. The other statements showed no obvious or statistically significant differences in the level of agreement pre-implementation and post-implementation.

Table 4.

Survey results*

Scores	Pre-implementation PRAQ					Post- implementation PRAQ
	1–3 % (n)	4–5 % (n)	6–7 % (n)	N /A† (n)	Median	1–3 % (n)	4–5 % (n)	6–7 % (n)	N /A† (n)	Median
Intake consultations
I knew which problems I wanted to discuss with the doctor	17 (7.2)	18 (7.7)	200 (85.1)	4	6	8 (4.2)	14 (7.3)	170 (88.5)	5	6
I discussed with the doctor the topics I wanted knew I wanted to discuss beforehand	22 (9.8)	28 (12.5)	174 (77.7)	15	6	16 (8.9)	22 (12.2)	142 (78.9)	17	6
Because of my conversation with the doctor, I understand better what causes my problems	14 (6.0)	14 (6.0)	206 (88.0)	5	6	6 (3.1)	10 (5.1)	178 (91.8)	3	6
The doctor and I chose the treatment together or together chose not to treat my apnoea	14 (6.0)	12 (5.2)	206 (88.8)	7	6	7 (3.6)	14 (7.3)	172 (89.1)	4	6
Because of my conversation with the doctor, I understand how the treatment can benefit me	12 (5.6)	12 (5.6)	192 (88.9)	23	6	4 (2.2)	10 (5.4)	172 (92.5)	11	6
I think the treatment will be worth it for me	10 (4.7)	17 (8.0)	196 (87.3)	26	6	4 (2.2)	18 (9.8)	161 (88.0)	14	6
Follow-up consultations
I knew which problems I wanted to discuss with the doctor	11 (5.1)	10 (4.7)	194 (90.2)	33	7	5 (3.4)	10 (6.9)	130 (89.7)	35	7
I discussed with the doctor the topics I wanted to discuss	10 (4.7)	14 (6.5)	190 (88.8)	34	6	8 (5.7)	9 (6.4)	123 (87.9)	40	6
There was enough attention for the complaints that I still have	7 (3.2)	6 (2.7)	206 (94.1)	29	7	2 (1.4)	2 (1.4)	144 (97.3)	32	7
My complaints have lessened since start of my treatment	21 (8.9)	36 (15.3)	179 (75.8)	12	7	19 (11.5)	25 (15.1)	121 (73.3)	15	6
In my opinion, my treatment is worth it for me‡	7 (2.9)	10 (4.2)	222 (92.9)	9	7	6 (3.4)	12 (6.9)	157 (89.7)	5	7
Usefulness of the PRAQ
The PRAQ-report was useful for preparing my consultation§	-	-	-	-	-	6 (6.5)	22 (23.9)	64 (69.6)	2	6
The PRAQ-report was useful during my consultation¶	-	-	-	-	-	3 (2.3)	18 (13.6)	111 (84.1)	29	6

*Scale 1–7 (1=completely disagree, 2=disagree, 3=disagree a little, 4=neither agree nor disagree, 5=agree a little, 6=agree, 7=completely agree).

†‘Not applicable’ (see online supplementary file 4) or missing.

‡Significant difference between pre-implementation and post-implementation (p=0.005, Mann-Whitney U test).

§Showing results for patients who indicated they had seen the PRAQ-report before their (intake or follow-up) consultation (n=94).

¶Showing results of patients who indicated the PRAQ-report was shown during their (intake or follow-up) consultation (n=161).

N/A, not applicable; PRAQ, Patient-Reported Apnea Questionnaire.

Patients showed high agreement with the two statements about the usefulness of the PRAQ-report, particularly regarding its use during a consultation (table 4). However, not all patients had completed the PRAQ and seen the PRAQ-report before or during their consultation. Patients who did not look up the PRAQ-report before their consultation may also have been the ones less interested in using the PRAQ-report, so the reported results may be somewhat biased towards are more positive evaluation (table 5).

Table 5.

Percentage of patients that completed and viewed the PRAQ, and patient opinion on usefulness of the PRAQ

	Intake (n=197)	Follow-up (n=180)
Completed PRAQ before consultation	77.7%	51.1%
Seen PRAQ-report before consultation*	40.0%	44.4%
Seen PRAQ-report during consultation*	74.1%	60.2%

*This percentage is a sub-percentage of the patients who indicated they completed the PRAQ.

PRAQ, Patient-Reported Apnea Questionnaire.

Electronic health record results

One hundred and twenty-five patients were included in the pre-implementation group, and 124 other patients in the post-implementation group. Patient characteristics are described in table 6. No differences were found with regard to how many patients with OSA received non-medical treatment (either no treatment at all or referral to a psychologist (table 7)), or in the number of patients for whom treatment was adjusted at the first follow-up consultation after starting CPAP treatment (table 8).

Table 6.

Patient file study: patient characteristics

	Pre-implementation (n=125)	Post-implementation (n=124)
Age (SD)	55.4 (12.0)	56.6 (15.7)
Gender	68% male	67.7% male
BMI (SD)	31.4 (6.5)	30.8 (6.1)
AHI (SD)	23.1 (16.1)	25.0 (18.5)
AHI <15	40.8%	33.9%
ESS (SD)	8.0 (4.8)	7.4 (5.0)
Start with CPAP at intake	78.4%	79.0%

AHI, Apnoea-Hypopnea Index; BMI, body mass index; CPAP, continuous positive airway pressure; ESS, Epworth Sleepiness Scale.

Table 7.

Treatment choice at intake*

	Pre-implementation (n=125)	Post-implementation (n=124)*	Pre-implementation, AHI <15 (n=51)	Post-implementation, AHI <15 (n=42)
Medical treatment for OSA (incl CPAP) (n,%)	123 (98.4)	123 (99.2)	49 (96.1)	41 (97.6)
No medical treatment for OSA (n,%)	2 (1.6)	1 (0.8)	2 (3.9)	1 (2.4)
Referred to psychologist (n, %)	2 (1.6)	0 (0)	2 (3.9)	0 (0)
No treatment (n, %)	0 (0)	1 (0.8)	0 (0)	1 (2.4)

*If nothing is indicated, no significant difference was found.

AHI, Apnoea-Hypopnea Index; CPAP, continuous positive airway pressure; incl, including; OSA, obstructive sleep apnoea.

Table 8.

Treatment adjustments and compliance in patients with CPAP at the first follow-up*

	Pre-implementation of PRAQ (n=98)	Post-implementation of PRAQ (n=98)	Missings
Adjustment of current treatment	45	36	N/A†
Switch to different treatment	5	9	N/A†
Referral to different specialisation	6	2	N/A†
CPAP compliance‡ (SD)	5:47 hours (2:11)	5:53 hours (2:10)	Pre-impl.: 11 Post-impl: 7
CPAP compliance<4 hours	25.0%	27.5%	Pre-impl.: 11 Post-impl: 7
Stopped CPAP treatment	4.1%	5.1%	N/A†

*If nothing is indicated, no significant difference was found.

†If nothing was noted down in the patient health record, it was assumed this did not take place. Therefore missings are not applicable.

‡Hours of CPAP use by patients who had stopped treatment altogether (see ‘stopped CPAP treatment’) are not included in this number.

CPAP, continuous positive airway pressure; impl., implementation; N/A, not applicable; PRAQ, Patient-Reported Apnea Questionnaire.

In both groups, 98 patients were prescribed CPAP. Patient characteristics did not differ between the two groups of patients with CPAP (data not shown). Compliance with CPAP treatment did not differ between the two groups (table 8).

Discussion

This exploratory study showed limited success regarding the uptake of the PRAQ in the daily clinical practice of sleep centres, and the improvement of patient-centredness of care. From the interviews it became clear that most patients were willing to complete the PRAQ and were generally positive about the usefulness of the PRAQ before the consultation (eg, because of feeling more informed) and during the consultation (due to the clear visual representation of their problems). This may therefore have lead to some improvement of preparation for the consultation by patients, and better communication, though this is not reflected in the results of the patient survey. Among healthcare professionals the willingness to use the PRAQ-report in consultations differed, as the perceived need was minimal. Most of the professionals that used the PRAQ also reported that the impact on their consultations was minor. Therefore, it is not surprising that comparison of health records pre-implementation and post-implementation of the PRAQ did not show any differences in treatment choice and CPAP compliance.

The interviews showed that the professionals mostly felt that they already sufficiently address the ‘symptom-like’ topics of the PRAQ (sleepiness, problems at night) in their usual care, in the context of setting a diagnosis. The topics of the PRAQ that are not necessary for setting a diagnosis, but could potentially be used to motivate patients for their treatment, were not seen as essential to discuss by many professionals. The limited perceived benefit of the PRAQ is likely also mitigated by the fact that many steps of the care process have to be covered during the intake consultation, including discussing the sleep study results and choosing a treatment. This leaves little extra time to discuss a patient’s quality of life and detailed treatment goals. Furthermore, burden of disease plays a limited role in setting a diagnosis when AHI ≥15, due to views on strict medical necessity of treatment, but also due to the driving ban for untreated patients. Therefore, adding the PRAQ to the current practice for OSA does not appear to be a sufficient trigger to increase attention to quality of life issues.

Patients generally held a more positive view towards the usefulness of the PRAQ. From the interviews it became clear that completing the PRAQ has the potential to give patients more insight into their OSA-related health complaints and encourages communication between family members. Furthermore, the patient survey results indicated that patients thought the PRAQ-report was useful for their preparation for the consultation and (when it was used by the healthcare professional) during the consultation.

Agreement to the patient survey statement ‘I feel like my treatment is worth it for me’ was significantly lower on the post-implementation survey than on the pre-implementation survey. The main difference was in the number of patients indicating ‘agree’ versus ‘completely agree’, meaning both pre-implementation and post-implementation of the PRAQ patients were very positive about their treatment. This being an exploratory study, statistically significant results should be interpreted with caution, and we deem the relevance of this finding to be limited.

There appears to be room for improvement of communication around the PRAQ, as there was confusion for some patients around the necessity of still discussing symptoms during the consultation. Whereas some patients seemed to be more interested in hearing their sleep study results than talk about their symptoms, for the healthcare professionals hearing about the patient’s symptoms in their own words is an essential part of the diagnosis. It may be beneficial to communicate the purpose of the PRAQ more clearly in the invitation email, and/or to instruct professionals to, at the beginning of their consultation, mention the PRAQ to patients and how its results will be addressed. More in-depth discussion with the field about what is most suitable or desirable in this context is needed.

In the past few years, several similar initiatives involving PROMs have been introduced in The Netherlands, such as the Assessment of Burden of COPD (ABC) tool,³⁴ the Nijmegen Clinical Screening Instrument for COPD,³⁵ the QLIC-ON PROfile for children³⁶ and myIBDcoach for patients with inflammatory bowel disease.³⁷ Studies into these applications show promising results regarding their benefits^{37 38} despite some resistance from professionals who do not believe in the added benefit or believe the tool would be more useful for different professionals within the care pathway.^{39 40} However, the healthcare professionals’ scepticism about the potential benefits of the PRAQ seems to be more extensive. Potentially, professionals will see greater benefit of the PRAQ in the context of the recently released new guidelines for OSA³⁰ with their greater emphasis on (improving) burden of disease, which were not yet available at the time of this study. However, the question remains whether a more ‘holistic’ approach to caring for OSA patients fits within the current setting of relatively short intake consultations which take place after the patients’ diagnostic sleep study. It may be necessary to move towards a reorganisation of care: for example to plan the intake consultations before the sleep study to allow for more focus on the individual patients’ symptoms and problems, and to specifically evaluate the necessity of doing a diagnostic sleep study. Additionally, integrating the PRAQ in the electronic health record will help professionals fit the PRAQ-report better into their workflow.

Another option that can be explored is to adapt the PRAQ itself or the context in which it is used, in order to fit better to healthcare professionals’ preferences. For example, an option would be to remove the domains of the PRAQ focused on symptoms that are (nearly) always discussed already, and instead put the focus on the additional domains. It is also possible to distribute the PRAQ to a more select group of patients, for example by moving the first measurement moment to the follow-up consultation, therefore targeting only patients with a diagnosis and treatment. It could then be used to identify those patients still experiencing problems. Downside to both of these adaptations is that they limit the option to monitor changes over time on all domains that are relevant for patients with OSA, while monitoring over time is what most interviewed healthcare professionals are interested in. Not having a baseline measurement would also limit the options to usefully study the PRAQ data on aggregate level. It may be most feasible to let sleep centres decide how they want to use the PRAQ in the context of what is desirable to them, which may also evolve over time. It is hoped that they will also take into account the patient perspective when deciding how to use the PRAQ.

Strengths and limitations of the study

The major strength of this study is that we used mixed methods, which provides insight into the reasons why the PRAQ does not work as intended. Many other studies on PROMs study only whether a PROM works, rather than why or how.

There are also limitations to the study. First, the survey used for this study was not tested and maybe not discriminative enough to show differences between the groups pre-implementation and post-implementation of the PRAQ. Potentially, patients who have not completed the PRAQ do not know that, for example, their preparation for the consultation could have been better than it currently was. Second, electronic health records were only studied in one of the included sleep centres. However, considering the information we collected in the interviews, we do not expect that we would have found different results in either of the other two sleep centres. Third, though technically there was enough time in this study for professionals to also use the PRAQ during the first follow-up consultation, practical implementation issues as well as a lack of initiative from healthcare professionals to actively check whether a follow-up PRAQ was available meant that it was not used often at this time point. Therefore we did not gain much insight into the potential use of the PRAQ for follow-up consultations. Lastly, only patients who looked up the PRAQ-report could give an opinion on its usefulness for preparing the consultation in the survey. However, patients who did not look up the PRAQ-report may also be generally less interested in these kinds of tools and, if they had looked it up, may have experienced it as less useful. Additionally, patients who have a more positive opinion on the PRAQ may be more likely to complete the items on its usefulness.

Conclusions

Using the PRAQ in the daily clinical practice of OSA is viewed as useful by patients, but the enthusiasm of healthcare professionals differed per individual and was generally not very great. Implementation of the PRAQ does not seem a sufficient trigger to focus more attention to quality of life during consultations, and in current practice does not show impact on treatment choice or CPAP compliance. However, new Dutch guidelines for OSA care that have recently been published may lead to a greater emphasis on quality of life for patients, making the integration of the PRAQ in clinical care potentially more useful.