Validation of online Asthma Control Questionnaire and Asthma Quality of Life Questionnaire

Abstract

Objective

Several newly developed eHealth applications use online questionnaires to monitor asthma control. The Asthma Control Questionnaire (ACQ) and Asthma Quality of Life Questionnaire (AQLQ) are two such commonly used questionnaires. These questionnaires are validated for use on paper. This study aims to validate them by assessing the agreement between online and paper versions of the ACQ and AQLQ.

Methods

Patients (aged 18 years and older) from the Self-Management in Asthma Supported by Hospitals, ICT, Nurses and General Practitioners (SMASHING)-trial and Davos@home study were included in this study. Patients completed both the paper and online Dutch versions of the ACQ and AQLQ in a random order within a 2-week interval. Agreement between the different versions was assessed with paired t-tests, intraclass correlation coefficients and Bland–Altman plots.

Results

In total 44 patients were eligible for analysis. The mean difference between the paper and online versions of the ACQ was 0.04 (p=0.40) and for the AQLQ it was 0.08 (p=0.06). The intraclass correlation coefficient scores were 0.94 for the ACQ and 0.95 for the AQLQ.

Conclusion

The online versions of the ACQ and AQLQ show high levels of agreement with the paper versions and can therefore be safely used in eHealth applications to respectively monitor asthma control and quality of life.

Short abstract

The self-administered online versions of the Asthma Control Questionnaire and Asthma Quality of Life Questionnaire show good agreement with the paper versions, and are therefore, valid measurement tools in eHealth applications http://bit.ly/2rGBbRf

Introduction

According to international guidelines the goal of asthma management is to achieve and maintain asthma control, including prevention of exacerbations and minimisation of symptoms [1]. However, in many patients these goals are not met [2]. Among other factors, behavioural aspects, such as poor medication adherence, contribute to uncontrolled asthma [3]. Additionally, according to guidelines, patients should be reviewed approximately twice a year at pre-specified appointments [1]. However, these appointments do not necessarily correlate with the moment a person's asthma is uncontrolled. Patients might experience these appointments as either unnecessary at that point in time (I have no symptoms now) or too late (my asthma has worsened for a longer period in time), which in turn influences behavioural aspects. Therefore, improving asthma self-management is currently a major goal [1]. To facilitate self-management, it is expected that online self-management tools will play an important role [4]. This view is also shared by patients themselves [5].

The use of online tools on mobile devices and other internet technologies is becoming more and more common. Research into digital support (eHealth) solutions is ongoing and increasingly accepted as an opportunity to reach more patients and facilitate the implementation of asthma self-management [6–8]. A systematic review by Morrison et al. [6] showed that digital interventions are effective at improving knowledge, reducing activity limitation, improving markers of self-management, improving quality of life, and optimising medication use. One study showed that patients preferred a Web-based system of monitoring asthma compared to a paper-based system [9]. In addition, we have shown that internet-based self-management support resulted in sustained improvements in asthma control and asthma-related quality of life [10, 11].

Two commonly used and well-validated paper questionnaires to track change in asthma outcomes are the Asthma Control Questionnaire (ACQ) and Asthma Quality of Life Questionnaire (AQLQ) [12, 13]. Of these paper-based questionnaires, self-administered versions and interviewer-administered versions exist. In a previous study we reported that assessment of asthma control by the ACQ is influenced by the type of administration [14]. Therefore, validation of online questionnaires is required. This current study aimed to compare the self-administered online ACQ and AQLQ versions with the self-administered paper equivalent in order to validate the online versions.

Materials and methods

Patients

Patients from the Self-Management of Asthma Supported by Hospitals, Information and communication technology, Nurses and General practitioners (SMASHING)-trial as well as patients from the Davos@home study were included in this study [10]. The projects were approved by the Medical Ethics Committee of the Leiden University Medical Center, Leiden, the Netherlands. The SMASHING-project studied the effect of communication technology through self-management programmes on asthma-related quality of life. The original SMASHING study was a 12-month multicentre randomised controlled trial in the Netherlands. Patients were randomised to receive either internet-based self-management combined with usual care or usual care alone between January 2006 and March 2007. Only patients from the intervention group who received both online and paper questionnaires were used in this study. The trial was registered at www.trialregister.nl/trial/464. In the Davos@home study, asthma patients' perceptions on eHealth were assessed using qualitative analyses. Patients were enrolled between May and September 2019. All patients gave informed consent.

Inclusion criteria were age ≥18 years and patients required asthma severity step ≥2 [1]. Patients were also required to use inhalation corticosteroids for at least 3 months in the year prior to the study. Severe comorbidity, inability to speak, write and understand Dutch were exclusion criteria.

Study design

Patients were randomised to either the paper–online group, who filled in paper versions of the ACQ and AQLQ at baseline and the online versions within 2 weeks, or the online–paper group, who filled in the online versions first and the paper versions within the 2-week period. The advised range of assessing reproducibility of measurements is 2–14 days [15]. Randomisation was performed to ensure the order of filling out questionnaires (online or on paper first) did not influence the results. A more detailed description of the study method and data collection can be found in the original SMASHING study manuscript [10]. For this study, Juniper [12] as the author of ACQ and AQLQ gave permission to use an online version within our internet-based self-management support system, providing that the layout of the online version was very similar to the original paper version. The Dutch versions of the questionnaires were used.

ACQ

The ACQ is a questionnaire that consists of six questions on a scale from 0 to 6 of symptoms. The questions are about limitations due to asthma and symptoms in the past week. A lower score corresponds with better asthma control. A Minimal Clinically Important Difference (MCID) of 0.5 is used [13]. The MCID indicates the minimal difference in mean scores that is regarded as important [16].

AQLQ

The standard AQLQ questionnaire consists of 32 questions with seven answer options. First, patients were asked whether they were limited in their daily activities and if so, to name the five activities that were affected the most. For each of these five activities the patients were then asked how much they were limited in the last 2 weeks on a seven-point scale. The remainder of the questions generally quantified the frequency and extent of discomfort. The questions are divided in four domains: activity limitation, symptoms, emotional function and exposure to environmental stimuli [12]. The mean results were calculated (minimum 1, maximum 7) where the higher the score, the better the quality of life is. A MCID of 0.5 is also agreed upon [17].

Analysis

To assess possible differences between online and paper-based ACQ and AQLQ, paired t-tests were used. Next, the intraclass correlation coefficient (ICC) was calculated to assess the agreement of the two versions of the questionnaires in a single measure and an identity plot was displayed. Bland–Altman plots were used to assess the agreement between the different versions of the ACQ and AQLQ. Bland–Altman plots show the agreement between the online and paper versions of the questionnaires, by relating the difference of the two measurements (online and paper) to the average of these measurements [18]. The range of differences where 95% of the observations lie within the Bland–Altman plots is indicated by the limits of agreement. In order to determine whether the two measurements in the Bland–Altman plot were different from each other, limits of 0.5 (based on the MCID) were used. All analyses were performed with STATA 14.2 (StataCorp, College Station, TX, USA).

Results

In total, 44 patients were included in our study (35 from SMASHING and 9 from Davos@home). Two patients failed to complete the ACQ questionnaire on paper and one patient failed to complete the electronic AQLQ questionnaire and they were therefore removed from the analysis for the ACQ or AQLQ respectively (figure 1). Every patient with at least one set of questionnaires (both online and paper) filled in was included. Of the total group of asthma patients, 25 patients filled in the paper questionnaires at baseline and the online versions within 2 weeks. Baseline characteristics are summarised in table 1. The majority of the patients were female and most patients were not current smokers. Based on paper ACQ measurements, 24 patients had controlled asthma (ACQ<0.75), 12 patients had partly controlled asthma (0.75≤ACQ<1.5) and 6 patients had uncontrolled asthma (ACQ≥1.5) [13, 19].

FIGURE 1.

Flowchart participant enrolment. SMASHING: Self-Management in Asthma Supported by Hospitals, ICT, Nurses and General Practitioners; AQLQ: Asthma Quality of Life Questionnaire; ACQ: Asthma Control Questionnaire.

TABLE 1.

Baseline characteristics based on paper questionnaire (n=44)

Age years	39.8±10.8
Females	33 (75%)
Smokers
Never	29 (66%)
Former	14 (32%)
Current	1 (2%)
Baseline ACQ	0.84±0.87
Baseline AQLQ	6.04±0.91

Data are presented as mean±sd or n (%). ACQ: Asthma Control Questionnaire; AQLQ: Asthma Quality of Life Questionnaire.

As shown in table 2 no significant differences were found between the online and paper versions of the ACQ and AQLQ. The differences in mean scores of the online versions compared to the paper versions of the ACQ (mean difference=0.04; p=0.40) and AQLQ (mean difference= −0.08; p=0.06) were minimal. The AQLQ can be divided in four domains and three domains showed no significant difference between the online and paper versions of the questionnaires, as shown in table 3. Only the symptoms domain showed a statistically significant difference (p=0.046), although the score difference (−0.09) was not clinically relevant as it was below the MCID of 0.5. Mean scores were also calculated for every individual question of the ACQ (table 4).

TABLE 2.

Correlation scores for the Asthma Control Questionnaire (ACQ) and Asthma Quality of Life Questionnaire (AQLQ) online and paper versions compared to one another

	Participants	Online	Paper	ICC	Difference	p-value	Limits of agreement
ACQ	42	0.85±0.87	0.88±0.87	0.94	0.04±0.31	0.40	−0.56, 0.64
AQLQ	43	6.14±0.86	6.06±0.90	0.95	−0.08±0.27	0.06	−0.61, 0.45

Data are presented as mean±sd unless otherwise stated. ICC: intraclass correlation coefficient.

TABLE 3.

Asthma Quality of Life Questionnaire (AQLQ) mean scores divided by domain

	Paper	Online	Mean difference (95% CI)	p-value
Total AQLQ	6.06	6.14	−0.08 (−0.16–0.00)	0.06
AQLQ activities	6.07	6.10	−0.03 (−0.14–0.08)	0.57
AQLQ environment	5.96	6.05	−0.09 (−0.23–0.05)	0.19
AQLQ emotional	6.32	6.45	−0.13 (−0.27–0.00)	0.06
AQLQ symptoms	5.99	6.09	−0.09 (−0.19–0.00)	0.046

TABLE 4.

Asthma Control Questionnaire (ACQ) mean scores divided by question (Q)

	Paper	Online	Mean difference (95% CI)	p-value
Total ACQ score	0.88	0.85	0.04 (−0.06–0.13)	0.40
ACQ Q1 (awoken)	0.53	0.49	0.05 (−0.15–0.25)	0.64
ACQ Q2 (morning)	0.88	0.81	0.07 (−0.11–0.25)	0.44
ACQ Q3 (limitations)	1.23	1.07	0.16 (−0.04–0.36)	0.11
ACQ Q4 (shortness of breath)	1.47	1.53	−0.07 (−0.29–0.15)	0.52
ACQ Q5 (wheeze)	0.76	0.81	−0.05 (−0.23–0.13)	0.60
ACQ Q6 (β2-agonist use)	0.50	0.45	0.05 (−0.07–0.17)	0.42

Intraclass correlation

The ICC scores (ACQ 0.94; AQLQ 0.95) represented a high agreement between online and paper versions. As shown in the identity plots, most of the points followed the line of identity (figure 2).

FIGURE 2.

Identity plots. The paper questionnaires plotted against the online versions for the a) Asthma Control Questionnaire (ACQ) and b) Asthma Quality of Life Questionnaire (AQLQ). The x=y line depicts an identical result for the paper and online versions. The agreement between the two versions is acceptable if the measurement falls between the upper and lower limits of the minimal clinically important difference (MCID) (mean difference ±0.5).

Bland–Altman plots

In figure 3 the Bland–Altman plots are shown. The majority of the patients fall between both limits of agreement and the MCID.

FIGURE 3.

Bland–Altman plots for the a) Asthma Control Questionnaire (ACQ) and b) Asthma Quality of Life Questionnaire (AQLQ). ---: mean difference. The area between the dashed lines (– – –) represents the minimal clinically important difference range (mean difference ±0.5) and the solid lines (—) represent the limits of agreement.

Discussion

In this study we showed that paper versus online self-administered versions of both the ACQ and AQLQ show good agreement. There were only minimal mean differences between paper and online versions of both questionnaires, none of which exceeded the MCID. The agreement as indicated by the intraclass correlation was good.

The ACQ and AQLQ paper-based questionnaires developed by Juniper [12, 13, 20] are well-validated instruments to measure asthma control and asthma-related quality of life, respectively. She carefully developed and validated several versions for application in different subgroups and settings. Shortened versions [21, 22] and versions for children [23, 24] are proving invaluable in the assessment and management of patients' health and are used as primary outcomes in research studies. In previous literature, validation of the ACQ and AQLQ were extensively studied for use in different circumstances. The study by Nguyen et al. [25] showed that the ACQ can be used in children. Translations of the questionnaires and use in different ethnic groups are also studied [26–31]. All concluded that the ACQ and AQLQ were valid in a broad approach.

The study by Honkoop [14] concluded that type of administration influenced the perceived level of asthma control. Self-administered online questionnaires tend to yield more severe symptoms than interviewer-administered questionnaires by the Practitioner Nurse. This indicates that the type of administration might influence the measurements. Indeed, two previous studies have shown significant bias and inadequate concordance between the original paper version and the electronic versions [32, 33]. Our study looked specifically at differences between results of different forms of self-administered questionnaires (online versus paper). We showed that when using a very similar paper and online layout, the method of acquiring self-administered responses did not yield different outcomes.

Different analysis methods to quantify agreement between the two versions of questionnaires all showed that paper and online usage of the ACQ and AQLQ yield comparable results. The patients from the SMASHING study had mild-to-moderate persistent asthma, whereas the patients from the Davos@home study had severe asthma. The heterogeneity of our study population strengthens the external validity of our findings.

One limitation of the study was only having a single comparison per patient. Every patient filled out every set of questionnaires only once due to restrictions in the study protocol. Multiple comparisons per patients over a prolonged time would have strengthened the study. Another limitation of the study is the small study population which might have resulted in an overestimated SD of differences. We verified the Bland–Altman plot in a control group that only filled in paper questionnaires, both at baseline and within 2 weeks. We saw that the limits of agreement were similar to the paper–online/online–paper comparison and that the limits of agreement were also for this control group not in the MCID range. This suggests that our findings reflect normal variations in asthma control.

This study showed that online versions of the ACQ and AQLQ yielded results with good agreement with the original paper versions. This means that eHealth applications based on our validated online format can be safely used to monitor asthma control and quality of life. Online questionnaires currently play an important role in many eHealth programmes and this study showed that these online questionnaires are valid measurements tools.

We conclude that online self-administered versions of the ACQ and AQLQ with a similar layout as the original paper versions are valid tools to monitor asthma control and quality of life.