Development of a simple binary response questionnaire to identify airflow obstruction in a smoking population in Argentina

Abstract

The CODE questionnaire (COPD detection questionnaire), a simple, binary response scale (yes/no), screening questionnaire, was developed for the identification of patients with chronic obstructive pulmonary disease (COPD). We conducted a survey of 468 subjects with a smoking history in 10 public hospitals in Argentina. Patients with a previous diagnosis of COPD, asthma and other respiratory illness were excluded. Items that measured conceptual domains in terms of characteristics of symptoms, smoking history and demographics data were considered. 96 (20.5%) subjects had a diagnosis of COPD according to the 2010 Global Initiative for Chronic Obstructive Lung Disease strategy document. The variables selected for the final questionnaire were based on univariate and multivariate analyses and clinical criteria. Finally, we selected the presence or absence of six variables (age ≥50 years, smoking history ≥30 pack-years, male sex, chronic cough, chronic phlegm and dyspnoea). Of patients without any of these six variables (0 points), none had COPD. The ability of the CODE questionnaire to discriminate between subjects with and without COPD was good (the area under the receiver operating characteristic curve was 0.75). Higher scores were associated with a greater probability of COPD. The CODE questionnaire is a brief, accurate questionnaire that can identify smoking individuals likely to have COPD.

Short abstract

The COPD detection questionnaire is a simple, binary response scale for the identification of patients with COPD http://ow.ly/KiEdF

Introduction

Chronic obstructive pulmonary disease (COPD) has become a major public health problem worldwide because of its high and increasing prevalence, morbidity and mortality. Under-diagnosis of COPD is a worldwide problem [1]; despite the existence of evidence-based diagnostic and treatment guidelines, many patients with COPD remain undiagnosed or misdiagnosed [2]. Hence, there is a need for a simple method to help identify persons who might have COPD. A simple, self-administered, self-scored tool to screen individuals for the disease may lead to increased awareness, earlier symptom recognition and the use of spirometry for accurate diagnosis.

There are screening questionnaires designed to identify patients with COPD [3–5]; however, only one has a translated and validated Spanish version [6]. It is important to develop the questionnaire from questions expressed by the same population in which it is going to be used. In addition, the dissemination of such a tool in the general population should encourage individuals to discuss their respiratory symptoms with a healthcare provider and is aimed at assisting physicians to identify those individuals who require spirometric assessment.

The aim of the present study was to develop a simple, binary response scale (yes/no) screening questionnaire for the identification of patients with possible COPD in a smoking population. This paper describes the item-selection and item-reduction phases of the development of the CODE questionnaire (COPD detection questionnaire), designed as a screening tool that can be used efficiently in primary care settings to detect those subjects in whom spirometry testing for COPD is appropriate.

Materials and methods

Working group and survey development

A working group of the Argentine Association of Respiratory Medicine was gathered to assist with development of the questionnaire. The working group guided development of specific survey question content and assisted in the evaluation of the study results. Based on clinical experience, the working group first identified conceptual domains relevant to the detection of COPD and easily identified by patients. From a systematic review of the literature [7], powerful questions to improve COPD detection were also included in the survey. Items that measured the conceptual domains in terms of presence, frequency, duration or quality of symptoms were considered. Local experience at the Hospital de Clínicas, Buenos Aires University, Buenos Aires, Argentina, with 400 current or former smokers was used for the questionnaire design. An abbreviated description of these items is shown in table 1.

TABLE 1.

Item content of variables tested for inclusion in the questionnaire

Conceptual domain	Abbreviated item content
Dyspnoea	Shortness of breath when walking fast or walking up small slopes
Cough	Cough most days for a period of 3 months in a year
	Cough most days for more than 2 years
Phlegm	Phlegm most days for 3 months in a year
	Phlegm most days for more than 2 years
Wheezing	Wheezing sometime in the last year
	Wheezing without a respiratory infection
History of smoking	Current smoker?
	Pack-years
	Have you smoked more than 100 cigarettes in your life?
Previous respiratory disease	Previous diagnosis of asthma
	Previous diagnosis of chronic bronchitis
	Previous diagnosis of emphysema
Environment	Biomass smoke at home
	Risk profession

Data collection

Consecutive individuals, >40 years of age, who were smokers or ex-smokers with a smoking history of ≥10 pack-years, attending any of the participating hospitals for a nonrespiratory cause, were prospectively surveyed during an 8-week period between November and December 2011. Subjects with a previous diagnosis of COPD, asthma and other respiratory illness were excluded. Clinical sites with broad representation throughout the country and with experience performing spirometry according to the American Thoracic Society (ATS)/European Respiratory Society (ERS) Task Force guidelines [8] were selected. Approval was obtained from the ethics committees of the institutions involved in the study, and written informed consent was obtained from each individual.

Prospectively, 10 centres were recruited in public hospitals in the cities of Buenos Aires, Vicente Lopez, La Plata, Neuquén, Mendoza, Rosario, Baigorria, Tucumán, Parana and Corrientes. All centres worked with the same model of spirometer, which was provided for this study. Personnel were trained in the procedures according to the ATS/ERS standards. Eligible respondents were enrolled after providing informed consent. Participants completed a questionnaire covering demographics, history of smoking and symptoms and then underwent pre- and post-bronchodilator spirometry.

Spirometry was performed pre- and post-bronchodilator according to ATS/ERS standards [8]. For the purposes of this study, COPD was defined based on the 2010 Global Initiative for Chronic Obstructive Lung Disease (GOLD) strategy document (post-bronchodilator forced expiratory volume in 1 s (FEV1)/forced vital capacity (FVC) <70%) [9]. Patients with spirometry tracings meeting the strict ATS criteria were included in the analysis.

Statistical analysis

We performed a univariate analysis to observe the relationship of the predictor variables with airway obstruction. Continuous variables were expressed as median (25th–75th percentile) and analysed with the Mann–Whitney U-test. The categorical variables were expressed as proportions and analysed with the Chi-squared test. A multiple logistic regression model was constructed in order to identify the questionnaire items for discriminating between persons with and without airway obstruction. The model was fitted including the items from the CODE questionnaire as independent variables (tested in the univariate analysis and including those variables with p<0.2) and COPD as the dependent variable in order to compare odds ratios, logistic regression coefficients with corresponding p-values and area under the receiver operating characteristic (ROC) curve (using a probability cut-off of 0.50). We applied the backward elimination strategy to obtain the final predictive model in order to reduce the number of items and to identify the items most predictive of COPD (item reduction). In general, item choice was based on the results obtained with the base models, clinical relevance and ease of administration of the questionnaire (statistical significance was not cause for exclusion). Goodness-of-fit was assessed by the Hosmer–Lemeshow statistic [10].

Questionnaire items were analysed using a binary response scale (yes/no), dichotomising continuous variables to promote an easier interpretation and application of the questionnaire. Items found to be statistically significant in the logistic regression analysis and those variables considered to have clinical relevance were included in the final questionnaire. Sensitivity, specificity, positive predictive value, negative predictive value and percentage correctly classified for the presence of obstruction in this final questionnaire were calculated. Statistical analysis of data was performed using SPSS 17.0 (SPSS Inc., Chicago, IL, USA).

Results

In all, 485 current and former smokers were evaluated. Of these, 17 (3.5%) were excluded due to not having spirometry that met the quality criteria, or a previous diagnosis of asthma, emphysema and chronic bronchitis. We excluded four subjects with FEV1/FVC <70% who responded to bronchodilators because of suspected asthma, leaving 468 patients for analysis. Of these, 333 (71.2%) were current smokers and 135 (28.8%) were former smokers. 96 (20.5%) subjects had airflow obstruction and we confirmed the diagnosis of COPD according to the GOLD document [9]. Patients with COPD were older, had a lower body mass index (BMI), a higher median pack-years of smoking, more symptoms (cough, phlegm, dyspnoea, wheezing), biomass smoke at home and professional risks, although the latter variables did not reach statistical significance. The median FEV1 in COPD subjects was significantly lower (2.14 L versus 2.76 L; p<0.001). Baseline characteristics and differences among patients with and without COPD are shown in table 2. Among subjects with COPD, 38 (39.6%) had GOLD stage I, 46 (47.9%) had GOLD stage II, 10 (10.4%) had GOLD stage III and only two (2.1%) had GOLD stage IV.

TABLE 2.

Baseline characteristics of the total population and differences among COPD and non-COPD patients

Conceptual domain	Total population	COPD	No COPD	p-value
Age years	55 (49–61)	58 (54–64)	54 (47–60)	<0.001
Males	229 (49)	66 (68.8)	163 (43.9)	<0.001
Body mass index kg·m−2	28 (24–33)	26 (22–29.5)	28 (24–33)	0.002
Wheezing
Any wheezing	228 (48.7)	53 (55.2)	175 (47)	0.15
Without infection	163 (34.8)	37 (38.5)	126 (33.9)	0.39
Cough
For 3 months	186 (39.7)	53 (55.2)	133 (35.8)	0.001
Chronic cough	153 (32.8)	46 (47.9)	107 (28.8)	<0.001
Phlegm
For 3 months	186 (39.8)	48 (50)	138 (37.2)	0.022
Chronic phlegm	136 (29.1)	39 (40.6)	97 (26.1)	0.005
Dyspnoea	253 (54.1)	60 (62.5)	193 (51.9)	0.063
History of smoking
Current smokers	333 (71.2)	60 (62.5)	273 (73.4)	0.036
Smoking history pack-years	30 (20–40)	40 (25–50)	28 (18.38–40)	<0.001
Environment
Fireplace	38 (8.1)	10 (10.4)	28 (7.5)	0.35
Profession at risk	88 (18.8)	19 (19.8)	69 (18.5)	0.78

Data are presented as median (25th–75th percentile) or n (%), unless otherwise stated. COPD: chronic obstructive pulmonary disease.

Variables were selected for multivariate analysis according to the results of the univariate analysis and the clinical criteria of the evaluating committee. For this purpose we dichotomised age as ≥50 years or <50 years and smoking history as ≥30 pack-years or <30 pack-years, considering these to be cut-offs of increased risk according to previous data and clinical criteria. Although BMI was also statistically significant, this variable was eliminated from consideration for the reduced subset because of its low discriminatory power and the difficulty of easily and reliably assessing it in a patient-reported questionnaire. The results of the multiple logistic regression analysis that modelled the probability of having COPD versus not having COPD are shown in table 3. Significant associations were observed for age ≥50 years, male sex, smoking history of ≥30 pack-years and chronic cough. Chronic phlegm and dyspnoea, although not statistically significant, were included in the model and in the subsequent questionnaire because of their clinical importance. The remaining variables did not reach significance. The area under the ROC curve of the model was 0.75 (95% CI 0.70–0.80; p<0.001) (fig. 1).

TABLE 3.

Results of multivariate logistic regression for having airway obstruction^#

Item	Odds ratio estimate
Age
<50 years	Reference
≥ 50 years	4.42
Male sex	2.67
Smoking history
<30 pack-years	Reference
≥ 30 pack-years	2.02
Chronic cough
No	Reference
Yes	1.80

^#: chronic phlegm and dyspnoea were also considered in the model but did not reach statistical significance. Hosmer–Lemeshow p=0.80.

FIGURE 1.

Receiver operating character- istic curves for predictors of airway obstruction. The dashed line represents the sum score of the chronic obstructive pulmonary disease detection questionnaire (area under the curve: 0.75).

Table 4 summarises the performance of the CODE questionnaire score in identifying patients with COPD across multiple score levels. In these analyses, scores were dichotomised to allow an analysis of cut-point performance relative to COPD diagnosis. The absence or presence of the six variables (age ≥50 years, smoking history ≥30 pack-years, sex, chronic cough, chronic phlegm and dyspnoea) were tested to assess the best cut-offs for sensitivity and specificity. None of the patients without any of these six variables (0 points) had COPD. Thus, the presence of at least one variable (1 point) in the questionnaire has 100% sensitivity for COPD. Higher scores produced higher specificity and a higher percentage of correctly classified COPD. Most subjects had multiple symptoms; only 20 (20.8%) had one or two symptoms. We analysed the presence of these six symptoms according to the severity of COPD and found that subjects in GOLD stage III or IV tended to have more symptoms than subjects in GOLD stage I or II; 83.4% of GOLD stage III or IV subjects reported five or six points on the score compared with 29.7% of GOLD stage I or II subjects.

TABLE 4.

Performance of CODE questionnaire (COPD detection questionnaire) cut-point scores in screening for airway obstruction

CODE score	Total subjects n	COPD subjects n	Sensitivity	Specificity	PPV	NPV	Correctly classified
0	20	0	NA	NA	NA	NA	NA
1	72	1	100	5.41	21.52	100	24.89
2	100	19	98.96	24.59	25.40	98.91	39.91
3	117	22	79.17	46.49	27.74	89.58	53.22
4	76	19	56.25	72.16	34.39	86.40	68.88
5	62	24	36.46	87.57	43.21	84.15	77.04
6	19	11	11.46	97.84	57.89	80.98	80.04

Data are presented as %, unless otherwise stated. COPD: chronic obstructive pulmonary disease; PPV: positive predictive value; NPV: negative predictive value; NA: not applicable.

Discussion

Our data showed that COPD was common in long-term smokers aged >40 years (20.5%) [11, 12]. There was a statistically significant association of symptoms (cough and phlegm; with dyspnoea close to significance), smoking history, age and male sex with the presence of COPD. However, the logistic regression model showed a significant association between COPD and age ≥50 years, male sex, smoking history ≥30 pack-years and chronic cough. We decided to include chronic phlegm and dyspnoea in the final questionnaire because of their clinical significance; these six variables show a moderate prediction of COPD in terms of sensitivity, specificity and predictive values. As in other reports, we show that smokers with COPD frequently report symptoms that are linked to FEV1; subjects with earlier GOLD stages (I and II) have fewer symptoms than subjects in GOLD stages III and IV [13].

This study represents, for the first time in Argentina, the development and initial validation of a simple, reliable, COPD screening questionnaire. Three COPD-related items (dyspnoea, chronic cough and chronic phlegm), a smoking history item (≥30 pack-years), and an age and sex item were used to construct the questionnaire. A binary sum (yes/no) score of these items resulted in a questionnaire that discriminated between patients with and without COPD, and could be easily adapted to an electronic format for simple scoring. In contrast to this simple questionnaire, some earlier COPD screening tools require clinical information from medical records [3, 14] or previous report of a physician diagnosis of COPD [15] for scoring. Others include more complicated calculations or skip patterns that make them difficult to self-administer [16–18]. The value of self-report surveys relative to spirometry has been demonstrated for COPD case finding [4, 15]. van Schayck et al. [15] identified an optimal combination of items from the National Health and Nutrition Examination Survey (NHANES) III for distinguishing between those with and without COPD, but because their study did not include reversibility testing, it is unclear how well the results will generalise to COPD diagnoses based on GOLD definitions of the disease. Freeman et al. [19] identified four features (age, cough, dyspnoea and wheezing) that identified patients with COPD among a primary care population with a positive smoking history, history of use of respiratory medications, or a history of asthma. Calverley et al. [3] developed a population-based screening questionnaire for COPD retrospectively, using NHANES III data [12]. Our questionnaire has a good diagnostic accuracy demonstrated by a sensitivity of 79% and a specificity of 46%, for a cut-off value of 3 points. This compares very well with previous questionnaires such as COPD population screener (COPD-PS), which showed a sensitivity of 66% and specificity of 86%, and with the analysis of Kotz et al. [20], who showed a sensitivity of 65% and specificity of 54% for COPD-PS.

It should be noted that our study has several limitations. First, as FEV1/FVC is known to decrease with age [9], a classification scheme relying on FEV1/FVC <0.7 across all age groups is likely to result in large false-positive rates among elderly respondents (aged ≥65 years). In our study 12.6% were elderly subjects and we cannot rule out a possible overdiagnosis. Secondly, evidence suggests that continuum-based scales have better psychometric properties than dichotomous yes/no scales [21]. It is the feeling of the working group that a dichotomous yes/no scale, despite its lower discriminative power, is easier to interpret and apply in general practice. An additional limitation reflects the source of the analytical samples studied. Subjects were recruited from a clinical setting to allow post-bronchodilator spirometric assessment. It is important to realise that the prevalence of COPD could be higher in the cohort studied than would be anticipated in general population samples, which may have increased the positive predictive value of the questionnaire.

The number of patients with GOLD stage III–IV is low, which is not surprising, because the participants were individuals attending the participating hospitals for a nonrespiratory cause. The tool is designed for screening and is expected to detect individuals who are still undiagnosed, most of whom will have a mild or moderate disease. It is not expected to be used for patients with advanced disease, who will be symptomatic and should have spirometry performed directly.

According to the results, a higher CODE score can generally be interpreted as indicating an increased likelihood of COPD. By contrast, none of the subjects with 0 points had COPD. The selection of a particular cut-point value can be adapted for a particular application. For example, if we consider in the questionnaire a cut-off of above 3 points we observe a sensitivity of 79.2% and a specificity of 46.5%; by contrast, a cut-off of above 4 points has a sensitivity of 56.2% and a specificity of 72.2%. If the goal is to identify as many individuals with potential COPD as possible, with less concern about including those who may not have the disease, a score corresponding to high sensitivity and lower specificity can be considered. This could result in significant healthcare resource expenditure. Nevertheless, it has been suggested that all smokers, regardless of spirometry results, should be helped to quit smoking [22]. The relationship between knowledge of lung function and smoking cessation is unclear, with some studies suggesting smokers are not more likely to successfully quit smoking when faced with abnormal spirometry results [23, 24]. New studies refute this conclusion. For example, in the study by Parkes et al. [25] all patients (smokers aged ≥35 years) had spirometry and equal exposure to cessation resources, but the intervention group who were told their lung age had validated cessation rates more than double those of the control group (13.6% versus 6.4%). If an application requires a higher level of certainty that all identified individuals do have COPD, a cut-off score associated with higher specificity is desirable. As such, the optimal scoring used should be adapted to the situation and take into account the healthcare utilisation implications of this decision.

Whereas spirometry is the only test that confirms the diagnosis of COPD, and should be carried out in people exposed to risk factors and who are symptomatic, it exceeds the capacity of healthcare systems and is very expensive. The CODE questionnaire can determine the risk of airflow obstruction because scores of 0–2 points have a low risk of suffering, and scores of 5 and 6 points have a higher chance of diagnosis of COPD. The questionnaire can help better identify subjects at high risk of COPD in order to perform lung function tests with a higher pre-test positive predictive value. It is not a tool to substitute for spirometry in the diagnosis of COPD.

Validation studies are necessary to confirm the performance of the CODE questionnaire among individuals in other populations. This will require a larger sample size randomly divided into a development and a validation cohort. Subsequent validation would need a study of external validity and generalisability in a different population and different settings.

Future studies may evaluate whether inclusion of additional items can improve the performance of the questionnaire. Cognitive interviews or focus groups with individuals from the general population may provide additional information about the acceptability and reliability of the CODE questionnaire. We conclude that the CODE questionnaire can be a simple and valuable method to detect people at risk of COPD.