Abstract
Background
A seven-item prescreening questionnaire (gender, age, pack-years smoking, dyspnea, sputum, cough, previous spirometry data) was developed for COPD detection in the primary care setting (PUMA Study) of four Latin America countries.
Objectives
To validate the PUMA prescreening COPD questionnaire externally in two different populations (primary care and general).
Methods
The PUMA prescreening COPD questionnaire score was applied to subjects from the Hospital Maciel, Montevideo (primary care), case-finding program and the PLATINO population (general) using PUMA study inclusion criteria. Post-bronchodilator FEV1/FVC <0.70 and lower limit of normal (LLN) criteria were used to define COPD. Area under the received operator curve (ROCAUC), sensitivity, specificity, predictive positive and negative values (PNV), number needed to treat (NNT), and best cut-points of the score were calculated.
Results
974 individuals from Hospital Maciel and 2512 from the PLATINO population were eligible, using post-bronchodilator FEV1/FVC <0.70, 45.1% and 18.7% had COPD, respectively, and using LLN 38.4% and 15.4% had COPD, respectively. From Hospital Maciel (post-bronchodilator FEV1/FVC <0.70), the best cut-point of ≥6 had moderate discriminatory power (ROCAUC 0.70), sensitivity 69.9%, specificity 62.1%, PNV 70.9%, and NNT of 3. The discriminatory power was 0.73 (ROCAUC) in the PLATINO population with three potential cut-points (Youden’s index): ≥3 (sensitivity 85.4%, specificity 46.9%), ≥4 (sensitivity 66.7%, specificity 66.5%), and ≥5 (sensitivity 51.5%, specificity 81.6%); the PNV at each cut-point was 93.3%, 89.9%, and 88.0%, respectively. The NNT was 5 for scores ≥3 and ≥4, and 4 for ≥5. The mean accuracy using the LLN for Hospital Maciel and PLATINO was 0.67 and 0.70, respectively.
Conclusion
External validation of the PUMA prescreening questionnaire in two Latin American populations (general and primary care) suggests moderate accuracy, similar to the original study in which the questionnaire was developed.
Keywords: COPD, diagnostic questionnaire, primary care, validation, Latin America
Introduction
COPD is a common condition that leads to physical disability and represents a major health problem worldwide.
Several studies have evaluated the prevalence of COPD underdiagnosis in different settings.1–4 Lamprecht et al evaluated the underdiagnosis of COPD using different population-based studies and reported that approximately 81% of COPD cases remain undiagnosed and that there is considerable variation between the locations.2 In contrast, Casas et al evaluated a high-risk primary care population from Latin America and found COPD underdiagnosis in 77% of cases when using a definition of post-bronchodilator FEV1/FVC<0.70, and underdiagnosis in 73% of cases when using a lower limit of normal (LLN) for FEV1/FVC definition.3 Underdiagnosis or non-diagnosis of COPD is important as these patients have significant limitations in their level of physical activity, reduced quality of life, increased mortality and also contribute considerably to the burden of health care.5–8 Therefore, the approach of trying to identify the underdiagnosis of COPD is essential in order to reduce the burden of the disease, delay disease progression and improve health through smoking cessation support.
Primary care is an ideal setting for implementing search strategies to detect COPD cases. However, general practitioners (GPs) have to handle multiple diseases; therefore, the availability of a simple tool to identify COPD cases is extremely useful for these physicians.
Different risk factors and symptom-based prescreening questionnaires have been developed for the early detection of COPD. These questionnaires aid the selection of patients for spirometry screening to confirm definitively the diagnosis of COPD.
Even though there are validated COPD prescreening questionnaires9–13 coming from developed countries they may not be applicable in the primary care or general population of developing countries such as those evaluated in the present study.
In an at-risk population visiting the primary care setting in four Latin American countries, Lopez et al assessed the accuracy of a prescreening score developed to detect COPD (the PUMA study).14 The simple score had an accuracy of 76% for detecting COPD (using a definition of post-bronchodilator FEV1/FVC <0.70). To date, there has not been an external evaluation of this prescreening questionnaire.
The aim of this study was to validate externally the PUMA study prescreening COPD questionnaire in two different populations from Latin America and to compare its performance with the findings from the original PUMA study.
Methods
Development of the PUMA questionnaire in the original sample
The development of the seven-item COPD diagnostic PUMA questionnaire has been described in detail elsewhere.14 Briefly, PUMA was a multicenter, multinational, cross-sectional, non-interventional study conducted in the primary care setting of four Latin American countries: Argentina, Colombia, Venezuela, and Uruguay. At-risk patients completed the interview and performed acceptable post-bronchodilator spirometry. The simple PUMA prescreening questionnaire assessed risk factors and respiratory symptoms (gender, age, smoking pack-years, dyspnea, sputum, cough, and previous spirometry data) to select high-risk patients for spirometry screening.14 The score could range from 0 to 9 (higher scores indicating higher COPD risk) with 0–2 points assigned for each category. The points applied to each variable in the original PUMA questionnaire are described in detail elsewhere14 and a modified version is shown in Table S1. The best cut-point of the simple score according to Youden’s index to discriminate between persons with and without COPD was ≥5 points and the discriminatory power represented by the received operator curve (ROC) was 0.76.14
Table S1.
Points applied for each variable in the PUMA questionnaire
| Variable | Score | |
|---|---|---|
| Sex | Female | 0 |
| Male | 1 | |
| Age (years) | 40–49 | 0 |
| 50–59 | 1 | |
| 60+ | 2 | |
| Smoking history (pack-years) | <20 | 0 |
| 20–30 | 1 | |
| >30 | 2 | |
| Dyspnea | No | 0 |
| Yes | 1 | |
| Chronic phlegm | No | 0 |
| Yes | 1 | |
| Chronic cough | No | 0 |
| Yes | 1 | |
| Spirometry history performed during life | No | 0 |
| Yes | 1 |
External validation samples
Patient data used for validation of the prescreening questionnaire were taken from the PLATINO population-based study15 and from the Hospital Maciel, Montevideo, primary care case-finding program; in both population samples, those patients fulfilling the original PUMA study inclusion criteria (≥40 years, current or ex-smokers [≥10 pack-years, ≥50 pipes/year or ≥50 cigars/year], and/or exposure to biomass smoke [wood or coal, for cooking or heating; exposure ≥100 hr/year]) were selected.14 In the Hospital Maciel sample, the criterion of exposure to biomass was not used because these data were not collected. For both the PLATINO population-based and the Hospital Maciel samples, the study designs were reviewed and approved, by the local Institutional Review Boards and Ethics committees, the studies were performed in accordance with the Declaration of Helsinki, and all patients provided written informed consent prior to participating.
Hospital Maciel, Montevideo, sample
The Hospital Maciel, Montevideo, implemented a case-finding COPD program from January 2010 to December 2015. The main objective of this program was to increase the availability of spirometry and enhance COPD diagnosis at the primary care level. The selection of patients for spirometry was based on GPs criteria for possible COPD. Each week, a respiratory technician from the pulmonary function laboratory of the Hospital Maciel attended the primary care centers to perform spirometry in those patients identified by the GPs. Patients completed a short risk factor and respiratory symptoms questionnaire that included questions on gender, smoking habits, respiratory symptoms, and prior spirometric testing. Spirometry tests were performed using a portable, ultrasound Easy One spirometer (ndd Medical Technologies Inc., Zurich, Switzerland) at baseline and 15 mins after inhalation of a bronchodilator (400 μg salbutamol), according to the American Thoracic Society criteria. A total of 1091 subjects completed the questionnaire and spirometry.
PLATINO study sample
The PLATINO study was a population-based survey performed in five Latin American cities; a total of 5314 subjects completed the interview and recorded spirometric measurements using a portable spirometer (Easy One spirometer, ndd Medical Technologies, Inc.) at baseline and 15 mins after inhalation of salbutamol 200 μg, according to the American Thoracic Society criteria of acceptability and reproducibility. Complete details of the methodology, participation rates and patients’ characteristics of the PLATINO study have been published elsewhere.16 Data were collected on several factors potentially associated with COPD, including gender, smoking habits, exposure to biomass smoke, respiratory symptoms, and prior spirometric testing.16 Copies of the questionnaires are available at the PLATINO website (http://www.platino-alat.org).
COPD definition
Definition and severity stratification of COPD proposed by GOLD was used: post-bronchodilator FEV1/FVC <0.70. We also performed parallel analyses using the post-bronchodilator LLN for FEV1/FVC as a criterion to define COPD.16
Statistical analysis
The characteristics of the patient populations in the two studies were described as absolute values and relative frequencies. The simple score was applied and then we calculated the area under the ROC and also the sensitivity, specificity, predictive positive value (PPV), predictive negative value (PNV), and the number needed to treat (NNT) for each point in the score. The Youden index (the sum of sensitivity and specificity-1) was used to determine the best cut-point. We calculated the power of our analyses, considering the obtained results, using the routine rocsize in Stata. For the study in Hospital Maciel, our sample had a power of 99.8% for the cut-off score of six points. In the PLATINO study, we had a power of 100% to estimate the parameters in any of the three major cut-off points (between a score of 3 and 5). All analyses were done using Stata 13.0 (StataCorp. 2013, Stata Statistical Software: Release 13, StataCorp LP, College Station, TX, USA).
Results
A total of 974 eligible individuals completed interviews and performed acceptable spirometry in the Hospital Maciel, Montevideo sample and 2512 subjects from the PLATINO baseline study (all sites) were eligible for the analysis. The flow chart for the patients is shown in Figure 1. Of these, 439/974 (45.1%) subjects in the Hospital Maciel sample and 450/2512 (18.7%) subjects from the PLATINO sample had COPD defined by post-bronchodilator FEV1/FVC <0.70 (Figure 2). Using the LLN criteria, 374/974 (38.4%) subjects in the Hospital Maciel sample and 364/2512 (15.4%) participants in the PLATINO study had COPD (Figure 2).
Figure 1.
The flow chart for the patients screening.
Figure 2.
Prevalence of COPD according to different criteria (post-bronchodilator FEV1/FVC <0.70 and post-bronchodilator LLN for FEV1/FVC) using the Hospital Maciel, Montevideo, sample and the PLATINO study baseline population. Abbreviation: LLN, lower limit of normal.
Description of sample characteristics according to each population sample is shown in Table 1. In the Hospital Maciel sample, there was a higher proportion of women than men, nearly half of the subjects were ≥60 years of age, 64% were heavy smokers (>30 pack-years), about 75% of subjects reported respiratory symptoms (dyspnea, cough, and phlegm) and previous spirometry had only been performed in about 8% of subjects. In the PLATINO study, the proportion of men was slightly higher than women, around one-third of subjects were ≥60 years of age, subjects smoked less (about 60% smoked <20 pack-years), dyspnea was the most common symptom (~47%), only about 25% of subjects reported cough and phlegm, and previous spirometry had only been performed in 13% of subjects.
Table 1.
Description of sample characteristics for the Hospital Maciel, Montevideo, and the PLATINO study samples
| Characteristics | Hospital Maciel, Montevideo | PLATINO study |
|---|---|---|
| n (%) [N=974] | n (%) [N=2512] | |
| Sex | ||
| Female | 531 (54.5) | 1225 (48.8) |
| Male | 443 (45.5) | 1287 (51.2) |
| Age (complete years) | ||
| 40–49 | 141 (14.5) | 972 (39.1) |
| 50–59 | 349 (35.8) | 755 (60.4) |
| 60+ | 484 (49.7) | 758 (30.5) |
| Pack-years smoked during life | ||
| <20 | 214 (22.0) | 1479 (59.2) |
| 20–30 | 139 (14.3) | 401 (16.1) |
| >30 | 621 (63.7) | 617 (24.7) |
| Dyspnea | ||
| No | 220 (22.6) | 1315 (52.9) |
| Yes | 754 (77.4) | 1171 (47.1) |
| Chronic phlegm | ||
| No | 343 (35.2) | 1914 (76.2) |
| Yes | 631 (64.8) | 597 (23.8) |
| Chronic cough | ||
| No | 225 (23.1) | 1866 (74.3) |
| Yes | 749 (76.9) | 645 (25.7) |
| Previous spirometry performed during life | ||
| No | 892 (91.6) | 2178 (86.8) |
| Yes | 82 (8.4) | 333 (13.2) |
Note: Maximum missing values for PLATINO are for variable age (n=27).
Description of COPD individuals’ characteristics from population sample according to the post-bronchodilator FEV1/FVC <0.70 and the LLN definitions are shown in Tables 2 and 3, respectively. Using the post-bronchodilator FEV1/FVC <0.70 definition, COPD patients in the Hospital Maciel sample were more likely to be men, approximately 60% were ≥60 years of age, heavy smokers (70% smoked >30 pack-years), with moderate to very severe airway obstruction (GOLD ≥2) and respiratory symptoms reported in >70% of patients (Table 2). Previous spirometry was recorded in only 11% of patients. In the PLATINO sample, COPD subjects were more likely to be men, ≥60 years of age (58%), there were fewer smokers (41% smoked <20 pack-years), with mild to moderate airway obstruction (GOLD 1–2) (Table 2). Dyspnea was the most common symptom (50%), while less than half reported cough and phlegm (~35%). Previous spirometry was recorded in only 22% of patients (Table 2). Similar results were found in both population samples for COPD defined using the post-bronchodilator LLN FEV1/FVC definition (Table 3).
Table 2.
Description of sample characteristics for the Hospital Maciel, Montevideo, and the PLATINO study samples in individuals with COPD defined using the post-bronchodilator FEV1/FVC <0.70 definition
| Variable | Hospital Maciel, Montevideo | PLATINO study |
|---|---|---|
| n (%) [N=439] | n (%) [N=450] | |
| Sex | ||
| Female | 191 (43.5) | 166 (36.9) |
| Male | 248 (56.5) | 284 (63.1) |
| Age (complete years) | ||
| 40–49 | 35 (8.0) | 67 (14.9) |
| 50–59 | 138 (31.4) | 122 (27.1) |
| 60+ | 266 (60.6) | 261 (58.0) |
| Pack-years smoked during life | ||
| <20 | 67 (15.3) | 185 (41.1) |
| 20–30 | 62 (14.1) | 84 (18.7) |
| >30 | 310 (70.6) | 181 (40.2) |
| Dyspnea | ||
| No | 81 (18.5) | 222 (49.9) |
| Yes | 358 (81.5) | 223 (50.1) |
| Chronic phlegm | ||
| No | 111 (25.3) | 298 (66.2) |
| Yes | 328 (74.7) | 152 (33.8) |
| Chronic cough | ||
| No | 76 (17.3) | 280 (62.2) |
| Yes | 363 (82.7) | 170 (37.8) |
| Previous spirometry performed during life | ||
| No | 390 (88.8) | 351 (78.0) |
| Yes | 49 (11.2) | 99 (22.0) |
| GOLD 2007 classification | ||
| I | 83 (18.9) | 247 (54.9) |
| II | 236 (53.7) | 171 (38.0) |
| III | 100 (22.8) | 25 (5.6) |
| IV | 20 (4.6) | 7 (1.56) |
Table 3.
Description of sample characteristics for the Hospital Maciel, Montevideo, and the PLATINO study samples in individuals with COPD defined using the post-bronchodilator LLN FEV1/FVC definition
| Variable | Hospital Maciel, Montevideo | PLATINO study |
|---|---|---|
| n (%) [N=374] | n (%) [N=364] | |
| Sex | ||
| Female | 169 (45.2) | 132 (36.3) |
| Male | 205 (54.8) | 232 (63.7) |
| Age (complete years) | ||
| 40–49 | 36 (9.6) | 84 (23.1) |
| 50–59 | 134 (35.8) | 106 (29.1) |
| 60+ | 204 (54.6) | 174 (47.8) |
| Pack years smoked during life | ||
| <20 | 55 (14.7) | 147 (40.4) |
| 20–30 | 52 (13.9) | 70 (19.2) |
| >30 | 267 (71.4) | 147 (40.4) |
| Dyspnea | ||
| No | 70 (18.7) | 185 (51.3) |
| Yes | 304 (81.3) | 176 (48.7) |
| Chronic phlegm | ||
| No | 89 (23.8) | 226 (62.1) |
| Yes | 285 (76.2) | 138 (37.9) |
| Chronic cough | ||
| No | 60 (16.0) | 220 (60.4) |
| Yes | 314 (84.0) | 144 (39.6) |
| Previous spirometry performed during life | ||
| No | 333 (89.0) | 280 (76.9) |
| Yes | 41 (11.0) | 84 (23.1) |
The area under the curve (AUCs) of the PUMA simple score with COPD as the outcome are shown in Figure 3A and B using the post-bronchodilator FEV1/FVC <0.70 and post-bronchodilator LLN FEV1/FVC definitions, respectively. Sensitivity, specificity, PPV, PNV, and NNT for each cut-point of the PUMA simple score are shown in Table 4. The mean accuracy (AUCs) of the PUMA simple score for detecting COPD using post-bronchodilator FEV1/FVC <0.70 definition for the Hospital Maciel and PLATINO samples was 0.70 and 0.73 (Figure 3A), respectively. Using the LLN definition for the Hospital Maciel and PLATINO samples, the mean accuracy was 0.67 and 0.70, respectively (Figure 3B). The best cut-point according to Youden’s index (sensitivity + specificity −1) for the PUMA simple score was ≥6 in the Hospital Maciel sample and between 3 and 5 in the PLATINO sample, for both definitions. Using the post-bronchodilator FEV1/FVC <0.70 definition, a subject having a score of <6 in the Hospital Maciel sample has a 70.9% chance of not having COPD (Table 4). In the PLATINO sample, a subject having a score of <5 has an 88.0% chance of not having COPD (Table 4). Similar results were observed with the LLN definition (Table 4).
Figure 3.
Area under the ROC for score (1 point for each category variable) with COPD as outcome using: (A) post-bronchodilator FEV1/FVC <0.70 and (B) post-bronchodilator LLN for FEV1/FVC as definitions for the Hospital Maciel, Montevideo, the PLATINO study, and the original PUMA study baseline populations as samples.
Abbreviations: LLN, lower limit of normal; ROC, received operator curve.
Table 4.
Sensitivity, specificity, PPV, PNV for each cut-point of proposed score (1 point for each category variable) using Hospital Maciel, Montevideo, and PLATINO study samples
| Score | Sensitivity | Specificity | PPV | PNV | NNT |
|---|---|---|---|---|---|
| % (95% CI) | % (95% CI) | % (95% CI) | % (95% CI) | ||
| Post-bronchodilator FEV1/FVC <0.70 definition as outcome | |||||
| Hospital Maciel, Montevideo | |||||
| ≥1 | – | – | – | – | – |
| ≥2 | 99.3 (98.0; 99.9) | 0.9 (0.3; 2.2) | 45.1 (42.0; 48.3) | 62.5 (24.5; 91.5) | 13 |
| ≥3 | 97.5 (95.6; 98.7) | 6.9 (4.9; 9.4) | 46.2 (43.0; 49.5) | 77.1 (62.7; 88.0) | 4 |
| ≥4 | 95.2 (92.8; 97.0) | 18.9 (15.6; 22.5) | 49.1 (45.7; 52.5) | 82.8 (74.9; 89.0) | 3 |
| ≥5 | 85.4 (81.8; 88.6) | 37.6 (33.5; 41.8) | 52.9 (49.1; 56.6) | 75.8 (70.2; 80.9) | 3 |
| ≥6 | 69.0 (64.5; 73.3) | 62.1 (57.8; 66.2) | 59.9 (55.5; 64.2) | 70.9 (66.6; 75.0) | 3 |
| ≥7 | 44.6 (39.9; 49.4) | 82.6 (79.1; 85.7) | 67.8 (62.1; 73.2) | 64.5 (60.8; 68.1) | 3 |
| ≥8 | 21.2 (17.5; 25.3) | 94.0 (91.7; 95.9) | 74.4 (65.8; 81.8) | 59.2 (55.9; 62.6) | 3 |
| ≥9 | 2.3 (1.1; 4.2) | 100.0 (99.3; 100.0) | 100.0 (69.2; 100.0) | 55.5 (52.3; 58.7) | 2 |
| PLATINO study | |||||
| ≥1 | 98.4 (96.8; 99.4) | 7.3 (6.2; 8.6) | 19.6 (18.0; 21.3) | 95.3 (90.6; 98.1) | 7 |
| ≥2 | 93.0 (90.3; 95.2) | 25.2 (23.2; 27.2) | 22.2 (20.4; 24.2) | 94.0 (91.6; 95.9) | 6 |
| ≥3 | 85.4 (81.8; 88.5) | 46.9 (44.7; 49.2) | 27.0 (24.7; 29.4) | 93.3 (91.6; 94.8) | 5 |
| ≥4 | 66.7 (63.1; 72.0) | 66.5 (64.4; 68.6) | 31.7 (28.8; 34.8) | 89.9 (88.3; 91.5) | 5 |
| ≥5 | 51.5 (46.7; 56.2) | 81.6 (79.8; 83.3) | 39.1 (35.1; 43.2) | 88.0 (86.4; 89.4) | 4 |
| ≥6 | 31.5 (27.2; 36.0) | 91.3 (89.9; 92.5) | 45.3 (39.7; 51.0) | 85.3 (83.7; 86.8) | 3 |
| ≥7 | 13.5 (10.4; 17.0) | 97.6 (96.8; 98.2) | 56.1 (46.1; 65.7) | 83.1 (81.5; 84.6) | 3 |
| ≥8 | 6.3 (4.2; 9.0) | 99.6 (99.2; 99.8) | 77.8 (60.8; 89.9) | 82.2 (80.6; 83.7) | 2 |
| ≥9 | 1.4 (0.5; 2.9) | 99.9 (99.7; 100.0) | 85.7 (42.1; 99.6) | 81.5 (79.9; 83.1) | 1 |
| Post-bronchodilator LLN FEV1/FVC definition as outcome | |||||
| Hospital Maciel-Montevideo | |||||
| ≥1 | – | – | – | – | – |
| ≥2 | 98.9 (97.3; 99.7) | 0.7 (0.2; 1.7) | 38.3 (35.2; 41.5) | 50.0 (15.7; 84.3) | – |
| ≥3 | 96.8 (94.5; 98.3) | 6.0 (4.2; 8.2) | 39.1 (35.9; 42.3) | 75.0 (60.4; 86.4) | 7 |
| ≥4 | 94.4 (91.5; 96.5) | 16.8 (13.9; 20.1) | 41.4 (38.1; 44.8) | 82.8 (74.9; 89.0) | 4 |
| ≥5 | 83.2 (79.0; 86.8) | 33.7 (29.9; 37.6) | 43.9 (40.2; 47.6) | 76.2 (70.6; 81.2) | 5 |
| ≥6 | 68.4 (63.5; 73.1) | 58.3 (54.3; 62.3) | 50.6 (46.1; 55.0) | 74.8 (70.6; 78.7) | 4 |
| ≥7 | 43.9 (38.8; 49.0) | 79.1 (75.7; 82.3) | 56.7 (50.8; 62.5) | 69.3 (65.7; 72.8) | 4 |
| ≥8 | 21.1 (17.1; 25.6) | 92.3 (89.9; 94.3) | 63.2 (54.1; 71.6) | 65.3 (61.9; 68.5) | 4 |
| ≥9 | 2.1 (0.9; 4.2) | 99.7 (98.8; 100.0) | 80.0 (44.4; 97.5) | 62.0 (58.9; 65.1) | 2 |
| PLATINO study | |||||
| ≥1 | 97.8 (95.7; 99.0) | 7.1 (6.0; 8.3) | 16.1 (14.6; 17.8) | 94.6 (89.6; 97.6) | 9 |
| ≥2 | 90.3 (86.8; 93.2) | 24.3 (22.4; 26.2) | 17.9 (16.2; 19.7) | 93.2 (90.7; 95.2) | 9 |
| ≥3 | 82.0 (77.6; 85.8) | 45.3 (43.1; 47.5) | 21.5 (19.4; 23.8) | 93.2 (91.4; 94.7) | 7 |
| ≥4 | 64.3 (59.1; 69.2) | 64.7 (62.6; 66.9) | 25.0 (22.2; 27.9) | 90.8 (89.2; 92.3) | 6 |
| ≥5 | 49.0 (43.8; 54.3) | 80.1 (78.3; 81.9) | 31.1 (27.3; 35.1) | 89.6 (88.1; 91.0) | 5 |
| ≥6 | 30.7 (26.0; 35.8) | 90.3 (88.9; 91.6) | 36.8 (61.3; 42.5) | 87.7 (86.2; 89.1) | 4 |
| ≥7 | 14.1 (10.7; 18.2) | 97.3 (96.5; 98.0) | 49.0 (39.1; 59.0) | 86.1 (84.6; 87.5) | 3 |
| ≥8 | 7.2 (4.8; 10.4) | 99.6 (99.2; 99.8) | 76.5 (58.8; 89.3) | 85.4 (83.9; 86.9) | 2 |
| ≥9 | 1.7 (0.6; 3.6) | 99.9 (99.7; 100.0) | 85.7 (42.1; 99.6) | 84.8 (83.2; 86.2) | 1 |
Abbreviations: PPV, predictive positive value; PNV, predictive negative value.
Discussion
In the present study, the validity of the recently developed PUMA COPD prescreening questionnaire was evaluated in two different patient samples: 974 at-risk patients from the Hospital Maciel primary care COPD case-finding program, and 2512 subjects from the PLATINO population-based study. The discriminative capacity of the PUMA questionnaire in both populations was similar to that found in the original study: using the COPD definition of post-bronchodilator FEV1/FVC <0.70, the ROCAUC was 0.70 and 0.73 for the Hospital Maciel and the PLATINO samples, respectively; using the LLN definition, the discriminative capacity of the questionnaire was lower at 0.67 and 0.70, respectively.
There are three externally validated prescreening questionnaires that assess risk factors and respiratory symptoms to select high-risk patients for spirometry screening: the COPD Diagnostic Questionnaire (CDQ),9 the Lung Function Questionnaire (LFQ),10,11 and the COPD Population Screener (COPD-PS).12 The predictive accuracy of these questionnaires in external validation studies was measured against post-bronchodilator FEV1/FVC <0.70 and ranged from 57% to 72%.13
The CDQ is a symptom- and risk factor-based COPD prescreening questionnaire that was performed in a primary care-based US and UK.9 Possible scores range from 0 to 38 (higher scores associated with higher COPD risk). Two cut-points (16.5 and 19.5) have been proposed to select patients for spirometry screening based on ROCAUC curves from the original development study.9,17 The CDQ has been externally validated in five studies focused primarily on European and Australian primary care populations.18–22 Most of the validation studies reported that a CDQ score of >16.5 had a sensitivity ranging from 80% to 91% and a specificity ranging between 24% and 49% for identifying those subjects who were subsequently found positive for COPD by spirometry testing.13 The use of a higher cut-point of 19.5 reduced sensitivity and PNV, but increased specificity and PPV. The discriminative ability of the CDQ to identify patients with and without COPD ranges from 0.65 to 0.72 (ROCAUC).
The LFQ and the COPD-PS include risk factors and respiratory symptoms.10–12 Scores range from 5 to 25, with lower scores associated with an increased risk of COPD. A threshold of ≤18 has been proposed as the cut-point for the risk of COPD that justifies using spirometry to confirm the diagnosis. This questionnaire was validated in a multicenter primary care study in 1288 smokers of at least 10 pack-years of exposure.23 The study reported a high unacceptable spirometry rate (31%), as well as an estimated sensitivity of 88% and specificity of 25%.
The COPD-PS assigns scores to age, history of smoking, dyspnea, sputum production and functional limitations related to dyspnea. The score can range from 0 to 10.12 A threshold of ≥5 has been proposed as a cut-point for the risk of COPD that justifies evaluation with spirometry. This questionnaire was validated in a Japanese population study (n=2357) and reported a sensitivity of 67% and specificity of 73% for a score ≥4, and 35% and 79%, respectively, for a score ≥5.24 The discriminative ability (ROCAUC) of the COPD-PS questionnaire to identify patients with and without COPD was 0.70 (score ≥4) and 0.57 (score ≥5).24
The PUMA questionnaire assigns scores to gender, age, smoking history (pack-years), dyspnea, sputum, cough and data from prior spirometry. The best cut-point of ≥5 had a moderate discriminatory power ROCAUC of 0.76 with a sensitivity and specificity of 74% and 65%, respectively.14 A subject with a score of <5 has a 91% (PNV) chance of not having COPD. The use of the PUMA score to identify patients with and without COPD has not been validated in other populations.
The present study evaluated the discriminatory capacity of the PUMA questionnaire score in individuals from two different settings (primary care and general population) using the original PUMA study inclusion criteria. The results from the Hospital Maciel, Montevideo (primary care sample), showed that the best cut-point of ≥6 had a moderate discriminatory power ROCAUC of 0.70, a sensitivity of 69.9%, a specificity of 62.1%, a PNV of 70.9% and a NNT of 3 (one person diagnosed with COPD for every three persons assessed spirometrically). When the questionnaire was applied to the general population sample from the PLATINO study the discriminatory power was slightly better with a ROCAUC of 0.73 and with three potential cut-points according to the Youden’s index: ≥3 (sensitivity 85.4%, specificity 46.9%), ≥4 (sensitivity 66.7%, specificity 66.5%), and ≥5 (sensitivity 51.5%, specificity 81.6%); the chances for not having COPD (PNV) at each cut-point were 93.3%, 89.9% and 88.0%, respectively. The NNT was 5 for scores ≥3 and ≥4, and it was 4 for a score ≥5.
This validation study, unlike the external validations of other questionnaires (CDQ, LFQ and COPD-PS)18–24 found a discriminatory capacity that was similar to that reported in the original PUMA study, both in the primary care and in the general population samples. The results also show a performance of the PUMA score in this external validation in a similar range to the other prescreening COPD questionnaires (CDQ accuracy 0.65–0.73; LFQ accuracy 0.65; and COPD-PS accuracy 0.57–0.70).18–24
This study has both strengths and limitations that need to be mentioned. The two populations used to validate the PUMA prescreening COPD questionnaire used the same inclusion criteria of the original study, with the exception of biomass in the Hospital Maciel sample. A particular strength of the study is that both the primary care and general population samples are large, and using two different population samples increases the spectrum of validity of the PUMA score. In addition to the lack of information on biomass in Hospital Maciel, other limitations are that the study validation samples included patients with and without respiratory symptoms, and this may underestimate the ability of the PUMA score to detect cases of COPD as defined by the GOLD document screening criteria (risk factors plus respiratory symptoms).16 This limitation also applies to the other prescreening COPD questionnaire studies that looked to detect spirometrically defined COPD in patients with risk factors but who were not necessarily symptomatic. Given the controversy of detecting asymptomatic airflow obstruction, this could be considered as an advantage of the prescreening COPD questionnaire in that asymptomatic patients would be less likely to score above the validated cut-points. Further research is needed to determine if the PUMA score performs better as a prescreening tool for COPD only in subjects meeting the GOLD screening criteria. In addition, both populations in the present validation come from Latin America, which limits the extrapolation of our results worldwide.
In summary, the results of this external validation study of the PUMA prescreening COPD questionnaire suggest moderate accuracy for detecting COPD using post-bronchodilator FEV1/FVC <0.70 definition (discriminatory power 70–73%) in subjects ≥40 years of age with risk factors for COPD (smoking history and/or exposure to biomass smoke) from Latin America in a primary care or general population setting. Further research is needed on the performance of the PUMA score as a prescreening tool for COPD in other populations around the world.
Supplementary material
Acknowledgments
We are grateful to the other investigators of the PLATINO team, namely, Rogelio Perez-Padilla, Adriana Muiño, Gonzalo Valdivia, and Jose Roberto B. Jardim. This observational study was funded by AstraZeneca Latin America. AstraZeneca had no input into the study design, analysis and interpretation of the result.
Data sharing statement
All relevant data are within the paper.
Author contributions
Conceived and designed the experiments: MVLV, MMO, AMM; performed the experiments: MVLV, MMO, CR, LR, AMM; analyzed the data: FCW, AMM; contributed reagents/materials/analysis tools: MVLV, MMO, CR, LR, AMM. MVLV, MMO, and AMM led the writing of the article but all authors provided input into revising the article and gave final approval of the version to be published, and agree to be accountable for all aspects of the work.
Disclosure
Larissa Ramirez is a full-time employee of AstraZeneca Latin America. Ana Menezes has been paid for her work as a statistician for the study.
The Hospital Maciel, Montevideo, data of this paper were presented at the European Respiratory Society 2017 Congress as a poster presentation with interim findings. The poster’s abstract was published in the European Respiratory Journal 2017; 50 (Suppl 61): PA1198. DOI: 10.1183/1393003. The authors report no other conflicts of interest in this work.
References
- 1.Bednarek M, Maciejewski J, Wozniak M, Kuca P, Zielinski J. Prevalence, severity and underdiagnosis of COPD in the primary care setting. Thorax. 2008;63:402–407. doi: 10.1136/thx.2007.085456 [DOI] [PubMed] [Google Scholar]
- 2.Lamprecht B, Soriano JB, Studnicka M, et al. Determinants of underdiagnosis of COPD in national and international surveys. Chest. 2015;148:971–985. doi: 10.1378/chest.14-2535 [DOI] [PubMed] [Google Scholar]
- 3.Casas Herrera A, Montes de Oca M, López Varela MV, Aguirre C, Schiavi E, Jardim JR. COPD underdiagnosis and misdiagnosis in a high-risk primary care population in four Latin American countries. A key to enhance disease diagnosis: the PUMA study. PLoS One. 2016;11(4):e0152266. doi: 10.1371/journal.pone.0152266 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Llordés M, Jaén A, Almagro P, et al. Prevalence, risk factors and diagnostic accuracy of COPD among smokers in primary care. COPD. 2015;12(4):404–412. doi: 10.3109/15412555.2014.974736 [DOI] [PubMed] [Google Scholar]
- 5.Miravitlles M, Soriano JB, García-Río F, et al. Prevalence of COPD in Spain: impact of undiagnosed COPD on quality of life and daily life activities. Thorax. 2009;64(10):863–868. doi: 10.1136/thx.2009.113506 [DOI] [PubMed] [Google Scholar]
- 6.Martinez CH, Mannino DM, Jaimes FA, et al. Undiagnosed obstructive lung disease in the United States. Associated factors and long-term mortality. Ann Am Thorac Soc. 2015;12(12):1788–1795. doi: 10.1513/AnnalsATS.201506-388OC [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Labonté LE, Tan WC, Li PZ, et al. Undiagnosed chronic obstructive pulmonary disease contributes to the burden of health care use. Data from the CanCOLD study. Am J Respir Crit Care Med. 2016;194(3):285–298. doi: 10.1164/rccm.201509-1795OC [DOI] [PubMed] [Google Scholar]
- 8.Gershon AS, Thiruchelvam D, Chapman KR, et al. Health services burden of undiagnosed and overdiagnosed COPD. Chest. 2018;153(6):1336–1346. doi: 10.1016/j.chest.2018.01.038 [DOI] [PubMed] [Google Scholar]
- 9.Price DB, Tinkelman DG, Halbert RJ, et al. Symptom-based questionnaire for identifying COPD in smokers. Respiration. 2006;73(3):285–295. doi: 10.1159/000090142 [DOI] [PubMed] [Google Scholar]
- 10.Hanania NA, Mannino DM, Yawn BP, et al. Predicting risk of airflow obstruction in primary care: validation of the lung function questionnaire (LFQ). Respir Med. 2010;104(8):1160–1170. doi: 10.1016/j.rmed.2010.02.009 [DOI] [PubMed] [Google Scholar]
- 11.Yawn BP, Mapel DW, Mannino DM, et al. Development of the Lung Function Questionnaire (LFQ) to identify airflow obstruction. Int J Chron Obstruct Pulm Dis. 2010;5:1–10. [PMC free article] [PubMed] [Google Scholar]
- 12.Martinez FJ, Raczek AE, Seifer FD, et al. Development and initial validation of a self-scored COPD Population Screener Questionnaire (COPD-PS). COPD. 2008;5(2):85–95. doi: 10.1080/15412550801940721 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Guirguis-Blake JM, Senger CA, Webber EM, Mularski RA, Whitlock EP. Screening for chronic obstructive pulmonary disease: evidence report and systematic review for the US preventive services task force. JAMA. 2016;315(13):1378–1393. doi: 10.1001/jama.2016.2654 [DOI] [PubMed] [Google Scholar]
- 14.López Varela MV, Montes de Oca M, Rey A, Casas A, Stirbulov R, Di Boscio V. Opportunistic COPD case-finding in primary care of four Latin America countries. Developing a simple screening tool: the PUMA study. Respirology. 2016;21(7):1227–1234. doi: 10.1111/resp.12834 [DOI] [PubMed] [Google Scholar]
- 15.Menezes AM, Perez-Padilla R, Jardim J, et al. Chronic obstructive pulmonary disease in five Latin American cities (the PLATINO study): a prevalence study. Lancet. 2005;366:1875–1881. doi: 10.1016/S0140-6736(05)67528-9 [DOI] [PubMed] [Google Scholar]
- 16.Vogelmeier CF, Criner GJ, Martinez FJ, et al. Global Strategy for the diagnosis, management, and prevention of chronic obstructive lung disease 2017 report. GOLD executive summary. Am J Respir Crit Care Med. 2017;195(5):557–582. doi: 10.1164/rccm.201701-0150WS [DOI] [PubMed] [Google Scholar]
- 17.Price DB, Tinkelman DG, Nordyke RJ, Isonaka S, Halbert RJ, COPD Questionnaire Study Group. Scoring system and clinical application of COPD diagnostic questionnaires. Chest. 2006;129(6):1531–1539. doi: 10.1378/chest.129.6.1531. [DOI] [PubMed] [Google Scholar]
- 18.Sichletidis L, Spyratos D, Papaioannou M, et al. A combination of the IPAG questionnaire and PiKo-6® flow meter is a valuable screening tool for COPD in the primary care setting. Prim Care Respir J. 2011;20(2):184–189. doi: 10.4104/pcrj.2011.00038 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Frith P, Crockett A, Beilby J, et al. Simplified COPD screening: validation of the PiKo-6® in primary care. Prim Care Respir J. 2011;20(2):190–198. doi: 10.4104/pcrj.2011.00040 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Stanley AJ, Hasan I, Crockett AJ, van Schayck OC, Zwar NA. Validation of the COPD diagnostic questionnaire in an Australian general practice cohort: a cross-sectional study. Prim Care Respir J. 2014;23(1):92–97. doi: 10.4104/pcrj.2014.00015 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Dirven JA, Tange HJ, Muris JW, van Haaren KM, Vink G, van Schayck OC. Early detection of COPD in general practice: implementation, workload and socioeconomic status. A mixed methods observational study. Prim Care Respir J. 2013;22(3):338–343. doi: 10.4104/pcrj.2013.00071 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Kotz D, Nelemans P, van Schayck CP, Wesseling GJ. External validation of a COPD diagnostic questionnaire. Eur Respir J. 2008;31(2):298–303. doi: 10.1183/09031936.00074307 [DOI] [PubMed] [Google Scholar]
- 23.Mintz ML, Yawn BP, Mannino DM, et al. Prevalence of airway obstruction assessed by lung function questionnaire. Mayo Clin Proc. 2011;86(5):375–381. doi: 10.4065/mcp.2010.0787 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Tsukuya G, Matsumoto K, Fukuyama S, et al. Validation of a COPD screening questionnaire and establishment of diagnostic cut-points in a Japanese general population: the Hisayama study. Allergol Int. 2015;64(1):49–53. doi: 10.1016/j.alit.2014.06.002 [DOI] [PubMed] [Google Scholar]