Performance of alternative COPD case-finding tools: a systematic review and meta-analysis

Abstract

Rationale

Guidelines recommend pre-/post-bronchodilator spirometry for diagnosing COPD, but resource constraints limit the availability of spirometry in primary care in low- and middle-income countries. Although spirometry is the diagnostic gold standard, we shall assess alternative tools for settings without spirometry.

Methods

A systematic literature review and meta-analysis was conducted, utilising Cochrane, CINAHL, Google Scholar, PubMed and Web of Science (search cut-off was May 01, 2020). Published studies comparing the accuracy of diagnostic tools for COPD with post-bronchodilator spirometry were considered. Studies without sensitivity/specificity data, without a separate validation sample and outside of primary care were excluded. Sensitivity, specificity and area under the curve (AUC) were assessed.

Results

Of 7578 studies, 24 were included (14 635 participants). Hand devices yielded a larger AUC than questionnaires. The meta-analysis included 17 studies and the overall AUC of micro-spirometers (0.84, 95% CI 0.80–0.89) was larger when compared to the COPD population screener (COPD-PS) questionnaire (0.77, 95% CI 0.63–0.85) and the COPD diagnostic questionnaire (CDQ) (0.72, 95% CI 0.64–0.78). However, only the difference between micro-spirometers and the CDQ was significant.

Conclusions

The CDQ and the COPD-PS questionnaire were approximately equally accurate tools. Questionnaires ensured testing of symptomatic patients, but micro-spirometers were more accurate. A combination could increase accuracy but was not evaluated in the meta-analysis.

Short abstract

Micro-spirometry testing in symptomatic patients could enable COPD diagnosis in settings without spirometry. Questionnaires are valuable, as they are easy to implement and test for symptomatic patients. A combination could increase diagnostic accuracy. https://bit.ly/3opHzUA

Introduction

COPD is uncurable but is to a large degree preventable, with treatable symptoms. Defining manifestations are dyspnoea, chronic cough, high sputum production and frequent lower respiratory tract infections [1]. In 2017, COPD prevalence was about 300 million cases worldwide, with a relative increase of 5.9% from 1990 and it ranked as the 7th disease in the global list of years of life lost [2–4]. COPD caused 3.2 million deaths in 2017, with an increase of 17.5% from 2007 [2] and 90% of COPD-related deaths occur in low- and middle-income countries (LMICs) [5].

A growing elderly population and an ongoing exposure to risk factors are reasons why prevalence is anticipated to increase further [1]. In general, a minimum of 25% of smokers that have smoked for 25 years might acquire a clinically significant COPD [6]. Additionally, indoor air pollution through burning of biomass plays a large role, especially in LMICs [7]. Asthma, old age, HIV infection, a history or current tuberculosis (TB) infection and genetic dispositions like α1-antitrypsin deficiency can further contribute to the course of COPD [1]. The social and economic burden created by COPD is substantial. Not only does COPD lead to a high degree of mortality, but the morbidity when living with severe COPD is also consequential. The direct healthcare expenditure on acute lung infections (exacerbations) is significant, but the indirect costs should be considered as well. Severely affected people are no longer able to work and often need care, causing a serious problem particularly in LMICs [1].

The gold standard in diagnosing airflow obstruction is pre- and post-bronchodilator spirometry. Diagnosis of COPD is likely in symptomatic patients that have a post-bronchodilator forced expiratory volume in 1 s (FEV₁)/forced vital capacity (FVC) ratio <0.7 or less than the lower limit of normal (LLN) [1]. It has been shown, when excluding patients with a history of asthma, that an early diagnosis of COPD is associated with a better outcome in terms of exacerbation rates, direct healthcare costs and mortality [8]. As lung function rates decrease the fastest in the mild stages of COPD, early diagnosis is important in order to slow this decline with smoking cessation therapy and medication [9]. It has been demonstrated that long-acting bronchodilators slow disease progression in the mild, early stages of COPD and increase quality of life [10, 11]. However, COPD is often undiagnosed or diagnosed late as resource constraints in primary care limit the availability of spirometry in many LMICs [12].

Guidelines recommend spirometry testing for diagnosing COPD in symptomatic patients but leave out what to do when no spirometry is available [13]. Alternative diagnostic tools have been developed and could be helpful in reducing the rate of underdiagnosis [14]. This work summarises all the alternative tools that were compared to post-bronchodilator spirometry in primary care populations and measures their quality. A work similar to this one was conducted in 2013, but many new tools have been developed since then. We aim to inform about best practice in COPD case-finding in a primary care setting without spirometry, ensuring that physicians around the world have easy access to current evidence when encountering a symptomatic patient likely to have COPD.

Methods

Search strategy and selection criteria

A systematic literature review was conducted to obtain an overview of the tools and to outline their sensitivity, specificity and area under the curve (AUC). Additionally, a meta-analysis was administered.

The PROSPERO registration number is: CRD42020197751 and this study adheres to the principles of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) [15]. Database searches and screening of titles, abstracts, full text and reference lists, as well as quality assessment, were completed independently by E. Schnieders and E. Ünal. Any discrepancies were resolved between all the authors. The systematic literature search considered research available up until May 01, 2020 in Cochrane, CINAHL, PubMed and Web of Science. Furthermore, results in Google Scholar were screened in groups of 50 until no more relevant articles could be found. Consequently, the first 400 hits from Google Scholar were screened. Peer-reviewed articles in English, French, German and Spanish were evaluated and there was no limitation regarding past publication dates.

The search terms were derived from three different categories: COPD, diagnosis and primary healthcare. For each of these categories, possible alternative words and medical subject heading (MeSH) terms were determined (see supplementary table E1).

According to the PICO scheme, all studies conducted in a primary healthcare cohort that compared an alternative and reproducible diagnostic tool for COPD to gold standard post-bronchodilator spirometry and measured the sensitivity and specificity of the tool, were included. Protocols were excluded and only tools that focused solely on diagnosing COPD or airflow obstruction, defined as post-bronchodilator FEV₁/FVC <0.7 or FEV₁/FVC <LLN according to the Global Initiative for Chronic Obstructive Lung Disease (GOLD) criteria [1], were considered. The studies had to report the post-bronchodilator spirometry test results for every participant or every participant with pre-bronchodilator obstruction. Finally, studies that did not validate their new tool in a sample different to the one in which it was developed were excluded.

The meta-analysis included tools that were evaluated in at least five studies, as the statistical package does not recommend a small number of studies and issues a warning when less than five studies are used [16].

Data analysis

All identified results were imported into Endnote X9 (Clarivate Analytics, Philadelphia, PA, USA) and, after the removal of duplicates, titles and abstracts were screened with Rayyan [17]. The reference lists of all the included studies were then screened for further eligible studies. Data was then extracted using an Excel spreadsheet for the following categories: study setting, time of data collection, tool, cut-off, sample size, sample criteria, sample size calculation, spirometry criteria, definition of COPD, COPD patients, smokers, sensitivity, specificity and AUC. In cases where there was a lack of clarity regarding the use of post-bronchodilator spirometry, study authors were contacted.

Quality assessment of the included studies was performed with the QUADAS tool, evaluating applicability and the risk of bias for the categories “patient selection”, “index test”, “reference test” and “flow and timing” [18]. If a category was assessed as “unclear” for bias, then relevant information to answer the signalling questions was missing (for details see supplementary table E2). Biased studies are mentioned in the discussion but were not excluded from the analysis, given the low number of studies available.

The 95% confidence intervals (CIs) for the AUC values in the systematic review were calculated using the Wilson interval and the corresponding forest plot was constructed with the “Forestplot” package for R software (The R Project for Statistical Computing, www.r-project.org). Additionally, the Youden index was calculated [19].

The coupled forest plot of studies in the meta-analysis was created with Review Manager version 5.3 (The Nordic Cochrane Centre, Copenhagen, Denmark). The meta-analysis was conducted with R software using a bivariate model [20]. For this model, the function “reitsma” from the R-package “mada” was applied [16]. The “mada” package calculated the summary receiver operating characteristic (ROC) curve and therefore the overall AUC of each tool. As multiple thresholds of one tool were assessed, calculating the overall sensitivity and specificity was not recommended [21]. The 95% CIs for the summary AUC values were calculated using bootstrap with the R-package “dmetatools”, which also tested for statistically significant differences between the summary AUC values [22, 23]. The heterogeneity of the sensitivities and specificities of each tool was assessed graphically with a scatterplot and the summary ROC curve.

Results

Study selection

Altogether, 7578 potentially eligible papers were identified (see supplementary figure E1). After removing duplicates, 4823 articles were included in title and abstract screening, and 72 papers qualified for full-text reading. Of these, 48 were excluded, mainly due to violation of the spirometry inclusion criteria. Additionally, reference lists of eligible studies were searched; however, no additional studies were found.

General study characteristics

Ultimately, 24 studies dating from 2006 until 2020 were included [24–47]. Tables 1–3 show study criteria and tool performance results for diagnostic hand devices (i.e. micro-spirometers), questionnaires and a combination of hand devices and questionnaires, respectively. The studies were conducted in 18 different countries that are all considered high-income countries, apart from Venezuela, which is an upper middle-income country. Sample sizes ranged from 104 participants to 3234 participants [28, 34]. Study characteristics varied but most studies only included participants that were over 35 years of age, had no previous diagnosis of COPD and were current or ex-smokers.

TABLE 1.

Hand devices

First author [ref.]	Year	Country	Sample size	Sample population	Device	Cut-off	Sensitivity %	Specificity %	AUC
Ronaldson [30]	2018	UK	216	≥35 years of age and current smokers	Wheezometer	Wheezometer score ≥3	4.4	93.6	0.49
					PEF	PEF <80%	86.1	67.5	0.77
					Micro-spirometer	FEV1 <80% pred, FVC <80% pred, or FEV1/FVC <70% pred	88.9	57.9	0.73
Fujita [38]	2020	Japan	1001	No previous diagnosis of asthma, TB and lung cancer	COPD-6 score#	COPD-6 score ≥30	85.7	74.6	0.82
Dickens [24]	2020	UK	544	≥40 years of age with respiratory symptoms or clinically diagnosed COPD	COPD-6	FEV1/FEV6 ≤0.77	82.8	85	NA
Labor [33]	2016	Croatia	227	40–60 years of age and ≥20 pack-years, with no previous diagnosis of COPD	COPD-6	FEV1/FEV6 ≤0.78	71	89	0.83
Kim [25]	2016	South Korea	190	≥40 years of age and >10 pack-years, with no previous COPD diagnosis	COPD-6	FEV1/FEV6 ≤0.77	72.7	77.1	0.76
Llordes [47]	2017	Spain	407	≥40 years of age and ≥1 pack-year, with no previous diagnosis of COPD	COPD-6	FEV1/FEV6 ≤0.78	87.9	72.3	0.87
Represas-Represas [42]	2016	Spain	362	≥40 years of age and >10 pack-years, with respiratory symptoms and no previous respiratory diagnosis	COPD-6	FEV1/FEV6 ≤0.78	91.2	52.8	0.8
Thorn [39]	2012	Sweden	305	40–85 years of age and >15 pack-years	COPD-6	FEV1/FEV6 ≤0.73	79.2	80.3	0.84
Miravitlles [45]	2012	Spain	173	Controls: ≥35 years of age and no previous diagnosis of COPD Cases: COPD stages I and II	COPD-6	FEV1/FEV6 ≤0.75	84.6	85	0.85
Hidalgo Sierra [44]	2018	Spain	155	≥40 years of age and ≥10 pack-years, with respiratory symptoms and no previous diagnosis of COPD	Piko-6	FEV1/FEV6 ≤0.69	100	97.2	NA
Van den Bemt [34]	2014	The Netherlands	104	≥50 years of age and ≥1 pack-year, with respiratory symptoms and no previous diagnosis of COPD	Piko-6	FEV1/FEV6 ≤0.72	93.2	85	0.94
Frith [43]	2011	Australia	204	≥50 years of age, smoker/ex-smoker and no previous diagnosis of obstructive lung disease	Piko-6	FEV1/FEV6 ≤0.74	81	71	0.85
Sichletidis [26]	2011	Greece	1078	≥40 years of age and no previous diagnosis of COPD	Piko-6	Post-bronchodilator FEV1/FEV6 ≤0.69	80	95	NA

AUC: area under the curve; PEF: peak expiratory flow; FEV₁: forced expiratory volume in 1 s; FEV₆: forced expiratory volume in 6 s; FVC: forced vital capacity; TB: tuberculosis; NA: not available. ^#: COPD-6 score was based on micro-spirometry results.

TABLE 3.

Questionnaires and hand devices

First author [ref.]	Year	Country	Sample size	Sample population	Combination	Cut-off	Sensitivity %	Specificity %	AUC
Fujita [38]	2020	Japan	1001	No previous diagnosis of asthma, TB and lung cancer	Changed CDQ and COPD-6 score	Joined score ≥27	87.8	73.1	0.87
Sichletidis[26]	2011	Greece	1078	≥40 years of age and no previous diagnosis of COPD	CDQ and Piko-6	Questionnaire score ≥17 and post-bronchodilator FEV1/FEV6 <0.7	72	97	NA

AUC: area under the curve; TB: tuberculosis; CDQ: COPD diagnostic questionnaire; FEV₁: forced expiratory volume in 1 s; FEV₆: forced expiratory volume in 6 s; NA: not available.

TABLE 2.

Questionnaires

First author [ref.]	Year	Country	Sample size	Sample population	Questionnaire	Cut-off#	Sensitivity %	Specificity %	AUC
Lopez Varela [37]	2019	Mexico	974	≥40 years of age and ≥10 pack-years, or ≥50 pipes per year or ≥50 cigars per year and/or exposure to biomass smoke	PUMA	≥5	85.4	37.6	0.70
Fujita [38]	2020	Japan	1001	No previous diagnosis of asthma, TB and lung cancer	Changed CDQ	≥18	81.6	59.7	0.73
Ronaldson [30]	2018	UK	216	≥35 years of age and current smokers	COPD-PS	Standard scoring criteria	53.3	78.6	0.66
					CDQ	Standard scoring criteria	81.5	52.3	0.67
Spyratos [28]	2017	Greece	3234	>40 years of age and ≥10 pack-years	CDQ	≥17	74	72	0.80
					COPD-PS	≥5	56	90	0.79
					LFQ	≤18	79	68	0.81
Weiss [31]	2017	Austria	775	≥40 years of age	SCSQ	≥2	67.1	58.9	0.66
Demirci [40]	2017	Turkey	357	40–65 years of age, with no previous history of obstructive lung disease	CAT	≥11	66.7	75.2	NA
Sogbetun [27]	2016	USA	376	No limitation	VAFOSQ	≥25	66	43	0.58
			255	No limitation	11-Q	≥5	94	9	0.61
			362	No limitation	COPD-PS	≥5	69	46	0.62
			376	No limitation	LFQ	≤18	95	14	0.66
			336	No limitation	Personal level screener COPD	Questionnaire algorithm	50	69	0.59
Lopez Varela [32]	2016	Argentina, Colombia, Venezuela, Uruguay	1540	≥40 years of age and ≥10 pack-years, or ≥50 pipes per year or ≥50 cigars per year and/or exposure to	PUMA	≥25 (weighted)	76.3	69.3	0.79
				biomass smoke		≥5 (simple)	74.2	64.8	0.76
Sogbetun [46]	2016	USA	278	All patients with a previously scheduled office visit	VAFOSQ (retrospective validation)	≥25	59.9	69.8	0.72
Llordes [47]	2017	Spain	407	≥40 years of age and ≥1 pack-year, with no	EGARPOC	13	72.9	58.3	0.68
				previous diagnosis of	RHSQ	>17	73.8	56	0.72
				COPD	COPD-PS	≥4	80.4	47.7	0.65
					Two screening questions	Both positive	59.8	51.7	0.56
Stanley [29]	2014	Australia	1054	40–85 years of age and former or current smokers, with no previous diagnosis	CDQ	≥16.5	79.7	46.8	0.71
				of COPD		≥19.5	63	70.1
Cui [35]	2012	China	150	≥40 years of age; COPD patients (n=100) and non-COPD patients (n=50)	Discriminant function model	NA	89	82	NA
Miravitlles [45]	2012	Spain	173	Controls: ≥35 years of age and no previous diagnosis of COPD Cases: COPD	COPD-PS (Spanish)	≥4	93.6	64.8	0.79
				stages I and II		≥5	65.4	88
Sichletidis [26]	2011	Greece	1078	≥40 years of age and no previous diagnosis of COPD	CDQ	≥17	80	95	NA
Frith [43]	2011	Australia	204	≥50 years of age, smoker/ex-smoker and no previous diagnosis of obstructive	CDQ	≥16.5	91	37	0.72
				lung disease		≥19.5	71	62
Kotz [36]	2008	The Netherlands	676	40–70 years of age and >10 pack-years, with respiratory symptoms and no previous	CDQ	≥16.5	89.2	24.4	0.65
				respiratory diagnosis		≥19.5	65.8	54.0
Price [41]	2006	UK, USA	246	≥40 years of age and smokers, with no previous respiratory	CDQ	≥16.5	58.7	77	0.82
				diagnosis		≥19.5	80.4	57.5

AUC: area under the curve; TB: tuberculosis; PUMA: PUMA study questionnaire; COPD-PS: COPD population screener; CDQ: COPD diagnostic questionnaire; LFQ: lung function questionnaire; SCSQ: Salzburg COPD-screening questionnaire; CAT: COPD assessment test; VAFOSQ: veterans airflow obstruction screening questionnaire; 11-Q: 11-Q COPD screening questionnaire; EGARPOC: EGARPOC study questionnaire; RHSQ: respiratory health screening questionnaire; NA: not available. ^#: cut-off based on questionnaire score.

Quality assessment

The quality assessment determined that each diagnostic accuracy study was fully applicable to the research question (see supplementary table E3 and supplementary figure E2). Regarding the risk of bias, two studies were defined as being at “high risk” of bias in the category “patient selection” because they used a case-control design [35, 45]. In the “index test” category, nine studies were defined as “high risk” as they did not pre-specify a threshold for their tool. These studies were mainly micro-spirometer studies whose aim was the determination of the optimal threshold for FEV₁/forced expiratory volume in 6 s (FEV₆). Three of the nine “high risk” studies were evaluating new questionnaires whose scoring was not yet established [32, 41, 46]. The two studies defined as being “unclear” for bias in the “index test” category assessed multiple diagnostic tests, of which at least one did not have a pre-specified cut-off. The five studies marked as “unclear” for bias in the “reference test” category did not mention blinding of the interpreters of the reference test. In the category “flow and timing”, most studies were assessed to be of “high risk” of bias due to the exclusion of enrolled participants. The main reasons for this were poor quality of spirometry or incomplete data, with some reporting loss to follow-up as well. One study with a “high risk” of bias in the “flow and timing” category reported a large time interval between the index and reference test [33]. Studies assessed to be “unclear” for bias did not specify if any study participants were excluded.

Systematic review

The 24 included studies evaluated one Wheezometer, one peak expiratory flow (PEF) device, one discriminant function model, 11 different micro-spirometer thresholds, 11 unique questionnaires and two combinations of questionnaire and micro-spirometer [24–47]. A Wheezometer is a hand-held pulmonary sound analyser and a PEF device is a small hand tool that measures the PEF rate. Micro-spirometers are hand-held and mobile spirometers. The threshold for diagnosing COPD with a micro-spirometer is evaluated using the FEV₁/FEV₆ ratio instead of the FEV₁/FVC ratio used in conventional spirometry. The micro-spirometers most evaluated in this review are the COPD-6 (Vitalograph Ltd., Ennis, County Clare, Ireland) and the Piko-6 (nSpire Health Inc., Longmont, CO, USA). Most questionnaires contained items such as age, smoking status and respiratory symptoms (e.g. wheezing, dyspnoea, coughing and the interference of symptoms with participants' daily lives).

A forest plot of the AUC of 38 tools, some of them identical, that were evaluated in 19 of the included studies is shown in figure 1. Results for hand devices, excluding the Wheezometer, showed higher AUC values on average than the questionnaires. A combination of questionnaire and micro-spirometer was evaluated in two studies. Only one of them determined the AUC [38]. The Youden index of the other study was calculated to be 0.69 [19, 26]. Both values were set in the same range as the hand device group excluding the Wheezometer. Eight studies evaluated more than one tool, which made inter-test comparison more meaningful [26–28, 30, 38, 43, 45, 47]. Six of these compared micro-spirometer use with questionnaires and micro-spirometers always showed a higher AUC or Youden index. Four studies compared multiple questionnaires and, in two of these four studies, the lung function questionnaire (LFQ) scored the highest AUC [27, 28]. The COPD diagnostic questionnaire (CDQ), also known as the international primary care airways guidelines (IPAG) questionnaire or the respiratory health screening questionnaire (RHSQ), yielded the highest AUC in the other two studies [30, 47]. In one of these four studies the LFQ and CDQ were compared directly. Here, the LFQ acquired the highest AUC (0.81), followed closely by the CDQ (0.80) and the COPD population screener (COPD-PS) questionnaire (0.79) [28]. The CDQ and the COPD-PS questionnaire were compared directly in three studies and the CDQ always obtained a slightly higher AUC [28, 30, 47]. Two cut-off values for the CDQ (≥16.5 and ≥19.5) were directly compared in four studies [29, 36, 41, 43]. In all studies, the cut-off value ≥19.5 yielded a higher Youden index in comparison to the threshold of ≥16.5.

FIGURE 1.

Systematic review: forest plot of area under the curve (AUC) values with 95% confidence intervals (CIs) of hand devices (mainly the micro-spirometers COPD-6 and Piko-6), questionnaires, or both hand devices and questionnaires. The COPD-6 score was based on micro-spirometry results. PEF: peak expiratory flow; CDQ: COPD diagnostic questionnaire; COPD-PS: COPD population screener; VAFOSQ: veterans’ airflow obstruction screening questionnaire; PUMA: PUMA study questionnaire; LFQ: lung function questionnaire; 11-Q: 11-Q COPD screening questionnaire; SCSQ: Salzburg COPD-screening questionnaire; EGARPOC: EGARPOC study questionnaire; RHSQ: respiratory health screening questionnaire.

Meta-analysis

The meta-analysis included the CDQ and the COPD-PS questionnaire, as well as the micro-spirometers if they stated an FEV₁/FEV₆ cut-off. The three tools were assessed in 17 out of 24 studies. Only identical diagnostic tools that were evaluated at least five times were considered, but thresholds could differ. Some CDQ studies evaluated two different cut-offs but did not specify which one qualified as best for a clinical setting. As only one cut-off can be assessed with the bivariate model, the cut-off yielding the highest Youden index was applied for these studies, which was always ≥19.5. The thresholds of the micro-spirometers and the COPD-PS questionnaire used in the analysis were the ones specified to be the most efficient in their respective diagnostic accuracy studies or the ones with the highest Youden index. The following studies were included in the analysis: 8 CDQ studies (7115 participants), 5 COPD-PS questionnaire studies (4392 participants) and 10 micro-spirometer studies (2671 participants) evaluating pre-bronchodilator micro-spirometry [24–30, 33, 34, 36, 39, 41–45, 47]. Every participant in the CDQ group was over 35 years of age. Furthermore, with the exception of one study, every participant in the CDQ group had a history of smoking [26]. Apart from two studies, no participants with a previous diagnosis of COPD were included in the CDQ group and COPD prevalence spanned between 10.3% and 41% [26, 28, 30, 36]. Additionally, the eight CDQ studies were conducted in five different countries. Three out of five studies in the COPD-PS questionnaire group included participants that were specified to be >35 years old [28, 30, 47]. Study criteria regarding history of smoking and a previous diagnosis of COPD varied in the COPD-PS questionnaire group and COPD prevalence ranged between 10.7% and 45.6% [28, 45]. Furthermore, the five COPD-PS questionnaire studies were administered in four different countries. In all micro-spirometer studies but one, participants were specified as being >35 years old [45]. Eight out of the 10 micro-spirometer studies included only smokers and, apart from three studies, no participants with a previous diagnosis of COPD were included [24, 39, 45]. COPD prevalence in the micro-spirometer group ranged between 18.9% and 62% [24, 33]. Moreover, the 10 micro-spirometer studies were conducted in seven different countries. Supplementary figure E3 depicts the descriptive analysis of the sensitivity and specificity of each tool in each study in a coupled forest plot. Figure 2 displays a scatterplot of the sensitivities and specificities, as well as the summary ROC curve and the prediction region, for each tool resulting from the meta-analysis. The true sensitivity and specificity of a future tool lies with 95% confidence within this prediction region. Micro-spirometers obtained a greater overall AUC (0.84, 95% CI 0.80–0.89) than the questionnaires. The data analysis indicated that the COPD-PS questionnaire yielded a higher summary AUC (0.77, 95% CI 0.63–0.85) than the CDQ (0.72, 95% CI 0.64–0.78) (see supplementary figure E4). The differences between summary AUC values were tested for all three combinations of tools (see supplementary table E4). Only the summary AUC values of micro-spirometers and the CDQ differed significantly. A forest plot containing the AUC values of the included studies, as well as the summary estimates of the overall AUC values resulting from the bivariate analysis, can be found in supplementary figure E5.

FIGURE 2.

Descriptive analysis and meta-analysis: scatterplot of sensitivities and specificities, as well as summary receiver operating characteristic (ROC) curves (including prediction region) for a) the COPD diagnostic questionnaire; b) the COPD population screener questionnaire; and c) micro-spirometers. AUC: area under the curve; CI: confidence interval; NA: not available.

Discussion

The aim of this study was to inform about current evidence concerning the optimal alternative diagnostic tool for COPD case-finding in primary care settings without spirometry.

Overall results

In the analysis of the systematic review, micro-spirometry almost always outperformed the analysed questionnaires. The combination of a hand device and a questionnaire was deemed as accurate at diagnosing COPD as the hand devices alone, given the similar AUC value and Youden index value. The meta-analysis determined that micro-spirometers are more accurate diagnostic tools than the CDQ and the COPD-PS questionnaire, although only the difference in summary AUC value compared to the CDQ was statistically significant. The COPD-PS questionnaire was a slightly more accurate diagnostic tool than the CDQ; however, the difference in summary AUC values was not statistically significant.

Micro-spirometers

Given the data analysed, micro-spirometers were a more accurate diagnostic tool for COPD in comparison to diagnostic questionnaires; however, only difference to the CDQ was significant. COPD diagnosis requires a symptomatic patient in addition to irreversible airflow obstruction. Of the 10 micro-spirometer studies, four were conducted in a patient cohort that was required to be symptomatic and almost all of the micro-spirometer study participants were required to have a history of smoking. Therefore, the conclusion that micro-spirometers are more accurate diagnostic tools is only valid for symptomatic patients or patients with a history of smoking and is not applicable to the general population. A systematic review and meta-analysis that answered a similar question, conducted by Haroon et al. [48], collected data up until December 2013. The eligibility criteria were different as only studies with participants that had no prior diagnosis of COPD were included. Furthermore, their spirometry criteria differed due to the acceptance of pre-bronchodilator spirometry as an adequate gold standard for the diagnosis of COPD. They concluded that micro-spirometers were more exact diagnostic tools for COPD in comparison to the CDQ, which was the only questionnaire included in their meta-analysis. The analysis in our work further supports their conclusions. Micro-spirometers are easier, quicker and cheaper to administer than conventional spirometry and could therefore be of great help in diagnosing symptomatic patients in resource-constrained, primary care settings.

Combination of tools

According to our systematic review, the combination of a micro-spirometer and a questionnaire was as capable in the diagnosis of COPD as micro-spirometers alone. COPD case-finding with a combination of a questionnaire and micro-spirometry has been deemed practical and has been recommended to be implemented into healthcare [49]. Apart from micro-spirometry, a PEF device can also be used in combination with a diagnostic questionnaire. A recently developed combination test in COPD diagnostic research is the COPD Assessment in Primary Care to Identify Undiagnosed Respiratory Disease and Exacerbation Risk questionnaire (CAPTURE) together with a PEF device. One study measured its sensitivity at 89.7% and its specificity at 78.1% [50]. Its Youden index value (0.68) is one of the highest in the range compared to our included studies. However, the CAPTURE+PEF evaluation study was excluded in this systematic review as not all study participants received post-bronchodilator spirometry. Soriano et al. [51] reported that a case-finding method comprised of the COPD-PS questionnaire and PEF led to a 90% reduction in the spirometry tests taken. A cross-sectional study from 2017 measured the sensitivity and specificity of respiratory questions in combination with PEF as 84% and 93%, respectively [52]. In summary, a combination of a questionnaire and a hand device can improve COPD diagnosis in primary care settings without spirometry. Diagnostic questionnaires can determine symptomatic patients and can therefore function as a pre-test to obstruction testing.

Questionnaires

Diagnostic questionnaires are still valuable tools, although micro-spirometers were assessed to be more accurate. This systematic review indicated that the LFQ was a slightly more robust diagnostic tool for COPD than the CDQ and the COPD-PS questionnaire [53–56]. According to the systematic review, the CDQ was slightly more accurate than the COPD-PS questionnaire. The meta-analysis demonstrated the contrary, although the difference was not statistically significant. Altogether the two questionnaires are approximately equally accurate in diagnosing COPD. The higher Youden index of the CDQ, with a threshold of ≥19.5 rather than a threshold of ≥16.5, suggested it might be the better option. A comparison of the CDQ and the COPD-PS questionnaire in a Japanese population determined that they were comparable in discriminating subjects with and without airflow limitation [57]. A validation study measured the sensitivity and specificity of the LFQ, with a threshold of ≤18, at 82.6% and 47.8%, respectively [58]. As in the two studies that were included here, this cut-off combines a high sensitivity with a low specificity. As stated above, COPD diagnosis requires a symptomatic patient. Few of the included questionnaire studies specified respiratory symptoms as an inclusion criterion, but the diagnostic questionnaires mentioned, namely the CDQ, LFQ and COPD-PS questionnaire, encompass questions about respiratory manifestations. In summary, diagnostic questionnaires for COPD can determine symptomatic patients and are easier to implement than hand devices, which is especially important in resource-limited settings.

Strengths and limitations

A strength of this work is that studies in four languages were assessed. Additionally, many new diagnostic tools were considered that have been developed since the last meta-analysis was conducted. Moreover, this review only included studies that used post-bronchodilator spirometry as a reference standard, which is defined as the gold standard by the GOLD guidelines [1].

As of 2016, the US Preventative Task Force undertook a systematic review resulting in it advising against screening for COPD in asymptomatic people, because there was no evidence of patient benefit [59, 60]. They concluded that more randomised and controlled studies in smokers, utilising the LFQ or CDQ as pre-screening questionnaires followed by micro-spirometry, are necessary. It is crucial to differentiate between COPD screening and case-finding. Screening is conducted in a population resembling the general population with a low prevalence of COPD. Case-finding is done in patients likely to have COPD due to respiratory symptoms or an exposure to noxious particles. Almost all included studies required participants to have a history of smoking, but potential non-smoking COPD patients and patients with exposure to biomass smoke might have been missed. Overall, only a few included studies required participants to be symptomatic, which is a limitation of this review. As we aimed to find an optimal case-finding tool, a realistic scenario would be if the tool were applied to symptomatic patients only. Unfortunately, there are few validation studies with this sample criterion and it was not feasible to only include studies with that requirement.

Given the strict eligibility criteria of this systematic review, many potentially valuable diagnostic tools for COPD were not analysed. Many recently developed questionnaires, like the veterans’ airflow obstruction screening questionnaire (VAFOSQ) and the PUMA study questionnaire, but also older examples like the LFQ, were eligible for the systematic review only, as they were solely tested twice [27, 28, 32, 37, 46]. Furthermore, only two combinations of a hand device and a questionnaire were assessed, and consequently they were not included in the meta-analysis. As such, definite statements about a combination of tools could not be made and overall the low number of included studies in the meta-analysis limits our results.

Another limitation of this meta-analysis is the heterogeneity of sensitivities and specificities (see figure 2). Heterogeneity can be assessed by the size of the prediction region and the distance of individual points from the summary ROC curve. According to these criteria, COPD-PS questionnaire studies show the most heterogeneity, followed by CDQ studies and finally micro-spirometer studies. Reasons for this could be that studies differed with regard to the patient's history of smoking, the inclusion of diagnosed COPD patients, the COPD prevalence and the country of conduct.

Regarding the risk of bias, it is possible that the diagnostic accuracy of some tools was evaluated optimistically, as can be observed in the “index test” category. “High risk” studies did not pre-specify a threshold for their tool. In the bivariate model only one cut-off per study could be assessed, therefore the optimal threshold resulting in the respective diagnostic accuracy study was chosen. As the optimal threshold was determined retrospectively, the stated quality measures might be lower in a different study sample. The exclusion of enrolled study participants from the analysis due to poor quality of spirometry could, for example, be another source of bias.

Using the AUC and the Youden index to search for the optimal diagnostic tool assumed that sensitivity and specificity were weighted as being equally important, which does not necessarily have to be the case. As a great proportion of COPD-related deaths occur in LMICs, a high specificity may be wanted to avoid unneeded strain on false positive participants and the healthcare system.

When available, spirometry should continue to be the first line of choice for COPD diagnosis, particularly considering that failure to make a correct diagnosis leads not only to underdiagnosis but also to overdiagnosis, with a related burden of disease and costs [61].

Suggestions for further research

The burden of COPD lies mainly in LMICs where spirometry is often not available in primary care settings. However, all 24 studies included in this review were, except for one, conducted in high-income countries. Therefore, further research regarding the applicability of these alternative diagnostic tools in low-resource settings is needed. Also, many questionnaires include respiratory symptoms and exposure to smoking, but not biomass fuel utilisation, which is an important risk factor in LMICs. Additionally, only two studies were included that evaluated a combination of a questionnaire and a hand device. As this combination could increase diagnostic accuracy, more research should be directed this way. When validating a COPD case-finding tool it should be ensured that only symptomatic patients are tested. Future studies should compare their tool to post-bronchodilator spirometry for every included participant or every included participant with pre-bronchodilator obstruction. Interpreters of index and reference tests should be blinded towards the results of the respective other test and exclusion of enrolled participants must be avoided.

Conclusion

We aimed to evaluate alternative diagnostic tools for COPD diagnosis in resource-constrained settings where no spirometry is available. In general, only symptomatic patients should be tested for COPD. The most accurate alternative diagnostic method to conventional spirometry, if sensitivity and specificity are equally weighted, is micro-spirometry. However, this was only shown in patients with a history of smoking and some symptomatic patient cohorts and the number of publications was low. Additionally, only the difference in accuracy to the CDQ was statistically significant. As they encompass questions about respiratory symptoms and are easy to implement, diagnostic questionnaires like the COPD-PS questionnaire and the CDQ are valuable diagnostic tools. Considering the systematic review, the meta-analysis, heterogeneity and CIs, both questionnaires are approximately equally accurate case-finding tools. They could be used as a test itself or as a pre-test for micro-spirometry or conventional spirometry. The combination of a questionnaire and micro-spirometry was deemed as accurate as micro-spirometry alone but was not evaluated in the meta-analysis. A combination could ensure obstruction testing of symptomatic patients only. When choosing an optimal diagnostic tool in primary care settings, a high specificity may be desired depending on resource availability. Alternative diagnostic tools for COPD can be of great help in resource-constrained, primary care settings. However, if post-bronchodilator spirometry is available, it should be used for final COPD diagnosis given its position as the gold standard.

Supplementary material

Please note: supplementary material is not edited by the Editorial Office, and is uploaded as it has been supplied by the author.

Supplementary_tables_and figures ERR-0350-2020_Supplementary_tables_and_figures ^{(512KB, pdf)}