Diagnostic accuracy of exhaled nitric oxide in asthma: a meta-analysis of 4,691 participants

Abstract

Asthma is a common airway inflammation, but current methods for diagnosing it are poor. Here we meta-analyze the available evidence on the ability of exhaled nitric oxide (eNO) in asthma to serve as a diagnostic marker of asthma. We systematically searched the PubMed and EMBASE databases, published data on sensitivity, specificity and other measures of diagnostic accuracy of eNO in the diagnosis of asthma were meta-analyzed. The methodological quality of each study was assessed by QUADAS-2 (quality assessment for studies of diagnostic accuracy). Statistical analysis was performed by employing Meta-Disc 1.4 software and STATA. And the measures of accuracy of eNO in the diagnosis of asthma were pooled using random-effects models. A total of nineteen publications reporting twenty-one case-control studies were identified. Pooled results indicated that eNO showed a diagnostic sensitivity of 0.78 (95% CI 0.76 to 0.80), specificity was 0.74 (95% CI 0.72 to 0.76). PLR was 3.70 (95% CI 2.84 to 4.81) and NLR was 0.35 (95% CI 0.26 to 0.47). DOR was 11.37 (95% CI 7.54 to 17.13). Exhaled nitric oxide show insufficient sensitivity and specificity for diagnosing asthma, eNO measurements may be useful in combination with clinical manifestations and conventional tests such as pulmonary function tests, assessment of bronchodilator response and bronchial challenge tests.

Introduction

Asthma is an airway inflammation as a serious health problem globally, people in each age stage could be affected by this chronic airway disease. If it uncontrolled, the patients’ daily life with it would severely limits and mortality. While early diagnosis of asthma reduces the socioeconomic impact of asthma and enhances patients’ quality of life significantly [1]. In clinical practice, diagnosis asthma is based on symptoms, pulmonary function tests, assessment of bronchodilator response, and bronchial challenge tests [2]. However, neither the symptoms nor pulmonary function tests can reflect the severity of airway inflammation [3,4]. Although assessment of bronchodilator response and bronchial challenge tests are a reliable tool for airway hyperresponsiveness, the results is not always consistent with the degree of inflammation [5]. In recent years, many studies indiated that exhaled nitric oxide (eNO) was an useful monitoring factor for asthmatic airway inflammation. However, the diagnostic value of exhaled NO for asthma is still debated [6]. For example, some studies have reported that levels of eNO in asthma provide high diagnostic sensitivity (91.0%) [7]. Other studies, however, have reported much lower corresponding values 26% [8]. So we meta-analyzed the available literature to gain a comprehensive status of the diagnostic usefulness of eNO in asthma.

Methods

The guidelines of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement [9] was accorded when we conducting this meta-analysis.

Search strategy and selection criteria

We searched Pubmed and EMBASE for meta-analyses existed that related to diagnostic accuracy of exhaled nitric oxide in asthma, no article was found. Then we identify eligible studies until December 31, 2015. Using “asthma” AND “exhaled nitric oxide” OR “eNO” OR “nitric oxide” AND “sensitivity” AND “specificity” AND “diagnosis” as the text search terms. Only English-language articles were considered. Identified articles in reference lists were also searched manually.

To be included in our study, the criteria were used (1) Information about the sensitivity, specificity of exhaled nitric oxide for diagnosis of asthma and number of patients was complete. (2) case-control design was performed. (3) Clear diagnostic criteria. Unpublished data, case reports, letters to editor, abstracts, review articles were excluded.

Data extraction and quality assessment

Two independent reviewers (Z.L. and W.Q.) assessed study eligibility and disagreements were consulted to resolve. The standard procedure was performed to extract data from the studies. Data as follows were retrieved: the name of the first author, the country of origin, the year of publication, the number of patients, asthma diagnosis standard, assay methods, cut-off values, sensitivity and specificity data, the numbers of true positive, false positive, true negative and false negative. The methodological quality of the studies assessed by the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) checklist, with a maximum score of 11 [10].

Statistical analyses

We used the standard methods recommended for meta-analyses of diagnostic test evaluations [11]. Analyses were performed using Stata, version 12 and Meta-Disc software (Zamora J, Muriel A, Abraira V. Meta-DiSc for Windows, XI Cochrane Colloquium. Barcelona, 2003). The following measures of test accuracy were computed: sensitivity, specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR) and diagnostic odds ratio (DOR). Overall diagnostic performance was assessed from summary receiver operating characteristic (SROC) curves [11,12]. These curves were plotted for each study using the sensitivity and specificity based on the single-test threshold identified within the same study [12,13].

We used a random-effect model to meta-analyze sensitivity, specificity, and other diagnostic measures [14,15]. We used chi-squared and Fisher’s exact tests to assess statistically significant variability (heterogeneity) across studies. To assess the effects of some methodological and clinical characteristics, we included cut-off value, diagnostic standard, eNO assay method as covariates in univariate meta-regression analysis (inverse variance weighted). The relative DOR (RDOR) was calculated according to standard methods to analyze the change in diagnostic accuracy in the study per unit increase in the covariate [16,17]. We tested for the potential presence of publication bias using Deeks’ funnel plots [18].

Results

Literature searches turned up 289 potentially eligible studies, and 267 were excluded based on review of titles and abstracts. The remaining 22 articles were read in full, and three [19-21] were excluded after read the full text because they did not display a sufficient data. In the end, nineteen publications [7,8,22-38] assessing the diagnostic performance of eNO in asthma were included in our analysis. One study [23] involved three case-control groups, and sufficient data were reported for each that we were able to treat the groups as three independent studies in the meta-analysis. Thus, the final meta-analysis included twenty-one studies from nineteen publications and the clinical characteristics of these studies are displayed in Table 1.

Table 1.

Characteristics of studies included in the meta-analysis^a

Citing no	Author	Study	Numbers of patients	TP	FP	FN	TN	Cut-off value (ppb)b	Assay method device	Asthma diagnosis standard	Quality score (QUADAS)
14	Ana Ma Fortuna	Spain (2007)	50	17	10	5	18	23	SIR N-6008, Madrid, Spain	GINA guidelines	9
15	Andrei Malinovschi	Sweden (2012)	108	35	23	10	40	15	NIOX Mino, Aerocrine AB, Solna, Sweden	GINA guidelines	9
15	Andrei Malinovschi	Sweden (2012)	62	12	6	7	37	22	NIOX Mino, Aerocrine AB, Solna, Sweden	GINA guidelines	9
15	Andrei Malinovschi	Sweden (2012)	112	18	14	14	66	17	NIOX Mino, Aerocrine AB, Solna, Sweden	GINA guidelines	9
16	Sung-Il Woo	Korea (2012)	245	95	10	72	68	22	NIOX Mino, Aerocrine AB, Solna, Sweden	NAEPP guidelines	9
17	Antonius Schneider	Germany (2009)	160	24	6	51	79	46	NIOX Mino, Aerocrine AB, Solna, Sweden	ATS guidelines	8
18	Kazuto Matsunaga	Japan (2011)	366	129	36	13	188	22	NIOX Mino, Aerocrine AB, Solna, Sweden	Significant airway reversibility and/orhyperresponsiveness	8
19	Sachs-Olsen C	Norway (2010)	227	8	6	23	190	20.4	EcoMedics AG, Duernten, Switzerland	Symptoms, history, Use of asthma medication	8
20	L P Malmberg	Finland (2003)	83	18	5	3	57	9.7	CLD 77 AM, Eco Physics, Duernten, Switzerland	Symptoms, history, Use of asthma medication	8
21	N Berkman	Israel (2005)	85	33	5	7	40	7	LR 2000, Logan Research, Rochester, UK	History	7
22	Joanna Jerzyn´ ska	Poland (2014)	1767	949	342	105	371	23	Model 280i nitric oxide analyzer; Sievers, Boulder, CO, USA	GINA guidelines	10
23	A. Florentin	France (2014)	178	8	12	11	147	25	Niox-Minow Analyser; Aerocrine, Stockholm, Sweden	ATS guidelines	8
24	Atsuro Fukuhara	Japan (2011)	61	33	2	9	17	23.9	NA623N; Chest MI, Tokyo, Japan	Conventional asthma diagnostic procedure	8
25	Danielle Cordeiro	Netherlands (2011)	114	33	6	9	66	27	NIOX Mino, Aerocrine AB, Solna, Sweden	GINA guidelines	10
26	Mar´ıa Pedrosa	Spain (2010)	114	26	22	9	57	40	NIOX Mino, Aerocrine AB, Solna, Sweden	ATS guidelines	9
27	Yakov Sivan	Israel (2009)	150	85	3	21	41	19	EcoMedics AG, Duernten, Switzerland	Conventional asthma diagnostic procedure	8
28	Luisa Bommarito	Italy (2008)	109	9	28	4	68	18.5	Sievers, Boulder, Colo., USA	Symptoms	8
29	Rajiv Arora	Texas (2006)	172	121	7	17	27	20	NIOX Mino, Aerocrine AB, Solna, Sweden	Symptoms, historypositive, histamine bronchoprovocation	9
30	Enrico Heffler	Italy (2006)	48	14	12	4	18	36	NIOX Mino, Aerocrine AB, Solna, Sweden	GINA guidelines	9
31	Antonius Schneider	Germany (2013)	393	75	60	79	179	25	NIOX Mino, Aerocrine AB, Solna, Sweden	Symptoms, history or lung functional test	8
32	Mikko Voutilainen	Finland (2014)	87	30	10	24	23	22	NIOX Mino, Aerocrine AB, Solna, Sweden	GINA guidelines	9

^aTN, true negative; TP, true positive; FN, false negative; FP, false positive.

^bppb: Parts per billion.

Study characteristics

The total sample size in the twenty-one studies was 4,691, comprising 2,269 patients with asthma and 2,422 without it. Asthma was diagnosed by GINA guidelines or ATS guidelines [8,10-12,14,15,18,20,21,25,39], in the remaining 9 studies, some asthma patients were diagnosed based on history, and some were diagnosed based on clinical symptoms and history. The cut-off value and diagnostic standard not exactly the same.

Diagnostic accuracy

Sensitivity for eNO in asthma diagnosis ranged from 0.26 to 0.91 in the twenty-one studies, and meta-analysis of sensitivity and specificity indicated a pooled sensitivity of 0.78 (95% CI 0.76 to 0.80) (Figure 1). Specificity ranged from 0.52 to 0.97 and meta-analysis showed a pooled specificity of 0.74 (95% CI 0.72 to 0.76) (Figure 2). PLR was 3.70 (95% CI 2.84 to 4.81) (Figure 3) and NLR was 0.35 (95% CI 0.26 to 0.47) (Figure 4). DOR was 11.37 (95% CI 7.54 to 17.13) (Figure 5). I² was 94.7 for sensitivity, 94.7% for specificity, 86.2% for PLR, 93.6% for NLR, and 79.6% for DOR.

Figure 1.

Forest plot of estimates of sensitivity for eNO in the diagnosis of asthma. Point estimates of sensitivity from each study are shown as solid circles, the size of which reflects the total number of cases and controls. Error bars show 95% confidence intervals. Numbers indicate the reference numbers of the studies.

Figure 2.

Forest plot of estimates of specificity for eNO in the diagnosis of asthma. Point estimates of specificity from each study are shown as solid circles, the size of which reflects the total number of cases and controls. Error bars show 95% confidence intervals. Numbers indicate the reference numbers of the studies.

Figure 3.

Forest plot of estimates of positive likelihood ratios for eNO in the diagnosis of asthma. Point estimates of positive likelihood ratios from each study are shown as solid circles, the size of which reflects the total number of cases and controls. Error bars show 95% confidence intervals. Numbers indicate the reference numbers of studies.

Figure 4.

Forest plot of estimates of negative likelihood ratios for eNO in the diagnosis of asthma. Point estimates of negative likelihood ratios from each study are shown as solid circles, the size of which reflects the total number of cases and controls. Error bars show 95% confidence intervals. Numbers indicate the reference numbers of studies.

Figure 5.

Forest plot of estimates of diagnostic odds ratios for eNO in the diagnosis of asthma. Point estimates of diagnostic odds ratios from each study are shown as solid circles, the size of which reflects the total number of cases and controls. Error bars show 95% confidence intervals. Numbers indicate the reference numbers of studies.

SROC curves were generated by plotting sensitivity against (1-specificity) for individual studies (Figure 6). The curves did not lie near the desired upper left corner, and the maximum joint sensitivity and specificity was 0.82, with an area under the curve (AUC) of 0.8428 (SEM 0.0225).

Figure 6.

Summary receiver operating characteristic curves for eNO. Each study is depicted as a solid circle, the size of which reflects the total number of cases and controls.

Multiple regression analysis

Across the twenty-one studies, the method device, asthma diagnosis standard and eNO cut-off values in the assay differed significantly (Table 1). Thus, we performed a meta-regression analysis to assess the effect of these different on the relative DOR (RDOR) of eNO in asthma diagnosis. It indicated the asthma diagnosis standard was affect the heterogeneity between the studies (Table 2).

Table 2.

Weighted meta-regression of the effects of the method device, asthma diagnosis standard and eNO cut-off values in the assay on diagnostic accuracy of asthma

Covariate	No. studies	Coefficient	RDOR (95% CI)	P
Cut-off, ppb
> 22	8	-0.489	0.61	0.298
≤ 22	13
Diagnosis
Guidelines	12	-0.933	0.39	0.0379
Non-guidelines	9
Device
NIOX Mino	12	-0.57	0.57	0.2171
Not NIOX Mino	9

Publication bias

Funnel plots showed some asymmetry (Figure 7), nevertheless, Deeks’ test gave a p value of 0.59, suggesting that our analysis did not have significant risk of publication bias.

Figure 7.

Funnel plot for evaluating publication bias among the twenty-one studies included in the meta-analysis. The log of the diagnostic odds ratio (DOR) is plotted against the standard error of log DOR; The latter serves as an indicator of sample size. Each article is shown as a solid circle, and the regression line is shown.

Discussion

Given the limitations of current methods for diagnosing asthma, researchers have explored whether eNO as diagnostic markers. These studies have given conflicting results about the diagnostic performance of eNO, so here we performed the present meta-analyzed. Our analysis suggests that eNO measurements by themselves are not sufficiently sensitivity (0.78) and specificity (0.74) to diagnose asthma, but they can provide complementary diagnostic information when used in combination with assays of conventional tests such as bronchial challenge tests.

Meta-analysis of the twenty-one included studies indicated a pooled DOR of 11.37 for eNO, not indicating a relatively high accuracy. DOR, which combines sensitivity and specificity data that serves as an aggregate indicator of test accuracy [40], is the ratio of the odds of positive test results in people with disease relative to the odds of positive test results in people without disease [41].

The SROC curve and the area underneath it present tradeoff between sensitivity and peciMNficity [41]. Meta-analysis showed eNO sensitivity to be 0.78; specificity, 0.74; maximum joint sensitivity and specificity, 0.82; and the area under the SROC curve, 0.8428. These results also not indicate a high accuracy.

DOR and SROC curve analysis are difficult to interpret and use in clinical practice [42], and likelihood ratios are more clinically meaningful for measuring diagnostic accuracy [42,43]. Therefore we meta-analyzed the pooled PLR and NLR. The PLR value of 3.70 suggests that patients with asthma have about 4-fold higher chance of being eNO assay-positive compared to patients without asthma, this is insufficient to serve as the sole basis for diagnosing asthma. At the same time, the NLR was 0.35, it means it has a 35% probability that the patient having asthma if the eNO assay is negative. This also provides evidence that such an assay is inadequate, on its own, for ruling out asthma.

We found significant heterogeneity in the data, so we examined the twenty-one studies more carefully. In all studies, the QUADUS-2 score in each study was relatively high. In addition, inter-study variation in eNO cut-off values and assay method device did not substantially affect diagnostic accuracy, the basis for the heterogeneity in our meta-analysis were from the inter-study variation in asthma diagnosis standard, and in any case, further large studies are needed to verify our findings, especially since we excluded possibly relevant studies that were not published in English or that were published only as conference abstracts or letters to the editor.

The present meta-analysis suggests eNO assays, which can be used to complement other tests, has a potential role for in screening and confirming a diagnosis of asthma, those may be more desirable non-invasive methods of choice for screening and diagnosing asthma in the future.

Conclusion

The available evidence suggests that the eNO assay should not be used on its own to diagnose asthma, but that it can be used to complement other tests including pulmonary function tests, assessment of bronchodilator response and bronchial challenge tests.

Disclosure of conflict of interest

None.