Diagnostic accuracy of vascular endothelial growth factor for malignant pleural effusion: A meta-analysis

Abstract

Since the diagnostic accuracy of conventional examinations for malignant pleural effusion (MPE) is limited, a number of studies have investigated the utility of pleural vascular endothelial growth factor (VEGF) in the diagnosis of MPE. The present meta-analysis aimed to determine the overall accuracy of a VEGF test in the diagnosis of MPE. A systematic review of studies published in English was conducted and the data concerning the accuracy of pleural VEGF assays in the diagnosis of MPE were pooled with random effects models. The overall test performance was summarized using receiver operating characteristic curves. Ten studies, based on 1,025 patients, met the inclusion criteria for the meta-analysis and the summary estimates for VEGF in the diagnosis of MPE were: sensitivity 0.75 [95% confidence interval (CI), 0.72–0.79], specificity 0.72 (95% CI, 0.68–0.76), positive likelihood ratio 2.94 (95% CI, 1.97–4.41), negative likelihood ratio 0.38 (95% CI, 0.27–0.51) and diagnostic odds ratio 9.05 (95% CI, 4.60–17.80). The summary receiver operating characteristic curve indicated that the maximum joint sensitivity and specificity was 0.75; the area under the curve was 0.82. Our findings suggest that the determination of pleural VEGF may improve the accuracy of MPE diagnosis, while the results of VEGF assays should be interpreted in parallel with conventional test results and other clinical findings.

Introduction

Pleural effusion is a common complication which most commonly results from cardiac failure, pneumonia and malignant neoplasms (1). However, it is occasionally difficult to differentiate malignant pleural effusions (MPEs) from benign effusions. The sensitivity of conventional cytological examination is just 60% (2), while closed pleural biopsy is only able to identify an additional 7% of the cytology-negative MPE patients (3). Image-guided percutaneous and thoracoscopic pleural biopsies provide a high sensitivity (4), but they may not be widely used in all facilities or be well-tolerated.

A number of tumor markers have been studied in attempts to improve the accuracy of MPE diagnosis. Two previously published meta-analyses (5,6) investigated the diagnostic value of the pleural carcinoembryonic antigen (CEA), carbohydrate antigens (CAs) 125, 15–3 and 19–9 and CYFRA 21-1 in MPE but failed to identify a reliable tumor marker with high sensitivity and specificity. Therefore, it is imperative to identify a novel pleural marker to increase diagnostic accuracy.

Vascular endothelial growth factor (VEGF), or vascular permeability factor, is a glycoprotein that functions as a mediator of angiogenesis. It is expressed by various types of tumors (7) as well as certain normal tissues, including the lung, kidney, adrenal gland, heart, liver and stomach mucosa (8). VEGF is pivotal in the formation of MPE, as it increases vascular permeability and vascular leakage of fluid (9,10). In addition, a high level of pleural VEGF has been found to be correlated with malignancy (11) and thus an increasing number of studies consider VEGF to be a marker for the diagnosis of MPE (12–14). However, conflicting results have been reported and the exact role of VEGF remains unclear. Therefore, we performed the present meta-analysis to establish the overall accuracy of pleural VEGF for diagnosing MPE.

Materials and methods

Search strategy and study selection

To find relevant studies, we performed searches of Pubmed (Medline), Embase, Web of Science and the Cochrane database up to November 30, 2011, using the key words ‘pleural effusion’, ‘malignant pleural effusions’, ‘vascular endothelial growth factor’, ‘sensitivity and specificity’ and ‘accuracy’. All searches were limited to English language publications concerning human studies. A manual search of the references of the retrieved articles was conducted subsequently. Conference abstracts and letters to the editor were excluded due to the limited data provided. A study was included in the present meta-analysis if it provided the sensitivity and specificity of pleural VEGF for the diagnosis of MPE. Two authors (Y.-C. Shen and M.-Q. Liu) independently screened the articles for inclusion. Disagreements between the reviewers were resolved by consensus.

Data extraction and quality assessment

The final articles included were assessed independently by two reviewers (Y.-C. Shen and M.-Q. Liu). Data retrieved from the studies included author, publication year, patient source, test method, cut-off value, sensitivity, specificity and methodological quality. To assess the trial methodology, the articles were reviewed independently by two authors (Y.-C. Shen and M.-Q. Liu) and assigned a quality score using the STARD (standards for reporting diagnostic accuracy, a guideline that aims to improve the quality of the reporting of diagnostic studies, maximum score 25) (15) and the QUADAS (quality assessment for studies of diagnostic accuracy, an evidence-based quality assessment tool to be used in systematic reviews of diagnostic accuracy studies, maximum score 14) tools (16).

Statistical analyses

The standard methods recommended for the diagnostic accuracy of meta-analyses were used in the present study (17). The following indices of test accuracy were computed for each study: sensitivity, specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR) and diagnostic odds ratio (DOR). The diagnostic threshold identified for each study was used to plot a summary receiver operating characteristic (SROC) curve (18). The average sensitivity, specificity and other related indices of the studies were calculated using a random-effects model (19). Spearman’s rank correlation was performed as a test for threshold effect. The χ² and Fisher’s exact tests were used to detect statistically significant heterogeneity across the studies. If there were enough studies, subgroup analyses would be performed to explore potential between-study heterogeneity (20). All analyses were performed using two statistical software programs (Stata, version 11; Stata Corporation, College Station, TX, USA and Meta-DiSc for Windows; XI Cochrane Colloquium, Barcelona, Spain). All statistical tests were two-sided and P<0.05 was considered to indicate a statistically significant result.

Results

Quality reports and study characteristics

Following independent review, 181 publications concerning VEGF and pleural effusions were considered to be eligible for inclusion in the analysis. Of these publications, 144 were excluded for being beyond the scope of the present study, one was excluded due to the lack of a control group (9), two letters to the editor were excluded due to the limited data they contained (21,22), four publications were excluded as they recruited <10 patients in one of study groups (10,11,14,23) and 20 were excluded as they did not allow data extraction or calculation of the sensitivity and specificity (12,13,24–41). The remaining 10 studies, based on 514 patients with MPE and 511 without MPE, were available for the meta-analysis (42–51). The diagnostic characteristics of these studies and their STARD and QUADAS scores are outlined in Table I. Of the 10 articles included, 8 had STARD scores ≥13 and 9 had QUADAS scores ≥10.

Table I.

Summary of the studies included in the meta-analysis.

								Quality scores
Author/year (ref.)	Country	Method	Cut-off	TP	FP	FN	TN	STARD	QUADAS
Fiorelli et al, 2011 (51)	Italy	ELISA	652 pg/ml	31	5	18	25	15	11
Chen et al, 2010 (50)	China	PCR	NA	76	4	16	32	14	10
Zhou et al, 2009 (49)	China	ELISA	1.6 ng/ml	44	25	18	39	17	12
Duysinx et al, 2008 (48)	Belgium	ELISA	382 pg/ml	44	18	20	21	17	11
Cheng et al, 2008 (47)	China	PCR	NA	11	9	3	5	12	9
Xue et al, 2007 (46)	China	ELISA	945.7 pg/ml	34	7	8	38	17	12
Shu et al, 2007 (45)	China	ELISA	959.25 pg/ml	15	2	17	47	20	13
Sack et al, 2005 (44)	Germany	ELISA	NA	77	50	19	68	16	11
Momi et al, 2002 (43)	Japan	ELISA	2000 pg/ml	38	14	0	75	15	11
Yeo et al, 1993 (42)	USA	IFA	10 pm	18	7	7	20	11	10

TP, true positive; FP, false positive; FN, false negative; TN, true negative; QUADAS, quality assessment for studies of diagnostic accuracy; STARD, standards for reporting diagnostic accuracy; ELISA, enzyme-linked immunosorbent assay; PCR, polymerase chain reaction; IFA, immunofluorometric assay; NA, not applicable.

Diagnostic accuracy

Forest plots of the sensitivity and specificity of these 10 studies concerning pleural VEGF assays in the diagnosis of MPE are shown in Fig. 1. The average sample size of the studies included was 102 (range, 28–214). The sensitivity and specificity ranged from 0.47 to 1.00 [mean, 0.75; 95% confidence interval (CI), 0.72–0.79] and from 0.36 to 0.96 (mean, 0.72; 95% CI, 0.68–0.76), respectively. The PLR was 2.94 (95% CI, 1.97–4.41), the NLR was 0.38 (95% CI, 0.27–0.51) and the DOR was 9.05 (95% CI, 4.60–17.80). χ² values of sensitivity, specificity, PLR, NLR and DOR were 44.02, 67.36, 57.84, 30.72 and 34.21, respectively, with all P-values <0.001, suggesting a marked heterogeneity among the studies.

Figure 1.

Forest plots of sensitivity and specificity for pleural VEGF for the diagnosis of MPE. The point estimates of sensitivity and specificity from each study are shown as solid circles. Error bars indicate 95% CI. VEGF, vascular endothelial growth factor; MPE, malignant pleural effusion; CI, confidence interval.

Fig. 2 shows the SROC curve plotting the true-positive against the false-positive rates of the individual studies. As a global measure of test efficacy we used the Q-value, which is the intersection point of the SROC curve with a diagonal line from the left upper corner to the right lower corner of the ROC space and corresponds to the highest common value of sensitivity and specificity for the test. This point does not indicate the only or even the best combination of sensitivity and specificity for a particular clinical setting, but represents an overall measure of the discriminatory power of a test. In the present meta-analysis, the maximum joint sensitivity and specificity of our study was 0.75 (the Q-value). The area under the curve (AUC) was 0.82, indicating that the level of overall accuracy was not high.

Figure 2.

Summary receiver operating characteristic (SROC) curve of pleural VEGF for the diagnosis of MPE. The size of each solid circle represents the sample size of each study included in the present meta-analysis. The regression SROC curve indicates the overall diagnostic accuracy. VEGF, vascular endothelial growth factor; MPE, malignant pleural effusion.

Discussion

Our meta-analysis evaluates the diagnostic role of pleural VEGF in MPE and our data demonstrate that determining pleural VEGF results in a moderate sensitivity of 0.75 (95% CI, 0.72–0.79) and a specificity of 0.72 (95% CI, 0.68–0.76). It appears that VEGF determination may be most applicable in screening for MPE, although the relatively low specificity of VEGF may not be sufficient to confirm the diagnosis of MPE. This trade-off has significant clinical implications.

The SROC curve presents a global summary of test performance and shows the trade-off between sensitivity and specificity. The results of the analysis based on the SROC curve revealed that the maximum joint sensitivity and specificity was 0.75, while the AUC was 0.82, suggesting that the level of overall accuracy was not as high as expected. DOR, the ratio of the odds of positive test results in patients with the disease relative to those in patients without the disease, is a single indicator of test accuracy that combines the data from sensitivity and specificity into a single number (52). The value of a DOR ranges from 0 to infinity, with higher values indicating a superior discriminatory test performance (higher accuracy). A DOR of 1.0 indicates that a test does not discriminate between patients with the disorder and those without it. In our meta-analysis, the mean DOR was 9.05, indicating that VEGF assays appeared to aid the diagnosis of MPE. Since the SROC curve and the DOR are not easy to interpret and use in clinical practice, while likelihood ratios are considered to be more clinically meaningful, we also presented PLR and NLR as measures of diagnostic accuracy. A PLR value of 2.94 suggests that patients with MPE have an approximately 3-fold higher chance of being VEGF assay-positive compared with patients without MPE, but this is not high enough for clinical practice. On the other hand, NLR was found to be 0.38 in the present meta-analysis. This means that, if the VEGF assay result was negative, the probability that the patient has MPE is 39%, which is not low enough to rule out MPE.

Although the present study was performed with a comprehensive search strategy and data extraction, our meta-analysis has several limitations. First, we excluded conference abstracts and letters to the editor. This may lead to publication bias, which may also be introduced by inflation of diagnostic accuracy estimates, since studies that report positive findings are more likely to be accepted for publication. In addition, due to the limited numbers of the studies included, we did not use the STARD and QUADAS scores to perform the meta-regression analysis to assess the effect of study quality on the relative DOR of VEGF in the diagnosis of MPE. For the same reason, we were unable to explore whether study design, including blinded, cross-sectional, consecutive/random and prospective designs, affects the diagnostic accuracy.

The results of the present meta-analysis suggest that VEGF may, to a certain extent, play a role in the diagnosis of MPE, while its diagnostic value is not satisfactory. The combination of VEGF with other markers or examinations in pleural effusion may aid the establishment of the diagnosis of MPE. For instance, the combination of VEGF mRNA and endostatin mRNA has been reported to result in a high-diagnostic performance, with a sensitivity of 95.7% and an accuracy of 93.8%, respectively (50). The combined use of cytological examinations and VEGF has been found to increase the detection rate of malignancy with respect to cytological examination (51). Although the traditional method for the diagnosis of MPE remains cytological and/or histological examination, pathologists do not recommend making a diagnosis based on cytological samples alone due to the high risk of diagnostic error. In addition, invasive thoracoscopy may not be available in all hospitals, so the VEGF test is not only a useful adjunct to conventional diagnostic tools in diagnosing malignancy, but also guides the inclusion of patients who may benefit from further invasive procedures.

In summary, pleural VEGF determination plays a role in the diagnosis of MPE, while the results of VEGF assays should be interpreted in parallel with clinical findings and the results of conventional tests.