Accuracy of FDG-PET to diagnose lung cancer in a region of endemic granulomatous disease

Abstract

Background

18 F-fluorodeoxyglucose positron emission tomography (FDG-PET) is used to evaluate suspicious pulmonary lesions due to its diagnostic accuracy. The southeastern United States has a high prevalence of infectious granulomatous lung disease, and the accuracy of FDGPET may be reduced in this population. We examined the diagnostic accuracy of FDG-PET in patients with known or suspected NSCLC treated at our institution.

Methods

279 patients identified through our prospective database, underwent an operation for known or suspected lung cancer. Preoperative FDG-PET in 211 eligible patients was defined by standardized uptake value, SUV > 2.5 or by description (“moderate” or “intense”) as avid. Sensitivity, specificity, positive and negative predictive values, likelihood ratios, and decision diagrams were calculated for FDG-PET in all patients and in patients with indeterminate nodules.

Results

In all eligible patients (n=211), sensitivity and specificity of FDG-PET were 92% and 40%. Positive and negative predictive values were 86% and 55%. Overall FDG-PET accuracy to diagnose lung cancer was 81%. Preoperative positive likelihood ratio for FDG-PET diagnosis of lung cancer in this population was 1.5 compared to previously published values of 7.1. In 113 indeterminate lesions, 65% had lung cancer and the sensitivity and specificity were 89% and 40% respectively. 24 benign nodules (60%) had false positive FDG-PET scans. 22 of 43 benign nodules (51%) were granulomas.

Conclusions

In a region with endemic granulomatous diseases, the specificity of FDG-PET for diagnosis of lung cancer was 40%. Clinical decisions and future clinical predictive models for lung cancer must accommodate regional variation of FDG-PET scan results.

INTRODUCTION

The proliferation of non-invasive imaging and interest in screening populations at high risk for non-small cell lung cancer (NSCLC) with computed tomography (CT) has increased pulmonary nodule detection rates [1, 2]. Clinicians rely heavily on available non-invasive diagnostic tools, and 18 F-fluorodeoxyglucose positron emission tomography (FDG-PET) has become widely accepted for the clinical diagnosis and staging of lung cancer in patients with suspicious pulmonary nodules [3, 4]. Recent studies of FDG-PET demonstrate a reduction in non-therapeutic resections (e.g. resection for benign lesions or metastatic disease) by 17 to 20% [5-8]. However, we questioned FDG-PET's accuracy in diagnosing lung cancer in patients with pulmonary lesions in our region where histoplasmosis is endemic.

A meta-analysis for FDG-PET accuracy to diagnose pulmonary lesions estimated a sensitivity of 94.2% and specificity of 83.3% [9]. Due to the suspected discrepancy between published specificity of FDG-PET, and our clinical observations of poor specificity, we sought to determine the accuracy of FDG-PET to diagnose lung cancer in our region of south-central United States where high endemic rates of granulomatous disease occur primarily from Histoplasmosis sp.

MATERIAL AND METHODS

Population

We identified 279 patients using Vanderbilt University Medical Center's Thoracic Surgery Quality Improvement database who underwent a surgical procedure for known or suspected lung cancer from January 1, 2005 to April 30, 2009. The Vanderbilt University Institutional Review Board approved this study (IRB #081298) and waived the need for individual informed consent. Clinical data elements were abstracted from the database and supplemented by chart review. Individuals without a reported lesion diameter were excluded from final analysis. Each lesion was confirmed by pathological examination following thoracotomy, thoracoscopy, mediastinoscopy, or bronchoscopy with biopsy for known or suspected NSCLC. In patients without cancer, additional review was conducted to determine, where possible, the etiology of the benign lesion.

Preoperative FDG-PET was performed in 216 patients. Five patients were excluded. One underwent re-operation after initial resection, two individuals had no reported lesion size preoperatively in their medical record, and two patients were excluded due to multiple nodules being evaluated. 211 patients were deemed eligible and reviewed further. Imaging data specific to the pulmonary nodule were abstracted from radiological reports found in the electronic medical record. FDG-PET results were classified into avid (likely for lung cancer) and non-avid (not likely for lung cancer) based upon radiologist description, or standardized uptake value (SUV) of the pulmonary lesion. An avid FDG-PET result was defined by a SUV of > 2.5 or by description such as “likely” or “malignant” and “moderate” or “intense” uptake by the reading radiologist.

Analysis

Contingency tables were created for the accuracy of FDG-PET scan to diagnose lung cancer in all patients and in patients without a pre-operative tissue diagnosis. Sensitivity, specificity, positive and negative predictive values, and positive and negative likelihood ratios were calculated for both groups. The observed values in our population were compared to results in the peer-reviewed literature. A test-treatment threshold graph was calculated using observed sensitivity and specificity to estimate positive and negative likelihood ratios. Observed ratios were compared to reported likelihood ratios.[10, 11] Overall accuracy was estimated in six strata of clinically determined tumor lesion sizes. Strata were ≤1 cm, >1 to 2 cm, >2 to 3 cm, >3 to 5 cm, >5 to 7 cm and >7cm. Sensitivity analysis of scans based upon year of the scan and whether the test used combined FDG-PET/CT or isolated FDG-PET were conducted.

RESULTS

In our population (n=211), the prevalence of lung cancer was 80%. Lesions ranged in size from 5 to 140 mm with a median of 25 mm. One hundred-thirteen patients had indeterminate nodules and their lung cancer prevalence was 65% (Table 1). Adenocarcinoma was the most common cancer histology (70/168, 42%), and granuloma was the most common benign diagnosis (22/43, 51%) (Table 1). Of these 22 granulomas, 7 were histoplasmosis by cytological staining or culture. Five of the 7 histoplasmosis granulomas were FDG-PET positive. Other known granulomatous diseases observed were blastomycosis (2), sarcoidosis (1), cryptococcosis (1) and Mycobacterium avium (1). The etiologies of the remaining ten granulomas were indeterminate by initial pathological staining.

Table 1.

Patient Characteristics

	All Patients	Indeterminate
	N=211	N=113
Gender: Male	53%	47%
Race: Caucasian	92%	93%
Age	64	63
Smoking Status – Never	21%	19%
– Ever	79%	81%
Pack-years	45	41
Predicted FEV1	77%	78%
Nodule Size (mm)	29	24
FDG-PET SUV (SD)a	7.0 (5.6)	6.6 (6.1)
Cancer Diagnosis	168 (80%)	73 (65%)
Adenocarcinoma	70	28
Squamous	56	26
NSCLC (not specified)	15	7
Large Cell	6	1
Small Cell	5	5
Bronchoalveolar Cell	5	2
Other Neuroendrocrine	5	0
Carcinoid	3	2
Melanoma	2	1
Lymphoma	1	1
Benign Diagnosis	43 (20%)	40 (35%)
Granuloma	22	21
Fibrotic	7	7
Inflammation	6	5
Hamartoma	4	4
Broncholith	1	1
Other	3	2

Mean values are reported for age, pack years, FEV1, and nodule size

^aScans with a reported maximum mean Standard Uptake Value (SUV) = 60 (28%), 151 scans did not have a maximum SUV reported.

SD – standard deviation

The sensitivity of FDG-PET for diagnosis of cancer in all 211 cases was 92% (95% CI: 86, 95). The specificity was only 40% (95%CI: 25, 56) (Table 2), far less than published data. Sensitivity and specificity in the 113 cases without a preoperative diagnosis was 89% (95% CI: 80, 95) and 40% (95%CI: 25, 57) respectively (Table 3). In our total population, the positive likelihood ratio for an avid FDG-PET scan was 1.5, and negative likelihood ratio observed was 0.2. A test-treatment plot of pre- and post-test probability of cancer compares the positive and negative likelihood ratios observed in our cohort (Figure 1) to published likelihood ratios (positive likelihood ratio 7.1 and negative likelihood ratio 0.06) for FDG-PET [9-11].

Table 2.

Contingency table for all patients (n=211)

FDG-PET	Cancer	Benign
Avida	154	26	PPVb 86% 95%CI: (80, 90)
Not-Avidc	14	17	NPVd 55% 95%CI: (36, 72)
			Prevalence 80% 95%CI: (74, 85)
Sensitivity 92% 95%CI: (86, 95)
Specificity 40% 95%CI: (25, 56)

The diagnosis was based upon pathological result of resected specimen.

^aFDG-PET avidity was defined by an SUV > 2.5 or moderate or intense uptake.

^bPPV = positive predictive value.

^cNot-Avid, cancerous lesions (n=14) include 6 with a pre-operative tissue diagnosis, 6 with documented growth, 1 with spiculated edge and 1 larger than 4cm in maximum diameter.

^dNPV = negative predictive value

Table 3.

Contingency table for patients with indeterminate lesions (n=113)

FDG-PET	Cancer	Benign
Avida	65	24	PPVb 73% 95%CI: (63, 82)
Not-Avidc	8	16	NPVd 67% 95%CI: (45, 84)
			Prevalence 65% 95%CI: (55, 73)
Sensitivity 89% 95%CI: (80, 95)
Specificity 40% 95%CI: (25, 57)

Patients did not have a preoperative diagnosis or had a non-diagnostic needle biopsy. The diagnosis was based upon pathological result of resected specimen.

^aFDG-PET avidity was defined by an SUV > 2.5 or moderate or intense uptake.

^bPPV = positive predictive value.

^cNot-Avid, cancerous lesions (n=8) include 6 with documented growth, 1 with spiculated edge and 1 larger than 4cm in maximum diameter.

^dNPV = negative predictive value

Figure 1.

Operative test-treatment decision diagram. Lines are the calculated likelihood ratios (LR) for the respective FDG-PET result based on the pre-test probability of cancer.

Example: In a patient with a 40% pre-FDG-PET probability of cancer, an avid FDG-PET scan gives a 55% probability of cancer in our observed population (Line A). A non-avid FDG-PET result yields a 12% likelihood of cancer (Line B). In this example using our data, FDG-PET result, whether avid or non-avid, does not change recommendation for resection. However, in other areas of the country, a 40% pre-FDG-PET probability of cancer results in a post FDG-PET probability of only 4% with a non-avid FDG-PET result (Line C) and an 84% post-FDG-PET probability of cancer in an avid FDG-PET (Line D).

False positive FDG-PET results were identified in 26 patients (26/43, 60.4%) with benign lesions: 12 granulomas, 6 fibrotic lesions, 3 inflammatory lesions, 1 broncholith, 1 hamartoma, and 3 of unknown etiology (Table 1). 5 of the 12 false positive granulomas were histoplasmosis. False negative results occurred in 14 of 168 (8.3%) patients with lung cancer: 6 occurred in adenocarcinoma greater than 1cm in maximum diameter; 1 each in smaller (9mm) lesions with adenocarcinoma and squamous cell carcinoma; and 6 were bronchoalveolar carcinoma or carcinoid tumors, histologies known to confound FDG-PET.

Overall accuracy was 81%. Accuracy varied somewhat by lesion diameter (Figure 2). The lowest accuracy (56%, 95%CI: 30%, 80%) occurred among 16 lesions with diameters ≤10 millimeters. The highest accuracy (95%, 95%CI: 68%, 99%) was among 19 lesions whose diameter was between 5 and 7centimeters. None of the groups by size were significantly different from the other 5 groups (p=0.08). Accuracy was less (72%) among those without a pre-operative diagnosis. Accuracy, sensitivity and specificity did not change among scans performed prior to 2008, when most FDG-PET without simultaneous CT scan occurred. Scans performed outside our institution (n=87) were less sensitive (86%) and specific (25%) compared to those performed at our medical center (N=123, sensitivity 97% and specificity 44%).

Figure 2.

FDG-PET accuracy for diagnosis of lung cancer by lesion diameter

*Accuracy = (True Positives + True Negatives) / N.

**All diameters are pre-operative, clinical maximum diameters by CT or PET/CT.

COMMENT

FDG-PET was far less specific (40%) in its diagnosis of cancer in patients residing in a region of endemic granulomatous disease when compared to previously published meta-analysis (83%). Our observed specificity was less than half that reported in meta-analysis or which has been used in most cost efficiency models [12-14]. In our cohort, sensitivity and specificity of FDG-PET was similar whether it included or excluded patients with a preoperative diagnosis of lung cancer. Nor were differences found between scans performed earlier in the study when compared to those performed more recently. FDG-PET scan was less sensitive to slowly growing carcinoid tumors and to lesions ≤ 10 mm in diameter. However, the sensitivity observed was similar (92% 95%CI: 86, 95) to that found in the literature (94.2%) [9, 15].

Almost half (46%) of the false positive scans observed in our population were specifically due to granulomatous disease. Croft et. al. reported similar results in their smaller, upper Midwestern (Iowa) cohort, with all imaging performed at a tertiary referral center, whereas our results are from the south-central US, from a variety of regional imaging centers [16]. Inflammation and granuloma were the most common benign results in their study as well. Fungal infections caused 66% false positives in one study using combined FDG-PET/CT imaging [17].

Lung nodules that are fungal in origin may be metabolically active and appear similar to malignancy on both CT and FDG-PET scans. Regional endemic prevalence of fungal disease confounds diagnosis of infection or cancer in asymptomatic individuals with pulmonary lesions. While our findings may be an extreme example of fungal disease [18, 19], histoplasmosis is endemic throughout much of the Mississippi, Ohio and lower Missouri River Valleys while coccidioidomycosis is endemic in the Southwest[20-22]. The efficacy of FDG-PET for the diagnosis of lung cancer in these regions of the country appears very different from other regions (Figure 1).

Clinicians ordering a FDG-PET scan for diagnosis of a pulmonary lesion must consider how the result of the scan will influence their decision-making. This decision depends upon the surgeon's pre-test estimate of the nodule's malignancy, the patient's operative risk, and their decision-to-operate threshold. For example, an FDG-PET scan in a patient with an estimated 40% pretest probability for lung cancer and an FDG-PET non-avid test result reduces the post-test probability of cancer to 4% using published negative likelihood ratio of 0.06 (Figure 1). Using the FDG-PET scan likelihood ratio of 0.20 as found in our study, a non-avid FDG-PET scan identifies a 12% post test probability of cancer. In this instance an FDG-PET scan whose accuracy reflects that found in the literature will change the surgeon's decision to a watch and wait strategy; however, an FDG PET scan with accuracy similar to our results, will have a post-test probability of cancer greater than 10% irrespective of test outcome. These patients may receive an FDG-PET scan for staging purposes, but the scan's information will not aid in diagnosis. Thus, the results of the FDG-PET scan would not affect the surgeon's decision to either surgically obtain a tissue diagnosis or follow the lesion with serial imaging. In similar regions of the country, pretest probabilities of cancer less than 6% or greater than 35% will not change the decision to operate for surgeons with a 10% post test likelihood of cancer.

The utility of FDG-PET for baseline staging of lung cancer is well-established [7, 15, 17, 23-26]. Our study addresses the accuracy of FDG-PET for the diagnosis of indeterminate lung nodules or lung masses in a high endemic area for granulomatous diseases. Our results were very similar to series previously reported from tertiary centers [16, 17]. Reporting of maximum SUV was limited in our population (28% with SUV). Our institution rarely reports SUV for various reasons [27-29]. However, the accuracy of FDG-PET scans performed at our institution, all of which were fusion FDG-PET/CT scans, was greater than scans performed outside our institution. The lack of SUV eliminates the possibility of using SUV as a continuous measurement of avidity and thus manipulating the sensitivity and specificity of the test based upon SUV.

Importantly, our study demonstrates the characteristics of FDG-PET scanning from a wide range of imaging centers. Forty-two percent of the scans were performed outside our institution. These scans used a variety of PET and PET-CT scanners, and were interpreted by local imaging physicians. Our results demonstrate the importance of utilization of pre-test probabilities and likelihood ratios that are regionally appropriate for clinical decision making in the management of indeterminate lung nodules or masses.

In summary, our study highlights limitations of FDG-PET to diagnose lung cancer in areas with endemic infectious lung disease. While false-negatives occurred in slowly growing, carcinoid tumors, over 40% false negative scans occurred in tumors larger than 1 cm with adenocarcinoma histology. Poor specificity due to multiple false positives from endemic granulomatous disease draws into question the cost-effectiveness of FDG-PET for diagnosing lung cancer in our region of the United States. Future cancer prediction models must consider the regional variation that may occur with FDG-PET scans. In addition, clinicians must be aware of FDG-PETs limitations to diagnose lung cancer and to determine whether a positive or negative result will change the operative decision. A personalized diagnostic plan developed for patients with suspected lung cancer depends upon the patient's risk factors for developing cancer [30, 31] and the accuracy of the diagnostic imaging that is specific to the patient's geographic location.

DISCUSSION

17. FDG-PET Does Not Accurately Diagnose Lung Cancer in an Area of Endemic Granulomatous Disease. Paper presented by Eric L. Grogan, MD, Nashville, TN. eric.grogan@vanderbilt.edu

Discussion by William F. Sasser, MD Missouri wfsassser@sbcglobal.net Dr. W. Sasser (St. Louis, MO):

Eric, Thank you very much for this presentation. I think it was very informative. In St. Louis, we experience many of the problems that you have because of the granulomatous nature of many of the pulmonary lesions. The Mississippi River area is very similar to the Cumberland River area in this respect.

In the manuscript you indicate that the imaging data were abstracted from dictated reports in the electronic medical records. There was no consistency of the multiple radiologists involved. Would it have been better and a stronger report if you had retrieved the disk and one common reader interpret all of the data? The SUV that you mentioned throughout the paper I assume was the max SUV. Dr. Cerfolio has pointed this out in the past that the max SUV is more reliable than just the mean SUV.

In one of your references, you used the meta-analysis from the JAMA in 2001. That was your standard for accuracy to diagnose pulmonary lesions. Most of these data that were published there were established many years before. That was before there was a combined PET-CAT instrument. That came into being in 2000. One of my other suggestions is that a more recent reference relating to the incidence of lung cancer involved with this would be appropriate. I know this was a paper was focused on pulmonary lesions. The PET does establish metastatic lesions as well as nodal lesions, which frequently are much easier to biopsy to get a diagnosis.

In closing, I would like to say that I appreciate the exercise you gave me in reviewing my statistical knowledge. Going back to words like sensitivity, specificity, accuracy, positive likelihood, negative likelihood, PPVs and NPVs has been very refreshing, and I appreciate the opportunity to relearn those.

17. FDG-PET Does Not Accurately Diagnose Lung Cancer in an Area of Endemic Granulomatous Disease. Response by Eric L. Grogan, MD, Nashville, TN.

DR. GROGAN: Thank you for your very insightful questions, and I think the audience would agree that they are right on.

Your first question was about the multiple radiologists reading the FDG-PET scans. I think that is both a strength and a weakness for this paper. This is a real world study. Many of the reports that have been done in the past have looked at single institutions with single reviewers, single PET scans, and that has been a strength of their paper. These are the patients that are referred to us, which include radiology reads from really all throughout the middle Tennessee, northern Alabama, southern Kentucky regions, and so we are sort of left with the data that we are given, and that is the reality and the advantage and disadvantage and limitations of this paper.

We went back and reviewed the reports and that was the data that the surgeons had at the time that they made the operative decision. So I think that is, as you mentioned, both a limitation but also a strength.

With regard to max SUV, that is what we used, and we categorized PET as avid if the SUV was dictated in the report greater than 2.5. If not, then we had to go with moderate to intense uptake as dictated by the radiologist that they were suspicious for cancer, and, as before, that is a strength and a limitation for our paper.

As far as the meta-analysis question, I think that is a very good point. However, even in the more current cost-effective modeling that Dr. Gould has done for us, the specificity numbers that he uses, which included even more current data, are very similar to what I have reported, which is around 80%. So although that is an older reference, I think even most of the data which comes from regions outside of the southeast still report specificity in the 80% range.

And with regard to metastatic and nodal disease, I think that is something for future work and we are looking into that. I don't currently have that data, but that is a very good point.

17. FDG-PET Does Not Accurately Diagnose Lung Cancer in an Area of Endemic Granulomatous Disease. Paper presented by Eric L. Grogan, MD, Nashville, TN. eric.grogan@vanderbilt.edu

Discussion by Robert J. Cerfolio, M.D., Alabama robert.cerfolio@ccc.uab.edu Dr. R. Cerfolio (Birmingham, AL): Congratulations on the paper. A few questions – first - you use an absolute value of 2.5. You would agree that a 1×1 × 1 cm tumor with a max SUV of 2.5 is very different than a 10× 10 × 10 cm tumor. We published about volume. If you haven't done it, could you go back and look at the max SUV per cm2 or cm3 of volume of tumor? That is number one.

Number 2 – if I understood slide correctly you are telling us that a 12% difference versus a 50 or 60% difference doesn't make a difference? In my life, if I can improve my odds from being wrong or right from 12% to almost 60%, I would take it every day.

17. FDG-PET Does Not Accurately Diagnose Lung Cancer in an Area of Endemic Granulomatous Disease. Response by Eric L. Grogan, MD, Nashville, TN.

DR. GROGAN: Your point with max SUV per cm2 is a good point. We have not done that. I think that is something we can look at as we go forward.

With regard to operative decision-making, I don't know about you, but I still would have a hard time watching a lesion in a healthy individual that has a 12% chance of malignancy, and I think comes down to a risk-benefit ratio for each individual patient. So we just have to be thoughtful with regard to the individual patient coming in, what their risks are.

DR. CERFOLIO: I agree, but have you considered the difference in the risk of waiting of a patient with a mass that has a low max SUV for example say 2.6, the risk of waiting three months, six months, for him and getting nodal disease is less than waiting for a patient with a maxSUV of 12.

17. FDG-PET Does Not Accurately Diagnose Lung Cancer in an Area of Endemic Granulomatous Disease. Paper presented by Eric L. Grogan, MD, Nashville, TN. eric.grogan@vanderbilt.edu

Discussion by Mark J. Krasna, M.D., Maryland markkrasna@catholichealth.net Dr. M. Krasna (Towson, MD): Eric, great paper. Another real world question. If you are confronted now with a 2 cm ground-glass opacity but the PET shows you no uptake, please tell us, how you would use it in that situation as opposed to a solid nodule where you have a high pretest probability that there might be malignancy with a smoking history, et cetera? What would you do with a GGO now, that comes in and you did the PET and the PET is negative?

17. FDG-PET Does Not Accurately Diagnose Lung Cancer in an Area of Endemic Granulomatous Disease. Response by Eric L. Grogan, MD, Nashville, TN.

DR. GROGAN: We had 14 false negatives, and a PET negative means that you are still going to have a greater than 10% chance of malignancy, and I think that in our population the decision has to be made based upon the CT scan in that particular patient depending on the pretest probability that that patient brings, and I think that is the key. We have to understand what the pretest probability is before we understand what the posttest probability with a positive or a negative PET result will be.