Did death certificates and a death review process agree on lung cancer cause of death in the National Lung Screening Trial?

Abstract

Background/Aims

Randomized controlled trials frequently use death review committees to assign a cause of death (COD) rather than relying on COD information from death certificates. The National Lung Screening Trial (NLST), a randomized controlled trial of lung cancer screening with low dose computed tomography versus chest x-ray for heavy and/or long-term smokers ages 55-74 years at enrollment, used a committee blinded to arm assignment for a subset of deaths to determine whether COD was due to lung cancer.

Methods

Deaths were selected for review using a pre-determined computerized algorithm. The algorithm, which considered cancers diagnosed during the trial, causes and significant conditions listed on the death certificate, and the underlying cause of death derived from death certificate information by trained nosologists, selected deaths that were most likely to represent a death due to lung cancer (either directly or indirectly) and deaths that might have been erroneously assigned lung cancer as the COD. The algorithm also selected deaths that might be due to adverse events of diagnostic evaluation for lung cancer. Using the review COD as the gold standard and lung cancer COD as the outcome of interest (dichotomized as lung cancer vs. not lung cancer), we calculated performance measures of the death certificate COD. We also recalculated the trial primary endpoint using the death certificate COD.

Results

1642 deaths were reviewed and assigned a COD (42% of the 3877 NLST deaths). Sensitivity of death certificate COD was 91%; specificity, 97%; positive predictive value, 98%; and negative predictive value, 89%. About 40% percent of the deaths reclassified to lung cancer COD had a death certificate COD of a neoplasm other than lung. Using the death certificate COD, the lung cancer mortality reduction was 18% (95% CI: 4.2-25.0), as compared with the published finding of 20% (95% CI: 6.7-26.7).

Conclusions

Death review may not be necessary for primary outcome analyses in lung cancer screening trials. If deemed necessary, researchers should strive to streamline the death review process as much as possible.

Introduction

The primary outcome in cancer screening randomized controlled trials is disease-specific mortality, and therefore accurate and unbiased assignment of the cause of death (COD) is critical. Trials can employ one of two methods to assign COD. One method uses COD information from the death certificate; information either can be used as listed or used to assign an underlying COD as per rules set forth by coding or government organizations, such as the United States National Center for Health Statistics. The other option is to establish a death review process that employs a committee, which is comprised of highly trained individuals who review medical records that document events leading up to the death, and can identify deaths that occur as sequelae of screening and treatment. The death certificate method is less resource-intensive and less costly than use of a death review process, but is assumed to be less accurate, in part because it could miss deaths that occur as sequelae of screening and treatment; those deaths, when identified through death review, are re-classified as primary-outcome deaths, and thus penalize the intervention. Use of a death review process requires that medical records be gathered, checked for completeness, redacted with regard to patient identifiers and study interventions, and made available to the reviewers. Reviews are time-consuming as record content must be synthesized into one COD. Most large cancer screening randomized controlled trials have used some form of death review to assign COD, but trial conclusions are usually similar when results using each method are compared with one another.¹

The National Lung Screening Trial (NLST) was a randomized trial of lung cancer screening comparing low dose computed tomography with chest x-ray for heavy/long-term smokers.² Institutional review board approval and patient consent were received. Participants were required to be between ages 55 and 74 years at enrollment and have a minimum of 30 pack-years of smoking history. Former smokers, who were enrolled in addition to current smokers, had to have quit within 15 years of randomization. A statistically significant twenty percent reduction in lung cancer mortality was observed with low dose computed tomography screening compared with chest x-ray screening.³ That analysis used “best information” for COD, with best information defined as the review COD if one was available and the death certificate COD otherwise. In this paper, we examine deaths that underwent death review to determine the degree of agreement on lung cancer COD assignment between review and death certificate CODs, with the aim of evaluating whether conclusions of the trial would have been the same had a death review process not been used.

Methods

The NLST death review process, formally called the endpoint verification process, has been documented in detail elsewhere.⁴ Death certificates for NLST participants were completed outside the auspices of NLST, with the cause of death section completed by the physician¹ in charge of the decedent’s care for the condition that resulted in death. Trial staff requested a copy of the death certificate and a trial nosologist used text recorded on the death certificate to assign an ICD-10 alphanumeric underlying COD code, as per the United States National Center for Health Statistics guidelines.⁶ A dichotomized lung cancer COD variable (death due to lung cancer versus other COD) was created, and for the remainder of this paper, that variable is referred to as the death certificate COD. A computerized algorithm considered all death certificate information (causes, significant conditions, and the assigned underlying COD) and history and timing of certain medical conditions, and identified deaths to be reviewed. A death could be selected through one or more of five pathways:

1. Any cause of death (including the assigned underlying COD) or significant condition noted on the death certificate matched an endpoint verification process COD of interest;

2. Deaths occurring in participants who were diagnosed during the trial with an endpoint verification process cancer of interest;

3. Deaths occurring within 60 days of select diagnostic procedures that occurred as a result of a screen suspicious for lung cancer or in conjunction with a diagnosis of lung cancer;

4. Deaths with chronic obstructive pulmonary disease noted on the death certificate or as the assigned underlying COD that occurred any time after select diagnostic evaluation procedures that occurred as a result of a positive screen or in conjunction with a diagnosis of lung cancer;

5. Deaths occurring within 6 months of screen suspicious for lung cancer or a clinically significant abnormality not suspicious for lung cancer.

CODs of interest were those that were most likely to represent a death due to lung cancer (either directly or indirectly) and deaths that might have been erroneously assigned lung cancer as the COD. Lung cancer was a cancer of interest; others were those that commonly metastasize to the lung, or vice versa. The algorithm also selected deaths that might be due to adverse events of diagnostic evaluation or treatment for lung cancer.

The death review committee included 5 physicians, including a medical doctor epidemiologist as chair, and members were blinded to trial arm assignment. The endpoint verification process resulted in assignment of a COD as well as a dichotomized lung cancer COD variable (death due to lung cancer versus other COD), with the latter referred to as the review COD for the remainder of the paper. A review COD of lung cancer was further classified as directly due to lung cancer or indirectly due to lung cancer, with the latter referring to deaths due to medical misadventures of lung cancer diagnostic or treatment. Using the review COD as the gold standard, we calculated performance measures for the death certificate COD. Those analyses reflect the final NLST dataset (data collected through September 28, 2010), and are restricted to deaths that were reviewed. When calculating a lung cancer mortality hazard ratio and 95% confidence interval (CI) that used the death certificate COD only, we used the same censoring date (January 15, 2009) and interim analysis statistical methods that were used to calculate published NLST primary findings³ so that a comparison to the published result would be possible.

Results

Forty-two percent of 3877 NLST deaths were reviewed, amounting to 824 deaths in the low dose computed tomography arm and 818 in the chest x-ray arm. Forty-five deaths in the low dose computed tomography arm and 57 deaths in the chest x-ray arm were reclassified. Table 1 presents cross-tabulations of the review COD and death certificate COD, dichotomized as lung cancer COD versus other COD, by screening arm. Percent agreement was nearly the same for the two arms (95% for the low dose computed tomography arm and 93% for the chest x-ray arm), as were sensitivity (low dose computed tomography: 92%; chest x-ray: 91%), specificity (97% for both), and positive predictive value (98% for both). Negative predictive value was 92% for the low dose computed tomography arm and 86% for the chest x-ray arm. In both arms, death review resulted in reclassification of 2% of deaths with a death certificate lung cancer COD to a review COD of “other”. Death review resulted in reclassification of 8% of deaths in the low dose computed tomography arm and 14% of deaths in the chest x-ray arm with a death certificate COD of “other” to a review lung cancer COD.

Table 1.

Performance measures by NLST trial arm: review COD versus death certificate COD

	Low dose computed tomography (n=824)				Chest x-ray (n=818)
		Review COD				Review COD
		Lung cancer	Other			Lung cancer	Other
Death certificate COD	Lung cancer	395	10	Death certificate COD	Lung cancer	460	9
	Other	35	384		Other	48	301
Sensitivity	0.92			Sensitivity	0.91
Specificity	0.97			Specificity	0.97
PPV	0.98			PPV	0.98
NPV	0.92			NPV	0.86
% agreement	0.95			% agreement	0.93
Kappa	0.89			Kappa	0.86

NLST: National Lung Screening Trial; COD: cause of death; PPV: positive predictive value; NPV: negative predictive value.

Death certificate COD refers to the underlying COD derived using the United States National Center for Health Statistics coding rules. Data collected as of September 28, 2010.

Table 2 presents information on those deaths that were reclassified. Nearly all deaths with a death certificate lung cancer COD that were reclassified (n=19) were reclassified as death due to a non-lung malignancy (n=8), a respiratory cause (n=5), or a cardiovascular cause (n=4). Most deaths with a death certificate COD of “other” that were reclassified to a review lung cancer COD (n=83) originally had been coded as death due to other malignancies (n=32), a cardiovascular cause (n=22), or a respiratory cause (n=13). Of those 83, 14 were classified as indirect lung cancer deaths. When examined by trial arm, the distributions of reclassification causes were similar for both directions of reclassification (that is, lung cancer COD to other COD and vice versa).

Table 2.

Lung cancer COD reassignments by NLST arm

Other cause	Death certificate: lung cancer Review: other cause (n=19 of 844)		Death certificate: other cause Review: lung cancer (n=83 of 768)
	Low dose computed tomography (n=10)	Chest x-ray (n=9)	Low dose computed tomography (n=35)	Chest x-ray (N=48)
Other malignancies	4 (0.40)	4 (0.44)	16 (0.46)	16 (0.33)
Cardiovascular disease	1 (0.10)	3 (0.33)	7(0.20)	15 (0.31)
Respiratory	4 (0.40)	1 (0.11)	5 (0.14)	8(0.17)
Complications of medical or surgical care	0 (0)	0 (0)	2 (0.06)	1 (0.02)
Other	1 (0.10)	1 (0.11)	5 (0.14)	8 (0.17)

NLST: National Lung Screening Trial; COD: cause of death. Others include (n=1 unless otherwise specified): dementia (n=2), pneumonia, septicemia (n=3), aplastic anemia, cerebral edema, diverticular disease, perforation of intestine, cirrhosis of liver, abnormal findings on diagnostic imaging of lung (n=2), accidental fall, and choking.

Absolute lung cancer mortality rates were slightly lower in both trial arms when calculations were made using death certificate COD as compared with best information (Table 3). A rerun of the final interim analysis using the death certificate COD led to a lung cancer mortality reduction of 18% (95% CI: 4.2-25.0), as compared with the published result of 20% (95% CI: 6.7-26.7), which used best information.³

Table 3.

Lung cancer deaths, mortality rates, and mortality reductions by method of COD ascertainment

	Best information*		Death certificate COD
	Low dose computed tomography	Chest x-ray	Low dose computed tomography	Chest x-ray
Lung cancer deaths	356	443	336	408
Lung cancer mortality rates (per 10,000 person-years)	247.1	309.1	233.2	284.6
Percent reduction in lung cancer mortality (95% confidence interval)	20.0 (6.7-26.7)		18.0 (4.2-25.0)

COD: cause of death.

^*Best information refers to the review COD if one was available and the death certificate COD otherwise. Data censored at January 15, 2009.

Discussion

It is imperative that cancer screening randomized trials have accurate information on the primary endpoint, typically disease-specific mortality, and that deaths due to sequelae of the screening process be classified in such a way that they count against the intervention of interest. But the use of a review process to assign COD, even for a subset of deaths, increases trial costs and could delay reporting. We have demonstrated that in the NLST, a large randomized trial of lung cancer screening, use of death certificate lung cancer COD would have led to the same conclusion as use of best information, which included review COD for those participants on whom it was available.

Doria-Rose et al. examined agreement between death review and death certificate COD assignment in four cancer screening trials conducted prior to the 1990s.¹ Death review in two lung cancer screening trials of male smokers, the Mayo Lung Project and the Johns Hopkins Lung Project, differed from that in the NLST, in that they included review of all deaths. Our arm-combined results for sensitivity (91%), specificity (97%), and positive predictive value (98%) are remarkably close to those from the Mayo and Hopkins trials (sensitivity: Mayo – 89% and Hopkins – 85%; specificity: 99% in both; positive predictive value: Mayo – 94% and Hopkins – 96%). Negative predictive value was lower in the NLST (89% versus 98% in Mayo and 96% in Hopkins), but that may be because the targeted approach selects non-lung-cancer deaths for review based on the chance that they were misclassified. As was the case in our analyses, the conclusions of the Mayo and Hopkins trials were the same regardless of whether death certificate or review COD was used. We conclude that if death review is deemed necessary in lung cancer screening trials, a review of a carefully chosen subset of deaths is a viable alternative to reviewing all deaths.

Our findings, while consistent with the Mayo and Hopkins findings, are most relevant to the current lung cancer screening trials. Screening in the NLST experimental arm was low dose computed tomography as opposed to chest x-ray and sputum cytology in Mayo and Hopkins. The NLST intervention, low dose computed tomography, is currently recommended as a lung cancer screening modality in the United States as a result of the NLST. The modalities used in the older lung cancer screening trials are either not recommended (chest x-ray) or are different due to advances in technology (sputum cytology). Another valuable feature of the NLST COD data is that they include deaths in women. NLST performance measures did not vary by sex (data not shown), suggesting that the Mayo and Hopkins death review findings are likely relevant to women as well. Our findings, however, are only relevant to lung cancer screening trials among heavy/long-term smokers, and they cannot speak to the need for death review in other trials of cancer screening. Doria-Rose et al.¹ also examined one randomized trial of colorectal cancer screening and one of breast cancer screening; for the colorectal trial, which reviewed all deaths, the degree of agreement on performance measures and primary endpoint was quite similar to that in the lung trials. No such similarity was seen for the breast trial, although only 80 deaths were reviewed, and selection criteria were employed that may have inherently led to poor performance measures (participants diagnosed with breast cancer with a death certificate COD other than breast cancer and participants with a suspect breast cancer COD due to other information on the death certificate). The NLST performance measures presented in this paper were dependent on selection criteria as well, and therefore our most important finding with regard to the need for death review is that use of death certificate information alone led to the same efficacy conclusion.

Midway through NLST, 120 deaths not chosen by the selection algorithm were randomly selected and submitted for death review to gauge the likelihood that the algorithm missed lung cancer deaths. In every instance, the death was assigned a review COD other than lung cancer, indicating that it is highly unlikely that a lung cancer death went uncounted in NLST.

Can death review be excluded from randomized trials of lung cancer screening? Our results suggest that death review may not be necessary for the purpose of primary outcome assessment, but to exclude it might cause some to question results of trials, particularly negative results. Cancer screening trials are major efforts; they are key to public health policy decisions and therefore it is critical that COD information is accurate. Additional review of the NLST COD assignment findings, including calculations of probabilities that certain selection criteria will trigger a change in COD assignment, could identify ways to further streamline the death review process for lung cancer screening trials.