When is cystic fibrosis not cystic fibrosis? The importance of appropriately classifying patients

To the Editor:

The COVID-19 pandemic and its impact on people with cystic fibrosis (pwCF) continues to be a concern for the CF community. At the start of the pandemic, there was a great deal of apprehension as to how people with CF, a chronic, life shortening multisystem disorder that importantly affects the lungs, would fare. We are therefore appreciative of efforts to further illuminate clinical outcomes for those infected with SARS-CoV-2 and read with great interest the study by Hadi and colleagues [1].

The study was a retrospective secondary data analysis of COVID-19 patients in the TriNetX Network (507,810 COVID-19 cases which included 422 patients (0.08%) with CF). COVID-19 cases were identified by using ICD-10 codes and Logical Observation Identifiers Names and Codes (LOINC) for positive laboratory tests. CF was identified by the ICD-10 code of E84. The primary endpoint of interest was a composite endpoint of death or mechanical ventilation within 30 days of LOINC Codes of a positive SARS-CoV2 test or COVID-19 diagnosis. Propensity score matching in a 1:1 ratio with age, race, diabetes, hypertension, chronic lung disease, chronic kidney disease, nicotine dependence, heart failure, ischemic heart disease, BMI, and gender as covariates was conducted. The authors found, “following robust propensity matching, pwCF had a higher hospitalization rate (RR 1.56, 95% CI 1.20–2.04), critical care need (RR 1.78, 95% CI 1.13–2.79), and acute renal injury (RR 1.60, 95% CI 1.07–2.39) as compared to patients without CF.”

We note multiple, critical shortcomings in study design that challenge the accuracy of their conclusions. First, the likelihood of misclassification is high. Use of ICD-10 coding to identify pwCF has NOT been validated. The authors might have considered combining the prescription of key CF therapies with the ICD-10 code to refine their method. Misclassification in such a small cohort of CF patients would lead to potentially large effects. This is a likely fatal flaw especially given the nature of this unusual CF cohort. When compared to pwCF reported to the CF Foundation Patient Registry (CFFPR), which has data on 80–84% of the US CF population [2], one can see marked differences in the characteristics of the populations (Table 1 ). In addition to the differences highlighted in the table, other concerns include the high number of patients with ischemic heart disease, rare in CF, and the small number of patients on typical CF therapies such as inhaled antibiotics or CFTR modulators. Interestingly, the authors make no mention of how many of these patients were on pancreatic enzyme replacement therapy.

Table 1.

Comparison of CF cohort identified by Hadi and colleagues using ICD-10 codes versus overall CF population and subgroup of pwCF reported to have COVID-19 in 2020 to the CFFPR.

Patient Characteristics	Hadi et al. (n = 422)	CFFPR, 2020 (n = 31,411)	COVID-19 2020 cohort (n = 993)
African American	13.7%	3.5%	2.8%
BMI ≥30—(ages 20 and up in CFFPR)	26.8	8.9%	10.5%
Nicotine dependence (smoking in CFFPR)	12.3%	0.6%	0.6%
Hypertension	48%	7.1%	5.6%
CKD	21.6%	0.2%	0.4%
Use of inhaled antibiotics	28.9%	44.8%	50.9%
Use of CFTR modulators	16.2%	66.93%	74.5%

Fifty-eight transplant patients (14%) were also included in the TriNetX CF cohort. This is highly problematic as these patients are immunosuppressed and thus at great risk for severe outcomes from COVID-19. There were two viable options for dealing with this issue, exclude the transplant patients from the cohort or include immunosuppression as a propensity matching variable. Neither of these choices were made.

This brings us to our final point that the overall design of this analysis seems to miss the point about how propensity matching should be used. Propensity score matching is intended to control for confounding bias when estimating treatment effects from observational study data in which treatment status is not randomly assigned. The authors test whether CF (as defined by ICD-10 coding) results in a higher risk of death or mechanical ventilation essentially treating CF status as the primary exposure of interest. What is described as propensity score matching is effectively an exercise in comparing two poorly defined populations: one that includes subjects who have CF and another without CF but a similar distribution of comorbidities. The utility of this comparison for pwCF is unclear. The fact that the outcomes in their composite endpoints were no different in their unmatched analysis compared to their “robust propensity matching” raises red flags that this approach is inappropriate. The matched analysis cannot overcome misclassification of CF status and selection bias associated with the underlying data source. The study authors do nothing to formally evaluate their use of propensity score matching nor do they quantify the possible impact of underlying sources of bias inherent in the input data on the effect estimates they report.

There are now multiple published reports that use both individual national registries as well as collaborative international efforts to identify and track pwCF with COVID-19 [[3], [4], [5]]. These studies demonstrate that pwCF infected with SARS-CoV-2 have better outcomes than expected. Outcomes are consistently worse for those who have been transplanted as well as for those with advanced lung disease. That does not mean that pwCF without one of these risk factors are protected from poor outcomes, but there is no published evidence to suggest that they have worse outcomes than the general public.

Specifically, in the US, where Hadi and colleagues performed their study [1], data collected in the CFFPR, tells a very different story. As of the week of October 11, 2021, a total of 1996 pwCF diagnosed with COVID-19 and 15 deaths (0.75% of those with COVID-19) have been reported to the CFFPR, including 7 transplant recipients. This is a markedly different number than reported by Hadi. They report 7 more deaths in pwCF over a period that is approximately 10 months shorter than data reported to the CFFPR. Table 1 demonstrates how different their ICD-10 derived cohort of pwCF and COVID-19 is from what has been reported to the CFFPR.

In summary, the paper by Hadi and colleagues has several fatal flaws. It is evident that identifying pwCF through an ICD-10 code resulted in substantial misclassification that render their results ungeneralizable to pwCF. Moreover, the inclusion of transplant recipients without using immunosuppression as a propensity matching variable is highly problematic. Lastly, the authors misuse of propensity matching constitutes a third fatal flaw.

We encourage continued study and publishing of data that helps elucidate the impact of COVID-19 on pwCF. The clinical and patient/family CF community thirsts for this knowledge.