A Causal Machine Learning Framework for Estimating the Impact of Cancer Diagnosis on Receipt of Advance Care Planning

ABSTRACT

Objective

Develop a causal machine learning (causal ML) framework for estimating how a diagnosis (cancer in this study) affects the likelihood of receiving a specific health care service (advance care planning in this study) and associated heterogeneity.

Study Setting and Design

Our proposed framework leverages the causal forest method, combined with a population‐weighted resampling and averaging over estimations strategy, to estimate average treatment effects (ATEs) and conditional average treatment effects (CATEs). Post hoc, we used best linear projections to identify covariates associated with variation in the CATEs. We illustrate the framework by applying it to a stratified random sample of patients, where the strata are defined by the crosstabulation of cancer diagnosis (diagnosed vs. not diagnosed) and ACP receipt (documented vs. not documented).

Data Sources and Analytic Sample

We extracted deidentified patient data from October 2019 to October 2024 (n = 87,772) with explanatory variables in three categories: demographics, morbidity, and health care system utilization.

Principal Findings

In application of the causal ML framework, we found that patients diagnosed with cancer at this health care system to be at least 17.2% more likely to have documented ACP than similar patients not diagnosed with cancer. We also found significant heterogeneity. For instance, a one standard deviation increase in in‐person outpatient visits was associated with an on‐average increase in the CATE estimate (by 6.1 percentage points), while a one standard deviation increase in hospital admissions, inpatient days, and surgical duration in minutes was associated with an on‐average decrease in the CATE estimate (by −1.3, −5.6, and −0.5 percentage points, respectively).

Conclusions

The proposed causal ML framework enables estimation of the effect of a diagnosis on receiving a relevant health care service. In the cancer diagnosis context, it can identify patient groups less likely to receive ACP, thus informing service allocation strategies.

Summary.

• What is known on this topic

  • ○
    Machine learning (ML) based methods can be effective at estimating average and heterogeneous treatment effects, but implementation requires carefully addressing underlying assumptions.
  • ○
    In the cancer diagnosis and health services provisioning context, not all patients diagnosed with cancer have had a documented ACP discussion with a provider.
  • ○
    There are often significant differences in which types of patients have had documented ACP discussions.
• What this study adds

  • ○
    This study proposes a causal machine learning (causal ML) analytical framework that health care systems and applied researchers can apply toward understanding how a diagnosis (e.g., cancer) affects receipt of health care services (e.g., ACP).
  • ○
    We propose that this causal ML framework can be used to tailor when and how health care services, such as ACP conversations, are offered to specific patient subgroups.
  • ○
    In demonstrating how the causal ML framework can be applied, this study finds that patients diagnosed with cancer are more likely to have had a documented ACP discussion with a provider, but primarily when the patient utilizes more outpatient services.

1. Introduction

Estimating how a diagnosis affects the services patients receive, and how that effect varies across patient subgroups, is a central challenge in applied health care research. Traditional approaches, such as logistic or linear regression, often require specifying the model in advance and may struggle with high‐dimensional, correlated data or non‐linear treatment effects. Causal ML methods, such as causal forests [1, 2], are a potentially effective alternative to address such challenges and are explicitly designed to estimate causal effects. The causal forest method can flexibly estimate both the average treatment effect (ATE) and the conditional average treatment effects (CATEs) for specific patient subgroups without specifying a parametric model, while adjusting for confounding through robust estimation strategies. Such capabilities make this method particularly valuable when estimating treatment effects from observational health data. However, the application of causal ML methods requires additional considerations, such as addressing assumptions about the research design and data to generate as accurate estimates as possible. Such considerations are challenging to implement without a framework to guide the decision making and application process.

Therefore, in this study, we propose and apply a causal ML framework that integrates three components: (1) the causal forest method to estimate average and conditional (heterogenous) treatment effects, (2) a population‐weighted resampling and averaging over estimations approach to improve robustness when working with stratified samples, and (3) best linear projections to identify patient‐level covariates most strongly associated with variation in treatment effects. This framework is designed for health system and research applications where understanding subgroup differences can directly inform service allocation strategies.

We illustrate the framework in the context of cancer diagnosis and advance care planning (ACP). ACP “enables individuals to define goals and preferences for future medical treatment and care, to discuss these goals and preferences with family and health‐care providers, and to record and review these preferences if appropriate” [3]. ACP is especially relevant to patients with increasing mortality or morbidity risk, such as those advancing in disease severity, but can also be relevant to anyone interested in documenting their preferences for end‐of‐life or resuscitative care [4, 5, 6, 7]. As of 2016, Medicare now includes ACP conversations as a billable code in the Medicare fee schedule as current procedure terminology (CPT) code 99497 for dedicated conversations of 15–30 min in length and 99498 for sessions more than 30 min in length [8, 9, 10].

One especially important group for which to consider ACP is patients diagnosed with cancer [11]. Those with cancer, and their families, are especially vulnerable to guilt and conflicting emotions if providers do not document preferences early in disease progression and care processes [12]. ACP can be especially beneficial for providing patients with cancer more control over end‐of‐life decisions as well as alleviating challenging decision making in the‐moment [12, 13, 14]. While the efficacy of ACP documentation on clinical and utilization outcomes has been debated in the literature [7, 15, 16, 17], and the findings with regard to the effectiveness of ACP documentation are quite mixed [5, 18], it is important to note that ACP provides information and offers choice. An informed patient can decide how to proceed, or at least understand what options are available and understand that difficult conversations, even if not documented, may be necessary. Thus, consistent with prior views in this area [4], we regard the discussions themselves as an important aspect of patient education and choice, but an under‐researched aspect of the process.

A particular challenge, however, is to understand how to best estimate what types of patients are most and least likely to receive ACP, particularly as prior research has shown utilization of the CPT codes (99497 and 99498) to be low and inconsistent in many cases [8, 19, 20]. Information about the propensity of certain types of patients to have documented ACP discussions could be used to understand which patient subgroups could benefit from more targeted ACP service allocation strategies [5]. While prior research regarding disparities in ACP is helpful and can inform health systems seeking to more equitably provide ACP [21, 22], the differences in who is and is not offered such counseling will likely differ by health system and by patient population. Further, predicting the mortality or morbidity risk for all patients and prioritizing ACP for those with the highest clinical risk [23] does not fully address service allocation disparities and can result in too many or too few patients being targeted for ACP conversation. Therefore, we developed a causal ML framework to address this challenge.

2. Methods

We developed our framework using a causal inference‐based design [24, 25], to which ML could be applied. We specifically applied the causal forest, an ML‐based, non‐parametric method [1, 2]. We also employed a strategy that combines population‐weighted resampling and estimate averaging to adjust for the stratified random sample, followed by best linear projections. In what follows, we explain the full framework and discuss this framework compared to logistic and linear regressions. We then discuss the application of this framework to cancer diagnosis and documented ACP context.

2.1. Causal Forest

We chose the causal forest method over a more traditional design, such as ordinary least squares (OLS) with interactions, as the causal forest method is known to perform well with high‐dimensional and correlated data, can identify non‐linear effects, and is doubly robust in that the approach adjusts for observables. Causal forests adjust for covariates similarly to regression adjustment but use a tree‐based method, with propensity to be treated by applying augmented inverse propensity weighting (AIPW) to estimations [2, 26, 27]. Causal forests can be used to estimate ATEs, which are the differences in expected outcomes between treatment and control groups, as well as CATEs, which are treatment effects conditional on covariates (i.e., for specific subgroups). The causal forest method is increasingly being used in health‐focused studies [26, 28, 29, 30, 31] and has been shown to be robust, even in comparison to traditional causal inference approaches [32]. We estimated the results in this study using R 3.6.2 and the generalized random forest R package (grf version 2.3.2).

Assumptions are essential to address when applying any causal inference‐based method. The causal forest method primarily assumes unconfoundedness, which can be satisfied through selection on observables thought to address possible latent variables, and overlap, in that the propensity for treatment is bounded away from 0 and 1. The stable unit treatment value assumption (SUTVA) requires assuming that treatment is consistent between individuals and that treated individuals do not interfere with each other. However, as noted in our limitations, bias will still be present if not all confounders are included in the data. Thus, we sought to address confounding to the extent possible, both through inclusion of as many variables as were available and through the use of a method (causal forest) specifically designed to address confounding.

2.2. Population‐Weighted Resampling and Averaging Over Estimations

In our dataset, we used a stratified random sample of patients to ensure greater representation of the minority groups (e.g., the cancer patients). As described in more detail in a later section, we developed four strata via the crosstabulation of having cancer (or not) and having ACP documented (or not) (see Table 1 for details). We applied the causal forest via a resampling strategy to account for the difference between the amount of data requested for each stratum versus the proportions of each stratum in the full patient population. Then, to adjust for such disproportion, we resampled the data 100 times by the proportions observed in the full population, as discussed further below, without replacement. For each of these 100 resamples, we re‐ran the causal forest and then report here the average estimates across all 100 of these causal forest runs. We suggest this strategy to be an important contribution toward establishing robustness, even in the presence of a relatively large dataset, as the smaller proportion of data sampled (e.g., 1% in some cases) is highly likely to lead to biased estimates if a resampling strategy is not employed.

TABLE 1.

Stratified random sampling description.

	ACP conversation = No	ACP conversation = Yes
Cancer diagnosis = No	n requested = 25,000	n requested = 25,000
	n received = 25,000	n received = 25,000
	n with no missing data = 21,287	n with no missing data = 20,559
	Actual % in full population = 94%	Actual % in full population = 3%
Cancer diagnosis = Yes	n requested = 25,000	n requested = 25,000
	n received = 25,000	n received = 12,772
	n with no missing data = 19,085	n with no missing data = 11,031
	Actual % in full population = 3%	Actual % in full population = 1%

Note: All data is for the 5‐year period preceding October 2024 for this one health care system; actual % in full population means the percentage of patients in the full population from this health care system that met the criteria for the cell (i.e., 94% of the patients in the full population had not participated in ACP conversations and had not been diagnosed with cancer). Percentages were provided rounded to the nearest full percentage point.

2.3. Best Linear Projection

We ran best linear projections to interpret how the CATEs estimated via the causal forest might differ by relevant covariates (i.e., demographic, morbidity, and health care utilization variables, in our case). A best linear projection is a doubly robust version of the OLS regression of the CATEs on the covariates, which incorporates inverse propensity score weighting to scale the residuals and can be used to determine which variables are linearly associated with the CATEs [33, 34].

2.4. Comparing to Other Methods

There are important differences between using logistic regression, linear regression, and our causal ML framework. To start, the DV for the best linear projections applied in this study is the CATEs (τ(x)), which measure the difference in outcomes that a specific individual would experience under treatment versus control, conditioned on the observables and adjusted via an inverse propensity score weighting method that rescales the residuals.

$$\tau \left(x\right)=E\left[Y_i\left(1\right)-Y_i\left(0\right)|X_i=x\right]$$

where $Y_i\left(1\right)$ and $Y_i\left(0\right)$ correspond to the outcome being observed had we assigned treatment or control to sample i.

To provide support for this choice of method, we compare our proposed method to a linear regression and a logistic regression. Given our research design, the DV for the linear regression is binary (ACP yes (1) vs. no (0)), so the regression is a linear probability model. This has obvious limitations when fitting a straight line to a DV bounded by zero and one. However, this method is compared here as the best linear projection is also linear in parameters, so it serves as a basis for comparison, at least in terms of functional form, even though the DV is different.

The logistic regression is more appropriate for a binary DV, which provides consistency since the best fit line is bounded by zero and one, but again the DV is different from that of the best linear projection. Thus, the main point is that the three methods have different purposes, and their estimation strategies are different.

As discussed previously, the causal forest method provides the most accurate estimations as it accounts for confounding factors and has many other beneficial properties [1, 2, 35]. We also note that the estimated effect sizes and significance levels are different for the three approaches, as can be seen in the Supporting Information Appendix S1. This is expected, since neither logistic nor linear regression accounts for the various issues in observational studies. For example, the causal ML approach can identify important variables such as the number of hospital admissions and the number of emergency department (ED) visits, while a linear regression cannot. Such differences highlight the importance of using a ML causal framework since we can control for confounding factors and create comparable case and control groups. Finally, when using linear and logistic regression, typically one assumes some prior knowledge of the main and interaction terms to include in the model, and thus they are hypothesized rather than identified. In contrast, a causal ML method, while still affected by confounding, is designed to more thoroughly account for unobservables and is designed to explicitly identify important interactions based on treatment effect differences.

3. Results

3.1. Research Setting and Design

We applied this framework to data from one of the largest health care systems in the southeastern US. This health care system includes 9 hospitals, 6 of which provide cancer care approved by the American College of Surgeons and 7 of which provide chemotherapy services. In total, this health care system has approximately 2200 beds, participates in an accountable care organization, and had more than 100,000 inpatient discharges in the fiscal year ending June 2024. This health care system offers ACP conversations to patients and wanted to better understand how to determine what types of patients were most and least likely to have documented ACP conversations.

Institutional Review Boards approved this study as exempt due to the data being deidentified and also considered it to be a quality improvement project. Given the observational nature of the study, we followed the Reporting of Studies using Observational Routinely‐Collected health Data (RECORD) [36] and the Standards for Quality Improvement Reporting Excellence (SQUIRE 2.0) guidelines [37].

To estimate the ATE, the difference in the probability of having documented ACP if diagnosed with cancer versus not diagnosed with cancer, our design is at the patient unit of analysis. We consider being diagnosed with cancer as the treatment condition and having a documented ACP discussion with a provider as a binary outcome, where 1 means that an ACP discussion with a provider in this health care system was observed (documented) at least once and 0 means otherwise.

Having a cancer diagnosis was assessed through evaluation of all ICD‐10 codes assigned to the patient in their records within the 5‐year period of data made available. Having a documented ACP discussion was measured by having CPT codes 99497 or 99498 or an ACP note as part of the medical record. We note that one limitation is that the CPT codes are used when the conversations are 15 min or longer and ACP conversations may be less than that for some patients. Thus, we also included whether an ACP note was present to account for shorter length discussions. However, if informal discussions occurred and were not coded as either 99497, 99498, or documented as an ACP note, then such discussions would not appear in our data.

Covariates, used both to identify heterogeneous treatment effects and to account for potential confounders, included features in three categories: demographics (e.g., sex, age, race/ethnicity, etc.), morbidity (e.g., Elixhauser Index, number of active items in the problem list), and utilization (e.g., number of ED visits, number of in‐person outpatient visits, etc.). Detailed descriptions of the variables are in the Supporting Information Appendix S1.

3.2. Data Source and Sample

Deidentified data was obtained from the data warehouse for this health care system, which is fed by EPIC, the electronic health record system used, and other clinical systems from the hospitals, clinics, and facilities throughout the integrated delivery network. Data were extracted and aggregated per patient (as opposed to per admission, visit, or discharge) and included demographic, clinical (i.e., morbidity), and utilization features. The four strata generated were (Table 1), (1) yes cancer, yes ACP (n = 12,772), (2) yes cancer, no ACP (n = 25,000), (3) no cancer, yes ACP (n = 25,000), and (4) no cancer, no ACP (n = 25,000). For each of the strata, we requested 25,000 patients to be randomly sampled from the full population. For the “yes cancer, yes ACP” stratum, for which 12,772 patient records were obtained, 10,257 (80.3%) had a documented ACP discussion after receiving a cancer diagnosis. The 12,772 patient records obtained for “yes–yes” strata were a census of the patients fitting these criteria who had interacted with this health system at least once in the past 5 years since data extraction. Thus, the total number of patient records included in this dataset is 87,772. One note, however, is that some patient records had missing data. We report the numbers below for how many patient records had full information for the variables considered in our analyses, which were the records ultimately included in our analyses.

One issue with stratified random sampling is that a predetermined number of records requested for each stratum does not reflect the actual population proportion of patients in each stratum, likely biasing estimates determined from data that is different in proportion from the population. Thus, we also requested from this health care system the percentage of patients in their population who: (1) met criteria for inclusion (i.e., ≥ 18 years of age, seen at least once by this health system in the past 5 years since data extraction) and (2) would be classified into each of the four conditions/strata mentioned before (cancer yes/no, ACP yes/no). The health care system analyzed the data warehouse, calculated the percentages, and provided us with the following information. One percent of their patient population who met our inclusion criteria were diagnosed with cancer and had ACP (i.e., cancer = yes, ACP = yes). Three percent of their patients were not diagnosed with cancer and had ACP (i.e., cancer = no, ACP = yes). Ninety‐four percent of their patients were not diagnosed with cancer and did not have ACP (i.e., cancer = no, ACP = no). Three percent of their patients were diagnosed with cancer, but did not have ACP (i.e., cancer = yes, ACP = no). These percentages were used in our analyses, via a resampling strategy, to reflect the true proportions in this population. To account for these population percentages, we kept all the patients without ACP among the cancer patients, while taking $8333$ out of the total $\mathrm{12,772}$ ACP patients with a cancer diagnosis (to preserve the ratio between non‐ACP and ACP at 3:1). Similarly, among the non‐cancer patients, we kept all the patients without ACP, while taking $3/94\times \mathrm{25,000}$ of the ACP patients without a cancer diagnosis (to preserve the ratio between non‐ACP and ACP at 94:3).

3.3. Data Description

As reported in Table 2, for the full sample without missing data, 56% were female, the average age was 63.04 years, with the note that due to deidentification, ages over 89 years were set to 90 years, 62% were White, 30% were Black, 8% were another race, 48% were married, and 47% were either former or current smokers. In the subsample of patients diagnosed with cancer (cancer = yes) and who had ACP conversations (ACP = yes), the average age was 72.02 years, the proportion of females was 49%, the proportion of those who are White was (70%), the proportion of those who are Black was (25%), and the proportion of those of another race was 6%. In the subsample of patients diagnosed with cancer (Cancer = yes) who did not have a documented ACP conversation (ACP = no), the average age was 67.81 years, the proportion of females was 57%, the proportion of those who are White was 72%, the proportion of those who are Black was 21%, and the proportion of those of another race was 6%. Additional descriptive statistics are available in Table 2 and detailed descriptions of each variable are included in the Supporting Information Appendix S1.

TABLE 2.

Descriptive statistics.

	All sample (n = 71,962)				ACP no, cancer Dx yes (n = 11,031)				ACP yes, cancer Dx yes (n = 20,559)
Variables	Min	Max	Mean	Std	Min	Max	Mean	Std	Min	Max	Mean	Std
Treatment: diagnosed with cancer	0	1	0.44	0.50	1	1	1.00	0.00	1	1	1.00	0.00
Subgrouping: diagnosed with metastatic cancer	0	1	0.03	0.16	0	1	0.11	0.31	0	1	0.03	0.18
Outcome: ACP discussion observed (ACP)	0	1	0.42	0.49	1	1	1.00	0.00	0	0	0.00	0.00
Demographics
Female	0	1	0.56	0.50	0	1	0.49	0.50	0	1	0.57	0.49
Preferred language English	0	1	0.96	0.20	0	1	0.97	0.17	0	1	0.97	0.17
Ethnicity: Hispanic or Latino	0	1	0.06	0.23	0	1	0.03	0.17	0	1	0.04	0.20
Race: White or Caucasian	0	1	0.62	0.49	0	1	0.70	0.46	0	1	0.72	0.45
Race: Black or African American	0	1	0.30	0.46	0	1	0.25	0.43	0	1	0.21	0.41
Race: other or patient refused to disclose	0	1	0.08	0.28	0	1	0.06	0.23	0	1	0.06	0.24
Married	0	1	0.48	0.50	0	1	0.51	0.50	0	1	0.60	0.49
Age (in years) a	18	90	63.04	17.20	18	90	72.02	11.80	18	90	67.81	13.62
Has insurance b	0	1	0.81	0.39	0	1	0.90	0.30	0	1	0.90	0.29
Morbidity c
Elixhauser index d	0	22	3.79	3.14	0	18	6.24	3.09	0	22	3.87	2.64
Problem list active item count	0	349	24.93	23.84	1	349	40.50	26.67	1	173	22.90	18.59
Chief complaints total count	0	745	25.05	27.74	0	498	33.32	29.80	0	316	26.18	24.91
Ancillary procedures total count e	0	25	0.45	1.32	0	23	0.39	1.08	0	17	0.49	1.32
Former smoker	0	1	0.33	0.47	0	1	0.46	0.50	0	1	0.36	0.48
Current smoker	0	1	0.14	0.35	0	1	0.12	0.32	0	1	0.09	0.29
Health care utilization c
Outpatient in‐person visit count	0	678	47.35	47.24	0	571	74.21	55.97	0	648	51.82	45.63
Outpatient non‐urg. care telemed. visits count	0	199	0.99	2.70	0	60	1.30	2.93	0	62	1.08	2.47
Outpatient urgent care telemed. visits count	0	104	0.05	0.65	0	29	0.04	0.54	0	25	0.05	0.51
Hospital admission count	0	464	2.05	3.81	0	77	3.69	3.64	0	35	1.17	1.78
Emergency department (ED) visit count	0	728	4.34	10.47	0	528	5.31	9.40	0	203	2.17	4.38
Providers seen count f	0	17,192	402.04	548.28	10	17,192	712.33	650.26	1	5158	293.43	306.76
Preoperative encounters count	0	7	0.10	0.37	0	5	0.15	0.45	0	7	0.12	0.40
Inpatient days total sum	0	790	10.80	21.94	0	629	21.88	26.07	0	289	4.40	9.65
Surgical duration in minutes total sum	0	29,500	80.74	357.50	0	29,500	139.86	539.10	0	18,918	74.63	281.27

Note: Detailed variable descriptions are available in the Supporting Information Appendix S1.

Abbreviations: ACP = advance care planning, Dx = diagnosis.

^a Ages over 89 years of age were set to 90 years of age, for deidentification purposes, thus skewing the averages downward somewhat. One row had an age of 0 (zero) years, which was clearly an error and was removed from description.

^b Has insurance is set to true (1) if at least one encounter was billed to a commercial payer, Medicaid, or Medicare.

^c All variables are calculated for the 5 years prior to October 2024, which means that the counts and the sums represent all utilization within the 5‐year period.

^d The Elixhauser Index was calculated using the “comorbidity” package in R (https://cran.r‐project.org/web/packages/comorbidity/index.html) [38, 39], using the comorbidities that were tracked in the EHR for this health care system.

^e Includes diagnostics services such as imaging and laboratory tests, and therapies such as physical therapy.

^f This is the number of providers seen, which can include all the providers (including case managers) on a team, which is why the numbers may seem high.

3.4. Data Analysis Results

In this subsection, we first report the ATE and then report details about the heterogeneity (CATEs) found in the results. As shown in Table 3, the average ATE over 100 causal forest runs, where each run is a random resample based on population weights without replacement, was 17.6%. In other words, and perhaps unsurprisingly, for those diagnosed with cancer in this health system, they were 17.6% more likely to have an ACP discussion documented in their medical record. The estimate for the untreated (ATU) is a little higher at 17.8%, suggesting some differences in characteristics between those who have cancer and those who do not. Thus, the treatment effect weighted for common support (overlap) [40] may be considered a more accurate estimate in this case, at 17.2%. While there is some variation in the effect estimations, the estimates are consistent and are all positive, suggesting that, on average, for this patient population, getting diagnosed with cancer caused an increase in the likelihood of having a documented ACP discussion.

TABLE 3.

Average treatment effects.

Average treatment effect (ATE)	0.176
Average treatment effect on the treated (ATT)	0.172
Average treatment effect on the untreated (ATU)	0.178
Average treatment effect in the overlap (ATO)	0.172

Note: ATE refers to the effect of treatment if it was applied to everyone in the population; ATT refers to the effect of treatment if it is applied only to those in the treatment group; ATU refers to the effect of treatment if it is applied only to those in the control group; ATO refers to the effect of treatment if it is applied only to those who could be either in the treated or control group based on similarities in their propensity to be treated.

Even though the average effect is positive, what is equally important in our view is to understand what types of patients with cancer were more or less likely to have had documented ACP conversations. This has important implications for how resources are allocated for service provisioning. An analysis to identify whether heterogeneity was present is available in the Supporting Information Appendix S1. The result of this analysis was that heterogeneity was indeed present. To examine the sources of this heterogeneity (Table 4), we estimated the best linear projection of the independent variables on the CATEs estimated by the causal forest [33, 34]. We ran the best linear projection results for 100 causal forest runs, and report the average, standardized results in Table 4. We further report the results for the population and subpopulations as follows: (1) the full sampled population, (2) only those individuals with cancer that is not metastatic (i.e., assumed to be less severe), (3) only those individuals with cancer that is metastatic (i.e., assumed to be more severe), (4) only those individuals with cancer where the cancer diagnosis was less than or equal to the median number of days between diagnosis and the last encounter (i.e., more recent cancer diagnosis), (5) only those individuals with cancer where the diagnosis was less recent (i.e., more than the median number of days between diagnosis and the last encounter), (6) only those individuals with cancer where the Elixhauser index was less than or equal to the median (i.e., lower comorbidities and severity), and (7) only those individuals with cancer where the Elixhauser index was greater than the median (i.e., higher comorbidities and severity). We note that the best linear projection results are most valid when subpopulations are not determined with respect to the treatment variable or outcome variable. These subsamples (not metastatic vs. metastatic, cancer Dx more recent vs. less recent, and cancer patients Elixhauser lower vs. higher) are partially determined with information related to treatment (cancer vs. not) and thus must be considered as potentially biased. Thus, the most valid results are those for the full population considered.

TABLE 4.

Best linear projection results (standardized).

	All population (n = 47,989)		Cancer population where cancer is not metastatic (n = 6823)		Cancer population where cancer is metastatic (n = 794)		Cancer population where Dx was more recent (n = 15,060)		Cancer population where Dx was not recent (n = 15,052)		Cancer population where Elx. index is low (n = 17,092)		Cancer population where Elx. index is high (n = 14,498)
	Avg. coeff.	% Sig. at 5%	Avg. coeff.	% Sig. at 5% same direc.	Avg. coeff.	% Sig. at 5% same direc.	Avg. coeff.	% Sig. at 5% same direc.	Avg. coeff.	% Sig. at 5% same direc.	Avg. coeff.	% Sig. at 5% same direc.	Avg. coeff.	% Sig. at 5% same direc.
Demographics
Female	0.005	0%	−0.04	24%	−0.046	12%	−0.022	7%	−0.063	32%	−0.078	40%	0.007	4%
Preferred language English	−0.060	63%	0.006	1%	−0.074	2%	0.008	4%	0.009	0%	0.017	1%	−0.030	11%
Ethnicity: Hispanic or Latino	0.012	0%	−0.09	7%	0.022	6%	0.026	0%	−0.209	13%	−0.105	4%	−0.039	0%
Race: White or Caucasian	0.006	0%	−0.076	15%	0.012	1%	−0.074	19%	−0.048	6%	−0.101	16%	−0.019	11%
Race: Black or African American	0.023	9%	−0.02	4%	0.003	0%	−0.047	8%	0.028	0%	−0.033	6%	0.002	0%
Married	−0.005	5%	−0.029	18%	−0.059	18%	−0.038	23%	−0.027	7%	−0.051	22%	−0.007	2%
Age	−0.024	100%	−0.08	90%	−0.169	92%	−0.165	100%	−0.01	1%	−0.114	88%	−0.072	96%
Has insurance	−0.037	100%	−0.072	99%	−0.112	76%	−0.082	100%	−0.046	19%	−0.100	99%	−0.038	67%
Morbidity
Elixhauser index	−0.020	94%	0.009	1%	−0.004	4%	−0.013	11%	0.024	5%	0.086	18%	−0.010	5%
Problem list active item count	0.002	0%	0.001	0%	−0.003	2%	−0.016	13%	0.022	6%	0.022	3%	−0.007	5%
Chief complaints total count	0.006	1%	0.029	29%	0.009	2%	0.036	35%	0.014	3%	0.028	4%	0.018	24%
Ancillary procedures total count	0.013	86%	−0.003	0%	0.016	13%	0.027	32%	−0.014	3%	−0.012	3%	0.009	17%
Former smoker	−0.003	1%	−0.024	9%	−0.001	5%	−0.025	12%	−0.03	7%	−0.044	11%	−0.001	5%
Current smoker	0.031	75%	0.048	16%	−0.011	1%	0.036	13%	0.021	3%	0.063	11%	0.014	4%
Utilization
Outpatient in‐person visit count	0.061	100%	0.012	2%	0.002	0%	−0.003	1%	0.034	33%	0.011	1%	0.008	5%
Outpatient non‐urgent care telemedicine visits count	−0.004	2%	0.001	1%	0.011	4%	0.016	16%	−0.003	1%	−0.009	1%	0.005	4%
Outpatient urgent care telemedicine visits count	0.006	27%	0.002	1%	−0.014	0%	−0.012	0%	0.007	1%	0.007	2%	−0.001	0%
Hospital admission count	−0.013	98%	−0.035	24%	−0.021	6%	−0.026	6%	−0.055	40%	−0.046	4%	−0.031	53%
Emergency department (ED) visit count	0.012	51%	0.034	5%	0.057	11%	0.025	2%	0.059	20%	0.107	19%	0.025	22%
Providers seen count	0.001	0%	0.015	10%	−0.005	0%	0.031	19%	0.003	0%	0.032	0%	0.012	13%
Preoperative encounters count	−0.003	1%	−0.008	13%	0.003	9%	−0.012	22%	−0.002	1%	−0.014	14%	0.000	0%
Inpatient days total sum	−0.056	100%	−0.073	99%	−0.097	49%	−0.094	97%	−0.077	92%	−0.150	85%	−0.048	91%
Surgical duration in minutes total sum	−0.005	72%	−0.002	0%	0	0%	−0.002	0%	−0.001	0%	−0.006	0%	−0.001	0%
Alpha (intercept)	0.378	100%	0.759	100%	1.318	98%	1.22	100%	0.309	15%	0.919	100%	0.653	100%

Note: All n values are approximate as each run might have slightly different n values due to missing data.

The dependent variable (DV) in a best linear projection is the predicted CATEs from the causal forest.

All non‐binary variables were standardized. Thus, the coefficients can be interpreted as the number of standard deviations the DV will change with respect to a one standard deviation change in an independent variable. The non‐binary variables are: age, Elixhauser index, problem list active item count, chief complains total count, ancillary procedures total count, outpatient in‐person visit count, outpatient non‐urgent care telemedicine visits count, outpatient urgent care telemedicine visits count, hospital admission count, emergency department (ED) visit count, providers seen count, preoperative encounters count, inpatient days total sum, and surgical duration in minutes total sum.

All data is from the past 5 years since data extraction (i.e., September 2024). Thus, all total counts and sums represent total count and sum for the 5‐year period prior to October 2024.

All population = Results from samples within the strata for the full patient population.

Not metastatic = Results from samples within the strata for only the population with cancer that has not metastasized, determined by whether the patient has a problem in the active problem list that includes the word “metastatic”; Metastatic = Results from the samples within the strata for only the population with cancer that has metastasized, determined the same way as mentioned above.

Dx was recent ≤ 792 days between Dx and last encounter (median); Dx was not recent > 792 days between Dx and last encounter.

Elixhauser score is low where score ≤ 3 (median); Elixhauser score is high where score is > 3.

The results are averaged across the 100 causal forest runs and subsequent best linear projections (via standard OLS regression) run for each of the 100 causal forests generated. Therefore, the coefficient is an average coefficient (avg. coeff.) as it is averaged across all 100 best linear projections sets of results. Significance was determined at the 5% level and the column “% Sig. 5%” represents the percentage of times the coefficient was significant at 5% in each of the 100 sets of results where the coefficient was in the same direction. Hundred percent means that the coefficient was significant in the same direction 100% of the time in the 100 best linear projections.

For demographic variables, in the full population results, we primarily found both preferred language English (binary variable, avg. coeff. −0.060, significant at 5% in 63 out of 100 runs), age (standardized continuous variable, avg. coeff. −0.024, significant at 5% in 100 out of 100 runs), and has insurance (binary variable, avg. coeff. −0.037, significant at 5% in 100 out of 100 runs) to each have a negative estimated effect. We did not observe much of a significant estimated effect for preferred language English in any of the subpopulation analyses. We did, however, observe negative estimated effects for age in all the subpopulation analyses, except for the subpopulation with longer time since cancer diagnosis (i.e., not recent; measured by being above the median number of days for the patients in this dataset). Similarly, for patients with insurance, we observed significant negative estimated effects for all subpopulations, except for the those who have had their cancer diagnosis for a longer time. We also note that we did not find many significant effects for sex, ethnicity (Hispanic or Latino), race (White or Black), or married.

For the morbidity variables, in the full population results, we primarily found that receiving more ancillary procedures (standardized continuous variable, avg. coeff. 0.013, significant at 5% in 86 of 100 runs) increases the estimated likelihood of receiving ACP when one is diagnosed with cancer, as does being a current smoker (binary variable, avg. coeff. 0.031, significant at 5% in 75 of 100 runs). We also found that a higher Elixhauser index was associated with a decrease in the estimated likelihood of having a documented ACP counseling session (standardized continuous variable, avg. coeff. −0.020, significant at 5% in 94 out of 100 runs). We do not find significant effects in more than half of the runs for these variables for any of the subpopulations.

For the utilization variables, we found that those with more in‐person outpatient visits (standardized continuous variable, avg. coeff. 0.061, significant at 5% in 100 of 100 runs) and ED visits in some cases (standardized continuous variable, avg. coeff. 0.012, significant at 5% in 51 out of 100 runs) were associated with an increase in the treatment effect estimate. More hospital admissions (standardized continuous variable, avg. coeff. −0.013, significant at 5% in 98 of 100 runs), more inpatient total days within the 5‐year period (standardized continuous variable, avg. coeff. −0.056, significant at 5% in 100 of 100 runs), and more surgical duration (standardized continuous variable, avg. coeff. −0.005, significant at 5% in 72 of 100 runs) were associated with a decrease in the treatment effect estimate. The results for in‐person outpatient visits were not significant for most runs in the subpopulations. The results for hospital admissions were somewhat significant for the higher Elixhauser Index subpopulation (standardized continuous variable, avg. coeff. −0.031, significant at 5% in 53 of 100 runs) but were not significant in the majority of runs for the other subpopulations. The inpatient days sum results were significant and negative in all the subpopulations, except for the metastatic subpopulation (standardized continuous variable, avg. coeff. −0.099, significant at 5% in only 49 of 100 runs). The surgical duration was not significant in any of the subpopulations.

4. Discussion

We proposed a causal ML framework for estimating how a diagnosis (cancer in this study) affects the likelihood of receiving a specific health care service (ACP in this study) and associated heterogeneity.

4.1. Principal Findings

We demonstrated that this causal ML framework can be effective for estimating not only average effects, but also heterogeneous effects. This framework can be especially useful to apply when seeking to allocate resources toward patients who may need the treatment the most, but are currently unlikely to receive it or have it documented.

In our results specific to this health care system, we found that patients diagnosed with cancer have an increased likelihood of having documented ACP. We also note, though, that not all patients with cancer, including many diagnosed with metastatic cancer, have had a documented ACP discussion with a provider in this health care system. Thus, one opportunity is an increased emphasis on providing ACP to all patients with cancer, with a priority placed on those with more severe cancer.

Regarding identifying specific subgroups, we primarily found that there was significant heterogeneity, and specifically that some types of patients (i.e., subgroups) were more or less likely than others to have an ACP discussion observed when diagnosed with cancer. For instance, we found that more ancillary procedures (e.g., diagnostics or therapeutic services) and that more outpatient in‐person visits lead to an increased likelihood of receiving ACP, suggesting perhaps that more time spent in‐person and receiving (or reading) diagnostic or therapy orders and results leads to more opportunities for ACP. This may also imply, however, that providing ACP is somewhat ad hoc, in that a provider will only offer such a discussion when it seems that utilization (i.e., number of visits or ancillary procedures) is high. Or, it could imply that ACP discussions were more efficiently offered in outpatient rather than inpatient settings, and perhaps this provides a patient flow efficiency advantage.

4.2. Limitations and Future Research

This study has several limitations. First, the data is from one health care system, albeit across multiple sites within this health care system and a large sample of patients, but treatment effects can only be estimated using this sample, not fully identified. Second, we may not capture all ACP sessions in the data, as either 99497 or 99498 were not billed, or the conversation was so short that an ACP note was not filled out. Another possibility is that the patient or caregiver proactively alerted a provider as to their wish not to hear about ACP options, and thus the conversation did not occur. It is also possible that ACP was offered by a provider outside of the focal health care system, for which we do not have data. Third, we make assumptions associated with causal inference methods (SUTVA, unconfoundedness, positivity), but these assumptions may be violated. For instance, we do not capture all possible confounders, which means that our effect estimates are not without bias and that we do not capture all the heterogeneous effects. For instance, some patients or caregivers may have had more intrinsic motivation for receiving ACP than others, and we did not observe such latent variables. Further, additional confounders not observed include the prevalence of other conditions associated with ACP, provider characteristics (including propensity to offer ACP), site characteristics (such as patient volumes and average time per appointment), whether cancer screening was received, and site‐specific ACP policies. Fourth, the offering of ACP conversations, and the associated billing codes, is a relatively new phenomenon. Given that it may take years for new practices to disseminate, it is likely that there is significant variation in which providers offer ACP and which do not. A more granular identification of which specific providers are more likely to have patients who have (or have not) been offered ACP is likely an important area for future research. Finally, the data extracted for this study was a random sample from the patient population at this health care system. Future work could consider larger and multi‐hospital system datasets.

4.3. Implications

Overall, we show that a causal ML framework can be used to estimate ATEs as well as heterogeneity and that, specifically, there is heterogeneity in which types of patients diagnosed with cancer have had documented ACP discussions with a provider. Implications for health care systems and applied researchers are that services such as ACP, which are often not uniformly offered, will benefit from more targeted resource application. This framework helps to identify which patient subgroups need to be more effectively targeted and can be extended to other diagnoses and services.

Conflicts of Interest

The authors declare no conflicts of interest.

Supporting information

Table S1: Variable descriptions

Figure S1: Target operating characteristic chart

Table S2: Comparison of coefficients from logistic regression, linear regression, and best linear projection of causal forest CATE estimates on covariates.

Baird A., Cheng Y., Lesandrini J., and Xia Y., “A Causal Machine Learning Framework for Estimating the Impact of Cancer Diagnosis on Receipt of Advance Care Planning,” Health Services Research 61, no. 2 (2026): e70039, 10.1111/1475-6773.70039.