Addressing biases in analysis of time-of-day of infusion: Evaluation of NCI/SWOG Trial S1404 among participants with high-risk resectable melanoma who received adjuvant anti-PD-1 therapy

Abstract

Background:

Multiple reports have suggested that receiving immunotherapy infusions “earlier” in the day is associated with improved outcomes, including longer overall survival and lower toxicity rates. However, the definition of “early” varies between publications. Reports also fail to account for confounding factors (including distance to infusion center), are subject to survivor bias (analyzing post-baseline factors at baseline), and do not adjust p-values for multiple comparisons when evaluating multiple potential thresholds for early versus late time-of-day of infusion.

Methods:

We analyzed a previously reported multi-center clinical trial evaluating pembrolizumab as adjuvant therapy for patients with resectable high-risk melanoma. Standard statistical methodologies that account for potential biases were used to evaluate the association between time-of-day of infusion and clinical outcomes.

Results:

628 participants received pembrolizumab and had time of first infusion recorded. Median age was 55 years, range 20-82. Odds of infusion before 11:00 hours increased by 32% over 12 months of therapy (p=0.013). Participants living further from their treating institution had later infusion times on average: odds of infusion before 11:00 decreased by 9% for each additional 50 miles (p=0.017). The optimal cut-point for first infusion time for OS was 15:48 with hazard ratio (HR)=1.40; changing the cut-point by 30 minutes earlier to 15:18 decreased HR to 0.98, indicating lack of robustness of the threshold. No significant association was identified between proportion of early infusions and outcomes in multivariable time-dependent Cox regression models.

Conclusions:

In this multi-center trial of adjuvant pembrolizumab for patients with high-risk melanoma, analyses that account for common sources of bias found no significant association between recurrence-free or overall survival and time-of-day of infusion.

BACKGROUND

Humans have diurnal variations in hormone levels and seasonal and/or circadian rhythms that could theoretically interact with and influence the outcome of immune checkpoint blockade cancer treatment. Multiple retrospective analyses have reported a significant association between time of infusion of immunotherapy and patient outcomes, with earlier times of infusion associated with improved outcomes.^1-7 In addition, data from one randomized trial of time-of day of chemo-immunotherapy among patients with non-small cell lung cancer has been reported.⁸ Other analyses have found seasonal associations with outcomes including toxicity.⁹ Many of these analyses are subject to potential biases including: not controlling for potential confounding variables (such as distance between patient and infusion center)¹⁰, p-values that do not adjust for multiple comparisons when evaluating multiple potential cut-points for “early” versus “late” infusions,¹¹ analysis of post-baseline data as baseline variables,¹² and failure to account for competing events.¹³ Our goal in this work was to analyze a well-annotated clinical trial cohort of participants with high-risk, resectable melanoma who received adjuvant anti-PD1 immunotherapy¹⁴ and use appropriate statistical analyses to avoid the potential pitfalls of prior reports.

METHODS

Our data resource was SWOG trial S1404 that has been previously reported (NCT02506153, funded by the National Cancer Institute [NCI]).¹⁴ We analyzed eligible participants who were randomized to the pembrolizumab arm (200mg intravenously every three weeks for 18 doses) and who received at least one dose of protocol therapy. The protocol and consent were reviewed and approved by SWOG, the NCI, the NCI Central Institutional Review Board, and the Institutional Review Board of each participating institution. All participants signed informed consent and were treated according to the Declaration of Helsinki.

Overall survival (OS) was measured from date of first infusion to death from any cause, with participants last known to be alive censored at date of last contact. Recurrence-free survival (RFS) was measured from date of first infusion to the first of either recurrence or death from any cause, with participants last known to be alive without recurrence censored at the date of last contact. Time to toxicity was measured from date of first infusion to date of first toxicity; recurrence or death before toxicity observation were analyzed as competing events. Treatment attribution was determined by treating investigator and possibly, probably, and definitely related toxicities were defined as treatment-related. Baseline characteristics were compared between groups using Fisher’s exact tests and Wilcoxon rank-sum tests. OS and RFS were estimated using the Kaplan-Meier method. Cumulative incidence of toxicity was estimated non-parametrically. Associations with OS and RFS were assessed with Cox regression models; associations with time to toxicity were assessed with cause-specific hazard models. Multivariable models controlled for the following pre-randomization covariates regardless of univariate significance: age (18-64 versus 65 and older), sex (female versus male), body mass index (<25, 25-29, 30 and higher), ECOG performance status (0 versus 1), randomization stratification factors PD-L1 status (positive, negative, not known) and stage (AJCC 7^th edition IIIA, IIIB, IIIC, IV), and BRAF status (mutated, wild-type, not known). Time-of-day of first infusion was analyzed as a baseline covariate; infusion times after the first treatment were analyzed with time-dependent regression models. Patterns in association between time of first infusion and RFS or OS were visualized with Martingale residual plots. Thresholds for “early” versus “late” time-of-day of infusion were evaluated with maximum log-rank statistics; p-values adjusting for multiple (correlated) comparisons were calculated using exact Gaussian statistics.¹¹ Sensitivity of selected thresholds was assessed by evaluating thresholds 15 and 30 minutes before and after the selected thresholds. Patterns in time of infusions over time were evaluated using linear mixed models with participant-level Gaussian random effects and generalized estimating equations with independent working correlation. Distance between participant and infusion center was calculated in miles based on zip code; distance was evaluated as a potential confounding variable due to prior reports of a prognostic association between distance and outcomes¹⁵ and the potential for association between distance and time of infusion. All times were local and reported in 24-hour format. Data follow-up cut-off was June 30, 2025. All analyses were completed with R version 4.5.1.

RESULTS

Six hundred forty-seven (647) eligible participants were randomized to the pembrolizumab arm; 628 participants who received at least one dose of pembrolizumab and had time-of-day of first infusion recorded were included in analyses. The analyses reported here include 277 RFS events and 148 deaths. Participant characteristics are summarized in Table 1.

Table 1:

Pre-randomization participant characteristics. N (%) reported.

Factor	N=628
Age at randomization
18 - 64 years	487 (78)
65 years and older	141 (22)
Sex
Female	258 (41)
Male	370 (59)
BMI at randomization
Under 25	149 (24)
25 – 29	233 (37)
30 and higher	246 (39)
ECOG Performance status
0	527 (84)
1	101 (16)
Stage (AJCC v7)
Stage IIIA (N2a)	73 (12)
Stage IIIB	305 (49)
Stage IIIC	210 (33)
Stage IV	40 (6)
Primary tumor ulceration
Ulceration present	234 (37)
Ulceration not present	266 (42)
Not reported	128 (20)
BRAF mutation
Wild type	169 (27)
Mutated	134 (21)
Unknown	325 (52)
PD-L1
Positive	518 (82)
Negative	91 (14)
Indeterminate	19 (3)

Abbreviations: AJCC, American Joint Commission on Cancer; BMI, body mass index; ECOG, Eastern Cooperative Oncology Group; HDI, high-dose interferon alfa-2b; ipi, ipilimumab; PD-L1, programmed death receptor-1 ligand.

Hour of first infusion is summarized with a histogram in Supplemental Figure 1. Martingale residual plots for time-of-day of first infusion were fairly flat across times of day and did not indicate a pattern consistent with earlier infusion times being associated with longer RFS or OS (Figure 1). Maximum rank statistics selected 13:18 for the RFS cut-point and 15:48 for the OS cut-point (Supplemental Figure 2); neither time was associated with a statistically significant difference in outcome (Figure 2). We note that the hazard ratio for OS favored later infusion times; if there were no association between time of day and outcome, we would expect that maximum rank statistics in different datasets would select cut-points in both directions (both favoring earlier infusion times and favoring later infusion times). The “sawtooth” shape of the log-rank statistics across potential cut-points (Supplemental Figure 1) is not consistent with earlier infusion times being associated with better outcomes. When evaluating thresholds 15 and 30 minutes before or after the “optimum” thresholds, hazard ratios become more null indicating the lack of robustness for the threshold to support a pattern consistent with earlier time of day being associated with improved outcomes (Table 2).

Figure 1:

Martingale residual plots for recurrence-free survival and overall survival. For each patient, their residual is equal to the difference between their event indicator (equal to 0 if censored, equal to 1 otherwise) and the probability of an event at their censoring or event time (probability based on calculated cumulative incidence). The largest value the residual can take is 1. Censored patients with long follow-up will have more negative residuals.

Figure 2:

Kaplan-Meier plots of recurrence-free survival (RFS) and overall survival (OS) by “optimal” cut-point based on time of first infusion. Nominal p-values do not adjust for multiple comparisons with selection of cut-point; adjusted p-value adjust for multiple comparisons.

Table 2:

Hazard ratios and 95% confidence intervals from univariate Cox regression models with different threshold for “early” versus “late” treatment administration for recurrence-free survival (RFS) and overall survival (OS).

Version	Model	Hazard ratio (95% confidence interval)
Recurrence-free survival
30 minutes before optimal	After 13:48 (reference = 13:48 or earlier)	1.15 (0.91-1.47)
15 minutes before optimal	After 13:03 (reference = 13:03 or earlier)	Same as optimal
“Optimal”	After 13:18 (reference = 13:18 or earlier)	1.24 (0.98-1.57)
15 minutes after optimal	After 13:33 (reference = 13:33 or earlier)	1.20 (0.95-1.51)
30 minutes after optimal	After 13:48 (reference = 13:48 or earlier)	1.14 (0.89-1.44)
Overall survival
30 minutes before optimal	After 15:18 (reference = 15:18 or earlier)	0.98 (0.66-1.45)
15 minutes before optimal	After 15:33 (reference = 15:33 or earlier)	0.86 (0.55-1.35)
“Optimal”	After 15:48 (reference = 15:48 or earlier)	0.70 (0.42-1.18)
15 minutes after optimal	After 16:03 (reference = 16:03 or earlier)	0.70 (0.38-1.30)
30 minutes after optimal	After 16:18 (reference = 16:18 or earlier)	0.80 (0.41-1.56)

Prior analyses have evaluated percent of infusions before or after a threshold time. We first evaluated whether there were any changes in time of infusion over the course of the up to 18 infusions a participant could receive. In this cohort, average time of infusion moved earlier in the day over the course of the year; for each additional month of therapy, the average time of infusion was 5 minutes earlier (95% confidence interval 2-7 minutes, p<0.001). Similarly, the odds of an infusion before 11:00 increased by 32% over the course of the 12-month treatment period (odds ratio = 1.32, 95% confidence interval = 1.10-1.59, p=0.003).

Zip code data were available on 445 (71%) of the participants. Demographics for participants with and without zip code data are summarizes in Supplemental Table 1. A smaller proportion of participants missing zip code data were on age 65 or older (17%) compared to those with zip code data (25%, p=0.035). Among those missing zip code data, 79% were PD-L1 positive versus 84% with zip coded data (p=0.052). Among those missing zip code data, 49% reported non-ulcerated primary tumors compared to 40% of those with zip code data (p=0.081). There were no significant differences in sex, performance, stage, or BRAF mutation incidence between the two groups. Among those with zip code data, on average the farther a participant’s zip code was from that of the treating institution, the later in the day infusions were scheduled; for each additional 50 miles between participant zip code and treating institution zip code, the odds of an infusion before 11:00 decreased by 9% (odds ratio = 0.91, 95% confidence interval 0.85-0.98, p=0.017).

Multivariable Cox regression analyses accounting for time of infusion as a time-dependent covariate (time-of-day changing with each infusion a patient received), distance to treating institution, and other known prognostic factors did not identify a significant association with either RFS or OS (Table 3). Conclusions remained similar in multivariable models not controlling for distance to treating institution (data not shown).

Table 3:

Hazard ratios and 95% confidence intervals from multivariable Cox regression models with different threshold for “early” versus “late” treatment administration for recurrence-free survival (RFS) and overall survival (OS). Models adjust for age at randomization, sex, BMI, ECOG performance status, control arm preference pre-randomization, AJCC stage, ulceration, BRAF status, PD-L1, and distance from treating institution.

Model	Hazard ratio (95% confidence interval)
Recurrence-free survival
Percent of infusions before 11:00 (per 10% increase)	0.92 (0.83-1.0)
Percent of infusions before 12:00 (per 10% increase)	0.93 (0.87-1.0)
Percent of infusions before 13:00 (per 10% increase)	0.98 (0.92-1.05)
Percent of infusions before 14:00 (per 10% increase)	1.01 (0.94-1.09)
Percent of infusions before 15:00 (per 10% increase)	1.01 (0.92-1.09)
Percent of infusions before 16:00 (per 10% increase)	1.02 (0.90-1.17)
Overall survival
Percent of infusions before 11:00 (per 10% increase)	0.95 (0.87-1.04)
Percent of infusions before 12:00 (per 10% increase)	0.93 (0.86-1.0)
Percent of infusions before 13:00 (per 10% increase)	0.94 (0.88-1.0)
Percent of infusions before 14:00 (per 10% increase)	0.96 (0.90-1.02)
Percent of infusions before 15:00 (per 10% increase)	0.99 (0.91-1.07)
Percent of infusions before 16:00 (per 10% increase)	1.15 (0.95-1.39)

Abbreviations: AJCC, American Joint Commission on Cancer; BMI, body mass index; ECOG, Eastern Cooperative Oncology Group; PD-L1, programmed death receptor-1 ligand.

To evaluate potential seasonal patterns, we analyzed the month of first infusion and found no association with RFS, OS, or time to treatment-related grade 3 or higher adverse event (Supplemental Figure 3). Time-of-day of first infusion and month of randomization were not significantly associated with time to treatment-related grade 3 or higher adverse events (Supplemental Table 2, Supplemental Figures 4 and 5); conclusions remained similar in multivariable models not controlling for distance to treating institution (data not shown).

DISCUSSION

Studying temporal relationships in therapeutic interventions using retrospective data requires careful attention to data nuances. A first concern when reviewing the literature to-date is the lack of consistency in the “optimal” time for defining “early” versus “late” infusions. Rather than publications validating prior cut-points, most analyses report a single “best” cut-point in their observed data. Spurious conclusions can be drawn when not accounting for multiple comparisons.¹⁶ The multivariable analyses here account for the multiple nuances present in these data. Our results suggest that it is premature to suggest a benefit from a specific threshold of “early” versus “late” infusion times in clinical patient management.

When analyzing a quantitative covariate, such as time of infusion, it is important to evaluate the shape of the association between the covariate and outcomes.¹⁷ With time-to-event outcomes, such as RFS and OS, Martingale residuals plots provide one way to visualize the association. In this prospective, randomized trial cohort, the relatively flat Martingale residual plots (Figure 1), do not indicate that a binary “early” versus “late” categorization of time-of-day of infusion would result in two groups with clinically meaningfully different outcomes. A recent report of long-term results of the Checkmate-238 study in a comparable patient population as S1404 did not find a significant association between infusions given earlier or later than 13:00.¹⁸ A single-institution retrospective analysis of patients with melanoma who had received single-agent or combination anti-PD-1/PD-L1 therapies did not find a significant association between one or more infusions after 16:00 among the first four infusions and the endpoints of progression-free or overall survival.¹⁹

Regardless of whether splitting data into two groups may be meaningful, a maximum rank statistic algorithm can be applied to a dataset, and the algorithm will generate a “best” cut-point. Because such algorithms evaluate multiple potential cut-points, it is important to adjust for multiple comparisons when reporting p-values.²⁰ If prior cut-points have been reported in the literature, it is important to analyze those cut-points in addition to reporting the “optimum” in the analyzed dataset. When multiple cut-points have been evaluated, a figure should be provided that includes all analyzed cut-points and summarizes the pattern of the statistics over all cut-points (Supplemental Figure 1).

The issues with analyzing post-baseline data as baseline data have been well-described in the literature for many years.^12,21 Survivor bias is a particular concern with in this setting because in our cohort infusion times, on average, moved to earlier times of day the longer a patient was on therapy. Analysis of variables such as “more than 75% of infusions early in the day” as a baseline covariate cannot account for this potential data feature.

This trial cohort represents one disease setting (melanoma), one treatment setting (adjuvant therapy), and one trial setting (multi-center trial conducted across more than 200 institutions), and therefore results from these analyses cannot necessarily be directly extrapolated to other disease and treatment settings. Patterns observed in this cohort, such as time of infusion moving to earlier times of day, may not be observed in other settings or cohorts.

We note that time-of-day of infusion with respect to circadian rhythms is an imperfect surrogate of diurnal variation in hormone processes. Hours from sunrise can vary by latitude, longitude, and day of year, further confounding interpretation of time-of-day data.

An additional consideration is the pharmacokinetics and pharmacodynamics of immune checkpoint blockade antibodies. In particular, anti-PD-1 antibodies such as nivolumab and pembrolizumab, which are the basis of most published analyses, have long plasma half-lives of over three weeks,^22,23 and saturate the PD-1 receptor target for even longer periods. The specific day and time-of-day of the intravenous infusion of an anti-PD-1 antibody does not result in greater PD-1 blockade effects at that time. It has been previously shown that the anti-PD-1 antibody saturates the PD-1 receptor on T cells at a wide range of doses.²² Once the anti-PD-1 epitope is occupied in all the available PD-1 receptors by a prior infusion, administering more anti-PD-1 antibody with a next intravenous infusion would not allow “piling-up” of the antibody onto the PD-1 receptor to have a stronger pharmacodynamic effect (two or more antibodies cannot bind to the same epitope on PD-1). Given the saturation of PD-1 before and after the antibody infusion at the currently approved doses and schedules of nivolumab and pembrolizumab, there is limited mechanistic rationale for association between time-of-day and outcomes, and the speculation that the effect may be related to circadian rhythms is unlikely. To-date, studies have indicated that immunogenicity of the cancer is the strongest predictor of effectiveness of these therapies, as evidenced by the response rates near 90% in highly immunogenic settings including desmoplastic melanoma and MSI-high tumors.^24,25 Hence, it is more likely that prior reports associating infusion time-of-day with improved patient outcomes are influenced by other factors, such as the observation on our cohort that patients who respond to therapy and proceed with the complete year-long treatment plan are more likely to have their infusion times scheduled earlier in the day later in their therapy course, compared with patients who start on the treatment and have disease progression or recurrence before they are offered earlier times of infusion.

These findings do not support reorganizing infusion center schedules to prioritize morning administration of adjuvant anti-PD-1 therapy for patients with melanoma, which would have substantial operational implications. Given, the recent publication of randomized results of chemoimmunotherapy among patients with non-small cell lung cancer, results of other ongoing randomized studies (such as NCT07155317) will help to definitively answer this question.

Supplementary Material

Supplemental: 1 table, 4 figures

Data Access Statement:

All datasets and R code to reproduce analyses are available by request following SWOG procedures (Policy 43: Requests for Participant Data): https://www.swog.org/sites/default/files/docs/2019-12/Policy43_0.pdf