PlanningCopilot: An agentic framework integrating ESAPI modules for autonomous treatment planning in lung radiotherapy

Abstract

Background

Consistently generating clinically acceptable plans without human intervention remains a challenge in radiotherapy. Rule‐based automation provides deterministic execution, and knowledge‐based planning (KBP) provides statistical dose estimation, but both often require manual refinement. Large language models (LLMs) offer clinical reasoning capability, but effective autonomous planning also requires a mechanism to execute complex planning actions within the treatment planning system (TPS).

Purpose

To develop and evaluate PlanningCopilot, an agentic system that utilizes the reasoning capability of LLM and a validated Eclipse Scripting API (ESAPI) optimization module integrating KBP initialization (“PlanAct”) to autonomously generate treatment plans. This study evaluates the system's ability to produce clinically acceptable plans for locally advanced non‐small cell lung cancer (LA‐NSCLC) and assesses its potential to refine performance by self‐learning.

Methods

PlanningCopilot was implemented as a multi‐agent framework linked to the TPS through PlanAct API. It comprises four specialized GPT‐4.1 agents that iteratively interact with the TPS: (1) an Evaluator agent that accesses the plan and generates plan quality reports, (2) a Supervisor agent that validates these reports before passing them to a Planner agent, (3) the Planner agent that executes initialization and optimization tasks through PlanAct API and planning guidelines, and (4) an optional Learner agent that synthesizes optimization history into Planner‐facing prompt addendums. We retrospectively analyzed 62 patients with conventionally fractionated LA‐NSCLC and compared original clinical plans with autonomous plans with and without the Learner agent. Measurement‐based patient‐specific quality assurance (PSQA) was performed on the first 21 autonomous IMRT plans in planning order.

Results

All autonomous plans met clinical dosimetric requirements, including those not achieved in the clinical plans and KBP (RapidPlan) plans. Paired Wilcoxon signed‐rank tests showed no significant differences between autonomous and clinical plans for Lungs D_mean (p = 0.371), Lungs V_20Gy (p = 0.449), Lungs V_5Gy (p = 0.309), Heart D_50% (p = 0.175), Esophagus D_mean (p = 0.750), Spinal Cord D_0.03cc (p = 0.422), and Plan D_0.03cc (p = 0.941). Furthermore, autonomous plans achieved significantly lower Esophagus D_0.03cc (p = 0.027). Compared with RapidPlan initialization, PlanningCopilot improved multiple dosimetric endpoints, including Lungs D_mean (p < 0.001), Lungs V_20Gy (p < 0.001), Lungs V_5Gy (p = 0.004), Heart D_50% (p = 0.037), and Plan D_0.03cc (p < 0.001), with the cost of higher Esophagus D_mean (p < 0.001) and Esophagus D_0.03cc (p < 0.001). In a subset of 18 cases requiring at least two iterations, applying Learner‐derived knowledge reduced required iterations by an average of 11.8% while maintaining comparable plan quality (p > 0.05). All 21 autonomous IMRT plans passed measurement‐based PSQA.

Conclusion

PlanningCopilot enables autonomous generation of clinically acceptable and deliverable treatment plans for LA‐NSCLC. It consistently satisfies clinical dosimetric requirements across varying anatomical complexities and improves optimization efficiency through self‐learning from prior optimization history.

1. INTRODUCTION

Lung cancer remains the leading cause of cancer‐related mortality worldwide. ¹ Among its forms, locally advanced non‐small cell lung cancer (LA‐NSCLC) is particularly challenging for radiation oncologists to manage. The primary difficulty is critical organs‐at‐risk (OARs), such as the lungs, esophagus, heart, and spinal cord, ² , ³ are near planning target volume (PTV) where we need to deliver high, curative doses (often 60–66 Gy). ² , ³ In traditional treatment planning, dosimetrists rely on trial and error by manually adjusting dose objectives and priorities, following the guiding principle of “As Low As Reasonably Achievable”. ⁴ This manual workflow is labor‐intensive and the final plan quality often depends heavily on the planner's individual skill. ⁵ , ⁶ Automation was demonstrated to standardize plan quality and speed up the workflow, with a goal that every patient gets an optimized plan with less dependence on the planner. ⁶ , ⁷

Previous automation approaches have had progress on uncomplicated cases. ⁸ Early methods used rule‐based scripts based on the Eclipse Scripting API (ESAPI). These methods worked for simple, routine cases but lack the flexibility to handle complex anatomies that require patient‐specific adjustments. ⁸ , ⁹ Subsequent innovations in knowledge‐based planning (KBP) utilized statistical models to predict achievable dose‐volume histograms (DVHs) from prior plans. ¹⁰ , ¹¹ While KBP significantly narrowed the optimization search space, it often required manual “steering” to handle difficult tradeoffs. Meanwhile, the quality of the training data is key for KBP's DVH predictions. ¹² , ¹³ , ¹⁴ Generative AI and large language models (LLMs) offer a new path. ¹⁵ , ¹⁶ Unlike statistical models that simply forecast achievable DVHs, LLMs can reason, and understand clinical protocols, and use tools to interact with treatment planning systems (TPS). However, LLM agents often struggle to turn good clinical intentions into anticipated DVH changes. ¹⁷ , ¹⁸ , ¹⁹ , ²⁰ , ²¹ , ²² They might understand what needs to happen, but they lack the dexterity to make it happen within the TPS, leading to plans that do not quite meet clinical standards. ¹⁷ , ¹⁸ , ¹⁹ , ²⁰ , ²¹ , ²²

The missing link is how expert humans work. When an optimization fails to yield acceptable dosimetric outcomes, an expert planner does not rely solely on adjusting numerical parameters in TPS's optimizer; rather, they create new “auxiliary contours” to control dose from spilling into sensitive areas and precisely improve conformity. Existing automation tools have struggled to replicate this dynamic behavior, particularly in complex geometries. ¹⁹ , ²⁰ They either lack the “hands” (the execution capacity) to draw these structures (via scripts) or the “brain” (the reasoning capacity) to know where to draw them (via LLM).

In this study, we introduce PlanningCopilot, an LLM‐based agentic planning system, to bridge the gap. PlanningCopilot addresses these disparities by integrating the validated “PlanAct” ESAPI module (the execution engine) ²³ with LLM‐based multi‐agents (the reasoning engine). The workflow starts with using the KBP prediction to rapidly generate a preliminary plan (called by an LLM Planner agent). Then, the LLM agents analyze the dose, identify weak spots, and use PlanAct to iteratively optimize the plan. By giving it both the ability to think and the tools to act, we hypothesize that PlanningCopilot can consistently achieve clinically acceptable plan quality across varying anatomical complexity in LA‐NSCLC cases.

2. METHODS AND MATERIALS

2.1. PlanningCopilot system overview

The PlanningCopilot system was designed to autonomously navigate the treatment planning process by mimicking the iterative decision‐making process of a dosimetrist. As illustrated in Figure 1, after receiving contoured volumes and prescription dose, PlanningCopilot autonomously drives the TPS to generate step‐and‐shoot intensity‐modulated radiation therapy (IMRT) plans. The workflow initiates with a PlanAct initialization (called by the Planner agent), which utilizes a fine‐tuned RapidPlan KBP model to generate a preliminary plan, establishing a reasonable dosimetric baseline. Four GPT‐4.1‐based (OpenAI, San Francisco, CA) LLM agents work collaboratively to fine‐tune the preliminary plan: (1) an Evaluator agent checks dose metrics against institutional clinical constraints to generate a plan quality report; (2) a Supervisor agent validates the Evaluator's plan quality report to minimize numerical and logical errors; (3) the Planner agent determines the next action based on the plan evaluation results and the plan guidelines; and (4) an optional Learner agent synthesizes past actions and the resulting dosimetric metrics into new planning knowledge, which is added to the Planner's prompt as “prompt addendums”. The Evaluator and Supervisor work collaboratively on the plan evaluation but were separated to provide an internal quality‐control step. PlanAct converts TPS‐derived DVH metrics into a structured input and the agent Evaluator analyzes the input, identifies unmet requirements, and generates a plan quality report, including pass/fail status for each clinical objective and suggestions to improve the plan quality. The Supervisor acts as an independent validation gate. Only when the Supervisor approves, the plan‐quality report is passed to the Planner for reference to select a PlanAct action. This design is to reduce propagation of stochastic errors into subsequent optimization decisions and avoids relying on a single agent to self‐validate its own output.

FIGURE 1.

Overview of the PlanningCopilot system. PlanAct converts TPS‐derived DVH metrics into a structured input and the agent evaluator analyzes the input. The centered Planner agent iteratively determines the next action by the supervisor agent‐checked plan quality report and by pre‐set plan guidelines. The optional learner synthesizes the history plan quality reports into new planning knowledge for updating the Planner agent. The Planner agent acts through the PlanAct API, a library of pre‐compiled ESAPI modules such as ReduceDvolume for sparing organs‐at‐risk and ReducePTVDmax for the active creation of auxiliary structures to control isodose spillage and hotspots. The PlanningCopilot system autonomously generates step‐and‐shoot IMRT plans (in an Eclipse V18.0 environment) for comparison against clinical VMAT benchmarks. All LLM agents are based on OpenAI's GPT‐4.1 model.

The Learner agent synthesized the complete optimization history from representative cases. Its input included all plan quality reports generated by the Evaluator and validated by the Supervisor, as well as all Planner reasoning reports produced at each optimization iteration. These records captured both the dosimetric evolution of the plan and the reasoning process used to select each subsequent optimization action. The Learner then generated Planner‐facing prompt addendums that summarized generalizable planning lessons rather than case‐specific rules. These addendums described effective action patterns, cautionary tradeoffs, and inefficient strategies observed during prior optimization trajectories. The resulting addendum was appended to the Planner prompt in subsequent cases to support more efficient action selection while still allowing the Planner to reason from the current case‐specific plan quality report. Representative examples of the Learner‐generated addendums are provided in the Supplemental Materials. In this study, the Learner agent was trained on three (out of 62) cases of varying complexity: high (ten anticipated optimization steps), medium (five steps), and standard (two steps).

The execution engine of the proposed autonomous PlanningCopilot system is the PlanAct API, a modular library of ESAPI‐based scripts designed to execute planning strategies. ²³ The Planner agent calls specific PlanAct modules (referred as PlanAct Core in this study) to execute deterministic planning tasks. To meet the plan objectives, the system utilized ReduceROIMean and ReduceDvolume to reduce OAR doses and a suite of structure‐contouring modules, specifically ReduceBodyDmax and ReducePTVDmax, to create auxiliary avoidance structures for hotspot management and dose containment. Plan finalization was executed via OptimizePlan and NormalizePlan, which triggered the Eclipse optimization engine and normalized the dose distribution to the prescription baseline. The PlanAct scripting module was built on ESAPI version 18.0 (Varian Medical Systems, Palo Alto, CA) and compiled with the. NET framework version 4.7.2. PlanningCopilot performed all optimizations on a standard workstation running Windows 10 Enterprise 2016, equipped with dual‐socket Intel Xeon Silver 4110 CPUs (2.10 GHz) and 32 GB of RAM, and no GPU was used. Within the Eclipse 18.0 environment, plans were optimized using Photon Optimizer 18.0.1, with final dose calculations computed via the Anisotropic Analytical Algorithm (AAA) 18.0.1.

2.2. Study design

We retrieved 62 anonymized stage II or IIIA LA‐NSCLC cases previously treated between November 2022 and January 2024 at our institution. The prescribed dose was 60 Gy (200 cGy × 30 fractions). The PTV volumes of this cohort range from 41.30 cc to 876.82 cc, with a mean of 410.89 cc (standard deviation = 218.03 cc). To evaluate the efficacy of PlanningCopilot and the Learner agent, we compared the dosimetric performance across four distinct planning strategies: (1) clinical plans; (2) RapidPlan initialization plans (3) autonomous IMRT plans generated by an Evaluator‐Supervisor‐Planner LLM strategy without the Learner agent (referred as LLM plans); and (4) autonomous IMRT plans generated by the Evaluator‐Supervisor‐Planner‐Learner LLM strategy with the Learner agent (referred as LLM‐Learn plans). All autonomous IMRT plans were generated using nine static fields and normalized to ensure 95% PTV coverage at 100% of the prescription dose. This study was approved by an Institutional Review Board.

2.3. Plan evaluation

As shown in Table 1, the plan quality evaluation metrics included PTV coverage, maximum dose (defined as D_0.03cc), and OAR constraints for esophagus, lungs, heart, larynx, and spinal cord. DVH objectives served as the primary tool for assessing treatment plan compliance with these clinical goals. ²⁴ , ²⁵

TABLE 1.

Clinical evaluation criteria used in this study. Rx: Prescription dose.

Structure	Metric	Goal	Description
PTV	D95%	100% Rx	Relative dose covering 95% of volume
	D0.03cc	≤ 110% Rx	Relative dose to 0.03 cc volume
Plan (global)	D0.03cc	≤ 110% Rx	Global maximum dose (0.03 cc)
Lungs	V20Gy	≤ 37%	Volume receiving ≥ 20 Gy
	Dmean	≤ 2000 cGy	Mean dose to structure
Heart	D50%	≤ 3000 cGy	Dose covering 50% of volume
Spinal Cord	D0.03cc	≤ 4500 cGy	Dose to 0.03 cc volume
Esophagus	Dmean	≤ 3400 cGy	Mean dose to structure
	D0.03cc	≤ 6600 cGy	Dose to 0.03 cc volume
Larynx	D3cc	≤ 4000 cGy	Dose to 3 cc volume
	D0.03cc	≤ 4600 cGy	Dose to 0.03 cc volume

Statistical dosimetric comparisons were performed using Stata Version 17.0 Basic Edition (StataCorp LLC, TX, USA). The paired Wilcoxon signed‐rank test was utilized to assess the null hypothesis that the median difference between paired samples was zero, with a p‐value threshold of less than 0.05 considered statistically significant.

Measurement‐based patient‐specific quality assurance (PSQA) was performed on the first 21 autonomous IMRT plans in planning order, without further selection (total monitor units 822–2742 MU). Verification fields were delivered on a Varian TrueBeam linear accelerator and acquired via the electronic portal imaging device (EPID). Predicted and measured portal dose images were evaluated using Eclipse's Portal Dosimetry function with gamma analysis criteria of 3%/2 mm and a 10% low‐dose threshold. Following AAPM Task Group 218, ²⁶ a tolerance limit of 95% of points with γ < 1.0 was applied.

2.4. Use of AI

During the preparation of this work, the authors used ChatGPT (versions 4o, 4.1, and 5) to assist coding for the PlanningCopilot system. The authors also used ChatGPT (versions 4o, 4.1, and 5) and Gemini (version 3.1 Pro) to generate icons used for Figure 1 and to assist in the evaluation of initial manuscript drafts. Following the use of these tools, the authors reviewed and edited the content as needed and took full responsibility for the accuracy, logic, and integrity of the final manuscript. The AI‐generated code was independently tested and validated to ensure reproducibility of the results.

3. RESULTS

3.1. Evaluator‐supervisor‐planner LLM strategy

The standard autonomous PlanningCopilot Evaluator‐Supervisor‐Planner LLM strategy (without the agent Learner's involvement, referred as the LLM strategy) successfully resolved all constraint violations observed in 12 (out of 62) manual clinical VMAT plans and satisfied the clinical goals in all 62 cases (Figure 2). This 100% success rate highlights the system's robustness in handling the diverse anatomical challenges present in the cohort. Of the 62 plans optimized using the standard Evaluator‐Supervisor‐Planner LLM strategy, 41 (66%) RapidPlan initialization met all clinical objectives within a single iteration. An additional 15 plans reached goal satisfaction within 2 to 10 iterations. Extended optimization was required for the remaining cohort, with five plans succeeding between 11 and 19 iterations, and only one plan failed to meet clinical goals within 20 steps, requiring 22 iterations. This indicates that the observed improvement was not solely attributable to the KBP initialization but also to the subsequent agent‐guided PlanAct refinements.

FIGURE 2.

(a) Summary of plan numbers by objective achievement and unmet requirements (b) Distribution of unmet dosimetric requirements. LLM: the standard evaluator‐supervisor‐planner LLM strategy. RapidPlan: plan initialized by PlanAct (the Planner agent called the initialization action) before optimization via PlanningCopilot.

The variation in iteration number reflected differences in patient anatomy and in the quality of the RapidPlan initialization. Plans close to the institutional objectives after initialization usually require only one refinement step. In contrast, cases with larger targets, proximity to serial organs, competing lung and hotspot constraints, or unfavorable initial dose spill required additional sequential actions. Since PlanningCopilot re‐evaluates the plan after each action and uses incremental corrections to avoid overcorrection, complex geometries naturally required more iterations before all objectives were simultaneously satisfied.

With plan normalization values to ensure 95% PTV coverage at 60 Gy (100% of the prescription dose), dosimetric comparison (Figure 3) for the full cohort (n = 62) using the standard Evaluator‐Supervisor‐Planner strategy reveals statistical equivalence (p > 0.05) of the autonomous IMRT plans to clinical VMAT plans across most metrics, including Lungs D_mean (p = 0.371), Lungs V_20Gy (p = 0.449) and Lungs V_5Gy (p = 0.309), Heart D_50% (p = 0.175), Esophagus D_mean (p = 0.750), Spinal Cord D_0.03cc (p = 0.422), and global plan D_0.03cc (p = 0.941). Furthermore, the autonomous IMRT plans showed significantly superior sparing of the Esophagus D_0.03cc by 1.14 Gy (p = 0.027) under the tested standardized tradeoff setting. Statistical evaluations of Larynx D_3cc and D_0.03cc are not available, as only three cases have a contoured Larynx structure. PlanningCopilot IMRT plans required 115% higher (p < 0.001) MUs than clinical VMAT plans.

FIGURE 3.

Dosimetric comparison for the full cohort (n = 62), including (a) lungs D_mean, (b) lungs V_20Gy, (c) lungs V_5Gy, (d) heart D_50%, (e) esophagus D_mean, (f) esophagus D_0.03cc, (g) spinal cord D_0.03cc, and (h) Plan (global) D_0.03cc. The horizontal dashed lines indicate the DVH objectives in each metric. Lungs V_5Gy does not have an institutional mandatory objective. p‐values in subfigures represent results of the paired Wilcoxon signed‐rank test. LLM: the standard evaluator‐supervisor‐planner LLM strategy. RapidPlan: plan initialized by PlanAct (the Planner agent called the initialization action) before optimization via PlanningCopilot.

Compared with RapidPlan initialization, PlanningCopilot improved multiple dosimetric endpoints, including Lungs D_mean (p < 0.001), Lungs V_20Gy (p < 0.001), Lungs V_5Gy (p = 0.004), Heart D_50% (p = 0.037), and Plan D_0.03cc (p < 0.001), with the cost of higher Esophagus D_mean (p < 0.001) and Esophagus D_0.03cc (p < 0.001), as shown in Figure 3, and 11.4% higher (p < 0.001) total monitor units (MUs), as shown in Figure S1.

All 21 autonomous IMRT plans passed measurement‐based PSQA. The mean gamma passing rate was 99.0% ± 1.1% (range 95.7–99.9%), and every plan exceeded the 95% tolerance limit (Table S1).

3.2. Evaluator‐supervisor‐planner‐learner LLM strategy

In a subset of 21 complex cases requiring at least two iterations, the Learner summarized knowledge from three training cases and generated addendum prompts for the Planner. With the updated prompts, the same Evaluator‐Supervisor‐Planner strategy (referred as the LLM‐Learn strategy) spent on‐average 11.8% fewer iterations in the remaining 18 cases and eliminated the need for > 20 iterations observed in the initial planning (Figure 4). This demonstrates that the knowledge summarized by the Learner effectively streamlines the optimization process in challenging cases.

FIGURE 4.

Improvement on numbers of iterations with the inclusion of the Learner agent. (Plans #8–18 are most challenging: more than 5 iterations in the standard LLM strategy; plans #1–7 are moderately challenging: 2–5 iterations in the standard evaluator‐supervisor‐planner LLM strategy).

Analyses (Figure 5) of the complex subset (n = 18) includes cases requiring at least two iterations where the Learner agent could actively intervene. They demonstrated that in this subset (n = 18), the Learner's efficiency gains did not compromise plan quality because no statistically significant dosimetric difference or MU difference (both p > 0.05) was found across all the evaluation metrics in the full Evaluator‐Supervisor‐Planner‐Learner LLM strategy compared to the standard Evaluator‐Supervisor‐Planner LLM strategy. Meanwhile, using the LLM‐Learn strategy, the autonomous IMRT plans were statistically equivalent (p > 0.05) to the clinical plans across all the evaluation metrics, with details shown in Figure S2. Compared with RapidPlan initialization, PlanningCopilot increased the total MUs (Figure S3) but improved dosimetric endpoints including Lungs D_mean (p = 0.002), Lungs V_20Gy (p = 0.007), Lungs V_5Gy (p = 0.022), and Plan D_0.03cc (p < 0.001), with the cost of higher Heart D_50% (p = 0.010), as shown in Figure 5.

FIGURE 5.

Dosimetric comparison for the complex subset (n = 18), including (a) lungs D_mean, (b) lungs V_20Gy, (c) lungs V_5Gy, (d) heart D_50%, (e) esophagus D_mean, (f) esophagus D_0.03cc, (g) spinal cord D_0.03cc, and (h) Plan (global) D_0.03cc. The horizontal dashed lines indicate the DVH objectives in each metric. Lungs V_5Gy does not have an institutional mandatory objective. p‐values in subfigures represent results of the paired Wilcoxon signed‐rank test. RapidPlan: plan initialized by PlanAct (the Planner agent called the initialization action) before optimization via PlanningCopilot. LLM: the standard evaluator‐supervisor‐planner LLM strategy. LLM‐learn: the full LLM strategy with the Learner's inclusion.

4. DISCUSSIONS

Consistently automating clinically acceptable plans without human intervention is a challenge in radiotherapy. This study proposed and evaluated PlanningCopilot that integrates the ESAPI‐based execution module PlanAct with LLM to automate treatment planning for LA‐NSCLC. When applied to 62 retrospective cases, the PlanningCopilot‐driven IMRT plans satisfied all institutional dose constraints and were statistically equivalent to clinical benchmarks. In contrast, some manual clinical VMAT plans exhibited minor deviations from ideal dose objectives, whereas the PlanningCopilot achieved a 100% compliance rate. These findings support the concept of combining semantic reasoning with deterministic execution, allowing the system to consistently generate clinically acceptable plans across diverse anatomical presentations without human intervention in satisfying the predefined institutional dosimetric criteria. However, these differences should be interpreted as protocol‐level compliance rather than direct evidence of overall plan‐quality superiority, because the original clinical plans were physician‐approved based on individualized anatomical tradeoffs, clinical judgment, and delivery considerations. Formal blinded physician scoring was not performed in this feasibility study.

PlanningCopilot was developed to address a key gap in current automated and autonomous planning approaches. Rule‐based scripting methods, including ESAPI‐based automation, provide deterministic and reproducible execution but are generally limited by predefined logic and may lack flexibility for patient‐specific anatomy. KBP methods, such as RapidPlan, provide statistical DVH estimations and can reduce planning time, but KBP‐generated plans often require additional steering and refinement to satisfy clinical goals. Recent feasibility studies ¹⁸ , ²⁰ and autonomous planning workflow studies on LLM‐based planning, including Zero‐Shot LLM agents, ¹⁷ GPT‐Plan, ¹⁸ DOLA, ¹⁹ GPT‐RadPlan, ²¹ have demonstrated the potential of LLMs for radiotherapy planning; however, current LLM‐based planning workflow studies remain focused primarily on parameter selection or objective tuning within the TPS. Comprehensive treatment planning extends beyond parameter tuning, because clinically acceptable plans often depend on the creation of specialized optimization structures to guide dose distribution during iterative refinement. In PlanningCopilot, LLM agents works with TPS to execute human‐like planning strategies. By decomposing the planning process into sequential steps of evaluation, reasoning, and execution, the system can identify specific deficits such as a local hotspot and selectively deploy a targeted plan action to address it. This iterative loop allows the system to flexibly adapt to anatomical variability that statistical models cannot anticipate effectively.

PlanningCopilot differs from prior autonomous planning studies by combining clinical reasoning, prior dosimetric knowledge, and executable treatment planning actions within one framework. RapidPlan provides an initial KBP‐based dosimetric baseline, while the LLM Planner selects deterministic actions through PlanAct. PlanAct is a key component of PlanningCopilot, as it converts high‐level planning decisions into executable treatment‐planning actions. Since PlanAct is implemented through ESAPI, PlanningCopilot operates within a commercial TPS‐compatible environment rather than relying only on offline dose prediction or external simulation. This design helps bridge reasoning and execution in autonomous treatment planning.

Potentially, improvements in KBP model quality could directly benefit PlanningCopilot. A better KBP initialization provides a stronger starting plan and may reduce the amount of downstream iterative refinement required. Recent multi‐criteria optimization (MCO) studies are relevant in this context, as MCO‐generated plans and tradeoff‐aware KBP models can improve the quality of RapidPlan training data and provide more informative patient‐specific tradeoff estimates. ¹³ , ²⁷ , ²⁸ These advances could further strengthen PlanningCopilot by improving its initialization step and by providing prior knowledge for Planner guidance or Learner‐derived rules. ²⁹

The proposed method addresses two major barriers preventing clinical adoptions of automated planning models. First, the need for manual tuning of KBP plans. Although RapidPlan and similar KBP models are usually fine‐tuned to adopt institution‐specific planning goals, it is common that the KBP‐generated plans are not fully clinically acceptable and need further fine‐tune. PlanningCopilot utilizes PlanAct to adapt the KBP initialization to institution‐specific requirements, reducing workload on manual plan tuning. Second, direct LLM‐to‐TPS interaction could be hampered by stochastic behaviors, such as irrational parameter value changes and contradictory optimization goals. ¹⁸ PlanningCopilot mitigates these risks through a constrained actuation layer where the LLM selects the tasks from a list of planning actions. This design balances the potential gain of creativity of LLMs and the potential loss due to a lack of deterministic strategy. The multi‐agent structure also provides an internal quality‐control mechanism before TPS execution. In PlanningCopilot, the Evaluator generates the plan‐quality report from TPS‐derived DVH metrics, whereas the Supervisor independently verifies the numerical interpretation, pass/fail logic, and consistency with institutional constraints before the report is passed to the Planner. This separation follows the general principle of multi‐agent LLM design, in which complex workflows are decomposed into specialized roles with structured interactions rather than handled by a single consolidated prompt. ³⁰ This design is also supported by prior studies showing that multi‐agent LLM strategies can enhance complex medical reasoning and improve decision making compared with single‐agent prompting. ³⁰ , ³¹

GPT‐4.1 was used in this implementation as it was one of the most practical models available for our implementation at the time of experiments and provided stable tool‐calling behavior and sufficient context capacity for structured multi‐agent prompts. The PlanningCopilot framework is model‐agnostic in principle, and the selection of GPT‐4.1 should not be interpreted as excluding other LLMs. A formal comparison of different LLM backbones remains an important direction for future work.

To reduce the total optimization time, PlanningCopilot only optimized step‐and‐shoot IMRT plans. Although VMAT and step‐and‐shoot IMRT differ in their delivery mechanics, delivery efficiency, and MUs, previous studies have demonstrated their dosimetric similarity in LA‐NSCLC. ³² , ³³ This established comparability validates our methodology of evaluating autonomous IMRT plans against the clinical plans. Notably, the PlanningCopilot system excelled at sparing the esophagus, dose to which commonly restricts the therapeutic window for centrally located tumors. These results suggest that the LLM agent's capacity for autonomous iterative refinement to uncover sparing opportunities that might be unachievable in a time‐pressured manual workflow. By consistently minimizing high‐dose hotspots and resolving complex constraint conflicts, the system demonstrates strong potential for clinical utility, particularly in high‐throughput environments or clinics aiming to standardize the quality of their treatment plans.

Integrating the “Learner” agent enables the agent Planner to improve without undergoing human‐driven prompt engineering. The Learner agent provides an additional mechanism for performance refinement. Although the standard Evaluator‐Supervisor‐Planner LLM strategy met all the DVH objectives across all the 62 cases included in this study, not every action driven by the agent Planner was effective. By synthesizing successful optimization strategies, the Learner agent synthesizes previous optimization strategies with their corresponding DVH results from previous cases into targeted “prompt addendums,” equipping the system with a dynamic memory bank to accumulate experience in both effective and less effective autonomous strategies. In this study, it was most useful for complex cases requiring multiple optimization iterations, where recurring tradeoff patterns could be identified from prior planning histories and converted into prompt addendums for subsequent action selection. This dynamic memory prevents the system from repeating past patterns of inefficient iterations; however, it is worth noting that the Learner can occasionally increase the number of iterations for specific cases. This typically occurs when a generalized strategy derived from training cases does not map to a case‐specific geometry. In such instances, the historical prompt addendum might instruct the Planner to attempt a highly aggressive dosimetric trade‐off (e.g., aggressively sparing an OAR), temporarily compromising PTV coverage. This necessitates additional compensatory iterations to rebalance the plan. Therefore, Learner agent is optional and should be interpreted as an exploratory performance‐refinement component rather than a requirement for plan acceptability. PlanningCopilot can generate clinically acceptable plans without the Learner, as demonstrated by the standard Evaluator‐Supervisor‐Planner workflow. Future work will further investigate how the Learner can improve PlanningCopilot from additional perspectives, including action efficiency, case‐specific tradeoff selection, prompt refinement, and progression from clinically acceptable planning toward consistently higher‐quality plan generation.

The present study has limitations. This study was retrospective and confined to a single institution's protocols and contouring styles. The cross‐institutional generalizability of PlanningCopilot is pending validation. In future work, we plan to use public dataset together with physician review to assess clinical acceptability in addition to dosimetric performance in more general scenarios. While the autonomous step‐and‐shoot IMRT plans matched VMAT quality, the autonomous step‐and‐shoot IMRT plans require several times more MUs than the clinical VMAT plans. The clinical standard often prefers VMAT for delivery efficiency, and future work must adapt the PlanningCopilot strategy for arc optimization. Additionally, the dosimetric comparisons among the autonomous IMRT plans and clinical VMAT plans were by statistics. Despite the full compliance with the clinical goals, the PlanningCopilot paused optimization at the “acceptable”, leaving the potential of achieving a “good” plan. Though demanding more resources, evaluations from expert oncologists would help to improve the solidity of the comparisons and to guide further optimization for higher plan quality.

Although the present implementation generated nine‐field step‐and‐shoot IMRT plans, the PlanningCopilot architecture is not inherently limited to static‐field planning. VMAT optimization was not included in the current feasibility study because the iterative PlanningCopilot workflow requires increased computation time on our current hardware. Static‐field IMRT was therefore used as a practical implementation for initial feasibility testing. However, the same workflow design can be extended to VMAT because the PlanAct action library includes arc initialization and VMAT‐specific initialization and optimization modules. In this feasibility study, the autonomous IMRT plans generally required higher MUs than both the clinical VMAT plans and RapidPlan initialization, suggesting increased beam modulation in the current implementation. This increase likely reflects the additional modulation introduced by sequential PlanAct refinements to satisfy all predefined dosimetric criteria. The Learner agent did not significantly increase MUs relative to the standard PlanningCopilot workflow, suggesting that its efficiency benefit was primarily related to reducing planning iterations with minimal impacts on the complexity of beam modulation. PSQA measurements confirmed that these plans are deliverable: all 21 measured plans passed, with a mean gamma passing rate of 99.0%, including a plan of 2742 MU. These findings indicate that delivery efficiency, rather than deliverability, remains the practical limitation of the current implementation. Future VMAT‐focused studies with upgraded computing hardware will include formal comparisons (incorporating dosimetric and physician's review) of dosimetric quality, monitor units, delivery time, optimization time, and plan complexity.

5. CONCLUSION

The present study demonstrates the clinical feasibility of PlanningCopilot, a multi‐agent large language model framework integrated with deterministic ESAPI scripting, for the autonomous treatment planning of LA‐NSCLC. Across a retrospective cohort of 62 cases, the system consistently met institutional dosimetric requirements. The autonomous plans demonstrated statistical equivalence to clinical baselines for most critical OARs, while achieving significantly superior sparing of the esophagus D_0.03cc (p = 0.027). Furthermore, the integration of a Learner agent enabled the system to synthesize past optimization history into actionable knowledge, thereby reducing planning iterations by an average of 11.8% in complex geometries without compromising plan quality. PlanningCopilot successfully bridges high‐level semantic reasoning and deterministic treatment planning system actuation, offering a reliable, adaptive, and efficient solution for autonomous lung radiotherapy planning.

CONFLICT OF INTEREST STATEMENT

The authors declare no conflicts of interest.

ETHICS STATEMENT

This study was approved by an Institutional Review Board of the Mount Sinai School of Medicine (STUDY‐23‐01493‐MODCR001).

Supporting information

Supporting Information