Skip to main content
NCBI home page

This is a preprint.

It has not yet been peer reviewed by a journal.

The National Library of Medicine is running a pilot to include preprints that result from research funded by NIH in PMC and PubMed.

medRxiv logoLink to medRxiv
[Preprint]. 2026 Jun 18:2026.06.16.26355782. [Version 1] doi: 10.64898/2026.06.16.26355782

Hard to Halt: Automation Bias in Agent-Driven Sequencing Prior Authorization Workflows

Mengshu Nie 1, Wendy Chung 1,8, Jessica Waxler 1, Michael Lee 2, Chunhua Weng 3, Rachel Lewis 4, Priyanka Ahimaz 4, Kai Wang 5,6, Cong Liu 1,7,8,*
PMCID: PMC13308439  PMID: 42369474

Abstract

Purpose:

Prior authorization (PA) for exome or genome sequencing is a time-consuming process that impedes timely rare disease diagnosis. Large language model-based browser agents offer potential for automating these workflows, but their clinical reliability remain uncharacterized.

Methods:

We developed a sandbox compromising a simulated ES/GS PA submission payer portal and a synthetic EHR containing 836 patient records spanning compliant profiles and deficient profiles with different types of issues. Gemini 3 Pro, Gemini 3 Flash, and Claude Opus 4.5 were evaluated on task completion rate, form completion accuracy, and appropriate withholding for deficient profiles.

Results:

Larger models achieved much higher task completion rates (Gemini 3 Pro 95.45%, Claude Opus 4.5 93.67%) compared to Gemini 3 Flash (56.05%), but nearly universally failed to withhold submission for deficient profiles whereas Gemini 3 Flash ironically demonstrated superior withholding performance (17.33%). In a non-agentic setting, Gemini 3 Pro correctly identified 91% of the issues in deficient profiles, indicating that withholding failure is attributable to the browser interaction rather than the model’s reasoning limitations.

Conclusion:

Current LLM-based browser agents exhibit a systematic bias towards form submission that poses risks in PA workflows. A modular, multi-agent architecture with human supervision is necessary for a safe clinical deployment.

Background

Prior authorization (PA) is an insurer-initiated process to verify medical necessity and coverage eligibility before approving prescribed procedures, services, or medications requested by healthcare providers.1 It has contributed significantly to healthcare administrative costs in the United States, estimated to account for $35 billion annually.2 These costs are driven in part by the high frequency of administrative errors in PA submissions, including missing documentation, coding mistakes, and eligibility mismatches which necessitate resubmission, appeal, and manual review.3 Across ACA Marketplace plans, such administrative reasons accounted for 25% of all in-network claim denials in 2024, substantially exceeding denials based on medical necessity (5%).4 These failures are often traceable to inaccuracies propagated from the EHR, where incomplete or incorrect data are incorporated into PA submissions without adequate verification.5 Beyond the monetary costs, the PA process is also time consuming. An AMA survey in 2024 showed that each physician and their staff spend an average of 13 hours on PA forms per week.6 This burden is particularly acute for clinical exome and genome sequencing (ES/GS) which have transformed rare disease diagnosis by increasing the yields and reducing cost per diagnosis.7,8 Obtaining PA approval for these genetic tests, especially in outpatient contexts, is a significant bottleneck: a study shows that approximately 20% of a total of 4535 PA requests for genetic testing across two children’s hospitals in Texas were denied, with ES carrying the second highest denial rate among all genetic tests.9 The follow-up appeal process after an initial denial can introduce weeks of additional delay, which further impedes timely diagnosis and treatment for patients.10

Currently, most PA submissions still rely on manual web portals or fax submission which contribute to transcription errors and inefficiency. Large language model (LLM)-based agents offer a potential solution to these administrative challenges, which could assist the genetic counselors, physicians and medical assistants who currently bear the operational burden of PA submission in genetics practices. Unlike static conversational systems, agentic LLMs execute multi-step tasks by iteratively reasoning, planning, and acting through external tools. The ReAct framework formalizes this approach by coupling stepwise reasoning with discrete actions, enabling interaction with APIs, databases, and web interfaces.11 Browser-based agents operationalize this paradigm for web workflows by navigating pages, parsing web representations and selectively incorporating visual context via vision-language models when needed.12 For PA submission, this enables end-to-end automation: logging into payer portals, navigating form fields, filling in structured EHR data (demographics, codes, insurance details), and submitting requests with confirmation capture. This closed-loop interaction aligns naturally with the structured, form-based design of PA systems, making ES/GS PA suitable target for agentic automation.

Before any agentic system can be responsibly deployed in a live clinical environment, its behavior must be rigorously examined under controlled conditions. Unlike conversational AI tools which can be constrained to text generation and are bounded by the turn of a conversation, agentic systems can execute consequential, irreversible actions, such as modifying EHR-linked records, or inadvertently disclosing protected health information to unauthorized parties. This loss of deterministic guardrails necessitates a sandbox-first evaluation strategy. Clinical sandboxes provide isolated, functionally equivalent replicas of production systems in which an agen’s reasoning chains, decision-making, action sequences can be assessed without risk of real-world harm.13–15 Sandbox testing not only reveals failure modes such as navigational dead ends, multi-step error propagation or payer portal-specific idiosyncrasies,16,17 but also provides an empirical upper bound on achievable performance under idealized conditions, informing the design of safeguards and human-in-the-loop oversight mechanisms required for eventual clinical deployment.18,19 In this study, we aim to investigate the behavior of an LLM-based browser agent in a controlled sandbox environment simulating the ES/GS PA workflow, characterize its task completion performance, identify systematic failure patterns, and establish a quantitative foundation to guide future clinical implementation.

Materials and Methods

Sandbox Environment

We developed a sandbox environment comprising two primary components (Figure 1(a)). First, a web portal with PA webform was constructed to emulate real-world payer forms which requires structured inputs including patient demographics, insurance subscriber information, provider and laboratory details, and relevant clinical data. We then implemented a searchable patient database that simulates an EHR environment from the provider’s side, in which each patient profile is represented as an information card with text-based materials. Agents in this study access both components and follow the instructions to complete PA-related tasks. The sandbox is publicly available at https://wes-wgs-pa-app-u2c8s.ondigitalocean.app/login.

Figure 1. Sandbox & Study Overview.

Figure 1.

Open in a new tab

(a) The sandbox interface is consisted of a user login page, a PA form, and a patient search function that directs to the synthetic EHR database. The payer portal requires authentication prior to form access. The EHR interface supports patient retrieval via name-based search, and each patient record contains structured data fields (demographics, subscriber, provider, laboratory, and test details) alongside a free-text clinical summary.

(b) A synthetic EHR database was constructed comprising four profile categories: (1) compliant profiles with error-free information and genetically relevant clinical description; (2) profiles containing date-related errors; (3) profiles with completely genetically irrelevant clinical information; and (4) profiles with duplicate patient names. Profiles of categories 2, 3, and 4 are generalized as deficient profiles since submission for these profiles are expected to be halted. All profiles were generated from predefined clinical condition templates and ICD codes, and subsequently augmented with LLM-generated free-text clinical narratives conditioned on the structured data.

(c) A browser-based agent was initialized with a standardized prompt to retrieve patient records, review clinical information, complete a PA form, and either submit the form or halt execution if issues were identified. A unique task ID was automatically assigned at the time of task creation.

d) Task outcomes for all patient profiles are evaluated against ground-truth data. For compliant profiles, submitted forms are assessed for form-level and field-level accuracy; halted submissions are flagged as undesigned withholding. For deficient profiles, submission is considered undesired, while halted submissions can be either correct withholding or undesigned withholding depending on whether the identified issue matches the ground-truth injected error. Tasks that terminate without producing either a submission or withholding decision, such as due to step limit exhaustion, data loss, or other technical failures, are classified as aborted.

Synthetic Patient Data

We constructed a synthetic patient cohort via a multi-stage data generation pipeline as visualized in Figure 1(b). To emulate the variability observed in real-world clinical data and evaluate the agent’s performance under different data context, we partitioned the patient profiles into four high-level groups: (1) compliant profiles, which are error-free records with genetically relevant conditions, or mixed with unrelated secondary medical histories; (2) error-injected profiles, in which we introduced perturbations reflecting common administrative errors; (3) irrelevant profiles, where the clinical descriptions are entirely genetically unrelated (e.g., physical injuries); and (4) homonymous profiles, where two patients share an identical name. Each group was further subdivided into categories corresponding to specific error modes or irrelevance patterns, as summarized in Table 1. Profiles of categories 2, 3, and 4 were generalized as deficient profiles since submission should be halted due to the injected issues. Note that the homonymous profile category itself is issue or error free; it is designed as deficient because ambiguity is intentionally created in the following stage where we only provide patient name to the agent for patient search. The synthetic data generation pipeline was explained in Supplementary Material Patient Data Generation.

Agent Setup and Task Execution

PA submissions were automated using Browser Use Cloud (Figure 1(c)), an open-source agentic framework that enables LLMs to orchestrate browser-based workflows such as webpage navigation and information retrieval. Three foundational LLM models were evaluated: Gemini 3 Pro (preview), Gemini 3 Flash and Claude Opus 4.5. For each patient profile, the agent was initialized with a base prompt (Supplementary Material Prompt 2) instructing it to: (1) navigate to the PA portal and authenticate using provided credentials; (2) retrieve the patient record via the search function using the patient’s full name; and (3) complete and submit the PA form, or halt submission and report issues if any discrepancies were identified. No explicit guidance was provided regarding potential error types or fields requiring scrutiny. A unique task ID was automatically assigned at the time of task creation, and each task was executed in an independent session within the controlled sandbox environment per patient profile and per model, with no shared state or memory across runs. A maximum of 40 browser interaction steps was enforced per task, and the thinking mode was enabled to produce explicit intermediate reasoning traces, including step-level evaluation, short-term memory, and next-action planning.

Evaluation

Figure 1(d) illustrates the evaluation framework used to assess agent performance across profile types. We first classify the agents’ task execution according to the following four outcome categories:

  1. Aborted: the agent failed to produce a form submission or a withholding reason due to technical issues, such as reaching the maximum number of allowed browser interaction steps (40 steps) or losing form data following a browser refresh.

  2. Submitted: the agent completed and submitted the PA form within the maximum 40 steps. This constitutes a correct completion outcome for compliant profiles and an erroneous completion outcome for deficient profiles.

  3. Correct withholding: the agent correctly identified the intentionally designed issue in a deficient patient profile, halted the submission process, and reported the identified issue. This outcome represents the desired completion outcome for deficient profiles.

  4. Undesigned withholding: the agent halted submission for reasons not supported by ground-truth criteria which may occur in both compliant and deficient profiles. The cited issues may constitute invalid bases for termination and reflect failures in reasoning, for example, incorrectly asserting that required information is missing when it is present, or failing to infer ICD codes or clinical rationale that are reasonably derivable from the patient profile. In some cases, these errors may also arise from artifacts of the synthetic dataset (e.g., postal codes or provider NPI numbers that do not conform to real-world formats).

Task-level metadata were subsequently retrieved via the Browser Use API, and submitted cases were identified by cross-referencing the full set of task IDs against the sandbox stored submission files. For tasks without a recorded submission, GPT-5.2 was then applied to the agent’s final output messages to categorize outcomes into aborted, correct withholding, or undesigned withholding (Supplementary Material Prompt 3). The agent may report both intentionally designed and other unanticipated issues in its output, in which case both Outcome 3 and 4 will be applied.

We then evaluated the agents’ ability to accurately complete and submit the PA forms from compliant patient profiles by comparing each agent-filled value against its corresponding ground-truth entry. Form fields were separated into two types: deterministic fields whose values can be directly extracted (i.e., via straightforward retrieval) from the patient record, including patient demographics, provider information, laboratory details, and test specifications; and interpretative fields which include the clinical rationale for testing (e.g., checkbox selections) and prior testing history (e.g., test type, results, and dates), requiring clinical reasoning, synthesis, or inference from narrative clinical text. To quantify statistical uncertainty in task-level outcomes, form, and field-level accuracy, we employed nonparametric bootstrap resampling (1000 iterations) to obtain empirical distributions, from which median point estimates and 95% confidence intervals were then obtained.

Results

Task Execution Outcomes

As shown in Figure 2(a), larger models exhibited substantially fewer technical failures, resulting in much higher overall task completion rates (95.5% for Gemini 3 Pro and 93.7% for Claude Opus 4.5) compared to only 56.1% for Gemini Flash. Among compliant profiles, Gemini 3 Pro achieved the highest submission rate (94.8% of all tasks). In contrast, Claude Opus 4.5 demonstrated frequent refusals driven by undesigned issues, which accounted for nearly 79.8% of failed submissions while the remainder were aborted (20.3%). For deficient profiles, all three models largely failed to identify injected issues and appropriately halt execution. Notably, the smallest model, Gemini Flash, exhibited the highest proportion of correct withholding decisions (17.3% of completed tasks). Conversely, Gemini 3 Pro demonstrated a near-zero halting rate on deficient profiles. Overall, these results indicate a trade-off: while larger models are less prone to technical failures and more likely to complete tasks successfully, they underperform relative to smaller models in detecting erroneous or clinically inappropriate cases and halting submission as instructed.

Figure 2. Main Experimental Results.

Figure 2.

Open in a new tab

(a) Proportions of task outcomes for three different models across compliant and deficient profiles, with error bars indicating 95% bootstrap confidence intervals.

(b) Proportions of correctly completed fields per submission among successfully submitted compliant profiles.

(c) Field-level accuracy across all required form fields with 95% bootstrap confidence intervals, reflecting the frequency where each field is correctly completed across runs.

To assess whether undesigned issues reflected limitations in dataset generation or the agent’s failure of reasoning, we conducted a manual review of all such cases for Claude Opus 4.5 (244 of 706 tasks). None of the undesigned issues corresponded to valid omissions; instead, 93.4% of them are attributed to hallucination (e.g., incorrectly claiming that subscriber date of birth, provider contact information, or address fields are missing); 6.6% results from inference failures (e.g., asserting that ICD-10 codes were absent when they were expected to be inferred from the clinical context).

Submitted Form Accuracy

Figure 2(b) shows the bootstrapped distribution of the percentage of correctly entered fields on submitted forms among compliant profiles across three LLMs, all of which have achieved a form-level accuracy over 90%. Gemini 3 Pro achieves the highest mean accuracy (94.4%) followed closely by Claude Opus 4.5 (93.72%). Gemini Flash exhibits lower mean accuracy (91.12%) and greater dispersion, indicating less consistent performance. Figure 2(c) presents the agent’s filling accuracies of each mandatory field on the form. All models achieve near-ceiling performance on some of the deterministic fields (e.g., patient name, sex, date of birth, and member ID etc.) with no statistically significant differences observed. Several other fields, however, exhibit much lower accuracy with greater variability across LLMs. One interesting example is the error pattern of the subscription relation field: larger models occasionally over-infer the subscriber-patient relationship (e.g., inferring a parental relationship from matching last names and age differences when the patient record indicates other or guardian), whereas the smaller model, Gemini Flash, sometimes leaves the subscriber-related fields blank even when the information is explicitly provided. Sporadic hallucinations are also observed on all three models (7.6% for Claude Opus 4.5, 5.7% for Gemini 3 Pro, and 4.2% for Gemini Flash), with address-related fields populated with entirely fabricated values which reduces the overall accuracy of those fields. For clinical fields, performance is markedly more variable and less accurate across both models and individual fields, ranging from approximately 60% to 100%. Fields tied to rationale-for-testing checkboxes (i.e., multiple congenital anomalies, autism, metabolic, and neurological indicators etc.) generally demonstrate relatively strong performance (80% to 100%). In contrast, fields related to prior testing history (e.g., prior test date, result, and type) which require extraction from unstructured clinical text exhibit the lowest accuracy (50% to 80%). These results indicate that while agents can reliably perform direct information extraction, hallucination errors persist, and performance degrades when tasks require clinical reasoning and interpretation of narrative data.

Ablation Studies

To further investigate the sources of agent performance limitations observed in the main experiment results described in the previous two sections, we conducted three ablation studies examining the role of the agentic setting itself in task execution, the step budget constraint, and prompt specificity. Table 2 summarizes the objective and samples of each ablation studies for reference.

Ablation 1: increasing step limit

In the main experiment, we found that Gemini Flash exhibits the highest rate of encountering technical issues most likely due to weaker long horizon planning and less effective state tracking across intermediate actions (e.g., the need to go back to patient records and check for information multiple times) compared to the other two advanced models. We therefore gradually increased the step limits and re-evaluated Gemini Flash’s performance on the subset of previously uncompleted tasks (323 out of 735) spanning all profile types. As shown in Figure 3(a), there is a significant boost when the step limit is increased from 40 to 55, with the proportion of completed tasks rising from 56.1% at 40 steps to 89.2% at 55 steps. The marginal gain in completion rate progressively diminishes with further increases, eventually saturating at over 99% at 85 and 100 steps. Figure 3(b) compares the form-level accuracies under different step limits, which is highest under 55 max steps (median 95.8%) and declines monotonically as the budget relaxes. This suggests that while loosening budgets may enable more tasks to complete end-to-end, it might also introduce extra noise through redundant reinteractions with already-completed fields. Figure 3(c) demonstrates the bootstrapped distribution of step counts over completed tasks. The sharp, narrow peak centered around 25~30 steps in the max_steps=40 group is not indicative of the model’s typical behavior given the high task failure rate, but rather a survivorship artifact: only a fraction of runs happened to find an efficient trajectory short enough to complete the task within the tight step budget. As the step budget is relaxed, the distribution progressively flattens and shifts rightward, reflecting the inclusion of longer, more tortuous trajectories involving backtracking that would have been prematurely truncated under tighter limits.

Figure 3. Ablation Study 1: Effect of Step Budget on Gemini Flash.

Figure 3.

Open in a new tab

Given the high rate of technical failures observed for Gemini 3 Flash under a 40-step constraint in the main experiment, we systematically increased the maximum allowed steps in this ablation study.

(a) Left: task completion rate (bootstrapped median with 95% confidence intervals) as a function of step limit. Right: distribution of task outcomes among completed tasks.

(b) Proportions of correctly completed fields per submission among successfully submitted compliant profiles across different step limits.

(c) Distribution of the number of steps consumed per completed task under each step limit.

Ablation 2: agent VS non-agent setting

To disentangle the contribution of agentic browser interaction workflow from the underlying LLM’s intrinsic reasoning capacity, we compared outcomes between the full agent setting and a non-agent baseline across the complete set of patient profiles, using Gemini 3 Pro as a representative model. In the non-agent setting, the same model was provided with the complete patient profile directly in context/prompt and asked to make a submission or withholding decision using pure LLM reasoning (Supplementary Prompt 4), without any additional browser interaction. As shown in Figure 4(a), in the non-agent setting, Gemini 3 pro can identify the injected issues in 91% of the deficient patient profiles, which indicates that the LLM itself possesses substantial capacity identify profile-level issues when information is presented directly in context. The sharp degradation in withholding performance in the agent setting therefore reflects a failure to reliably surface and integrate critical information when interacting with the browser environment and dynamically constructing context, rather than a fundamental limitation in the model’s reasoning capacity. In addition, Gemini in non-agentic setting achieved 0% submission rate even for compliant profiles, as the model consistently flagged undesigned issues inherent to the synthetic data, such as provider NPI numbers deviating from real-world formatting standards and clinical records insufficiently supporting expedited test requests.

Figure 4. Ablation Studies 2 and 3: Role of Agent Execution and Prompting.

Figure 4.

Open in a new tab

(a) Comparison between agent-based and non-agent (chat-only) settings. In the non-agent condition, a similar prompt with patient profiles was provided to Gemini 3 Pro in a chat API to elicit submission or withholding decisions along with identified issues. Task outcomes are compared across settings. Technical errors are not applicable in the non-agent condition due to the absence of browser execution. For deficient profiles, the model may identify both injected (ground-truth) and undesigned issues within a single instance; therefore, proportions do not sum to 100%.

(b) Agent performance on 125 deficient profiles (25 profiles from each of type 2a, 2b, 2c, 3 and 4) with enhanced prompt instructions (e.g., explicit emphasis on reviewing date-related fields). The panel reports the distribution of task outcomes across profile categories.

Ablation 3: prompt specificity

Given the low correct withholding rates observed for profiles with injected issues or errors, we tested whether targeted prompt augmentation could improve error detection without providing explicit information about what exact errors to look for. Specifically, the phrase “carefully review date-related information in the patient record” was appended to the standard prompt used for the main experiment (Supplementary Prompt 5), which was then evaluated on all deficient profile types using Gemini 3 Pro. As shown in Figure 4(b), this prompt modification produced substantial improvements in correct withholding for date-sensitive profiles: profiles with subscriber age error improved to 96% correct withholding, and profiles with chronological errors reached 100%. However, the benefit was not uniform across other deficient profile types: profiles with missing specimen collection date, which is a required field in the PA portal, showing only 36% correct withholding rate. Similarly, the improvement was neither generalized to the other two sample types with non-date-specific issues: 0% for profiles with irrelevant medical information and 29.2% of the correct withholding for profile with duplicate patient names. These results demonstrate that while targeted prompt engineering can meaningfully improve error detection for specific error types, broader improvements will likely require more structured validation mechanisms beyond prompt-level guidance.

Discussion

In this study, we present the PA-submission sandbox and an evaluation framework for LLM-powered agentic systems specifically designed for PA submission of ES/GS. Unlike existing web agent benchmarks that primarily assess whether an agent can successfully complete a task assuming valid inputs,18,20 our sandbox additionally evaluates whether agents can follow general safeguard instructions to halt the process when issues are presented, a capability increasingly recognized as critical for safe deployment but systematically understudied in existing evaluation frameworks.21,22 Given that clinical documentation errors are pervasive in real-world practice,23,24 we believe the capacity to withhold action under uncertainty is a critical yet underexplored attribute for any productively deployable LLM-powered clinical agent. By running browser-based agents on synthetic PA forms and patient profiles, we find that current models exhibit a systematic bias toward form submission that poses considerable patient safety risks.

Our results carry several important implications. First, while frontier LLMs demonstrate reasonable capacity for identifying clinical issues in isolation (i.e., non-agentic setting), performance degrades substantially when tasks are compounded in an agent mode where the agent is required to simultaneously identify documentation errors and conditionally submit only if none are found. In our study, the agents particularly appear engineered toward form completion and submission despite the presence of errors. This behavior likely reflects a bias introduced during reinforcement learning fine-tuning, where reward signals have been optimized for successful tool calls rather than appropriate abstention.25–27 This sycophantic completion bias is a well-documented consequence of Reinforcement Learning from Human Feedback (RLHF) which poses a direct risk concern in clinical agentic workflows.28

Second, relaxing the maximum step limit substantially increases task completion rates, while also leading the agent to consume more steps to complete the same tasks under a looser budget. The improvement in completion with higher step limits can be possibly explained by the self-conditioning effect in which models are more likely to make mistakes in the subsequent steps when context contains earlier mistakes.29 Under a tight constraint, the agent has little to no margin for error recovery; mistakes made at intermediate steps (e.g., incorrect navigation or form interactions) often require multi-step corrections that exhaust the remaining budget before completion, resulting in failure to complete the task. In addition, tighter constraints incentivize concise execution, whereas looser budgets promote more deliberative, verification-oriented strategies that increase step usage.30,31 This explains why agents tend to consume more steps to complete the same task under relaxed budgets. Cost-effectiveness and energy efficiency should therefore be also considered in design. In our experiments, each step required around 17000 tokens with Claude Opus 4.5($5/million input tokens, $25/million output tokens) versus ~11300 tokens with Gemini models ($0.5/million input tokens, $3/million output tokens for Gemini 3 Flash; $2/million input tokens, $12.million output tokens for Gemini 3 Pro). Even though the smaller model needed a larger max step limit to reach a comparable submission success rate, its overall costs remained lower.

Third, while augmenting agent prompts with additional error-specific information improves performance on the targeted error types, this benefit does not generalize to other error categories. Prior work shows that prompt engineering is highly context- and task-dependent, often improving some subtasks while degrading others, and remains sensitive to small variations in phrasing or format.32 This brittleness makes exhaustive prompt engineering an impractical strategy for clinical deployment, as it would require anticipating every possible failure mode in advance. In practice, a structured onboarding or “shadowing” phase, during which the agent observes real submission workflows, encounters errors, and iteratively refines its own prompts or skill documents, is likely necessary to achieve broad coverage.33 However, it should be noted that purely intrinsic self-correction without external feedback signals may not reliably improve reasoning performance and could in some cases degrade it,34 suggesting that human-in-the-loop validation remains essential during any such onboarding process.

Finally, our results suggest that reliable error detection and form submission may be fundamentally difficult to achieve within a single autonomous agent with current models. Instead, a more controlled, modular workflow may be a more appropriate architecture for clinical deployment than a fully end-to-end autonomous agent. This preference is also consistent with the fact that the current human PA workflow is a coordinated team effort rather than individual responsibility. In practice, PA submission for ES/GS is divided into distinct roles: administrative staff retrieve and compile patient information from the EHR; a genetic counselor or PA specialist reviews the documentation for completeness and clinical appropriateness; and a designated submitter (often a medical assistant) enters and submits the finalized form to the payer.35,36 A multi-agent pipeline can mirror the same workflow naturally. The first agent retrieves and organizes a given patient’s clinical profile from the EHR. The second agent then independently evaluates the profile for documentation errors or clinical inconsistencies. Only upon confirmation of a complete, deficient-free profile does a third agent proceeds to finalize the PA request. This staged design is consistent with emerging evidence that multi-agent designs outperform monolithic LLM agents in clinical tasks.37,38 Critically, this pipeline also naturally involves human oversight between stages, positioning the human reviewer as a deliberate participant in this multi-agent workflow.

Human oversight within this pipeline should not just be deferred to the final submission step, but rather first embedded between profile-review and submission agents. The output of the profile-review agent should be presented to a genetic counselor or specialist in a human-readable format, listing each identified issue with the supporting evidence drawn from the patient record before any submission action is triggered. Only after explicit sign-off at this intermediate stage would the submission agent proceed, and human oversight may also extend to verification of the completed PA form itself. The final human review on completed PA form could be exception-based or confidence-weighted, in which the form-filling agent flags fields it populated with low confidence (e.g., rationale for testing fields) for targeted human inspection, reserving full form review for complex cases such as expedited requests. This layered verification preserves the efficiency gains of automation while ensuring that human still holds sufficient accountability.

Supplementary Material

Supplement 1
media-1.docx (1.6MB, docx)
Supplement 2
media-2.docx (1.6MB, docx)

Funding Statement

This study was funded by the National Institute of Health / National Human Genome Research Institute (NIH / NHGRI) under award number R01HG120566.

Funding Statement

This study was funded by the National Institute of Health / National Human Genome Research Institute (NIH / NHGRI) under award number R01HG120566.

Footnotes

Declaration of AI and AI-assisted technologies in the writing process

During the preparation of this work the author(s) used Claude and ChatGPT in order to improve phrasing and the flow of writing. After using this tool/service, the author(s) reviewed and edited the content as needed and take(s) full responsibility for the content of the publication.

Ethics Declaration

Not applicable, there is no human subject or real patient data involved in the study.

Conflict of Interest

The authors declare no competing interests for this study.

Data Availability

The source code of the sandbox and experiment runs is available at https://github.com/stormliucong/wes-wgs-pa-app.

References

  • 1.Marcus BS, Bansal N, Saef J, et al. Burden with No Benefit: Prior Authorization in Congenital Cardiology. Pediatr Cardiol. 2024;45(1):100–106. doi: 10.1007/s00246-023-03255-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Sahni NR, Istvan B, Stafford C, Cutler D. Perceptions of prior authorization burden and solutions. Health Affairs Scholar. 2024;2(9):qxae096. doi: 10.1093/haschl/qxae096 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Some Medicare Advantage Organization Denials of Prior Authorization Requests Raise Concerns About Beneficiary Access to Medically Necessary Care. Office of Inspector General | Government Oversight | U.S. Department of Health and Human Services. April 27, 2022. Accessed June 1, 2026. https://oig.hhs.gov/reports/all/2022/some-medicare-advantage-organization-denials-of-prior-authorization-requests-raise-concerns-about-beneficiary-access-to-medically-necessary-care/ [Google Scholar]
  • 4.kffmichellel. Claims Denials and Appeals in ACA Marketplace Plans in 2024. KFF. March 24, 2026. Accessed April 29, 2026. https://www.kff.org/patient-consumer-protections/claims-denials-and-appeals-in-aca-marketplace-plans-in-2024/ [Google Scholar]
  • 5.Kim MK, Rouphael C, McMichael J, Welch N, Dasarathy S. Challenges in and Opportunities for Electronic Health Record-Based Data Analysis and Interpretation. Gut and Liver. 2024;18(2):201–208. doi: 10.5009/gnl230272 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Fixing prior auth: Nearly 40 prior authorizations a week is way too many. American Medical Association. April 24, 2025. Accessed April 29, 2026. https://www.ama-assn.org/practice-management/prior-authorization/fixing-prior-auth-nearly-40-prior-authorizations-week-way [Google Scholar]
  • 7.Stark Z, Schofield D, Alam K, et al. Prospective comparison of the cost-effectiveness of clinical whole-exome sequencing with that of usual care overwhelmingly supports early use and reimbursement. Genet Med. 2017;19(8):867–874. doi: 10.1038/gim.2016.221 [DOI] [PubMed] [Google Scholar]
  • 8.Clark MM, Stark Z, Farnaes L, et al. Meta-analysis of the diagnostic and clinical utility of genome and exome sequencing and chromosomal microarray in children with suspected genetic diseases. NPJ Genom Med. 2018;3:16. doi: 10.1038/s41525-018-0053-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Smith HS, Franciskovich R, Lewis AM, et al. Outcomes of prior authorization requests for genetic testing in outpatient pediatric genetics clinics. Genet Med. 2021;23(5):950–955. doi: 10.1038/s41436-020-01081-x [DOI] [PubMed] [Google Scholar]
  • 10.Anderson KE, Darden M, Jain A. Improving Prior Authorization in Medicare Advantage. JAMA. 2022;328(15):1497–1498. doi: 10.1001/jama.2022.17732 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Yao S, Zhao J, Yu D, et al. ReAct: Synergizing Reasoning and Acting in Language Models. arXiv. Preprint posted online March 10, 2023:arXiv:2210.03629. doi: 10.48550/arXiv.2210.03629 [DOI] [Google Scholar]
  • 12.He H, Yao W, Ma K, et al. WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models. arXiv. Preprint posted online June 6, 2024:arXiv:2401.13919. doi: 10.48550/arXiv.2401.13919 [DOI] [Google Scholar]
  • 13.Luo L, Kim SE, Zhang X, et al. A clinical environment simulator for dynamic AI evaluation. Nat Med. 2026;32(3):820–827. doi: 10.1038/s41591-026-04252-6 [DOI] [PubMed] [Google Scholar]
  • 14.Hager P, Jungmann F, Holland R, et al. Evaluation and mitigation of the limitations of large language models in clinical decision-making. Nat Med. 2024;30(9):2613–2622. doi: 10.1038/s41591-024-03097-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Jiang Y, Black KC, Geng G, et al. MedAgentBench: A Virtual EHR Environment to Benchmark Medical LLM Agents. NEJM AI. 2025;2(9). doi: 10.1056/AIdbp2500144 [DOI] [Google Scholar]
  • 16.BrowserArena: Evaluating LLM Agents on Real-World Web Navigation Tasks. Accessed April 30, 2026. https://arxiv.org/html/2510.02418v2
  • 17.Zhu K, Liu Z, Li B, et al. Where LLM Agents Fail and How They can Learn From Failures. arXiv. Preprint posted online September 29, 2025:arXiv:2509.25370. doi: 10.48550/arXiv.2509.25370 [DOI] [Google Scholar]
  • 18.Drouin A, Gasse M, Caccia M, et al. WorkArena: how capable are web agents at solving common knowledge work tasks? In: Proceedings of the 41st International Conference on Machine Learning. Vol 235. ICML’24. JMLR.org; 2024:11642–11662. [Google Scholar]
  • 19.Qiu Y, Yao H, Ren P, Tian X, You M. Regulatory sandbox expansion: Exploring the leap from fintech to medical artificial intelligence. Intelligent Oncology. 2025;1(2):120–127. doi: 10.1016/j.intonc.2025.03.001 [DOI] [Google Scholar]
  • 20.Zhou S, Xu FF, Zhu H, et al. WebArena: A Realistic Web Environment for Building Autonomous Agents. arXiv. Preprint posted online April 16, 2024:arXiv:2307.13854. doi: 10.48550/arXiv.2307.13854 [DOI] [Google Scholar]
  • 21.Levy I, Wiesel B, Marreed S, Oved A, Yaeli A, Shlomov S. ST-WebAgentBench: A Benchmark for Evaluating Safety and Trustworthiness in Web Agents. arXiv. Preprint posted online March 2, 2026:arXiv:2410.06703. doi: 10.48550/arXiv.2410.06703 [DOI] [Google Scholar]
  • 22.Kirichenko P, Ibrahim M, Chaudhuri K, Bell S. AbstentionBench: Reasoning LLMs Fail on Unanswerable Questions. In: 2025. Accessed April 30, 2026. https://openreview.net/forum?id=OkHC30LLpO&referrer=%5Bthe%20profile%20of%20Samuel%20J.%20Bell%5D(%2Fprofile%3Fid%3D~Samuel_J._Bell1)
  • 23.Bowman S. Impact of electronic health record systems on information integrity: quality and safety implications. Perspect Health Inf Manag. 2013;10(Fall):1c. [PMC free article] [PubMed] [Google Scholar]
  • 24.Bell SK, Delbanco T, Elmore JG, et al. Frequency and Types of Patient-Reported Errors in Electronic Health Record Ambulatory Care Notes. JAMA Netw Open. 2020;3(6):e205867. doi: 10.1001/jamanetworkopen.2020.5867 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Sharma M, Tong M, Korbak T, et al. Towards Understanding Sycophancy in Language Models. arXiv. Preprint posted online May 10, 2025:arXiv:2310.13548. doi: 10.48550/arXiv.2310.13548 [DOI] [Google Scholar]
  • 26.Perez E, Ringer S, Lukosiute K, et al. Discovering Language Model Behaviors with Model-Written Evaluations. In: Rogers A, Boyd-Graber J, Okazaki N, eds. Findings of the Association for Computational Linguistics: ACL 2023. Association for Computational Linguistics; 2023:13387–13434. doi: 10.18653/v1/2023.findings-acl.847 [DOI] [Google Scholar]
  • 27.Shapira I, Benade G, Procaccia AD. How RLHF Amplifies Sycophancy. arXiv. Preprint posted online February 1, 2026:arXiv:2602.01002. doi: 10.48550/arXiv.2602.01002 [DOI] [Google Scholar]
  • 28.Shapira I, Benade G, Procaccia A. How RLHF Amplifies Sycophancy. 2026. doi: 10.48550/arXiv.2602.01002 [DOI] [Google Scholar]
  • 29.Sinha A, Arun A, Goel S, Staab S, Geiping J. The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMs. arXiv. Preprint posted online March 13, 2026:arXiv:2509.09677. doi: 10.48550/arXiv.2509.09677 [DOI] [Google Scholar]
  • 30.Liu T, Wang Z, Miao J, et al. Budget-Aware Tool-Use Enables Effective Agent Scaling. arXiv. Preprint posted online November 21, 2025:arXiv:2511.17006. doi: 10.48550/arXiv.2511.17006 [DOI] [Google Scholar]
  • 31.Li J, Zhao W, Zhang Y, Gan C. Steering LLM Thinking with Budget Guidance. arXiv. Preprint posted online June 16, 2025:arXiv:2506.13752. doi: 10.48550/arXiv.2506.13752 [DOI] [Google Scholar]
  • 32.Chai M, Zomorrodi AR. Prompt engineering does not universally improve Large Language Model performance across clinical decision-making tasks. arXiv. Preprint posted online December 28, 2025:arXiv:2512.22966. doi: 10.48550/arXiv.2512.22966 [DOI] [Google Scholar]
  • 33.Madaan A, Tandon N, Gupta P, et al. Self-Refine: Iterative Refinement with Self-Feedback. arXiv. Preprint posted online May 25, 2023:arXiv:2303.17651. doi: 10.48550/arXiv.2303.17651 [DOI] [Google Scholar]
  • 34.Huang J, Chen X, Mishra S, et al. Large Language Models Cannot Self-Correct Reasoning Yet. arXiv. Preprint posted online March 14, 2024:arXiv:2310.01798. doi: 10.48550/arXiv.2310.01798 [DOI] [Google Scholar]
  • 35.Uhlmann WR, Schwalm K, Raymond VM. Development of a Streamlined Work Flow for Handling Patients’ Genetic Testing Insurance Authorizations. Journal of Genetic Counseling. 2017;26(4):657–668. doi: 10.1007/s10897-017-0098-3 [DOI] [PubMed] [Google Scholar]
  • 36.Bajguz D, Danylchuk NR, Czarniecki M, Selig JP, Sutphen R, Kaylor J. Utilization of genetic testing: Analysis of 4,499 prior authorization requests for molecular genetic tests at four US regional health plans. Journal of Genetic Counseling. 2022;31(3):771–780. doi: 10.1002/jgc4.1543 [DOI] [PubMed] [Google Scholar]
  • 37.Chen X, Yi H, You M, et al. Enhancing diagnostic capability with multi-agents conversational large language models. npj Digit Med. 2025;8(1):159. doi: 10.1038/s41746-025-01550-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Klang E, Omar M, Raut G, et al. Orchestrated multi agents sustain accuracy under clinical-scale workloads compared to a single agent. npj Health Syst. 2026;3(1):23. doi: 10.1038/s44401-026-00077-0 [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplement 1
media-1.docx (1.6MB, docx)
Supplement 2
media-2.docx (1.6MB, docx)

Data Availability Statement

The source code of the sandbox and experiment runs is available at https://github.com/stormliucong/wes-wgs-pa-app.

Articles from medRxiv are provided here courtesy of Cold Spring Harbor Laboratory Preprints

RESOURCES