Disparities in AI‐Based Prior Authorization for Head and Neck Reconstruction: A Large Language Model Analysis

ABSTRACT

Introduction

Large language models (LLM) are being rapidly integrated into healthcare, particularly to streamline time‐ and labor‐intensive administrative processes. However, the potential for artificial intelligence (AI) systems to demonstrate bias when employed for insurance authorization remains poorly understood. As insurers increasingly adopt AI to make coverage decisions, this study examined bias in LLM‐driven prior authorization in otolaryngology, using oral cavity squamous cell carcinoma (SCC) as a case study.

Methods

Using OpenAI's generative transformer GPT‐4o, this study assessed for LLM bias when simulating insurance coverage decisions for head and neck cancer reconstruction. A standardized clinical scenario was constructed involving patients with T2N2 oral cavity SCC, all requiring surgical resection and reconstruction. The LLM was prompted to choose between a radial forearm free flap (RFFF) and split‐thickness skin graft (STSG) for reconstruction, across 19,900 simulations. Patient profiles were systematically varied by age, sex, race/ethnicity, zip code‐based income level, socioeconomic status (SES), and substance use history.

Results

The LLM output showed significant disparities in approval decisions. RFFF was more frequently approved for younger, Asian, or white patients from high‐income zip codes or high SES backgrounds (p < 0.0001). Older, Black, and Hispanic patients, and those from lower‐income areas or with substance use histories, were less likely to receive RFFF authorization (p < 0.0001). On sensitivity analysis, inclusion of tumor‐specific information markedly skewed recommendations towards RFFF across sociodemographic backgrounds.

Conclusion

In this experimental study, the LLM's outputs exhibited significant disparities for oral cavity cancer reconstruction based on patient demographic variables in the setting of limited clinical information. Inputs to LLMs for clinical decision‐making should include pertinent and detailed information to reduce the risk of bias. As insurers increasingly integrate AI for prior authorization, recognition of its biases, rigorous safeguards, and increased regulatory governance are needed to promote equitable health care.

1. Introduction

The adoption of large language models (LLMs) in healthcare is rapidly expanding, with applications spanning clinical documentation, patient education, clinical decision support, and administrative workflows [1]. Among these, prior authorization, which has historically been a labor‐intensive, time‐consuming, and opaque process, has emerged as an important area for AI integration. LLMs are trained on petabytes of text data from Internet and book archives and can produce advanced human‐like output informed by deep and broad knowledge. Despite these advantages and the ability to offer efficiencies previously unattainable, concerns about algorithmic bias have grown, particularly as these models are often deployed without full transparency or human oversight. Pre‐training data used to develop LLMs are often proprietary, and may include antiquated or prejudiced text, false news sources, and social media, which may contribute to LLM outputs that perpetuate prejudiced content [2].

Prior studies have demonstrated that LLMs may exhibit biased decision‐making, which could in turn have harmful downstream effects. A study by Omar et al showed that LLMs prescribe invasive interventions at a higher frequency in patients who are Black, LGBTQIA+, or unhoused [3]. When a LLM was used to simulate physicians deciding life‐saving care between patients in a resource‐scarce environment, medical decision‐making was biased by patients' race, gender, sexual orientation, political affiliation, and age [4]. These preliminary findings raise concerns that if AI systems are embedded in high‐stakes clinical decision environments, such as insurance coverage, they may reproduce or even amplify pre‐existing health disparities.

However, large health insurers are increasingly incorporating AI into their authorization process [5, 6, 7]. Reports indicate that AI use in health insurance may increase denial rates by up to 16‐fold, sometimes without individualized review [8]. Legal action has followed, with allegations of wrongful denials secondary to the use of these AI algorithms [9, 10]. Despite this, little research has directly assessed whether current LLMs display bias when applied to insurance coverage decisions, especially for complex‐high‐cost procedures such as head and neck reconstruction.

Using OpenAI's generative pretrained transformer (GPT‐4o) as a test case, this study explored potential biases in LLMs from the perspective of an insurance company making prior authorization decisions for patients with head and neck cancer. We hypothesized that LLM‐based decisions would vary based on patient sociodemographic characteristics.

2. Materials and Methods

A standardized clinical scenario involving patients with T2N2 squamous cell carcinoma of the lateral oral tongue was constructed. Each simulated patient was scheduled to undergo a partial glossectomy and neck dissection for a resectable but life‐threatening cancer. All clinical variables, including tumor staging, imaging, physical exam, prognosis, and baseline health status, were held constant to ensure medical equivalence across cases. The purpose was to isolate whether the model's recommendations changed based solely on patient demographics, rather than clinical need. The scenario (Supporting Information S1: File 1) was presented to GPT‐4o in the role of a medical review officer working for an insurance company in a resource‐limited environment. The model was asked to decide whether to approve a radial forearm free flap (RFFF) or a split‐thickness skin graft (STSG) for surgical reconstruction. The model was instructed to choose only one option and then briefly explain the decision.

The only variables that differed between simulated patients were age (30 vs. 60 years), sex, race/ethnicity, zip code–based income level, socioeconomic status (high vs. low), and substance use history (tobacco, alcohol, or illicit drugs). High‐ and low‐income zip codes were selected based on U.S. Census data for California [11]. Each simulation presented a patient profile with a unique demographic combination.

The prompt was intentionally designed to be clinically ambiguous; it did not indicate that one reconstructive method was preferred or medically necessary over the other. This design allowed both RFFF and STSG to be considered reasonable options depending on individual patient factors or institutional practice, enabling us to isolate whether demographic characteristics alone influenced the model's decisions. In practice, tumor characteristics such as diameter, depth of invasion, perineural or lymphovascular invasion, and involvement of adjacent tissues such as floor of mouth and mandible, are factored into the choice of reconstruction [12]. These tumor‐specific factors were intentionally excluded from the simulation to isolate the effect of patient demographics on the model's decision‐making.

A sensitivity analysis (Supporting Information S1: File 2) was performed to assess whether inclusion of tumor‐specific information in separate independent simulations would affect the LLM outcome. The additional prompt included information about size of the expected defect involvement of the lateral tongue, and extension to the floor of mouth. The additional prompt also explicitly stated that RFFF is expected to provide better functional outcomes.

Using the OpenAI Application Programming Interface (API) connected via Python (Python Software Foundation, Wilmington, Delaware), 100 independent simulations were conducted for each patient permutation, totaling 19,900 queries. All simulations were performed using default OpenAI API settings (temperature = 1.0, top_p = 1.0, frequency_penalty = 0.0, presence_penalty = 0.0). The model was explicitly instructed to provide an unbiased answer when choosing between radial forearm free flap (RFFF) and split‐thickness skin graft (STSG). Each query was submitted independently to prevent memory or contextual influence from prior prompts. Patient order was randomized within each iteration to minimize ordering bias. If no response was generated, the prompt was retried up to five times. Model outputs were recorded as either “RFFF” or “STSG,” and the accompanying rationale was stored for qualitative analysis. The prompt and model instructions are included in Supporting Materials. Statistical analysis was performed using chi‐squared tests with pairwise comparisons, with significance defined as p < 0.05. The study was deemed exempt from institutional review board (IRB) oversight.

3. Results

A total of 19,900 independent queries were submitted to GPT‐4o across all patient permutations. Table 1 summarizes the RFFF approval rates by demographic profile. The highest approval rates were observed among 30‐year‐old Asian males (73%), 30‐year‐old Asian females (71%), and 30‐year‐old White males (60%). The lowest approval rates occurred among patients with substance use histories (e.g., 55‐year‐old alcohol drinkers, 16% and 55‐year‐old smokers, 18%), and those from low‐income zip codes, such as 93,641 (17%).

Table 1.

Rates of prior authorization approval for radial forearm free flap (RFFF) over 100 GPT‐4o simulation iterations, stratified by patient age, sex, and race/ethnicity. Each simulated patient had identical cancer staging and health status, with only demographic variables varied. Approval percentages reflect the proportion of model responses recommending RFFF over split‐thickness skin graft (STSG).

Patient Profile	Approval rate for RFFF(%)	Approval rate for STSG(%)
30‐year‐old Asian female	71	29
30‐year‐old Asian male	73	27
30‐year‐old Black female	36	64
30‐year‐old Black male	30	70
30‐year‐old Hispanic female	37	63
30‐year‐old Hispanic male	36	64
30‐year‐old White female	54	46
30‐year‐old White male	60	40
60‐year‐old Asian female	38	62
60‐year‐old Asian male	48	52
60‐year‐old Black female	34	66
60‐year‐old Black male	44	56
60‐year‐old Hispanic female	28	72
60‐year‐old Hispanic male	38	62
60‐year‐old White female	43	57
60‐year‐old White male	49	51
Patient living in zip code 90013 (Los Angeles, CA)—low SES	46	54
Patient living in zip code 90021 (Los Angeles, CA)—low SES	64	34
Patient living in zip code 93641 (Miramonte, CA)—low SES	17	83
Patient living in zip code 94022 (Los Altos, CA)—high SES	47	53
Patient living in zip code 94024 (Los Altos, CA)—high SES	44	56
Patient living in zip code 94027 (Atherton, CA)—high SES	49	51
Patient living in zip code 94506 (Danville, CA)—high SES	36	64
Patient living in zip code 94507 (Alamo, CA)—high SES	42	58
Patient living in zip code 95202 (Stockton, CA)—low SES	28	72
Patient living in zip code 95559 (Phillipsville, CA)—low SES	23	77
30‐year‐old Asian patient of high SES	40	60
30‐year‐old Asian patient of low SES	45	55
30‐year‐old Black patient of high SES	57	43
30‐year‐old Black patient of low SES	44	56
30‐year‐old Hispanic patient of high SES	34	66
30‐year‐old Hispanic patient of low SES	38	62
30‐year‐old White patient of high SES	24	76
30‐year‐old White patient of low SES	39	61
55‐year‐old heavy alcohol drinker	16	84
55‐year‐old illicit drug user	28	72
55‐year‐old smoker	18	82
55‐year‐old nonsmoker, non‐alcohol user, non‐drug user	43	57

3.1. Simulation 1: Age, Sex, and Race/Ethnicity

In the first simulation, patient profiles differed by three demographic dimensions, by age, sex, and race (Figure 1). The model output showed significant statistically significant disparities among 30‐year‐old Black females, 30‐year‐old Black males, 30‐year‐old Hispanic males, 60‐year‐old Black females, and 60‐year‐old Hispanic females (all p < 0.001), who were significantly less likely to be approved for RFFF compared to their white and Asian counterparts.

Figure 1.

Differences in insurance authorization outcomes from a large language model when comparing patients based on three demographic axes: race, sex, and age. Bold values indicate statistically significant comparisons, whereas gray shaded boxes indicate non‐significant comparisons. Values > 1 (red) represent preference for group 1 for RFFF, whereas values < 1 (blue) represent preference for group 2 for RFFF. RFFF, radial forearm free flap; yo, year old.

In contrast, Asian and white patients were more likely to receive RFFF approval, regardless of sex (p < 0.001). When comparing younger to older white patients, no significant age‐related differences were observed. However, younger Asian patients were significantly more likely to be approved for RFFF than older Asian patients (p < 0.001).

Rationales for RFFF frequently cited “superior functional and aesthetic outcomes” or “higher quality of life after surgery.” STSG was often justified by statements such as “provides adequate coverage,” “faster recovery,” or “more cost‐effective for the insurance company.” Although age was sometimes referenced, sex and race were consistently omitted from the model's rationale.

3.2. Simulation 2: Race and Socioeconomic Status

The second simulation compared 30‐year‐old patients across combinations of race and socioeconomic status (SES) (Figure 2). High‐SES Asian patients were significantly more likely to receive RFFF approvals than other groups. Notably, even among high‐SES patients, white and Hispanic individuals had lower RFFF approval rates compared to Asians. Among patients with low SES, approval rates did not significantly differ by race.

Figure 2.

Differences in insurance authorization outcomes from a large language model when comparing patients based on two demographic axes: race and socioeconomic status. Bold values indicate statistically significant comparisons, whereas gray shaded boxes indicate non‐significant comparisons. Values > 1 (red) represent preference for group 1 for RFFF, whereas values < 1 (blue) represent preference for group 2 for RFFF. pt, patient; RFFF, radial forearm free flap; SES, socioeconomic status; yo, year old.

The model's explanations often included contradictory reasoning. Patients with high SES were recommended RFFF for reasons such as “high SES can often facilitate comprehensive post‐operative care.” In contrast, patients of low SES were recommended STSG for reasons such as “reducing time away from work and associated costs.” In several instances, the model asserted that “SES or race are not relevant to the clinical decision,” even when clear differences in outcomes were observed.

3.3. Simulation 3: Zip Code and Geographic Location

The third simulation evaluated patients based on zip code of residence, comparing the five highest‐income and five lowest‐income zip codes in California (Figure 3). Patients from high‐income areas were significantly more likely to be approved for RFFF (p < 0.001). No significant differences were observed when comparing between high‐income zip codes.

Figure 3.

Differences in insurance authorization outcomes from a large language model when comparing patients based on zip code of residence, classified as either a high socioeconomic status zip code or a low socioeconomic status zip code. Bold values indicate statistically significant comparisons, whereas gray shaded boxes indicate non‐significant comparisons. Values > 1 (red) represent preference for group 1 for RFFF, whereas values < 1 (blue) represent preference for group 2 for RFFF. RFFF, radial forearm free flap.

Explanations provided for choosing RFFF in this simulation included statements such as, “given the urban location of Los Angeles, CA, there is likely easier access to the specialized care needed for complex reconstruction,” and “Atherton, CA is an affluent area where high‐quality healthcare is expected.” In contrast, explanations for STSG included: “Stockton, CA is an area where local healthcare facilities could be more familiar with performing and managing skin grafts, which generally require less specialized postoperative care and resources,” and “Phillipsville, CA is a rural area which may limit access to highly specialized postoperative care.”

3.4. Simulation 4: Substance Use History

In the final simulation, approval rates were compared among patients with or without a history of smoking, alcohol use, or illicit drug use (Figure 4). Patients with any form of substance use were significantly less likely to be approved for RFFF (p < 0.001). Among the three categories, alcohol use was associated with the lowest RFFF approval rate, followed by tobacco and illicit drug use.

Figure 4.

Differences in insurance authorization outcomes from a large language model when comparing patients based on history of substance use (smoking, alcohol, or illicit drugs). Bold values indicate statistically significant comparisons, whereas gray shaded boxes indicate non‐significant comparisons. Values > 1 (red) represent preference for group 1 for RFFF, whereas values < 1 (blue) represent preference for group 2 for RFFF. Abbreviations: RFFF, radial forearm free flap; yo, year old.

For both smoking and alcohol history, the LLM output used both poor wound healing as a rationale for and against RFFF. Namely, STSG was chosen due to “the risks of poor wound healing from impaired blood flow”, yet for other patients, RFFF was chosen due to “the superior vascular supply which is crucial in patients with poor wound healing”. For illicit drug users, STSG was commonly chosen due to concerns for the “patient's ability to adhere to the complex postoperative care.”

3.5. Sensitivity Analysis

Addition of tumor‐specific information to the prompt markedly skewed the LLM output across all sociodemographic variables. For all permutations of age, sex, and race, inclusion of tumor‐specific information resulted in a recommendation of RFFF reconstruction in 99%–100% of profiles. Across permutations of race and SES, the additional information led to a recommendation of RFFF in 98%–100% of profiles. For substance use history, the additional tumor‐specific information led to a recommendation of RFFF for 90% of smokers, 77% of alcohol users, 60% of illicit drug users, and 100% of patients without smoking, alcohol, or illicit drug use.

4. Discussion

This experimental study demonstrated that an LLM, when tasked with simulated insurance decision‐making for head and neck cancer reconstruction, demonstrated significant disparities in decision‐making based on patient demographics in the setting of limited clinical information. Across nearly 20,000 iterations, the model preferentially approved RFFF for patients who were younger, Asian or white, from higher‐income zip codes or higher socioeconomic status (SES) backgrounds. In contrast, patients who were older, Black or Hispanic, had a history of substance use, or lived in low‐income or rural areas were disproportionately denied RFFF and instead approved for STSG. On sensitivity analysis, inclusion of tumor‐specific information such as expected defect involvement of the lateral tongue, extension to the floor of mouth, and anticipated improved functional outcomes with RFFF, skewed recommendations to RFFF markedly across sociodemographic profiles.

This study's findings raise ethical, clinical, and operational concerns as health insurers increasingly explore the use of AI and LLMs to streamline prior authorization workflows. By omitting clinical variation and controlling for disease severity and tumor factors, this study was designed to isolate the influence of demographics alone in the test case. Our findings tested over 19,900 queries indicate that the LLM consistently produced differing authorization recommendations for reconstructive options based on patient demographics, despite identical clinical presentations. As such, when AI systems are used for prior authorization without appropriate validation, oversight, or safeguards, they risk undermining patient care and interfering with surgeon expertise, clinical judgment, and procedural decision‐making. The appeals process for prior authorization denials often requires additional documentation and peer‐to‐peer discussions, which contributes to physicians' administrative burden and burnout. Additional time and effort taken to counter prior authorization denials may also prolong wait times for surgery [13].

The reconstructive decision between RFFF and STSG in oral tongue cancer is nuanced and multifactorial. Resection of tongue cancer can lead to functional deficits of dysarthria and dysphagia, which negatively impact quality of life. Both RFFF and STSG can be viable surgical options for reconstruction of glossectomy defects, each with distinct advantages and limitations. RFFF is a time‐intensive procedure, typically performed at tertiary‐care institutions by fellowship‐trained microvascular surgeons and requires close postoperative level monitoring and specialized nursing care [14], factors of which contribute to higher hospital charges. In contrast, STSG is a less labor‐intensive procedure with decreased donor site morbidity, faster recovery, and reduced operative time. STSG is a less costly procedure and often performed in the ambulatory setting [15]. Studies have shown that patients who undergo RFFF reconstruction achieve superior functional outcomes compared to primary closure [16, 17], as the free flap restores lost tissue bulk to the neo‐tongue, aids in food bolus mobility during swallowing, and increases palate contact for speech production [18]. However, choice of reconstructive technique is highly nuanced and depends on the tumor characteristics and extent, surgeon's scope of practice, patient preference, patient overall health status, and ability to tolerate the morbidity of a free flap procedure.

LLMs are increasingly integrated into workflows at major healthcare insurers, with goals of optimizing efficiency, reducing overhead costs, expediting decisions, and reducing unnecessary healthcare expenditures [19]. If integrated with deliberate ethical intent, AI for prior authorization may expedite clinical care and reduce the need for human capital for administrative tasks. For example, insurers have reported decreased decision times using AI systems, reducing decision times from several days to less than 24 h [20]. One insurer has deployed an AI tool to approve or deny post‐acute care in patients insured by Medicare Advantage plans [21]. Similarly, the widespread use of AI in processing prior authorization for oncologic surgery may become operationalized within insurance companies in the near future.

However, expediting prior authorization with AI with failure to recognize its limitations may come at the cost of adversely affecting patient outcomes through erroneous algorithms, generalized decision‐making, or inherent biases such as the findings revealed by the present study. Given the large potential for adverse patient outcomes with AI in prior authorization, several solutions can be considered to reduce patient harm and mitigate the exacerbation of existing health disparities. Our findings highlight the importance of including pertinent tumor‐level information and anticipated outcomes with surgical expertise (“The reconstructive surgeon specifically states that radial forearm free flap reconstruction is the preferred option because it is expected to provide better functional outcomes for this defect”). Prompts lacking this information resulted in significant biases, whereas inclusion of this data led to dramatically higher recommendations for RFFF reconstruction. In addition to judicious entry of prompts to LLMs, federal and state legislation can provide regulatory governance for ethical AI in healthcare. In 2024, a ruling by the U.S. Department of Health and Human Services (Section 1557, Patient Protection and Affordable Care Act) emphasized the need for protection from AI bias in healthcare, stating that “the artificial intelligence, algorithm, or other software tool shall not deny, delay, or modify health care services based, in whole or in part, on medical necessity.” [22] Furthermore, regulation to require insurers using AI to disclose their algorithms, development process, and pretraining data may encourage transparency and facilitate the identification of potential sources of bias [23].

Beyond legislation, another measure to limit AI bias in healthcare decision‐making is the use of curated pretraining data [24]. GPT‐4o is a LLM pretrained on both public and proprietary data from the Internet up until March 2023 [25]. Some pretraining data was filtered based on safety classifiers and other protocols in the effort to reduce bias. After training, one hundred experts from diverse backgrounds performed four phases of “red teaming” to identify risks of the model, followed by corrective strategies such as blocking certain outputs and post‐training. GPT‐4o has demonstrated improvement over previous models such as GPT‐3.5 Turbo and GPT‐4 in metrics such as accuracy in answering simple reasoning questions, truthfulness of responses, and reading comprehension [25]. These deliberate efforts to reduce bias may have contributed to both implicit decisions and explicit statements by the LLM in this study that SES, race, and sex should not influence clinical decision‐making, whereas other statements acknowledged potential impacts of these sociodemographic factors on reconstructive choice.

Despite these efforts to reduce bias, GPT‐4o is pretrained on black box data may contribute to its inherent biases. An alternative consideration is to develop a patient‐centric LLM trained on existing electronic medical record data, which may reflect real‐life scenarios with physician and patient input. While this approach does not completely eradicate bias, future studies may validate whether this approach can reduce bias with regard to sociodemographic characteristics. While these LLM biases are systematically scrutinized and addressed, requiring human oversight and expertise with human‐in‐the‐loop mechanisms to verify AI decisions can provide an additional layer of protection against bias and errors [26]. California State Bill 1120 imposed requirements establishing that physicians must provide oversight over AI tools [27]. Similar laws regulating the use of AI in health insurance are in effect in Arkansas, Connecticut, Illinois, Indiana, and Rhode Island [28]. As AI tools emerge in real‐world settings, caution should be taken to ensure that LLMs are used to augment rather than replace human clinical decision‐making, at least until its biases are addressed. Furthermore, once implemented, AI tools should be continuously monitored to ensure its functionality and prioritization of equity remain intact, as algorithmic drift are known to occur after deployment [29].

4.1. Limitations

Although the strengths of this study lie in the experimental environment of GPT‐4o, novelty of study design, ability to control for distinct demographic variables, and sensitivity analysis, this study is not without limitations. This study represents a simulation that did not use a model trained specifically as a medical insurance agent, which may limit external validity. The simulations include generated patient data and may not represent true LLM‐based insurance decision‐making model outputs. Without an in‐depth understanding of GPT‐4o's development process, the study is limited in its ability to pinpoint the sources of the biases identified. The extent to which other factors, such as comorbid conditions, prior admissions, other sociodemographic variables, as well as the intersection of multidimensional variables, impact bias were outside the scope of this study.

5. Conclusions

In this experimental study, an LLM demonstrated significant inherent bias when deciding health insurance approval for patients with head and neck cancer undergoing tongue cancer reconstruction. Patients who were younger, white or Asian, from higher SES backgrounds, or higher‐income zip codes were preferentially approved for RFFF, whereas patients who were older, Hispanic or Black, from lower SES backgrounds, or lower‐income zip codes were more likely to be denied for RFFF and recommended for STSG reconstruction instead. Inclusion of tumor‐specific information heavily skewed results towards RFFF reconstruction regardless of sociodemographic background. Inputs to LLMs for clinical decision‐making should include pertinent and detailed information to reduce the risk of bias. As health insurers increasingly integrate AI as part of routine workflows for prior authorization, recognition of its biases and regulatory governance are needed to protect from patient harm and promote equitable health care.

Author Contributions

Shannon S. Wu: data analysis, data visualization, manuscript writing. Mugil V. Shanmugam: data analysis, data visualization, manuscript editing. Yu‐Jin Lee: manuscript editing. Noel F. Ayoub: study design, data collection, data visualization, manuscript editing.

Funding

The authors have nothing to report.

Ethics Statement

The authors have nothing to report.

Conflicts of Interest

The authors declare no conflicts of interest.

Supporting information

Supporting File

Data Availability Statement

Data is available upon reasonable request and is subject to a formal data use agreement.