Impact of AI-Augmented informed consent on preoperative anxiety and patient understanding: A prospective Observational Study

Abstract

Objective

This study evaluated the clinical integration of a large language model (LLM) as a supportive communication tool within the preoperative informed consent pathway.

Methods

Participants completed the State-Trait Anxiety Inventory (STAI)-1, STAI-2, and Amsterdam Preoperative Anxiety and Information Scale (APAIS) scales after undergoing both the standard informed consent process and the subsequent ChatGPT consultation. Additionally, they completed a five-item questionnaire to assess their experience with the digital consultation. Anxiety scores were statistically compared before and after the AI consultation. Patient satisfaction regarding the digital consultation process was analyzed. Additionally, ChatGPT’s responses to the eleven most frequently asked questions were evaluated by three independent urologists in terms of accuracy, comprehensibility, and appropriateness.

Results

Anxiety scores decreased significantly. STAI-1 decreased by 3.2 points and STAI-2 by 1.3 points (both p<0.001). ROC analysis identified minimal clinically significant difference (MCID) thresholds of 1.5 for STAI-1 and 0.5 for STAI-2, indicating that these reductions were clinically significant. In contrast, APAIS scores decreased by 0.89 points (p < 0.001), but the ROC analysis was not significant (AUC = 0.559, p = 0.327), suggesting limited clinical relevance. Subgroup analyses showed that reductions in STAI scores generally exceeded the MCID thresholds, whereas APAIS changes did not consistently achieve clinical meaningfulness. Most participants reported that the ChatGPT consultation contributed to their understanding of the surgery (68%).

Conclusion

LLM-assisted consultation may serve as a scalable adjunct to clinician-led consent by enhancing perioperative communication and reducing situational anxiety, while requiring continued clinical oversight for safe implementation.

Introduction

As the first quarter of the 21st century concludes, an artificial intelligence (AI) revolution has significantly impacted various sectors, including healthcare. The main applications of AI software in healthcare include tumor detection in imaging systems, clinical decision-making support systems, and the patient information-consent process supported by chatbots.^1–3

Anxiety is one of the most prominent concerns among patients scheduled for surgery. Insufficient preoperative information regarding the upcoming medical procedure and anesthesia has been shown to contribute to elevated levels of preoperative anxiety.^4,5 Increased preoperative anxiety is associated with adverse outcomes, including heightened postoperative analgesic requirements, an elevated risk of delirium, and prolonged recovery periods. ⁶ Accordingly, the management of preoperative anxiety in surgical patients is essential, with adequate patient education representing a key component of this process.

Previous studies have indicated that patients scheduled for medical procedures increasingly turn to online resources, alongside conventional physician consultations, to understand their health status and the planned interventions. ⁷ However, the prevalence of misinformation online can hinder access to accurate knowledge and may even lead to patient confusion. This constitutes a contributing factor to heightened preoperative anxiety among patients. In this context, the large language model (LLM) developed by OpenAI can comprehend everyday questions, extract information from extensive databases, and convey it in reassuring human-like language. ⁸ This capability may contribute to a growing tendency among patients to rely on AI-driven conversational agents for pre-procedural medical information. However, based on the available literature, there are limited data to date that have examined the association between AI-based chatbots and patients’ preoperative anxiety.

Although LLMs possess the capacity to transfer by simplifying complex information, the accuracy of their outputs should not always be regarded as reliable. ⁹ This can be attributed to the following rationale: LLMs are developed by processing vast datasets from various online digital sources, which may include inaccurate information. Furthermore, as stated by OpenAI, this data processing occurs independently of verifying the information’s accuracy. ¹⁰ In light of this, it is critically important to assess the reliability of the information provided by LLMs before they are routinely employed for preoperative patient education, whether by individuals or healthcare professionals.

In this study, we aimed to examine the contributions of the LLM-assisted consultation to the preoperative informed consent process and associations with patients’ preoperative anxiety levels, understanding of the planned surgery, and overall satisfaction. To the best of our knowledge, this is the first study in the literature to investigate the association of the LLM-assisted consultation with preoperative anxiety in urology. Furthermore, by analyzing the accuracy of LLMs’ information, we aim to elucidate the potential role of LLMs in the informed consent process for surgical patients.

Materials and methods

This was a single-center prospective observational study conducted at the Department of Urology, Zonguldak Bülent Ecevit University. This study was conducted between 01.02.2025 and 01.05.2025 in our clinic among patients scheduled to undergo percutaneous nephrolithotomy (PNL), retrograde intrarenal surgery (RIRS), ureterorenoscopy (URS), transurethral resection of the prostate (TUR-P), bipolar enucleation of the prostate (BipolEP), transurethral resection of bladder tumor (TUR-M), radical nephrectomy, radical prostatectomy, inguinal orchiectomy, cystoscopy, intravesical botox injection, and internal urethrotomy. Patients undergoing the selected urological surgical procedures were screened for eligibility. Those who agreed to participate and completed the study questionnaires were enrolled. Individuals who declined participation or were unable to provide reliable questionnaire responses were excluded.

For this study, Turkish-validated versions of the State-Trait Anxiety Inventory (STAI)-1 for assessing state anxiety, STAI-2 for evaluating trait anxiety, and Amsterdam Preoperative Anxiety and Information Scale (APAIS) for measuring preoperative anxiety were employed.^11–14 In the first phase, patients scheduled for surgery underwent the standard informed consent process through a conventional physician consultation, after which they completed the STAI-1, STAI-2, and APAIS scales. In the second phase, they engaged in a conversation with the ChatGPT-4o AI chatbot, accessed via a dedicated ChatGPT Plus subscription account.

The chatbot interaction was intentionally designed to reflect real-world patient use of publicly available AI-based conversational agents following a routine physician consultation. Patients were instructed only to describe their medical condition and the surgical procedure recommended by their physician. No information regarding the treating institution, clinic, physician, or study objectives was provided to the chatbot. Participants were free to ask questions concerning their disease, treatment options, indications for surgery, expected benefits, potential complications, postoperative recovery, or any other issue they considered relevant. No predefined prompts, standardized question lists, custom instructions, or procedure-specific templates were used. Likewise, the researchers did not provide any pre-loaded contextual information or prompt engineering to the chatbot. The supervising clinician did not interfere with, guide, or modify the conversations and had no influence on the content of the patients’ questions. This approach was chosen to simulate the increasingly common practice of patients seeking additional information from AI-based tools after physician consultations to better understand their condition, evaluate treatment options, and explore the potential benefits and risks of proposed interventions.

After this interaction, patients were asked to complete the STAI-1, STAI-2, and APAIS scales again. The association between LLM-assisted consultation and patients’ preoperative anxiety was analyzed by comparing STAI-1, STAI-2, and APAIS scores obtained before and after the chatbot interaction. Subsequently, a five-item satisfaction questionnaire was administered using a 5-point Likert scale to evaluate patients’ experiences with the AI chatbot interaction (Figure 1). Responses 4 (agree) and 5 (strongly agree) were considered positive answers to the corresponding statement. To assess whether the difference in anxiety scores between groups was clinically significant, the minimal clinically important difference (MCID) scores were calculated using Receiver Operating Characteristic (ROC) analysis. Clinical success was determined by patients’ responses to the satisfaction questionnaire’s fifth question, which assessed overall satisfaction. An anchor-based ROC approach was selected because the primary objective was to identify a clinically meaningful threshold linked to patients’ perceived benefit rather than one based solely on statistical distribution.

Figure 1.

The questionnaire used to evaluate patient satisfaction.

The surgical procedures were classified into four subgroups: stone disease surgery (RIRS, PNL, URS), prostatic obstruction surgery (TUR-P, Bipolep), oncological surgery (TUR-BT, radical nephrectomy, radical prostatectomy, inguinal orchiectomy), and others (cystoscopy, botox injection, internal urethrotomy). The survey responses were analyzed according to these subgroups to assess whether the type of surgery influenced the outcomes.

In the final stage, three urologists independently assessed the answers to the most frequently asked questions within the subgroup of patients scheduled to undergo the same surgical procedure for accuracy, comprehensibility, and appropriateness. For the evaluation, the urologists used a scale of 1 to 5, with 1 indicating very poor and 5 indicating very good. In cases of disagreement, the two most concordant responses among the three were considered valid. The Fleiss’ kappa coefficient was calculated to evaluate the level of agreement among the urologists’ assessments. This analysis included 11 questions. The questions and their answers are presented in Table 4.

Table 4.

The most frequently asked questions, their answers, and the evaluations of the answers by urologists.

Questi̇ons	Chatgpt's answers	Evaluati̇on of the answers
Will a stent be placed after the kidney stone surgery? Why is it used?	Yes, after your kidney stone surgery, a temporary stent may be placed from your kidney to your bladder. The stent is used to maintain urine flow after surgery, prevent blockage from stone fragments, protect the kidney from pressure, and reduce pain The stent typically remains in place for 1 to 3 weeks and is later removed painlessly in an outpatient setting.	ACCURACY	5
		COMPREHENSİBİLİTY	5
		APPROPRİATENESS	4
		Question frequency: (%)	68.9
Will kidney stones come back after the kidney stone surgery?	Yes, kidney stones can come back even after successful surgery. Surgery removes the existing stones, but it does not eliminate the underlying factors that may cause stones to form. To reduce the risk of recurrence, it’s important to: Stay well hydrated by drinking plenty of water throughout the day. Follow dietary recommendations based on the type of stone you had (e.g., reducing salt or oxalate intake). Monitor and manage medical conditions such as hyperparathyroidism or metabolic disorders if present. Attend regular follow-up appointments and imaging to check for early signs of new stones.	ACCURACY	5
		COMPREHENSİBİLİTY	5
		APPROPRİATENESS	4
		Question frequency: (%)	55.1
What will be done during the surgery? (The patient is asking about prostate obstruction surgery.)	During the surgery, the part of your prostate that is blocking the flow of urine will be removed. This is usually done using a special instrument inserted through the urethra, so there is no need for an external incision.Depending on the type of surgery (such as TURP or HoLEP):The surgeon will remove the overgrown prostate tissue that is causing the blockage.This helps to open up the urinary channel, allowing urine to flow more freely from the bladder.A catheter will be placed in your bladder at the end of the procedure to help drain urine while the area heals.The surgery is typically performed under spinal or general anesthesia and usually takes between 30 and 90 minutes.	ACCURACY	5
		COMPREHENSİBİLİTY	4
		APPROPRİATENESS	5
		Question frequency: (%)	81
Will a urinary catheter be placed ? (The patient is asking about prostate obstruction surgery.)	Yes, after surgery for prostatic obstruction (such as TURP, HoLEP, or other procedures), a urinary catheter is typically placed. The catheter is used to allow urine to drain while the operated area heals, prevent urinary retention due to temporary swelling or clot formation, monitor bleeding or flush out any blood clots with irrigation if needed. The catheter usually remains in place for 1 to 3 days, depending on the type of surgery performed and your recovery status. Your healthcare team will remove it when they determine it’s safe to urinate on your own.	ACCURACY	5
		COMPREHENSİBİLİTY	5
		APPROPRİATENESS	5
		Question frequency: (%)	70.2
Will I be completely asleep during the surgery (The patient is asking about prostate obstruction surgery.)	Yes, during the surgery, you will be completely asleep under general anesthesia or you may receive spinal anesthesia, which numbs the lower half of your body while you remain awake but sedated. The choice of anesthesia depends on: The type of surgery, Your overall health condition, The anesthesiologist’s and surgeon’s recommendation. In either case, you will not feel pain during the procedure.	ACCURACY	4
		COMPREHENSİBİLİTY	4
		APPROPRİATENESS	5
		Question frequency: (%)	67.5
Will I be able to urinate comfortably after the surgery? (The patient is asking about prostate obstruction surgery.)	Yes, one of the main goals of prostate surgery for benign prostatic obstruction (such as TURP or HoLEP) is to relieve urinary symptoms and improve urine flow. Most patients report significant improvement in their ability to urinate after recovery. However, in the early period following surgery, you may temporarily experience:Mild burning or discomfort while urinating.Frequent urination or an urgent need to go.A small amount of blood in the urine These symptoms usually resolve within a few days to weeks as the surgical area heals. If healing progresses well, you should notice a stronger, more continuous urine stream and less straining during urination compared to before the surgery.	ACCURACY	5
		COMPREHENSİBİLİTY	5
		APPROPRİATENESS	5
		Question frequency: (%)	67.5
My doctor said there might be a tumor in my bladder. Is the tumor malignant? (The patient is asking about TUR-BT.)	At this stage, we cannot say for sure whether the tumor is malignant (cancerous) or benign (non-cancerous). The purpose of the TUR-BT (Transurethral Resection of Bladder Tumor) surgery is both to remove the tumor and to obtain tissue samples for detailed pathological examination.After the surgery:The removed tissue will be sent to a laboratory. Pathologist will examine the sample under a microscope to determine whether it is cancerous, and if so, how aggressive it is.The results usually become available within 7–10 days.These results will guide the next steps in your treatment, if needed. Please remember that not all bladder tumors are cancerous, and even if cancer is present, many cases can be effectively treated, especially when detected early.	ACCURACY	4
		COMPREHENSİBİLİTY	5
		APPROPRİATENESS	4
		Question frequency: (%)	57.1
Will the tumor be removed entirely with the surgery? (A patient scheduled for radical nephrectomy due to renal cell carcinoma)	Yes, the purpose of radical nephrectomy is to completely remove the kidney that contains the tumor, along with the surrounding fatty tissue and sometimes nearby lymph nodes or adrenal gland if necessary. In most cases of localized renal cell carcinoma (RCC):The entire tumor can be successfully removed with surgery.If the cancer has not spread beyond the kidney, radical nephrectomy is often considered curative.After the surgery, the removed tissue will be examined by a pathologist to assess whether the surgical margins are clear and if there are any signs of spread.However, if the cancer is more advanced or has invaded nearby structures or blood vessels, additional treatments or close follow-up may be necessary.	ACCURACY	5
		COMPREHENSİBİLİTY	5
		APPROPRİATENESS	5
		Question frequency: (%)	80
Can this cancer come back? (A patient scheduled for radical prostatectomy due to prostate cancer)	Radical prostatectomy (RRP) is performed to completely remove the prostate gland and, if necessary, surrounding tissues affected by cancer. The goal is to eliminate all cancer cells confined to the prostate. However, as with many cancers:There is still a chance of recurrence, especially if cancer cells have already spread outside the prostate before surgery.The risk depends on several factors, such as the Gleason score, tumor stage, surgical margins, and PSA levelsafter surgery.Some patients may require additional treatment after surgery (such as radiation therapy or hormonal therapy) if there are signs of residual or recurrent cancer. After surgery, you will be closely monitored with regular PSA blood tests and follow-up visits. A rising PSA level after surgery can be an early sign of recurrence. While recurrence is possible, many men live long, cancer-free lives after successful surgery—especially when the disease is caught early and fully removed.	ACCURACY	5
		COMPREHENSİBİLİTY	4
		APPROPRİATENESS	5
		Question frequency: (%)	50
I’ve been told I have cancer in my testicle, and they want to take me to surgery immediately. Is surgery for testicular cancer done right away?	Yes, in most cases, surgery for testicular cancer (called radical inguinal orchiectomy) is performed very soon after diagnosis, often within a few days. This is because: Testicular tumors can grow and spread quickly, especially non-seminomatous germ cell tumors. The surgery is both diagnostic and therapeutic — it removes the tumor and allows for precise histopathological classification. Early surgery improves the chance of cure and reduces the risk of metastasis to lymph nodes or other organs such as the lungs or liver. While it is not considered an “emergency” in most cases, it is treated as an urgent or high-priority procedure that should not be delayed unnecessarily.	ACCURACY	5
		COMPREHENSİBİLİTY	5
		APPROPRİATENESS	5
		Question frequency: (%)	100
I had prostate surgery nine months ago, but now I’m experiencing urinary difficulty again. How is this possible? Will this be a permanent condition?	The fact that you were unable to urinate after the first surgery and required a second operation suggests a complication such as re-stenosis (new narrowing), bladder neck contracture, or complete obstruction. Your inability to urinate again may be due to: Recurrent obstruction in the urethra or bladder neck, bladder muscle weakness (detrusor underactivity), or urethral stricture (narrowing of the urinary passage). This does not necessarily have to be a permanent condition. If there is a new blockage, it can usually be corrected. If the bladder muscle has become weak, it can still be managed effectively with appropriate strategies. I strongly recommend seeing a urologist again for a comprehensive evaluation. With the correct diagnosis, a suitable and effective treatment plan can be made.	ACCURACY	5
		COMPREHENSİBİLİTY	5
		APPROPRİATENESS	5
		Question frequency: (%)	50

The frequency of the questions in the table was calculated proportionately to the number of patients scheduled to undergo the same surgical procedure.

Privacy and data security

All chatbot–patient interactions were conducted in a counseling room within the urology clinic under the supervision of an authorized clinician. Neither patients nor researchers used personal ChatGPT accounts. Instead, an account created solely for this study was used, with both chat history and training features disabled. The chatbot was accessed via the ChatGPT website (https://chatgpt.com) using the GPT-4o model through a ChatGPT Plus subscription on a hospital computer. Patients were verbally instructed not to enter any identifiable personal information, such as names, dates of birth, or identification numbers. The supervising clinician ensured adherence to these instructions. To prevent the potential effect of the previous question on others, each question was initiated in a separate chat session. There was no predetermined time limit for the chatbot interaction. Patients were free to end the consultation whenever they felt their questions had been sufficiently addressed. In all cases, the interaction did not exceed approximately 15 minutes. All interactions were logged anonymously and stored in de-identified form solely for research purposes.

Ethics and consent to participate declarations

The local clinical research ethics committee at Zonguldak Bulent Ecevit University reviewed and approved the study protocol before initiation (Approval No: 2025/02-5). The protocol complied with the tenets of the Declaration of Helsinki. Before the commencement of the study, all participants were provided with an informed consent form and thoroughly informed about the study’s objectives and procedures. Verbal and written informed consent were subsequently obtained for participation in the study.

Statistical analysis

Continuous variables were presented as mean ± standard deviation (SD) or median and (min-max), while categorical variables such as responses to the satisfaction survey were expressed as frequencies and percentages. The normality of continuous variables was assessed using the Shapiro–Wilk test. Paired samples were compared using the paired t-test for normally distributed data and the Wilcoxon signed-rank test for non-normally distributed data to evaluate pre- and post-chatbot intervention differences in STAI-1, STAI-2, and APAIS scores. Effect sizes for pre–post comparisons were calculated using Cohen’s d for paired samples, and interpreted as trivial (<0.2), small (0.2–0.5), moderate (0.5–0.8), or large (>0.8). The internal consistency of the five-item satisfaction questionnaire was evaluated using Cronbach’s alpha coefficient. P<0.05 was considered to indicate a statistically significant result. SPSS software (IBM SPSS Statistics for Windows, version 25.0; IBM Corp) was used for the analyses. The Fleiss’ kappa coefficient calculations were performed in Python using the statsmodels library. No formal a priori sample size calculation was performed. However, based on a paired-samples design, a two-tailed alpha level of 0.05, 80% statistical power, and a conservative small-to-moderate effect size (d = 0.30), approximately 90 participants would be required for the primary analysis. Therefore, the final sample of 100 patients was considered adequate for the study objectives. Subgroup analyses were exploratory and were not specifically powered.

This study was reported in accordance with the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines.

Results

One hundred patients were enrolled in the study. The median age of the study participants was 65 years (range: 18–83), and the majority were male (n=82, 82%). Regarding educational status, 40% (n=40) had completed primary school or less, 35% (n=35) were high school graduates, and 25% (n=25) held a university degree. The distribution of planned surgical procedures was as follows: RIRS, eight patients (8%); PNL, six (6%); URS, fifteen (15%); TUR-P/BipolEP, thirty seven (37%); TUR-BT, seven (7%); radical nephrectomy, five (5%); radical prostatectomy, eight (8%); inguinal orchiectomy, three (3%); cystoscopy, seven (7%); botox injection, two (2%); and internal urethrotomy, two (2%) (Table 1).

Table 1.

The demographics and clinical characteristics of patients.

Characteristic	header
Age (median, min&max; years)	65 (18-83)
Sex (n, %)
Male	82 (82%)
Female	18 (18%)
Educational Level (n, %)
Primary School or lower	40 (40%)
High School	35 (35%)
University	25 (25%)
Operation Category (n, %)
Stone Disease	29 (29%)
RIRS	8 (8%)
PNL	6 (6%)
URS	15 (15%)
Prostatic Obstruction	37 (37%)
TUR-P/BipolEP	37 (37%)
Oncology	23 (23%)
TUR-BT	7 (7%)
Radical Nephrectomy	5 (5%)
Radical Prostatectomy	8 (8%)
İnguinal Orchiectomy	3 (3%)
Others	11 (11%)
Cystoscopy	7 (7%)
Botox injection	2 (2%)
Internal urethrotomy	2 (2%)

min: Minimum, max: Maximum. RIRS: Retrograde Intrarenal Surgery.

PNL: Percutaneous nephrolithotomy, TUR-P: Transuretral Resection- Prostate.

URS: Ureterorenoscopy, BipolEP: Bipolar enucleation of the prostate.

TUR-M: Transuretral Resection-Bladder tumor.

Following consultation with the AI chatbot, the scores of all three anxiety scales (STAI-1, STAI-2, and APAIS) were significantly reduced in the entire cohort (36.7 (10.4)- 33.5 (9.8)/p<0.001, 43.0 (8,4)- 41.7 (8.1))/p<0.001, and 14.0 (5.9)- 13.1 (4.9)/p<0.001, respectively). Effect size analysis revealed a moderate effect for STAI-1 (Cohen’s d = −0.60), and small effects for STAI-2 (d = −0.39) and APAIS (d = −0.43). For the STAI-1, the area under the ROC curve (AUC) was 0.870 (95% CI: 0.791–0.948, p < 0.001), and the MCID was 1.5 (sensitivity: 81%, specificity: 81%). For the STAI-2, the AUC was 0.730 (95% CI: 0.632–0.828, p<0.001), and the MCID was 0.5 (sensitivity: 65%, specificity: 84%). Given these values, the observed reductions in STAI-1 and STAI-2 exceeded the clinically significant threshold. However, ROC analysis for APAIS did not reach statistical significance (AUC = 0.559, p = 0.327), indicating that this change may not be clinically meaningful. Among the 29 patients who underwent stone surgery, the APAIS score remained unchanged (12.5 (4.8)- 12.0 (3.9)/p=0.212) following AI chatbot consultation, whereas both STAI scores showed a significant decrease (34.5 (8.3)- 32.7 (8.7)/p=0.005, 42.8 (10.1)- 41.5 (9.5)/p=0.007 respectively). Effect sizes were moderate for both STAI-1 (d = −0.57) and STAI-2 (d = −0.54). Both STAI reductions met the ROC-derived MCID thresholds, supporting their clinical relevance. All three scores (STAI-1, STAI-2, and APAIS) significantly decreased following AI chatbot consultation in patients who underwent surgery for prostatic obstruction (n=37), (36.5 (11.7)- 33.1 (11.3)/p=0.001, 42.9 (8.4)- 41.4 (7.8)/p= 0.049, 13.7 (5.4)- 12.8 (4.4)/p=0.007, respectively). STAI-1 decreased by 3.37 points (95% CI: 1.50–5.23, p=0.001) and STAI-2 by 1.43 points (95% CI: 0.08–2.85, p=0.049). Effect sizes were moderate for STAI-1 (d = −0.61) and small for STAI-2 (d = −0.34) and APAIS (d = −0.47). Both reductions exceeded the MCID thresholds and were therefore clinically significant. However, APAIS scores decreased by 0.94 points (95% CI: 0.27–1.61, p=0.007), but given the non-significant ROC result for APAIS, the clinical relevance remains uncertain. Among the 23 patients who underwent oncological surgery, a significant reduction was observed in two of the three scores following AI chatbot consultation, except for the STAI-2 score, which reflects trait anxiety (37.5 (10.7)- 32.6 (9.1)/p=0.004, 43.6 (7.7)- 42.7 (7.9)/p=0.195, 15.5 (7.3)- 14.1 (6.6)/p=0.016, respectively). Effect sizes were moderate for STAI-1 (d = −0.68) and APAIS (d = −0.54), and small for STAI-2 (d = −0.28). STAI-1 decreased by 4.9 points (95% CI: 1.8–8.04, p=0.004), a reduction above the MCID threshold and thus clinically relevant. In the patient group that underwent other types of surgery, a significant reduction was observed in the STAI-1 and STAI-2 score, which measures state and trait anxiety (41.0 (9.4)- 38.5 (8.1)/p=0.005, 42.7 (6.3)- 41.1 (5.7)/p= 0.020, 16.1 (6.0)- 15.1 (4.8)/p=0.184, respectively). Effect sizes were large for both STAI-1 (d = −1.07) and STAI-2 (d = −0.83), though this subgroup comprised only 11 patients and should be interpreted with caution. Both reductions exceeded the MCID thresholds and were clinically relevant (Table 2).

Table 2.

Changes in Preoperative Anxiety Scores Before and After ChatGPT-Based Consultation Across the entire cohort and Surgical Subgroups.

​	Before contact with chatGPT.	After contact with chatGPT.	Mean difference	Confidence interval		P-value
				Lower	Upper
ENTİRE COHORT (n=100)
The STAI – 1 Score (mean, sd)	36.7 (10.4)	33.5 (9.8)	3.2	2.12	4.21	<0.001
The STAI – 2 Score (mean, sd)	43.0 (8,4)	41.7 (8.1)	1.3	0.63	1.94	<0.001
The APAIS Score (mean, sd)	14.0 (5.9)	13.1 (4.9)	0.89	0.47	1.30	<0.001
PATİENTS UNDERGOİNG STONE SURGERY (n=29)
The STAI – 1 Score (mean, sd)	34.5 (8.3)	32.7 (8.7)	1.8	0.60	3.00	0.005
The STAI – 2 Score (mean, sd)	42.8 (10.1)	41.5 (9.5)	1.3	0.45	2.22	0.007
The APAIS Score (mean, sd)	12.5 (4.8)	12.1 (3.9)	0.41	-0.24	1.07	0.212
PATİENTS UNDERGOİNG PROSTATİC OBSTRUCTİON SURGERY (n=37)
The STAI – 1 Score (mean, sd)	36.5 (11.7)	33.1 (11.3)	3.37	1.50	5.23	0.001
The STAI – 2 Score (mean, sd)	42.9 (8.4)	41.5 (7.8)	1.43	0.08	2.85	0.049
The APAIS Score (mean, sd)	13.7 (5.4)	12.8 (4.4)	0.94	0.27	1.61	0.007
PATİENTS UNDERGOİNG ONCOLOGİCAL SURGERY (n=23)
The STAI – 1 Score (mean, sd)	37.5 (10.7)	32.6 (9.1)	4.9	1.8	8.04	0.004
The STAI – 2 Score (mean, sd)	43.6 (7.7)	42.7 (7.9)	0.86	-0.48	2.22	0.195
The APAIS Score (mean, sd)	15.5 (7.3)	14.1 (6.6)	1.34	0.27	2.41	0.016
OTHER PATİENTS (n=11)
The STAI – 1 Score (mean, sd)	41.0 (9.4)	38.5 (8.1)	2.45	0.91	3.9	0.005
The STAI – 2 Score (mean, sd)	42.7 (6.3)	41.1 (5.7)	1.63	0.31	2.95	0.020
The APAIS Score (mean, sd)	16.1 (6.0)	15.1 (4.8)	1.0	-0.56	2.56	0.184

The STAI: Trait Anxiety Inventory, The APAIS: The Amsterdam Preoperative Anxiety and Information Scale.

Exploratory subgroup analyses by educational attainment showed no significant differences in STAI-1 (p = 0.639) or APAIS (p = 0.367) change scores. However, a significant difference was observed for STAI-2 change scores (p = 0.035). Post hoc analysis demonstrated a significant difference between the high school and university groups (p = 0.040), reflecting a smaller reduction in STAI-2 scores among participants with a high school education (Supplementary table).

The satisfaction questionnaire analysis indicates that none of the patients selected the lowest response (strongly disagree) to any question. A total of 68% of the patients (n=68) reported that the consultation with the AI chatbot helped enhance their understanding of the planned surgery, with 47% (n=47) agreeing and 21% (n=21) strongly agreeing (mean 3.87, SD=0.76/median 4). Similarly, 66% reported that ChatGPT provided clear and understandable answers (mean = 3.84, SD = 0.76/median 4), while 62% felt more informed in addition to their physician consultation (mean = 3.62, SD = 0.74/median 4). The fourth question, whether the LLM-assisted consultation was easy and comfortable, received the lowest number of positive patient responses (55%, mean 3.65, SD=0.74/median 3). The overall satisfaction rate for the LLM-assisted consultation was 64%. The satisfaction questionnaire demonstrated excellent internal consistency, with a Cronbach’s alpha coefficient of 0.94 (Table 3).

Table 3.

Analysis of participants’ responses to the satisfaction questionnaire.

Statement	Rating (1–5)					Mean (SD)	Median (min-max)
	1	2	3	4	5
1. The conversation with ChatGPT helped me better understand my upcoming surgery.	0 (0%)	2 (2%)	30 (30%)	47 (47%)	21 (21%)	3.87 (0.76)	4 (2-5)
2. ChatGPT provided clear and understandable answers to my questions.	0 (0%)	2 (2%)	32 (32%)	46 (46%)	20 (20%)	3.84 (0.76)	4 (2-5)
3. The ChatGPT interaction contributed to my feeling more infored in addition to the doctor’s consultation.	0 (0%)	1 (1%)	35 (35%)	48 (48%)	16 (16%)	3.79 (0.71)	4 (2-5)
4. It was easy and comfortable for me to communicate with ChatGPT.	0 (0%)	3 (3%)	42 (42%)	40 (40%)	15 (15%)	3.65 (0.74)	3 (2-5)
5. Overall, I was satisfied with the information provided through the ChatGPT consultation.	0 (0%)	4 (4%)	34 (34%)	43 (43%)	19 (19%)	3.62 (0.74)	4 (2-5)

In evaluating the responses to the 11 most frequently asked questions for accuracy, 9 of the 11 questions received the highest score of 5, while the remaining 2 received a score of 4. The Fleiss’ kappa score for accuracy is 0.649, indicating a moderate-to-good level of inter-rater agreement. About comprehensibility and appropriateness, 8 of the 11 questions received a score of 5, and 3 received a score of 4. The Fleiss’ kappa scores are 0.570 and 0.694, reflecting moderate and good levels of agreement, respectively (Table 4).

Discussion

This study suggests an association whereby the LLM-assisted consultation during the informed consent process may help patients better understand the upcoming procedure and reduce preoperative anxiety. Effect size analysis supported these findings: STAI-1 demonstrated a moderate effect (Cohen’s d = −0.60) across the entire cohort, while STAI-2 and APAIS showed small effects (d = −0.39 and −0.43, respectively), suggesting that the association was most pronounced for state anxiety. Given that the chatbot interaction was a brief informational intervention delivered immediately before surgery, a greater influence on state anxiety than on trait anxiety would be expected. This interpretation is supported by previous literature indicating that educational and counseling interventions are generally more effective in reducing procedure-related situational anxiety than underlying anxiety traits. ¹⁵ Furthermore, most patients who participated in the study reported that ChatGPT provided clear and understandable responses, which offered additional benefits to clinician consultation in supporting a more effective patient information process. In the current study, a relatively lower proportion of patients found the LLM-assisted consultation easy and comfortable. This situation may be attributable to the fact that a substantial portion of the study population had an educational level of primary school or below. The overall satisfaction rate the patients reported was deemed acceptable and promising. Notably, exploratory subgroup analyses revealed that reductions in state anxiety (STAI-1) were similar across educational subgroups. This finding may suggest that educational attainment did not substantially influence patients’ ability to engage with information provided during the LLM-assisted consultation. Although a statistically significant difference in trait anxiety (STAI-2) change scores was observed between the high school and university subgroups, absolute changes were small across all groups, warranting cautious interpretation given the exploratory nature of the analysis and the limited statistical power for subgroup comparisons.

The STAI-1 scores in our study were consistent with findings from another study conducted in the Turkish population. ¹⁶ A further shared feature between the two studies was the use of APAIS, an additional tool for assessing preoperative anxiety. However, Aykent et al. did not identify a linear correlation between the APAIS and STAI-1 scales and proposed that APAIS may not be suitable for evaluating preoperative anxiety in patients. ¹⁶ Our study identified a significant reduction in state anxiety scores, as measured by the STAI-1 and STAI-2, following the LLM-assisted consultation, across the entire cohort and within almost all subgroup analyses (except for STAI-2 in oncological surgery). However, this finding did not hold for the APAIS scores. Notably, the APAIS scores in our study decreased following AI chatbot consultation among patients scheduled for prostate obstruction and oncological surgeries, as well as the entire cohort. However, no significant reduction was observed after the LLM-assisted consultation in patients undergoing stone surgery or in those undergoing other procedures, including cystoscopy, botox injection, and internal urethrotomy. Moreover, the ROC analysis result didn’t reach statistical significance, indicating that this change may not be clinically meaningful. It is plausible that these differences stem from the lack of a linear correlation between the STAI and APAIS scales, as noted in the previously mentioned study.

This study found that patients scheduled for oncological surgery exhibited clinically significant reductions in STAI-1 scores, which assess state anxiety, following the LLM-assisted consultation. However, no significant change was observed in STAI-2 scores, which measure trait anxiety. This observation is reasonable, as patients in this study primarily interacted with ChatGPT during the informed consent process regarding surgery and anesthesia. In contrast, patients undergoing oncological surgery may experience heightened concerns related to the need for additional postoperative treatments, cancer-associated health complications, and survival outcomes. This interpretation is further supported by existing literature reporting elevated levels of trait anxiety in oncological patients.^17,18 Accordingly, the lack of a significant reduction in STAI-2 scores following the LLM-assisted consultation is not unexpected. Nonetheless, AI chatbot consultation may still offer benefits for these patients, as it supports them in managing anxiety related to their upcoming surgical procedures.

Standardized informed consent forms developed by surgical clinics often fail to address patients’ individual concerns because they focus on addressing the most frequently asked questions.^19,20 Moreover, these documents may contain complex medical terminology difficult for patients to comprehend.^20,21 When physicians place excessive reliance on these forms and cannot devote sufficient attention to the informed consent process, it may hinder the development of an effective physician–patient relationship. ²² As a result, patients may perceive themselves as inadequately informed and experience heightened anxiety regarding the upcoming procedure.^4,5 In response, they often turn to online sources for more information. ⁷ However, with the internet and social media increasingly functioning as marketing platforms, the information found may disproportionately emphasize the positive outcomes of medical interventions while overlooking potential risks. Conversely, some sources may exaggerate negative aspects. This inconsistency in available information can create confusion and contribute to increased levels of preoperative anxiety.

At this point, patients must receive adequate information from physicians at the centers where they will undergo the procedure. However, some physicians may struggle to communicate the information they intend to convey to patients in a simplified and understandable manner.^20,22 For some patients, even engaging in such discussions with a clinician may increase anxiety, as exemplified by the white coat phenomenon. ²³ To resolve the issue, LLMs—capable of scanning vast datasets and responding in language tailored to patients’ comprehension levels—present a promising solution. In our study, most participants reported that the LLM-assisted consultation contributed to their understanding of the upcoming surgical procedure, provided additional information to the physician consultation, and delivered clear and comprehensible responses. An important aspect of the present study is that the chatbot interaction was intentionally left open-ended. Patients were not provided with predefined questions or standardized prompts and were free to discuss any issue related to their disease or planned procedure. This approach was chosen because the primary aim was not to evaluate a structured educational program, but rather to reflect how patients increasingly use publicly available AI tools after physician consultations to seek additional information and clarification. Although this may have introduced variability in the content of the interactions, it also mirrors routine real-world use and may partly explain why many participants perceived the consultation as helpful and informative. Moreover, expert clinical review concluded that most of ChatGPT’s responses were accurate, comprehensible, and appropriate.

Recent studies suggest that AI-based conversational agents are becoming increasingly integrated into healthcare communication and patient education, largely due to their ability to provide personalized, interactive, and easily accessible information ²⁴ Wenyi Gan et al. investigated the impact of the LLM-assisted consultation on anxiety in 55 patients scheduled for total knee arthroplasty. Consistent with our findings, they found lower anxiety scores and greater satisfaction with preoperative education in the LLM-assisted consultation group. Their study also incorporated an analysis of postoperative knee function and pain; however, no statistically significant differences were observed between the two groups in these outcomes. ²⁵ Similarly, Akdogan et al. investigated the effects of the LLM-assisted consultation on anxiety and depression in newly diagnosed cancer patients before chemotherapy. They found that the LLM-assisted consultation led to a reduction in both anxiety and depression before the initiation of chemotherapy. ²⁶ Based on these findings, it is suggested that LLM-based AI chatbots have the potential to enhance patient education within the informed consent process and thereby reduce preoperative anxiety.

Our findings are consistent with the existing literature, suggesting that LLM-assisted consultation may contribute to patients’ understanding of the planned surgical procedure, while the information provided by the LLM demonstrated high levels of accuracy, comprehensibility, and appropriateness. Furthermore, the anxiety-reducing effects observed in this study may not be attributable solely to the accuracy and comprehensiveness of the information provided. The characteristically reassuring, supportive, and empathetic communication style of LLMs may also have contributed to reduced anxiety, independent of informational content. Disentangling the effects of informational quality from those of communication style remains an important direction for future research. Additionally, patients’ engagement with and responses to LLM-assisted consultations may be shaped by their pre-existing attitudes toward AI, perceived usefulness, and trust in AI-based technologies. Recent evidence from Turkey suggests that these dimensions represent distinct constructs that may independently influence how individuals interact with AI in healthcare settings. ²⁷ Nevertheless, it has also been suggested that the quality and accuracy of the LLM-generated content may not be optimal and that it may contain statements that could lead to misunderstandings among patients. ⁹ The authors strongly recommend that any consultation between patients and AI chatbots be conducted under the supervision of a healthcare professional, preferably a specialist clinician.

A key strength of this study is that it provides early empirical evidence regarding the association between LLM-assisted consultation during the informed consent process and preoperative anxiety among patients undergoing urological procedures. Although the pre–post design minimizes interpersonal variability, the lack of an independent control group prevents us from fully excluding potential confounders, such as natural time-dependent reductions in anxiety, repeated-testing effects, or the influence of additional attention. Consequently, it remains uncertain whether the observed association was attributable to the LLM-assisted consultation itself or to the provision of supplementary information following the physician consultation. LLM performance evolves continuously through model updates. Therefore, the present findings reflect the characteristics of GPT-4o available via ChatGPT Plus during the study period and may not be directly generalizable to other model versions or platforms. Future studies should compare different LLM versions and platforms to assess the reproducibility of these findings. The principal limitation of our study is that the MCID values derived from the ROC analysis in our cohort were lower than the threshold values (a 5–8 point change) recommended in the literature. This finding suggests that the clinical effect size associated with the LLM-assisted consultation on anxiety may be limited. Accordingly, our results should be interpreted with caution, and further research with larger samples is warranted. Although educational subgroup analyses were exploratory given the sample sizes involved, the findings suggest that reductions in state anxiety were consistent across educational levels. Although a formal a priori sample size calculation was not performed before study initiation, the final sample size was considered adequate for the primary analyses. However, subgroup analyses should be interpreted with caution, as the study was not specifically powered for these comparisons. Future studies with larger samples should further examine the role of educational attainment as a potential moderator of LLM-assisted consultation outcomes.

Conclusions

This study provides evidence that the LLM-assisted consultation has potential benefits for the informed consent process. The LLM-assisted consultation may reduce patients’ preoperative anxiety and help them better understand the surgical procedure they are scheduled for. Despite the high accuracy, comprehensibility, and appropriateness of the information provided by LLM observed in this study, there is insufficient evidence to recommend its adoption in routine clinical practice.

ORCID iDs

Yavuz Mert Aydın https://orcid.org/0000-0002-6287-6767

Tevfik Erdem Özarslan https://orcid.org/0009-0004-4082-5709

Ethical considerations

The local clinical research ethics committee (Zonguldak Bulent Ecevit University) reviewed and approved the study protocol before initiation (Approval No: 2025/02-5).

Consent to participate

The protocol complied with the tenets of the Declaration of Helsinki. Before the commencement of the study, all participants were provided with an informed consent form and thoroughly informed about the study’s objectives and procedures. Verbal and written informed consent were subsequently obtained for participation in the study.

Data Availability Statement

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.*