Conversational artificial intelligence for pre-procedural patient preparation: implementation, validation and patient satisfaction

Abstract

Preparing patients for cardiac catheterization requires critical but repetitive tasks. We conducted a prospective evaluation of the AI voice assistant, Sofiya, for pre-procedural calls in our center. A customized Large Language Model was trained on medical knowledge and deployed on an agentic AI framework with conversational AI. Sofiya called patients to provide instructions, collect clinical data, and answer questions or redirect the patient to a nurse. The study consisted of a 90-day stabilization period with collaboration between clinicians and developers (Phase I), followed by a 90-day real-world application led by nurses (Phase II). The primary outcome was the overall rate of successfully completed calls reaching the end of the script with all patients answering all clinical questions. From January 16, 2025, to July 17, 2025, 1431 patients received 1606 calls from Sofiya. Phase I had 806 calls whose completion rate gradually improved, reaching an average of 86.4%. High completed call rate (87.9%) was maintained in Phase II with an additional 800 calls. AI system errors were observed in 48 (6.0%) calls during Phase I and 24 (2.6%) in Phase II. A voice-based AI assistant can successfully augment routine tasks of pre-procedural patient preparation, allowing nurses to focus on clinical aspects of patient care.

Introduction

Cardiac catheterization is an invasive procedure that is used to diagnose and treat patients with suspected or confirmed coronary artery disease through angiograms¹. With more than one million cardiac catheterizations performed annually, it is one of the most widely performed cardiac procedures in the U.S². Pre-procedural patient education and data collection are critical yet resource-intensive components of clinical care, particularly for procedures such as cardiac catheterization. Effective pre-procedural communication improves patient preparedness, reduces anxiety, enhances compliance with instructions, and contributes to better procedural outcomes, including higher success rates and fewer complications^3–5. However, traditional methods, notably phone calls conducted by clinical staff, are labor-intensive, subject to variability in information delivery, and limited by time constraints and staffing shortages⁶. In fact, health-care professionals are generally considered to be in one of the highest-risk groups for experiencing burnout due to the nature of their work^7–10. Chronic exposure to job stress has been identified as the main reason for physicians and nurses to leave their jobs¹¹. Workplace stress and burnout are driven by excessive workload, administrative hurdles, workplace violence and harassment, inadequate staffing, and emotional weight of caring for critically ill patients¹².

Artificial Intelligence (AI) - driven conversational agents are transforming healthcare by enhancing the efficiency and effectiveness of patient care and monitoring. AI systems have demonstrated benefits across several areas: treating mental health conditions^13–16 and chronic diseases^17,18, managing chronic pain¹⁹, promoting healthy lifestyles²⁰ and delivering tailored patient education^21,22. AI-powered nursing assistants can help alleviate the administrative burden in hospitals and clinics by automating routine tasks such as scheduling appointments, providing reminders and handling repetitive queries from patients^23,24. The automation of non-urgent tasks can help nurses focus on more complex clinical decision-making and improve patient care. AI conversational agents have been successfully used to follow-up patients after orthopedic surgery²⁵, in hypertension treatment²⁶, and after healthcare intervention for cancer, asthma and ureteroscopy²⁷. To the best of our knowledge, there hasn’t been a previous attempt to use a voice-based conversational AI agent for pre-procedural patient preparation.

In this study, we evaluate the capacity of AI to effectively deliver tailored pre-procedural instructions, assess patient readiness, and capture critical pre-procedural data. As a high-volume catheterization laboratory, our physicians perform more than 16,000 procedures every year. The day before the procedure, each patient receives a 15-20-minute phone call from a nurse with pre-procedural instructions and a clinical questionnaire. Patient preparedness is extremely important for a successful and safe procedure, and not following pre-procedural instructions might lead to cancellations or complications due to allergy reaction, bleeding, or aspiration^3,28. To augment and optimize the pre-procedural patient workup, we implemented a voice-based conversational AI system named Sofiya in our cardiac catheterization laboratory (CCL). Our goal was to help optimize nurses’ workload by automating repetitive administrative tasks and patient data collection and allowing them to focus on critical issues of direct patient-facing care, while maintaining a high level of patient satisfaction. The total number of successfully completed calls made by Sofiya was evaluated during a two-phase implementation period. In addition, we evaluated patient satisfaction using post-procedural surveys.

Results

Patient characteristics

Over the six-month study period, from January 16 to July 17 2025, 1431 patients scheduled for cardiac catheterization received 1606 calls from Sofiya with pre-procedural instruction and a clinical questionnaire (Supplementary Table 1). The patients' mean age was 67.1 ± 10.4 years, and 28% were female. The study population was ethnically diverse and consisted of 52% white, 17% black, 16% Asian, and 13% other ethnicities. Most patients had previously diagnosed single- or multi-vessel coronary artery disease (73%), and more than a third of patients had a previous percutaneous catheterization procedure. Baseline demographic and clinical characteristics were similar in the two phases of the study (Supplementary Table 1).

AI system call completion rate

During the 6-month study period, the total number of Sofiya calls was 1606; 806 calls were performed in the first 90-day optimization period (Phase I) and 800 calls in the subsequent 90-day routine-use Phase II (Table 1). The total completion rate was 696 (86.4%) and 703 (87.9%) in Phases I and II, respectively. In the completed call groups, the number of fully automated and not requiring human intervention calls in each study period was 295 (36.6%) and 341 (42.6%). Sofiya calls requiring a quick protocol-driven callback from a nurse to clarify medication use or answer a patient’s question were 303 (37.6%) and 268 (33.5%) in Phases I and II. In Phase I, 98 (12.2%) calls had an appropriate conversion to a human call, with the most frequent reason being patient refusal to talk to AI (n = 35). A similar number of calls, 94 (11.8%), were converted to a nurse call in Phase II.

Table 1.

Completion call rates in the prospective two-phase evaluation of an AI voice assistant, Sofiya, implementation for pre-procedural patient preparation

	Phase 1	Phase 2
Patient calls	N = 806	N = 800
Call completion	696 (86.4%)	703 (87.9%)
Patient self-service resolution, no callback	295 (36.6%)	341 (42.6%)
Protocol-driven callback needed	303 (37.6%)	268 (33.5%)
Clinical questionnaire	170 (21.1%)	172 (21.5%)
Clinical clarifications	65 (8.1%)	44 (5.5%)
Human callback requested	12 (1.5%)	12 (1.5%)
Logistical, cancel, reschedule	56 (6.9%)	40 (5.0%)
Appropriate human call conversion	98 (12.2%)	94 (11.8%)
AI refusal	35 (4.3%)	22 (2.8%)
Scheduling issues	22 (2.7%)	35 (4.4%)
Not patient speaking	20 (2.5%)	12 (1.5%)
Human data entry error	17 (2.1%)	8 (1.0%)
Asked to speak to human	4 (0.5%)	17 (2.1%)
Call non-completion	110 (13.6%)	97 (12.1%)
Patient-related issue	62 (7.7%)	73 (9.1%)
No answer	42 (5.2%)	47 (5.9%)
Hung up (usually at start)	20 (2.5%)	26 (3.3%)
AI system reliability issue	48 (6.0%)	24 (3.0%)
Technical error	23 (2.9%)	21 (2.6%)
Patient validation failed	14 (1.7%)	1 (0.1%)
Hallucination	6 (0.7%)	0 (0.0%)
Disruptive conditions (TV, noise)	5 (0.6%)	2 (0.3%)

Sofiya’s performance after a 90-day optimization period (Phase I) and during a subsequent 90-day routine operation under nurse supervision and no active optimization (Phase II).

In Phase I, 110 (13.6%) calls did not reach the end and were classified as incomplete, either due to patients not answering the phone or hanging up, 62 (7.7%), or due to AI system errors, 48 (6%) (Table 1). A total of 97 (12.1%) calls were incomplete in Phase II, 73 (9.1%) due to patient-related issues and 24 (3.0%) due to AI system issues. Most of the AI issues were associated with technical errors due to a temporary downtime of the portal, 23 (2.9%) and 21 (2.6%) in Phases I and II. AI hallucination rate leading to an incomplete call was low, 6 (0.7%) in Phase I, indicating that accurate responses to patients’ questions were provided in 99.3% of the calls. In one instance, Sofiya provided a positive answer to a patient asking whether they need to stay overnight after the procedure despite the lack of information in the knowledge base. In another call, Sofiya provided the wrong callback number. After reviewing the calls, nurses had to call these patients and clarify the information. There was no incidence of AI hallucination leading to an incomplete call in Phase II.

Temporal trends of weekly successfully completed calls according to AI technology upgrades and new feature implementations in Phase I of the study are shown in Fig. 1a. The rate of successfully completed calls increased from 58.3% to 92% during the study period. Based on the clinical team and patient feedback, we implemented a series of major upgrades with the main focus to improve patient engagement and response rate, including: (1) splitting patient name and date of birth (DOB) fields to simplify verification; (2) sending a text message notifying the patient about a scheduled appointment and incoming AI voice call from Sofiya; (3) improving DOB validation; (4) switching to a new phone number with the hospital’s area code and prefix to help patients differentiate Sofiya’s calls from unsolicited calls; and (5) upgrading and updating the customized LLM. As a result of the updates and new feature implementation, the weekly rate of completed calls had consistently exceeded 90% during the last month of the study. After the initial optimization period, we were able to maintain high weekly rates of completion calls during the subsequent 90-day period of routine operation (Fig. 1b) except for the week of June 16^th, when most Phase II technical errors were identified (Table 1).

Fig. 1. Temporal trends of call completion rates.

The weekly rate of successfully completed calls (blue line) during prospective two-phase evaluation of an AI voice assistant, Sofiya, for pre-procedural patient preparation. a Temporal trends (orange line) of the completed call rates according to Sofiya’s major updates (red dotted line) and new features implementation (orange dashed lines) during the first three month of the evaluation study (Phase I). b Completed call rates during the subsequent three months of a routine operation without significant changes to the system (Phase II). For both phases, the green dotted line represents their respective average completion rates. AI, Artificial Intelligence; DOB, Date of Birth; Txt, text; LLM, Large Language Model.

Clinical efficiency and nursing workload reduction

Sofiya-assisted patient preparation averaged 8.9 min compared to 20 min in the manual approach without AI. This reduction in nursing workload corresponds to an annual time saving of 26,869 min ((636 + 571) × (20 min–8.9 min)) × 365/182). This time saving is equivalent to 37.3 twelve-hour nursing shifts per year.

Patient satisfaction

The overall patient satisfaction score was 94.6% and 98.1% in Phase I and II respectively (Table 2). Sofiya achieved the highest score on the questions regarding phone call “matching procedure experience”, “aligning with expectations”, and “instruction clarity”.

Table 2.

Patient satisfaction score

		Phase I	Phase II
		(n = 542)	(n = 571)
1	Instruction clarity	97.2	99.1
2	Adequate preparation	94.5	97.4
3	Easy to understand	94.1	97.4
4	Align with expectations	97.4	99.1
5	Phone call matching procedure experience	97.4	99.1
6	Level of comfort during call	93.9	98.3
7	Level of detail provided	95.2	98.3
8	Effectiveness of questions answered	95.7	99.1
9	Areas that were unclear or lacking	89.1	96.5
10	Overall call effectiveness in preparing for procedure	92.1	97.4
10-Question Average Satisfaction %		94.6	98.1

Discussion

Healthcare systems are increasing efforts to utilize AI to automate and assist clinical staff in time-consuming and repetitive tasks related to patient engagement, caregiving, and data management. Despite efforts, most prior studies have been limited to text-based chatbots or controlled research environments rather than routine clinical use^29,30. The majority of healthbots use a finite-state input when a dialog is led by the system and follows a predetermined algorithm, and only a few apps use machine learning and natural language processing³¹. In this prospective study, we deployed a voice-based AI platform, Sofiya, which conducts pre-procedural telephone calls for patients scheduled for cardiac catheterization in a high-volume CCL. Sofiya called patients scheduled for cardiac catheterization procedures to provide instructions (arrival time, directions, parking, diet, etc.), collect medical information (allergies, medications), and answer questions.

Based on the analysis of 1606 calls to 1431 patients, we demonstrated that such an AI system was able to successfully complete most patient calls. In the first 90-day optimization period, the overall rate of successfully completed calls was 696 (86.4%). In the completed calls group, 295 (36.6%) calls were fully automated and did not need a callback from the nurses after review, 303 (37.7%) calls required a quick protocol-defined clarifying call from a nurse, and 98 (12.2%) calls were appropriately converted to a human call. One hundred and ten (13.7%) calls did not reach the end and were classified as incomplete due to patient-related issues (patients not answering the phone or hanging up) 62 (7.7%) or AI system reliability issues 48 (6.0%). During the first implementation phase, the effectiveness of AI automation showed consistent improvement with each system upgrade reaching a weekly completion rate exceeding 90% at the end of the phase. We were able to maintain a high completed call rate, 703 (87.9%), and low rates of AI errors, 24 (3%), in the subsequent 90-day nurse-led routine operation period without system updates.

The findings suggest Sofiya’s strong ability to reach patients, provide instructions, collect clinical variables, and either answer patient’s questions or report a call as requiring human assistance. The flexibility of our LLM-based system is important in real-world pre-procedural calling, where patient questions often do not follow a fixed script but still require safe escalation when outside the system’s supported scope. In our study, more than one-third of all calls were fully automated and did not require human intervention. Another third of the calls required a quick callback from a nurse clarifying specific clinical findings identified and documented by Sofiya, allowing nurses to focus their attention on the clinical aspects of the call. Those patients who completed a call with Sofiya patients expressed an overall positive view of their experience, confirming that AI-assisted phone calls have the potential to optimize clinical processes while maintaining high levels of patient satisfaction. The study’s comprehensive user-centered evaluation metrics allowed us to assess the impact of the AI system on our clinical tasks and its effectiveness in user-interactive conversations³².

In addition to performance and reliability assessment of the new AI system, the study provides valuable insights into the implementation process and major barriers to achieving a high level of call completion. Insufficient patient engagement and response rate at the beginning of the optimization period (Phase I) was significantly improved after introducing a text message notification prior to Sofiya’s call and changing the phone number to a hospital number (Fig. 1a). Optimizing patient name and DOB validation further contributed to improved completion rate at the end of the study period. AI automation showed significant improvement during the 3-month optimization phase and reached a high level of completed calls in the last improvement cycle of over 90%.

Preparing patients for cardiac catheterization requires critical but repetitive tasks to ensure that the patient is ready for the procedure and the clinical team is prepared for special circumstances such as allergies and medications. Occupational burnout syndrome is characterized by emotional exhaustion and depersonalization, and leads to decreased effectiveness at work as well as diminished quality of patient care³³. Without adequate intervention, the global deficit of health-care workers is estimated to reach 18 million by 2030³⁴. As a result, as procedure volumes rise and workforce shortages intensify, this manual approach is becoming increasingly unsustainable. Conversational AI systems can help reduce the workload and resulting burnout by automating routine tasks such as providing medical reminders, scheduling appointments, and managing repetitive queries from patients. By automating the non-urgent and non-critical tasks, conversational AI systems can help health-care professionals focus on more important critical aspects of direct patient-facing care and care that better utilizes their expertise. Our estimate of nurse workload reduction yielded an average of 8.9 minutes per patient prepared with Sofiya, a significant reduction from the 20-minute baseline average. Annually, for patients completing the full protocol, this efficiency gain reclaims 26,869 min, equivalent to 37.3 twelve-hour nursing shifts. By reducing routine non-clinical tasks by approximately two months per year, Sofiya enables skilled nurses to refocus their time on clinical work.

Although the overall patient feedback was overwhelmingly positive, Sofiya was unable to reach 57 patients due to the patient’s refusal to engage with AI. It is possible that the number of AI refusals will decrease in the future as patients become more comfortable and familiar with similar technology used in other industries, as many services shift from in-person to digital access. However, the process might be more complicated and involve broader concerns within the health-care system. In a recent survey of U.S. adults, most respondents reported low trust in their health-care system to use AI responsibly (65.8%) and low levels of trust in their health-care system to ensure that AI tools would not harm them (57.7%)³⁵. The main predictor of low trust was previous experiences of discrimination while seeking care, while neither lack of AI knowledge nor health literacy was associated with these attitudes toward AI. As AI-based tools are implemented into clinical practice, it is important to address patient concerns, provide education on how the tools are being used, and what safeguards are placed to protect them. In our use case, we ensured a low threshold for calls to be converted to a human in favor of patient satisfaction.

Despite the rapid development of AI technology, significant challenges must be addressed before AI can be widely implemented in clinical practice²³. Key barriers include lack of knowledge about AI, concerns regarding its reliability and accuracy, issues related to data privacy and security, worries about the impact on patient relationships, and fears of job loss due to automation³⁶. AI education and training, technical assistance, active involvement of clinicians in AI systems development, institutional support and regulatory oversight can help overcome some of the barriers and facilitate the implementation of AI in healthcare^37,38. To be able to call patients, Sofiya had to undergo extensive and rigorous review by a series of institutional committees assessing data security, AI risks, compliance, and ethical issues. We addressed the possibility of errors by careful monitoring of all calls and performing a clinician attestation for each AI-generated report. All calls were reviewed for accuracy by RN, particularly focusing on the clinical variables collected before each encounter. One of the most important and core ethical considerations is balancing technology and the “human touch.” AI integration should be approached carefully due to the significance of the nurse–patient relationship and ethical concerns regarding the “human” aspect of nursing. In addition, patients’ attitude toward AI can significantly influence its acceptance, underscoring the importance of a patient-centered approach to the implementation of clinical AI.

Patients bring diverse levels of understanding across numerous dimensions, including language, clinical expertise, generalized interest, and overall reading comprehension. The ability to adapt content and communications can be truly transformative to fostering greater patient understanding, engagement and, ultimately, adherence to their care plans over time. In contrast to text-based mobile or web-based AI technology, voice-enabled systems provide human-like verbal communication, a more natural and intuitive experience, and help remove technological barriers in cases of lower literacy. These capabilities make it more likely for patients to understand the procedure, the actions they need to undertake, and the rationale for these actions. Digital health elevates health equity, but only for those with access to new technology³⁹. Patients who lack reliable access to the internet or a smartphone may experience a gap in AI-based care. Sofiya does not require any additional technology or devices and is able to effectively reach patients using either cell phones or landlines.

Sofiya experienced common limitations of speech recognition tools such as background noise, accents, speaking styles, and vocabulary size. Some of the classifications, such as “no answer” and “hung up” in the patient-related reasons for a call not being completed, could potentially fall into another category, but we did not interrogate the patients for the underlying reason. Patients who required a family member, caregiver, or interpreter to complete the call were not included in the study in compliance with hospital policy. As a result, the findings may not fully generalize to patients who rely on caregiver-mediated communication, including some older adults and patients with limited English proficiency. Since the feedback questionnaire is specifically investigating whether a patient felt that Sofiya prepared them for a procedure, we only queried patients who were able to get to the end of the call script and finish the clinical questionnaire. A significant limitation of the assessment is compliance bias, as these patients are more likely to provide favorable feedback. Patients who refused AI engagement or had a technical error are not represented in the feedback. In addition, patient satisfaction survey was not previously validated.

Enhancements are planned to include a callback feature, multilingual support, adding text messages with directions and parking instructions, assisting patients with directions within the hospital, expanding to include patients scheduled for biopsy and peripheral disease procedures, and including limited medication instructions. EHR integration will further automate the workflow and simplify the administrator’s role. Additionally, we are working on automating post-discharge follow-up calls.

In conclusion, the study describes a two-phase implementation project of a voice-based conversational AI system, Sofiya, in the pre-procedural clinical workflow for cardiac catheterization. Sofiya called patients scheduled for the procedure to provide pre-procedural instructions, collect medical information, and answer patient’s questions. Sofiya was able to successfully complete most of the calls in both implementation phases, 696 (86.4%) during the optimization period and 703 (87.9%) in the subsequent real-word performance phase. More than one-third of all calls were fully automated and did not need a callback from the nurses after review, and another third of the calls required a quick callback from a nurse clarifying specific clinical findings identified by Sofiya. Conversational AI systems have the potential to automate routine components of pre-procedural patient preparation, allowing healthcare professionals to redirect their focus toward more complex clinical tasks.

Methods

In this section, we describe details of the pre-procedural phone calls, the conversational AI platform, and the two-phase implementation study.

Pre-procedural phone call

A registered nurse (RN) calls a patient the day before the procedure (1) to provide scheduling and logistics details (arrival time, where to check in, parking and transportation advice), overview of the procedure, fasting instructions, and post-procedure planning; (2) to collect relevant clinical data, such as current medications and provide instructions (which prescription medications to take and which ones to hold, such as oral anticoagulants), allergies and comorbidities (any allergies to contrast dye, iodine, or shellfish, as well as dialysis, and active fever symptoms); (3) to answer patient’s questions and generate a report for the electronic health records (EHR). Pre-procedural preparation workflow before AI implementation is described in Table 3. We aimed to automate the repetitive administrative tasks of the call, collect data, as well as answer simple patient inquiries to allow the nurses to focus on more complex clinical encounters.

Table 3.

Clinical workflow before and after implementation of AI voice assistant, Sofiya, for pre-procedural calls

	Before AI implementation	After AI implementation
	RN’s old workflow	RN’s new workflow	Sofiya’s workflow
Call initiation	RN calls patient; no patient answer is a challenge		Sofiya calls per Admin-entered schedule; re-dials if no answer
Providing basic and logistical information (time, place, directions, etc.)	RN spends time providing basic information		Sofiya provides information during the call after patient validation
Collecting clinical data on medications and allergies	RN collects clinical data and enters it into the EHR as a note	RN validates and calls the patient back if needed	Sofiya collects the data and generates a report
Answering patient’s questions	RNs answer simple, repetitive, as well as complex questions	RN verifies Sofiya’s answers; provides answers only if needed	Sofiya answers questions based on training or redirects the patient to a nurse
Report	RNs manually enter the report into the EHR		Admin scans Sofiya’s reports to EHR

RN registered nurse, Admin administrator, EHR electronic health record system.

Conversational AI system, Sofiya

Sofiya is a customized conversational LLM and agentic AI system that incorporates our preferred instructions, patient call script for obtaining clinical information, and general information about our center. Unlike rule-based automated call systems that depend on predefined decision trees and fixed response mappings, the LLM-enabled agentic architecture allows Sofiya to interpret varied patient phrasing, respond to compound or unexpected questions, and maintain conversational flow across multi-step interactions. To reduce inaccurate responses, the system was constrained by domain-specific instructions, curated knowledge sources, escalation rules for unsupported topics, and mandatory RN review of all encounters.

The system includes: (1) an AI-powered response generator module, which is a LLM trained on medical and procedural knowledge to provide contextual responses to inquiries in text format; (2) an AI-powered speech recognition and voice synthesis module that serves as an intermediary between the response generator and the patient, utilizing text-to-speech (TTS) and speech-to-text (STT) services; (3) a call manager module that handles call routing between the patient and the AI modules through a VOIP gateway; and (4) an agentic AI orchestration module that coordinates the components to create a coherent voice call (Supplementary Figure 1).

The new AI-powered workflow is shown in Table 3 and Fig. 2. Sofiya initiates a call using a schedule and automatically re-dials the patient if needed. Following patient validation, Sofiya (1) provides basic and logistical information; (2) collects patient’s data on allergies, blood thinner use, diabetes status, dialysis, medication pump, and fever; (3) answers patient’s questions or redirects the patient to a nurse. Sofiya provides brief pauses allowing patients time to write down important information, the option to repeat any details, and the ability to hold and wait upon the patient’s request. After the call, Sofiya updates the call’s status and monitoring dashboard, facilitating review by a nurse. After the call has been verified by the RN, Sofiya generates a call encounter report, which is then added to the patient’s EHR.

Fig. 2. Sofiya workflow diagram.

(1) Sofiya contacts the patient scheduled for catheterization and provides instructions, collects clinical variables, and answers questions; (2) The conversation, summary, and clinical variables are made available for review on the online portal; (3) Registered nurses (RNs) review calls on the portal and call the patient back based on protocol-driven reasons if needed; (4) RN makes necessary comments or changes and verifies the AI-generated report validity of the information; (5) The report is scanned into the electronic health record (EHR) of the patient. AI, Artificial Intelligence; IV, Intravenous.

Customization and implementation

We customized the system through close collaboration among cardiologists, nurses, and AI software developers. Sofiya ingested documentation detailing procedure and location information, as well as an extensive list of frequently asked questions and answers to help it respond to patient inquiries, including clinical and procedural queries. The virtual assistant was instructed not to provide any medication-related advice and to report such patients' requests for human review. In the monitoring dashboard, Sofiya classifies all calls according to pre-defined criteria based on the outcome of each encounter (Fig. 3). “Completed” calls were those that successfully reached the end of the script and where no key clinical questions were answered with a “yes”. “Needs verification” calls were those that reached the end of the script, but where at least one of the clinical questions was answered “yes” (for example, the question about anticoagulant use). “Needs attention” calls were those that did not reach the end of the script due to a patient hang-up, inability to reach the patient, or a system issue. The nursing team reviews all calls in order of priority, starting with calls classified as “Needs Attention”, then those classified as “Needs Verification”, and finally those classified as “Completed” calls. A classification of “Waiting on Patient” indicates that a voicemail was identified and a message left requesting the patient to call back to Sofiya within a two-hour window, where they can independently initiate their intake process. All calls are reviewed for accuracy, particularly focusing on the clinical variables collected before each encounter is verified by an RN, and a report can be generated. Calls that started in the states “Completed” or “Needs Verification” become “Verified” upon clinician review, and calls starting as “Needs Attention” become “Resolved” upon clinician review. The nurse reviewing calls was able to return to direct patient care after all calls were either “Verified” or “Resolved”.

Fig. 3. Sofiya portal monitoring dashboard showing the status of each call.

For each AI-assisted call on the portal, a new record is created reflecting the status of the call, allowing the nurses to prioritize their review in order of urgency. AI, Artificial Intelligence.

Education and training

Interacting with AI tools is still a relatively rare experience, especially in a healthcare setting. Nurses underwent one-on-one training with Sofiya superusers who understood the full spectrum of features. Sofiya superusers were RNs from the catheterization laboratory who had previously performed standard pre-procedural calls. They received hands-on, in-person training in portal navigation, call-status review, transcript and report verification, redial functions, and escalation workflows during the initial implementation period. After gaining proficiency, they served as local champions and first-line support for other nurses adopting the AI-assisted workflow. While reviewing calls and verifying reports came naturally as it mirrored closely their prior workflow, some of the new aspects to their workflow were more of a challenge. Once nurses passed the learning curve, they were in favor of the new AI-augmented workflow. The learning curve primarily involved navigating Sofiya’s portal, reviewing transcripts and autogenerated reports, managing redial workflows, and triaging calls by status. Early support needs varied according to baseline digital literacy, but most nurses were able to use the system independently within 1–2 days.

Implementation study design

We conducted a two-phase prospective implementation study involving consecutive patients who underwent a catheterization procedure and received a phone call from Sofiya for pre-procedural preparation between January 16, 2025 and July 17, 2025. Sofiya’s performance was assessed after a 90-day optimization period with rapid-cycle improvements, system updates with close input from clinicians and the AI developers (Phase I) followed by a subsequent 90-day real-world performance period without further system optimization and nurse-led oversight (Phase II).

During the study period, patients who met the following criteria were selected for Sofiya’s calls: (1) patients scheduled for cardiac catheterization; (2) English-speaking patients who did not require an interpreter; and (3) patients who were the primary contact and did not need a family member to take the call. Administrative staff created patient profiles in the portal and recorded their arrival times at the hospital. To prevent overwhelming the phone lines, patient calls were made in batches to accommodate those who called back immediately. If there was no response, the numbers were automatically redialed. Calls were expected to be completed shortly after they began, unless they went to voicemail. In such cases, patients had a two-hour window to return the call. The encounter report generated by Sofiya was reviewed by the RN for accuracy, with a particular focus on the clinical data. Calls were triaged by the dashboard status, with priority given to “Needs Attention” and “Needs Verification” encounters. Nurses cross-checked key clinical variables against the transcript and performed a callback when clarification was required, usually within 30 min after Sofiya's call. “Completed” call scripts were reviewed last to rule out any out-of-script questions or clinical variables being missed. With this workflow, no Sofiya encounter was left without RN review. After validation, the reports were uploaded to the patient’s EHR (Fig. 2).

Study outcomes and metrics

All transcripts of calls made by Sofiya were evaluated by study personnel, who categorized calls as either successfully completed or incomplete based on the following definitions:

• Successfully completed calls reached the end of the script with patient answering all the questions and having no remaining unanswered questions after the instructions were delivered and either (1) not requiring a clarifying callback due to patient self-service resolution (fully automated calls shown in Fig. 2, pathway 1-2-4-5); or (2) protocol driven clarifying callback due to a clinical question, patient’s request to speak with a person or service recovery call related to logistics or scheduling; or (3) appropriate conversion to a human call such as refusal to interact with AI, request to cancel or reschedule the procedure, or only a family member being available to speak. AI-generated reports for two completed calls with and without patient callback are shown in Fig. 4. RN’s assistance with the patient’s question regarding blood thinner medication was required in the second example shown in Fig. 4b. The automatically generated report reflects the RN’s instructions on medication use.

• Incomplete calls were calls that did not reach the end of the script. To better understand Sofiya’s ability to communicate with the patients, we categorized the reasons for incomplete calls as either (1) patient-related (patients not answering the phone or hanging up) or (2) AI system reliability issues (technical error, patient validation failure, disruptive conditions interfering with speech recognition, or hallucinations, instances in which AI generated an unsupported or incorrect response not grounded in the approved workflow content) (Fig. 5).

Fig. 4. Reports of successfully completed AI calls.

a AI-generated report of a fully automated call with no nurse call back required; b AI-generated report of a call with a comment (red box) requiring a nurse to call back due to a medication reason. AI, Artificial Intelligence.

Fig. 5. Report of an incomplete AI call.

The call was classified as incomplete due to the patient hanging up after receiving instructions without completing the medical history questionnaire (red box). AI, Artificial Intelligence; DOB, Date of Birth; NPO, Nil Per Os.

The primary study outcome was the overall completion rate for each phase, calculated as [(the number of successfully completed calls) ÷ (total number of calls) × (100)] during the study period.

To evaluate how upgrades in the conversational AI system impacted system performance, in the optimization period (Phase I), we analyzed trends in completed calls for each upgrade using a Plan-Do-Study-Act (PDSA) approach, a widely accepted method in healthcare improvement⁴⁰. The interactive four-step PDSA model was used to implement and assess small and incremental quality improvement changes in Phase I. For this analysis, we calculated the weekly completion rates as follows: [(number of successfully completed calls) ÷ (total number of calls) × (100)] during the week.

We used the Sofiya portal call logs in order to create a data-driven estimate of the length of time for nurses to prepare a patient for their procedure. For each day, we extracted the time of the last signed report and then subtracted the time of the second signed report (as the starting time of the first report was not available). The daily duration of Sofiya-assisted work was divided by the number of patients to obtain a per-patient average time. Because the nurses were not always focusing solely on their Sofiya task (taking lunch breaks or focusing on other work), we used a formula to exclude instances where reports were signed 40 or more minutes after the previous one. For the calls performed without the AI system (Legacy cases), we obtained an estimate of 20 min per patient from the nurses who handle these calls. The 40-minute exclusion logic covered 92% of patients and corresponded to double the average legacy call duration. The resulting time for a Sofiya-assisted case was 8.9 min including the time to review transcripts, where the patient never needed to be contacted by the RN and those that spend a long time on the phone. Using the following formula and the data from Table 1, we estimated an average time saved per patient, and ultimately, an annualized time savings: Annualized time saved = ((Patient self-service calls + Protocol-driven callbacks) × (Legacy case (min) – Sofiya case (min))) × 365/182.

After their procedure, patients who completed an AI-assisted call were asked to complete a post-procedural survey. The survey was developed locally for the study to assess the instruction clarity, perceived preparedness, comfort during the call, and whether patient questions were adequately addressed. (see Supplemental Table 2 for the survey questions). The Customer Satisfaction Score (CSAT) was calculated using the formula: [(number of satisfied patients) ÷ (total number of patients) × (100)]⁴¹.

Institutional review and hospital policy changes

Multiple Mount Sinai Health System (MSHS) committees, Digital and Technology Partners groups as well as the medical board reviewed and approved Sofiya, including legal implications, risk and compliance assessment, as well as ethical and security concerns. Based on the discussions, the Front Door Acknowledgement policy was updated by the MSHS medical board to notify patients about the possibility of MSHS using AI tools to assist in making decisions about patient care.

Ethics approval and consent to participate

Retrospective analyses of Sofiya operational data were performed as part of routine healthcare operations and health-system quality assessment, using data generated during standard clinical care and program implementation, with no additional patient contact or data collection for research purposes. Accordingly, this component was treated as a non-research quality improvement activity and did not require informed consent. The patient satisfaction survey, not a part of routine operations, was reviewed by the Program for the Protection of Human Subjects (PPHS) / Institutional Review Board (IRB) at the Icahn School of Medicine at Mount Sinai (IORG0000124) in accordance with the Declaration of Helsinki and the PPHS office determined that this STUDY-25-00134 was EXEMPT human research as defined by DHHS regulations (45 CFR 46.104 (d)).

Author contributions

A.K. designed and supervised the study. A.V., D.P., and Y.V. drafted and revised the manuscript. A.V. and D.P. performed the data analysis and system implementation. B.O., G.P., and Y.B. developed the clinical protocols and managed staff implementation. J.B., S.S., and N.G. designed the study framework and interpreted the results. A.R. and B.D. oversaw regulatory compliance and digital governance. All authors reviewed and approved the final manuscript.

Data availability

No data is shared as part of this article. Deidentified data on AI phone calls can be made available after review and approval of a proposal by the corresponding author.

Code availability

Statistical code underlying the results of this publication can be provided by the corresponding author upon request.

Competing interests

The authors declare no competing interests.

Supplementary information

The online version contains supplementary material available at https://doi.org/10.1038/s41746-026-02959-x.