Table 2. Themes and Quotes From Physician Likelihood to Recommend Generative Artificial Intelligence (AI) Drafted Replies to Their Colleagues, on a Scale of 0 to 10 Where 0 is Not At All and 10 is Definitely.

Likelihood to recommend	Quote
Likelihood: 9 or 10
Tone and value: acknowledgment of the robotic tone of AI replies, recognizing their role in initiating patient interactions, and serving as a valuable baseline.	“Though the replies sound very robotic still, they’re extremely helpful for generating the baseline response to what you’d want to say to a patient.”
Potential for improvement and mimicry of physician language: anticipation for AI-generated replies to improve and emulate the communication style of individual physicians, enhancing personalization and human-like interactions.	“I can’t wait for them to get even better, to the point where they can mimic each physician’s language/tone.”
AI replies’ place and role: recognition of AI-generated replies’ valuable role in health care workflows, aiding in workload management and effective patient communication, and contributing to workflow efficiency.	“I think AI responses have its place. [I] worry about inaccuracies that I may miss due to busy workload. I have been very impressed with [a] few of the responses.”
Hope for reduced supervision: expressing hope for AI advancements leading to reduced supervision, envisioning a future where AI can function autonomously while maintaining high-quality patient communication.	“Great initiative which requires supervision. Hopefully there would be time when minimal supervision would be needed.”
Likelihood: 7 or 8
Tone and empathy: recognition of AI-generated replies for their kind and empathetic tone, aiding in maintaining respectful and caring interactions with patients.	“Helpful in drafting responses, provides more empathy into a response without me taking time to type it all out.”
Time savings and future expectations: appreciation for saving time and enhancing efficiency by initiating tailored drafts swiftly, compared with starting from scratch; optimism about future enhancements.	“Not perfect but decreases time I spend on it and has a kind tone.”
	“While not perfect, I think there have been a good number of cases where I use the draft as a starting point. I expect the AI responses to get better over time.”
Alleviating pressure to address patient concerns online only: perceived relief from the pressure of responding solely through MyChart, with AI aiding appropriate recommendations for in-person evaluations when necessary, thus lessening workload.	“AI generated messages often appropriately recommend that the patient be evaluated in person for specific concerns. Sometimes clinicians feel pressured to deal with patient concerns by MyChart alone. The use of AI generated messages can take away this pressure.”
Recommendation to colleagues: general endorsement of AI-generated draft replies to colleagues, emphasizing its potential benefits in starting drafts and infusing empathy into responses, despite the need for some editing.	“I generally would recommend auto-generated draft replies to colleagues because it seems to be net even—it may be helpful to start a draft, but most of the time, I am editing the replies, so it is not completely/automatically helpful.”
Likelihood: 0-6
Tone and language: critique of AI-generated replies for being excessively polite, formal, impersonal, and not aligning with the desired direct and concise tone in patient interactions.	“Messages were too nice and wordy, but sometimes offered good advice.”
	“Not personalized to the specific patient. I tend to personalize my response in different ways depending on the patient.”
Efficiency and time savings: recognition of AI replies as helpful starting points, but often requiring extensive editing, reducing potential time savings compared with drafting from scratch.	“I found I used them most when I was covering for other providers and did not know the patients as well so did not need to provide as customized of a message.”
Challenges in clinic population and workload: noting challenges in AI replies’ applicability, particularly regarding urgent and specific patient needs, where recommending appointments may not be appropriate or feasible.	“It ended up creating more work for me and ultimately always recommended scheduling an appointment. Our clinic is largely populated with persons who are struggling with multimorbidity and SDOH [social determinants of health] and need urgent assistance rather than a future appointment.”
Improvement and future potential: acknowledgment of AI potential, urging significant improvement in understanding patient queries, offering accurate information, and considering context for appropriate responses.	“I think it has potential, but is not anywhere near where it needs to be to be useful.”