Table 5. Qualitative Encoding of Free-Text Comments From Postsurveys.

Theme	Representative quotations	No. of comments
		Negative	Neutral	Positive	Total
Draft message voice and/or tone	Positive: “I was impressed by the tone that varied based on patient’s concerns and questions, and felt messaging was overall very professional and clear.” Negative: “I think the drafts are great but can further be improved if it did not sound robotic and had a more personable touch.”	14	2	10	26
Future use	Positive: “Please continue to allow us to utilize this tool and spread to other SHC clinics!” Negative: “I still think it’s a good idea but not ready for real life situations.”	1	0	18	19
Draft message tool utility	Positive: “Overall this is a very helpful tool.” Negative: “Also, it struggled with having draft replies of more nuanced concerns.”	4	2	13	19
Draft message content relevance	Positive: “I especially appreciated the one example where a patient mentioned having frequent UTIs on a certain medication, and the response had pulled in the last 3 lab results from urinalysis!” Negative: “The Reponses often did not accurately reflect the questions. Sometimes way off. Often vague.”	9	1	8	18
Impact on workflow	Positive: “It helped with the ‘translation’ cognitive work that I hadn’t ever realized I was doing before process of translating my medical understanding into patient-facing language.” Negative: “I have to read the actual draft before starting to work on the actual request, as I don’t know if the response is even appropriate.”	9	0	7	16
Impact on time	Positive: “It helped save me a lot of time starting from scratch.” Negative: “Right now, it is just piling on top of the work that we are already doing, and it is faster for me to type a prose response that I have generated myself.”	1	0	12	13
Draft message length and/or brevity	Positive: “However, the responses are very thorough. I had a patient that needed a refill and the draft wrote out almost a whole letter when I typically would maybe just write a short sentence saying ‘Yes, I will send!’” Negative: “Overall the responses seemed unnecessarily wordy in noncontributory ways.”	8	2	1	11
Draft message content accuracy	Positive: “I found the AI-generated draft replies pretty accurate and helpful.” Negative: “Sometimes, the AI response was not completely accurate, but it was not difficult to make minor tweaks to the draft.”	5	0	4	9
Impact on patient engagement	Positive: “This may have a positive impact on patient satisfaction with longer messages.” Negative: “Patients can tell these responses were AI generated, they are formatted like the AI responses we get on airline websites.”	2	2	3	7
Draft message content completeness	Positive: “Good things are AI can capture all the elements in the message patient sent and address each element.” Negative: “The AI responses were a great initial draft, though often required some additional information or editing.”	4	0	1	5
Total	NA	57	9	77	143

Abbreviations: AI, artificial intelligence; NA, not applicable; SHC, Stanford Health Care; UTI, urinary tract infection.