Fig. 4. Note generation interface design.

a Speech Transcripts Panel is for presenting the transcripts of the conversations (right) and medical information recognized from the transcripts (left). When clicking on a medical term, the positions where it appears are highlighted as yellow bars (or an orange bar if it is the one currently being shown). b Note Writing Panel is a regular text box for note writing and editing. Clinicians can add information from (a) into (b) by dragging and dropping the text in (a) into the desired position in (b). c Raw Notes Panel presents the raw notes including handwritings, drawings, photos, and/or videos. Clinicians have the option to switch between (a) and (c). d ICD 10 List contains a list of ICD 10 codes recognized from the conversation and used for billing purposes.