A novel method for evaluating physician communication: A pilot study testing the feasibility of parent-assisted audio recordings via Zoom

Abstract

Objective

Quality of physician consultations are best assessed via direct observation, but require intensive in-clinic research staffing. To evaluate physician consultation quality remotely, we pilot tested the feasibility of parents using their personal mobile phones to facilitate audio recordings of pediatric visits.

Methods

Across four academic pediatric primary care clinics, we invited all physicians with a patient panel (n = 20). For participating physicians, we identified scheduled patients from medical records. We invited parents to participate via text message and phone calls. During their adolescent's appointment, parents used their mobile phone to connect to Zoom for remote research staff to audio record.

Results

In Spring 2021, five of 20 (25%) physicians participated. During a nine-week period, we invited parents of all 54 patients seen by participating physicians of which 15 (28%) completed adult consent and adolescent assent and 10 (19%) participated. For 9 recordings, at least 45% of the conversation was audible.

Conclusions

It was feasible and acceptable to directly observe physician consultations virtually with Zoom, although participation rates and potentially audio quality were lower.

Innovation

Patients used their cellular phone calling features to connect to Zoom where research staff audio-recorded their physician consultation to evaluate communication quality.

Highlights

• •Parents and physicians found Zoom audio recordings of in-person visits acceptable.
• •Population representativeness and staffing efficiencies may be increased.
• •The percentage of parents recruited (19%) was lower than in-clinic research.
• •Some words were missed: 70% of the recordings had ≥ 83% of audible minutes.
• •This strategy reduced clinical workflow interruptions and the spread of infections.

Physician communication is a strong predictor of patient acceptance and compliance with health recommendations [[1], [2], [3], [4]]. To measure and improve physicians' effectiveness, rigorous analysis of physician communication is essential. Assessing physician communication is more accurate when performed via direct observation (an observer present or audio/video recordings) rather than relying on physician report [5]. Direct observation typically requires research staff in clinics to identify interested patients, consent participants, and observe clinical encounters or manage recording equipment [6,7].

Given widespread use of mobile phones and increased use of virtual platforms in healthcare [8,9], virtual recordings of clinic visits with patients facilitating recording is a promising alternative. First, patients and physicians are comfortable with recordings, and recordings do not alter the ability to discuss issues openly [10]. Second, teleconference software can offer platforms in which physicians are increasingly comfortable, patients' mobile phones only need calling features, and recording capacity can be restricted to the research staff [11].

While patient-controlled audio recordings via an iOS app has been used to increase patient recall [12], we are unaware of any studies where patients use their mobile phone calling features to facilitate audio recordings for the purpose of evaluating physician communication. This study aimed to pilot test the feasibility of parents collecting audio-recordings of their physician's consultation using the calling features of their mobile phones to connect to research staff-controlled Zoom meetings.

1. Methods

We focused the study on 11- to 12-year-olds who had not received the human papillomavirus (HPV) vaccine because our primary interest was pilot testing the feasibility of this method to evaluate physicians' HPV vaccine recommendations. Within the four University of Florida - Gainesville pediatric primary care clinics, we identified and invited all pediatric physicians with their own panel of 11- to 12-year-olds (n = 20). The clinics serve pediatric populations for whom 51% are enrolled in Medicaid and patient race is 50% White and 28% Black. Among participating physicians, we invited parents of 11- to 12-year-olds with any type of scheduled appointments during our study period (March 25 and May 26, 2021). We tracked participation rates and determined the percentage of decipherable minutes of the audio recordings (decipherable/undecipherable minutes).

The study was approved by the University of Florida (UF) Institutional Review Board (IRB). The IRB verified that the Business Associates Agreement with Zoom did not exclude recordings from the Personal Health Information covered encounters. The IRB required research staff take several precautions: (1) create call links within Personal Health Information protected Zoom accounts; (2) disable the parents' ability to record; (3) close the Zoom room once the parent arrives; (4) store Zoom recordings on UF owned and maintained computers; (5) immediately transfer recordings to UF secure servers; and (6) destroy recordings as soon as metrics were recorded.

With written approval from the department chair, a co-author and practicing physician (LAT) sent an email invitation to each identified physician. Once a physician expressed interest in participating, research staff sent a link to a REDCap electronic consent form and, when requested, explained the consent in individual virtual meetings [13,14]. Physicians provided electronic consent signatures in REDCap.

We identified potential participants by reviewing the participating physicians' scheduled appointments under a Health Insurance Portability and Accountability Act waiver of informed consent. We invited parents in chronological order of scheduled appointments. Approximately 2 to 5 days prior to an adolescent's appointment, we sent a text message from the clinic inviting parents to participate in a study on doctor-patient communication during their adolescent's upcoming visit. The message stated parents would receive $50, notified them of phone call follow-up, included a phone number and weblink for more information, and offered an option to opt-out of future invitations. The webpage summarized the study and allowed parents indicate their interest via REDCap. Between 1 and 2 days after the text message, we called parents who had not opted-out.

Research staff spoke directly to parents and adolescents to obtain informed consent / assent via electronic signatures collected in REDCap. To ensure that the participating physician was aware of which visits were recorded, we alerted the physician of the participating patient's name and visit date in advance.

Approximately two hours prior to the adolescent's visit, research staff sent parents a Zoom link. Parents did not need to use the Zoom app and were instructed to call the phone number and enter the passcode to join the call. Research staff initiated the Zoom call 30 min prior to the adolescent's scheduled appointment and waited in the Zoom room for the parent to arrive. Parents called the Zoom number when they were in the exam room waiting for the physician. After a largely inaudible first recording, parents were instructed to place their phone on a table in the exam room. Research staff began recording via Zoom recording features when they heard the physician enter the room.

2. Results

Among 20 invited physicians, five contacted the study coordinator and all five agreed to participate (25%). During a nine-week period, the five participating physicians saw 54 patients who were 11- to 12-year-olds and had not received the HPV vaccine (weekly average = 6 visits, range 1 to 14). Overall, 10 of 54 families participated for a 19% participation rate.

In response to the text messages, no parents called or opted-out and four parents indicated interest via REDCap. All parents who completed the interest form were reachable by phone, consented to participate, and created a recording of their adolescent's visit (Table 1). Parents who completed the interest form represent 7% of invited parents and 40% of participating parents.

Table 1.

Response rates among invited parents.

	Overall	Interest form	Phone contact
	n	n(%)	n(%)
Contacted	54	4 (7%)	26 (48%)
Consented	15	4 (27%)	11 (73%)
Participated	10	4 (40%)	6 (60%)

Research coordinators called 44 of 50 parents who did not complete the interest form to invite them to participate. We reached 26 parents by phone (48% of invited and 59% of called), among which 11 (42%) consented to participate and six ultimately participated representing 60% of participating parents. Among parents reached on the phone, nine refused (17% of invited and 20% of called). No parents expressed dissatisfaction about being invited.

Among the 15 consenting parents, 10 participated (66%) (Table 2). The most common reason for non-participation was rescheduled appointments (13%). One parent withdrew prior to the appointment because upon further discussion their adolescent did not want to participate.

Table 2.

Dispositions among consenting parents (n = 15).

	Number of parents	Percentage
No show at appointment	1	7%
Rescheduled appointment	2	13%
Participation total reached	1	7%
Withdrew	1	7%
Participated	10	66%

None of the 10 participating parents had difficulty connecting to the Zoom phone line. Recordings averaged 23 min (range 13 to 37 min). For all but the first recording, 44% of the minutes of the visit were audible. For 70% (7/10) of the recordings, at least 83% of the minutes were audible.

3. Discussion and conclusion

3.1. Discussion

This study documents a novel method for evaluating quality of physician communication. It was feasible to audio record physician consultations by recruiting parents to use their mobile phone calling features to connect to Zoom. Our one-time email invitation garnered 25% of invited physicians to participate and all participating physicians were comfortable with the procedures. Most importantly, parents created audible recordings without difficulty. This small pilot study indicates that parents facilitating recording of their adolescent's clinical encounter may be an effective strategy to directly observe physician consultations.

Compared to similar studies conducting audio recordings of physician consultations for adolescents, our recruitment rate (19%) was higher than a study that recruited participants virtually and had in-clinic research staff facilitate the recordings (8%), but lower than a study with recruitment and recordings handled by in-clinic research staff (66%) [15,16]. However, research staff did not need to travel to clinics and could recruit parents and record visits from multiple clinics on the same day. The interest form linked from the text message required the least research staff time and recruited families who were most likely to participate.

There were limitations and strengths. First, the population was limited by geography and age; thus, results may not be generalizable to more rural areas or other patient age groups. Second, data collection was limited to one recruitment email to physicians and did not include demographics or measures of time spent on tasks. On the other hand, approaching potential participants days prior to the clinic visit gave families more time to consider participating.

3.2. Innovation

Our approach differs from prior studies in purpose and the lack of any specialty software or hardware. To evaluate communication quality, patients used their cellular phone calling features to connect to the health system's preexisting HIPAA compliant Zoom line where research staff recorded the physician consolation. Other studies have used clinic- and patient-controlled audio recordings to aid patient recall and have required app installation or clinic-controlled recording devices [12,17].

3.3. Conclusions

In this pilot study, parent-facilitated audio recordings of adolescents' clinical visits were feasible and acceptable. We found two potential benefits of this method. First, staffing efficiencies due to omitting the need for traveling to clinics would likely exist for studies with multiple clinical sites spread over a wide geographic area. Second, the patient-facilitated recording strategy likely reduced the research burden on the clinics in two ways: (1) the strategy had little or no influence on the clinical workflow - an essential element to facilitate successful clinical research [18], and (2) the reduced research staff in clinics may increase the safety of the research by limiting infection spread.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.