Abstract
Introduction
Social distancing requirements during COVID-19 pose a challenge to conducting traditional academic detailing, which typically involves in-person peer education visits to improve patient outcomes. The main alternative is to conduct virtual academic detailing delivered through web-based technology, but this approach is fraught with many challenges. This study aimed to examine the feasibility and acceptability of a virtual academic detailing program implemented among health care providers.
Methods
The academic detailing program focused on appropriate opioid prescribing and chronic non-cancer pain management among a sample of providers. An initial in-person visit was followed by a virtual visit up to 8 weeks later. Videoconferencing was used to conduct the virtual visit with telephone as a backup. Feasibility was assessed whether the virtual visits could happen, and acceptability was assessed by provider satisfaction. Validated measures of Provider Satisfaction with Academic Detailing (PSAD) and Detailer Assessment of Visit Effectiveness (DAVE) with a 5-point Likert-type scale were used. Higher scores corresponded to higher satisfaction and greater perceived effectiveness. Non-parametric and parametric statistical tests were used to compare instrument summary scores across visits and between groups. Pairwise analyses across visits only included instrument responses for providers who participated in both visits and completed both surveys in their entirety.
Results
There were 127 (90 %) initial in-person visits completed out of 141 visits scheduled, with a survey response rate of 96 %. Out of 120 virtual follow-up visits scheduled, 92 (77 %) were conducted, and 56 surveys (61 %) were collected. There was a high level of satisfaction with the initial and follow up virtual academic detailing visits, though, among providers who participated in both visits and had completed surveys (n = 50), initial visits had slightly higher scores (mean difference = −2.94 [95 % Confidence intervals: −4.38, −1.50], p < 0.001). There was no significant difference in detailer perception across the two visits as seen in the scale summary score (0.05 [−0.56, 0.66], p = 0.86) and two individually reported items related to feasibility (0.07 [−0.29, 0.42], p = 0.72) and conversation (−0.05 [−0.28, 0.17], p = 0.63). Forty-one (44.6 %) virtual visits were conducted using WebEx, where video and screen sharing of visit content was possible, while the remaining 51 (55.4 %) were conducted using a telephone. There was no significant difference in provider satisfaction between WebEx vs. telephone visits (-1.47 [-4.99, 2.05], p = 0.82). Provider satisfaction was also not impacted by any technical difficulties as reported by the detailer (-0.04 [-3.30, 3.38], p = 0.98).
Conclusion
The results slightly favor in-person visits and suggest that virtual detailing visits need to incorporate strategies that minimize technical difficulties and prevent participants from defaulting to less favorable technology. Future research opportunities include evaluating the effectiveness of a virtual versus in-person delivery of AD program on outcomes such as providers’ opioid prescribing behavior.
Abbreviations: AD, Academic Detailing; ILPMP, Illinois Prescription Drug Monitoring Program; PSAD, Provider Satisfaction with Academic Detailing; DAVE, Detailer Assessment of Visit Effectiveness
Keywords: Virtual academic detailing, Provider satisfaction, Technology
1. Introduction
Academic detailing (AD) is an evidence-based educational outreach method designed to provide healthcare providers with up-to-date information that informs their practice and improves patient care [1,2]. AD has been used to improve prescribing behaviors [3,4], and impact clinician management of patients with chronic pain [5], hypertension [6,7], and HIV [8]. AD is traditionally conducted in-person and one-on-one with a clinician by specially trained personnel (i.e. academic detailer) [9]. However, in-person visits may not be feasible due to geographic distance or circumstances that warrant social distancing, such as pandemics [10,11]. In such a case, conducting virtual AD may be an attractive alternative to in-person detailing. However, the feasibility and effectiveness of virtual AD are not well understood or established. Among the few studies that report utilizing virtual AD, heterogeneity in program development and implementation are apparent, with limited emphasis on study design, sample size, and validated instruments to assess outcomes [10,12,13].
As part of the Prevention for States initiatives in collaboration with the Illinois Prescription Monitoring Program (ILPMP), the research team successfully implemented an opioid-focused AD program in a large health care system in the Chicago region during the summer of 2018 [14]. Subsequently, we sought to pilot a program that incorporated virtual AD implemented among health care providers located in the southernmost counties of Illinois. The primary objective of our study was to assess the feasibility and acceptability of virtual AD visits among health care provider participants [15]. Within this study, feasibility was defined as the extent to which virtual detailing could be successfully implemented, i.e. participation in an AD program via response rates, and acceptability was how well it was received by them, i.e. as measured by a provider satisfaction with the AD visit. The study aimed to compare: (1) provider satisfaction and detailer assessment for the in-person and the follow up virtual AD visits, (2) provider satisfaction and detailer assessment for virtual AD visits with and without video conferencing and screen sharing capabilities, (3) provider satisfaction when technical difficulties were reported during the virtual visit, and (4) provider satisfaction by survey collection method.
2. Material and methods
Healthcare providers with prescriptive authority practicing in the 31 southernmost counties of Illinois were recruited to voluntarily participate in an AD program focused on appropriate opioid prescribing from November 2018 through June 2019. Health care providers’ offices were located on average 300 miles away from the study team site based in Chicago. Because of the geographic distance, the program was designed to deliver the follow-up AD visit using WebEx (Cisco Systems Inc., San Jose, CA) no later than eight weeks after the initial in-person AD visit. The WebEx platform includes videoconferencing and screen sharing, components that help to simulate in-person face-to-face interactions that occur in traditional AD. Telephone calls were utilized as a backup. Feasibility was operationalized by measuring the extent to which scheduled virtual visits were completed and the quality of the virtual visits through the Detailer Assessment of Visit Effectiveness (DAVE), a validated instrument that can be used to evaluate AD in lieu of direct clinician feedback [16]. Acceptability was operationally measured using the Provider Satisfaction with Academic Detailing (PSAD) instrument, a 9-item validated measure for assessing provider satisfaction with an AD visit [17].
The academic detailers were 22 clinical pharmacists from the University of Illinois at Chicago College of Pharmacy who were trained in AD. The detailers were trained by research team members who completed formal training from National Resources Center for Academic Detailing in April 2018. The standardized training included presentations related to AD, discussions on program aims and logistics (i.e., scheduling visits and traveling details), and procedures for administering and filling out the instruments. Training also included visit simulations where detailer skills were assessed.
Materials discussed with the providers across the two visits included: (1) six key messages from the Centers for Disease Control and Prevention (CDC) Guideline for Prescribing Opioids for Chronic Non-Cancer Pain [18]; (2) individual provider opioid prescribing metrics (obtained from the ILPMP); and (3) additional resources that addressed provider questions and barriers to practice (e.g., brochures on Illinois naloxone standing order, list of local addiction treatment centers, etc.). The second visit covered different material related to opioid prescribing not covered in the initial visit that was identified as relevant to the prescriber [19]. The detailer also followed up on any provider questions and suggested action by the detailer to the provider based on the conversation during the first visit. To increase provider rapport and trust, every effort was made to keep the same detailer across the two visits.
After each visit, the provider was asked to complete the PSAD. In the first visit, the PSAD was completed in-person at the time of the visit by self-report using paper and pen was returned to the detailer prior to leaving the clinician’s office. During the second visit, the survey was administered electronically. To simulate the first visit, the PSAD survey was embedded within the WebEx meeting to be administered towards the end of the call. In case the visit was conducted via telephone, a survey link was emailed to the provider after the call. The detailers also completed DAVE electronically after each visit. The item response for both the validated instruments was based on a Likert-type scale, with “not at all” = 1 to “extremely” = 5. The 9-item PSAD instrument was scored by summarizing all item responses with a maximum score of 45. The DAVE was scored by summarizing the first three items, with a maximum score of 15, and two individually reported items. Detailers also documented their interaction with the provider using field notes. During the follow-up virtual visit, the detailers also recorded the experience interfacing with the WebEx platform (i.e., technical difficulties experienced; whether telephone use was needed).
The data was analyzed using SAS version 9.4 (SAS Institute, North Carolina). Provider characteristics were analyzed using the Chi-square test for categorical variables and paired t-test for continuous variables. Nonparametric (i.e., Mann-Whitney U and Wilcoxon Signed Ranks tests) and parametric (i.e., Independent and paired t-tests) statistical tests were used to evaluate provider satisfaction and detailer visit assessment scores. To compare satisfaction across the two visits, instrument responses were analyzed only for providers who participated in both visits and completed both surveys in their entirety. Additional analysis was done to evaluate provider satisfaction scores among different types of virtual visits conducted (e.g., visits conducted where screen sharing & video possible vs. visits with telephone only).
The providers could claim up to 0.5 Continuing Education (CE) credits per visit by participating. The study was approved by the University of Illinois at Chicago’s Institutional Review Board. Provider consent for participation was obtained during the initial in-person visit.
3. Results
Out of 141 appointments scheduled, 127 (90.1 %) initial in-person visits were completed (Table 1 ). Of the 120 follow up visits scheduled, 92 (76.7 %) virtual AD visits were completed, resulting in a 72.4 % retention rate. Differences in visit completion were not statistically significant (p = 0.38). There were 122 (96.1 % response rate) initial provider satisfaction surveys collected from the 127 in-person visits conducted. This was statistically different than the 56 (60.8 %) electronic surveys collected from the 92 virtual visits conducted (p = 0.03). Provider characteristics were similar across both visits (visit 1, visit 2), with Doctor of Medicine providers (63.0 %, 60.9 %) and Nurse Practitioners (25.2 %, 29.4 %) representing the majority of participating providers (p = 0.63), an almost equal proportion of males and females(p = 0.90), and over half of providers (62.2 %, 70.6 %) from specialties related to primary care (p = 0.18). The average length of the first visit was approximately 19 min (SD = 9.0), while the average follow-up visit lasted 12 min (SD = 7.1), which was significantly shorter (p < 0.001).
Table 1.
Provider and Visit Characteristics.
| visit 1 | visit 2 | p-values | |
|---|---|---|---|
| In-person | Virtual | ||
| (n = 127) | (n = 92) | ||
| Scheduled visits (n) | 141 | 120 | |
| Completed visits, n (%) | 127 (90.1) | 92 (76.7) | 0.38 |
| Provider Survey Responsea, n (%) | 122 (96.1) | 56 (60.8) | 0.02 |
| Length of visit (in minutes), mean (SD) | 19.3 (9.0) | 12.4 (7.1) | <0.01 |
| Prescriber Characteristics, n (%) | |||
| Primary Care Providersb | 100 (78.7) | 79 (85.9) | 0.18 |
| Non-Primary Care Providersc | 27 (22.3) | 13 (14.1) | |
| Male | 66 (52.0) | 47 (51.1) | 0.90 |
| Female | 61 (48.0) | 45 (48.9) | |
| Doctor of Medicine | 80 (63.0) | 56 (60.9) | 0.63 |
| Nurse Practitioner | 32 (25.2) | 27 (29.4) | |
| Physician Assistant | 12 (9.4) | 9 (9.8) | |
| Doctor of Osteopathic Medicine | 1 (0.8) | 0 (0.0) | |
| Otherd | 2 (1.6) | 0 (0.0) | |
| Years of Practice, mean (SD) | 13.7 (11.2) | 12.9 (11.1) | <0.01 |
p-values were calculated at a significance level of α = 0.05.
For visits 1 and 2, there were 115 and 54 surveys, respectively, that were completed in their entirety (i.e., no missing item responses).
Primary care providers included the following specialties: Family Medicine (n=63), Internal Medicine (n=16), Pediatrics (n=17), OBGYN (n=1), and Women’s health (n=1).
Non-Primary care providers included the following specialties: Allergy/Immunology (n=1), Behavioral Health (n=6), Ear Nose and throat (n=1), Gastroenterology (n=2), Hematology/Oncology (n=4), Neurology (n=3), Podiatry (n=1), Rheumatology (n=1), Surgery (n=3), Urology (n=5).
Other provider types include DPM (Doctor of Podiatric Medicine) and LCPC (Licensed Clinical Professional Counselor).
Non-parametric and parametric tests resulted in the same statistical conclusions. Therefore, parametric results (i.e. independent and paired t-tests) were reported [20]. Providers indicated high levels of satisfaction with the initial and follow up AD visit (Table 2 ). Fifty-six follow-up provider surveys were collected, but two were incomplete. Additionally, four providers did not fill out an initial visit survey. Among providers who received both visits and completed both surveys (n = 50), satisfaction was higher in the first visit compared with the follow up (mean = 41.9 (SD = 4.0) vs. 39.0 (5.9), p < 0.001). There was no significant difference across the two visits in detailer perception of the visit effectiveness as seen in the scale summary score (9.9 (3.4) vs. 9.7 (2.2), p = 0.86) and two individually reported items related to feasibility (3.44 (1.3) vs. 3.3 (1.1), p = 0.72) and conversation (4.2 (0.7) vs. 4.3 (0.7), p = 0.63). Among the 92 follow up virtual AD visits, 41 (44.6 %) were conducted using a medium where videoconferencing and screen-sharing for detailing content was possible, while 51 (55.4 %) of the calls were conducted using a telephone.
Table 2.
Provider Satisfaction with Academic Detailing (PSAD) and Detailer Assessment of Visit Effectiveness (DAVE) for In-person and Virtual Academic Detailing Visits.
| Providers who received both visits |
||||||||
|---|---|---|---|---|---|---|---|---|
| (n = 50)a | ||||||||
| Visit 1 |
Visit 2 |
Visit 1 | Visit 2 | Mean Diff (95 % CI) | ||||
| In-person |
Virtual |
In-person | Virtual | |||||
| Provider Satisfaction with Academic Detailing (PSAD) | n | mean (SD) | n | mean (SD) | mean (SD) | mean (SD) | p-value | |
| The detailer was knowledgeable | 122 | 4.74 (0.45) | 56 | 4.60 (0.56) | 4.82 (0.38) | 4.60 (0.57) | −0.22 (−0.39, −0.06) | 0.01 |
| The detailer was an effective communicator | 121 | 4.84 (0.36) | 56 | 4.57 (0.53) | 4.88 (0.32) | 4.56 (0.54) | −0.30 (−0.45, −0.15) | <0.01 |
| AD is an effective way to get updated on important topic(s) | 122 | 4.63 (0.59) | 56 | 4.35 (0.90) | 4.66 (0.68) | 4.38 (0.85) | −0.30 (−0.52, −0.07) | 0.01 |
| The printed material was useful | 120 | 4.49 (0.76) | 55 | 4.05 (0.80) | 4.64 (0.63) | 4.06 (0.79) | −0.58 (−0.82, −0.33) | <0.01 |
| I would be receptive to future visits | 122 | 4.52 (0.89) | 56 | 3.91 (1.19) | 4.56 (0.83) | 3.94 (1.13) | −0.67 (−0.96, −0.38) | <0.001 |
| This topic was relevant to my practice | 121 | 4.34 (1.12) | 55 | 4.09 (1.23) | 4.60 (0.94) | 4.14 (1.19) | −0.48 (−0.78, −0.19) | 0.00 |
| This is an important topic | 121 | 4.77 (0.55) | 56 | 4.71 (0.59) | 4.76 (0.51) | 4.74 (0.56) | −0.06 (−0.23, 0.12,) | 0.52 |
| The key messages are feasible to implement in my practice | 118 | 4.40 (0.97) | 56 | 4.10 (1.07) | 4.46 (0.95) | 4.18 (1.00) | −0.32 (−0.64, 0.00) | 0.05 |
| The key messages were consistent with my practice | 121 | 4.52 (0.85) | 55 | 4.39 (0.82) | 4.54 (0.86) | 4.38 (0.83) | −0.19 (−0.47, 0.10) | 0.20 |
| Summary Score | 115 | 41.3 (4.70) | 54 | 38.7 (6.06) | 41.9 (3.96) | 39.0 (5.88) | −2.94 (−4.38, −1.50) | <0.001 |
| Detailer Assessment of Visit Effectiveness (DAVE) | ||||||||
| The visit was informative/useful to the provider | 127 | 3.22 (1.05) | 92 | 3.25 (0.94) | 3.42 (1.05) | 3.34 (0.98) | −0.03 (−0.27, 0.20) | 0.78 |
| The provider is willing to implement the key points | 127 | 3.24 (1.23) | 92 | 3.42 (1.04) | 3.44 (1.26) | 3.64 (1.04) | 0.07 (−0.15, 0.28) | 0.56 |
| The provider is likely to change his/her/their practice as a result of this visit | 127 | 2.66 (1.23) | 92 | 2.86 (1.09) | 3.00 (1.27) | 2.74 (1.08) | 0.02 (−0.33, 0.37) | 0.90 |
| Scale Summary Score | 127 | 9.14 (3.24) | 92 | 9.54 (2.07) | 9.86 (3.38) | 9.72 (2.21) | 0.05 (−0.56, 0.66) | 0.86 |
| It is feasible for the provider to implement the key points | 126 | 3.15 (1.26) | 92 | 3.35 (1.09) | 3.44 (1.30) | 3.30 (1.09) | 0.07 (−0.29, 0.42) | 0.72 |
| The conversation went smoothly | 127 | 4.25 (0.66) | 92 | 4.16 (0.80) | 4.18 (0.66) | 4.32 (0.68) | −0.05 (−0.28, 0.17) | 0.63 |
p-values were calculated at a significance level of α = 0.05.
Statistical analysis was only evaluated among providers who received both visits and completed both surveys.
There was no difference in provider satisfaction between the two types of calls conducted (38.7 (6.0) vs. 38.7 (6.2), p = 0.97) (Table 3 ). Furthermore, there was no difference in the summary score for the detailer’s perception of visit effectiveness (9.6 (1.8) vs. 9.5 (2.3), p = 0.79).
Table 3.
The Impact of Optimal Video Conferencing Technology on Provider Satisfaction with Academic Detailing (PSAD) and Detailer Assessment of Visit Effectiveness (DAVE).
| Was video conferencing and screen sharing possible during the call? |
||||||
|---|---|---|---|---|---|---|
| yesa |
nob |
Mean Diff (95 % CI) | p-value | |||
| Provider Satisfaction with Academic Detailing (PSAD) | n | mean (SD) | n | mean (SD) | ||
| The detailer was knowledgeable | 33 | 4.57 (0.56) | 23 | 4.65 (0.57) | −0.08 (−0.38, 0.23) | 0.62 |
| The detailer was an effective communicator | 33 | 4.57 (0.50) | 23 | 4.56 (0.58) | 0.01 (−0.28, 0.3) | 0.94 |
| AD is an effective way to get updated on important topic(s) | 33 | 4.39 (0.86) | 23 | 4.30 (0.97) | 0.09 (−0.41, 0.59) | 0.72 |
| The printed material was useful | 33 | 4.00 (0.79) | 22 | 4.13 (0.83) | −0.14 (−0.58, 0.31) | 0.54 |
| I would be receptive to future visits | 33 | 3.87 (1.21) | 23 | 3.95 (1.18) | −0.08 (−0.73, 0.58) | 0.81 |
| This topic was relevant to my practice | 32 | 4.12 (1.18) | 23 | 4.04 (1.33) | 0.08 (−0.60, 0.77) | 0.81 |
| This is an important topic | 33 | 4.72 (0.57) | 23 | 4.69 (0.63) | 0.03 (−0.29, 0.36) | 0.85 |
| The key messages are feasible to implement in my practice | 33 | 4.15 (1.03) | 23 | 4.04 (1.14) | 0.11 (−0.48, 0.70) | 0.71 |
| The key messages were consistent with my practice | 32 | 4.36 (0.90) | 23 | 4.43 (0.73) | −0.07 (−0.52, 0.38) | 0.75 |
| Summary Score | 32 | 38.7 (6.05) | 22 | 38.7 (6.22) | 0.07 (−3.47, 3.33) | 0.97 |
| Detailer Assessment of Visit Effectiveness (DAVE) | ||||||
| The visit was informative/useful to the provider | 41 | 3.21 (0.90) | 51 | 3.27 (0.98) | −0.06 (−0.45, 0.34) | 0.78 |
| The provider is willing to implement the key points | 41 | 3.48 (0.95) | 51 | 3.37 (1.11) | 0.12 (−0.32, 0.55) | 0.60 |
| The provider is likely to change his/her/their practice as a result of this visit | 41 | 2.90 (1.13) | 51 | 2.84 (1.06) | 0.06 (−0.4, 0.52) | 0.80 |
| Scale Summary Score | 41 | 9.60 (1.82) | 51 | 9.49 (2.27) | 0.12 (−0.75, 0.99) | 0.79 |
| It is feasible for the provider to implement the key points | 41 | 3.26 (1.16) | 51 | 3.43 (1.04) | −0.16 (−0.62, 0.29) | 0.48 |
| The conversation went smoothly | 41 | 4.34 (0.61) | 51 | 4.01 (0.90) | 0.32 (−0.01, 0.65) | 0.06 |
p-values were calculated at a significance level of α = 0.05.
This includes visits where the provider joined the virtual visit using 1) computer for both audio and video (n−8); 2) computer for video and telephone for audio (n−9); and 3) WebEx mobile application for both audio and video.
Providers joined the virtual visit only using the telephone, where video and screen sharing the academic detailing visit materials was not possible.
Almost half of the time (45 of 92 visits), the detailers reported there was difficulty experienced by either themselves or by the provider during the virtual AD visit. There were 16 visits where the detailer self-reported technical challenges experienced, 13 visits where the detailer reported technical challenges experienced by the provider, and 16 visits where the detailer reported both. Among providers who completed the PSAD survey, there was no significant difference in provider satisfaction when any technological difficulties were present (38.7 (5.6) vs. 38.6 (6.6), p = 0.93) (Table 4 ).
Table 4.
The Impact of Technological Challenges as Reported by Detailers on Provider Satisfaction with Academic Detailing (PSAD).
| Were there any technical difficulties?a |
||||||
|---|---|---|---|---|---|---|
| No |
Yesb |
Mean Diff (95 % CI) | ||||
| Provider Satisfaction with Academic Detailing (PSAD) | n | mean (SD) | n | mean (SD) | p-value | |
| The detailer was knowledgeable | 28 | 4.57 (0.57) | 28 | 4.64 (0.56) | −0.07 (−0.37, 0.23) | 0.64 |
| The detailer was an effective communicator | 28 | 4.50 (0.58) | 28 | 4.64 (0.49) | −0.14 (−0.43, 0.14) | 0.32 |
| AD is an effective way to get updated on important topic(s) | 28 | 4.36 (0.91) | 28 | 4.36 (0.91) | 0.00 (−0.49, 0.49) | 1.00 |
| The printed material was useful | 27 | 4.07 (0.68) | 28 | 4.04 (0.92) | 0.04 (−0.4, 0.48) | 0.86 |
| I would be receptive to future visits | 28 | 4.00 (1.09) | 28 | 3.82 (1.31) | 0.18 (−0.47, 0.82) | 0.58 |
| This topic was relevant to my practice | 28 | 4.18 (1.16) | 27 | 4.00 (1.33) | 0.18 (−0.49, 0.85) | 0.60 |
| This is an important topic | 28 | 4.75 (0.52) | 28 | 4.68 (0.67) | 0.07 (−0.25, 0.39) | 0.66 |
| The key messages are feasible to implement in my practice | 28 | 4.04 (1.00) | 28 | 4.18 (1.16) | −0.14 (−0.72, 0.44) | 0.62 |
| The key messages were consistent with my practice | 28 | 4.32 (0.82) | 28 | 4.46 (0.84) | −0.14 (−0.59, 0.30) | 0.90 |
| Summary Score | 27 | 38.70 (5.59) | 27 | 38.67 (6.60) | −0.04 (−3.30, 3.38) | 0.98 |
p-values were calculated at a significance level of α = 0.05.
Technical difficulties were reported by the detailer.
There were 11 visits where the detailer self-reported technology difficulties, 7 visits where the detailer reported only provider experienced technology difficulties, and 9 visits where the detailer reported both kinds of technical difficulties.
There were 18 (33.3 %) surveys collected toward the end of the virtual call using the WebEx polling function, and 36 (66.7 %) collected using a survey link sent via email after the call ended (Table 5 ). How satisfaction was collected did not significantly impact provider satisfaction (38.7 (5.6) vs. 38.7 (6.6), p = 0.98).
Table 5.
The Impact of Survey Administration on Provider Satisfaction with Academic Detailing (PSAD).
| Google poll |
Web-Ex |
Mean Diff (95 % CI) | ||||
|---|---|---|---|---|---|---|
| Provider Satisfaction with Academic Detailing (PSAD) | n | mean (SD) | n | mean (SD) | p-value | |
| The detailer was knowledgeable | 38 | 4.58 (0.6) | 18 | 4.67 (0.49) | −0.09 (−0.41, 0.24) | 0.59 |
| The detailer was an effective communicator | 38 | 4.53 (0.56) | 18 | 4.67 (0.49) | −0.14 (−0.45, 0.17) | 0.36 |
| AD is an effective way to get updated on important topic(s) | 38 | 4.21 (0.96) | 18 | 4.67 (0.69) | −0.46 (−0.96, 0.05) | 0.08 |
| The printed material was useful | 37 | 3.95 (0.85) | 18 | 4.28 (0.67) | −0.33 (−0.79, 0.13) | 0.15 |
| I would be receptive to future visits | 38 | 3.84 (1.22) | 18 | 4.06 (1.16) | −0.21 (−0.9, 0.48) | 0.54 |
| This topic was relevant to my practice | 37 | 4.00 (1.31) | 18 | 4.28 (1.07) | −0.28 (−0.99, 0.44) | 0.44 |
| This is an important topic | 38 | 4.74 (0.55) | 18 | 4.67 (0.69) | 0.07 (−0.27, 0.41) | 0.68 |
| The key messages are feasible to implement in my practice | 38 | 4.05 (1.14) | 18 | 4.22 (0.94) | −0.17 (−0.79, 0.45) | 0.59 |
| The key messages were consistent with my practice | 38 | 4.50 (0.69) | 18 | 4.17 (1.04) | 0.33 (−0.14, 0.80) | 0.16 |
| Summary Score | 36 | 38.19 (6.19) | 18 | 39.67 (5.83) | −1.47 (−4.99. 2.05) | 0.82 |
p-values were calculated at a significance level of α = 0.05.
4. Discussion
Our study found that virtual AD visits had lower scheduled visit completions than in-person visits, though these differences were not significant. Additionally, technological difficulties and resorting to less favorable communication mediums occurred frequently. Furthermore, there were significant challenges in gathering provider surveys associated with virtual visits than in-person visits.
There was a high level of satisfaction with both visits indicating that in-person and virtual detailing visits were well accepted among those who completed a provider satisfaction survey. Providers reported slightly lower satisfaction with the follow-up virtual visit conducted using either the teleconferencing application WebEx or a telephone call. However, the difference cannot be attributed solely to the change in AD delivery (i.e., technology-based vs. in-person) since other factors may have contributed to the change, such as the specific information discussed at the visit or differences in survey administration technique. Provider satisfaction during the virtual visits was not impacted by the type of technology used nor by technological difficulties if encountered. Furthermore, detailers' perception of the AD visit effectiveness was not statistically different across the two visits, nor was it impacted by needing to use telephone vs. WebEx, which may suggest that detailers perceived the visits were effective regardless of the medium used to conduct the visits.
The strengths of our study include that all providers received an initial in-person visit that was intended to help establish an initial relationship and rapport with the provider, which has been previously recommended [10,12]. Our study utilized two novel psychometrically validated instruments to assess provider satisfaction and detailer perception of AD visits [16,17]. The study also attempted to evaluate differences among the virtual visit (i.e., video conferencing available versus telephone only) and if technical challenges were encountered since such nuances may affect the quality of interaction between detailer and provider. Since the statistical conclusions were the same across parametric and non-parametric tests, we choose to report parametric results (i.e. independent and paired t-tests) to avoid committing a type II error (i.e., there is no difference in provider satisfaction when there truly is one). This seems appropriate given that parametric tests are sufficiently robust when sample sizes greater than 10 per group despite non-normal distributions present [20].
There were several limitations with our study. The providers who participated in the AD visits may be systematically different than providers who declined to participate or dropped out after the visit. Therefore, our findings may not be generalizable across all providers. Retention across the two visits may have been influenced by the provider’s experience with the first in-person visit. A post hot analysis found that compared to those who were retained, providers who dropped reported AD topic presented was less relevant (3.92 (1.3) vs. 4.5 (0.9), p < 0.05) though overall satisfaction scores (40.0 (5.8) vs. 41.8 (4.1), p = 0.05) was not statistically different. Therefore, since these providers who dropped felt that the AD topic was less relevant to their practice, they likely were less inclined to participate in a follow visit, regardless of its medium.
Our pilot study was also not limited to primary care providers, who account for nearly half of the opioid prescriptions prescribed in the United States due to concerns about the limited sample size [21]. However, additional analysis showed no difference in the initial visit satisfaction between PCPs and non-PCPs who participated (41.5 (4.7)vs. 40.4 (4.7), p = 0.27), and among those who were retained (41.8 (4.0) vs. 41.5 (4.3), p = 0.83) though sample sizes were small in the non-PCP group (n = 11). These findings may suggest that providers’ decision to drop out may be due to individual preferences and not necessarily related to specialty. Furthermore, there was a higher retention rate among primary care providers (79 %) and was comparable to the retention rate in the parallel opioid AD program aimed at PCPs (p = 0.66) [14].
Another limitation was that the second visit PSAD response rate was significantly lower than the near-complete response rate for the first visit, which could have led to nonresponse bias. The authors believe the higher nonresponse was due to how the survey was administered and collected during the second visit. If a survey link had to be emailed after the call, it would require more effort for the provider to fill it out during their own time. It is worth noting that 51 of the 54 (96 %) completed surveys were submitted by PCPs. This means that more than two-thirds of the PCPs who participated in the virtual visit provided a completed PSAD survey.
Additionally, more than half of the follow-up calls were conducted via telephone, which does not simulate the traditional face-to-face interaction between the detailer and provider as intended. We agree with other studies that suggest video-enabled digital communication as a preferred tool because it emulates the in-person interaction critical for AD [1,10,12]. However, our findings show provider satisfaction may not be significantly impacted in situations necessitating using a less favorable technology. Furthermore, technological challenges present during the virtual visit do not appear to significantly impact the provider satisfaction of the virtual AD visit.
Also, the study was not powered due to small sample sizes in some of the sub-analysis to detect a difference in satisfaction and perceived AD visit effectiveness. Therefore, there could have been a difference in provider satisfaction and detailer assessment of AD, but we were unable to detect it. However, the observed differences in satisfaction across comparator groups were relatively small and may not be consequential even if statistical significance was detected. Nevertheless, when there were sufficient sample sizes, parametric tests resulted in similar findings [20]. Finally, while almost half of the visits experienced some kind of technical challenge, specific details related to those challenges were not captured. Potential challenges might include providers’ unfamiliarity with the technology and internet connectivity issues. Detailers also self-reported experiencing technical difficulties over a third of the time.
Scheduling longer visit times in anticipation of potential technical problems may improve the implementation of virtual AD. Similar to in-person visits, virtual visits were scheduled for 15–20 min, which afforded little time for troubleshooting issues with technology if they arose. Providing clinicians with “how-to” documents on operating the teleconference platform may preclude difficulties resulting from user unfamiliarity. Training detailers to troubleshoot technical issues efficiently would equip them with skills to quickly resolve any anticipated technology problems. Exploring the reasons for virtual visit cancelations will be informative. Perhaps providers perceive it to be less socially acceptable to cancel an in-person appointment last minute, especially if the detailer is already waiting at their office, than if the meeting is virtual. Finally, finding ways to mimic the ease and immediacy of electronic survey collection may improve survey response rates with virtual visits. Some of these considerations have been incorporated into a current AD study (CDC grant number: 5R01CE003156-02) that has pivoted to virtual detailing due to the coronavirus disease of 2019 (COVID-19) pandemic.
Virtual AD may be an attractive alternative compared to traditional in-person AD during the current COVID-19 pandemic that requires social distancing [[22], [23], [24]]. Successful virtual AD programs will need to incorporate strategies addressing the challenges identified in our study. However, in the current COVID-19 climate, since many providers and health systems have been forced to utilized telemedicine, some of these challenges due to user inaccessibility and unfamiliarity with technology may be less prevalent [11,22]. Therefore, due to the necessity warranted by this pandemic, it may be much easier to implement a virtual AD program than before [22]. AD has developed and refined over several decades and recently became a key strategy used to educating clinicians about opioid prescribing and appropriate pain management. Integration of AD into health policy has occurred in states like Illinois, which mandated educational outreach to providers (i.e. 305 ILCS 5/12-4.52 new) starting in 2020. However, the global pandemic due to COVID-19 has required a rethinking of approaches to educational outreach, particularly AD. Virtual detailing is an option to traditional face-to-face AD in an era of social distancing due to pandemics such as COVID-19 and when resources are limited. Our findings are important to inform future designs and delivery of AD programs. Future research should extend the examination of virtual AD program effectiveness to prescribing behavior and patient outcomes.
Ethics approval
Approved by the University of Illinois at Chicago institutional review board.
Disclosure
The views expressed in this manuscript are those of the authors and do not necessarily represent the views of those acknowledged or our funding sources.
Funding
This work was supported in part by the Centers for Disease Control and Prevention: Prevention for States (PfS) via the Illinois Department of Human Services and Illinois Prescription Monitoring Program (Grant#: 49I1099M40, 2018) and the Centers for Disease Control and Prevention (Grant #: R01CE003156-01-00, 2019).
Summary table
What was already known on this topic
-
•
Academic detailing is an evidence-based method that can positively impact clinician behavior and improve patient outcomes
-
•
Virtual academic detailing has not been well studied.
What this study added to our knowledge
-
•
Using technology to conduct academic detailing visits is a possible alternative to traditional AD visits.
-
•
The scheduled visit completion rate was higher for in-person visits than virtual visits.
-
•
Provider satisfaction for both in-person and virtual AD visits was high.
-
•
For providers who received both visits, provider satisfaction was higher with the initial in-person visit though the difference in satisfaction cannot be attributed solely to the change in AD delivery.
-
•
Technical difficulties or the use of less favorable technology to conduct virtual visits may not affect the provider’s satisfaction with AD visit.
CRediT authorship contribution statement
Mary H. Smart: Conceptualization, Data curation, Project administration, Formal analysis, Visualization, Writing - original draft, Writing - review & editing. Monika Rao Mandava: Conceptualization, Data curation, Writing - original draft, Writing - review & editing. Todd A. Lee: Funding acquisition, Conceptualization, Writing - original draft, Writing - review & editing. A. Simon Pickard: Supervision, Funding acquisition, Conceptualization, Writing - original draft, Writing - review & editing.
Declaration of Competing Interest
The authors report no declarations of interest.
Acknowledgments
Illinois Academic Detailing Visits And New Evidence CEnter (Illinois ADVANCE).
Illinois Prescription Monitoring Program: Craig Berberet; Stan Murzynski; Ed Dowllar; Sarah Pointer, PharmD; Andrew Hollo.
Expert panel: David Liebowitz, MD; Scott Glaser, MD; Chris Herndon, PharmD; and Julie Adkins DNP.
Theodore Ross, MD, from Southern Illinois Healthcare Foundation; Keith Griffeth and Dana Leefers, PharmD, from the Hospital Sisters Health System; and Cathy Harte from Southern Illinois Healthcare.
Footnotes
Supplementary material related to this article can be found, in the online version, at doi:https://doi.org/10.1016/j.ijmedinf.2020.104365.
Appendix A. Supplementary data
The following is Supplementary data to this article:
References
- 1.Soumerai S.B., Avorn J. Principles of educational outreach (‘academic detailing’) to improve clinical decision making. JAMA. 1990;263(4):549–556. [PubMed] [Google Scholar]
- 2.Carroll J.J., Green T.C., Noonan R.K. 2018. Evidence-Based Strategies for Preventing Opioid Overdose: What’s Working in the United States: an Introduction for Public Heath, Law Enforcement, Local Organizations, and Others Striving to Serve Their Community. [Google Scholar]
- 3.Chhina H., Bhole V.M., Goldsmith C., Hall W., Kaczorowski J., Lacaille D. Effectiveness of academic detailing to optimize medication prescribing behaviour of family physicians. J. Pharm. Pharm. Sci. 2013;16(4):511–529. doi: 10.18433/j3kk6c. [DOI] [PubMed] [Google Scholar]
- 4.Kisuule F., Wright S., Barreto J., Zenilman J. Improving antibiotic utilization among hospitalists: a pilot academic detailing project with a public health approach. J. Hosp. Med. 2008;3(1):64–70. doi: 10.1002/jhm.278. [DOI] [PubMed] [Google Scholar]
- 5.Larson M.J., Browne C., Nikitin R.V., et al. Physicians report adopting safer opioid prescribing behaviors after academic detailing intervention. Subst. Abus. 2018:1–7. doi: 10.1080/08897077.2018.1449175. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Siegel D., Lopez J., Meier J., et al. Academic detailing to improve antihypertensive prescribing patterns. Am. J. Hypertens. 2003;16(6):508–511. doi: 10.1016/s0895-7061(03)00060-8. [DOI] [PubMed] [Google Scholar]
- 7.Elnaem M.H., Nik Mohamed M.H., Huri H.Z. Pharmacist-led academic detailing improves statin therapy prescribing for Malaysian patients with type 2 diabetes: quasi-experimental design. PLoS One. 2019;14(9) doi: 10.1371/journal.pone.0220458. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Treloar C.J., Higginbotham N., Malcolm J., Sutherland D., Berenger S. An ‘Academic detailing’ intervention to decrease exposure to HIV infection among health-care workers. J. Health Psychol. 1996;1(4):455–468. doi: 10.1177/135910539600100404. [DOI] [PubMed] [Google Scholar]
- 9.National Resource Center for Academic Detailing (NaRCAD). Introductory Guide to Academic Detailing. https://www.narcad.org. (Accessed 18 June 2020).
- 10.Ho K., Nguyen A., Jarvis-Selinger S., Lauscher H.N., Cressman C., Zibrik L. Technology-enabled Academic Detailing: computer-mediated education between pharmacists and physicians for evidence-based prescribing. Int. J. Med. Inform. 2013;82(9):762–771. doi: 10.1016/j.ijmedinf.2013.04.011. [DOI] [PubMed] [Google Scholar]
- 11.Smith A.C., Thomas E., Snoswell C.L., et al. Telehealth for global emergencies: implications for coronavirus disease 2019 (COVID-19) J. Telemed. Telecare. 2020 doi: 10.1177/1357633X20916567. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Hartung D.M., Hamer A., Middleton L., Haxby D., Fagnan L.J. A pilot study evaluating alternative approaches of academic detailing in rural family practice clinics. BMC Fam. Pract. 2012;13 doi: 10.1186/1471-2296-13-129. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Baldwin L.M., Fischer M.A., Powell J., et al. Virtual educational outreach intervention in primary care based on the principles of academic detailing. J. Contin. Educ. Health Prof. 2018;38(4):269–275. doi: 10.1097/CEH.0000000000000224. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Lee T.A., Pickard A.S., Saffore C. Tailored approaches to opioid-related academic detailing of urban and rural prescribers. Paper Presented at: Rx Drug Abuse & Heroin Summit; April 24, Atlanta, GA; 2019. [Google Scholar]
- 15.Bowen D.J., Kreuter M., Spring B., et al. How we design feasibility studies. Am. J. Prev. Med. 2009;36(5):452–457. doi: 10.1016/j.amepre.2009.02.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Smart M., Monteiro A., Saffore C., et al. Development of an Instrument to Assess the Perceived Effectiveness of Academic Detailing. J Contin Educ Health Prof. 2020;40(4):235–241. doi: 10.1097/CEH.0000000000000305. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Monteiro A., Smart M., Saffore C., Lee T.A., Fischer M.A., Pickard A.S. Development of a measure of prescriber satisfaction with academic detailing. Value Health. 2019;22:S260. doi: 10.7573/dic.2021-9-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Dowell D., Haegerich T.M., Chou R. CDC guideline for prescribing opioids for chronic pain - United States, 2016. MMWR Recomm. Rep. 2016;65(1):1–49. doi: 10.15585/mmwr.rr6501e1. [DOI] [PubMed] [Google Scholar]
- 19.Saffore C.D., Tilton S.T., Crawford S.Y., et al. Identification of barriers to safe opioid prescribing in primary care: a qualitative analysis of field notes collected through academic detailing. Br. J. Gen. Pract. 2020;70(697):e589–e597. doi: 10.3399/bjgp20X711737. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Norman G. Likert scales, levels of measurement and the “laws” of statistics. Adv. Health Sci. Educ. Theory Pract. 2010;15(5):625–632. doi: 10.1007/s10459-010-9222-y. [DOI] [PubMed] [Google Scholar]
- 21.Levy B., Paulozzi L., Mack K.A., Jones C.M. Trends in opioid analgesic-prescribing rates by specialty, US, 2007–2012. Am. J. Prev. Med. 2015;49(3):409–413. doi: 10.1016/j.amepre.2015.02.020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Hoffman J.D., Shayegani R., Spoutz P.M., et al. Virtual academic detailing (e-Detailing): a vital tool during the COVID-19 pandemic. J. Am. Pharm. Assoc. (2003) 2020 doi: 10.1016/j.japh.2020.06.028. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Dedeilia A., Sotiropoulos M.G., Hanrahan J.G., Janga D., Dedeilias P., Sideris M. Medical and surgical education challenges and innovations in the COVID-19 era: a systematic review. In Vivo. 2020;34(3 Suppl):1603–1611. doi: 10.21873/invivo.11950. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.National Resource Center for Academic Detailing (NaRCAD). NaRCAD e-detailing Community of Practice. https://www.narcad.org/e-detailing.html. (Accessed 18 June 2020).
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.