Online Participation in a Pharmacy Residency Research Certificate Program

Conducting a research or quality improvement project is a key component of pharmacy residency training. In fact, the American Society of Health System Pharmacists (ASHP) Accreditation Standard for Postgraduate Year 1 and Post Graduate Year 2 (PGY1 and PGY2) Pharmacy Residency Programs includes Goal R2.2, which states residents must demonstrate the “ability to evaluate and investigate practice, review data, and assimilate scientific evidence to improve patient care and/or the medication-use system.” ¹ However, studies have identified deficits in knowledge and skills that may impact a resident’s ability to successfully complete a research project.^{2 -4} For instance, knowledge of study design and biostatistics appears to be low among residents, as demonstrated by an evaluation of more than 150 PGY1 pharmacy residents where the mean score on 10 items assessing this knowledge was only 47%. ³ Similarly, approximately 225 pharmacy residents were administered an 18-item test assessing research knowledge before and after residency training. ⁴ No improvements in scores were observed at the completion of residency.

In 2018, we created a pharmacy residency research certificate program aimed at providing residents at our institution with a more formalized approach to research training. This has been previously described. ⁴ In 2019, we expanded the program by offering it to residents in 2 nearby residency programs. Residents at our institution were required to attend in-person sessions, whereas residents at nearby programs participated online. The purpose of this article is to evaluate confidence and attitude toward research in residents participating in a research certificate program in-person versus online.

Our residency research certificate program was developed to provide residents with training related to the conduct, dissemination, and interpretation of research. ⁵ The program consists of seven 1-hr-long sessions covering topics such database management, data description, statistical test selection, statistical software, reporting research findings, medical writing, study design, and tips on incorporating research into a pharmacy career. Residents were required to attend or view all 7 sessions and score ≥80% on an end of the year assignment to earn a certificate.

For the 2019 to 2020 residency year, we designed our program in a way that would allow residents at nearby residency programs to participate. This involved uploading all material (eg, session handouts, readings) to an online learning management system and making session recordings available. These session recordings were offered through 2 modalities. First, a videoconference was set up for each session to allow participates to view and participate in the sessions in real time. Second, recordings from each session were posted to the online learning management system. While residents at our program were asked to attend each session in-person, participants at the nearby programs were not required to attend and could listen to session recordings either in real time or at their convenience. This was done because sessions were offered at our institution during a time that was already designated for resident education; however, this time was not reserved at other participating programs. Furthermore, asking participants to travel from their site to our center during this time block would have likely interfered with other residency responsibilities. All residents participating in the research certificate at other sites (ie, online) were PGY1s. Both PGY1 and PGY2 residents from our program participated in-person; however, only PGY1s were included in this analysis to make this in-person group more comparable with those participating online.

A postevaluation was sent to all participants. The postevaluation had 2 open-ended questions soliciting feedback on the strengths and areas for improvement for the program as well as several items assessing attitudes toward research (n = 2) and confidence (n = 6) in research-related skills. ³ Attitude and confidence items were assessed using a scale ranging from “strongly disagree” to “strongly agree.” The proportion of participants answering “agree” or “strongly agree” for each item was calculated and compared between those participating in-person versus online using chi square or Fisher exact tests. IBM SPSS, version 22.0 (IBM Corp., Armonk, New York) was used to perform statistical analyses and a P value <.05 was considered statistically significant. This project was considered program evaluation by our Institutional Review Board (IRB) and was thus exempt from IRB oversight.

Of the 15 PGY1 pharmacy residents at our institution, 14 participated in the research certificate program and were included in this analysis as the in-person cohort. The PGY1 that did not participate was in a 2-year residency program and chose to participate in the second year of residency. All 6 PGY1 residents at the nearby programs participated in the program and are included in this analysis as the online cohort.

Responses to the items assessing confidence and attitude toward research are shown in Figure 1. More than half of all respondents selected “agree” or “strongly agree” for 7 of the 8 items. Across all items, a numerically higher proportion of online participants answered “agree” or “strongly agree.” This was most pronounced for the question assessing residents’ confidence in discussing statistical methods with their colleagues, where 100% of online participants answered “agree” or “strongly agree” versus only 35.7% of in-person participants (P = .01). Written feedback on the certificate program was generally positive in both cohorts, with many residents commenting on the relevance of the program to their research projects and primary literature evaluation.

Figure 1.

Responses to items assessing attitude and confidence among online versus in-person participants.

Our pharmacy research certificate program was completed by 20 PGY1 residents, of which 14 participated in-person and 6 participated online. After completion of the program, those participating online answered questions assessing confidence and attitude toward research more favorably.

We hypothesize that the observed numerical differences between those participating online versus in-person were a result of the increased convenience of online participation. Online participation may have allowed residents to choose the place and time of learning, which could have improved the learning experience. Moreover, the recordings could be stopped or paused and restarted at any time. In contrast, residents participating in-person may have had a more difficult time shifting from patient care and other residency responsibilities for the hour block of time dedicated to each session.

Similar to our current evaluation, several studies have compared outcomes in courses offered in-person versus online.^{6 -8} For instance, a study assessed biostatistics knowledge among 58 participants who had a degree in medicine or science and were randomized to an onsite course arm (which was 3.5 days in duration) or an online course arm (which was 3.5 weeks in duration). ⁶ Increases in median scores on a biostatistics knowledge test from baseline to 3 months after the course were similar among those in the onsite versus online arms (16% versus 12%; P = .59). Similarly, Porter and colleagues evaluated performance before and after an online versus classroom immunization course among 140 pharmacy students and found no difference in grades between the 2 groups. ⁷ Moreover, Summers et al compared an in-person versus online statistics class among 38 nursing students and found no difference in grades between online versus in-person participation. ⁸ However, satisfaction was higher among in-person participants.

Our analysis has several limitations worthy of discussion. First, we had a small sample size, which limited our ability to detect statistical differences between the online and in-person groups. Nonetheless, our residency program is one of the largest in the country and a similar analysis with a larger sample size would not be possible at many programs; thus, we wanted to share our findings. Second, the in-person and online groups were from different residency programs and could have received different research training outside of the certificate course. However, we are not aware of any formal research training outside of our research certificate program at any of the participating residency programs. Third, we did not compare differences in knowledge or publication rates between groups, but this could be evaluated in future analyses.

In conclusion, we offered a residency research program that allowed for both online and in-person participation. Residents participating online answered attitude and confidence questions more positively than the in-person participants after completion of the certificate program, which may reflect increased convenience for those participating online.