Effectiveness of an Online Preceptor Development Program Across Health Professions

Abstract

Introduction

An online Preceptor Development Program (PDP) was developed to meet the needs of geographically dispersed preceptors across health professions. We aimed to measure the audience, their engagement, and effectiveness of an online PDP developed and implemented amid the COVID-19 pandemic.

Methods

The mixed methods study included survey and attendance data for live and asynchronous formats. T-tests compared overall session perception to self-reported session impact. Objective alignment and self-reported measures of impact were analyzed around Kirkpatrick’s levels of reaction, learning, and behavior.

Results

Participants engaged in live and/or asynchronous PDP sessions from various professions, specialties, and geographical locations. Quantitative findings indicated significant associations between overall session perception and knowledge (session 2), competence (sessions 1, 2), and performance (session 1). Objectives were met, and key learning takeaways were reported. While most participants indicated no barriers to implementation, two barriers identified were a lack of time and uninterested students.

Conclusion

Participants were afforded flexibility and choice and likely benefitted in the areas of knowledge, competence, and behavior. This online PDP effectively addressed preceptor needs in common areas, including feedback. Future program development may include advisory group input and interactive learning opportunities.

Introduction

Clinical preceptors are a vital component of clinical education programs. Enhancing the teaching skills of preceptors helps ensure that learners receive high-quality experiences in authentic clinical care settings. Preceptor development is a specialized form of faculty development, defined as “all activities health professionals pursue to improve their knowledge, skills, and behaviors as teachers and educators, leaders and managers, and researchers and scholars, in both individual and group settings” [1]. To help ensure the quality of these preceptor development opportunities, accrediting bodies (e.g., LCME [2], AOA [3]) establish standards for training clinical preceptors and evaluate institutions based on compliance with those standards.

While accrediting bodies require preceptor development, few recent peer-reviewed studies assess the effectiveness of the preceptor development program (PDP). Several healthcare fields have provided recent reviews of PDP offerings and consensus around key components. However, the implementation and effectiveness of these PDPs are not widely represented in peer-reviewed literature. This study addresses this gap by describing and researching one PDP conducted online.

Recent Preceptor Development Literature

Pharmacy and nursing are represented in several peer-reviewed studies on PDPs. A three-part pharmacy article series resulted in a National Preceptor Development Program Prototype [4], three reviews summarized pharmacy preceptor programs [5–7], a consensus of topics was produced by nursing professionals [8], and a multi-disciplinary online PDP was studied [9]. A small number of studies stem from medical specialties addressing the need to develop preceptors [10–12], while others address face-to-face (F2F) workshop offerings within the medical field [13–15]. One study described preceptor development across multiple health and medical professions (i.e., [9]).

The gaps in recent peer-reviewed research serve as foundational motivation for the present study. The PDP described in this study was offered across multiple health and medical professions, and we measured the program’s effectiveness, as defined by meeting the session objectives and participants’ perceived impact on their clinical teaching. Minor and colleagues described the role Virtual Faculty Development (VFD) has played in serving geographically dispersed faculty, enabling higher participation and invitation of external speakers without associated travel costs [16]. VFD has grown in prevalence in recent years, accelerated by the COVID-19 pandemic [16, 17]. This acceleration drives a need for up-to-date research around PDPs, in general and online.

Online Preceptor Development Programs

Online formats have played an important role in PDPs, including web-based modules, videoconferenced sessions, and online collaborative networks [6, 7]. These tools have supplemented F2F formats, enabling the extension of learning after the live delivery of the PDP. Examples include online self-assessment tools and professional development training portfolios [6]. Preference varies for F2F and fully online PDPs — with some authors noting the importance of F2F interaction for relationship development [18] while others state that F2F interaction may not be essential to creating an effective PDP [19]. Further study of PDP effectiveness is needed, including a comparison between F2F and online PDPs [7].

When analyzing their online PDP for physician assistants and nurse practitioners, Heusinkvelt and Tracy reported that more than half of the participants preferred the fully online format over other approaches, including blended, simulation, and F2F offerings [9]. Additionally, participants expressed interest in the self-paced format to counter the challenge of time constraints. Other online PDPs offered asynchronous content in the form of podcasts [20] or via a learning management system (LMS) to create a choose-your-own-adventure style of pharmacy preceptor development for possible use at the national level [4]. The PDP studied in this article is comparable to those offered entirely online, which seek to reach a geographically dispersed audience and provide choice and flexibility using live and asynchronous formats.

Purpose and Research Questions

The purpose of this study was to assess the initial impact of a seven-part online, module-based Preceptor Development Program (PDP) by addressing three research questions (RQ):

1. Who was the audience?

2. How did they engage in the PDP series?

3. Were the PDP sessions effective?

Session effectiveness was defined by meeting session objectives and indicating participants’ perceived impact on their clinical teaching.

Materials and Methods

Target Audience and Marketing Avenues

An initial preceptor development request came from the College of Osteopathic Medicine (COM) at Des Moines University (DMU), a Midwestern graduate health professions institution, targeted at a specified group of geographically dispersed preceptors. The Director for Continuing Medical Education (CME) then expanded the target audience to include all clinical programs at DMU and their associated degrees. Marketing materials for the online PDP were shared widely through DMU’s CME program.

Health Professions Fields

The health professions included in the target audience were based upon the DMU preceptor audience, including doctors of osteopathic medicine and allopathic medicine, physical therapists, physician assistants/associates, and doctors of podiatric medicine.

Preceptor Availability

This PDP was developed and implemented beginning in June 2020. Due to the COVID-19 Pandemic, it was anticipated that preceptors would have limited availability related to schedules, workload, time of day, access to materials, and the inability to attend in-person events.

Topic Selection and Delivery Format

Considering preceptors’ needs, availability, and speaker experience, an online PDP was developed to address topics of interest to preceptors. The seven-part PDP addressed the following topics:

1. Improving Feedback Using the ARCH Model

2. Five Micro-skills Model to Facilitate Learning in the Clinical Setting

3. Teaching in the Presence of Patients

4. Incorporating Teaching in a Busy Practice

5. Evaluating the Learner with Reliability and Validity

6. Preceptor-Student Boundaries

7. Teaching in the Hospital Setting

The abovementioned include the ARCH Model and the Five Microskills Model. The ARCH Model is a feedback model defined by four components: (1) Ask and Allow for self-assessment, (2) Reinforce thoughts and actions done well, (4) Confirm areas needing correction, and (5) Help the learner with the development and implementation of an improvement plan [21]. The Five Microskills model [22] is a model for facilitating the process of clinical thinking and involves five steps including “Get a commitment, (2) Probe for supporting evidence, (3) Teach general rules, (4) Reinforce what was done right, (5) Correct mistakes” (p. 420).

The sessions were offered live via videoconferencing software. The speaker primarily used a lecture format with a small amount of interactivity built in through questions and answers (synchronous) and quizzes (asynchronous). Each session lasted 30 min, with alternating noon and early evening sessions. Sessions were recorded and archived for asynchronous viewing within the CME program’s learning management system (LMS). As a result, participants could attend live sessions, view asynchronous recordings, or engage with some combination of these two formats throughout the series. Session attendance was voluntary, and participants received CME session by session. The live sessions of this series began in June 2020 and ended in January 2021. The asynchronous engagement data was obtained for this study between June 2020 and April 2021.

Evaluation of Sessions

Evaluation surveys were distributed electronically to all participants following the session. The evaluation survey response was voluntary and anonymous. The evaluation survey was developed to meet CME requirements and was later used for research purposes. Questions remained consistent between live and asynchronous surveys, enabling grouping and comparison of data. All survey items were identical for live and asynchronous audiences, except for one. In one survey, the term “practice” was used while in another the term “clinical teaching” was employed. These terms were considered synonymous and conceptually consistent, aligning with our audience and the PDP series content, which primarily focuses on clinical teaching as the primary targeted practice of the preceptors. Each session had 15 standard questions ranging from 3 to 8 session-specific objective alignment questions (Table 1, Appendix). Modified Kirkpatrick’s levels [21] were identified conceptually for relevant survey items (indicated in Table 1) for organizational and meaning-making purposes. The data analysis section provides more detail on how the modified Kirkpatrick’s levels were used for analysis.

Table 1.

Evaluation survey items

Question #	Please indicate the extent to which you agree with the following statements:	Response type	Modified Kirkpatrick level
1	The content was appropriate to my practice and teaching of students	Likert	Reaction (Level 1)
2	This activity will make me more effective in my practice (“practice” was replaced by “clinical teaching” for live evaluation)	Likert	Behavior – Self-Reported Changes (Level 3A)
3	This activity was balanced and free of commercial bias	Likert	*
4	If repeated, I would recommend this presentation to a colleague	Likert	Reaction (Level 1)
5	Quality of the instructional process and presentation including the effectiveness of educational methods	Likert	Reaction (Level 1)
6	Speaker's teaching effectiveness, knowledge, and organization	Likert	*
7	Speaker's ability to communicate ideas and information clearly	Likert	*
8 (multiple)	Met objective: [Session-specific objective listed here]	Likert	Reaction (Level 1)
9	Please describe any ‘pearls’ or takeaway messages	Open text-based response	Learning – Knowledge & Skills (Level 2B)
10	This educational activity will result in a change in my: (Knowledge, Competence, Performance, Patient Outcomes, Community, No change)	Multiple selections enabled six response options	Behavior – Self-Reported Changes (Level 3A)
11	Please note any changes or improvements in the care of your patients that you plan to make as a result of participating in this educational activity. If no changes are identified, please explain why (program format, content not appropriate, nothing learned, etc.)	Open text-based response	*
12	Please identify any barriers that you perceive in implementing these changes. Select all that apply (No barriers, Uninterested student, Lack of resources, Lack of administrative support, Lack of consensus or professional guidelines, Lack of time to teach medical students, Lack of time to assess/counsel patients, Patient compliance issues, Not applicable to my practice, Other)	Multiple selections enabled nine response options; Text entry for “other” option	Self-Reported Behavior (Level 3A)
13	How will you address these barriers in order to implement these changes in your practice?	Open text-based response	Self-Reported Behavior (Level 3A)
14	Please provide any additional comments and/or suggestions below	Open text-based response	*
15	Do you have any unanswered questions? If yes, please explain	Open text-based response	*
16	What topics would you like to see in upcoming preceptor development activities?	Open text-based response	*

All Likert scale items were single selection on a 5-point scale (1 = strongly disagree to 5 = strongly agree). Items marked with * were excluded from the analysis due to lack of clear alignment with research questions, lack of sufficient data for that item, or space limitations

Research Design and Data Collection

Qualitative and quantitative data were gathered through an electronic survey. We utilized an explanatory mixed methods analysis approach [24]. Causal-comparative research principles guided the study of group differences after the PDP was completed [25]. Data collection included live and asynchronous attendance, evaluation survey responses, and other participant demographics, including geographic location, profession, and specialty.

Data Analysis

Phase 1: Overall Session-Level Descriptive Statistics

Attendance data for all seven PDP sessions were reported for the type of session view (live or asynchronous), geographic location, and indicated medical or health professional. Phase 1 of the analysis addressed RQ1 (Who was the audience?) and RQ2 (How did they engage in the series?). DMU-specific preceptor data was collected through registration data based on the question: “Are you currently precepting DMU medical students?”.

Phase 2: Quantitative Analysis

Sessions with a survey response sample size below thirty (N < 30) were excluded from quantitative analysis, narrowing the analysis to sessions 1, 2, and 3. To measure session effectiveness (RQ3), phase 2 analysis compared measures of two types: perception of the session experience and anticipated effects of the session.

Perception of the Session Experience

We selected four Likert scale survey items that conceptually aligned with modified Kirkpatrick’s Levels 1: Reaction and 3A: Behavior – Self-Reported Changes (see Table 1: Q1, 2, 4, 5). Although we did not collect direct behavioral data, we included participants’ responses to the item related to anticipated effects of the session as an indicator of Level 3A in our analysis. This grouping of items allowed us to quantitatively estimate participants’ overall perception of the session and, thus provide an indication of “Overall Session Effectiveness”, as it included Level 1: Reaction (Q1, 4, 5) and Behavior – Self-Reported changes (Q2). For each session, we conducted a Cronbach’s alpha test to assess internal consistency (α > 0.80) within this group of responses. Subsequently, we calculated individual means for each participant in each session. We only included live participant response data in phase 2 since not all selected perception items were available within the asynchronous format.

Anticipated Effects of the Session

We compared the individual means with participants’ responses to an item, indicating whether the session was expected to result in a change in knowledge (K), competence (C), and/or performance (P). These changes were measured dichotomously (1 = anticipated effect, 0 = no anticipated effect).

Comparison of Measures

To gauge effectiveness, as defined by participants’ perception of the session, we compared two measures: the participants’ perception of the session experience and their expected session effects. We wanted to inspect if a group-level relationship was present between these two measures. If a statistical relationship emerged, it could imply that participants’ perceptions were linked to potential impacts on their practice. In essence, a positive perception coupled with anticipated impact might indicate the session’s effectiveness in influencing participants’ clinical teaching. To accomplish this, we conducted two-tailed t-tests (assuming unequal variance) to compare individual means for each session between those who reported no effect in K, C, and/or P and those who reported an impact in K, C, and/or P. A significant result from the t-test indicated that the observed difference was unlikely to have occurred by chance, suggesting an association between the perception of the session and the accompanying self-reported outcome. We considered the findings to be statistically significant if the p-value was less than 0.05.

Phase 3: Mixed Methods Analysis

Phase 3 addressed session effectiveness (RQ3) from a mixed methods perspective to further explain the quantitative findings. Likert scale objective data were reported as group means (5-point scale, range from 1 = strongly disagree to 5 = strongly agree). Likert scale and qualitative open response data relevant to modified Kirkpatrick’s levels 1 (Reaction), 2B (Learning – Knowledge & Skills), and 3A (Behavior – Self-Reported Changes) [21] were grouped based on level. Table 1 shows the relevant modified Kirkpatrick’s levels. While phase 2 excluded the asynchronous participants due to our selected grouping of items, phase 3 added the asynchronous data to further learn about overall session perceptions.

Descriptive statistics were utilized for Likert scale data. In addition, qualitative responses were inspected for representative examples to make meaning of the associated open-response survey items, mixing during analysis and interpretation [24].

Results

Three research questions (RQ) were addressed in our study: RQ1: Who was the audience?, RQ2: How did the audience engage in the PDP series?, and RQ3: Were the PDP sessions effective? Results from analysis phase 1 addressed RQs 1 and 2, while results from analysis phases 2 and 3 addressed RQ3.

RQ1: Who Was the Audience?

Number of Participants

The audience included 237 participants with a cumulative 338 session views across the 7-part series, including both live (288) and asynchronous views (50). Of the registrants, 108 reported they are currently precepting DMU students, 99 were not currently precepting DMU students, and 30 declined response. Further details about session attendance and types of engagement will be addressed in RQ 2 and Table 2.

Table 2.

Session views and general survey response rates

Session # (Time of day, CST)	Session title (Abbreviated title)	Total views	Live attendees	Asynch. views	# Survey respondents	Survey response rate
1 (Noon)	Improving Feedback Using the ARCH Model (ARCH)	100	91	9	43	44.0%
2 (Evening)	Five Microskills Model to Facilitate Learning in the Clinical Setting (5 Microskills)	48	36	12	37	77.1%
3 (Noon)	Teaching in the Presence of Patients Via Direct Observation (TPP)	51	47	4	37	72.6%
4* (Evening)	Incorporating Teaching into a Busy Practice (Busy Practice)	47	41	6	17*	36.2%
5* (Noon)	Evaluating the Student with Reliability and Validity (Evaluating)	40	35	5	12*	30.0%
6* (Evening)	Preceptor-Student Boundaries (Boundaries)	29	17	12	8*	31.0%
7* (Noon)	Teaching in the Hospital Setting (Hospital)	23	21	2	6*	30.4%

An asterisk (*) references a low sample size (N) for evaluation survey responses. These sessions were excluded from the session analysis and findings as a result

Professions and Specialties

Participants were from twelve professions, including Doctor of Osteopathic Medicine (DO), Doctor of Allopathic Medicine (MD), Physical Therapy (DPT, PT), Physician Assistant (PA), Registered Nurse (RN), Doctor of Podiatric Medicine (DPM), and Social Work. Professions marked as “other” included Academics/Education, Occupational Therapy, Administration, Public Health, Dietary/Nutrition, and Chiropractic. Each profession in the original target audience involved some participants in the sessions, with additional professions attending outside the target audience. Table 3 depicts viewership from the top five represented professions and how they engaged (under RQ2).

Table 3.

Attendance for top five represented professions

Session # (Time of day, CST)	Session title (Abbreviated title)	DO Total N (% synch)	MD Total N (% synch)	Nurse Total N (% synch)	PT Total N (% synch)	PA Total N (% synch)
1 (Noon)	Improving Feedback Using the ARCH Model (ARCH)	35 (91%)	36 (97%)	3 (100%)	3 (100%)	9 (78%)
2 (Evening)	Five Microskills Model to Facilitate Learning in the Clinical Setting (5 Microskills)	13 (85%)	15 (87%)	1 (100%)	2 (50%)	4 (50%)
3 (Noon)	Teaching in the Presence of Patients Via Direct Observation (TPP)	18 (94%)	19 (100%)	1 (0%)	0 (N/A)	4 (100%)
4 (Evening)	Incorporating Teaching into a Busy Practice (Busy Practice)	15 (87%)	11 (100%)	3 (67%)	7 (29%)	4 (75%)
5 (Noon)	Evaluating the Student with Reliability and Validity (Evaluating)	9 (89%)	13 (92%)	3 (67%)	7 (100%)	4 (75%)
6 (Evening)	Preceptor-Student Boundaries (Boundaries)	8 (88%)	7 (86%)	0 (N/A)	3 (33%)	2 (100%)
7 (Noon)	Teaching in the Hospital Setting (Hospital)	7 (71%)	8 (100%)	0 (N/A)	0 (N/A)	1 (100%)

All percentages were rounded (> or = 0.5 rounded up) to the nearest whole percent

Each participant reported one or more various specialties, including Family Medicine, Internal Medicine, Emergency Medicine, Paediatrics, Academics/Education, Podiatry, and Obstetrics. The figures within the following section provide a breakdown of views across the top five represented professions.

Geographic Locations

Participants reporting their location represented five countries, with 98% of participants joining from the United States. Twenty-three different United States were represented, with the host state holding the highest representation (117, 54%). In addition, two other Midwestern states were represented (30 cumulative participants, 14% of US participants), and 1 Western state (10 participants, 5% of US participants). Within the host state, participants came from cities dispersed across the state, primarily urban, with the highest from the host city (40, 44% of host state participants) and nearby suburbs (34 cumulatively from 6 suburbs, 38% of host state participants).

RQ2: How Did the Audience Engage in the PDP Series?

Participant Views

Of the 237 total participants in the PDP series, the number of sessions viewed varied, ranging from 1 to 5 out of 7. Two types of participant engagement emerged: those who viewed live sessions only and those who engaged in both live and asynchronous views throughout the series.

Session Views

The most viewed session was Session 1: Improving Feedback Using the ARCH Model, followed by four sessions with a range of 40–51 viewers. Further details on attendance for the entire series are depicted in Table 2, including an evaluation of survey respondents and response rate. Due to the low sample size of evaluation survey respondents in the last three sessions, the remainder of this paper will only focus on session-level findings from the first three sessions of the series. Table 3 shows the session data for the top five represented professions with synchronous viewership in parentheses. The most represented professions were medical (DO and MD), with nurses, physical therapists, and physician assistants in attendance in lower numbers. Since the host institution offers DO, PT, and PA programs, their attendance was expected. Nurses and MDs may precept DMU students or find the series valuable for mentoring students outside of DMU. Most of the professions had high synchronous viewership. Lower viewership in session four (PT) may indicate scheduling conflicts with the live session time or strong interest, leading attendees to watch asynchronously.

RQ3: Were the PDP Sessions Effective?

Quantitative Results: Overall Session Effectiveness

Overall session effectiveness was measured quantitatively by exploring the relationship between two measures: participants’ perceptions of the session experience and their anticipated effects of the session. Table 4 shows the group means of individuals’ session perception (Conceptual group of survey items Q1, 2, 4, 5) and associated Cronbach’s alphas for sessions 1, 2, and 3, along with the number of survey respondents reporting K, C, and/or P. A significant t-test result indicates that the group means difference was unlikely to have occurred by chance, so there was likely an association between the perception of the session and the accompanying self-reported impact. Only synchronous survey data was included for this section, as survey Q4 was absent from the asynchronous survey data.

Table 4.

Quantitative results: Overall session effectiveness

Session #: Abbreviated title (Live session participants only)	Overall reaction (Group mean of individual means)	“This educational activity will result in a change in my…” (Number and % affirmative responses of total N) — synch participants only	T-test p-value
1: ARCH (N = 36)	4.51 (SD = 0.65) α = 0.94	K: 26 (72%)	0.15
		C: 26 (72%)	0.01*
		P: 32 (89%)	0.07
2: 5 Microskills (N = 25)	4.52 (SD = 0.63) α = 0.88	K: 15 (60%)	0.02*
		C: 14 (56%)	0.02*
		P: 19 (76%)	0.25
3: TPP (N = 37)	4.45 (SD = 1) α = 0.83	K: 26 (70%)	0.23
		C: 24 (65%)	0.38
		P: 29 (78%)	0.33

Asynchronous participant response data was excluded, as survey item 4 was not present in the asynchronous survey. However, this exclusion was deemed appropriate as the sample size for asynchronous participant responses was low for sessions 1, 2, 3 (1: N = 8, 2: N = 13, 3: N = 0)

Cronbach’s alpha (α) is reported for each session’s survey participant responses to survey items 1, 2, 4, and 5. K refers to Knowledge, C refers to Competence, and P refers to Performance. Survey respondents could mark K, C, and/or P in their responses, so the numbers will not total to 100%

An asterisk (*) indicates a statistically significant t-test (p < 0.05)

Mixed Methods Results

We defined effectiveness as a combination of alignment with session objectives (Likert scale data) and change indicators related to three modified Kirkpatrick’s levels (Likert scale and open response data). Table 5 presents the Likert scale response means across all objectives for the first three sessions. Overall, most participants responding to the survey perceived that session objectives were met, indicating these were effective sessions. Survey data related to Kirkpatrick levels 1 and 2B are reported in Table 6. The set of data in Table 6 suggests that most participants perceived these sessions as appropriate, that they would recommend the content to others, and many had impactful takeaways. While the data on learning was limited, we included example themes to provide an idea of potential areas of learning that participants perceived during the session. Table 7 summarizes the session-level survey data related to Kirkpatrick level 3B.

Table 5.

Alignment with Session-Level Objectives

Session #	Abbreviated session title	Mean value “Met session objectives” Likert response (Range across items)	Mean % agree Likert response (Range across items)
1	ARCH (N = 43; 5 items)	4.63 (4.40–4.81)	93.96% (86.05–100%)
2	5 Microskills (N = 37, 5 items)	4.64 (4.49–4.78)	95.68% (94.59–97.30%)
3	TPP (N = 35, 4 items)	4.48 (4.35–4.54)	88.84% (78.38–91.67%)

Each session had a set of specific objectives, ranging in number from 3 to 8. (see the Appendix for a full list of session objectives). Mean % Agree was calculated by summing the % Strongly Agree and Agree responses for each item, then taking the average across items

Table 6.

Summary of modified Kirkpatrick’s levels 1: Reaction and 2B: Learning – Knowledge & Skills

Session #	Abbreviated session title	Appropriate to practice*	Would recommend* (Live only)	‘Pearls’ or takeaway messages (examples from open-text responses, N of similar responses)
1	ARCH	4.63 (N = 39)	4.47 (N = 36)	Involving student in self-assessment, Importance of wait-time (N = 16 open responses)
2	5 Microskills	4.54 (N = 37)	4.6 (N = 25)	Facilitating student critical thinking, Importance of wait-time Use Microskills to provide feedback (N = 15 open responses)
3	TPP	4.57 (N = 37)	4.54 (N = 37)	Value of signage notifying patient of student doctor presence, Discussing expectations with student at beginning of rotation, Defining roles (N = 12 open responses)

An asterisk (*) indicates that these items represent the mean of the Likert scale response

Table 7.

Indicators of modified Kirkpatrick level 3B, Behavior: Participants’ perceived impact of session

Session #	Abbreviated session title	“This activity will make me more effective in my practice/clinical teaching.”	“This educational activity will result in a change in my…”* (synch and asynch participants)	“Please note any changes or improvements… you plan to make” (example themes from open responses)
1	ARCH	Avg: 4.48 % Agree: 90.91% (N = 44)	K: 70.45% C: 70.45% P: 81.82% 2 + : 70.45%	Focus on student self-assessment and strengths, Share approach with colleagues, Utilize ARCH model in precepting (N = 9)
2	5 Microskills	Avg: 4.54 % Agree: 89.19% (N = 37)	K: 62.16% C: 51.35% P: 72.97% 2 + : 59.46%	Utilize wait time, Utilize Microskills in precepting, Positive reinforcement with students, Use teaching moments throughout day (N = 15)
3	TPP	Avg: 4.49 % Agree: 91.89% (N = 37)	K: 70.27% C: 64.86% P: 78.38% 2 + : 64.86%	Utilize sign about student doctor, Explain direct observation to students, Improve communication with students (N = 6)

All participants — both synchronous and asynchronous — are included in this data

Likert responses were averaged for the group and reported as % agree, which included all “Strongly Agree” and “Agree” responses

For the column marked with an asterisk (*), data are reported by % of survey respondents who indicated each impact category (Knowledge — K, Competence — C, Performance — P), and % who indicated 2 + categories. Note that participants could have selected 1, 2 or all 3 categories, so %s indicated will not add to 100%

Most participants cited “No Barriers” to implementation for the first 3 PDP sessions (ARCH: 61.36%, 5 Microskills: 64.86%, TPP: 45.95%). The most reported barriers to implementation in these sessions were “Lack of time to teach medical students” (ARCH: 18.18%, 5 Microskills: 24.32%, TPP: 37.94%) and “Uninterested student” (ARCH: 9.09%, 5 Microskills: 16.22%, TPP: 18.92%).

Discussion

This study presented data about the audience, engagement, and potential impact of an online PDP offered during the COVID-19 pandemic. These findings identified affordances, benefits, anticipated barriers, and lessons learned to improve future online PDP offerings.

Affordances

The online PDP afforded preceptors choice and flexibility. Participants attended noon and evening sessions as well as live and asynchronous formats. Most attendees were from medical fields (MD, DO), with multiple professions represented (see Table 3) and higher instances of synchronous as compared to asynchronous engagement. However, the availability of an asynchronous option likely helped attendees navigate schedule demands and select the most relevant topics. The geographic dispersal of preceptors across the United States did not hinder participation in this PDP, as all content was readily available online through videoconferencing and the LMS. Attendees from a wide variety of locations participated. While the intended audience included professions connected with the host institution, the variety of professions and specialties represented a broader reach of the PDP. This shows that the PDP can support preceptors in professions or specialties who traditionally have less access to faculty development opportunities.

Benefits and Barriers

As evidenced by the survey data, the sessions did what they were intended to do – effectively meeting the session objectives in common areas of need for preceptor development. The statistically significant associations between participant perception of the session and their perceived impact (Table 4) indicate that the self-reported impact was likely not due to chance but related to the quality of the sessions. These significant findings suggest that the overall perception of the sessions was likely associated with a potential change in participant knowledge (5 Microskills), and competence (ARCH, 5 Microskills). We recognize these as indicators of effectiveness, with additional data and exploration needed. More data on the actual impact would be important, potentially through follow-up with the participants. We acknowledge the small sample size in survey responses from many of the PDP sessions limited our ability to measure the effectiveness of those sessions.

The mixed methods findings add further detail to these quantitative indications of effectiveness. For instance, over half of survey respondents for the first three sessions indicated at least one potential impact area and sometimes more than one. In addition, the qualitative responses on takeaways note areas the participants found most salient and readily applicable – including utilization of the feedback models, sharing them with colleagues, and improved communication with students (see Table 7). Therefore, we acknowledge that these sessions, at minimum, had some potential impact or benefit to participants, with the need for further study to determine the effects realized.

We must also consider barriers to implementation to get a realistic picture of the potential impact. Most survey respondents for sessions 1 and 2 reported no barriers, implying actual future implications based on the sessions. For those who stated barriers, lack of time to teach and uninterested students were most frequently reported. These barriers provide insight into potential future needs for preceptor development and support.

Improvements to the PDP

We recognize that all faculty development programs are imperfect and have limitations. However, considering the findings and limitations of this study, we see the potential for improvement in three areas: gathering input from an advisory group, enabling interactive engagement, and providing opportunities for follow-up support.

While the development team considered common preceptor needs based on the expertise and knowledge of the intended audience, future PDP design would benefit from the input of an advisory group. This study indicates the effectiveness of the first three sessions in meeting session objectives and having a perceived impact. However, further exploration is needed to understand topic-specific responses. For example, while our expertise suggests feedback is an important area for preceptor development, and our audience indicated a positive response to this content, an advisory group would provide further insights for planning future sessions. An advisory group was used by Mulherin and colleagues [19] and referenced in other initiatives [7] to ensure the program aligned with goals and objectives. We would create an advisory group of preceptors from various professions, specialties, and locations to ensure the PDP matches needs and format preferences. While the survey process enabled feedback after implementation, the low response rate for some sessions inhibited our ability to assess session effectiveness. Input from an advisory group of preceptors would help better needs matching before implementation and elicit more meaningful post-implementation feedback.

While choice and flexibility enabled the audience to view the sessions during convenient times, the PDP provided limited interactive opportunities. While our findings suggest many participants have few barriers to implementation, we did not provide or measure any longitudinal supports following the session. We speculate that interactivity, including possible community-building, may lead to more lasting collaborative supports and, in turn, a more long-term impact of session takeaways. Interactivity is valuable to learning at all levels, including interactions between the presenter and learners and among learners. Steinert and colleagues [23] recommend faculty development approaches that support intentional community building, ongoing collaboration, and experiential learning – all of which involve interactions among faculty developers and participants. We want to increase interactivity in a future PDP by incorporating small group discussions or ongoing interaction between participants in a format similar to a learning community. Similarly, interactivity through a learning community approach would enable opportunities for follow-up support. We recognize that our PDP has the potential to spark collaboration and experiential learning. However, in the future, we would intentionally design these opportunities within the PDP itself.

Appendix. Session-level objectives

Session #	Session title	Session objectives
1	Improving Feedback Using the ARCH Model	a. List the components of the ARCH Feedback Model b. Describe the components of the ARCH Feedback Model c. Describe the purpose of the ARCH Feedback Model d. Articulate barriers to using the ARCH Model e. State how those barriers might be overcome
2	Five Microskills Model to Facilitate Learning in the Clinical Setting	a. Describe the RIME Model in terms of the roles a student plays in the clinical setting b. State the Five Microskills Model for clinical teaching c. Describe how the utilization of wait-time can facilitate success of using the Five Microskills d. Identify strategies to using the Five Microskills Model e. Identify barriers to using the Five Microskills Model
3	Teaching in the Presence of Patients Via Direct Observation	a. Describe why direct observation of the learner is important b. Practice strategies for making direct observation acceptable to the learner and to the patient c. Use strategies for making the observation process purposeful and systematic d. Identify strategies for getting your staff involved
4	Incorporating Teaching into a Busy Practice	a. List strategies for teaching efficiently in a busy practice/clinical environment b. Employ strategies to set learner and staff expectations that will facilitate efficient teaching c. Describe how to make the orientation of the student facilitate teaching and learning efficiency
5	Evaluating the Student with Reliability and Validity	a. Define the term summative assessment b. Define the term formative assessment c. Define the term norm-referenced evaluation d. Define the term criterion-referenced evaluation e. Define the term reliability with reference to the evaluation of student performance f. Define the term validity with reference to the evaluation of student performance g. Describe strategies to enhance the reliability and validity of your evaluation of the student
6	Teacher-Student Boundaries	a. Name the standards/documents (COCA/LCME) that speak to teacher-student boundaries b. Describe the consequences (personal and professional) that can result from teacher-student boundaries c. Develop personal guidelines that will help prevent teacher-student boundary crossings
7	Teaching in the Hospital Setting	a. Describe current trends/strategies for teaching in the hospital/in-patient setting b. Describe teaching and logistical strategies to enhance bedside teaching rounds c. Describe how knowledge of teacher and learning “personality type” can be used to enhance the effectiveness of bedside teaching rounds d. Describe how specific teaching strategies (e.g. Five Microskills, Wait-time, RIME framework) can be utilized to enhance instruction and learning in the hospital setting

Declarations

Ethical Approval

Des Moines University approved this study with exempt status (IRB-2021-10).

Conflict of Interest

The authors declare no competing interests.

Disclaimers

The authors alone are responsible for the content and writing of the article.