Effectiveness of a Weekly Faculty Conversation Forum About Teaching

Abstract

Objective

To evaluate the effectiveness of holding weekly 60-90 minute conversation forums for faculty members to discuss, explore, and reflect on various teaching topics in a relaxed, informal, interactive format.

Methods

Weekly, 60-90 minute sessions were held for faculty members of the University of Illinois College of Pharmacy. A 15-item retrospective pretest-posttest questionnaire was developed and administered at the end of the first year of implementation to evaluate the participants’ perceived knowledge, abilities, and confidence gains relative to becoming effective educators.

Results

Eleven faculty members completed the questionnaire. All respondents tended to agree (6/11) or agreed (5/11) that their confidence as educators improved after attending the conversation forums. In addition,7 respondents tended to agree and 4 agreed that their ability to self-assess their teaching had improved.

Conclusions

An ongoing weekly conversations forum provides faculty members opportunities to explore and learn about facets of teaching in a safe, informal environment.

INTRODUCTION

It is important that senior faculty members prepare junior faculty members for careers as researchers and educators. The need to adopt a model that facilitates faculty mentoring is becoming more prominent. This is in part because of the great numbers of new faculty members entering academia. Typically, junior faculty members are hired with little “basic training” in teaching and have little, if any teaching experience. Often, they adopt the teaching norms, attitudes, and “strategies” to which they have been exposed as students, residents, or fellows. Impacting their selection of a teaching strategy are the pressures of other job responsibilities, which leave them little time and energy to experiment with different strategies that would allow them to discover their own “best practices.” Hopefully, they view their role as educators as challenging and rewarding and thereby select the best strategies and most appropriate norms. Their instructional tasks can be daunting. It is incumbent upon the colleges and schools to provide appropriate “help” mechanisms to get junior faculty members “off on the right foot” with respect to teaching and educating professional students.

Indeed, all colleges and schools of pharmacy are familiar with the final Accreditation Council for Pharmacy Education (ACPE) Standards and Guidelines, Effective July 1, 2007.¹ Specifically, Standard Number 26 encompasses “Faculty and Staff Continuing Professional Development and Performance Review.” Guideline 26.1 states that the college or school “must have in place or provide support for programs and activities for faculty and preceptor continuing professional development as educators, researchers, scholars and practitioners, commensurate with their responsibilities in the program.¹”

According to Johnson and Ridley, “mentoring relationships (mentorships) are dynamic, reciprocal, personal relationships in which a more experienced person (mentor) acts as a guide, role model, teacher, and sponsor of a less experienced person (protégé).”² A review of the literature for mentoring models that could be adapted to this challenge discovered 2 potential mentoring models. The more traditional model falls under the heading of a “Protégé Model."³ Usually, this model involves a senior faculty member working with a single junior faculty member in their own discipline and/or department. Or it may involve 2 faculty members from different disciplines within the school, college or university. These pairings of faculty members can be formal assignments or informal when the pairs find each other at random.³ Regardless, this is a hierarchical type model. Campbell and Chalmers have described the duties of and the benefits of mentoring using this approach.^4-5

Another model is referred to as the “Networking Model.”⁶ Haring describes the networking model as a more egalitarian approach to faculty mentoring that puts junior faculty members in groups in which participants contribute to each others’ successes.⁶ Usually, the participants are junior faculty members representing different departments, divisions, or disciplinary specialties, thereby making it non-hierarchical in nature. What brings the participants together is the exploration of a common interest, eg, the many facets of classroom teaching or how to prepare for tenure. In the networking model, a senior faculty member acts as a facilitator or a coach who guides or empowers the participants to help one another. His/her principle role is to prompt discussion, keep the group on track, and create a safe environment in which faculty members can share their classroom experiences, problems, concerns, and successes. Unlike a more traditional mentoring approach, for example, the protégé model, in the networking model, the senior faculty member fades in and out of the group as needed.

The group's purpose in the networking model is goal-oriented, non-cathartic, and tends to support and effect change within the institution. It is an ideal approach for faculty members, both junior and senior, who may feel intimidated by working one-on-one in the area of teaching. Also, due to its non-hierarchical structure and potential to bring diverse groups of new faculty members together, it can serve as a positive influence to help retain minority faculty members, particularly if the group contains more than 1 minority faculty member who is confronting similar issues.⁷

It is also possible that a mentee may have more than 1 mentor for different roles/needs, eg, research mentor, teaching mentor, life-in-balance mentor. As an example, at Purdue University, faculty members can select more than one “teaching mentor” from members of the Teaching Academy depending upon the skills of the mentor (eg, small group discussion, large lecture presentations, problem-based techniques) and the needs of the mentee. One mentor simply might not be capable of fulfilling all of the faculty member's needs to help improve his/her teaching skills.

As Abraham Zaleznik wrote, “Great teachers take risks. They bet initially on talent they perceive in younger people. And, they risk emotional involvement in working closely with their juniors. The risks do not always pay off, but the willingness to take them appears crucial in developing leaders.”⁸ This quote applies not only to leaders but to leaders who are teachers also. The networking model was selected for the “Conversations about Teaching” forum series.

Thus, the objectives of the “Conversations about Teaching” forum series were: (1) to provide faculty members with an environment where they could become reflective of their teaching and, (2) to equip faculty members with the basic skills, attitudes, and support necessary to become committed effective educators.

METHODS

Initially, all college of pharmacy faculty members were contacted via e-mail with an announcement about the “Conversations About Teaching” forum series and provided a description of its intent and information about the first meeting. All faculty members who were interested in improving their teaching skills were invited to attend. Also, emphasis was made in the e-mail that a faculty member did not have to attend all sessions and could “come and go” as his/her schedule permitted. Prior to the initiation of the series, the Dean, Department Heads, and College's Executive Committee with representation from all departments were notified about it and of the start-up date and time.

At the first session, the faculty members were provided with a goal-ranking exercise (ie, a needs assessment). Input from the completed needs assessments were used to determine which topics would be emphasized during future meetings.

The series was convened weekly for 60 minutes for the last 12 weeks of the 15-week semester, allowing faculty members to begin their teaching responsibilities for that semester. During the first academic year, sessions for the fall semester were held on Tuesday afternoons and for the spring semester on Wednesday afternoons. Because of clinic/service obligations for some faculty members, sessions were scheduled for the late afternoons (eg, 2:30 pm or 3:30 pm).

To assess the effectiveness of the series, a 15-item self-assessment instrument, using a 4-point rating scale was developed and used to evaluate the participants’ perceived knowledge, abilities, and confidence relative to becoming effective educators. The study used a single group posttest design with a retrospective component for 9 of the items (Table 1). Because of the challenge in evaluating perceived responses when the internal construct is likely to change (ie, response shift bias) a retrospective component was deemed appropriate.^9-12 In addition to the 9 pretest-posttest items, 6 items were designed and used to measure outcomes as a result of faculty members attending the “Conversations about Teaching” forum series. The Office for the Protection of Research Subjects (OPRS), Office of the Vice Chancellor for Research, UIC, granted approval for this research project (Research Protocol #2003-0246).

Table 1.

Statements Included on an Attitudinal Survey Instrument Used in a Faculty Discussion Forum, “Conversations About Teaching”

Responses were analyzed using the Rasch model (Winsteps version 3.51). Descriptive statistics and paired student t tests were performed using SPSS, version 12.0 for Windows.

The retrospective pretest-postetest questionnaire was forwarded to participating faculty members via e-mail and a reminder was sent encouraging these faculty members to return the anonymous questionnaire within 2 weeks. To retain anonymity, the questionnaire was returned to the campus mailbox of the session facilitator.

RESULTS

Demographic Survey Outcomes

Of those who completed the demographic survey instrument as part of the retrospective pretest-posttest questionnaire, 10 respondents were from the Department of Pharmacy Practice and 1 member was from a basic science department. Nine individuals had earned doctor of pharmacy degrees and 2 had earned doctor of philosophy degrees. Eight respondents were non-tenure, clinical assistant professors and 2 respondents were nontenure, clinical associate professors. One individual was a tenured associate professor of medicinal chemistry.

Five respondents had less than 5 years teaching experience; 4 respondents had between 6 and 10 years teaching experience; 1 respondent had between 11 and 15 years teaching experience, and 1 respondent had between 16 and 20 years teaching experience. Five individuals indicated that they devoted greater than 40% of their effort to teaching. Another 3 respondents indicated that they devoted greater than 50% of their effort to teaching. One person indicated devoting greater than 80% of his/her effort to teaching, while another indicated greater than a 25% effort toward teaching. One respondent failed to answer this question.

When asked, “in the scope of my current position, I perceive my teaching as…..,” 10 respondents indicated that it was very important. One other individual indicated that it was important. The last survey question asked about the respondent's belief in how well they had been mentored in the area of teaching while on the faculty. Eight respondents indicated that they were wholly inadequately mentored. Three respondents indicated that they were adequately mentored.

Outcomes of the goal-ranking exercise allowed the participants to share their expectations for the series. Topics were grouped according to themes, eg, pedagogy, professional faculty issues, personal growth and understanding. These are demonstrated in Table 2.

Table 2.

Topics Included in a Faculty Discussion Forum, “Conversations About Teaching”

The Rasch Rating Scale Criteria were met, verifying construction of a measurement scale. Evaluation of INFIT and OUTFIT statistics for the retrospective pre-postetest questionnaire were less than 1.4 and greater than 0.6, supporting the model unidimensionality and local independence requirements. The Pearson separation index for the assessment instrument calculated was equal to 2.00, with a reliability of 0.80, analogous to Cronbach alpha. Further, the group means for subject ability measures were -0.02 logits (±-1.50 logits) and 1.66 logits (±1.70) for the pretest and posttest, respectively. The difference in group means (dependent t test) demonstrated a significant self-perceived improvement from pretest to posttest (t = 6.26, p < 0.001)

The Rasch Expected Score Map depicted in Figure 1 demonstrates the 9 items administered in a retrospective pretest/posttest format. The sample distributions for persons responding “initially” and “now” are provided in different colors to help the reader interpret the findings. Examples for interpretations of 5 of the items are provided below.

Figure 1.

Rasch analysis of faculty members’ responses to a questionnaire.

The expected score map demonstrates that the second easiest item for the faculty to endorse positively was “My knowledge of what to include in a letter of recommendation.” Most subjects rated their ability as “good” initially and “very good” now, after the educational intervention. Responses to “My ability to use techniques to motivate students in class” and “My knowledge of active learning approaches to employ in my teaching” demonstrated similar improvement from the conversation forums. The most difficult item for faculty members to endorse positively was “My ability to use skills to coordinate a team taught course.” Initially, approximately 84% rated their knowledge as “weak” or “fair” and now, after the intervention, approximately 65% rated their knowledge as “good” or “very good.”

Results of the 6 outcome items demonstrated that after attending the “Conversations” series, all 11 respondents tended to agree (6/11) or agreed (5/11) that their confidence as educators had improved. In addition, 7 respondents tended to agree and 4 agreed that their ability to self-assess their teaching had improved. By virtue of the series, 7 tended to agree and 4 agreed that they were more aware of their teaching philosophies. Eight of eleven respondents tended to agree and 3 respondents agreed that their knowledge of what to include in a course syllabus had improved and all respondents now understood how to create a teaching portfolio. Lastly, 10 respondents tended to agree and the remaining faculty member agreed that their ability to use techniques to promote active learning in their teaching had improved because of the series.

DISCUSSION

The demographics of survey respondents corresponded closely with the actual faculty members in attendance during the “Conversation” forums. It was not surprising to the authors that the majority of faculty members who responded were from the Department of Pharmacy Practice and carried the rank of clinical assistant professor or associate professor. The nature of our institution, a Research I University, and the structure of our doctor of pharmacy curriculum requires that the heaviest instructional load be carried by our clinical faculty members. Throughout the curriculum, specific basic science faculty members shoulder their departments’ teaching loads. The majority of these faculty members are tenure track and carry the rank of either full or associate professor. Reasons for their lack of participation may include an inability to dedicate the time to participate, scheduling conflicts, not envisioning the “Conversations” to be of direct benefit, and/or a simple lack of interest. These chosen basic science faculty members are becoming more involved in subsequent offerings of “Conversations.” Interestingly, with the addition of the basic science faculty members, the topics for discussion have not changed. It also appears that when there is a particular issue that needs “airing,” eg, “should professional students be allowed to attend national conferences/meetings during the academic year?” the “Conversations” series has provided a forum for discussion, oftentimes led by a senior faculty member, not necessarily the facilitator.

Topics for discussion emanated from the groups on a weekly basis, using Table 2 as a “springboard” to identify them. In addition, faculty members were encouraged to bring “hot topics” that they had encountered in the scientific and/or professional literature for discussion. The discussions often provided opportunities for digressions into interesting tangents. For example, one of these tangents involved the use of the DiSC Classic Personal Profile System 2800 method to determine a student's behavioral and learning profile and to relate that information to the type of environment most conducive to that student's educational success, and to take that information one step further to look at the type of educator a specific profile might be.¹³ This then led to a discussion of the Myers-Briggs Personality Profile. An added benefit was that faculty members who did not know their Myers-Briggs nor DiSC profiles were able to determine them prior to the respective sessions, thereby encouraging faculty members to become personally involved in the discussion. Another example of a “hot topic” was a discussion of service learning led by a faculty member and how she incorporated this type of learning experience into her clinical clerkship rotation. Another faculty member volunteered to lead a discussion on the pros and cons of Web-based instruction in the classroom. An additional finding was that the faculty-led topics were very well received

Initially, scheduling on Tuesdays (fall semester) and Wednesdays (spring semesters) was performed to get a “routine, cyclical” schedule, and help faculty members plan on a weekly basis. However, it became apparent that by doing so, the series was disenfranchising those who really wanted to be a part of the program on a routine basis for the entire academic year, not just either semester. Thus, offerings in subsequent academic years have been scheduled on alternating Tuesdays and Wednesdays during the semester to accommodate all interested faculty members. In addition, some faculty members commented on the room in which the “Conversations” series was held. Initially, it was in a small conference room. However, as the sessions grew larger in attendance, the weekly meetings were moved to a larger, conference room.

Originally each “Conversations” meeting was scheduled for 60 minutes. However, depending on the topic, meetings could last beyond the 60 minutes to almost 90 minutes. In fact, for some topics the meetings would have gone beyond 90 minutes if conflicts with the room had not occurred. This provided a continual room scheduling challenge to the facilitator.

At the beginning of any new experience, it is sometimes difficult to gain a true appraisal of one's own knowledge and skills. Thus, the retrospective pre-posttest questionnaire design was effective as it allowed the respondent to reflect on the experience. Clearly, the majority of the participating faculty members consider teaching to be very important, and yet they indicated that their previous mentoring in the area of teaching has been “wholly inadequate.” A majority of faculty members who attended “Conversations,” reported that they devoted a minimum of 40% of their time to teaching activities. This is a high percentage of one's time devoted to an area in which they have little guidance or training. This confirmed initial impressions about mentoring related to the teaching at the College and the primary basis for conceptualizing, initiating, implementing, and evaluating the “Conversations About Teaching” forum.

When asked “what did you personally like about the Conversations series,” respondents indicated the informal exchange of ideas, having group discussions with a variety of faculty members (eg, basic scientists, clinical faculty members), networking with other faculty members, discussing effective teaching strategies, and receiving support from their colleagues. Further, having the opportunity to discuss various issues in an open, safe atmosphere was described by some participants as “liberating.” All faculty members were aware of their own personal growth as they reflected on their teaching, and as an added dividend, they perceived that their teaching skills improved.

The Rasch Analysis demonstrated that the “easy to endorse” items were those items which could be perceived to be more tangible or concrete to the participants, and could be implemented easier and whose results were observerable. These were items with which the faculty member could become familiar and establish their skills quicker (eg, writing a letter of recommendation, how to motivate students in class, techniques to minimize academic dishonesty, knowledge of active-learning approaches). Further, the analysis demonstrated that the “hard to endorse items” were those topics which could be perceived as more complex, requiring a higher level of interpersonal communication skill and experience (eg, skills to coordinate a team-taught course, working with others in a team-taught course). Also, 2 of the topics addressed, an ability to construct student handouts and develop an effective course syllabus, are continuing developmental processes. It will be important as the series continues to follow these faculty members longitudinally to determine whether these abilities continue to develop with time.

The attitudinal survey instrument also asked the respondents for suggestions for improving the series. They indicated that it would be helpful to ask faculty members to bring items or “hot topics” for discussion to the attention of the facilitators beforehand. Thus, when sessions were devoted to examination creation and item writing, for example, participants were asked to bring samples of their examination questions beforehand so that these could be copied and shared with the other participants. Those faculty members submitting samples of their examination questions were then asked to describe their thought process to create the questions and rationale for the type of questions utilized in the examination. This provided fertile ground for discussion and, in several instances, resulted in constructive ways from those in attendance to improve the items further in the future. Other suggestions included one to provide a weekly outline of the topic to keep the discussion “on track.” Usually, this would be difficult as the topic is decided at the beginning of the session. However, with “planned” topics for discussion, including those delivered by various faculty members (eg, Web-based instruction), this would be quite possible and provide a framework for the discussion.

During the course of the series, several faculty members were precepting fourth-professional year doctor of pharmacy students in a clinical clerkship rotation. These faculty members were encouraged to bring their students along to the sessions. Suffice it to say, unsolicited comments from the students demonstrated that for them it was an “eye-opening” experience. Observing faculty members discussing educational issues demonstrated to them the faculty members’ passion for educating students and seeking means to improve their teaching. Often, students provided their opinions and/or insights into the discussion. Interestingly, in the final assessment, 1 faculty member wrote that the series facilitators should “assess whether doctor of pharmacy students should attend all sessions or consider having 1 session a month for “faculty only” as some faculty members may be more open to discuss “thorny” issues. Another faculty member suggested that perhaps the student attendees should lead a discussion or provide a listing of topics they would like have the faculty member discuss, eg, ways to improve the curriculum and the students’ educational experience.

Summative faculty member comments provided about the series included that it was a “good idea, but it's hard for me to attend.” And this was observed by the facilitators. It was difficult for most faculty members to dedicate time or take time out of busy schedules to participate on a weekly basis. Those in a clinical or academic setting know that one's schedule can be easily disrupted for a variety of reasons, and on a moment's notice. Overall, participants thought the “Conversations” forums were a great venue to discuss new ideas and bring up instructional problems.

Based on feedback from the first year, the series has been held on alternating Tuesday and Wednesday afternoons during the fall and spring semesters since that time allowing for and enhancing faculty participation as much as possible throughout the academic year. This has helped to minimize having some faculty members being able to participate only during one semester of the academic year. Another discovery was the value of a reminder e-mail via a listserv a day or 2 before “Conversations” would meet. Hence, having a dedicated faculty member facilitator performing this service ensuring that the meetings take place is key to the program's success. Whenever possible, faculty members should be solicited to volunteer to lead future discussions of interest to them and their colleagues (eg, Web-based instruction, service-learning). Usually, participating faculty members are happy to do this with advanced notice and sufficient time to prepare. Also, it helps develop their self-confidence in this environment. Lastly, the authors have benefited immensely, too, from this experience. As Bennis and Thomas write, “the mentor sees in the prospective protégé a youthful energy and enthusiasm—we are tempted to call it “wide-eyed” enthusiasm, to bolster the point—that triggers some primal desire to nurture, teach, and protect. However, the mentor justifies his or her readiness to lavish time and other resources on the mentored (a desire to give back to the community; ‘she reminds me of myself at that age’), the real reason for the mentor's devotion may be as old as time and as inescapable as chemistry. It may be that the mentor benefits physiologically…just as he or she surely benefits socially.”¹⁴ There are no truer words.

SUMMARY

The current trend is to hire new faculty members in clinical, non-tenure tracks in schools/colleges of pharmacy upon completion of their residency training. For those newly hired tenure-track, basic or clinical science, faculty members having a solid research orientation, does not guarantee they are well steeped in teaching. Thus, it is increasingly important to emphasize to all pharmacy faculty members that there exists a cadre of faculty mentors and/or programs such as “Conversations” to welcome them, ease their adjustment, and guide them at this critical time in their faculty development.¹⁵

The first offering of “Conversations about Teaching” forum was a positive, successful experience for those participating. Faculty members, junior and senior, who attended wanted some form of mentoring to guide them in their teaching, and when performed in a safe and informal atmosphere, faculty members responded in a positive and constructive manner that encouraged and improved their academic development. Pharmacy faculty members who self-selected to become part of the “Conversations About Teaching” forum experience were committed to their teaching and to the improvement of their teaching.

Successful mentoring, kept simple and informal, requires only a time and place where a discussion about teaching can occur. The construction of “Conversations” was facilitated by having concrete and abstract topics for discussion, thereby allowing the faculty member to enter the interaction at their own comfort level. Having a dedicated, senior faculty member to coordinate and facilitate the series is vital to its success.