Career Skills Assessment in a Doctor of Pharmacy Curriculum

Abstract

Objective. To assess students’ knowledge of, perceived importance of, and confidence in six career skills areas (curriculum vitae/resume writing, interviewing skills/business attire, phone interviews, thank you notes, business/dining etiquette, and networking) before, immediately after, and six months after participating in a career skills workshop.

Methods. All students in a senior-level seminar course participated in the same simulation/performance-based workshop that was coupled with verbal or rubric-based feedback for each of the areas.

Results. Ninety-one students participated in the study and all students’ knowledge significantly increased over the study as determined by study baseline, conclusion, and six-month follow-up assessments. At study follow-up, knowledge increased an average of +7.1 percentage points from baseline. Multivariate analysis indicated significant increases in confidence from baseline to follow-up ranging from +0.15 to +0.29 across the six workshop areas, with resume/CV preparation having the highest increase. From study onset to follow-up, students perceived that the six career skills areas were above the average importance midpoint (3.0).

Conclusion. The workshop was effective in increasing students’ knowledge and confidence of essential career skills vital to pursuing post-graduate employment. These career skills are important for helping students distinguish themselves in a competitive job market.

INTRODUCTION

As the number of pharmacy schools and colleges across the United States increases, new pharmacy student graduates face growing competition for post-graduate employment or training opportunities.^1-3 It is important for pharmacy students to be equipped with a broad knowledge of professional career skills, such as interviewing, resume/curriculum vitae (CV) development, and business dining etiquette. Career skills training is advocated in the 2016 Accreditation Council for Pharmacy Education (ACPE) Standards.⁴ Standard 15.4 (student services) requires schools and colleges to provide career-pathway counseling to students. Standard 4 (self-awareness and professionalism) guides programs to help students commit to continuous professional development, evaluate their strengths and weaknesses, and recognize that their professionalism is constantly evaluated.⁴ Although standards 4 and 15 encourage programs to teach students about career skills in the doctor of pharmacy curriculum, the standards do not outline how to deliver this training, thus leaving programs to determine how to incorporate career skills training into an already crowded curriculum.⁴ Programs also need to design a structured training program so that all students are educated versus allowing students to electively participate.

One way to ensure that all students receive foundational career skills training is to embed professional development topics into the final year of the curriculum, such as in a senior seminar (capstone) course. Senior seminar courses are usually offered during the fourth-professional year when students are participating in their advanced pharmacy practice experiences (APPEs). Placement in this stage of the program is advantageous because the last professional year is usually the time when students interview for jobs or post-doctoral training. One limitation of this placement is that many seminar courses focus on research or clinical topic presentations, but some institutions have incorporated teaching of career skills into their seminar courses.^5-12 Many of the courses described in the literature use lecture-based training, optional faculty feedback, and voluntary participation in elective sessions offered to small groups of pharmacy, graduate, or other types of students.^5-8,10 Career skills topics in the training often include resume and curriculum vitae writing and interviewing techniques.⁹ The literature lacks data in evaluating student outcomes, such as retention of career skills knowledge and student-perceived importance of career skills, for a required core course using interactive performance-based workshops, assessments, and feedback. Data is lacking on whether students perceive if they possess career skills knowledge, if they believe that career skills are important, if they use career skills during their job search, or if these perceptions and knowledge change over their last professional year of pharmacy school.

This study was designed to evaluate a series of career skills workshops within a required senior seminar course. The six career skills areas are writing resumes and curricula vita, networking, interviewing skills/business attire, phone interviews, thank you notes, and business/dining etiquette. The workshops were divided into a residency/post-doctoral training and community/hospital/other track. Each track included feedback from an evaluator with expertise related to the track. The main study question was to determine if the career workshops improved students’ knowledge and confidence in the six career skills areas. This question was important because although pharmacy faculty may agree that training students in career skills is necessary, it is unclear which skills should be emphasized during the training, and if students use all six career skills during their job search over the year. The objectives of this study were to assess students’ knowledge of six career skills areas and students’ perceptions of the use of and confidence in the skill areas during the job search process.

METHODS

The career skills workshop was incorporated in the senior seminar course, which is a yearlong (fall and spring), two-credit hour course conducted across two campuses. This is a required course for fourth professional year students (P4), and is held once a week for two hours. The first three sessions of the course are devoted to the development of career skills. The workshop covers the six career skills areas and is taught in a simulation/performance-based (vs lecture-based) manner. This workshop has been offered to students for approximately 10 years now; however, various revisions to it have been made based on student and faculty feedback.

For the 2015-16 workshop, three simulation days were offered. On the first day, a general overview of the workshop was provided along with a review session of CV and resume development. All students completed a dining simulation where they ate cupcakes and drank punch to practice dining etiquette such as table settings, appropriate dinner options to choose during an interview, and cutting and eating food. During this session, students also learned about networking and appropriate business dinner conversation. Students practiced “small talk,” asked appropriate questions during the “down time” of an interview, and received feedback about their efforts. There was no formal student assessment of this simulation.

For the next two simulation sessions, students rotated through five stations. The stations were all pass fail and students were not formally graded. The stations included CV/resume review, 10-minute in-person interview, 10-minute phone interview, recent graduate panel interview, and thank you note writing (post-interview etiquette). Prior to these sessions, the students were asked to select one of two workshop tracks: the residency/post-doctoral track or the community/hospital/other track. The designation of the specific track was used when matching students with phone and in-person interviewers. Best efforts were made to match their selected track with a faculty member or preceptor with experience in that specific area to provide the most realistic experience. The in-person and phone interviewers were given a list of questions to use so as to avoid using the same questions. A different set of questions was provided to interviewers of students in the residency track. However, all students within the same track received the same questions.

Since the session was designed as a simulation, students were asked to participate in the workshop as if they were going to an actual interview. Nine faculty members and preceptors conducted the 10-minute in-person interview session. The students were assigned an interview time and location. Each student answered the interview questions and received feedback on their responses and their attire from the faculty member or preceptor. The interviewer completed a rating scale (1=poor, 5=excellent) of the student’s overall interview skills and professional appearance, along with a summary of at least one strength and weakness of the interview. Students received the rating scale and written strength and area of improvement as their feedback. For the 10-minute phone interview session, eight faculty members and preceptors conducted the interviews. The students reported to a designated office and were given the name and number of the interviewer. They called the interviewer on the phone and answered questions related to their track. The faculty or preceptor evaluated the students using a rubric and returned the feedback to students after the interview was completed. The rubric was similar to the in-person interview rubric, except the rating for professional appearance was replaced with a rating for projection and tone of voice. For the CV/resume review session, approximately 10 faculty members were asked to participate, and the session was conducted over a 30-minute period. Students brought copies of their CV or resume and received individual feedback from a faculty member about the content; no formal grading tools were used for this assessment. For the discussion panel session, the students self-selected the panel discussion they would like to attend. One panel included current PGY-1 and PGY-2 pharmacy residents and graduate students in the program. The other panel included recent graduates who were currently practicing in community, hospital, and managed care pharmacy. Prior to this session, the students were asked to think of two questions they would like to ask the panel. The panel members were asked to briefly introduce themselves and describe how they prepared for interviews. They were also asked to provide examples of success and failure during their interview process. For the final station, students completed a thank you card for an assigned person they interacted with from the two “interview” days and a rubric was used to evaluate the note.

In 2015-2016, P4 students were verbally invited to complete the study materials as part of the career skills workshop requirement in the senior seminar course. While students were required to participate in the workshops and complete the tests and surveys, they could indicate if they did not want their data included in the study. Responses to the test and survey were kept confidential and participants were informed that the results would only be reported in aggregate. Survey and test data were collected using Desire2Learn (D2L Ltd., Kitchener, ON, Canada).

A survey consisting of nine demographic questions and 21 Likert-type scale items related to student perceptions of four sections (importance, confidence, knowledge and training) was created for the study. Each section had five questions about the workshop topics which resulted in 20 questions. The section on perceptions of importance asked about one additional workshop topic area, the residency/post-graduate training panel, resulting in 21 questions. Since this workshop was formatted as a question and answer session, and not as a simulation, no additional questions were asked on the survey. Likert-type scale items were divided into five categories ranging from 1=not at all important to 5=very important. Sample survey questions were: How important is each workshop topic to your job search? How confident do you feel about each workshop topic? How much training have you received in each workshop area? How knowledgeable do you feel about each workshop topic? The demographic questions asked participants’ age range, gender, previous work experience with a range of years, job title (pharmacy technician, intern, non-pharmacy, other), job secured post-graduation including employer name and job title, and previously earned degrees.

In addition to the survey, there was a 15-item multiple-choice test. The test had four questions on CVs and resumes, two on interviews/business attire, three on phone interviews, three on dining etiquette, one on networking, and two on thank you cards. The study received IRB-expedited approval. Prior to administering the test and survey, five volunteers not enrolled in or affiliated with the course pilot tested the clarity of the test and survey questions.

Students completed the pre-workshop test (#1) and survey (#1) on D2L. One week after the workshop, students completed the post-workshop test (#2) and survey (#2) on D2L. In the spring semester, six months after the workshop, students completed the follow-up test (#3) and survey (#3). The same test and survey questions were used for all three time points. Data on D2L was stored in a password-protected secure network and could only be accessed by course coordinators and study personnel.

Descriptive statistics was used to summarize demographic variables and the study outcomes, using a midpoint of 3.00 for Likert-based measures. Associations between response scores across the time periods (ie, from baseline to workshop and follow-up) were tested via a multivariable, cross-sectional, time-series logistic or generalized least squares regression that also controlled for potential effects of age, sex, years of previous work experience, and current employment status.¹³ Random effects models were used with Huber-White standard errors to yield robust statistical inference.¹³ To control for multiple comparisons, a Šidák correction was incorporated to adjust the a priori .05 alpha level for statistical significance (ie, familywise error rate).¹⁴ All analyses were conducted using Stata MP 14.2 (College Station, TX).

RESULTS

Overall, out of 106 students, 91 (85.8%) participated in the investigation across the three response periods, with a total of 249 valid response sets collected. Most participants were 26-30 years old (46.2%, n=42), female (61.5%, n=56), previously employed (82.4%, n=75), currently working (71.4%, n=65), and had a bachelor’s degree prior to pharmacy school (71.4%, n=64). Full descriptive statistics appear in Table 1.

Table 1.

Student Demographics Data

The baseline score for the professional skills development workshop test was 60.2% (15.5), which increased significantly +7.1 percentage points on average to 67.3% (12.5) at follow-up (p=.001) (Table 2). Additionally, 22% (n=20) obtained an overall score ≥70% at baseline, increasing to 42.9% (n=39) at follow-up (p<.001). After adjusting for age, sex, years of previous work experience, and current employment, the multivariable analysis forecasted a +4.0 percentage point increase per response period (p<.001) and a 1.69 times higher odds of achieving a score of ≥70% per response period (p=.005).

Table 2.

Professional Skills Development Workshop Test

Specifically relating to test items, 5 of the 15 overall questions yielded correct responses at ≥70.0% or higher at baseline. These were Item 2: “In what order should items be listed in your resume or CV?” (88.9% initially answered correctly); Item 8: “When you are ordering from a menu during a business interview, what is the best food to order?” (92.6% initially answered correctly); Item 12: “What is the best reason why there is an increase in phone interview use for applicants?” (87.7% initially answered correctly); Item 14: “During a 30-minute phone interview, you should limit your number of questions to” (70.4% initially answered correctly); and Item 15: “Please identify the best email address for a resume or CV.” (72.8% initially answered correctly). At follow-up, three additional items were associated with responses ≥70.0%. These were Item 6: “Where is the water glass placed on a formal dinner setting?” (89.0% answered correctly); Item 7: “When and where should you place your dinner napkin?” (86.8% answered correctly); and Item 13: “Prior to a phone interview, what is an important recommendation to follow?” (78.0% answered correctly).

Across the baseline period through the workshop and follow-up, results of the multivariable regression analysis of each respective test item indicated that six questions were associated with significant increases per response period (Table 2). One question was related to the resume workshop, one for interviewing, two for dining etiquette, one for thank you cards and one for phone interviews.

At baseline, and generally throughout each response panel (Table 3), participants identified above average responses across each of the six items concerning “Importance”; scores were significantly and consistently above the midpoint of 3.0 (“Average Importance”) (p<.05). “Confidence” at baseline was significantly above 3.0 for Item 1 (Resume and CV Preparation response=3.3, p=.001), Item 2 (Face-to-Face Interview Skills/Professional Attire response=3.47, p<.001), and Item 5 (Post-interview Etiquette response=3.4, p<.001). “Formal Training” was significantly below an average of 3.0 at baseline for Item 2 (2.69, p=.007), Item 3 (Business Dining Etiquette and Small Talk=2.38, p<.001), Item 4 (Phone Interviews=2.01, p<.001), and Item 6 (Post-graduate and Residency Panel=2.31, p<.001). Mixed baseline results were observed for “Knowledge”; above-average responses were observed for Item 2 (3.41, p<.001) while below average responses were found for Item 4 (2.78, p=.041).

Table 3.

Outcomes of a Professional Skills Workshop^a

The multivariable analysis that controlled for variation across demographic and employment factors indicated significant increases in all items relating to “Confidence,” “Formal Training,” and “Knowledge” (p<.05), ranging from increases of +0.15 per response period (ie, Confidence Item 3 – “Business dining etiquette and small talk,” p=.017) to +0.42 per response period (ie, Formal Training Item 4 – “Phone interviews,” p<.001). “Importance,” which was significantly reported to be across all items at baseline, was not associated with a significant change per response period (p>.05).

DISCUSSION

The workshop was successful at improving students’ knowledge about six career skills areas. While these results were expected because of training, student knowledge was improved and retained over a six-month period. Of the 15 knowledge questions, students received a passing score on only six of the questions at baseline, which averaged to about one question per area. This suggests that while students may have had some superficial knowledge of the six topic areas, they were lacking the depth of knowledge that would be helpful during a future employment search.

For the nine questions that students failed at baseline, there were six questions that less than 50% of the class answered correctly; furthermore, three of these six questions had less than 20% of the class answering correctly at baseline. Less than 20% of the class answered the question related to resumes correctly, which was surprising because it was previously thought that students had received some earlier training in other courses about resumes. This result reveals that although they may have received training, they did not retain the knowledge. It is also possible that answers for the question may not be discriminating enough and need revision. This result was also alarming because a CV or resume is an important gatekeeper in the job search, and is what employers use to screen applicants. If students do not have a well-prepared resume or CV, they may not be asked to interview for the position. Further review of the results also reveal that with the two questions related to resumes and CVs that less than 20% of the class answered correctly, student performance at the end of the workshop and follow-up decreased even further. It is thought that this may have occurred due to the entirely performance-based nature of the resume and CV review. Approximately 10 faculty members evaluated the resumes or CVs and it is possible that students were given different advice related to email addresses or what to include on a resume versus a CV. The lack of a unified message delivered in a lecture or in a handout may have caused this confusion with the faculty and students, since many may believe that content and formatting of resumes and CVs are a matter of opinion. In the future, this data offers guidance about how to revise this section of the career workshop so that all students are taught the same guidelines. A resume rubric will also be created to consistently assess resume writing expectations.

Similarly, a question was asked about “small talk”/networking and scores decreased from baseline to follow-up. This was possibly related to how this topic was taught. Students participated in a large group simulation on business/dining etiquette. Afterward, they participated in a 30-minute discussion about small talk and were asked to reflect on their elevator speech. Students did not receive feedback on their elevator speech. Students may have benefitted from faculty feedback and from having more general content added to this topic. In future studies, a second test question should be added that covers elevator speeches.

Overall, it is important to note that these knowledge results are limited because one to four questions were asked per content area and although the test questions were related to the workshop objectives, more questions may have been needed to better assess student knowledge in the six areas. There were seven questions that students scored >70% on at the follow-up. Three questions were from resumes, two from thank you cards, one from interviewing, and one from networking/small talk. These results indicate areas of improvement for future workshop offerings. The poor scores may have resulted from multiple evaluators teaching and assessing content. Additional training materials and revised grading rubrics will be used in the future to better ensure that students acquire knowledge that is retained. In addition, future studies of career skills knowledge could include comparing a lecture-based workshop to a simulation/performance-based workshop to see if students retained their knowledge from the simulation workshop at six months compared to receiving the content in a lecture.

Results from the second study objective related to topic relevance and confidence in the topic revealed that prior to the start of the workshop, students perceived the workshop as important. This would imply that they were motivated to participate in the workshop since believing that a topic is relevant is related to motivation.¹⁵ Relatedly, students’ confidence and perception of knowledge gains in the topics significantly increased over time at the six-month follow-up. This increased confidence is important because it also affects motivation, which may ultimately lead to increased confidence when applying and interviewing for post-graduate employment. Students indicated that they used the workshop topics during their job search, which is an important measure of success for the program.

A limitation of this study is that students were not asked if they felt the way the topics were presented influenced their perceptions of confidence, relevance, or knowledge gains. Future studies could evaluate the workshop delivery design and include employer surveys that rate applicants’ overall performance and confidence in and knowledge of career skills. Future workshop topics could include negotiation skills in salaries and contracts as this is an area that affects an individual’s future earning potential.

CONCLUSION

A simulation/performance-based career skills workshop is an effective training program to increase students’ knowledge and confidence in six areas: resumes/CVs, business/dining etiquette, small talk/networking, interviewing, phone interviews, and thank you cards. The design of the workshop used simulations in each of the areas coupled with faculty feedback, which promoted long-term retention of the content. One of the most important aspects of the study revealed that the students thought the material was important from the study onset and there was an increase in every item for confidence, formal training, and knowledge over the study duration, indicating that the training achieved its objectives. Similar workshops could be offered at other pharmacy schools to systematically train all students in this important developmental area.