Perceptions of Pharmacy Students, Faculty Members, and Administrators on the Use of Technology in the Classroom

Abstract

Objectives. To gather and evaluate the perceptions of students, faculty members, and administrators regarding the frequency and appropriateness of classroom technology use.

Methods. Third-year pharmacy students and faculty members at 6 colleges and schools of pharmacy were surveyed to assess their perceptions about the type, frequency, and appropriateness of using technology in the classroom. Upper-level administrators and information technology professionals were also interviewed to ascertain overall technology goals and identify criteria used to adopt new classroom technologies.

Results. Four hundred sixty-six students, 124 faculty members, and 12 administrators participated in the survey. The most frequently used and valued types of classroom technology were course management systems, audience response systems, and lecture capture. Faculty members and students agreed that faculty members appropriately used course management systems and audience response systems. Compared with their counterparts, tech-savvy, and male students reported significantly greater preference for increased use of classroom technology. Eighty-six percent of faculty members reported having changed their teaching methodologies to meet student needs, and 91% of the students agreed that the use of technology met their needs.

Conclusions. Pharmacy colleges and schools use a variety of technologies in their teaching methods, which have evolved to meet the needs of the current generation of students. Students are satisfied with the appropriateness of technology, but many exhibit preferences for even greater use of technology in the classroom.

INTRODUCTION

As higher education institutions increase their use of technology and new technologies are developed, there is increasing pressure on faculty members to use these new technologies in the classroom and to appropriately modify the educational methods they use.^1,2 This pressure, in large part, is attributable to different learning preferences of the new generation of students, who are referred to as “digital natives.”³ These students have not only an overall greater preference for technology use but also a strong desire to be engaged in and to interact during the learning process.

Technology can enhance the teaching and learning experience through ready access to information, increased collaboration, and student engagement.⁴ Some technologies have been documented to improve teaching and learning. For example, student engagement, active learning, and learning outcomes were improved with the use of audience response systems and lecture podcasting.^5-10 Online and asynchronous learning are some of the newer teaching methods possibly attributable to the increased use of technology in education. Although students may be frustrated with technology at times, asynchronous learning is preferred by students.¹¹ Additionally, learning has been enhanced by a combination of live lectures and online learning strategies.¹²

Some have argued that technology is occasionally used just for the sake of technology and that allowing technology to dictate the teaching approach is ineffective.¹³ Mixed results have been found for the use of Microsoft PowerPoint, for example. While it receives positive reviews as a tool that stimulates creativity, it promotes low student responsibility for and passivity toward learning.¹⁴ Interestingly, pharmacy students indicate that they rely heavily on PowerPoint presentations for their learning.¹⁵ Similarly, various features of course management systems have different influences on learning.¹⁶ Simply using a course management system as a means to distribute information does not substantially enhance the learning experience; however, the discussion features and the support for increased interactions allow students to become more engaged in learning.

The Technology in Pharmacy Education and Learning Special Interest Group (TiPEL SIG) of the American Association of Colleges of Pharmacy (AACP) catalogued a database of instructional technologies available at colleges and schools of pharmacy. In gathering these data, however, the group did not quantify the extent to which these technologies were used by faculty members and students.¹⁷ The objective of this investigation was to evaluate perceptions of pharmacy faculty members, students, and administrators regarding types, frequency, and appropriateness of educational technology use in the classroom.

METHODS

To investigate perceptions regarding the type, frequency, and appropriateness of technology use in the classroom, opinions of the main stakeholders (ie, faculty members, students, and administrators) were solicited. A collaboration of 6 members of the AACP Academic Leadership Fellows Program (ALFP) and the group’s dean facilitator developed 2 survey instruments (1 each for faculty members and students) and interview questions for administrators. After discussions at the initial ALFP retreat in August 2010 and extensive literature and Web reviews, the group developed 3 survey instruments, which were revised several times to clarify survey items. Face validity of the survey instrument was determined by a consensus of 7 study investigators. Technologies evaluated in the survey instruments were: course management systems (eg, Blackboard [Blackboard Inc., Washington, DC], Moodle [Moodle HQ, Perth, Western Australia]), audience response systems (eg, TurningPoint [Turning Technologies, Youngstown, Ohio], iClicker [Macmillan New Ventures, NY, NY]), lecture capture (eg, Tegrity [Tegrity USA, Burr Ridge, IL], Echo360 [Anystream, Dulles, VA], podcasts (eg, iTunesU [Apple, Cupertino, CA]), Web 2.0 tools (ie, blogging, wikis, social media), smart boards (ie, interactive white boards), online testing, and simulation technologies. These technologies were chosen because they were identified by the TiPEL survey as being among the most commonly used¹⁷ and were available at the specific institutions to be surveyed. To ensure clarity and accuracy in responses, survey instruments were individualized for each college and school of pharmacy to include specific names for respective technologies used by each institution.

The intent of the research project was not just to quantify the use of various technologies in the classroom but also to determine whether these technologies were used appropriately to enhance teaching and learning. Appropriateness of use was determined by querying students and faculty members about whether they believed the technology was being used effectively. Additionally, students were asked if specific technologies enhanced their learning experience, and faculty members were asked whether specific technologies enhanced their teaching and student learning. Faculty members were also asked to report whether they had changed their teaching methods to incorporate available technology for the purpose of meeting the needs of the current generation of students.

Along with questions regarding the frequency and appropriateness of classroom technology use, students were asked to report their interest in further increasing the use of technology. Faculty members were asked to report the frequency with which they experienced technical difficulties and how quickly the issues were resolved; technology development needs in the classroom; pressure they may feel from students, colleagues, or administrators to adopt new technologies in the classroom; and their success with and perceived barriers to implementing technology. Information on each participant’s age, sex, and self-reported comfort level with technology was also collected.

ALFP participants were challenged to connect with high-level university administrators to gain perspectives on challenges not previously considered. Group members agreed that interview data from deans, presidents, and provosts (ie, nontechnical administrators) would compare well with interview data from information technology administrators (ie, technical administrators). Scripted interview questions were constructed to improve consistency among multiple interviewers. Questions differed for technical and nontechnical administrators. When consent was provided by the administrators, interviews were recorded and transcribed for analysis for common themes. Interviews focused on administrators’ vision for technology in the classroom, perceptions of technology impact on teaching and learning, and perceived successes and barriers for implementation.

Third-year (P3) students, all faculty members, and 2 administrators from 6 colleges and schools of pharmacy were surveyed in the spring semester of 2011. P3 students were chosen for the study because they had the entire spectrum of experiences with classroom technology throughout their pharmacy education. The 6 colleges and schools of pharmacy, which were evenly split between private and public higher education institutions, were geographically dispersed across the United States (California, Georgia, Massachusetts, Minnesota, Virginia, Wisconsin). Institutional review board approval was obtained from each institution.

Data were combined in Microsoft Excel and analyzed with PASW v18.0 (SPSS Hong Kong Headquarters, Quarry Bay, Hong Kong). Descriptive statistics were used to categorize the frequency of use of the technologies and for demographic information for students and faculty members. Comparisons between groups for Likert-type questions were accomplished with Mann Whitney U and Kruskal-Wallis tests, based upon number of groups compared. Pearson chi-square tests were used to compare nominal level variable results to each other. For all statistical analyses, a probability of <0.05 was considered significant.

RESULTS

A questionnaire was sent to 734 P3 students. Four hundred sixty-five students from 6 schools completed the questionnaire, for a response rate of 63%. Response rates within the 6 schools ranged from 100% to 27%. Because of problems administering the faculty questionnaire at 1 institution, results reflect the data from only 5 institutions. Of the 182 questionnaires sent to faculty members, 124 were completed, for a response rate of 69%. Response rates within the 5 schools ranged from 92% to 37%. Table 1 contains the demographic characteristics of student and faculty participants.

Table 1.

Demographic Characteristics of Students (N=466) and Faculty Members (N=124) Participating in a Study of Technology Use in the Classroom

Student Perceptions

The first technology addressed in the student questionnaire concerned course management systems. All 6 schools surveyed used a course management system of some type. Ninety-two percent of the students reported that they used a course management system “frequently” (defined as between 75%-100% of their courses) in pharmacy school. The next highest response category was “often” at 7% (defined as 50%-75% of their courses). Only 6 students reported use of a course management system in less than 50% of their courses. When asked if faculty members were effectively using this technology, 92% either agreed or strongly agreed that it was being effectively used. Ninety-one percent of the students strongly agreed or agreed that the course management system enhanced their learning experience (Table 2), whereas 78% of the students strongly agreed or agreed that it increased communication between faculty members and students. Despite the high percentages of courses already using a course management system, 85% of students agreed or strongly agreed that a course management system should be used by more faculty members and in more courses.

Table 2.

Pharmacy Students’ Perceptions Regarding Use of Educational Technologies (N=466)

All 6 colleges and schools used an audience response system. However, when asked how often an audience response system was used in their courses, 37% of students indicated “rarely” (defined as use in <25% of the courses), and 30% indicated “sometimes” (defined as use in 25%-50% of the courses). Students had mixed opinions about the effectiveness of faculty members’ use of this technology, with 53% responding positively and 45% responding negatively (Table 2). Despite disparate opinions about its effectiveness, 72% of the students strongly agreed or agreed that use of an audience response system enhanced their learning experience. Similarly, 74% of the students strongly agreed or agreed that this technology engaged them during class, and 67% strongly agreed or agreed that more faculty members should use it in their courses.

Five of the 6 colleges and schools used lecture-capture technology (ie, digital audio and/or video recording of lectures). When asked about how often lecture capture was used in the school, “sometimes” and “often” were the most common responses at 34% and 29%, respectively. Seventy-two percent of students strongly agreed or agreed that faculty members effectively used lecture capture (Table 2), and 80% of students either strongly agreed or agreed that lecture capture enhanced their learning experience. When students were asked whether this technology should be used in more courses, 75% of students either strongly agreed or agreed.

The 5 colleges and schools that used lecture capture systems also distributed those lectures using podcasting. Responses to how often the podcasts were made available in particular courses were mixed, with 24% reporting “frequently” and 23% reporting “rarely.” Students believed podcasts enhanced their learning experience, with 63% positive responses (Table 2). Roughly half (51%) of students reported that they listened to the podcasts when studying for a course. The common perception that students skip class when podcasts are available was not supported by our data, given that only 8% of students reported skipping class and relying on the podcast to learn course material. Seventy percent of students exhibited a preference for greater use of podcasts in their courses.

Five of the colleges and schools used online testing in their curriculum. Forty-two percent of students reported that online testing occurred “sometimes,” while 32% reported “rarely.” Fewer than half of the students (37%) reported that they liked online tests or would like to see more online tests administered (41%). Half of the 6 schools used smart board technologies and Web 2.0 tools in the classroom, while only 1 school used simulation technology.

Students were also asked general technology questions. The first question related to overall technology use in the classroom. Fifty-two percent of students were satisfied with the amount of technology used in the classroom, while 40% would prefer greater use. A minority (6%) would have liked to see less technology used in the classroom. When responses were compared by self-report, those who considered themselves tech-savvy demonstrated a preference for greater use of technology in the classroom (p<0.01). When comparing across colleges and schools, there was a significant difference in students’ desire for technology in the classroom (p=0.002). There was also a significant difference by gender, with male students preferring greater use of technology than female students (p=0.02). No difference was observed when desire for technology and age were compared. Respondents were asked about the frequency of technical difficulties that occurred at their college or school. Sixty-five percent of students either strongly agreed or agreed that the frequency of technical difficulties encountered was acceptable. There were no significant differences in responses by gender, age, or how tech-savvy the students were. A third question revealed that 91% of the students strongly agreed or agreed that technology-based teaching methodologies used by faculty members met their academic needs.

Faculty Member Perceptions

Course management systems were the most commonly used classroom technology by faculty members (64%). The remainder of the top technologies used included lecture capture (46%), podcasts (44%), online testing (36%), and audience response systems (31%). When faculty members were asked whether they effectively used the technologies available to them, 77% reported believing they used course management systems effectively and 51% reported the same for audience response systems. Similarly, when asked which technologies enhanced teaching, the percentages of faculty members who named course management and audience response systems surpassed the others by an overwhelming majority (82% and 75%, respectively). Podcasts, smart board technologies, and lecture capture were thought to have a positive impact by 42%, 42%, and 39% of faculty members, respectively. When asked their opinion of the impact of these technologies on student learning, a majority of faculty members believed course management, audience response systems, and podcasts had a positive impact on student learning (81%, 77%, and 53%, respectively).

Faculty members perceived more pressure from colleagues (41%) and administrators (40%) than from students (26%) to incorporate additional technology in the classroom. There were significant differences among the 5 colleges and schools in the percentage of faculty members who felt pressure from colleagues and administrators. When private and public pharmacy colleges and schools and colleges were compared, faculty members from private institutions felt greater pressure from both colleagues and administrators to adopt technology (p=0.001 and p=0.003, respectively).

Faculty members were asked about technical difficulties, typical resolution time, and preference for training. Forty-six percent of faculty members reported having experienced technical difficulties frequently. When difficulties occurred, 69% of faculty members reported that the technical difficulties were resolved quickly. When difficulties were compared by private and public schools, private school faculty members reported fewer technical difficulties (p<0.001) but also reported that when difficulties occurred, they were resolved more slowly (p<0.001) compared with the experience of their counterparts at public institutions. A majority of faculty members (61.3%) felt it was inappropriate to use Web 2.0 tools in the classroom. Eighty-one percent of faculty members reported having changed their teaching methodology to accommodate the needs of the current generation of students.

Administrator Perceptions

Twelve administrators from 6 universities were interviewed. Nontechnical leaders who were interviewed included a university president, 2 vice provosts for teaching/learning, and 3 pharmacy college deans. Technical leaders who were interviewed included a chief information officer and 5 vice presidents or directors of technology. Table 3 presents common themes from the interviews regarding overall goals of technology on campus, decisions to select and purchase technology, and barriers to implementation. Technical leaders identified lecture capture, course management systems, and audience response systems as being the most widely implemented technologies on campus, as well as the top 3 most useful technologies for students and faculty members. Nontechnical leaders identified course management systems as the most useful technology on campus. Five of the nontechnical administrators expressed the opinion that the main role of technology in the classroom was to assist learning (83%).

Table 3.

Common Themes From Interviews With Administrators in a Study of Classroom Technology Use (N=12)

DISCUSSION

This report provides further insight on the use of educational technology by faculty members as well as perceptions from students and faculty members about the appropriateness of technology use in a convenience sample of 6 colleges and schools. Monaghan and colleagues surveyed 89 pharmacy colleges and schools and reported availability of educational technologies: course management systems (100%), audience response systems (89%), electronic testing (80%) and lecture-capture technology (70%).¹⁷ The availability of these educational technologies among the 6 institutions surveyed in our study was similar. Our analysis demonstrated, however, that technology was not uniformly used by faculty members. Students reported that course management systems and lecture-capture technology were the most frequently used technologies and that audience responses systems and electronic testing were used less frequently. These findings were also confirmed by faculty member responses. Students reported that faculty members most effectively used course management systems and lecture-capture technologies and identified these as most valuable to their learning. These perceptions are supported by the literature, which indicates that course management systems and podcasting engage students in learning and have a positive influence on learning outcomes.^10,16 Faculty members reported having confidence in the appropriate use of course management systems and audience response systems and thought that these had the greatest impact on learning outcomes. These findings are congruent with those of previous studies.^5-9,16

In a survey of pharmacy student experiences with educational technology, Stolte and colleagues reported that students most often used electronic class presentations, materials posted on course Web sites, and e-mail.¹⁸ Students in this study were also comfortable or very comfortable with the use of technology (89%), a finding similar to that in the current study, in which 83% of students identified themselves as tech-savvy. Although students in our study were younger than the cohort studied by Stolte and colleagues, neither study demonstrated significant differences in the use of technology among those <25 years of age compared with that among older students. Our results indicated significantly higher preferences for greater use of educational technology among tech-savvy and male students. Previous studies have demonstrated gender differences in technology comfort and preference. This finding may be attributable in part to the industry’s practice of marketing electronic games more to boys than girls and suggests that this early childhood experience influences technology preferences in education.^19,20

Substantial generational differences emerged when student and faculty respondents were compared. While only 0.6% of student participants identified themselves as “baby boomers,” more than 38% of faculty members fell into this category. Further, 89% of students identified with Generation Y, whereas only 12% of faculty members did so. Despite the generational gap between faculty members and students, the 2 groups were similar in the self-reported technical abilities: 20% and 53% of students strongly agreed or agreed that they were tech-savvy, and 15% and 55% of faculty members strongly agreed or agreed with this characterization.

In contrast to the investigators’ hypothesis that faculty members feel pressure to adopt new technologies from the digital-native generation of students, they feel more pressure from colleagues and administrators than from students. This finding can be explained by the recent proliferation of online learning and mobile technology at many institutions.²¹ Faculty members also compete with one another and may perceive pressure to use new technology or gadgets in the classroom when they see or hear about others becoming more technologically advanced in their teaching.

There were also differences between private and public institutions with respect to greater pressures to adopt new technology. These differences may be explained by a greater emphasis placed on teaching at private institutions compared with research-focused public institutions. While faculty members at private schools reported significantly fewer technical difficulties, problems were resolved significantly slower than at public institutions. This finding may reflect differences in technology support and resources between the 2 types of institutions.

Eighty-one percent of faculty members reported having changed their teaching methodologies to accommodate the preferences and needs of the current student generation. While we did not ascertain which specific changes in methods were made, this finding was confirmed by 91% of the students, who either agreed or strongly agreed that the teaching methodologies used by faculty members met their needs. Our study also suggests that faculty members’ concerns that students might skip live lectures and instead rely solely on recorded lectures (eg, podcasts) are unfounded. Student engagement and active learning in the classroom are likely to entice students to attend class, regardless of the availability of podcast recordings.

Our analysis represents opinions of 466 student pharmacists and 124 pharmacy faculty members in a convenience sample of 6 private and public institutions geographically distributed around the United States. While these numbers reflect a large sample of students and faculty members, they represent <1% of all students enrolled in the ACPE-accredited pharmacy programs (N=58,915) and 2% of all faculty members (N=6,400). ²² We believe that the response rates of 63% and 69%, respectively, are acceptable; however, because there was significant variation among colleges and schools in response rates, the sample in the study may not accurately represent the entire population of faculty members and students at all 6 schools. Other limitations of the study include inability to capture all characteristics of faculty members, students, administrators, and institutions involved. Further, we did not capture institutional culture and emphasis on the use of educational technology. Additionally, we chose to determine “appropriateness” of technology use by asking students and faculty members to report whether the technologies were used effectively and whether they had a positive impact on teaching and student learning. These are subjective measures of perceptions that may have been interpreted differently by different survey participants. Objective evidence of the impact of technology on student learning outcomes is challenging to ascertain because of the many variables affecting learning.

CONCLUSIONS

The colleges and schools of pharmacy surveyed used a variety of educational technologies. Although faculty members reported changing their teaching methods in response to the needs of the current generation of students, and students reported overall satisfaction with the appropriateness of technologies used, many exhibited a preference for even greater technology use. Despite the availability of technologies, not all faculty members used them in the classroom or felt that there is value to all technologies available. Given limited resources and support, administrators were faced with the challenge of adopting new educational technologies to meet the increased interest from faculty members and students. Faculty members who are avid technology users and advocates should direct future efforts toward objectively documenting the true impact of specific technologies and new teaching methodologies on student learning outcomes.