Abstract
Objectives:
Several public health education programs and government agencies across the country have started offering virtual or online training programs in emergency preparedness for people who are likely to be involved in managing or responding to different types of emergency situations such as natural disasters, epidemics, bioterrorism, etc. While such online training programs are more convenient and cost-effective than traditional classroom-based programs, their success depends to a great extent on the underlying technological environment. Specifically, in an online technological environment, different types of user experiences come in to play—users’ utilitarian or pragmatic experience, their fun or hedonic experience, their social experience, and most importantly, their usability experience—and these different user experiences critically shape the program outcomes, including course completion rates. This study adopts a multi-disciplinary approach and draws on theories in human computer interaction, distance learning theories, usability research, and online consumer behavior to evaluate users’ experience with the technological environment of an online emergency preparedness training program and discusses its implications for the design of effective online training programs. .
Methods:
Data was collected using a questionnaire from 377 subjects who had registered for and participated in online public health preparedness training courses offered by a large public university in the Northeast.
Results:
Analysis of the data indicates that as predicted, participants had higher levels of pragmatic and usability experiences compared to their hedonic and sociability experiences. Results also indicate that people who experienced higher levels of pragmatic, hedonic, sociability and usability experiences were more likely to complete the course(s) they registered for compared to those who reported lower levels.
Discussion:
The study findings hold important implications for the design of effective online emergency preparedness training targeted at diverse audiences including the general public, health care and public health professionals, and emergency responders. Strategies for improving participants’ pragmatic, hedonic, sociability and usability experiences are outlined.
Conclusion:
There are ample opportunities to improve the pragmatic, hedonic, sociability and usability experiences of the target audience. This is critical to improve the participants’ learning and retention as well as the completion rates for the courses offered. Online emergency preparedness programs are likely to play a crucial role in preparing emergency responders at all levels in the future and their success has critical implications for public health informatics.
Introduction
Preparing the public health workforce to mitigate, respond to, and recover from natural and man made disasters is not a minor undertaking. Both governmental and non governmental organizations have called on universities and other educational institutions to develop programs to efficiently and effectively train our public health workforce [1,2,3,4]. Many educational institutions across the country have responded to this by developing and offering virtual or online programs that incorporate ‘canned courses’—i.e. courses that do not require an instructor and instead allow students to download the materials and self-learn at their time of convenience [5].
The effectiveness of such online courses depends on delivering rich learning experiences for the students. However, unlike traditional classroom-based education, the online environment is not under the control of an instructor. Students’ learning experience in such online situations could be affected by not only the structure and content of the course but also the student interactions facilitated by the technology-based infrastructure and the usability of such infrastructure. Thus, to measure the effectiveness of online courses, we need to go beyond the evaluation tools that are currently used to evaluate offline or classroom-based courses and use tools that provide a more holistic view of users’ online learning experience.
Specifically, to understand and evaluate the learning experience in an online program, we need to draw on our understanding of people’s behavior in online environments. Prior studies in consumer psychology and human computer interaction offer an appropriate foundation for this. Research in consumer psychology indicate that experience has two primary dimensions—a utilitarian (or cognitive) dimension and a hedonic (or affective) dimension [6,7,8,9,10]. However, in an online environment, factors that are either related to the technology itself or to the interactions of the people with the technology could also shape such experience. Prior studies in the area of human-computer interaction and computer-mediated communication [11, 12, 13] indicate the relevance of two other dimensions—sociability experience and usability experience. In this study, the online offerings of an emergency preparedness program offered by a public university in the Northeast was evaluated on the above four dimensions of online user experience.
In addition, in this study we also examine whether online user experience had any impact on course completion. Prior research in this area has shown that online distance education courses often have higher non-completion rates than traditional in-class courses [14, 15]. The reasons cited for this include student isolation and technological barriers which in turn de-motivates students and lead to course drop out [16, 17, 18]. The current study will provide insights into how the technological environments can be developed so that users (i.e. students) would not only learn but also have a positive experience that in turn enhances the probability of course completion.
Further, we empirically show that higher levels of student self-motivation do not translate into course completion, which in turn emphasizes the need to focus on student’s experience during the course to enhance program success.
The remainder of this paper is organized as follows: The next section reviews the background literature and theories for this study: 1) workforce development for public health emergency preparedness, 2) online learning environment and online consumer experience, 3) cognitive affective learning, 4) social Learning theory, 5) usability in distance-learning environments, and 6) motivation and course completion. Following that we formally define our study research questions. This is followed by the methodology section which includes details on data collection and data analysis. Next, we discuss the study results and their implications. The report ends with a brief conclusion and key recommendations for improving the users’ online experience and thereby enhancing program effectiveness. .
Background
Workforce development for public health emergency preparedness
In 1997, the U.S. Department of Health and Human Services issued a report titled ‘The Public Health Workforce: An Agenda for the 21st Century’, which highlighted the gap in training and preparation for public health professionals for emergency preparedness [2]. It is estimated that there are around 500,000 people in the public health workforce at the federal, state and local levels. In addition, there are around 3 million people working in the healthcare system (private and non profit) who play a key role in public health emergencies [1]. In case of an emergency situation, be it an epidemic, terrorist attack or a natural disaster, these are the people who will be deployed to the front lines and the report raised concerns regarding their training and readiness. According to their assessment this “compelling and urgent programmatic forces are making enhanced training and education opportunities for public health professionals a necessity.” [1].
As a result, in September 2000, the Center for Disease Control (CDC) and the Association of Schools of Public Health together brought out a plan to develop a national network of public health preparedness centers. As part of the plan, they funded several University programs to start Centers of Public Health Preparedness (CPHP) in around 10 regions across the country. CPHPs in all these regions have been offering relevant courses to train the public health workforce for emergency preparedness [4].
The Institute of Medicine later released a report in 2003 titled “Who Will Keep the Public Healthy?” that not only reiterated the need for education and training for the public health workforce, but also stated that online distance learning was the best solution to train this large number of diverse public health workers in a cost-effective manner. This has led several state universities and local governmental agencies to start their own online education programs [19, 20] for training the public health workforce in emergency preparedness. Despite the growing numbers of such programs, there have been very few initiatives focused on evaluating the online learning environments of these programs, especially for the CPHP offerings, other than the evaluations done by CDC itself.
Online (or distance) education is definitely a cost effective and efficient way of training such large numbers of public health workforce. However, in order to evaluate such programs, one needs to adopt an interdisciplinary perspective as diverse aspects (technology, social, etc.) assume importance. This study offers a theory based framework drawn from multiple disciplines to evaluate the online environments of such distance education programs.
Online learning environment and online experience
Online classrooms and learning environments are inevitable to meet the demands of training requirements for public health emergency preparedness. It provides the economies of scale and convenience that will not be available in traditional classroom settings. There are innumerable benefits for students from online distance learning, flexibility and convenience perhaps being the most important [16]. However, this is not without many disadvantages and problems [21,22,23]. Student isolation [17] and student frustrations [18] have been found to be two of the major disadvantages with online distance education. A recent study on an online medical self-paced course noted that the major challenges were technological problems and the fact that opportunities for social interaction was much lower [16]. Another study found that there were eight main factors that impede online distance education: administrative issues, lack of social interaction, academic skills, technical skills, learner motivation, time and support for studies, cost and access to the Internet, and technical problems [24].
Sustained frustrations and isolation can impede learning, especially the cognitive and affective dimensions of the learning experience [25]. Studies in this area show that these would also decrease the storage and processing capacity of working memory [26, 27]. In addition, frustration and anxiety are major factors that lead to de-motivation among students [25]. Motivation is critical for this kind of online learning environments [28]. It becomes even more critical when training public health emergency preparedness workers as many students are much older, have full time jobs and other work and family commitments, as compared to young college students [20].
Cognitive Affective learning
Similar to the research in consumer psychology, where pragmatic and hedonic component of experience received much attention, in the education and learning literature, the cognitive (pragmatic) and affective (or hedonic) dimensions of learning has been the focus of many researchers. The cognitive dimension was considered most critical for learning in many of the earlier studies.
While the cognitive dimension is critical, researchers also began discovering that there is an affective dimension that impacts learning, memory, retention and inference making. More recently this component received even more focus in the context of online learning which led researchers in the MIT Media Lab to work on affective agents where a robotic computer aims to improve user’s motivation to learn. The robotic computer is capable of expressing affect by rewarding or showing pleasure when the learner does something right, and when the learner gets distracted, it would try to entertain the learner and so on. There has also been significant work done in developing affective interface agents that are capable of working as teaching assistants in monitoring and managing online distance learning [29, 30, 31]. The objective of this line of research is to detect the affective or emotional state of the learner and provide appropriate affective or hedonic support to keep the learner engaged in the content and also motivate them to complete the tasks before them.
Research in consumer online behavior shows that when users are engrossed in the online activity, they do not keep track of time and get into a state of “flow” [32, 33, 34]. This stream of research suggests that when people are provided with activities that they get engrossed in and start deriving fun from, they reach a state of flow [34]. In the online learning environment, if students are provided with activities that they could get immersed in and achieve a state of flow, it would not only improve learning but also enhance the course completion rates and the student retention rate. Hence, while the cognitive dimension of learning leads the student to evaluate the pragmatic value or the usefulness of the course content, it is the affective component that enables the user to have a better hedonic experience.
Social Learning theory
Another relevant stream of research adopts the social learning perspective in which the conjecture is that knowledge is socially constructed and occurs when individuals engage in discourse about a subject matter [35, 36, 37]. Knowledge is embedded in individuals, and by providing effective communication channels and opportunities to interact with one another—either socially or in a classroom setting—it would lead to more knowledge transfer and creation, and in turn offer a richer learning environment [36, 37].
This perspective has been widely accepted in the context of online distance education and it is often emphasized that student interactions are central and critical for a successful learning experience and consequently the success of online courses [35, 38]. These interactions could be with other students or with the instructor. In the context of online public health emergency preparedness courses, especially CPHP courses, almost all the courses are ‘canned courses’ without an instructor or fellow students. This could potentially affect the sociability experience and thereby impact learning and course completion rates.
Usability in distance-learning environments
As mentioned previously, technological barriers and usability issues are the two most often cited reasons for student frustrations and poor completion rates. Several studies have considered the usability issues of different online courses [39, 40, 41] and have broadly concluded that usability is a critical factor in determining the success of any online course.
Usability is the extent to which a user can successfully accomplish the tasks with effectiveness and efficiency [42]. In the distance education context, usability would be the effective and efficient accomplishment of learning related tasks or goals in the online environment (with or without using specified tools for that system). In the context of emergency preparedness training courses, it is a critical evaluation component as users’ interaction with the system is more than users’ interaction with the instructor.
Usability issues are more widely accepted by course providers as a potential problem and many understand the need to rectify them. However, usability issues are much more difficult to evaluate as users often attribute usability issues to their own lack of skills or a problem at their end (for example, their problematic home computer or Internet connection). In addition, many specific usability questions such as “is navigation through the website easy or difficult?” can be answered in two different ways – navigation through the website is easy or difficult for ‘everybody else who is skilled in computing technology’; or navigation through the website is easy or difficult for ‘me’ specifically. Analysis of the results also becomes difficult as users may hold different technology standards, different levels of skills, and access to different levels of technological assistance.
To overcome these measurements issues, in the current study, we used a simple pre-validated scale to evaluate whether the overall technological environment was easy/difficult; confusing/not confusing; consistent/inconsistent; stressful/not stressful; simple/complicated and tiring/not tiring. This usability tool has been found to be effective in understanding whether the overall usability experience was satisfactory to the user [43, 44].
Motivation and course completion
Lack of motivation has been cited as one of the major impediments to online learning [15, 24, 45, 46]. Motivation to enroll for courses could come from both internal forces and external forces [47].
Intrinsic (or internal) motivation has been indicated as one of the key factors that drive people to register for courses as it reflects a person’s need to enhance their skill set, their market value, self-esteem, etc. Extrinsic (or external) motivation relates to one’s profession including mandatory job requirements, CPE credits, suggestion from one’s boss and colleagues, etc. Extrinsic motivation would also include motivation from educational institutions or the course providers (e.g. instructors, universities, and program administrators). However, the CPHP is not organized to provide this kind of motivation. Hence, the main sources of extrinsic motivation seemed to be from their own professional life.
While both intrinsic and extrinsic motivation could certainly lead students to register for courses, there is no evidence yet that this would lead to course completion. In this study we empirically examine whether there is any significant difference in the intrinsic and extrinsic motivation levels of students who completed the courses and that of students who did not complete the courses.
Research Questions:
The above literature review suggests that poor online course experience (that in turn may arise from a lack of instructors, lack of social interactions, technological problems in the online courses, etc) could de-motivate students and lead them to drop the courses that they had registered for. The discussion also suggests that motivation to enroll for a course, while an important factor, may not be enough to ensure that the student completes the course.
Thus, in our empirical study, we address two research questions that reflect the above two issues. First, are there any significant differences in students’ online course experiences (pragmatic, hedonic, sociability and usability) based on their course completion status? Second, are there any significant differences in students’ intrinsic and extrinsic motivation levels based on their course completion status?
Based on the theories and concepts outlined previously, we define student’s online course experience—i.e. the overall experience a student derives from his or her interaction in the online course environment—along four dimensions: pragmatic, hedonic, sociability, and usability.
Pragmatic experience is the pragmatic or utilitarian value the student experiences in the online learning environment. This dimension is related to goal-oriented behavior [33] of the student and reflects whether the student found the experience in the online learning environment useful, valuable, and/or worthwhile [43, 44]. The hedonic dimension is the intrinsic value the customer derives from the interactions in the online learning environment. It reflects the enjoyment and excitement students derive during the learning process as well as during their interactions in the online learning environment. The sociability dimension is the social experience students derive from the interactions in the online learning environment. It captures students’ perceptions regarding the overall openness, friendliness and politeness of the community in the learning environment [11,48]. Even though there weren’t much human-human interactions in this study context, there were human-computer interactions and such interactions can also lead to sociability experience [48]. The usability dimension is defined as the students’ experience in navigating and using the online materials. As such, this dimension captures the ease of use and clarity of the technological features of the online learning environment. Higher levels of usability experience reflect the ability of the student to navigate and participate in the online learning environment smoothly and effortlessly and without any obstructions or annoyances that might distract them from their goals or interests [11].
Next, we describe our empirical study..
Method
Data collection and Data analysis
Data was collected using a Web-based questionnaire from students who had registered for the courses offered by a CPHP based in a large public university in upstate NY. Emails were sent to approximately 2700 students who had enrolled in one or more of the courses during the past one year. Each email briefly described the study and invited the student to respond to a survey—the link to the survey was included in the email (the survey was available from the CPHP’s Web site). There were 415 responses to the email invite. 38 responses had to be excluded from the analysis due to high amount of missing data. Thus, there were a total of 377 usable responses.
Data was collected on different aspects of the online program, including, student motivations, student profile, and their overall experience with the CPHP Web site and with the courses (specifically, the 4 dimensions of user experience—pragmatic, hedonic, sociability and usability). The questionnaire was built using existing scales for measuring each of the variables. Student’s online course experience was measured using an existing validated scale designed to measure online experience [43, 44]. A tool to measure student motivation to enroll was developed by the CPHP staff for an earlier study and was adapted and used in this survey. Course completion data was collected using a simple yes/no question as to whether they completed all the courses they had registered for.
A factor analysis of the data related to student motivation yielded two distinct factors— intrinsic motivation and extrinsic motivation. See Table 1 for items and their factor loadings. The intrinsic motivation factor included 4 items and extrinsic motivation factor also had 4 items.
Table 1:
Factor scores for ‘motivation’
| Intrinsic motivation | |
| I registered to gain more knowledge | .922 |
| I registered myself for personal development | .922 |
| I registered myself for professional development | .887 |
| I registered to do something useful/constructive | .810 |
| Extrinsic motivation | |
| It was required for other educational programs | .839 |
| It was recommended by someone outside my workplace | .703 |
| It provided continuing education credit | .620 |
| It was required/highly recommended for my job | .588 |
Similarly factor analysis for each of the online experience dimensions were done separately. Items and factor loadings are provided in Table 2. As can be seen in Table 2, pragmatic experience was measured using a 7 item scale (reliability α = 0.96), hedonic experience by using a 9 item scale (α = 0.95), sociability experience by using a 5 item scale (α = 0.87), and usability experience by using 6 item scale (α = 0.91).
Table 2:
Factor scores for Online Experience
| Pragmatic | Scores |
|
| |
| Valuable/Not Valuable | .938 |
| Practical/Impractical | .918 |
| Relevant/Irrelevant | .915 |
| Informative/Not Informative | .905 |
| Worthwhile/Worthless | .904 |
| Productive/Not Productive | .903 |
| Useful/Not Useful | .893 |
|
| |
| Hedonic | |
|
| |
| Stimulating/Boring | .914 |
| Exciting/Not exciting | .892 |
| Captivating/Not captivating | .872 |
| Fun/Not fun | .856 |
| Satisfying/Unsatisfying | .846 |
| Enjoyable/Not enjoyable | .831 |
| Entertaining/Not entertaining | .809 |
| Deeply engrossing/Not deeply engrossing | .803 |
| Pleasant/Unpleasant | .802 |
|
| |
| Sociability | |
|
| |
| Inviting/Uninviting | .851 |
| Friendly/Unfriendly | .840 |
| Polite/Impolite | .808 |
| Personal/Impersonal | .799 |
| Social/Unsocial | .748 |
|
| |
| Usability | |
|
| |
| Simple/Complicated | .866 |
| Easy/Difficult | .858 |
| Confusing/Not Confusing | .827 |
| Not Tiring/Tiring | .827 |
| Consistent/Inconsistent | .826 |
| Stressful/Not Stressful | .807 |
An independent sample t test was used to compare the means of the four experience dimensions (pragmatic, hedonic, sociability and usability) between students who completed all the courses they registered for and students who did not complete one or more of the courses they registered for. The data was analyzed using SPSS, all the experience dimensions were entered as test variables and the item ‘Did you complete all the courses you registered for’ was entered as the grouping variable. Similarly, an independent sample t test was used to compare the means of intrinsic and extrinsic motivation between students who completed all the courses they registered for and students who did not complete all the courses they registered for.
Results and Discussion
Majority of students had registered for just one course. Specifically, 157 people (41.6%) registered for 1 course; 73 people (19.4%) registered for 2 courses, and 24 people (6.4%) had enrolled for a course that was not listed in the survey. The study sample also included students from 31 countries although the large majority was from the United States. The number of female students was much higher (61%). This represents the actual student population ratio at this CPHP. Racial distribution was as follows: 73.7% White non Hispanic, 7.6% black non-Hispanic, 5.4% Hispanic or Latino and 5% South East Asian. This distribution also mirrors the student population distribution at this CPHP.
The mean and standard deviation for all the 4 dimensions of online experience and the two factors of motivation are provided in Table 3.
Table 3.
Means and Standard Deviation of Study Variables
| Variables | Mean | S.D |
|---|---|---|
| 1. Pragmatic experience | 6.1 | 1.12 |
| 2. Hedonic experience | 5.0 | 1.26 |
| 3. Sociability experience | 4.9 | 1.22 |
| 4. Usability experience | 5.6 | 1.15 |
| 5. Intrinsic motivation | 5.4 | 1.96 |
| 6. Extrinsic motivation | 3.8 | 1.52 |
Online experience & course completion
Analysis of the data indicates that, overall, participants had higher levels of pragmatic and usability experience compared to hedonic and sociability experience (see mean values in Table 3).
The results from the independent sample t-test showed that there was significant difference in the scores for all the 4 dimensions of experience between students who completed the course and students who did not. Results are shown in Table 4.
Table 4.
T test results for online experience & course completion
| Experience | Means | N | Std. deviation | DoF | T value |
|---|---|---|---|---|---|
| Pragmatic | Yes 6.2 No 5.5 |
275 67 |
.96 1.48 |
79.9 | 3.89*** |
| Hedonic | Yes 5.1 No 4.4 |
274 67 |
1.15 1.53 |
84.9 | 3.67*** |
| Sociability | Yes 5.1 No 4.3 |
275 67 |
1.07 1.57 |
81.6 | 3.68*** |
| Usability | Yes 5.8 No 4.9 |
275 67 |
1.04 1.32 |
87.3 | 4.92*** |
DoF – Degrees of freedom
Yes – Completed all the courses they enrolled
No – Did not complete all the courses they enrolled
p<.001;
p<.01;
p<.05
The mean scores for pragmatic experience for students who completed the courses (M =6.2, SD =.96) was significantly higher than for those who did not complete the courses (M=5.5, SD=1.48); t(79.9)=3.89, p<.001. The mean scores for hedonic experience for students who completed the courses (M =5.1, SD =1.15) was significantly higher than for those who did not complete the courses (M=4.4, SD=1.53); t(84.9)=3.67, p<.001. Similarly, the mean scores for sociability experience for students who completed the courses (M =5.1, SD = 1.07) was significantly higher than for those who did not complete the courses (M=4.3, SD=1.57); t(81.6)=3.68, p<.001. Finally, the mean scores for usability experience for students who completed the courses (M =5.8, SD =1.04) was significantly higher than for those who did not complete the courses (M=4.9, SD=1.32); t(87.3)=4.92, p<.001.
Overall, the results support the broader study thesis that students who experience higher levels of pragmatic, hedonic, sociability and usability experiences are more likely to complete the course(s) they registered for compared to those who report lower levels. In other words, these results indicate that people who dropped out had less positive online experience on all the four dimensions—pragmatic, hedonic, sociability and usability. The four-dimensional online experience questionnaire is useful in such situations where one can capture the underlying experience and derive insights on what aspect of the user experience really leads to non completion.
Our analysis also shows that students rated ‘hedonic’ experience and ‘sociability’ experience lower than ‘pragmatic’ and usability experience. For sociability experience, a sizeable number of the students gave a rating of 4 (neutral) on a scale of 1 to 7, which indicates that they did not perceive sociability to be either negative or positive. It could also indicate the lack of sociability experience in this CPHP online program.
Motivation and course completion
There was no statistically significant difference in students’ extrinsic motivation levels between those who completed all the courses they registered for and those who didn’t (See Table 5). There was mild statistically significant difference in students’ intrinsic levels between those who completed all the courses they registered for and those who did not (M=5.2, SD =2.01); t(106.7) = −3.2, p<.05. However, the results from the independent sample t test indicate a negative effect. In other words, students who completed all the courses they registered for had lower intrinsic motivation levels compared to those who did not complete all the courses they registered for. This indicates that lower levels of intrinsic motivation do not imply that they would drop out from the course. On the same lines, higher levels of intrinsic motivation do not imply that they would complete the course. In short, the results from this study indicate that student motivation (both intrinsic and extrinsic) is not a good predictor of course completion.
Table 5.
T test results for Motivation and course completion
| Motivation | Means | N | Std. deviation | DoF | T value |
|---|---|---|---|---|---|
| Intrinsic | Yes 5.2 No 6.0 |
238 59 |
2.01 1.62 |
106.7 | −3.2* |
| Extrinsic | Yes 3.8 No 3.9 |
213 51 |
1.45 1.78 |
66.7 | −.259 |
DoF – Degrees of freedom
Yes – Completed all the courses they enrolled
No – Did not complete all the courses they enrolled
p<.001;
p<.01;
p<.05
This finding combined with the earlier finding further indicates the importance of students’ online experience (all the four dimensions) for maintaining student interest and ensuring that they complete the courses. In other words, while motivation may play a key role in bringing the student to the program (i.e. enrolling for the course), it is their perceived experience during the online course that critically determines whether or not they would complete the course.
Study Implications
The results from this study have several implications for CPHPs, and more generally, for similar online training programs. First, this study indicates the need to focus on the four key dimensions of user’s online experience (i.e. their underlying feelings and perceptions) rather than on ad-hoc issues. Prior studies and evaluations have mainly considered specific problems perceived by the course provider rather than the actual user experience. The evaluation tool described here brings out users’ sentiments about different aspects of the program and gives a much more fundamental and holistic understanding of the program’s potential weaknesses and areas for improvement.
The study also highlights the importance of hedonic and sociability experience for students in such online training environments. Many online courses focus mainly on the pragmatic value of a particular course for the students and neglect the potential hedonic experience. While pragmatic experience is important and should be the primary focus, boredom and lack of fun can make students weary and de-motivated. Including elements that enhance fun and entertainment as part of the learning experience would be invaluable. Even in face-to-face classroom situations, instructors constantly try to include fun activities such as field trips, role-playing, including videos and movie clips etc that are relevant for the content of the course. The type of fun activities would be different in an online course (a few suggestions are provided in Table 6), but necessary especially for training programs that use ‘canned courses’.
Table 6.
Key strategies for improving course completion rates and overall program success
| Key Strategies | How to | Benefits for students | Benefits for CPHP |
|---|---|---|---|
| 1) Improving pragmatic experience |
Expectations management:
|
|
|
| 2) Improving hedonic experience |
Improve fun and entertainment:
|
|
|
| 3) Improving sociability experience |
Improve possibilities for social interaction.
|
|
|
| 4) Improving usability experience |
1) Improve usability experience by using some of the standard usability practices (Nielson, 2000).
3) Make CPHP courses accessible through Mobile phones Mobile phones are now very well equipped with fast connections and readable interfaces. |
|
|
As discussed previously, social learning theory suggests the importance of, sociability experience in learning; the current study findings indicate that sociability experience is equally important to ensure higher course completion rates. Good sociability experience prevents students from “feeling lonely”, and more importantly, enables them to engage in “active learning”. Indeed, student interactions have been found to be critical for the success of many online distance education courses [49, 50, 51]. Such interactions allow students to feel that they are part of a community of learners and share experiential knowledge that enhance the overall quality of learning.
Finally, this study found that majority of the students who enrolled in these programs were self-driven or self-motivated. Intrinsic factors such as professional and personal development seem to drive these public health professionals to enroll in such training courses. At the same time, such motivation did not translate into ensuring course completion. This implies that rather than depend on student motivation, course providers would need to provide such self-motivated individuals with a positive and engaging online learning experience to ensure high levels of course completion.
Conclusions and Key Recommendations
Key recommendations that follow from the study findings are given below (also summarized in Table 6).
1) Improving pragmatic experience: In this study, the majority of the students found the courses to be useful and valuable (the mean score for pragmatic experience was higher than those for the other three experience dimensions). However, this is still relative to the very low hedonic and sociability experiences and indicates the potential for improvement. An important means to enhance pragmatic experience is effective student expectations management. Students should be able understand upfront what they will be getting out of each course. This can be done by bringing more clarity to course descriptions and also detailing as to what specific goals students will be able to accomplish by taking each course. It will also help to indicate who would benefit by taking a particular course.
2) Improving Hedonic experience: Hedonic or fun and entertainment from these courses were rated quite low. It is true that fun and entertainment is not one of the primary objectives of these courses. However, as mentioned previously, when people get engrossed in the learning material, their learning and retention of the material are typically much higher [52]. In addition, they would try to finish the courses, instead of procrastinating and/or getting distracted with other things. An effective way to improve hedonic experience is to create more interactive and fun user interface. For example, one could incorporate video clips made in ‘Second Life’ that will give the user a personal view of a disaster and how things could be as he/she approaches a disaster area in addition to being a fun experience. Play2train http://play2train.us/wordpress/ developed by Idaho Bioterrorism Awareness and Preparedness Program using ‘Second Life’ is a good example of this.
3) Improving Sociability experience: Sociability experience was another weak factor in CPHP courses... As noted previously, positive sociability experience would enable students to feel that they are part of the overall community of students who are enrolled in the program. One solution would be to provide students with access to an online community/forum within the CPHP that will enable interactions with fellow students as well as with the CPHP staff. This would not only improve students’ sociability experience, but also enhance their learning and networking potential, and in turn, improve student retention. Developing such forums is a very cost-effective solution with proven benefits given the low cost of associated information technologies.
4) Improving Usability experience: It is important to ensure that the design of the online environment provides seamless and enjoyable navigation experience for the user. Best practices in usability include offering simple and clutter less user interface, intuitive navigational features, and avoiding technological jargons in user guidance. . In addition, usability can be significantly improved by offering online programs on mobile platforms and thereby catering to today’s public health worker who is likely to be very mobile. If courses can be accessed through smart phones (this would require redesigning the interface to fit the mobile device) it would improve the convenience factor significantly.
In conclusion, there are ample opportunities to improve the pragmatic, hedonic, sociability and usability experiences of the target audience. This is critical to improve the participants’ learning and retention as well as the completion rates for the courses offered. Online emergency preparedness programs are likely to play a crucial role in preparing emergency responders at all levels in the future and their success has critical implications for public health informatics.
However, we need more studies in the future to understand the factors that affect students’ overall experience in the online learning environment of CPHP courses. Future research could focus on understanding how the experience (pragmatic, hedonic, sociability and usability) would impact student performance in the courses, student learning and retention of the subject matter, and more importantly, their real life job performance. In addition, conducting qualitative studies with a cohort group of students could help us better understand the factors that shape the overall experience specific to this set of population as well as whether such online training is an effective long term solution for training our public health workers.
References
- [1].Centers for Disease Control and Prevention (US) Executive Summary. 2002. Feb. Bioterrorism and public health emergency preparedness and response: a national collaborative training plan: strengthening preparedness at the frontlines. [Google Scholar]
- [2].Department of Health and Human Services, Public Health Service (US) The public health workforce: an agenda for the 21st century, a report of the public health functions project, 1997. Available from: URL: http://www.health.gov/phfunctions/pubhlth.pdf.
- [3].Institute of Medicine of the National Academies Who will keep the public healthy: educating public health professionals for the 21st century. (Accessed on May 1, 2010). November 4, 2002. Available from: URL: http://www.iom.edu/report.asp?id=4307. [PubMed]
- [4].Office for Domestic Preparedness, Department of Health and Human Services (US) ODP approach for blended learning key program information. February 10, 2003 Version 1. (Accessed on May 1, 2010) Available from: URL: http://www.ojp.usdoj.gov/odp/training_bl.htm.
- [5].Office of Public Health Preparedness and Response (OPHPR) Centers for Public Health Preparedness (CPHP) Program. (Accessed on May 1, 2010) Available from: URL: http://emergency.cdc.gov/cdcpreparedness/cphp/background.asp.
- [6].Forgas JP, editor. Feeling and Thinking: Affective influences on social cognition. New York: Cambridge University Press; 2000. [Google Scholar]
- [7].Isen AM, Labroo AA. Some ways in which positive affect facilitates decision making and judgment. In: Schneider S, Shanteau J, editors. Emerging Perspectives on Judgment and Decision Research 2003. NY, Cambridge: pp. 365–393. [Google Scholar]
- [8].Overby JW, Lee EJ. The effects of utilitarian and hedonic online shopping value on consumer preference and intentions. J Bus Res. 2006;59:1160–1166. [Google Scholar]
- [9].Voss KE, Spangenberg ER, Grohmann B. Measuring the hedonic and utilitarian dimensions of consumer attitude. J Mktg Res. 2003;XI:310–320. [Google Scholar]
- [10].Dhar R, Wertenbroch K. Consumer choice between hedonic and utilitarian goods. J Mktg Res. 2000;XXXVII:60–71. [Google Scholar]
- [11].Nambisan P, Watt JH. Managing customer experience in online communities. Journal of Business Research. In press. [Google Scholar]
- [12].Nielsen J. Designing Web Usability: The Practice of Simplicity. Indianapolis: New Riders; 2000. [Google Scholar]
- [13].Shneiderman B, Catherine P. Designing the User Interface: Strategies for Effective Human-Computer Interaction. 4th ed. Boston: Addison Wesley Professional; 2004. [Google Scholar]
- [14].Dupin-Bryant PA. Pre-entry variables related to retention in online distance education [Electronic version] The American Journal of Distance Education. 2004;18(4):199–206. [Google Scholar]
- [15].Huett JB, Kalinowski KE, Moller L, Huett KC. Improving the Motivation and Retention of Online Students Through the Use of ARCS-Based E-Mails. American Journal of Distance Education. 2008;2008;22(3):159–176. [Google Scholar]
- [16].May L, Acquaviva KD, Dorfman A, Laurie Posey L. Medical Student Perceptions of Self-Paced, Web-Based Electives: A Descriptive Study. American Journal of Distance Education. 2009;23(4):212–223. [Google Scholar]
- [17].Twigg CA. Is technology a silver bullet? Educom Review. 1997:28–29. [Google Scholar]
- [18].Hara N, Kling R. Students' Frustrations with a Web-Based Distance Education Course. First Monday. 1999;4(12) (December) at http://www.firstmonday.dk/issues/issue4_12/index.html. [Google Scholar]
- [19].Harrison LM, Davis MV, MacDonald PDM, Alexander LK, Cline JS, Alexander JG, Rothney EE, Rybka TP, Stevens RH. Development and Implementation of a Public Health Workforce Training Needs Assessment Survey in North Carolina. Public Health Rep. 2005;120(Suppl 1):28–34. doi: 10.1177/00333549051200S107. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [20].NYSDOH Roadmap: Strengthening the Public Health Workforce in New York State 2006. [Accessed on May 1, 2010. Available from URL: http://www.health.state.ny.us/press/reports/docs/strengthening_the_public_health_workforce.pdf.
- [21].Bromley H, Apple MW. Education/Technology/Power: Educational computing as a social practice. Albany, N.Y: SUNY Press; 1998. [Google Scholar]
- [22].Jaffee D. Institutionalized resistance to asynchronous learning networks. Journal of Asynchronous Learning Networks. 1998;2(2) http://www.aln.org/alnweb/journal/vol2_issue2/jaffee.htm. [Google Scholar]
- [23].Wegerif R. The Social dimension of asynchronous learning networks. Journal of Asynchronous Learning Networks. 1998;2(1) http://www.aln.org/alnweb/journal/vol2_issue1/wegerif.htm. [Google Scholar]
- [24].Muilenburg LY, Berge ZL. Student barriers to online learning: A factor analytic study. Distance Education. 2005;26(1):29–48. [Google Scholar]
- [25].Jonassen DH, Grabowski BL. Handbook of individual differences, learning, and instruction. Hillsdale, N.J: Erlbaum; 1993. [Google Scholar]
- [26].Darke S. Anxiety and working memory capacity. Cognition and emotion. 1998;2(2):145–154. [Google Scholar]
- [27].Darke S. Effects of anxiety on inferential reasoning task performance. Journal of Personality and Social Psychology. 1988;55(3):499–505. doi: 10.1037//0022-3514.55.3.499. [DOI] [PubMed] [Google Scholar]
- [28].Abrahamson CE. Issues in interactive communication in distance education. College Student Journal. 1998;32(1):33–43. [Google Scholar]
- [29].Choy SO, Ng SC, Tsang YC. Software agents to assist in distance learning environments. Educause Quarterly. 2005;28(2):34–40. [Google Scholar]
- [30].Jafari A. Conceptualizing Intelligent Agents for teaching and learning. EDUCAUSE Quarterly. 2002;25(3):28–34. [Google Scholar]
- [31].Neji M, Ben Ammar M. Agent-based Collaborative Affective e-Learning Framework. The Electronic Journal of e-Learning. 2007;5(2):123–134. available online at www.ejel.org. [Google Scholar]
- [32].Csikszentmihalyi M. Flow: The Psychology of Optimal Experience. Harper & Row; New York, NY: 1990. [Google Scholar]
- [33].Hoffman DL, Novak TP. Flow Online: Lessons Learned and Future Prospects. Journal of Interactive Marketing. 2009;23(1):23–34. (pdf) [Google Scholar]
- [34].Ghani JA, Deshpande SP. Task Characteristics and the Experience of Optimal Flow in Human-Computer Interaction. The Journal of Psychology. 1994;128(4):381–391. [Google Scholar]
- [35].Hill JR, Song L, West RE. Social Learning Theory and Web-Based Learning Environments: A Review of Research and Discussion of Implications. American Journal of Distance Education. 2009;23(2):88–103. [Google Scholar]
- [36].Henning W. Everyday cognition and situated learning. In: Jonassen D, editor. Handbook of research on educational communications and technology. 2nd ed. Mahwah, NY: Erlbaum; 2004. pp. 143–168. [Google Scholar]
- [37].Schmitt FF. The Justification of Group Beliefs. In: Schmitt FF, editor. Socializing Epistemology: The Social Dimensions of Knowledge. Maryland: Rowman and Littlefield; 1994. pp. 257–88. [Google Scholar]
- [38].Garrison DR, Cleveland-Innes M. Facilitating cognitive presence in online learning: Interaction is not enough. The American Journal of Distance Education. 2005;19(3):133–148. [Google Scholar]
- [39].Tselios N, Avouris N, Dimitracopoulou A, Daskalaki S. Evaluation of Distance-learning Environments: Impact of Usability on Student Performance. International Journal of Educational Telecommunications. 2001;7(4):355–378. [Google Scholar]
- [40].Duggan MH, Adcock AB. Animated Agents Teaching Helping Skills in an Online Environment: A Pilot Study. Journal of Interactive Online Learning. 2007;6(1):56–71. [Google Scholar]
- [41].Volery T, Lord D. Critical success factors in online education. International Journal of Educational Management. 2000;14(5):216–223. [Google Scholar]
- [42].Karat J. Evolving the scope of user-centered design. Communications of the ACM. 1997;40(7):33–38. [Google Scholar]
- [43].Nambisan P, Gustafson D, Pingree S, Hawkins R. Patients’ Sociability and Usability Experience in Online Health Communities: Impact on Attitudes towards the Healthcare Organization and its services. International Journal of Web-based Communities (Special Issue on Web-based Communities and Healthcare) 2010;6(4):395–409. [Google Scholar]
- [44].Nambisan P. Evaluating Patient Experience in Online Health Communities: Implications for Healthcare Organizations. Health care Management Review. doi: 10.1097/HMR.0b013e3182099f82. In press. [DOI] [PubMed] [Google Scholar]
- [45].Astleitner H, Keller J. A model for motivationally adaptive computer-assisted instruction. Journal of Research on Computing in Education. 1995;27(3):270–280. [Google Scholar]
- [46].Gabrielle D. The effects of technology-mediated instructional strategies on motivation, performance, and self-directed learning. 2003. PhD diss, Florida State University, Tallahassee. (Accessed on May 10, 2010) Available at http://gabrielleconsulting.com/docs/gabrielleaect.pdf.
- [47].Driscoll MP. Psychology of learning for instruction. Boston: Allyn and Bacon; 1994. [Google Scholar]
- [48].Kreijns K, Kirschner PA, Jochems W. The Sociability of Computer-Supported Collaborative Learning Environments. Educational Technology & Society. 2002;5(1):8–22. [Google Scholar]
- [49].Brown RE. The process of community-building in distance learning classes. Journal of Asynchronous Learning Networks. 2001;5(2):18–35. [Google Scholar]
- [50].Swan K. Building learning communities in online courses: the importance of interaction. Education, Communication & Information. 2002;2002;2(1):23–49. [Google Scholar]
- [51].Richardson JC, Swan K. Examining social presence in online courses in relation to students’ perceived learning and satisfaction. Journal of Asynchronous Learning Networks. 2003;2003;7(1):68–88. [Google Scholar]
- [52].Malone TW, Lepper MR. Making Learning Fun: A Taxonomy of Intrinsic Motivations for Learning. In: Snow RE, Farr MJ, editors. Aptitude, Learning and Instruction. Hillsdale, NJ: Ebraum; 1987. pp. 223–253. [Google Scholar]