EVALUATING A HYBRID WEB-BASED BASIC GENETICS COURSE FOR HEALTH PROFESSIONALS

Abstract

Health professionals, particularly nurses, continue to struggle with the expanding role of genetics information in the care of their patients. This paper describes an evaluation study of the effectiveness of a hybrid basic genetics course for healthcare professionals combining web-based learning with traditional face-to-face instructional techniques. A multidisciplinary group from the National Institutes of Health (NIH) created “Basic Genetics Education for Healthcare Providers” (BGEHCP). This program combined 7 web-based self-education modules with monthly traditional face-to-face lectures by genetics experts. The course was pilot tested by 186 healthcare providers from various disciplines with 69% (n=129) of the class registrants enrolling in a pre-post evaluation trial. Outcome measures included critical thinking knowledge items and a Web-based Learning Environment Inventory (WEBLEI). Results indicated a significant (p<0.001) change in knowledge scores. WEBLEI scores indicated program effectiveness particularly in the area of convenience, access and the course structure and design. Although significant increases in overall knowledge scores were achieved, scores in content areas surrounding genetic risk identification and ethical issues regarding genetic testing reflected continued gaps in knowledge. Web-based genetics education may help overcome genetics knowledge deficits by providing access for health professionals with diverse schedules in a variety of national and international settings.

Introduction

Health professionals, particularly nurses, continue to struggle with the expanding role of genetics information in the care of their patients. Despite the completion of the sequencing of the human genome, it is clear that health professionals across disciplines remain confused about the role of genetics information in the care of their patients (Guttmacher & Collins, 2002). “Our increased understanding of the interactions between the entire genome and non-genomic factors that result in health and disease is paving the way for an era of genomic medicine, in which new diagnostic and therapeutic approaches to common multi-factorial conditions are emerging” (Feero et al., 2010, p. 2001). Physicians, as well as other health professionals, can be at the forefront of designing effective, efficient, and equitable systems that integrate genetics into the services they provide, but this requires a new way of thinking (Jenkins & Collins, 2003). Education of healthcare practitioners and the public has not kept pace with the accelerated changes in modern medicine resulting from the expanding role of molecular genetics. These educational deficiencies will make it imperative for healthcare providers and the public “ to do some work on their own to learn about the genes and genomes that will progressively change medical practice” (Varmus, 2002, p. 1527).

Supporting patients and their families regarding their genetic concerns can no longer only be the responsibility of genetic specialists (Barr & McConkey, 2007; Read et al., 2004). Scientific advances and their ethical, legal and social implications have contributed to expanded roles for nurses (Burton & Stewart, 2003). Nurses who have genetics expertise and possess competencies in genetics will be best prepared to meet these expanded roles (Nicol 2002; Tavernier, 2009). Nursing educators and clinicians face significant barriers that impede the adoption of genetic knowledge into nursing education programs and clinical practice (Spruill, Coleman, & Collins-McNeil, 2009). Chief among the obstacles is the misperception that genomics is not a priority for optimal clinical practice as well as the lack of qualified faculty to teach genomics (Hoop et al., 2010). Successfully establishing genetics nursing competencies requires that organizational and nursing leadership understand the relevance of genetics and genomics to the nursing community (Jenkins & Calzone, 2007). Competencies in genomics allow the provider to present basic genetic information, suggest appropriate genetic testing and give the provider the ability to know when to refer a patient and family to genetic specialists or counselors (Tavernier, 2009). A study of health visitors in the United Kingdom found that the majority of respondents supported the core competencies set forth by the Nursing and Midwifery Council but they had a limited view of their own role in providing support for parents of children undergoing genetic testing and felt they needed more education in relation to genetics (Barr & McConkey, 2007).

A landmark consensus document delineating genetics core competencies of knowledge, skill, and attitudes of all health professionals was published by a working group from the National Coalition for Health Professional Education in Genetics (NCHPEG) [Core Competencies Working Group, 2001]. Through collaborations with national and international professional associations, NCPHEG has continued to create and update resources for the designation of discipline-specific genetics competencies (Calzone et al., 2002; Jenkins et al., 2001; Prows et al., 2005). The American Nurses Association (ANA) published Essential Nursing Competencies and Curricula Guidelines for Genetics and Genomics in 2006(Consensus Panel on Genetic/Genomic Nursing Competencies, 2006). This document can be utilized to prepare nurses to incorporate genetic and genomic knowledge into research, clinical practice and education (Jenkins & Calzone, 2007; Dodson & Lewallen, in press).

Web-based Healthcare Provider Education

New opportunities for developing well-designed, engaging and learner-centered activities exist with advances in science and technology and with the development of new paradigms for training (Khan, 2001). These accelerated changes and the need for distributed learning environments is particularly critical in healthcare settings. With increases in instructional technology both nationally and internationally, and with the accompanying need to provide diverse opportunities for nursing students and nurses in clinical practice, web-based educational programs are becoming viable options for self-paced, flexible and affordable learning (Horiuchi et al., 2009; Tung & Chang, 2008).

The web has been identified as a promising medium for providing genetics education to audiences with differing, yet potentially overlapping needs (Guttmacher, 2001). However, few studies have examined the efficacy of basic genetics courses, particularly web-based educational programs for healthcare professionals. A multidisciplinary group from the National Institutes of Health (NIH) created “Basic Genetics Education for Healthcare Providers.” This program combined 7 web-based self-education modules (Table 1) with monthly traditional face-to-face lectures by genetics experts. The curriculum was based on the NCHPEG knowledge, skill and attitude competencies recommended for health professionals. In addition to the core competencies, a module on pharmacogenetics was included because the topic is applicable to clinical research practice at the NIH. The self-paced learning modules were designed at the novice level to review basic and preparatory information regarding the topic. Each module contained “test your understanding” questions for learners’ self-assessment of the content. The one-hour lectures presented additional detail to expand the learner’s knowledge base as well as application of the topic to both clinical practice and research initiatives in a variety of clinical specialties.

Table 1.

Basic Genetics for Healthcare Providers Modules

Module	Content
1. Genetics Basics	Nomenclature (RNA, DNA, protein) Genetics and genomics Human variation
2. From Disease to Genes	Breast Cancer, Tay-Sachs, Cystic Fibrosis Mapping of the Human Genome Genotype
3. Learning From the Family	Family medical history Constructing pedigrees Genetic Testing
4. Ethical and Social Challenges of Human Genome Research	Goals of the human genome project Mapping and sequencing National Human Genome Research Institute Laws, policies and ethics Genetic discrimination
5. Clinomics	Integration of genetics into clinical care Genomic healthcare Nursing roles and opportunities
6. Pharmacogenomics 101	Terminology Biology basics Drug disposition Metabolic capacity Viral genomics
7. Putting It All Together	Presentation of Case Study Patient and family education Developing a pedigree Genetic testing for recessive inheritance

On the first of every month the registered learner was given access to web-based instructional module. The modules were interactive with suggested exercises for each module. The learner had 24/7 access to a “majordomo” and electronic access to other students registered for the course. The majordomo was available for support from understanding the material to helping with any technological concerns. On the fourth week of each month, the learner attended a face-to-face lecture provided by a well-known content expert in the field of genetics. Each lecture was also video-streamed and then included into the module’s content twenty-four hours post lecture. The following paper is a description of a pilot study to evaluate a hybrid web-based basic genetics course designed for healthcare providers.

Methodology

Research design and objectives

A prospective pretest posttest pilot study was conducted with a convenience sample of healthcare professionals at the National Institutes of Health, Clinical Center. The Clinical Center (CC) is a 234-bed research hospital in Bethesda, Maryland that supports the Intramural Research Program (IRP) of the National Institutes of Health. It opened in 1953 and remains the largest inpatient facility in the USA devoted exclusively to clinical research. The objectives of this study were to 1) determine learner outcomes including change in knowledge and self-efficacy; and 2) to explore learner perceptions of the effectiveness of a basic genetics course that combined web-based learning with traditional face-to-face instructional techniques.

Respondents

One hundred and eighty-six health care providers agreed to enroll in the pilot course. Sixty-nine percent of those enrolled in the BGEHPC pilot agreed to participate in the evaluation study. The resulting study sample (N=129) consisted of nurses involved in clinical research as well as a small number of other allied health professionals including dieticians, social workers and lab technicians (Table 2). The sample was predominately female (91%)), Caucasian (81%), and had 16 or greater years of job experience (64%). Twenty-three percent of the sample reported having no web-based training experience while 41% reported having no genetics related education.

Table 2.

Baseline Demographic Characteristics

	N	%
Sample
Participants	127	98
Age
20–30 years old	15	12
31–40 years old	26	20
41–50 years old	42	33
>50 years old	44	35
Gender
Men	11	9
Women	116	91
Education
Doctorate or MD	3	2
Master’s	54	43
Bachelor’s	50	39
Associate	7	5
Diploma	6	5
Other	4	3
Job Title
Clinical Research Nurse	66	52
Research Nurse Coordinator	14	11
Advanced Practice Nurse	25	20
Nurse Manager/Executive	7	6
Other Allied Health	14	11
Race/Ethnicity
White-Caucasian	103	81
Black African-American	7	5
Latino (a)- Hispanic	5	4
Asian-Pacific Islander	9	7
Other
Work Schedule
Full-time	113	89
Part-time	13	10
Years of Job Experience
Less than 1 year	11	9
1–5 years	7	6
6–10 years	15	12
11–15 years	10	8
16–20 years	21	17
>20 years	60	47
Web-Based Experience
None	29	23
Mandatory reviews	40	31
College course	8	6
Continuing education	13	10
Other	4	3
Mandatory reviews & college courses	3	2
Mandatory reviews & continuing education	21	17
College courses and continuing education	5	4
Genetics Education Experience
None	52	41
Inservices-lectures	28	22
Journal-book reading	7	6
Formal course work	11	9
Other or combination	25	20

The sample size varies as a function of missing data.

Questionnaire development

Pre and post course assessments consisted of two parts. For the pretest, Part I included ten questions that addressed the learner’s perception of their ability and comfort to learn basic genetics using web-based format. Part II of both the pretest and posttest included ten problem-based questions utilizing basic genetic concepts. The post assessment consisted of two parts with Part I including a modified version of the Web-based Learning Environment Inventory, (WEBLEI). The modified WEBLEI instrument consisted of four subscales (access, interaction, response and results) utilizing a Likert scale with five response options of 1 (Almost Never), 2 (Seldom), 3 (Sometimes), 4 (Often), and 5 (Almost Always). A mean score ranging from 0 to 5 was then calculated for each scale. Three scales (emancipatory, co-participatory, and qualia) are built upon the work of (Tobin, 1998). The emancipatory subscale focuses on convenience, efficiency and autonomy for the learner’s emancipatory activities. The co-participatory subscale focuses on six categories to encourage learning in a virtual community, flexibility, reflection, quality, interaction, feedback and collaboration. Tobin (1998) described the six components of the qualia subscale as enjoyment, confidence, accomplishments, success, frustration and tedium. The fourth subscale added by Chang and Fisher (2001) described as “results” focuses on information structure and the design of on-line material. Chang and Fisher also renamed the other three original subscales as the access, interaction, and response subscales. The final version of the original WEBLEI included 32 items with eight items per scale. Statistical analyses, Cronbach’s alpha reliability coefficient, factor analysis, and discriminant validity, indicate that the WEBLEI is a reliable and valid instrument (Chang & Fisher, 2001; Chang & Fisher, 2003). The modifications of the WEBLEI were in order to make the inventory that was designed and used predominately in higher education settings to be more relevant to the current study’s healthcare environment. Table 3 describes the specific items used from each WEBLEI subscale for this study. The modified version of the WEBLEI used for this study included a total of 22 items, with 6 items in the access scale; 5 items in the interaction scale; 5 items in the response scale; and 6 items in the results scale. Additionally, since this basic genetics course did not include online interaction among students but did include a tutor known as the “majordomo,” item 13 on the scale was modified to read “the majordomo responds promptly to my queries”. The final modified version used included Internal consistency of the modified version of the WEBLEI subscales for this study was good with Cronbach’s alphas ranging from 0.73 to 0.92 (Table 3).

Table 3.

Cronbach’s Alpha for each Modified Web-based Learning Environment Inventory (WEBLEI) Subscale

Modified WEBLEI Subscales	Items Included	Cronbach’s Alpha	N
Access (Emancipatory)	1, 2, 4, 5, 6, 7	0.87	67
Interaction (Co-participatory)	9, 10, 11, 12, 13	0.73	52
Response (Qualia)	18, 19, 20, 23, 24	0.92	61
Results (Information Structure & Design)	25, 26, 27, 28, 29, 31	0.92	65

Ethics Approval

Once permission was granted through the NIH Clinical Center Office of Human Subjects Research, volunteers were solicited through email and flyers inviting staff to register for a new pilot course in basic genetics. Information describing the course format and contents was sent to all Clinical Center nurses. In addition, each course applicant was invited to participate in the pilot evaluation study using cover letters on their registration packets. The paper-pencil pre-assessment was sent to each participant and upon completion of the pre-assessment, the participant will have access to the web-based learning. All study participants were informed that they could withdraw from the study at any time and their data would be withdrawn from the analysis. They were further informed that regardless of study participation they could continue to be enrolled in the basic genetics course. The pre and post assessments were assigned a unique number that was be linked to each participant. The code was kept secured by the Principal Investigator until the completion of the study. Once the assessments were entered into a database, the code linking names to numbers was destroyed. All data were analyzed in aggregate and there were no identifiers link to an individual person attending the course.

Data Analysis

Quantitative data were analyzed using the statistical package for the Social Sciences (SPSS) for Windows 15.0. Descriptive statistics (percentages, frequencies, means and standard deviations) and inferential statistics were used to describe the sample and evaluate changes in knowledge scores post completion of the basic genetics web-based education program. Pretest scores were normally distributed however the post-test scores did not meet the assumptions of normality so a nonparametric Wilcoxon signed rank test was conducted to evaluate mean differences in total scores pre and post the genetics education program. Statistical significance was set at a p value of ≤ .05 for this educational program evaluation.

Results

A total of 69 (53.5%) of the participants completed the post test assessment and of these 58 (45%) completed the knowledge questions and were included in the analysis. Frequencies and percentages for each of the individual knowledge items answered correctly pre and post the basic genetics education program are presented in Table 4. Percentages reflect improvement in all areas except risk identification where the percentage of participants (77.6%) answering correctly after the basic genetics course was lower than the percentage of participants (79.5%) answering correctly before the course. Pretest total scores resulted in a mean of score of 16.32(SD ± 7.04) while posttest total scores increased to a mean of 24.81 (SD ± 8.81). Results from the nonparametric Wilcoxon signed rank test indicated a statistically significant change in total knowledge scores for participants who completed the basic genetics web-based course, z = −4.57, p < 0.001.

Table 4.

Frequencies and Percentages for each Knowledge Item Answered Correctly Pre and Post the Hybrid Web-based Basic Genetics Course

	Knowledge Pretest	Knowledge Posttest
	N = 127	N=58
	n (%)	n (%)
Labeling a Pedigree	16 (12.6)	32 (56.1)
Pedigree Inheritance Pattern	60 (47.2)	44 (77.2)*
Genetic Testing	33 (26.0)	25 (43.9)*
ELSIa of Workplace Discrimination	12 (9.4)	14 (24.6)*
Genetics Terminology	17 (13.4)	25 (43.9)*
Inheritance Patterns	6 (4.9)	11 (22.9)**
Probability Calculation	80 (63.0)	46 (79.3)
Risk Identification	101 (79.5)	45 (77.6)
Accessing Genetic Resources	13 (10.2)	24 (41.4)
ELSIa of Genetic Testing in Minors	20 (15.7)	20 (34.5)

^*total N for posttest on this knowledge item was 57.

^**total N for posttest on this knowledge item was 48.

^aELSI (Ethical Legal and Social Issues)

WEBLEI scores indicated program effectiveness particularly in the area of convenience, access and the course structure and design. In this study WEBLEI mean results of 4.35, 4.13, 3.68, and 4.20 out of a possible 5.00 were obtained from the Access, Interaction, Response and Results subscales respectively (Table 5). Mean results on the Access scale (4.35) indicate that the healthcare providers found the web-based basic genetics course easily accessible and provided flexibility for meeting individual learning needs. An Interaction mean of 4.13 for this study confirmed that participants often or almost always perceived that they needed to be self-disciplined about their learning but also indicated that they could often or almost always contact the “majordomo” for things they did not understand. The mean score for the Response scale (3.68) was the lowest of the four scales which is a reflection of whether participants felt that they were able to access the learning activities at convenient times and whether they felt they could work at their own pace to achieve the basic genetics learning objectives. Mean scores for the Results scales were high (4.20) with participants agreed that the organization of the lessons were easy to follow, the structure kept them focused on what was to be learned and that the presentation of the content was clear.

Table 5.

Means and Standard Deviations for each Modified Web-based Learning Environment Inventory (WEBLEI) subscale

WEBLEI Subscales	M	±SD	N
Access (Emancipatory)	4.35	0.61	67
Interaction (Co-participatory)	4.13	0.69	52
Response (Qualia)	3.68	0.95	61
Results (Information Structure & Design)	4.20	0.77	65

The seven live lectures that accompanied the web-based modules received positive evaluations however they were not well attended with the number of evaluations completed between 9 to 55 participants. The most well attended lecture was the “Ethical Legal and Social Implications” (ELSI) of genetics lecture. One of the students at the ELSI lecture commented that “the speakers were open to questions and to discussion afterwards” while several others commented on the strength of using actual case studies as exemplars. Finally, one participant commented that the knowledge would be “helpful for possible patient issues and public issues”.

Limitations

The positive results of this study may be limited by the posttest response rate of 45% and the fact that the participants who chose to complete the knowledge questions may have also been limited to the ones that felt most confident that they had increased their knowledge regarding genetics by completing the online “Basic Genetics Education for Healthcare Providers” course. Secondly, the open-ended responses provided in written form by the participants who attended lectures were reviewed in aggregate which limited the authors ability to attribute quotations to specific participants. The generalizability of these results is also limited in that a large proportion of the sample included nurses caring for clinical trials participants in a research hospital.

Discussion

The need for healthcare providers, and particularly nurses, to understand the expanding role of genetics information in the care of their patients is essential. Although the genetic competencies developed for nurses are clear, how to achieve these competencies is much less prescriptive and will depend on the audience and their current level of basic genetics knowledge and experience. There may be several reasons why the learners did not score in the 85% range or higher on the knowledge post test. First, some learners are poor test takers. Second, some learners commented that they were not able to attend all the lectures or finish the modules due to limited amounts of time to participate in the learning activities and competing priorities of clinical area patient care needs. Finally, genetic information is complex and may require study time or repeated exposure to the content to achieve mastery.

Similar to other studies using web-based education as a means for training healthcare professionals, the merit of this computer-aided learning proved to be the convenience of study at any time and at any location, aspects surrounding the autonomy of self directed learning, and the opportunities for repeated study (Blake, 2010; Horiuchi et al., 2009; Kim, et al., 2001). A study of a new hybrid model for online nursing courses by Tung and Chang (2008) also supports the premise that compatibility, or the users values, experiences and needs, is the most important contributing factor affecting students intention to use online nursing courses.

The results of the WEBLEI subscales in this study were similar to a study conducted with Australian high school science and physics students (Chandra & Fisher, 2009). However, the mean Interaction score for the study described in this paper (4.13) is higher that the Chandra and Fisher mean Interaction score (3.58) which indicated that the students neither agreed nor disagreed that they could communicate with their teachers and other students. The addition of a “majordomo” available for class queries during the basic genetics course may have accounted for the higher mean Interaction scores in the current study. In addition, all course participants had the ability to contact students and presenters by e-mail. The participants also may have interacted with the presenters or fellow students in daily clinical practice, which may have provided opportunity for additional communication.

Equally important as the content of web-based multimedia course offering, is the manner with which it is ultimately disseminated to a broader audience. Ease of navigating the content, ability of the program to operate correctly using different browsers and computer configurations and responsiveness of technical support impact the learner’s ability to complete web based training successfully. The failure to deliver a multimedia education program effectively has the potential to be the single greatest cause of system failure (Boyle, 1997). Using the results from this pilot evaluation study, a revised version of the Basic Genetics Education for Healthcare Providers was developed and implemented through a collaboration with nursing experts from the NIH, Clinical Center, National Cancer Institute (NCI) and NHGRI. Delivering the program effectively is the crucial phase in disseminating this multimedia basic genetics course to a broader local, national, and international audience.

Conclusions

Although significant increases in overall genetic knowledge scores were achieved, scores in content areas surrounding genetic risk identification and ethical issues regarding genetic testing reflect continued gaps in knowledge thus providing guidance to course developers for improved content and/ or learning activities in these specific areas. Web-based genetics education such as the one described in this evaluation study may help overcome genetics knowledge deficits by providing access to health professionals with diverse schedules in a variety of national and international settings. Adding opportunities for live lectures and classroom discussion to the web-based genetics modules appeared to strengthen the program further. These monthly lectures may have served to overcome a limitation set forth by previous studies that described the potential of web-based learning being a “relatively lonely process” (Horiuchi et al. 2009, p. 148).

“Although the effect of genomic discovery on the day-to-day practice of medicine has not been well quantified, it probably remains small in primary care and nonacademic settings as compared with, for example, oncology practice in an academic medical center. Regardless of where medicine is practiced, genomics is inexorably changing our understanding of the biology of nearly all medical conditions” (Feero et al., 2010, p.2002). As frontline clinicians, nurses will play a growing role in the delivery of genomic information to patients and their families. Nurses must possess at least basic genetic and genomic competencies to deliver this often complex information in a way that can be understood by their patients.