Field Test Results of Sex- and Gender-Specific Health Multimedia Case-Based Learning Modules

Abstract

Background: The sex- and gender-specific health (SGSH) multimedia case-based learning modules (MCBLMs) were developed to address the absence of validated or peer-reviewed material that incorporates topics of sex and gender differences into medical curricula. This article provides the methodology for development of the modules and reports the results of a field test of the modules in different medical educational settings.

Methods: MCBLMs were created by a multidisciplinary committee of scientists, health profession educators, and students. Two modules, osteoporosis and diabetes, were tested in various settings based on the curricular needs at each of the five accredited institutions. Each module consisted of a pretest and three interactive, multimedia stand-alone sections with post-tests. Scores on the tests were compared using a paired-samples t-test. A postmodule survey was used to evaluate the format.

Results: Four hundred eighteen students participated in the field testing. For the 194 who completed the osteoporosis module, the post-test scores (M = 13.71, standard deviation [SD] = 2.09) were significantly higher than the pretest scores (M = 10.54, SD = 2.41), p < 0.001. Post-test scores for the 285 who completed the diabetes module (M = 16.55, SD = 2.46) were also significantly higher than the pretest scores (M = 13.71, SD = 2.09), p < 0.001. The postmodule survey showed positive acceptance of the format with an average score of 3.54/4 for osteoporosis and 3.45/4 for diabetes.

Conclusion: The SGSH MCBLM field testing results show that the modules have a positive effect on content knowledge in multiple settings and are well accepted by learners.

Introduction

Every patient has a sex and a gender. Sex refers to biological differences, the chromosomes, hormonal profiles, and internal and external sex organs.¹ Gender refers to psychosocial factors of how a person is influenced by a culture and environment (gender norms), how the person relates to the cultural environment (gender identity), and how the person relates to others (gendered relationships).² There is compelling evidence that sex and gender have a profound impact on patients' health and medical care.³ These differences between men and women in health and disease extend beyond the reproductive system to incidence, diagnosis, treatment, and outcomes of multiple diseases.

Deliberate educational attention to curricula for future health care professionals that provide data on how sex and gender influence health and disease is essential to fundamentally improve many of the present disparities in both health care research and patient care,⁴ yet, in a national survey of medical students, only 31.1% identified sex- and gender-based topics in their medical school curricula.⁵ While there are some sources available for sex and gender content, including the National Institute of Health, the Office of Research on Women's Health, and continuing professional education from multiple sources, these are not specifically designed for medical students who may have little or no preexisting expertise in this area. Furthermore, health care educational programs and institutions have not found ways to incorporate these findings into their curricula due to lack of easily accessible and validated resources.⁶

To address the absence of validated or peer-reviewed material that incorporates topics of sex and gender differences into medical curricula, the Laura W. Bush Institute for Women's Health at the Texas Tech University Health Sciences Center (TTUHSC) developed the sex- and gender-specific health (SGSH) multimedia case-based learning modules (MCBLMs). These modules have the potential to provide a transformational change in health profession education because many health professionals and health profession faculty do not presently realize the significance of sex and gender differences for their patients.^5,7,8 This article provides the methodology for development of two of the modules, osteoporosis and diabetes, and reports the results of their field testing in different medical educational settings at accredited medical school programs in the United States.

Methods

Module creation

The MCBLMs were created by a committee selected from a multidisciplinary triad of scientists, health profession educators, and students based on their expertise on the topic (“Texas Tech Triad”). This triad was supplemented with instructional design faculty from the Texas Tech University School of Education to ensure that adult learning theories and design were included throughout development, assessment, and testing.⁹ The instructional framework was designed to avoid the major obstacles encountered in traditional settings: overemphasis on lecture-based classroom learning; separation between clinical and classroom instruction; and the opportunistic nature of medical education that is dependent on the medical cases that happen to be available at any given time. To address these challenges, the instructional design team developed modules based on the cognitive-affective theory of learning with media,^10,11 an evidence-based, pedagogical framework to “enhance problem-solving and decision-making abilities in dynamic, real-world settings.”^9,p.39 This conceptual framework builds on well-established multimedia learning theories (e.g., Mayer's cognitive theory of multimedia learning¹²) by expanding the traditional cognitive perspective to include affective and motivational aspects of learning.

After researching and creating the content, selecting the educational videos, and writing the scripts for the virtual patients and caregivers, the triad forwarded the content to the instructional design team for editing and copyright checks. Once the module content was complete, it was sent to national content experts for peer review. Final edits were made based on the peer review feedback, after which the instructional design team created and published the modules to the SGSH website (www.sexandgenderhealth.org), providing global access to the educational materials.

Module content

Each module consists of three stand-alone sections (Parts 1, 2, and 3) that follow a patient with osteoporosis or diabetes over time. In some cases, a male and a female patient are followed simultaneously to highlight the sex differences. Each part is designed to take ∼20 minutes to complete. The instructor may assign only one part or the entire three-part module for completion based on curricular needs. Figure 1 provides a screenshot to demonstrate the organization of the content of the osteoporosis module.

FIG. 1.

Screenshot of osteoporosis module illustrating the organization of Part 1.

Each module starts with a 20-item pretest to assess baseline content knowledge. Each part then begins with an introduction to the case followed by a video trailer of a patient consultation in a physician's office. The video presentations bring the learner up to date on the patient's chief complaint and relevant medical history and illustrate important issues in dealing with sex- and gender-specific perceptions of disease. Learning objectives, targeted to teach higher order thinking, are then clearly stated. The expert section follows, delivering didactic information and exploring the disease state from the point of view of a variety of professionals, including physicians, research scientists, pharmacists, and other interprofessional team members. Additional patient/provider videos illustrate the sex-based physiology underlying the disease and explore recommendations for treatment based on the biological sex of the patient. Each part concludes with a case solution and summary. Finally, there is a 20-item post-test to reassess the student's knowledge of the content focusing on significant sex and gender differences. The pre- and post-tests are designed to measure the impact on Kirkpatrick Level 2b: modification of knowledge/skills.¹³ At the completion of the three-part module, a nine-item survey using a four-point Likert scale (Strongly Disagree—Disagree—Agree—Strongly Agree) was used to evaluate the learners' acceptance of the module format as a learning tool.

Selection of site and settings

Ten institutions were invited to participate in field testing the modules, either through the Sex and Gender Health Collaborative or through their engagement in the 2015 Sex and Gender in Medical Education Summit (http://sgbmeducationsummit.com). Five institutions (Alpert Medical School of Brown University, The Mayo Clinic College of Medicine, Texas A&M University, TTUHSC, and University of Utah School of Medicine) accepted the opportunity to field test the first two modules, Module 1: osteoporosis and Module 2: diabetes. Setting and class size were based on the individual curricular needs of each institution and included a semester-long elective, a 6-week transition block, a 5-week selective, a stand-alone assignment during a block, and a prework assignment in a classroom session. Institutional review board approval was obtained at each of the participating sites.

All students were required to complete at least one assigned module, including the following: the pretest; Parts 1, 2, and 3; and the post-test. All students except the Alpert Medical School of Brown University students also had the option to complete the nine-item postmodule survey to evaluate the learner's acceptance of the module format as a learning tool. The postmodule survey was no longer available when the Alpert Medical School of Brown University students completed the modules. All of the remaining 317 students had the option to complete the survey after at least one of the modules (those who took both modules could complete the survey after each module). Paired samples t-tests were used to evaluate the pre- and post-test data. A mixed-design analysis of variance was performed to examine the effects of sex on the test scores.

Results

Of the 418 students who participated in the field testing, 199 identified as men, 213 identified as women, 1 identified as a transgender man with his responses grouped with the men, and 5 students did not select a gender option. Module 1 was completed by 194 students and Module 2 was completed by 281 students.

The Module 1 post-test scores (M = 13.90, standard deviation [SD] = 2.50) were significantly higher than the pretest scores (M = 10.54, SD = 2.41), p < 0.001. The Module 2 post-test scores (M = 16.55, SD = 2.46) were also significantly higher than the pretest scores (M = 13.71, SD = 2.09), p < 0.001.

Analysis of performance based on location and educational setting is reported in Table 1. Module 1 post-test scores improved significantly in the semester-long elective, the 6-week transitional block, the 5-week selective, and the stand-alone assignment. Post-test scores for Module 1 did not show improvement in the setting of prework assignment for a classroom session with only three students completing the module. Module 2's post-test scores improved significantly in all of the learning settings.

Table 1.

Field Test Institutions and Incorporation of Modules

Institution	Setting	Type of learner	Total No. of students (men/women)	Students who took both modules (men/women)	Total students per Module 1 and 2	Pretest average (SD), Max.: 20	Post-test average (SD), Max.: 20	t-Test results (p value)
Alpert Medical School of Brown University	Semester-long elective	MS1, MS2, MS3, MS4	Total: 101; M: 31; W: 69; transgender-man: 1	M: 20; W: 36	Module 1: 68	10.29 (2.29)	13.91 (2.84)	<0.001
					Module 2: 56	13.16 (2.20)	16.45 (2.58)	<0.001
The Mayo Clinic Medical College	Six-week transitional block between years 2 and 3	MS2, MS3	Total: 44; M: 28; W: 16	M: 16; W: 7	Module 1: 36	11.33 (2.19)	13.67 (1.88)	<0.001
					Module 2: 27	13.37 (2.57)	16.22 (2.31)	<0.001
Texas A&M University	Five-week selective	MS2	Total: 10; M: 3; W: 7	M: 2; W: 4	Module 1: 6	8.33 (1.97)	12.67 (3.01)	0.012
					Module 2: 9	14.11 (1.17)	17.00 (2.12)	0.013
TTUHSC	Stand-alone assignment during block	MS2	Total: 156; M: 84; W: 68; no gender selected: 4	M: 9; W: 6	Module 1: 81	10.62 (2.49)	14.30 (2.12)	<0.001
					Module 2: 86	13.86 (2.14)	16.95 (2.42)	<0.001
University of Utah School of Medicine	Prework assignment for classroom session	MS2	Total: 107; M: 53; W: 53; no gender selected: 1	M: 6; W: 0	Module 1: 3a	9.00 (3.61)	8.00 (2.00)	0.423
					Module 2: 103	13.96 (1.87)	16.33 (2.49)	<0.001
Total		Total: 418; M: 199; W: 213; transgender-man: 1; no gender selected: 5			Module 1: 194	10.54 (2.41)	13.90 (2.50)	<0.001
					Module 2: 281	13.71 (2.09)	16.55 (2.46)	<0.001

^aOne student did not complete post-test in Module 1.

MS1, first-year medical student; MS2, second-year medical student; MS3, third-year medical student; MS4, fourth-year medical student; SD, standard deviation; TTUHSC, Texas Tech University Health Sciences Center.

Module 1 was completed by 126 students, with 123 students completing the postmodule survey to evaluate acceptability of the module as a learning tool (98%) with an average score of 3.54/4. Surveys of Module 2 were completed by 212 of 225 (94%) students with an average score of 3.45/4. The survey results showed positive results for each individual item as well (Table 2).

Table 2.

Nine-Item Postmodule Survey: Evaluation of Modules as Learning Tools

	Module 1 osteoporosis (n = 123 total completed surveys),atotal completed modules = 126	Module 2 diabetes (n = 212 total completed surveys),atotal completed modules = 225
TTUHSC	80 Surveys (81 modules)	83 Surveys (86 modules)
Mayo Clinic Rochester	35 (36)	24 (27)
University of Utah	3 (3)	102 (103)
Texas A&M	5 (6)	3 (9)
(1) The partial patient interviews or “movie trailers” at the beginning of the modules helped me to be engaged in the module content.	M = 3.27 (SD = 1.09)	3.26 (1.09)
(2) Interacting with the various characters in the modules (e.g., experts) helped me to be engaged in the module content.	3.45 (1.04)	3.30 (1.05)
(3) The practice questions with feedback in the expert sections helped me to learn the instructional material.	3.92 (0.91)	4.00 (0.80)
(4) Seeing the entire patient interviews at the conclusion of the modules helped me to integrate what I learned from the module.	3.45 (1.03)	3.43 (1.05)
(5) Inserting the test questions within the complete patient interviews at the conclusion of the modules was an effective way to assess my learning.	4.00 (0.75)	3.93 (0.74)
(6) Receiving feedback on my responses to the test questions directly from the health care expert in the complete patient interview was a good learning experience.	3.94 (0.82)	3.85 (0.83)
(7) I learned a lot from this module.	3.65 (1.03)	3.44 (1.00)
(8) I would like to take more online modules such as this in the future.	3.10 (1.24)	2.83 (1.19)
(9) I would rather learn the information presented in this module in this online format than from a traditional classroom environment.	3.06 (1.29)	3.00 (1.22)
Average	3.54/4	3.45/4

^aThe number of completed surveys is less than the number of total students because the survey was no longer open when Alpert Medical School of Brown University students completed the modules. All other students had the option to complete a survey after each module, and therefore, students who completed two modules may have submitted two surveys.

Ninety-eight men and 94 women participated in Module 1 pre- and post-tests; 137 men and 139 women participated in Module 2 pre- and post-tests. The scores on the pretests did not differ significantly between men and women for either module, and there was a significant improvement on the post-tests scores for both men and women. However, as seen in Table 3, the scores on the post-tests for both modules were significantly higher for women (M = 14.29, SD = 2.36) compared to men (M = 13.47, SD = 2.58), p = 0.023, for Module 1, and for women (M = 17.07, SD = 1.89) compared to men (M = 16.01, SD = 2.87), p < 0.001, for Module 2.

Table 3.

Test Scores by Gender

Module	Gender	Pretest	Post-test	Significance (p value)
Module 1a	Men: 98	10.64 (2.37)	13.47 (2.58)	<0.001
	Women: 94	10.44 (2.48)	14.29 (2.36)	<0.001
		p = 0.577	p = 0.023
Module 2b	Men: 137	13.59 (2.30)	16.01 (2.87)	<0.001
	Women: 139	13.84 (1.89)	17.07 (1.89)	<0.001
		p = 0.323	p < 0.001

^aNumber differs from Table 1—one student did not select gender option; one student identified as a transgender-man.

^bNumber differs from Table 1—five students did not select gender option.

Discussion

The results of this field test of the SGSH MCBLMs suggest that both modules improved content knowledge in all settings except Module 1 as the prework assignment for classroom session. The results from these classroom sessions most likely reflect the small number of students completing the modules in that setting. The utility of the module in preclass assignment settings needs further investigation. Use of the modules improved post-test scores for both men and women. Interestingly, women performed significantly better than men on the post-tests. It is not clear whether this reflects a personal as well as a patient interest in health care information or whether women are more motivated to learn about sex and gender differences. In addition, the modules were well accepted as a learning tool. Overall, field testing of the SGSH modules was successful in demonstrating that they can be implemented in a variety of settings with measurable outcomes and without major changes in the existing curriculum.

Strengths

The SGSH modules have several strengths. First and most importantly, the use of the cognitive-affective theory of learning with media assures that the modules focus on enhancing problem-solving and decision-making abilities rather than simple memorization. Second, the fact that the modules were field tested in a variety of settings underlines their adaptability. Finally, the built-in assessments allow for readily measurable outcomes. The data show that both of the online sex and gender modules significantly improved medical knowledge pertaining to sex and gender differences in osteoporosis and diabetes in a variety of learning environments. Furthermore, the evaluation tool demonstrated that the format is well received by the learner, including the “movie trailer,” the interactions with the characters, the practice questions with feedback, and the general format of the modules.

Limitations

The improvement in medical knowledge, although a welcome outcome, can only demonstrate an impact on Kirkpatrick Level 2b: modification of knowledge/skills. Subsequent impact on Level 3: behavioral change is beyond the scope of an online module and is dependent on the opportunities for application of the new concepts in the real world. Another limitation was that the students were not stratified by gender beyond binary (man, woman) and transgender categories. This was discussed at length by the authors, but at the time the study was performed there was no universally accepted gender designation. In addition, it was felt that the small numbers in each gender category would weaken the statistical analysis. The main limitation of this study is the use of multiple sites with varying methods of integrating the modules into their curricula. However, this limitation may be viewed as a strength, since the modules were designed to offer teaching institutions the opportunity to use them with minimal disruption of the existing curriculum.

Conclusion

The development of the SGSH modules demonstrates an efficient and effective way to integrate new sex and gender curricular content across unique educational environments. Incorporating new concepts into medical education can be a daunting task. Topics such as sex and gender that cover broad areas are especially difficult without significant adjustment of existing curricula. Many schools attempt to develop proprietary stand-alone courses, but the time and expense involved in development make it difficult to achieve and, especially, to maintain. Furthermore, most of these courses do not involve field testing.

The model presented here included the development of adaptable, web-based, peer-reviewed educational resources utilizing foundational adult learning theories. This model is one that could be applied to other topics within health profession education. In fact, additional modules have been created covering cardiovascular disease, alcohol use disorder, and infectious disease. In this digital age with digitally adept learners, the opportunities to create and share quality educational resources are vast. Institutions must move past the era of creating costly stand-alone curricula. Countless resources have been expended on institutionally centered curricula that subsequently become outdated as funding and human resources become limited. Web-based, high-quality, adaptable, and open-sourced educational resources could change the landscape of integrating sex and gender content into health profession curricula. These modules, developed by interprofessional teams, are also suitable for nursing, pharmacy, and other health profession students. In addition, they may serve an important role in graduate education as well as continuing professional education since those who are presently in training, teaching, or practicing may not be familiar with this content.

Author Disclosure Statement

No competing financial interests exist.