Teaching Health Literacy Using Popular Television Programming: A Qualitative Pilot Study

Abstract

Background

Teaching of health and medical concepts in the K-12 curriculum may help improve health literacy.

Purpose

The purpose of this project was to determine acceptability and preliminary efficacy of pilot implementation of a health literacy curriculum using brief clips from a popular television program.

Methods

Participants included 55 ninth-grade students in a low-income school with a high proportion of minority students. The curriculum used three brief interspersed segments from the television show ER to teach basic topics in cardiology. After the 30-minute experimental curriculum, students completed open-ended surveys which were coded qualitatively.

Result

The most common codes described “enjoyment” (N=28), “acquisition of new knowledge” (N=28), “informative” (N=15), “interesting” (N=12), and “TV/video” (N=10). We found on average 2.9 examples of medical content per participant. Of the 26 spontaneously-generated verifiable statements, 24 (92.3%) were judged as accurate by two independent coders (κ=0.70, P=.0002).

Discussion

Use of brief segments of video material contributed to the acceptability of health education curricula without detracting from students’ acquisition of accurate information.

Translation to Health Education Practice

Health education practitioners may wish to include brief clips from popular programming to motivate students and provide context for health-related lessons.

BACKGROUND

Health literacy—the degree to which individuals obtain, process and understand basic health information and services needed to make appropriate health decisions¹—is now recognized as a critical determinant of health care outcomes and health care costs.² In the United States, nearly 90 million people are considered to have limited health literacy,² and their increased rates of hospitalization and emergency services utilization^3-5 may lead to as much as $69 billion in avoidable health care costs each year.⁶ In order to help reduce poor health literacy in the U.S., the Institute of Medicine recommends increased teaching of health and medical concepts in the K-12 curriculum.² However, the best method of achieving this integration is unclear.

Currently, educators in a variety of classroom settings use television-based education to produce stimulating, compelling programming that is motivating to students.^7-9 These techniques have been used since early in the development of the field of health education.¹⁰ Youth have even been shown to learn health-related information from popular television in the community setting.^{11, 12} Collins et al. surveyed a nationally representative sample of U.S. adolescents regarding a particular episode of the popular TV show Friends that included a depiction of condom failure. They found that 20% of episode viewers had talked with an adult about the episode and that 10% had talked with a parent or other adult about condom effectiveness because of this single episode.¹¹ In the Ivory Coast, 65% of a random sample of 2150 adolescents and adults had viewed at least one episode of the soap opera SIDA dans la Cite about HIV disease and viewers who had seen 10 or more episodes were significantly more likely to have used condoms at last intercourse, even after controlling for multiple covariates.¹²

Although there are potential benefits to utilization of television in classroom materials, it is also well known to educators that their exclusive use can lead to other undesirable effects, such as reduced interaction, disjunction between the television programming and the educational experience, and undue distraction from the educational material.^10,13,14 One creative solution to retain the benefits and minimize the undesirable effects may be to present interactive lessons based on a brief (e.g., two-minute) segment of dramatic television. These lessons would use the televised segment as a case-based springboard¹⁴—showing only about 30 seconds of the program at a time—while interspersing rich content-based, interactive discussions related to specific health literacy objectives. Thus, of the complete lesson, only a small fraction would involve direct screen time. Although in theory this may be a creative and valuable combination of educational techniques, it is not clear if such a lesson would achieve its goals of being both acceptable to students and educational for them.

PURPOSE

The purpose of this project was to determine acceptability and preliminary efficacy of pilot implementation of a brief health literacy curriculum using brief clips of material from a popular television program. We hypothesized that such a program would be feasible to implement, compelling to students and able to provide valuable educational information related to health.

METHODS

Design, Setting, and Participants

We designed and implemented a brief health literacy curriculum in a public high school. We selected a qualitative evaluation for this pilot implementation project because we were interested in students’ perceptions—in their own words—of how acceptable they found the programming and what they felt they learned. Additionally, this was the most appropriate study design because of the innovative nature of the curriculum.

We selected a public high school in a low-income area of Pittsburgh, PA, with a high proportion of minority students: 53% of students receive free or reduced school lunch and 35% are African American. We selected a school with these demographics because low income and minority individuals have been identified as being at risk for poor health literacy.²

Our participants were a convenience sample of 55 diverse male and female ninth-grade students at this school. This represented all incoming students who opted to participate in an optional ninth-grade orientation, or about 20% of all 276 ninth-grade students enrolled at the school. The orientation consisted of eight rotations, each of which lasted 30 minutes. These rotations focused on various aspects of orientation to the school, such as meeting school personnel and becoming familiar with extra-curricular activities. The curriculum described in this manuscript was presented as one of the rotations. All of the 55 students who were present for this orientation participated in this project.

Curriculum

A complete outline of the 30-minute curriculum is available from the authors. In brief, the lesson was titled “Have a Heart” and focused on basic topics in cardiology, including chest pain, heart attacks and congestive heart failure. The objectives of the lessons were that, by the end of the lesson, we expected students to be able to: (1) List different conditions that can cause chest pain, (2) Describe what causes a heart attack, and (3) Define congestive heart failure. For our popular television text we selected the first two minutes of Episode 20 of season 7 of ER, titled “Fear of Commitment.” Season 7 was selected since it was the most recent programming available at the time of implementation. This particular segment was chosen because: (1) it was at the beginning of an episode, so no back-story was necessary, and (2) it involved an undiagnosed case of chest pain, easily opening the door for development of differential diagnosis.

We divided the selected two-minute text into three segments. The curriculum began with segment 1 in which an elderly man complained of chest pain. This was followed by a seven-minute interactive and didactic discussion of causes of chest pain. In segment 2, a physician in the program brought up the possibility that the man is having a heart attack. This transitioned into a second seven-minute discussion of the causes of a heart attack. Finally, in segment 3 it was determined that the man has congestive heart failure. This led to a third seven-minute discussion of the definition and causes of congestive heart failure and then a five-minute summary of all covered material. The curriculum was conducted by the study principal investigator B.P. and a third year medical student B.S.

Procedures

We obtained approval from the school administration and the Institutional Review Board of the University of Pittsburgh for the project. During the orientation, the 55 students rotated through different classrooms in four groups of 10-20 students each. Our curriculum was offered in one of these classrooms, and all students rotated through all classrooms. After experiencing the 30-minute experimental curriculum, each student completed a self-administered anonymous survey. They received granola bars as thanks for their efforts. Students did not disclose their identities on their surveys and were not required to respond for credit of any type. We used these methods to maximize validity.

On these surveys students responded to demographic items, one closed-ended item and two open-ended questions assessing the curriculum. Demographic data included gender, race (White, Black, or Other) and age. The closed-ended item asked “Did you enjoy this lesson today?” The first open-ended item was “Why or why not?” referring to the closed-ended item regarding enjoyment. The second open-ended item was “What did you learn today?”

Coding Process and Analysis

We employed what Miller and Crabtree describe as an “editing” approach to coding student responses to the two open-ended survey questions.¹⁵ This involved using an iterative process where the study team examined the open-ended responses and then developed a coding scheme that best reflected the nature and complexity of the data. Our editing approach was guided by the two central questions that were given to the participants: “Did you enjoy this lesson today? Why or why not?” and “What did you learn today?” To capture the first question we chose to develop a codebook which recorded the themes related to receptiveness toward the lecture and the reasons why or why it was not well received. For the second question we sought to directly capture the knowledge participants recounted in their answers by using an “in vivo” coding scheme.

Acceptability and Overall Impressions: Thematic Analysis

For the first phase of coding, we developed a thematic analysis. Our investigative team developed a tentative list of codes based on the research questions, the participants’ responses, and the published literature. Authors S.Z. and D.W. conducted the majority of the qualitative analysis. They each read all responses, wrote memos on what they individually perceived to be emerging key themes and then discussed the emerging coding ideas together. Based on these initial memos and the discussion process, they developed a thematic codebook which condensed similar or redundant codes and which used a system of open codes designed to incorporate any new themes that may have emerged.

The first 40% of responses using this codebook were completed by S.Z. and D.W. together, with the remaining 60% being completed separately. The team used separate files in Atlas.ti 5.2, a qualitative software analysis and management program, to record all coding decisions.¹⁶ Once the qualitative coding was complete, the inter-rater reliability of the remaining 60% of responses that were coded independently was calculated using Cohen's Kappa scores. The kappa score for the thematic coding was 0.81 (P > 0.0001) which Landis and Koch¹⁷ describe as “near perfect” inter-rater reliability. All discrepancies between the two coders were then discussed and adjudicated among S.Z., D.W., and B.P., which allowed for there to be a final approved data set.

Preliminary Efficacy of Curriculum: In Vivo Coding and Physician Review

For the second phase of study coding, we sought to capture the depth of the participants’ knowledge from the lecture using a process known as “in vivo” coding. In vivo coding is a technique whereby codes emerge directly from the words written or uttered by study participants.¹⁸ For this phase of coding, S.Z. and D.W. together used the in vivo coding mechanism within Atlas.ti to record all medical terms/statement in the participants’ answers. This second phase also involved the use of different Atlas.ti files for the coders than those used for completing the thematic analysis. For the in vivo coding the verbatim words of a participants’ response were highlighted. Using the Atlas.ti “in vivo” function those exact words were automatically added to the codebook. For example, in one participant's response—“Chest pain can occur for many reasons”—the terms “Chest pain can occur for many reasons” were highlighted and transformed into a verbatim code in the Atlas.ti codebook.

To clarify how accurately the participants recalled the lecture's content, the coders isolated all statements that contained medical information that may or may not be correct in reality. These statements were given to the principal investigator and medical physician B.P. and to another physician with expertise in internal medicine K.K. Both were charged with independently determining whether each medical claim was accurate or inaccurate. Examples of statements viewed by the two medical providers as largely accurate included: “[I learned] that some people might have liquid in their lungs, and [might] think they are having a heart attack” and “Heartburn isn't really in your heart, but in your stomach and esophagus.” Examples of statements viewed by the providers as largely inaccurate included: “Showing how blood is not going to the heart like O₂ and A F [atrial fibrillation]” and “I learned that the definition of CPR was cardiopulmonary.” The two physician coders initially agreed on 24 of the 26 statements (Percent Agreement = 92.3%, Cohen's kappa = 0.70, P = 0.0002). After adjudication via discussion, the physicians agreed on all 26 determinations of accuracy.

RESULTS

Our sample consisted of all 55 students who were present at the orientation. Our participants’ mean age was 14.6. Of the sample 49% (N = 27) were female, 73% (N = 40) were White, and 27% (N = 15) were Black. Our final results included both thematic analysis and in vivo coding.

Acceptability and Overall Impressions: Thematic Analysis

Of the 55 participants, 50 (91%) stated that they did enjoy the experimental curriculum, two stated they did not enjoy it (3.6%), and three had a mixed impression (5.4%). The two individuals who did not enjoy it stated: “No, [I did not enjoy it] because I fell asleep” and “No, [I did not enjoy it] because I didn't really like learning about chest pain and heart attacks.” The three individuals with mixed responses are quoted verbatim in Table 1.

Table 1.

Open-ended, Verbatim Responses to Enjoyment Item

Coder Assessment	Quotation
Negative (N = 2 comments)	No because I fell asleep.
	No, because I didn't really like learning about chest pain and heart attacks.
Mixed (N = 3 comments)	I guess I liked it. It was good. Its good to know what could happen and what happens with the heart and lungs.
	I thought that it was a good lesson, but it was too hot in the room.
	Yes and no, because it seems that you could have heart failure and strokes.
Positive (N = 50 comments; all comments not included here)	Yes I enjoyed the lesson today. It was informative and used television to convey facts. I learned a lot of information concerning the heart and chest area.
	I did enjoy this lesson today because it just connected to me.
	Yes, I did enjoy this lesson. I enjoyed this because I learned a lot about the heart. I learned that chest pain can be from a heart attack or heart burn. It can be from many different things. I enjoyed it because I got to learn.
	Yes, I did. I learned new things and I enjoyed this lesson and the person that told it, he was polite.
	Yes because I learned about different kind of chest pains and what happens when you might have a heart attack, and I also learned some health slang. Most of all, I liked it because we got to watch a movie. I also liked the presentation.
	I really did enjoy the lesson today. I enjoyed it because I learned a whole lot. I also enjoyed it because it was a different lesson than we have had the rest of the week. It was cool to actually have a professional come talk to us.
	Yes, I liked it because it's interesting. I wanna be a nurse so I think it was a nice opportunity for us. This information could help me in the future. I also like it because I could understand what my pap-pap went through.
	Yes, I liked it because it helped me learn a lot about the heart and lungs. It taught me a lot of things I didn't know, and the teachers were nice and helpful.
	Yes, it was cool. All about heart attacks and what to do if some one has a heart attack.
	Yes, I actually learned stuff that I didn't understand before. It was very interesting.
	Yes, it was very interesting. It makes me want to get into a medical field, like a nurse or doctor. I learned a lot of things I never did before.
	Yes, because it taught us a lot about the body and what can happen to people. It also taught us a lot about different health problems that cause chest pain.
	Yes, I did enjoy this lesson today because it gave me a better understanding of the symptoms and consequences of a heart attack. I thought the showing of two scenes from the show ER was a great example of someone experiencing a heart attack.
	Yes, I did. There were a quite a few things I learned and I realized I knew stuff I didn't know I knew. I'm not like my sister, and want to be a doctor, but I think it is really interesting to know about the body and how it works and to know things that can go wrong in our bodies, and to be aware of it. I've never really thought of things like this before, and it's kind of cool to see how this stuff can work.

There were 20 thematic codes in the final codebook that were ultimately collapsed into 11 code groups based on discussion of redundancy and consensus of coders (Table 2). The most common codes described “enjoyment or fun” (N = 28) and “Acquisition of new knowledge (general or medical)” (N = 28). There were also many statements describing the “educational / informative” nature of the program (N = 15) and that it was “interesting” (N = 12).

Table 2.

Thematic Codes and Examples

Code or code group	N	Example(s)
Educational / informative	15	I found out a lot about many different problems that go on in the torso area of your body. ... it was detailed and educational. It was informative and used television to convey facts. Yes, it was direct and they explained things that were happening.
Enjoyed / fun	28	Yes, it was fun learning about how a person helps someone who is having a medical problem. Yes, it was cool. I enjoyed it because I learned a whole lot. I enjoyed learning about the things that were going wrong, like what they were called and what caused them. I really enjoyed this lesson today.
General preparedness	8	It was very informative in case this might happen to me or anyone around me. I enjoyed it because I learned a lot of important things that I might need to know later in life. I enjoyed this class because I like to learn about things that can happen so that I know what to do if it happens to someone.
Health career	8	When I grow up, my goal is to become a doctor, so every time I hear new things, I take it in, and that let's me know more and also to be better. I wanna be a nurse so I think it was a nice opportunity for us. I want to be a vet, and any kind of information I can get about anything medically related is great. Yes, I enjoyed the commentary because I wish to pursue a career in the medical field, and I learned a little about what I will probably deal with in the future.
Interactive	2	I also liked it because it was like solving a case. [I liked it because] it was interactive.
Interesting	12	It was interesting to learn about these things. I think it is really interesting to know about the body and how it works.
Movie / TV format	10	The lesson was enjoyable and having a TV show is a great way to explain things. Most of all, I liked it because we got to watch a movie. I thought the showing of two scenes from the show ER was a great example of someone experiencing a heart attack. I learned what a heart attack is. It is decreased oxygen. CHF = Congestive heart failure, and what you can learn by watching a simple tv program.
New knowledge / medical knowledge	28	I enjoyed this because I learned a lot about the heart. ... [this lesson] gave me a better understanding of the symptoms and consequences of a heart attack. I learned what CPR stands for, and what CHF stands for. And to have heart failure, something has to cause it, like a heart attack. I also learned what a heart attack is. I learned how and what can cause a heart attack, and what can cause asthma. Yes, because it taught us a lot about the body and what can happen to people. It also taught us a lot about different health problems that cause chest pain. I also learned some health slang.
Presenter (nice / polite / successful)	6	I really enjoy listening to successful people and learn about what they do. I enjoyed this lesson and the person that told it – he was polite. The teachers were nice and helpful. I did because the speaker helped keep me interested.
Relatives in health field	4	My dad is also a paramedic and the movie helped me see what he has to do sometimes. I also like it because I could understand what my pap-pap went through. Yes, because it showed me what my uncle used to do.
Shocking/scary	2	I was shocked by the lesson. It is also scary to learn how fast that stuff can happen.

Ten of the participants specifically noted that the use of the television episode ER enhanced their interest in the lecture (Table 2). Some participants appeared to gain pleasure primarily from watching the video segment of ER: “Most of all, I liked it because we got to watch a movie,” while for others the video segment appeared to exemplify the concepts the speaker was conveying: “The lesson was enjoyable and having a TV show is a great way to explain things.” No comments described disappointment at the brevity of the video segments. Two comments referred to the “interactive” case-based nature of the curriculum.

Many students felt that the curriculum was beneficial to them either because they have a specific interest in a health-related career (“health career,” N = 8) or merely because this may be important information in general (“general preparedness,” N = 8). Related to this, four students referred to the fact that the program helped them better understand a relative's experience as either a patient or a health professional (N = 4). Finally, six comments described the presenter(s) as nice, polite, or professional and two students conveyed a sense of “shock” or “scariness” to the episode.

Preliminary Efficacy of Curriculum and Physiciaan Review

The in vivo coding focused on capturing the medical content articulated by the participants in response to the question “What did you learn today?” Using Atlas.ti, this process resulted in 105 codes being used 162 times across all 55 cases. For our analysis, we operationally defined “medical content” as the use of a medical term, medical process and/or a description of a biological function. For the lower range of what was considered “medical content,” this segment was exemplary: “Yes, I learned some things about the heart and lungs.” For the upper range, this segment was exemplary: “I learned what causes chest pain, which can lead to a heart attack, the definitions for CHF, CPR and Rapid A-fib, and the heart circuit from the heart to the lungs.” Based on the in vivo coding, we found that on average 2.9 examples of medical content were given per participant, with a mode of 2. The spectrum ranged from having no examples of medical content (55% of the total sample) to 9 examples (1.8% of the total sample).

Twenty-six examples of medical content went beyond a mere statement that medical material was learned and included statements that could be verified as either true or false, such as: “I learned that you don't only have to have asthma to have a heart attack, and when you do have a heart attack, that there are many reasons why.” Each of these statements was assessed as either true or false by two physician coders working independently. Of these 26 statements, 24 (92.3%) ultimately were judged as true. Examples of accurate and non-accurate statements are included in Table 3.

Table 3.

Verifiable Knowledge-Based Statements

Physician Assessment	Quotation
Incorrect (N = 2 comments)	I learned that the definition of CPR was Cardiopulmonary.
	Showing how blood is not going to the heart like 02 and AF.
Correct (N = 24 comments; all comments not included here)	I learned that CHF is congestive heart failure.
	A heart attack is when oxygen is not getting to the heart and then you have chest pain and stuff.
	I also learned broken bones can cause chest pain.
	When you have a heart attack it means not enough oxygen to the heart.
	I learned that just because you have a pain in your chest, it doesn't mean you are having a heart attack.
	Heart burn isn't really in your heart, but in your stomach and esophagus.
	[Congestive heart failure] means the heart is not pumping enough blood, so it backs up to the lungs then the whole body and you can tell if the person has that because you can listen to the fluid in their lungs.
	We learned about the difference between CHF and a heart attack, like what you think what might be a heart attack could be CHF which is actually in your lungs.
	I also learned how CHF is caused and a lot more like about O3 is ozone.

DISCUSSION

This pilot study demonstrates that it is feasible to implement a curriculum interspersing relatively, brief television segments (approximately 30 seconds each) with substantial health education material. Furthermore, the experience was highly acceptable and even compelling to ninth-grade students for a variety of reasons that seem related to both the format and the content of the curriculum. Finally, the study found that students were able to reproduce accurate scientific and/or medical information they had gained during the experience, and that the vast majority of the verifiable statements they made were accurate.

These findings suggest that this may be a valuable format in which to present health educational material. In particular, the fact that video was utilized seemed to contribute to the acceptability of the curriculum, despite the fact that the total video time was less than two minutes. Ten students made statements to this effect, such as “I really like that movie, it was very interesting” and “Most of all, I liked it because we got to watch a movie.” However, no students were disappointed in the relative brevity of the video segments.

These findings also suggest that the use of video material did not detract from students’ acquisition of accurate information. Independent reviews by medical doctors demonstrated that students were able to give multiple examples of scientific material they had learned during the lesson and that the vast majority of the verifiable statements made were correct. Thus, it would seem there was not undue distraction from educational material, which can be a concern, related to use of video format.^{10, 13}

The results also offer insights into other elements of the curriculum that may have contributed to its success and may be emulated in a variety of other curricula. Although only two students directly praised the interactive “problem solving” approach related to a case study,^{13, 14} a number of other comments—especiaJly those related to practicality of the information—implied that this format was attractive. Medical education at the graduate level currently features increased engagement with interactive case studies.^19-21 The results suggest that health education at the secondary level similarly may benefit from increased utilization of case-based methods. Although these methods may be valuable for other grade levels as well, we should be sensitive to the different educational needs of those at various developmental stages.

It is also notable that students frequently related connections between themselves, their families, and the health sciences. Many students expressed a desire to pursue a health-related career, some told of parents who worked in health care, and others described afflicted family members. Still others appreciated the information even if they did not specifically desire to enter a health-related field. This information provides a prudent reminder that students’ connection with health care is a “double-edged sword”: while it may be valuable to capitalize on this familiarity and interest by presenting case-based health lessons, it is also important to realize that such cases may bring up topics of high sensitivily to particular individuals. For example, a student may experience emotional distress if he has a parent who previously smoked cigarettes and a case involves someone who dies related to prior cigarette use. Two students did, in fact, refer to the information as “shocking” or “scary” even without invoking the situations of particular friends or family.

This study was limited in that there may have been selection bias of participants. Although all 55 of those eligible participated, these students may not represent the full student body since they had self-selected to participate in an optional orientation. In particular, it is possible that these students were more engaged in education overall and that those who did not participate may have been less receptive to the intervention. A second limitation is that the qualitative analyses were conducted on written material, which was a logistical constraint of our design. Interviews or focus groups may have garnered richer, more spontaneous thoughts. Third, although we endeavored to objectively assess the accuracy of facts spontaneously generated by the students, it is a limitation that we did not rigorously assess what students actually learned using a validated instrument. Finally, our study was limited in that we only tested students at one point in time, without baseline or delayed follow-up assessment. It may be valuable for future investigations to follow students for recall of information later in high school.

TRANSLATION TO HEALTH EDUCATION PRACTICE

Despite its limitations. this study offers a template for development of innovative, compelling and valuable health education curricula that use brief segments of popular television programming as a “springboard” to rich educational material. Thus, health education practitioners may wish to utilize curricula that use brief dips from popular programming to motivate students and provide context for health-related lessons.

Although further research is necessary to confirm the efficacy and developmental appropriateness of these curricula, integrating techniques such as these into health-related lessons may help answer the Institute of Medicine's call to increase teaching of health and medical concepts in the K-12 curriculum to combat the epidemic of poor health literacy in the United States.