Measuring conceptual health knowledge in the context of oral health literacy: preliminary results

Abstract

Objectives

Health literacy encompasses several abilities including word recognition, reading comprehension, communication skills, and conceptual knowledge. To date, conceptual knowledge has not been included in oral health literacy research. This study assesses the validity and reliability of a new instrument and describes conceptual oral health knowledge among a sample of low-income adults.

Methods

One hundred Baltimore adults were administered the Rapid Estimate of Adult Literacy in Medicine (REALM), Short Test of Functional Health Literacy in Adults (Short-TOFHLA), and a new survey of conceptual oral health knowledge. Respondents were also asked about sociodemographics, dental health, and utilization.

Results

Psychometric analysis was used to identify a subset of oral health knowledge questions from the new survey instrument. The resulting Comprehensive Measure of Oral Health Knowledge (CMOHK) was categorized into three levels of knowledge (poor, fair, good). Nearly one-third of Baltimore adults exhibited the lowest level. CMOHK scores were significantly associated with age, education level, and word recognition (REALM). CMOHKs cores were not associated with reading comprehension (Short-TOFHLA) or dental care visits. Instrument reliability was good (Cronbach alpha = 0.74).

Conclusions

This preliminary study yielded a new measure of oral health conceptual knowledge, available for use in future oral health literacy studies. The author presents a conceptual framework of oral health literacy that separates health literacy into four unique components and places decision-making at the center. Future studies are needed to determine whether this framework is supported by empirical data and leads to improvements in oral health and reductions in health disparities.

Introduction

Health literacy is defined as “the degree to which individuals have the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions” (1). Individuals with limited health literacy are likely to encounter difficulties navigating the dental health-care system and managing their oral health (2). Limited health literacy is also likely to explain some oral health disparities (3).

National surveys reveal that limited health literacy is widespread. According to the 2003 National Adult Assessment of Literacy, about 22 percent of adults have only basic literacy skills, and as many as 14 percent of adults have below-basic abilities (4).Among those most likely to be affected are adults with low socioeconomic status, seniors, and those for whom English is a second language.

Researchers hypothesize that an individual’s health literacy is represented by a constellation of skills and abilities, including word recognition, reading comprehension, communication proficiency, and conceptual knowledge (5). To date, accepted measures of health literacy have focused on only word recognition, reading comprehension, and one facet of conceptual knowledge – the ability to interpret numbers (also referred to as numeracy). Measures of communication skills and nonnumeric conceptual knowledge have not been developed and their absence has restricted researchers’ ability to advance effective interventions.

The purpose of this study was to create a conceptual oral health knowledge instrument for use in oral health literacy research. Study findings are discussed in the context of a conceptual framework that lists word recognition, reading comprehension, conceptual knowledge, and communication skills as independent, yet related health literacy components. The framework also introduces a theoretical pathway linking health literacy with oral health decision-making and outcomes.

Methods

Data for this study were collected as part of the Baltimore Health Literacy and Oral Health Knowledge Project (BHLOHK Project), a cross-sectional investigation conducted during 2007 and 2008. The purpose of the BHLOHK Project was to develop and validate a survey instrument of oral health conceptual knowledge for use in health literacy research and to describe conceptual knowledge in a sample of low-income Baltimore adults.

Survey development

The process of developing the conceptual knowledge instrument began with the consideration of topic areas that were directly related to oral health decision-making and outcomes. After several rounds of consensus building, the research team decided on four main topic areas: a) basic knowledge of oral health; b) dental caries prevention and management; c) periodontal disease prevention and management; and d) oral cancer prevention and management. Once these four areas were identified, a list of candidate questions was generated by the team for each topic. Some of the candidate questions were adapted from existing surveys (6), however, most were developed anew. Each of the candidate questions (41 in total) were written in an open-ended format for use during pilot-testing.

Pilot testing

A convenience sample of 16 adults from Baltimore was selected to participate in the pilot testing sessions. Research staff from the Schaefer Center for Public Policy administered the sessions at a survey research laboratory on the University of Baltimore campus. Each of the 41 open-ended questions was read aloud to the participants and their answers were transcribed on paper forms. During the pilot-testing sessions, participants were also asked to complete the Rapid Estimate of Adult Literacy in Medicine (REALM) (7) and a shortened version of the Test of Functional Health Literacy in Adults (Short-TOFHLA) (8, 9), as well as a short survey of sociodemographics, dental health history, and dental visit behaviors. The REALM is a word recognition test consisting of medical terms with varying complexity. The Short-TOFHLA consists of a brief test of medical text comprehension.

At the completion of the pilot testing session, participants were brought together in groups of four for debriefing. Each group was asked to comment on the informed consent process, question formatting, and survey administration. Responses to the open-ended questions and comments from the debriefing conferences were used to develop the final version of the conceptual oral health knowledge survey.

Final survey development and administration

The pilot testing sessions revealed that some questions were redundant and also showed that specific topics were not adequately covered. After another round of consensus building to address these revelations, the research team increased the number of questions on the survey from 41 to 44 – two questions relating to the prevention of dental caries and periodontal disease were removed and five basic knowledge questions were added (i.e., one question regarding knowledge of dental implants and four questions regarding dental terms and anatomy). They also used answers to the open-ended pilot-study questions to generate multiple-choice response options for each of the questions. The resulting final version of the survey contained 20 basic oral health knowledge items, and eight items, each, regarding the prevention and management of dental caries, periodontal disease, and oral cancer, respectively. As a final check of face and content validity, a group of 15 practicing dentists in Maryland were asked to comment on the multiple-choice items. Their input was used for final editing. No additional pilot-testing was conducted.

Funding allowed a total of 100 adults during the final data collection stage. Researchers from the Schaefer Center for Public Policy selected participants at random from a list of Baltimore residents who had documented landline telephones. Telephone numbers were matched against mailing addresses to maximize the number of residences in the sample. In order to facilitate the objectives of the research project, respondents were drawn mainly from areas in Baltimore where the US Bureau of the Census indicated lower levels of educational achievement in comparison with the general population of the city. A total of 231 adults were contacted in order to reach the target sample size of 100, representing a 43.3 percent response rate. Interviewed participants received a $25 payment and a packet containing a toothbrush, dental floss, toothpaste, and a selection of oral health-related brochures. Participants were also provided information about available safety-net dental clinics in the Baltimore area in case they needed care.

The 44-item oral health knowledge survey was administered to participants via face-to-face interviews in small conference rooms at the University of Baltimore. Eight trained interviewers were used for data collection. Instructions and other related information were scripted to minimize variation across interviews. The oral health knowledge survey was printed in large font and placed in a bi-fold binder so that the respondent could see the questions and response categories while the interviewers read the questions aloud. Before interviews began, respondents were reminded that if they were not sure of an answer, or if they did not know the correct response to a question, that it was acceptable to answer,“I don’t know.”

The REALM and Short-TOFHLA were also administered during the final data collection sessions. In addition, interviewers asked questions about age (recoded as: 18–44 years, 45–64 years, ≥65 years), sex, race, education level (recoded as: <12 years, 12 years, >12 years), and household income (recoded as: unknown, $0–$25,000, ≥$25,001). Participants were also asked whether a physician or dentist had ever told them that they had tooth decay, periodontal disease, or mouth cancer. They were also asked about the timing of their last dental visit.

Data management and analysis

Responses were entered into a Microsoft Excel spreadsheet and imported into the SAS for Windows statistical software program (Version 9.1) (10). Data from the oral health knowledge survey, REALM, Short-TOFHLA, and sociodemographic and dental history/dental visit surveys were linked via unique code numbers. The final analytical data set contained no personal identifiers.

Data collection and management processes were approved by institutional review boards at the University of Maryland, Baltimore and the University of Baltimore. The study was funded by the National Institute of Dental and Craniofacial Research, National Institutes of Health (1 R03 DE016934).

Results

The majority of adults in the sample were African American women, aged 45–64 years, with an education level of 12 years, and an annual household income between $0 and $25,000 (Table 1). Given the small number of respondents who were not African American, race-specific statistical tests were excluded from subsequent descriptive analyses.

Table 1.

Sample Characteristics, Baltimore, Maryland, 2008 (n = 100)

Characteristics	n	Percentage
All	100	100.0
Age (years)*
18–44	30	30.3
45–64	43	43.4
≥65	26	26.3
Sex
Male	45	45.0
Female	55	55.0
Race*
African American	92	93.9
Other	6	6.1
Education level*
<12 years	20	20.2
12 years	50	50.5
>12 years	29	29.3
Household income
Unknown	15	15.0
$0–$25,000	51	51.0
≥$25,001	34	34.0

^*Total does not sum to 100 because of missing values.

Table 2 shows the distribution of REALM and Short- TOFHLA scores, by selected sociodemographic characteristics. Only 42 percent of the study sample exhibited the highest level of medical word recognition and, in general, REALM scores were higher among young men, those with an education level of >12 years, and an annual household income of ≥$25,001. By comparison, 80 percent of the sample was grouped into the highest level of reading comprehension. Short-TOFHLA scores were higher among women <65 years of age, those with education level of ≥12 years, and those with an annual household income of >$25,001.

Table 2.

REALM and Short-TOFHLA Scores, by Selected Characteristics, Baltimore, Maryland, 2008 (n = 100)

	REALM				Short-TOFHLA
Characteristics	<4th grade (0–18)	4th to 6th grade (19–44)	7th to 8th grade (45–60)	High school (61–66)	Inadequate (0–16)	Marginal (17–22)	Adequate (23–36)
	n (percentage)
All	5 (5.0)	21 (21.0)	32 (32.0)	42 (42.0)	12 (12.0)	8 (8.0)	80 (80.0)
Age (years)*
18–44	2 (6.7)	7 (23.3)	5 (16.7)	16 (53.3)	1 (3.3)	2 (6.7)	27 (90.0)
45–64	1 (2.3)	6 (14.0)	20 (46.5)	16 (37.2)	1 (2.3)	3 (7.0)	39 (90.7)
≥65	1 (3.8)	8 (30.8)	7 (26.9)	10 (38.5)	10 (38.5)	2 (7.7)	14 (53.8)
Sex
Male	3 (6.8)	11 (24.4)	11 (24.4)	20 (44.4)	8 (17.8)	3 (6.7)	34 (75.5)
Female	2 (3.6)	10 (18.2)	21 (38.2)	22 (40.0)	4 (7.3)	5 (9.1)	46 (83.6)
Education level*
<12 years	2 (10.0)	8 (40.0)	6 (30.0)	4 (20.0)	5 (25.0)	3 (15.0)	12 (60.0)
12 years	2 (4.0)	9 (18.0)	19 (38.0)	20 (40.0)	4 (8.0)	3 (6.0)	43 (86.0)
>12 years	0 (0.0)	4 (13.8)	7 (24.1)	18 (62.1)	3 (10.3)	1 (3.4)	25 (86.2)
Household income
Unknown	1 (6.7)	3 (20.0)	4 (26.6)	7 (46.7)	1 (6.7)	2 (13.3)	12 (80.0)
$0–$25,000	3 (5.9)	11 (21.6)	22 (43.1)	15 (29.4)	8 (15.7)	6 (11.8)	37 (72.5)
≥$25,001	1 (2.9)	7 (20.6)	6 (17.7)	20 (58.8)	3 (8.8)	0 (0.0)	31 (91.2)

^*Total does not sum to 100 because of missing values.

REALM, Rapid Estimate of Adult Literacy in Medicine; Short-TOFHLA, Short Test of Functional Health Literacy in Adults.

We used Classical Test Theory psychometrics (11) to identify the subset of the initial 44 knowledge questions with the highest reliability. The resulting survey, herein referred to as the Comprehensive Measure of Oral Health Knowledge (CMOHK), contained 10 of the initial basic knowledge questions (e.g. “How old are children when they get their first adult tooth? A) About 1 year old; B) About 3 years old; C) About 6 years old; D) About 13 years old”), six dental caries prevention and management questions (e.g., “When a person has a small cavity, how does the dentist usually treat it? A) Prescribing antibiotics; B) Placing a filling in the tooth; C) Pulling the tooth; D) Adding a dental implant”), five periodontal disease prevention and management questions (e.g. “Which of the following behaviors may cause periodontal disease? A) Biting your fingernails; B) Eating spicy foods; C) Drinking too much coffee; D) Smoking cigarettes”), and two oral cancer prevention and management questions (e.g. “What is the most common sign of cancer inside the mouth? A) A sore that lasts more than two weeks; B) Pain when you open your mouth; C) Gums that bleed when you brush; D) Teeth that have black spots on them”). Cronbach alpha for the 23 subset questions was 0.74.

CMOHK scores were divided into three categories. Scores from 0 to 11 represented “poor,” 12–14 corresponded to “fair,” and 15–23 represented “good” conceptual oral health knowledge. Table 3 shows the distribution of these scores, by selected sociodemographic characteristics. Overall, 42 percent of subjects were in the highest CMOHK category. Women aged ≥65 years, with an education level of >12 years, and an annual household income of ≥$25,001 (among those with known income) were most likely to have had good oral health conceptual knowledge. In comparison with the negatively skewed distributions of the REALM and Short-TOFHLA, CMOHK scores approximated a normal distribution (mean = 13.25; skewness = −0.366; kurtosis = 2.730) (Figure 1).

Table 3.

Oral Health Knowledge Scores by Selected Characteristics, Baltimore, Maryland, 2008 (n = 100)

	CMOHK
Characteristics	Poor (0–11)	Fair (12–14)	Good (15–23)
		n (percentage)
All	29 (29.0)	28 (28.0)	42 (42.0)
Age (years)*
18–44	15 (50.0)	6 (20.0)	9 (30.0)
45–64	9 (20.9)	15 (34.9)	19 (44.2)
≥65	5 (19.2)	7 (26.9)	14 (53.9)
Sex
Male	17 (37.8)	14 (31.1)	14 (31.1)
Female	15 (23.6)	18 (25.5)	22 (50.9)
Education level*
<12 years	12 (60.0)	5 (25.0)	3 (15.0)
12 years	11 (22.0)	17 (34.0)	22 (44.0)
>12 years	6 (20.7)	6 (20.7)	17 (58.6)
Household income
Unknown	6 (40.0)	2 (13.3)	7 (46.7)
$0–2013$25,000	17 (33.3)	15 (29.4)	19 (37.3)
≥$25,001	7 (20.6)	11 (32.3)	16 (47.1)

^*Total does not sum to 100 because of missing values.

CMOHK, Comprehensive Measure of Oral Health Knowledge.

Figure 1.

Side-by-side comparison of Rapid Estimate of Adult Literacy in Medicine (REALM), Short Test of Functional Health Literacy in Adults (Short-TOFHLA), and Comprehensive Measure of Oral Health Knowledge (CMOHK) score distributions, Baltimore, Maryland (n = 100).

Table 4 shows the distribution of CMOHK scores, by REALM and Short-TOFHLA scores. In general, CMOHK and REALM scores were consistent; those with the highest CMOHK score were also likely to have had the highest REALM score. By contrast, CMOHK and Short-TOFHLA scores were not consistent. Almost one-third of persons with “good” oral health knowledge scored in the lowest category of the Short-TOFHLA. Chi-square/Fisher’s exact test analysis confirmed that the CMOHK and REALM were significantly associated with one another (Fisher’s exact test P-value <0.01) whereas the CMOHK and Short-TOFHLA were not (chi-square P-value = 0.62). Note that REALM and Short-TOFHLA scores were significantly associated with one another (Fisher’s exact test P-value < 0.01).

Table 4.

REALM and Short-TOFHLA Scores, by CMOHK Scores, Baltimore, Maryland, 2008 (n = 100)

	CMOHK
Characteristics	Poor (0–11)	Fair (12–14)	Good (15–23)
	n (Percentage)
All	30 (30.0)	28 (28.0)	42 (42.0)
REALM
>4th grade (0–18)	3 (60.0)	2 (40.0)	0 (0.0)
4th to 6th grade (19–44)	12 (57.1)	6 (28.6)	3 (14.3)
7th to 8th grade (45–60)	10 (31.2)	11 (34.4)	11 (34.4)
High school (61–66)	5 (11.9)	9 (21.4)	28 (66.7)
Short-TOFHLA
Inadequate (0–16)	3 (25.0)	5 (41.7)	4 (33.3)
Marginal (17–22)	3 (37.5)	3 (37.5)	2 (25.0)
Adequate (23–36)	24 (30.0)	20 (25.0)	36 (45.0)

REALM, Rapid Estimate of Adult Literacy in Medicine; Short-TOFHLA, Short Test of Functional Health Literacy in Adults; CMOHK, Comprehensive Measure of Oral Health Knowledge.

Differences in sociodemographic associations across the three health literacy surveys were also found. Both age (Chi-square P-value < 0.05) and education level (chi-square P-value = 0.01) were significantly associated with CMOHK scores. For the REALM, only education level (Fisher’s exact test P-value = 0.04) was associated and for the Short-TOFHLA, only age (Fisher’s exact test P-value < 0.05) was significant. There were no statistically significant associations between health literacy (as measured either via the REALM, Short-TOFHLA, or CMOHK instruments) and having had a dental care visit in the preceding year.

Discussion

This investigation yielded a new measure of conceptual oral health knowledge suitable for use in future studies of oral health literacy. In preliminary testing, face and content validity were affirmed by consensus building among members of the research team. Reliability was supported by the high Cronbach alpha score. Criterion validity was supported by the statistically significant association between the CMOHK and REALM, an established health literacy instrument.

Although we anticipated that the CMOHK would also be associated with the Short-TOFHLA, the fact that we did not find a significant association between the CMOHK and Short-TOFHLA scores was not seen as evidence against the instrument’s validity. One possible explanation for this unexpected finding was the low number of respondents. Funding for this project allowed only 100 persons in the study sample. Power analysis suggested that about 500 persons would have been necessary to find a statistically significant association between the CMOHK and Short-TOFHLA, given the same conditions and power of 80 percent. It is possible that the CMOHK and Short-TOFHLA would have been statistically associated had the sample size been larger.

Another possible explanation for the lack of an association between the CMOHK and Short-TOFHLA is that the former might have been more closely related to word recognition than it was to reading comprehension, and our disparate statistical results were simply a reflection of these differences. This explanation is feasible because the theoretical basis for the REALM is the link between correct pronunciation of a word and knowledge of that word. A person who is unable to correctly pronounce the word “diabetes,” for example, probably does not know what diabetes is. The Short-TOFHLA, by contrast, is a reading comprehension test that does not necessarily require conceptual knowledge of health or disease.

Given that the REALM and Short-TOFHLA were differentially associated with the CMOHK and given that selected sociodemographic variables were differentially associated with all three health literacy instruments, we conclude that the REALM, Short-TOFHLA, and CMOHK are likely measuring unique health literacy attributes. Figure 2, a conceptual framework for the pathway between health literacy and oral health, illustrates this conclusion by expressing health literacy via four unique components: word recognition, reading comprehension, conceptual knowledge, and communication skills. Note that “communication skills” was included in the framework, despite being absent from the present study, because this factor represents one of the skills and abilities that researchers hypothesize as comprising health literacy (5). Note also that “appropriate decision-making” is at the center of the conceptual framework because this factor is integral to Ratzan and Parker’s definition of health literacy (1). As such “appropriate decision making” forms a bridge between the four individual health literacy components and “oral health outcomes.” Placing “appropriate decision making” at the center of the framework makes sense intuitively as it is difficult to imagine how word recognition, for example, might influence an individual’s unmet oral health needs without also influencing the numerous health-related decisions that are being made by that individual, along the way.

Figure 2.

Conceptual framework for associations between different components of health literacy* and oral health outcomes.

The relations of each health literacy component to one another are also explored by the conceptual framework. We hypothesize that the first three health literacy components influence decision-making whereas the fourth component is an extension of decision-making. That is, we hypothesize that reading skills, word recognition, and conceptual knowledge influence a person’s ability to make good decisions. We also hypothesize that, in order to translate appropriate health decisions into action, a patient must engage in meaningful communication with his or her practitioner. Future investigations will test whether there is empirical evidence for the existence of these four separate components and whether they independently affect decision-making and selected oral health outcomes via the defined pathways.

Note that the conceptual framework also reflects the present study’s findings. Our analyses showed that there were significant associations (signified by solid arrows) between word recognition and conceptual knowledge and between word recognition and reading comprehension. Our analyses also showed that there was no significant association between reading comprehension and conceptual oral health knowledge (signified by a dashed arrow). Note that the arrow is dashed because the association might have become significant had the sample size been larger (as previously discussed).

In summary, the conceptual framework separated health literacy into four components, each measured by different survey instruments. The framework also highlighted the importance of decision-making, showing that poor conceptual knowledge and difficulties with reading and word recognition probably do not influence health outcomes directly. Instead, poor knowledge and poor reading skills likely influence a myriad of decisions which, in turn, impact health and well-being. Finally, the framework highlights the unique role that communication plays in the process. Whereas decision-making serves as a bridge between selected components of health literacy and oral health, effective communication serves as an additional bridge between decision-making and health. In other words, individuals could not understand their health options unless they comprehended the messages they were receiving from others, and they could not move their decisions to action until their decisions were effectively communicated to a health professional.

Together, the preliminary results of the present study and the conceptual framework provide the rationale for including a measure of conceptual health knowledge in future investigations of oral health literacy. The present study also supports including multiple measures of health literacy in future studies, as word recognition, reading comprehension, and conceptual knowledge appear to reflect unique attributes. Of all of these health literacy components, however, we argue that conceptual knowledge may be one of the most useful for policymakers and practitioners. Those interested in improving oral health decision-making would find it more useful to direct interventions at specific areas of poor understanding than they would in directing interventions toward poor reading comprehension or poor word recognition.

One additional benefit of the CMOHK over the REALM and Short-TOFHLA relates to the distribution of their respective scores. Whereas the CMOHK exhibited a normal distribution in the present study, scores for the REALM (mean = 51.80; skewness = −1.518; kurtosis = 5.255) and Short-TOFHLA (mean = 28.76; skewness = −1.345; kurtosis = 3.700) were both negatively skewed. Because of these differences, the CMOHK is better suited for differentiating health literacy levels at the lower end of the scale than is either of the other two measures. Researchers should investigate whether measures of conceptual knowledge in nondental topic areas exhibit similar findings.

It is too early to know whether conceptual oral health knowledge (or any of the other health literacy components) is associated with decision-making because no measures of decision-making were included in the present study. Future studies will need to test the veracity of “appropriate decision-making” being at the center of the conceptual framework. It is also too early to know whether conceptual oral health knowledge is associated with oral health disparities. Although the present study showed that CMOHK scores were not significantly associated with dental visits, another study of oral health literacy found no significant association as well (12). Note that two other published studies did not provide any clarification on the matter, as they included dental visit frequency only as a covariate (13, 14). Future studies should explore whether health literacy is associated with more detailed measures of oral health care utilization, such as receipt of preventive services, episodic versus regular attendance, having a usual source of care, and use of emergency departments and nondental providers for care. Other oral health outcome measures will also need to be included in future studies, particularly ones secondary to decision-making.

The current study showed that conceptual oral health knowledge is lacking among low-income adults in Baltimore. Preliminary findings also suggested that conceptual oral health knowledge may represent a unique component of health literacy. A new conceptualization of the pathway between health literacy and oral health has been presented that has the potential to change how studies of decision-making in dentistry are conducted from here on. Future studies are needed to determine whether these conceptualizations are supported by empirical data and whether they lead to improvements in oral health and reductions in disparities.