Developing an Acceptability Assessment of Preventive Dental Treatments

Abstract

Objectives

Early childhood caries (ECC) is very prevalent among young Hispanic children. ECC is amenable to a variety of preventive procedures, yet many Hispanic families underutilize dental services. Acceptability research may assist in health care planning and resource allocation by identifying patient preferences among efficacious treatments with the goal of improving their utilization. The purposes of this study were (a) to develop a culturally competent acceptability assessment instrument, directed toward the caregivers of young Hispanic children, for five preventive dental treatments for ECC and (b) to test the instrument's reliability and validity.

Methods

An instrument of five standard treatments known to prevent ECC was developed, translated, reviewed by focus groups, and pilot tested, then tested for reliability. The instrument included illustrated cards, brief video clips, and samples of the treatments and was culturally appropriate for low-income Hispanic caregivers. In addition to determining the acceptability of the five treatments individually, the treatments were also presented as paired comparisons.

Results

Focus groups and debriefing interviews following the pilot tests established that the instrument has good face validity. The illustrated cards, product samples, and video demonstrations of the five treatments resulted in an instrument possessing good content validity. The instrument has good to excellent test–retest reliability, with identical time 1–time 2 responses for each of the five treatments 92 percent of the time (range 87 to 97 percent), and the same treatment of the paired comparisons preferred 75 percent of the time (range 61 to 90 percent).

Conclusions

The acceptability instrument described is reliable and valid and may be useful in program planning efforts to identify and increase the utilization of preferred ECC preventive treatments for target populations.

Introduction

The importance of patient and community participation in the design and evaluation of health care services is gaining recognition (1-3). Newton suggested that public, patient, and caregiver acceptability of treatment is a characteristic of service provision that should be included when planning new services, improving existing services, and measuring service quality (4). Acceptability research, ascertaining the perceptions of the prospective decisionmakers or recipients of care, has the potential to investigate whether patient treatment preferences are in agreement with recommendations arising from clinicians and evidence-based practice. Determining treatment acceptability from potential recipients may assist in health care planning, deciding how to allocate limited financial resources, and identifying preferences among possible efficacious treatments (5). The benefits of involving potential recipients in health care planning include improved implementation of research findings, better care, improved health, participative democracy, public accountability, and transparency (3). This approach of evaluating patient/caregiver perception of available treatments differs from consumer-based research aimed at product development and from program evaluation, which focuses upon patient experience.

The purposes of this study were (a) to develop a culturally competent acceptability assessment instrument for five preventive dental treatments for early childhood caries (ECC), directed toward the caregivers of young Hispanic children and (b) to test the instrument's reliability and validity.

Acceptability Research Addressing Pediatric Dental Treatments

There is a paucity of acceptability research on pediatric preventive dental treatment (3,5). Studies have focused on the acceptability of various behavioral management techniques used in pediatric dentistry (6,7), parental acceptance of treatment modalities for their children's decayed primary teeth (monitor, restore, or extract) (8), and parent and child acceptance of restorative materials for children with caries (tooth-colored or amalgam) (9). In summary, the acceptance of dental treatment has been related to esthetics, health concerns, clinician recommendations, previous treatment experiences, demographic and economic factors, and receiving treatment education (6-9).

The nascent field of acceptability research has, for the most part, only investigated restorative treatment preferences. Yet, there is an outstanding need to address patient, caregiver, and public preferences, attitudes, and values for preventive procedures that could increase the public participation in preventive treatments. In the case of children, parents are responsible for making the health care decisions, providing informed consent, and obtaining services that can have a long-term impact on their children's health and well-being.

The Importance of Understanding Caregiver Acceptability of Preventive Pediatric Treatments

Dental caries is the most common chronic disease in children and is amenable to a variety of preventive measures (10-12). ECC of the primary teeth can result in pain, infection, tooth loss, malocclusion, difficulty in chewing, malnutrition, sleep disruption, attention deficit, poor speech articulation, low self-esteem, slower social development, missed school days, and increases the risk for continued tooth decay throughout childhood (12-14). Nationally, 19 percent of children aged 2 to 5 years have untreated caries in their primary teeth with a far higher percentage of untreated caries among Mexican-American children (32 percent) and children living below the federal poverty level (33 percent) (15). In California, 35 percent of the 36 million residents are Hispanic, and 7 percent of the population is under the age of 5 (16). An oral health assessment of school-children in Alameda County (CA) revealed that 23 percent of kindergarteners from high-income schools and 46 percent from low-income schools had untreated decay, while a study in the Mexican-American community in nearby San Francisco found that 43 percent of children younger than 5 had ECC (13,14).

Incomplete understanding of caregiver acceptability of dental treatments for young children may contribute to underutilized pediatric dental services. Despite available dental services, a significant percent of families who are eligible for services do not utilize them (17). While Hispanic/Latino children are more likely to live in poverty and lack health insurance than children from other groups, reasons for underutilizing available dental services go beyond income and insurance status (17). Nonfinancial barriers associated with Hispanic/ Latino children underutilizing services include provider practices and behaviors, such as inadequate communication and sparse availability of Spanish-speaking providers, poor patient education, lack of a regular source of care, and a low parental educational attainment, which resulted in parental alternative beliefs regarding the etiology and treatment of disease (17). Lack of knowledge about available services and mistrust of the health care system have also been associated with underutilized dental services (18). Several additional studies have shown that minority children from low-income families, who lack insurance and whose parents are less well educated, are less likely to receive comprehensive dental care and are more likely to have acute dental disease (14,19-21). For these reasons, we chose to focus on preventive ECC treatments in a primarily Hispanic population.

Guidelines for Developing an Acceptability Assessment

Techniques for assessing acceptability include both quantitative and qualitative methods such as questionnaires, interviews, and focus groups, and it behooves researchers to utilize at least two of these methods to determine the extent that the findings from each method complement the other (4). For example, standardized vignettes describing the health care options allow for systematically measuring the acceptability of these services (4). Ideally, all services being assessed would be presented as equally effective and have similar side effects and costs, such that the choice could be made based solely on acceptability (4). Quantifying results using choice-based conjoint analysis has been recommended as a valid, reproducible, and internally consistent strength of preference measure (2).

Health literacy and limited English proficiency are concerns in the Hispanic population, particularly in low-income groups where lower education levels and language barriers may be present (22). Communication barriers arising from these factors are significantly lowered when information is presented visually as well as verbally, the messages are tailored and simplified, and comprehension is confirmed by an iterative “teach-back” approach (23,24). Additionally, cultural values and beliefs influence attitudes and behaviors toward health, prevention, and disease (25-27). Therefore, when developing an acceptability assessment for a specific group, for example caregivers of Hispanic children, community partners should be involved at the earliest stage to help define research objectives and have ongoing input into the research project (28).

Methods

We developed the assessment instrument to evaluate caregiver acceptability and preferences for preventive dental treatments for young Hispanic/Latino children. The sequence of development activities was the following: initial instrument development in consultation with behavioral and pediatric psychologists and a pediatric public health dentist; translation; focus group testing; pilot testing; and test–retest reliability evaluation. These activities took place over a 9-month period and are described in the next sections. An acceptability evaluation study is currently taking place in two Head Start and Early Head Start Centers in the East Bay region of the San Francisco Bay Area. The study was approved prior to initiation by the University of California San Francisco (UCSF) Institutional Review Board.

Initial Development of the Acceptability Assessment

Our primary objective was to develop a caregiver acceptability assessment that would be culturally and linguistically appropriate for low-income Hispanic/Latino families who may have limited health literacy or limited English proficiency. We followed several of Newton's (4) recommendations, described previously, for developing assessments of consumer acceptability of dental services.

We initially selected five standard treatments known to prevent ECC, based on the current scientific literature. We chose established treatments because our aim was to determine which of the existing treatments would be most acceptable and preferable to parents. These include three treatments for children: fluoride varnish, toothbrushing with fluoride toothpaste, and xylitol in foods and drinks; and two treatments for mothers: xylitol in gum and chlorhexidine rinse. All five treatments share certain key characteristics – scientific evidence of safety and effectiveness, readily available, and approximately equally low cost – thereby reducing the possibility that acceptability is based on these factors. Treatments varied along other dimensions, however, that are known to create differences in acceptability – for example, in smell, color, taste, frequency of application, ease of use, and in this situation, whether it was a preventive treatment for the mother or child.

Illustrated Treatment Cards

As part of the initial development, we created cards that described the variable aspects of each treatment in a visual, easy-to-read format. We focused on five characteristics of the treatments: (a) who would receive the treatment; (b) what the treatment was; (c) when or how often it was given; (d) where it would take place; and (e) what it tasted or smelled like. Based on the Flesch–Kincaid grade evaluation, the text on the cards was written at the fourth-grade level and was accompanied by illustrations. The illustrations showed Hispanic children, parents, and health providers engaging in oral hygiene activities (Figure 1).

Figure 1. Sample treatment card: brushing teeth with fluoride toothpaste.

Participants

We recruited participants for focus groups, pilot testing, and test–retest reliability analyses (total n = 78). Parents or primary caregivers who were at least 18 years of age and who had a Hispanic child aged 1 to 5 years attending the Head Start Centers were eligible to participate. Our sample consisted of primarily female Spanish-speaking immigrants from Mexico (98 percent), mean age 30 years old with a mean of 10 years of education. The assessment was conducted either in Spanish or English, as each participant preferred.

Focus Groups

We conducted two focus groups of caregivers of young children (n = 13) to ascertain the types of the information they would require in order to choose or reject specific dental treatments for themselves or their child. We then presented the illustrated treatment cards that we had developed for the study and asked parents to discuss their opinions of the cards for presentation style and content. Following the focus groups, we revised the treatment cards to address the caregiver feedback.

Pilot Testing and Test–Retest Assessments

We trained a Latino bilingual research associate from the community to conduct the acceptability assessment interviews. She conducted pilot interviews followed by debriefing sessions to test the protocols (n = 34). We revised the assessment in an iterative manner, based on feedback from the debriefing sessions. When participant feedback indicated that the protocols were clear and easy to follow, we recruited a small sample of participants (n = 31) to complete the acceptability assessment at two time points approximately 2 weeks apart.

Acceptability Assessment

The assessment consists of two sections: (a) the presentation and evaluation of the five separate treatments and (b) the presentation and evaluation of 10 paired comparisons of the five treatments. In the introduction to the first section, the interviewer stated that all treatments that will be presented are safe, effective, require no sedation or restraint, and are not intended to replace their regular oral hygiene habits. The assessment took about 15 minutes to complete.

Section 1: Presentation and Evaluation of Each of the Five Treatments

The interviewer presented a vignette for each of the five treatments, including the following three types of information: (a) samples of the supplies used in the treatment; (b) a video clip or photograph that illustrates the actual treatment; and (c) illustrated treatment card (Figure 1) that includes basic information about the treatment and is accompanied with a verbal description. Each of the five treatments was presented in a random order to prevent sequence effects in the ratings. Following this, the interviewer ascertained understanding through asking the participant to give a basic description of each treatment. If the participant did not demonstrate understanding, the interviewer presented the illustrated card with the basic information again. When the participant showed understanding, the interviewer assessed their basic acceptability of the treatment by asking the following three questions: (a) “is this treatment okay for a 1- to 2-year-old to receive” (yes, no, not sure); (b) “is this treatment okay for a 3- to 5-year-old to receive” (yes, no, not sure); and (c) “If this treatment was offered for (target child), how sure are you whether you would want him/her to receive it” [1 to 5 response scale ranging from 1 (very sure would not want) to 5 (very sure would want)]. For the treatments for mothers, the interviewer asked two questions: (a) “is this treatment okay for (target child's mother) to receive” (yes, no, not sure) and (b) “if this treatment was offered for (target child's mother), how sure are you whether you would want her to receive it” [1 to 5 response scale ranging from 1 (very sure would not want) to 5 (very sure would want)].

Section 2: Presentation of 10 Paired Comparisons

We anticipated that participants might report consistently high acceptability for all five treatments because they are noninvasive, are of low complexity, and many are already well embedded as daily habits (toothbrushing, chewing gum, etc.). Also, despite steps to establish a rapport, the participants might still have a deferential attitude to the interviewer and thus express favorable responses for all of the interventions. Therefore, to attempt to differentiate outcomes and obtain information regarding preferences among the five treatments, we presented the treatments in pairs (e.g., fluoride varnish paired with xylitol gum), for a total of 10 pairs. The interviewer presented the paired treatment cards with a brief explanation as needed and asked the participants to choose their preferred treatment in each pair. Each of the 10 pairs was presented in a random order. This information allows us to discern preferences among treatments and gain a sense of overall preferences. After completing this activity, the interviewer reviewed each of the 10 pairs and asked the participant to explain why she had chosen each particular treatment.

Analysis Plan

At each phase of development, we assessed the results obtained from the participants to inform the next iteration of the assessments. For the focus group and pilot testing data, the research team discussed participant comments and then reached consensus regarding revisions to the assessment. Using the data collected during the test– retest phase, percentage raw agreement and polychoric correlations were calculated to determine the reliability between time 1 and 2 acceptability and preference responses. Kappa was not used because it is especially limited when the prevalence (percentage of one response category) is very high or very low (29).

Results

Validity

Focus groups with community members and debriefing interviews with study participants showed this acceptability assessment instrument to have good face validity. It also had good content validity because of the variety and specificity of its components, which included verbal descriptions, illustrated cards, product samples, and video demonstrations of the five preventive dental treatments.

Reliability of the Ratings of the Five Preventive Treatments

Acceptability ratings for each of the five preventive treatments were highly reliable. On average, time 1–time 2 responses were identical (scale 1 to 5) for each of the five treatments 92 percent of the time (range 87 to 97 percent; Table 1). The polychoric correlations were 0.99 for all of the treatments except for chlorhexidine. The time 2 rating for the chlorhexidine rinse treatment had no variability (100 percent in one response category); hence, the correlation is inestimable. In cases where ratings changed for the five treatments, 83 percent of the changes were increases in acceptability.

Table 1. Test–Retest Caregiver Acceptability of and Preferences for Preventive Dental Treatments: Percent Raw Agreement, Polychoric Correlations, and Asymptotic Standard Errors (n = 31).

Acceptability item	% Agreement from T1 to T2	T1-T2: polychoric correlation	Asymptotic standard error
Acceptability rating: individual treatments (1-5 response range)
Toothbrushing with fluoride toothpaste	94	0.99	<0.01
Fluoride varnish	97	0.99	<0.01
Xylitol in food	94	0.99	<0.01
Xylitol in gum	90	0.99	<0.01
Chlorhexidine rinse	87	*
Preferences among treatments: 10 pairs of treatments
Fluoride varnish versus chlorhexidine	87	0.88	0.12
Fluoride varnish versus xylitol in food	84	0.86	0.12
Fluoride varnish versus toothbrushing with fluoride toothpaste	65	0.34	0.27
Fluoride varnish versus xylitol gum	90	0.99	0.00
Toothbrushing with fluoride toothpaste versus xylitol in food	71	0.47	0.27
Xylitol in gum versus xylitol in food	77	0.79	0.14
Xylitol in food versus chlorhexidine rinse	61	0.38	0.26
Toothbrushing with fluoride toothpaste versus xylitol in gum	77	0.72	0.18
Toothbrushing with fluoride toothpaste versus chlorhexidine rinse	65	0.44	0.24
Chlorhexidine rinse versus xylitol in gum	74	0.69	0.18

^*Ratings at T2 lacked any variability; therefore, correlations could not be calculated.

T1, time 1; T2, time 2.

Reliability of the 10 Matched Pairs of the Five Treatments

The time 1–time 2 reliability of 10 matched pair preferences showed that responses were highly consistent from time 1 to 2. Averaging across the 10 pairs from time 1 to time 2, the participants chose the same treatment 75 percent of the time (range 61 to 90 percent). The polychoric correlations ranged from r = 0.34 to 0.99 with six of them being >0.69 (Table 1). To determine whether overall preferences for individual treatments were reliable from time 1 to time 2, we summed the preferences for each of the five treatments at each time period. Polychoric correlations ranged from r = 0.57 to 0.79, thus indicating that the participants tended to prefer particular treatments consistently across time.

Discussion

This paper describes the development of a valid, reliable, and culturally competent acceptability assessment instrument of five preventive dental treatments for ECC, directed toward the caregivers of young Hispanic children. The instrument measured both patient acceptance for each of the five treatments and patient preference for the treatments when compared with one another. This acceptability assessment instrument was found to have both good face validity and content validity. The information on the five preventive dental treatments was presented visually as well as verbally, text was minimal and written at the fourth-grade level, and comprehension was confirmed by an iterative “teach-back” approach.

This acceptability assessment instrument was found to have excellent test–retest reliability for the ratings of the five preventive treatments, with identical time 1–time 2 responses for each of the five treatments 92 percent of the time (range 87 to 97 percent). The small changes in the ratings for the five preventive treatments tended to reflect an increased acceptance of the treatments, which most likely reflects increased familiarity and knowledge of the treatments because of the strong educational component of the instrument.

The instrument also had good test–retest reliability for the 10 matched pair preferences, as participants chose the same treatment 75 percent of the time (range 61 to 90 percent). While most paired preferences were highly correlated (0.69 to 0.99), some were not (0.32 to 0.47). This may reflect a lack of a defined preference within some treatment pairs. The paired preferences may have been affected by increased familiarity as well. In many cases, the participants expressed interest in learning more about treatments or in purchasing the products described in the scenarios (i.e., xylitol gum and xylitol sugar for food preparation). Thus, repeated exposure likely accounts for some of the variability in the preferences.

The ultimate goal is to use this assessment to determine the most acceptable intervention to provide for families in various communities to help prevent ECC. The sample size used during pilot and test–retest procedures was not intended to provide definitive results about the actual caregiver preferences for procedures to prevent ECC, as this is currently being assessed in a community sample of Hispanic families. This acceptability assessment instrument has also been adapted for an African-American target population, and data collection has begun with the caregivers of young African-American children.

The methods described in this paper could be applied to develop other acceptability assessment instruments within and outside the field of dentistry and for other defined populations. This acceptability assessment instrument will be available as a resource on the National Institutes of Health-funded UCSF Center to Address Disparities in Children's Oral Health website (http://www.ucsf.edu/cando/).