Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2017 Dec 1.
Published in final edited form as: Community Dent Oral Epidemiol. 2016 Aug 12;44(6):564–576. doi: 10.1111/cdoe.12250

Effectiveness evaluation of Contra Caries Oral Health Education Program for improving Spanish-speaking parents’ preventive oral health knowledge and behaviors for their young children

Kristin S Hoeft 1,*, Judith C Barker 2, Stephen Shiboski 3, Estela Pantoja Guzman 4, Robert A Hiatt 5
PMCID: PMC5097025  NIHMSID: NIHMS799932  PMID: 27517458

Abstract

Objectives

To determine the effectiveness of the Contra Caries Oral Health Education Program (CCOHEP) for improving low-income, Spanish-speaking parents’ oral health knowledge and behaviors for their young children. Mexican American children in the United States suffer disproportionately high prevalence and severity of early childhood caries, yet few evaluated, theory-based behavioral interventions exist for this population. CCOHEP is a theory-based curriculum consisting of four 2-hour interactive classes designed for and by Spanish speakers and led by designated community health educators (promotoras). Topics included children’s oral hygiene, caries etiology, dental procedures, nutrition, child behavior management and parent skill-building activities.

Methods

Low-income Spanish-speaking parents/caregivers of children aged 0–5 years were recruited through community services in an agricultural city in California. Survey questions from the Oral Health Basic Research Facts Questionnaire measuring oral health related behaviors and knowledge were verbally administered before, immediately after, and 3 months after attendance at CCOHEP. Five questions measured aspects of parental tooth brushing for their children (frequency, using fluoridated toothpaste, brushing before bed, not drinking or eating after nighttime brushing, adult assistance), three questions measured other oral health behaviors, and 16 questions measured oral health-related knowledge. Analyses of within-person changes between pre- and posttests, and again between post-test and three month follow up consisted of McNemar’s test for binary outcomes and sign tests for ordinal outcomes.

Results

Overall, 105 caregivers participated in CCOHEP (n= 105 pretest, n=95 posttest, n=79 second posttest). At baseline, all parents self-reported doing at least one aspect of toothbrushing correctly, but only 13% reported performing all five aspects according to professional guidelines. At posttest, 44% of parents reported completing all aspects of tooth brushing according to professional guidelines (p<.001). Statistically significant improvements were seen in 4 aspects of toothbrushing (p≤.008) between pretest and posttest (all but adult assistance). The second posttest showed 3 of these improvements were maintained, while adult assistance and the other reported behaviors improved (p≤.008). Between pretest and posttest, checking child’s teeth monthly and frequency of sweet drinks consumption improved (p≤.008) while frequency of eating sweet foods did not change. Knowledge was high at baseline (mean 12.83 of 16), but 6 knowledge items improved significantly between pretest and posttest. Improvements were maintained at second posttest.

Conclusions

CCOHEP improved low-income Spanish-speaking parents’ oral hygiene knowledge and self-reported behaviors for their young children, and change was sustained 3 months after the end of the intervention. Future, more rigorous evaluation of the intervention is recommended.

INTRODUCTION

Early childhood caries (ECC) is tooth decay of the primary dentition in children 71 months of age or less1. Left untreated, it is a chronic and painful condition affecting quality of life and interfering with child’s ability to perform necessary daily activities such as eat, sleep, talk, learn and play24. It also can affect speech, self-image, and put children at higher risk for dental problems later in life36. ECC affects 24% of all 2–4 year old children in the United States (U.S.)7. Among low income families, however, 43% of low-income Mexican American children experience ECC, over 9% more than low-income non-Hispanic white and black children7. Mexican American children also have more severe, extensive and untreated decay2. Though progress has been made in decreasing prevalence of dental caries generally in the United States, this youngest age group has witnessed an increase in ECC prevalence in recent years, with low-income children suffering higher increases than those in the general population79. Although rural/urban differences in ECC in nationally representative samples have not been studied, small localized studies of rural Mexican American and Latino populations have found very high prevalence of ECC, including high burden of untreated ECC1013.

While it is well established that Mexican American children suffer from a larger burden of ECC, very little has been firmly established about what causes the disparity between them and non-Hispanic white and black children. Small studies in single locations with heterogeneous Latino populations have found some factors affecting Latino children under age 5 years to be: parental knowledge, oral hygiene habits like brushing with fluoride toothpaste, dental care utilization, mother’s untreated caries status, and child cooperation while brushing, though few studies exist comparing these factors to non-Hispanic white or black groups1419.

While ECC is caused and sustained through a wide variety of factors, it can be prevented or ameliorated through modifiable behaviors, including twice-daily toothbrushing with fluoride toothpaste, establishing regular dental care by age 1 year, preventive care like fluoride varnish, and consuming diets with a low frequency of sugar-containing snacks and drinks2022. However, in two small studies of low-income Latino populations, nearly half of Latino 2-year olds did not meet this toothbrushing recommendation15,17.

Few oral health interventions exist targeting low-income Spanish-speaking parents of young children23,24. Moreover, very few have been evaluated or theory-based2527. While motivational interviewing is one educational approach that includes parents and has shown promising results in both behavior change and caries prevention, it has not been extensively evaluated for Spanish-speaking populations.28,29 Oral health promotion programs for children generally are not culturally or linguistically specific for Latino or Mexican American populations; tend to target older children through schools and preschools; and rarely include parents/caregivers in their education program24,30,31. Yet parents/caregivers are important for children 5 years and under, as it is the parent who primarily manages the child’s diet, supervises their behavior, undertakes oral care practices and seeks professional dental services. For example, Douglass and colleagues found that children who had parents assist in brushing their teeth had lower prevalence of caries than children whose parents did not assist15.

Contra Caries Oral Health Education Program (CCOHEP) is a curriculum for Spanish-speaking parents of children up to five years of age, consisting of 2-hour participatory/interactive sessions led by lay people trained as promotoras or community health outreach workers. Four people with parenting or childcare experience were hired as promotoras, and were trained primarily using CCOHEP itself, including more in-depth oral health detail so that they could field likely questions, as well as be proficient at group facilitation, study-specific procedures and ethics. CCOHEP was designed around the constructs in Bandura’s Social Cognitive Theory which addresses personal, social and environmental dimensions of behavior such as self-efficacy which has been shown to be related to maternal tooth brushing behavior for their children.32,33 For example, the second class session that focused on oral hygiene topics consisted of several items, described here along with a parenthetical note of where/how they fit with the theoretical model. These items are: explanation of how toothbrushing with fluoride toothpaste prevents cavities using both a biomedical explanation and the analogy of protecting your house from ants (these being outcome expectations); participants using disclosing tablets to brush their own teeth; demonstration of proper brushing and flossing technique and materials for adults and children using models (observational learning, situation); demonstration and practice of positions to brush a child’s teeth including giving feedback to a partner (observational learning and behavioral capability); practice flossing on a model (behavioral capability); “lift the lip” exam: discussion of behavior management and motivational techniques for brushing children’s teeth including group sharing and trouble-shooting to help parents be able to brush children’s teeth under challenging circumstances (emotional coping response, self-efficacy): setting toothbrushing goals to revisit at the next class (self-control): and, providing participants with toothbrushes and fluoridated toothpaste for all family members (environment). The target population participated in curriculum development and provided feedback to refine content and activities.34 The curriculum aims at increasing parents’ knowledge and skills using various didactic approaches as well as skill-building through diverse activities. Contra Caries consists of four sessions: 1- caries etiology and reducing germ sharing, 2- parent-assisted toothbrushing with fluoride toothpaste, flossing, and child behavior management during toothbrushing, 3- reducing sugar intake, snacking, diet, and bottle use, and 4- the tooth decay process, fluoride, and how to initiate and what to expect during dental visits. Classes are designed to (a) provide an understandable rationale for parents about why toothbrushing and other protective behaviors matter for young children and (b) to collectively lead to improved quality and quantity of oral hygiene. Development, details of content and acceptability of the curriculum are reported elsewhere34.

The objective of this paper is to determine the effectiveness of the Contra Caries Oral Health Education Program (CCOHEP) for improving Latino parents’ oral health related knowledge and self-reported behaviors for their young children both immediately after attendance at the educational intervention as well as three months after completion of the intervention.

METHODS

This study employed a single group, pre- post- test design. A pretest was administered at baseline, followed by a month-long intervention, an immediate posttest (Posttest1), and a second posttest (Posttest2) four months after baseline (3 months after completion of the intervention) (Figure 1). All study procedures are in full accordance with The Code of Ethics of the World Medical Association (Declaration of Helsinki) and were reviewed and approved beforehand by the Institutional Review Board at the University of California, San Francisco (Approval number 11-05603). The study was undertaken with the understanding and written informed consent of each adult participant.

Fig 1.

Fig 1

Timeline of CCOHEP Study Events

Data collection took place in an agricultural city with a population of 150,498, in northern California35. Approximately 75% of the city’s population is Hispanic, and 68.5% speak a language other than English at home. The city does not have optimally fluoridated tap water. The median income is $49,264, with 21% of the city population below the federal poverty line.

Recruitment lasted from August through December of 2011. Individuals were enrolled in a rolling fashion into intervention class groups with a goal of having 10 to 14 parents in each class, and at least 8 classes in total. Each class received the same four-session curriculum. Each of the four promotoras was scheduled to lead at least two classes. Posttest2 surveys were administered through March 2012. Participants received a $20 grocery store gift card for each Pretest and Posttest1 survey, and $30 for Posttest2 survey. They also earned a $5 gift card for each session they attended (maximum $20 for complete attendance), distributed after the final session.

Participants were parents or caregivers of a child aged 0–5 years, recruited through flyers or personal contacts through community services such as the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC), migrant farmworker daycare services, and low-income apartment complexes. Participants were usually mothers, but some fathers, grandparents, babysitters or foster parents participated as well. No sample size calculation was used to inform the sample size, as no existing effect size data was available and there was only a modest budget with which to carry out the research. Rather, a target sample size of 100 participants was decided on, based on the largest number that the pilot study budget would allow and the number of participants who could be recruited and finish classes in the timeframe of the study.

Inclusion criteria included: adult, aged 18+ years; self-identification as Hispanic or Latino; Spanish-speaking; low-income (≤200% US 2011 federal poverty level); plan to live in the study city for next 6 months; parent or caregiver for a child aged 0 to 5 years.

Questionnaires were verbally administered before and after attendance at CCOHEP (baseline and 1 month after baseline), and again 3 months after completion of the CCOHEP (4 months after baseline) (Figure 1). Questionnaire administration was conducted by bilingual researchers, not the promotora leading the classes. Posttest1 questionnaires were administered immediately at the conclusion of the class or if participants couldn’t stay after the final session, were scheduled individually at the study office or individual’s homes within a few days of the end of the class. Follow-up between Posttest1 and Posttest2 was done through phoning and texting individual participants 2 months after classes ended to remind them we would like to meet with them in 1 month. At the 3 months point after their class ended, participants were again phoned or texted to schedule the final data collection appointment in the study office, participant’s home, or at a community location such as WIC, a park, or an apartment building’s common room.

Questions included demographic characteristics for the caregiver and their child. If there was more than one child under age 5 years in their family, we asked the participant to respond about the child closest to age 3 years. Questions about oral health knowledge and self-reported behavior, taken from the Oral Health Basic Research Factors Questionnaire (BRFQ), measured aspects of ideal toothbrushing behavior for children’s teeth (5 questions: frequency of daily brushing, using fluoridated toothpaste, brushing before bed, no drinking/eating after nighttime brushing, adult assistance); behaviors around dental visits and diet (3 questions); and knowledge around oral hygiene causes and prevention measures (16 questions).

Questions were coded in a binary fashion, as either meeting the behavior or knowledge recommended by the American Academy of Pediatric Dentistry (AAPD), or not21. A summary score was created by summing 5 aspects of toothbrushing, with one point given for each recommended behavior resulting in a maximum score of 5; and a binary variable representing correctly doing all 5 aspects of toothbrushing was also established. Data were entered into a Microsoft Access database. Then 10% of the questionnaires were randomly selected to be double entered, comparison showing 100% agreement between the two data entries.

Analyses of within-person changes between pre- and posttest1 (initial change), as well as between posttest1 and posttest 2 (delayed change or maintenance) were based on McNemar’s test for binary variables, t-test for continuous, and sign test for ordinal categorical variables. We also used logistic and ordinal logistic regression models to evaluate the association between demographic factors and the following outcomes: a binary indicator of correctly performing all 5 aspects of toothbrushing behaviors at baseline, an ordinal measure of whether behavior change was observed, and a binary indicator of loss to follow-up. Stata 13 Software was used for all analyses36.

Questionnaire knowledge items answered as “don’t know” were coded as an incorrect answer; “don’t know” behavior items were excluded from analyses for that question. To assess the possible impact of loss to follow up on results, we compared distributions of demographic variables between individuals who stayed in the study, and those who were lost to follow up. We also assessed results based on imputed responses using the following two procedures: setting missing responses to the corresponding previous value, and also setting all missing responses to baseline values, to reflect expected results if participants lost to follow up did not exhibit changes in knowledge or behavior.

RESULTS

We reached the target number of parents/caregivers in CCOHEP, allowing five extra enrollees (n= 105 pretest, n=95 Posttest1, n=79 Posttest2). The 105 participants enrolled in the study were divided into 13 classes, with a mean of 7.7 participants per class at baseline (range 5–14). The rate of attrition between surveys was low (10 people did not take Posttest 1 (90% retention from baseline) and 26 did not take Posttest2 (75% retention from baseline); and there were no differences in baseline toothbrushing behaviors, parent age, parent education level, child age, child gender or child birth order between those who stayed in the program and those lost to follow up. A majority (89%) of participants had perfect attendance, attending all 4 class sessions, and those who missed classes reported it due to being sick or having a sick child. Details of the format and procedures during sessions and reasons for non-attendance or drop-out are provided elsewhere34. Study participants were mainly Mexican-born mothers with high school or lower education, caring for predominantly US-born children (Table 1).

Table 1.

Self-Reported Demographic Characteristics of Low-Income Spanish-Speaking Parent or Caregiver and their Child Closest to 3 Years of Age (N=105)

Caregiver Characteristic Count (%) or Mean ± SD; median; range
Mothers 81 (77%)
Age (years) 33.7 ± 8; median=33; range=18–57
Caregiver birth country
U.S. 11 (10%)
Mexico 91 (87%)
Other 3 (3%)
Years living in the U.S. if foreign born (n=94) 12.3 ±6.6; median=11; range=3–31
Years completed in School
6 years or less 35 (33%)
7–11 years 18 (17%)
High School diploma 33 (31%)
More than High School 19 (18%)
Self-rated oral health
Excellent 3 (3%)
Very Good 3 (3%)
Good 19 (18%)
Average (translated as “regular”) 53 (50%)
Bad 27 (25%)
Number of children in household 2.4 ± 1.1; median=2; range=0–5*

Child Characteristic

Age (years) 3.0 ± 1.3; median=3; range=0–5
Female 47 (45%)
U.S. Born 102 (98%)
First born child 38 (36%)
Never had dental visit 14 (13%)
Received fluoride varnish in past year 50 (48%)
Currently take fluoride supplement* 12 (18%)
Child stopped using a baby bottle before age 2 years 29 (31%)
Caregiver-rated child oral health
Excellent 14 (13%)
Very Good 10 (10%)
Good 44 (42%)
Average (translated as “regular”) 35 (33%)
Bad 2 (2%)

This table adapted from Table 3 in Hoeft et al BMC Oral Health49

*

Caregivers with zero children were grandparents or childcare providers who did not have children under 5 of their own, but cared for such children regularly

n=65 for the question about fluoride supplements, due to poorly followed skip pattern in the survey

n=94; (9 children under age 2 still using the bottle)

At baseline, 13% of participants self-reported performing all 5 toothbrushing behaviors correctly; the average number of correct behaviors was 2.6 ±1.6. Data from the five aspects of toothbrushing are reported for all three timepoints in Table 2. The most commonly reported behaviors were brushing child’s teeth twice a day (82%) and using fluoridated toothpaste (86% of those who brushed). The least reported behavior was brushing the child’s teeth before bedtime every day in the last week (22%). At baseline, there were no significant differences in parent age, parent education level, child gender or child birth order between those participants who reported all 5 toothbrushing behaviors, and those who didn’t (p>0.1). For toothbrushing frequency at baseline, however, child age mattered: parents reporting about children one year old or younger were less likely to report brushing their child’s teeth twice daily than were parents of older children (p=0.018).

Table 2.

Self-Reported Toothbrushing Behaviors at Baseline and Two Follow-Up Points

Prestest (Baseline) (N=105) Posttest1 (1 month) (n=95) Posttest2 (4 months) (n=79) Within-person change between Pretest and Posttest1 Within-person change between Posttest1 and Posttest2
TOOTHBRUSHING BEHAVIORS
Average number of correct behaviors 2.6 ±1.6
n=105
4.2±0.9
n=95
4.6±0.6
n=71
*
p<0.0001

p=0.0004
Performed all 5 toothbrushing behaviors correctly 14 (13%)
n=105
42 (44%)
n=95
47 (66%)
n=71

p<0.0001

p=0.0033
Brush twice a day or more 84 (82%)
n=102
94 (99%)
n=95
69 (90%)
n=77

p<0.0001

p=0.7815
Brush with fluoridated toothpaste 69 (86%)
n=80 additional 21 (20%) answered “don’t know” and were excluded
92 (96%)
n=95
75 (99%)
n=76

p=0.008

p= 0.5637
Brushed child’s teeth before bed every day last week 22(22%)
n=100
78 (82%)
n=95
60 (81%)
n=74

p= 0.0001

p= 0.7963
Child does not eat or drink anything after brushing his teeth and before going to bed 47 (47%)
n=100
76 (80%)
n=95
60 (92%)
n=74

p<0.0001

p= 0.0076
Adult assistance with brushing child’s teeth every night 55 (57%)
n=96
64 (67%)
n=95
70 (95%)
n=74

p=0.2568

p= 0.0001
*

arrow symbols denote the direction and significance of the change:

↑ Statistically significant improvement

↓ Statistically significant decrease

→ No statistically significant change

Results from Posttest1 (immediately after completion of CCOHEP, 1 month after baseline) showed 44% of participants reporting performing all 5 toothbrushing behaviors correctly, and the average number of correct behaviors was 4.2±0.9. Of participants with less than perfect scores at baseline, 79% improved at least one behavior between baseline and Posttest1. McNemar’s test for pairwise comparison examined within-person change between Pretest and Post1, assessed initial changes from the intervention, and found 4 of the 5 reported toothbrushing behaviors showing statistically significant improvement between Pretest and Posttest 1. Only adult assistance with brushing child’s teeth every night did not statistically improve between the first two timepoints.

McNemar’s test was used again between Posttest1 and Posttest2, assessing whether initial changes were sustained, further improved, or were lost over the three months after the intervention. The change between Posttest1 and Posttest2 is not statistically significant for the first 3 of the 5 reported behaviors, indicating sustained changes between the two time points. These three behaviors are brushing the child’s teeth twice a day, using fluoridated toothpaste, and brushing before bed every night in the previous week. There are statistically significant improvements for the last 2 reported behaviors, not eating or drinking after brushing but before going to bed and adult assistance with brushing child’s teeth every night. This latter behavior hadn’t initially improved at Posttest1. These changes are also illustrated in Figure 2. Results did not change when missing values were recoded as the last value.

Fig 2.

Fig 2

Comparison of Proportion (%) of Parents Reporting Performing AAPD Recommended Oral Hygiene Practices for Their Young Child between Pretest, Posttest1, Posttest2

Results for other reported oral health behaviors - monthly checking of child’s teeth and mouth (“lift the lip”), frequency of drinking sweet drinks, and frequency of eating sweet foods- are reported in Table 3. The number of caregivers who reported performing “lift the lip” monthly significantly increased between baseline and Posttest1, and was then maintained to Posttest2. Drinking of sweet drinks once a day or less also significantly improved in a positive direction between baseline and Posttest1, from one-third to 77% of participants, but then decreased non-significantly between Posttest1 and Posttest2 to just under two-thirds (63%). The number of caregivers giving their children sweet foods less frequently than every day, did not change significantly across the three time points: just under one-half the sample reported this behavior at all 3 time points. Thirteen children were using a bottle at baseline, and only 3 of these children managed to stop using the bottle by the end of the evaluation period (two at Posttest1, and one at Posttest2). The three that stopped were the oldest of the thirteen, two were aged 3 years and one was aged 2 years 8 months. Of the 10 who continued using the bottle, two were just over two years of age at baseline while the rest were under 18 months.

Table 3.

Self-Reported Other Oral Health Behaviors at Baseline and Two Follow-Up Points

Pretest (Baseline) (N=105) Posttest1 (1 month) (n=95) Posttest2 (4 months) (n=79) Within-person change between Pretest and Posttest1 Within-person change between Posttest1 and Posttest
OTHER ORAL HEALTH BEHAVIORS
Check the child’s teeth each month (“lift the lip”) 76 (75%)
n=103
94(99%)
n=95
72 (95%)
n=76

p<0.0001

p= 0.1797
Child ingests sweet drinks once a day or less 35 (33%)
n=105
73 (77%)
n=95
48 (63%)
n=76

p=0.0082

p= 0.1306
Child eats sweet foods less than every day 42 (40%)
n=104
45 (47%)
n=95
37 (49%)
n=76

p= 0.2568

p= 0.8575

↑ Statistically significant improvement

→ No statistically significant change

Overall, knowledge was high at baseline, with an average baseline score of 12.8 ±1.6 out of a total possible of 16 (Table 4). All caregivers knew that drinking soda is bad for children’s teeth; children’s teeth should be brushed twice a day; and sharing a toothbrush with your child is bad for their teeth. Almost all, 92% or higher, knew that it is necessary to go to the dentist at times other than when children’s teeth have a problem; fluoride toothpaste should be used when brushing; checking children’s teeth each month is good for their teeth; that eating or drinking something after brushing teeth but before going to bed was bad for teeth; and that caregiver brushing and flossing their own teeth was good for their child’s teeth. These knowledge items stayed high at Posttest1 and Posttest2.

Table 4.

Oral Health Knowledge at Baseline and Two Follow-Up Points

Pretest (Baseline) (N=105) Posttest1 (1 month) (n=95) Posttest2 (4 months) (n=79) Within-person change between Pretest and Posttest1 Within-person change between Posttest1 and Posttest
KNOWLEDGE
Total Knowledge Score 12.8±1.6
range=9–16
n=100
15.2±0.7
range=13–16
n=87
15.2±0.6
range=14–16
n=76

p<0.0001

p=0.2983
Answered all knowledge questions correctly (score=16) 3 (3%)
n=100
27 (31%)
n=87
22 (29%)
n=76

p<0.0001

p=1.00
Drinking soda is bad for children’s teeth 105 (100%)
n=105
89 (100%)
n=89
78 (100%)
n=78

N/A

N/A
It is not necessary to go to the dentist unless children have a problem with their teeth (false) 102 (98%)
n=104
88 (99%)
n=89
78 (100%)
p=0.5637

N/A
It is best to use fluoride toothpaste to brush child’s teeth 98 (93%)
n=105
89 (100%)
n=89
78 (100%)
n=78

p=.0082

N/A
Checking your child’s teeth each month for changes or spots is good for their teeth 101 (96%)
n=105
89 (100%)
n=89
78 (100%)
n=78

p=0.0455

N/A
A child eating something after brushing their teeth is bad for their teeth 97 (92%)
n=105
85 (96%)
n=89
78 (100%)
n=78

p=0.3173

p=0.0455
Putting fluoride varnish on children’s teeth is good for their teeth 79 (76%)
n=104
89 (100%)
n=89
78 (100%)
n=78

p<0.0001

N/A
Using a sippy cup at meal time doesn’t affect a child’s teeth 23 (22%)
n=105
31 (35%)
n=89
25 (32%)
n=78

p=0.0077

p=0.7316
Cavities are caused by bacteria in the mouth 78 (74%)
n=105
89 (100%)
n=89
77 (100%)
n=77

p<0.0001

N/A
Baby teeth are not very important because they don’t stay in the mouth for long (false) 65 (63%)
n=103
86 (97%)
n=89
78 (100%)
n=78

p<0.0001

p=0.1573
Sharing a toothbrush with your child is bad for their teeth 105 (100%)
n=105
89 (100%)
n=89
78 (100%)
n=78

N/A

N/A
Eating chips is bad for your child’s teeth 83 (79%)
n=105
87 (99%)
n=88
78 (100%)
n=78

p<0.0001

p=0.2173
Using the same spoon to taste your child’s food and feed them is bad for child’s teeth 92 (88%)
n=105
89 (100%)
n=89
78 (100%)
n=78

p=0.0016

N/A
Drinking a sippy cup with milk at bedtime is bad for your child’s teeth 69 (66%)
n=105
85 (96%)
n=89
78 (100%)
n=78

p<0.0001

p=0. 7316
Brushing and flossing your own teeth is good for your child’s teeth 104 (99%)
n=105
88 (100%)
n=88
78 (100%)
n=78

p=0.3173

N/A
Age children can brush their teeth alone (age 6 years or higher coded as correct) 31 (30%)
n=105
90 (95%)
n=95
69 (87%)
n=79

p<0.0001

p=0.083
A child’s teeth should be brushed at least 2 times a day 104 (100%)
n=104
95 (100%)
n=95
78 (100%)
n=78

N/A

N/A

↑ Statistically significant improvement

→ No statistically significant change

Seven knowledge items were lower at baseline (between 30% and 88%), but all improved to 95% or above by Posttest1 and were maintained at Posttest2. These knowledge items were about: fluoride varnish being good for teeth; cavities being caused by bacteria; importance of baby teeth; eating chips being bad for teeth; using the same spoon to taste your child’s food and feed them; using a sippy cup with milk at bedtime is bad for a child’s teeth; and, the age at which children can brush their teeth alone. In particular, at baseline only 30% of parents knew that children should be at least 6 years old to brush their teeth independently; some (10%) even suggesting that children as young as 1 and 2 years could brush unassisted. Caregivers giving the correct age at Posttest1 rose considerably, to 95%, but fell slightly to 87% at Posttest2; explained by 9 parents who reported the correct answer only immediately after the intervention, reverting back to a lower number similar to their baseline response by Posttest2.

One knowledge item, that using a sippy cup with milk at meal time is neither good nor bad for a child’s teeth, had the least number of correct answers at baseline (22%). While it improved slightly to 35% at Posttest1, it stayed quite low compared to all the other knowledge items.

Sensitivity analyses setting all missing values to the previous value did not result in any significant changes in findings. Setting missing Posttest2 values to baseline values did result in different findings for checking a child’s mouth (“lift the lip”) such that if all people missing at Posttest2 reverted to their original behavior of checking the child’s mouth, only 88% would have been doing the behavior correctly at Posttest2, a decrease from Posttest1 (p=0.0039). For knowledge questions, the question about it being bad for a child’s teeth to eat something after having their teeth brushed before bed, would not have improved significantly between Posttest1 and Posttest2 (p=0.1797), and the question asking about the age at which children can brush their teeth alone would have decreased between Posttest1 (88%) and Posttest2 (74%) (p=0.0017). All other items were not affected by the sensitivity analyses.

Analyses to examine the difference in rates of reported behavior change found no significant differences in toothbrushing behavior change or knowledge change by caregiver age, caregiver education, child age, child gender or child birth order.

DISCUSSION

The Contra Caries Oral Health Education Program successfully carried out 13 classes of 4 sessions each, retaining 90% of participants throughout the month of the class. CCOHEP was effective at improving low-income Spanish-speaking parents’ oral hygiene knowledge and behaviors for their children under age 5 years. At baseline, only 13% of parents self-reported providing optimal toothbrushing behaviors for their children, despite scoring high on oral health knowledge measures. Immediately after attending CCOHEP, however, optimal caregiver-reported behavior performance improved to 44%, and 3 months after attendance, rose to 66%.

Four of the five reported toothbrushing behaviors improved between Pretest and Posttest1, especially brushing at night. And all 5 behaviors had improved by the three month follow up. The one behavior with limited improvement between baseline and the first follow up was adult help with brushing. The need for adult help with brushing every day was little known or performed at baseline; and posttests revealed this to be a difficult behavior for parents to improve, despite skill-building activities and discussion in CCOHEP classes, especially to improve to daily assisting rather than just occasional help. It is possible that this behavior competed with life demands, schedules, or was a low priority for parents to change37. Seven parents improved the frequency of adult assistance from never or “sometimes” to “most of the time” between baseline and Pretest1 but did not achieve the optimal “all of the time” measure used in these analyses. Recent work by Benadof and colleagues describe the stages of children learning how to brush their teeth including a “Stage 3: road to tooth brushing independence” which includes alternating between independent brushing by the children and parent-assisted brushing, this pattern of intermittent parent assistance is something that increased after participation in CCOHEP, but was not captured in the dichotomous variable of the AAPD guidelines38. Another interesting component of this reported behavior was that 11 parents who reported assisted brushing at baseline subsequently reported not assisting their child with brushing at Posttest1. Previous qualitative research with Mexican-origin fathers in rural California found that they perceived themselves to be helping and supervising their children brush their teeth by only reminding them to brush—not actually physically assisting them39. It is possible that these 11 parents who no longer reported assisting their children’s toothbrushing had that same assumption at baseline and thought they were helping their children brush, but once they learned through Contra Caries that children need actual physical assistance rather than simple reminding, they changed their answer at Posttest1, suggesting that their baseline answer should also have been that they don’t assist their children with toothbrushing.

Other reported oral health behaviors of checking a child’s teeth monthly, and drinking sweet drinks once a day or less improved from Pretest to Posttest1, and were maintained thereafter. However drinking sweet drinks only improved to 77%, showing an area that could use further intervention or reinforcement. This low improvement, as well as no reduction in frequency of daily consumption of sweet foods, is not surprising given how difficult it is to change dietary behaviors. Likewise, the limited change in reported bottle use, with few people stopping bottle use by the recommended age of 18 months, matches previous comments from a similar population about the age parents perceive it to be acceptable for children to discontinue using a baby bottle34,.

Knowledge was generally high at baseline, in line with previous reports that despite this population having high caries prevalence, they have high basic knowledge about the detrimental effects of sugar consumption and that brushing teeth can improve oral health18,19,40,41. The more nuanced areas of knowledge that were lower at baseline - such as knowledge about fluoride varnish being good for teeth, cavities being caused by bacteria, baby teeth being important, eating chips being bad for teeth, it being bad to use the same spoon to taste your child’s food and then feed them, using a sippy cup with milk at bedtime being bad for a child’s teeth, and the age at which children can brush their teeth alone – were also similar to previous research with similar low-income Spanish-speaking Latino populations. This suggests that such populations generally may not have detailed knowledge around how caries are formed and perpetuated40. These complex areas of knowledge not only improved but were maintained after participation in the Contra Caries program.

Promotora interventions are usually quite acceptable, including for CCOHEP34, and having Spanish-speaking lay people from their own community be the one leading CCOHEP classes in a peer education approach allows for high cultural sensitivity and draws on the value of community, likely resulting in increased attendance, retention and a comfort in the setting. This might have been especially important with respect to being able to ask questions that they may not have felt comfortable asking in the dental clinic. However, outcome results of promotora education programs are sometimes mixed, though some have proven effectiveness42,43. This study adds to the literature demonstrating that promotora interventions can be effective as well as acceptable. Promotora interventions have occasionally been used for oral health, but this is the first effectiveness study of such an intervention. As Latino children continue to exhibit poorer oral health than their non-Hispanic white and black counterparts, the promotora health education model is gaining popularity. Having available a promising, evaluated curriculum like CCOHEP is valuable to those community organizations and public health agencies looking to improve health disparities for this vulnerable population.

There are some components of CCOHEP that are similar to motivational interviewing, such as individuals selecting their behavior change goals and providing participant-driven education29. However, CCOHEP includes a social support and group setting that participants particularly liked; this is typically absent in motivational interviewing because it usually has an individual focus.34 More examination of the mechanisms of action in both motivational interviewing and CCOHEP and their overlap is warranted.

This study is limited by lack of a control group, but within-person comparisons help reduce threats to validity. The major threat to this analysis is social desirability bias, in that parents who were exposed to the classes might have learned the correct answers (behaviors) from the class and reported those learned answers, but might not have been able to fully execute them at home. There is no way to know for sure if reported answers in the post-tests reflect actual behaviors or perceptions of desired behaviors. We attempted to minimize the latter by expressing the importance of honesty when filling out the questionnaire, having separate research staff unrelated to the promotora educators collect survey data, and using within-person change scores which minimizes the risk of confounding from individual characteristics.

Additional possible threats to validity include maturity (child development), history, and testing44. Although the one month delay between baseline and Posttest 1 is unlikely to be long enough for major child development to occur, the three month period between Posttest1 and Posttest2 is more likely affected by development and warrants further exploration in future studies, especially for behaviors like parent assistance with toothbrushing44. Another possible concern is that the incentives provided ($5 per session, and $20 or $30 per survey) could have been coercive. However, these low amounts have not previously been coercive.

Attrition was also a potential issue: 10 participants (9.5%) were lost to follow up between Pretest & Posttest1, and 16 (16.8%) lost between Posttest1 and Posttest2. This is lower than in other evaluated interventions25,26, however, and sensitivity analyses showed that there would be no significant differences if missing participants scored the same as their previous value. If missing participants only scored their baseline values at all subsequent follow-ups, there would have been lower Posttest2 performance on the behavior of checking a child’s teeth monthly, and less improvement in two knowledge items, but the other improvements, including reported behavior performance, would still have been statistically significant, suggesting that loss to follow up was not a strong driver in our findings.

Children’s caries and the oral health behaviors targeted by CCOHEP and examined in this study are shaped by many different social determinants.4548 CCOHEP was designed with an awareness of influences outside of an individual parent, the parent themselves, and the dynamic between the parent and child, and attempted to address all these levels of influence. However, longer follow-up study would have been helpful to determine if CCOHEP was enough to sustain these changes over the long term, especially as parents were back in the environments that shaped their original behaviors in the first place and further removed from CCOHEP.

Despite these limitations, this study makes a valuable contribution and is a key first step in evaluating a new intervention, providing initial data supporting future, larger evaluations of CCOHEP. This study is one of the first effectiveness evaluations of an oral health education intervention targeting low-income Spanish speaking parents of young children.

Contra Caries Oral Health Education Program improved Spanish-speaking parent oral hygiene knowledge and caregiver-reported behaviors for their children aged 0–5 years. Findings support previous research that many parents have some basic knowledge about children’s oral health, but that this knowledge lacks depth and detail and does not always result in the related health promoting behavior. This study provided more useful details in the practical application of oral health knowledge. CCOHEP can improve parents’ detailed knowledge of complex concepts and health promoting behaviors in this vulnerable population, knowledge which can be absorbed and maintained. Further research with randomization, a control group and longer follow up is warranted.

Acknowledgments

This research was supported by the National Institute of Dental & Craniofacial Research of the National Institutes of Health under Award Numbers R21DE019210 (funded project development and data collection); F31DE023282 (data entry and analysis, manuscript writing); U54DE019285, U54DE019275, & U54DE019259 (Basic Research Factors Questionnaire or BRFQ - development of survey items) and by a REAC (Cox fund) grant from the University of California San Francisco. The content is solely the responsibility of the authors and does not necessarily represent the official views of the funders.

We thank the study participants; the field research team (especially Sarah M. Rios); Drs. Jane Weintraub and Abby Alkon for input and consultation in drafting the original curriculum; Darby Henry for assistance with data entry; the promotoras (Lorena De Guzman, Diana Rincon Ruiz, Yadira Rubio Valdez and Leonor Santos); community advisory board; staff from Community Oral Health Services in Monterey County, CA; and, the Center to Address Disparities in Children’s Oral Health (CAN DO) at the University of California, San Francisco.

Contributor Information

Kristin S. Hoeft, Department of Epidemiology & Biostatistics, Center to Address Disparities in Children’s Oral Health (CAN DO), University of California, San Francisco, 3333 California Street, Suite 485, San Francisco, CA 94143.

Judith C. Barker, Department of Anthropology, History & Social Medicine, Center to Address Disparities in Children’s Oral Health (CAN DO), University of California, San Francisco

Stephen Shiboski, Department of Epidemiology & Biostatistics, University of California, San Francisco.

Estela Pantoja Guzman, Department of Anthropology, History & Social Medicine, University of California, San Francisco.

Robert A. Hiatt, Department of Epidemiology & Biostatistics, University of California, San Francisco

References

  • 1.American Academy of Pediatric Dentistry. Definition of Early Childhood Caries (ECC) Available from http://www.aapd.org/assets/1/7/D_ECC.pdf.
  • 2.United States Department of Health and Human Services. National Institute of Dental and Craniofacial Research. Oral health in America: a report of the Surgeon General. Rockville, MD: 2000. [Google Scholar]
  • 3.Kramer PF, Feldens CA, Ferreira SH, Bervian J, Rodrigues PH, Peres MA. Exploring the impact of oral diseases and disorders on quality of life of preschool children. Community Dent Oral Epidemiol. 2013;41:327–335. doi: 10.1111/cdoe.12035. [DOI] [PubMed] [Google Scholar]
  • 4.Filstrup SL, Briskie D, da Fonseca M, Lawrence L, Wandera A, Inglehart MR. Early childhood caries and quality of life: child and parent perspectives. Pediatr Dent. 2003;25:431–440. [PubMed] [Google Scholar]
  • 5.Nuttall NM, Steele JG, Evans D, Chadwick B, Morris AJ, Hill K. The reported impact of oral condition on children in the United Kingdom. Br Dent J. 2006;200:551–555. doi: 10.1038/sj.bdj.4813586. [DOI] [PubMed] [Google Scholar]
  • 6.Thomson WM, Poulton R, Milne BJ, Caspi A, Broughton JR, Ayers KM. Socioeconomic inequalities in oral health in childhood and adulthood in a birth cohort. Community Dent Oral Epidemiol. 2004 Oct;32:345–353. doi: 10.1111/j.1600-0528.2004.00173.x. [DOI] [PubMed] [Google Scholar]
  • 7.Dye BA, Thornton-Evans G. Trends in oral health by poverty status as measured by Healthy People 2010 objectives. Public Health Rep. 2010;125:817–830. doi: 10.1177/003335491012500609. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Statistics NCfH. Healthy People 2010 Final Review. Hyattsville, MD: 2012. [Google Scholar]
  • 9.Vargas CM, Ronzio CR. Disparities in early childhood caries. BMC Oral Health. 2006;6(Suppl 1):S3. doi: 10.1186/1472-6831-6-S1-S3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Quandt SA, Clark HM, Rao P, Arcury TA. Oral health of children and adults in Latino migrant and seasonal farmworker families. J Immigr Minor Health. 2007;9:229–235. doi: 10.1007/s10903-006-9033-7. [DOI] [PubMed] [Google Scholar]
  • 11.Finlayson TL, Gansky SA, Shain SG, Weintraub JA. Dental utilization by children in Hispanic agricultural worker families in California. J Dent Oral Craniofac Epidemiol. 2014;2:15–24. [PMC free article] [PubMed] [Google Scholar]
  • 12.Lukes SM, Simon B. Dental services for migrant and seasonal farmworkers in US community/migrant health centers. J Rural Health. 2006;22:269–272. doi: 10.1111/j.1748-0361.2006.00044.x. [DOI] [PubMed] [Google Scholar]
  • 13.Woolfolk M, Hamard M, Bagramian RA, Sgan-Cohen H. Oral health of children of migrant farm workers in northwest Michigan. J Public Health Dent. 1984;44:101–105. doi: 10.1111/j.1752-7325.1984.tb03059.x. [DOI] [PubMed] [Google Scholar]
  • 14.Watson MR, Horowitz AM, Garcia I, Canto MT. Caries conditions among 2–5-year-old immigrant Latino children related to parents’ oral health knowledge, opinions and practices. Community Dent Oral Epidemiol. 1999;27:8–15. doi: 10.1111/j.1600-0528.1999.tb01986.x. [DOI] [PubMed] [Google Scholar]
  • 15.Douglass JM, Tinanoff N, Tang JM, Altman DS. Dental caries patterns and oral health behaviors in Arizona infants and toddlers. Community Dent Oral Epidemiol. 2001;29:14–22. [PubMed] [Google Scholar]
  • 16.Nurko C, Aponte-Merced L, Bradley EL, Fox L. Dental caries prevalence and dental health care of Mexican-American workers’ children. ASDC J Dent Child. 1998;65:65–72. [PubMed] [Google Scholar]
  • 17.Weintraub JA, Prakash P, Shain SG, Laccabue M, Gansky SA. Mothers’ caries increases odds of children’s caries. J Dent Res. 2010;89:954–958. doi: 10.1177/0022034510372891. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Boggess KA, Urlaub DM, Moos MK, Polinkovsky M, El-Khorazaty J, Lorenz C. Knowledge and beliefs regarding oral health among pregnant women. J Am Dent Assoc. 2011;142:1275–1282. doi: 10.14219/jada.archive.2011.0113. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Cortes DE, Reategui-Sharpe L, Spiro A, Iii, Garcia RI. Factors affecting children’s oral health: perceptions among Latino parents. J Public Health Dent. 2012;72:82–89. doi: 10.1111/j.1752-7325.2011.00287.x. [DOI] [PubMed] [Google Scholar]
  • 20.Twetman S, Axelsson S, Dahlgren H, et al. Caries-preventive effect of fluoride toothpaste: a systematic review. Acta Odontol Scand. 2003;61:347–355. doi: 10.1080/00016350310007590. [DOI] [PubMed] [Google Scholar]
  • 21.American Academy of Pediatric Dentistry. Guideline on Infant Oral Health Care. 2014 Available from http://aapd.org/media/Policies_Guidelines/G_InfantOralHealthCare.pdf.
  • 22.Weintraub JA, Ramos-Gomez F, Jue B, et al. Fluoride varnish efficacy in preventing early childhood caries. J Dent Res. 2006;85:172–176. doi: 10.1177/154405910608500211. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Ramos-Gomez FJ. A model for community-based pediatric oral heath: implementation of an infant oral care program. Int J Dent. 2014:156821. doi: 10.1155/2014/156821. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Kranz AM, Rozier RG, Zeldin LP, Preisser JS. Oral health activities of Early Head Start and Migrant and Seasonal Head Start programs. J Health Care Poor Underserved. 2012;23:1205–1221. doi: 10.1353/hpu.2012.0090. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Weber-Gasparoni K, Reeve J, Ghosheh N, et al. An effective psychoeducational intervention for early childhood caries prevention: part I. Pediatr Dent. 2013;35:241–246. [PMC free article] [PubMed] [Google Scholar]
  • 26.Weber-Gasparoni K, Warren JJ, Reeve J, et al. An effective psychoeducational intervention for early childhood caries prevention: part II. Pediatr Dent. 2013;35:247–251. [PMC free article] [PubMed] [Google Scholar]
  • 27.Nurko C, Skur P, Brown JP. Caries prevalence of children in an infant oral health educational program at a WIC clinic. J Dent Child (Chic) 2003;70:231–234. [PubMed] [Google Scholar]
  • 28.Garcia R, Borrelli B, Dhar V, et al. Progress in Early Childhood Caries and Opportunities in Research, Policy, and Clinical Management. Pediatr Dent. 2015;37:294–299. [PubMed] [Google Scholar]
  • 29.Borrelli B, Tooley EM, Scott-Sheldon LA. Motivational Interviewing for Parent-child Health Interventions: A Systematic Review and Meta-Analysis. Pediatr Dent. 2015;37:254–265. [PubMed] [Google Scholar]
  • 30.Miller AP, Kameka M, Young-Whiting C. The effects of an oral health intervention on caregivers of Head Start children. J Natl Black Nurses Assoc. 2012;23:52–58. [PubMed] [Google Scholar]
  • 31.Albino J, Tiwari T. Preventing Childhood Caries: A Review of Recent Behavioral Research. J Dent Res. 2016;95:35–42. doi: 10.1177/0022034515609034. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Finlayson TL, Siefert K, Ismail AI, Sohn W. Maternal self-efficacy and 1–5-year-old children’s brushing habits. Community Dent Oral Epidemiol. 2007;35:272–281. doi: 10.1111/j.1600-0528.2007.00313.x. [DOI] [PubMed] [Google Scholar]
  • 33.Bandura A. Social Foundations of Thought and Action: A Social Cognitive Theory. Englewood-Cliffs, NJ: Prentice Hall; 1986. [Google Scholar]
  • 34.Hoeft KS, Rios SM, Pantoja Guzman E, Barker JC. Using community participation to assess acceptability of “Contra Caries”, a theory-based, promotora-led oral health education program for rural Latino parents: a mixed methods study. BMC Oral Health. 2015;15:103. doi: 10.1186/s12903-015-0089-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.United States Census Bureau. QuickFacts. Salinas (city), California: Available from http://quickfacts.census.gov/qfd/states/06/0664224.html. [Google Scholar]
  • 36.Stata Statistical Software: Release 13 [computer program] College Station, TX: StataCorp LP; 2013. [Google Scholar]
  • 37.Hoeft KS, Masterson EE, Barker JC. Mexican American mothers’ initiation and understanding of home oral hygiene for young children. Pediatr Dent. 2009;31:395–404. [PMC free article] [PubMed] [Google Scholar]
  • 38.Benadof D, Polk D, Documet P. Stages and transitions in the development of tooth brushing skills in children of Mexican immigrant families: a qualitative study. J Public Health Dent. 2015;75:337–342. doi: 10.1111/jphd.12108. [DOI] [PubMed] [Google Scholar]
  • 39.Swan MA, Barker JC, Hoeft KS. Rural Latino farmworker fathers’ understanding of children’s oral health. Pediatr Dent. 2010;32:400–406. [PMC free article] [PubMed] [Google Scholar]
  • 40.Hoeft KS, Barker JC, Masterson EE. Urban Mexican-American mothers’ beliefs about caries etiology in children. Community Dent Oral Epidemiol. 2010;38:244–255. doi: 10.1111/j.1600-0528.2009.00528.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Horton S, Barker JC. Rural Latino immigrant caregivers’ conceptions of their children’s oral disease. J Public Health Dent. 2008;68:22–29. doi: 10.1111/j.1752-7325.2007.00078.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Balcazar H, Alvarado M, Cantu F, Pedregon V, Fulwood R. A promotora de salud model for addressing cardiovascular disease risk factors in the US-Mexico border region. Prev Chronic Dis. 2009;6:A02. [PMC free article] [PubMed] [Google Scholar]
  • 43.Koniak-Griffin D, Brecht ML, Takayanagi S, Villegas J, Melendrez M, Balcazar H. A community health worker-led lifestyle behavior intervention for Latina (Hispanic) women: feasibility and outcomes of a randomized controlled trial. Int J Nurs Stud. 2015;52:75–87. doi: 10.1016/j.ijnurstu.2014.09.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Shadish, Cook, Campbell . Experimental and Quasi-Experimental Designs for Causal Inference. Boston: Houghton Mifflin; 2002. [Google Scholar]
  • 45.Fisher-Owens SA, Gansky SA, Platt LJ, et al. Influences on children’s oral health: a conceptual model. Pediatrics. 2007;120:e510–520. doi: 10.1542/peds.2006-3084. [DOI] [PubMed] [Google Scholar]
  • 46.Moimaz SA, Fadel CB, Lolli LF, Garbin CA, Garbin AJ, Saliba NA. Social aspects of dental caries in the context of mother-child pairs. J Appl Oral Sci. 2014;22:73–78. doi: 10.1590/1678-775720130122. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Boyce WT, Den Besten PK, Stamperdahl J, et al. Social inequalities in childhood dental caries: the convergent roles of stress, bacteria and disadvantage. Soc Sci Med. 2010;71:1644–1652. doi: 10.1016/j.socscimed.2010.07.045. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Newton JT, Bower EJ. The social determinants of oral health: new approaches to conceptualizing and researching complex causal networks. Community Dent Oral Epidemiol. 2005;33:25–34. doi: 10.1111/j.1600-0528.2004.00190.x. [DOI] [PubMed] [Google Scholar]
  • 49.Hoeft KS, Guerra C, Gonzalez-Vargas MJ, Barker JC. Rural Latino caregivers’ beliefs and behaviors around their children’s salt consumption. Appetite. 2015;87:1–9. doi: 10.1016/j.appet.2014.11.031. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES