Calibration of Self-Reported Oral Health to Clinically Determined Standards

Abstract

Objective

Self-report of oral health is an inexpensive approach to assessing an individual’s oral health status, but it is heavily influenced by personal views and usually differs from that of clinically determined oral health status. To assist researchers and clinicians in estimating oral health self-report, we summarize clinically determined oral health measures that can objectively measure oral health and evaluate the discrepancies between self-reported and clinically determined oral health status. We test hypotheses of trends across covariates, thereby creating optimal calibration models and tools that can adjust self-reported oral health to clinically determined standards.

Methods

Using National Health and Nutrition Examination Survey (NHANES) data, we examined the discrepancy between self-reported and clinically determined oral health. We evaluated the relationship between the degree of this discrepancy and possible factors contributing to this discrepancy, such as patient characteristics and general health condition. We used a regression approach to develop calibration models for self-reported oral health.

Results

The relationship between self-reported and clinically determined oral health is complex. Generally, there is a discrepancy between the two that can best be calibrated by a model that includes general health condition, number of times a person has received health care, gender, age, education, and income.

Conclusion

The model we developed can be used to calibrate and adjust self-reported oral health status to that of clinically determined standards and for oral health screening of large populations in federal, state, and local programs, enabling great savings in resources used in dental care.

Introduction

Oral health is an important component of an individual’s overall health. In the United States, tooth decay affects more than 90% of adults over age 40, and advanced gum disease affects 5%–15% of adults. Statistics show that over 40% of adults of low socioeconomic status have at least one untreated decayed tooth; most adults show signs of gum disease; severe gum disease affects about 14% of adults aged 45 to 54 years; and the annual incidence of coronal caries is higher than 30% among adults (1–4). Tooth decay affects more than one-fourth of U.S. children aged 2–5 years old and half of those aged 12–15 years old. About half of low-income children, aged 6–19 years old, have had decay (4). Furthermore, studies have indicated that oral health is related to systemic diseases through a number of routes, such as blood circulation (4–6). Oral diseases can significantly affect quality of life (7–11).

Measuring people’s oral health is complicated. Clinically, oral health is usually measured by at least four dimensions: dental caries, periodontal diseases, oral lesions, and need for tooth replacement. For each of these dimensions, measurements could be taken for each tooth, for up to 32 teeth. Measurement becomes even more complicated since dental caries can be measured by surfaces and by coronal and root caries; periodontal disease also presents complexity since measurements can be taken of pocket depth, or by loss of attachment, and can be measured for as many as six points.

The large number of oral health measures and the complications in measuring oral health status have caused different oral health indices to be created, including both clinical and self-reported measures, such as the decayed, missing, and filled teeth (DMF) index for dental caries (12); the Papillary, Marginal, Attached Gingiva (PMA) Index (13); the Periodontal Index (14); and the current method of measuring periodontal destruction (used by the National Institute of Dental and Craniofacial Research [NIDCR]) (15) for periodontal disease. Typically, periodontal status is assessed by some measure that incorporates clinical attachment loss, a concept first introduced by Ramfjord with the Periodontal Disease Index (PDI) (15–16). Marcus et al. utilized expert opinion to identify key factors and their respective weights that contribute to defining a clinical measure of oral health status (17–19). These epidemiologic measures are further complicated by their relationship to clinical decision making. By nature epidemiological measures look at one, or at most two, clinical signs of disease whereas the process of clinical decision-making requires consideration of multiple disease parameters, behavioral as well as socioeconomic factors. Hence, the relationship between epidemiologic measures and clinical decision making is complicated.

Patient self-report is the most convenient and cost-effective mechanism for obtaining first-hand health outcome information. However, self-reported measures are heavily influenced by personal beliefs; cultural background; and social, educational, and environmental factors. In addition, they often provide different assessment and values from those of clinically determined standards. The Oral Health Impact Profile (OHIP), based on self-report measures, reflects the negative effects of a person’s oral condition (17–23), is a widely used measure. A number of studies have examined the congruence of self-reported oral health status with clinically determined oral health status. Generally, patients are less likely to adequately assess their periodontal status and the presence of caries than they are to assess the number of their teeth, restorations, and the presence of fixed and removable prosthetics. The latter aspects are much more congruent with clinically assessed measures (24–26). As with Jones and colleagues (27), we hypothesized that patients’ self-reports of their oral health are different from their clinically determined oral health status. Hypotheses of trends across covariates such as age, gender, and education were also tested. For instance, more education is hypothesized to be associated with better oral health.

Other studies have examined the relationship between self-reported oral health status and clinical findings and have concluded that patients’ subjective reactions to their oral conditions strongly influence their perceived oral health, indicating that, although patients have difficulty assessing their clinical status, their perceptions play an important role in evaluating the outcomes of dental care and understanding of their health behaviors (28, 29).

The development of statistical calibration models that can take the advantage of self-reported oral health information while correlating such information to clinically determined standards is particularly useful in dental care programs (30–32). Well-constructed and selected oral health calibration models can use self-reported oral health information, along with other subjective factors related to oral health and general health, as well as sociodemographic characteristics; to directly calibrate self-reported oral health values to those of clinically determined standards.

In this paper, we use oral health data from the National Health and Nutrition Examination Survey (NHANES) to:

• evaluate the distribution of self-reported oral health and clinically determined oral health

• create summarized clinical oral health measures that can objectively measure oral health

• evaluate the discrepancies between self-reported oral health and clinically determined oral health

• test hypotheses of trends across covariates

• create and select optimal models for calibrating the self-reported oral health to clinically determined standards using calibration tools such as software program and nomogram

• use receiver operating characteristic curves to assess predictive accuracy of the calibrated model for its performance of predicting clinically determined oral health

• use cross-validation and bootstrap methods to validate the calibration model.

Methods

Descriptive statistics

A number of publicly available national surveys contain oral health information. Among the different surveys, the National Health and Nutrition Examination Survey offer comprehensive dental and oral health datasets with both self-report and clinical exam measures. Now an ongoing survey, NHANES has a long history of collecting oral health data and has matured through four waves, with its results released in 2-year waves. NHANES analytical instructions (33) recommend that two or more subwaves be used in analyses so that the estimates obtained are nationally representative. We used the 1999–2000 and 2001–2002 NHANES waves, which have a large number of common oral health measures for a total sample of 21004 individuals.

The NHANES components we used in the analysis are Demographics, Oral Health (Dentition), Oral Health (Periodontal/Lower), Oral Health (Periodontal/Upper), Oral Health (Referral), Oral Health (Questionnaire), Current Health Status, Hospital Utilization, and Medical Conditions.

NHANES Analytic Data contain measures from domains of self-reported oral health; clinically determined oral health, which come from three separate databases, Dentition, Periodontal, and Referral; and overall general health and patient characteristics.

1. Self-reported Oral Health. The measures used in this domain include “General condition of mouth and teeth,” “Limit foods because of teeth problems,” “Sip liquids to aid swallowing foods,” “Difficulties swallowing foods,” and “Mouth feels dry when eating meal.”

2. Clinically Determined Oral Health. In our analysis, actual summarized clinical measures are based on either professional judgment or direct clinical measures. We include the following nine items: “Edentulous,” “Root restorations,” “Assessment of soft-tissue lesion,” “Periodontal needs,” “Untreated Caries/Restorative needs,” “Gingival/oral hygiene instruction (OHI) needs,” “‘Overall recommendation for care,” “Significant needs,” and “Root Caries.”

3. Subject Characteristics. In the NHANES Analytic Data, such measures include demographics (e.g., age, gender, race, education, and marital status), socioeconomic status, and general health condition. For race/ethnicity, four mutually exclusive and exhaustive categories were created: non-Hispanic White (referred to as White), non-Hispanic Black (Black/African Americans.), Hispanic of any race (Hispanic, which was combined from Mexican Americans and other Hispanics), and those of other races, such as Asian or multiple races (Other).

Self-reported and clinically determined measures may have different ranges (for example, dichotomous, or four- or five-level Likert scales); therefore, the first step is to rescale each of the items so that all the items have the same range (e.g., ranges from 0 to 5):

$$x_{i(\mathit{rescale})}=\frac{x_i-min}{max-min}\times R,$$

where x_i is the original scale, min and max are the minimum and maximum values, respectively, and R is the range of the rescaled item. To take a simple example, assessment of soft-tissue lesion has 2 levels (0=“No”; 1=“Yes”). If we want the range of the rescaled item to be within 0 to 5 for a person with soft-tissue lesion, then the corresponding rescaled measure is (1–0)/(1–0) × 5 = 5. By summing these items and then rescaling again to the range of 0 to 100, we can obtain the unweighted indices.

Alternatively, weighted indices, which are the linear combinations of the rescaled items, were derived by principal component analyses (PCA). PCA transforms the rescaled items into a new set of measures that can be ranked by relative importance. We use the first principal component that can explain most of the variation. The formula for deriving weighted indices is

$${\mathit{Index}}_{\mathit{weighted}}={\sum }_{i=1}^k{w}_{1i}{x}_{i(\mathit{rescaled})},$$

where w_1i are the weights derived from PCA.

We evaluated the difference between self-reported and clinically determined oral health as follows: For continuous oral health measures, the differences between self-report and clinical standard were calculated and the distributions of the differences were obtained and tested (for mean=0) with one sample t-test. For categorical oral health measures, two-way tables were created to show the concordance between self-reported and clinically determined oral health. Chi-square statistics were calculated to test the concordance.

Multiple regression models

Calibration models have been used in various areas to improve the accuracy and precision of measurements (30, 31). Since we summarized clinically determined oral health as a continuous score, we used multiple regression to fit calibration models using self-reported oral health score, as well as the subject’s general health and characteristics as covariates.

In order to identify the best calibration model among all possible models with a different number of selected covariates, we used the adjusted coefficient of determination (adjusted R²) as the goodness of fit criterion for model selection.

Receiver operating characteristic curves

We used receiver operating characteristic (ROC) curves to assess predictive accuracy and characterize the sensitivity and specificity of the self-reported oral health and calibrated oral health for their performance in predicting clinically determined oral health. Data analyses were performed using SAS (version 9; SAS Institute, Cary, NC) software. Significance for the areas under the ROC curves (AUC) was tested. Gonen (2007)’s SAS macro was used to compare two ROC curves (34). The variance of the AUCs as well as the covariance between them can be computed using the general principles of the theory of U-statistics (35) and bootstrap methods (34). Chi-square test was used to test the significance for AUC under the ROC curves.

Validation analysis

To validate our calibration model, we used cross-validation and bootstrap methods. In cross-validation, the data are divided into k segments (usually of equal size) and one part is set aside for testing while the remaining k-1 parts are used for training the model. This process is repeated for each segment and the resulting measures of accuracy are averaged over the k segments. An alternative to cross-validation is to use bootstrap samples. A bootstrap sample is obtained by sampling from the observed data with replacement (34). While for cross-validation, K was a small number (on the order of 5 to 10), for bootstrap validation, this number will typically be larger (at least several hundreds or thousands). We generated 1,000 bootstrap samples. Each bootstrap sample is then used as a training sample. The difference between the AUC estimated from using the original data and the bootstrap samples is a measure of optimism. This difference was then subtracted from the resubstitution estimate of the measure of interest (e.g., AUC) to obtain the bootstrap-validated estimate.

Since NHANES uses a multilevel complex survey design, all the analysis but cross-validation and bootstrap methods incorporated the sample weights, stratification, and clustering of the design as suggested by the NHANES Analytic Guideline (33).

Nomograms

After a calibration model is developed mathematically, some simple calibration tools are usually developed to allow calibration to be done easily. Nomograms, calibration tables, and personal digital assistant (PDA) programs are the three most commonly used tools, allowing to approximate visual computation of the calibrated values.

Nomograms are graphs consisting of curves of different variables that are intersected by a line cutting through these curves to provide relationships between these variables. These methods are useful for screening the oral health of large samples and for evaluating the oral health of large populations. They will enable substantial savings to be realized in effort and resources in preventive and restorative dental care by more efficiently addressing the needs of underserved populations.

We developed a nomogram based on our final selected model. It is a graphical calculating device that can calibrate self-reported oral health to the clinical standards based on patient characteristics. The values of each covariate correspond to certain points in the plot. This relationship is built through the regression coefficients. The sum of the itemized points and the intercept of the regression equation result in the calibrated value.

Results

Table 1 shows distributions of the 1999–2002 NHANES subjects’ characteristics. Sixty-three percent were under 49 years old and 49% were male. For education, 62% completed high school or beyond. About two-thirds of the subjects (68%) were White, followed by 16% Hispanic, 12% Black, and 5% Other. Most subjects (85%) rated their general health (very) good or excellent

Table 1.

Distributions of NHANES 1999–2002 subjects’ characteristics

a. Characteristics	Unweighted		Weighted
	N	(%)	N	(%)
Age (years)
18–34	3695	(35.0)	6.49×107	(31.5)
35–49	2424	(22.9)	6.50×107	(31.6)
50–64	2011	(19.0)	4.27×107	(20.7)
65+	2438	(23.1)	3.34×107	(16.2)
Gender
Male	9,660	(48.9)	1.36×108	(48.7)
Female	10,099	(51.1)	1.43×108	(51.3)
Education
Less than high school	9,660	(58.7)	9.57×107	(37.5)
High school diploma/GED	2,563	(15.6)	5.37×107	(21.0)
More than high school	4,217	(25.6)	1.06×108	(41.3)
Race
White	7,354	(37.2)	1.88×108	(67.5)
Black	4,710	(23.8)	3.27×107	(11.7)
Hispanic	6,917	(35.0)	4.39×107	(15.8)
Other	778	(3.9)	1.41×107	(5.0)
Marital Status
Married	5,218	(42.0)	1.10×108	(52.1)
Widowed	882	(7.1)	1.24×107	(5.9)
Divorced/separated	1,102	(8.9)	2.29×107	(10.8)
Never married/living with partner	5227	(42.1)	6.59×107	(31.2)
Annual Family Income
$0 to $19,999	6921	(36.6)	7.34×107	(27.7)
$20,000 to $44,999	5548	(29.3)	7.53×107	(28.4)
$45,000 to $74,999	3139	(16.6)	5.48×107	(20.6)
$75,000 and higher	2649	(14.0)	5.61×107	(21.1)
General Health Condition
Excellent	6,341	(32.1)	8.33×107	(29.9)
Very good	5,048	(25.6)	8.31×107	(29.8)
Good	5,466	(27.7)	7.51×107	(27.0)
Fair	2,381	(12.1)	2.97×107	(10.7)
Poor	509	(2.6)	7.27×106	(2.6)
Times Received Health Care Last Year
None	3,065	(15.5)	4.35×107	(15.6)
1–3	9,942	(51.3)	1.39×108	(50.0)
4–9	4,558	(23.1)	6.36×107	(22.8)
≥10	2,174	(11.0)	3.22×107	(11.6)
Seen Mental Health Professional last year
Yes	1,251	(7.3)	2.19×107	(8.4)
No	15,924	(92.6)	2.41×108	(91.6)
Had Cancer
Yes	799	(8.4)	1.60×107	(8.1)
No	8,662	(91.5)	1.82×108	(91.9)

Table 2 displays the distribution of self-reported oral health. For the general condition of mouth and teeth, 70% answered (very) good. The majority of the subjects (76%) never limited their food intake because of oral problems. Seven percent sipped liquid to aid in swallowing foods. Five percent had difficulty swallowing foods. Regarding main reasons of last dental visit for those who used dental care, 57% went in on their own for a checkup/exam/cleaning, 28% went because something was wrong.

Table 2.

Distribution of self-reported oral health

Characteristics	Unweighted		Weighted
	N	(%)	N	(%)
General Condition of Mouth/Teeth
Very good	5,381	(29.6)	8.58×107	(31.6)
Good	6,988	(38.4)	1.04×108	(38.4)
Fair	4,083	(22.5)	5.52×107	(20.4)
Poor	1,723	(9.5)	2.59×107	(9.5)
Limit foods Because of Teeth Problems
Never	7,856	(74.4)	1.56×108	(75.5)
Seldom	1,105	(10.5)	2.28×107	(11.0)
Sometimes	976	(9.2)	1.73×107	(8.4)
Often	254	(2.4)	4.41×106	(2.1)
Very often	197	(1.9)	3.15×106	(1.5)
Always	169	(1.6)	2.74×106	(1.3)
Sip Liquids to Aid Swallowing Foods
Yes	522	(7.3)	1.02×107	(7.0)
No	6,611	(92.7)	1.36×108	(93.0)
Amount of Saliva in Mouth
Don’t notice it	6,447	(90.4)	1.33×108	(91.3)
Too little	333	(4.7)	6.47×106	(4.4)
Too much	340	(4.8)	6.10×106	(4.2)
Difficulties swallowing foods
Yes	367	(5.1)	7.70×106	(5.3)
No	6,764	(94.8)	1.38×108	(94.7)
Mouth feels dry when eating meal
Yes	347	(4.9)	6.82×106	(4.7)
No	6,777	(95.0)	1.39×108	(95.2)
When did you last visit a dentist?
Less than 6 months	7,340	(40.4)	1.23×108	(45.5)
6 months to 1 year	3,095	(17.0)	4.30×107	(15.9)
1 to 2 years	2,254	(12.4)	3.26×107	(12.0)
2 to 3 years	1,112	(6.1)	1.65×107	(6.1)
3 to 5 years	989	(5.4)	1.51×107	(5.6)
More than 5 years	1,723	(9.5)	2.71×107	(10.0)
Never have been	1,610	(8.9)	1.30×107	(4.8)
Main reason for last dental visit
Went in on own for checkup/exam/cleaning	9,391	(56.7)	1.42×108	(55.1)
Was called for checkup/exam/cleaning	763	(4.6)	1.37×107	(5.3)
Something was wrong, bothering, or hurting	4,615	(27.9)	7.28×107	(28.2)
Went for treatment of a condition that dentist discovered at earlier checkup or examination	1,071	(6.5)	1.83×107	(7.1)
Other	652	(3.9)	1.03×107	(4.0)

Table 3 shows the distribution of clinically determined oral health. Twenty-one percent of the sample had untreated caries; 10% had root caries; 20% had periodontal needs; 5% had at least one tooth with loss of attachment (LOA) greater than 6 mm measured from mid-facial or mesial; 29% had gingival or OHI needs; and 7% had root restorations.

Table 3.

Distribution of clinically determined oral health

Characteristics	Unweighted		Weighted
	N	(%)	N	(%)
Untreated Caries
Yes	4,392	(24.2)	5.60×107	(20.6)
No	13,793	(75.8)	2.15×108	(79.4)
Root Caries
Yes	937	(11.9)	1.55×107	(9.8)
No	6,928	(88.0)	1.42×108	(90.2)
Soft-Tissue Lesion
Yes	33	(0.2)	6.65×105	(0.2)
No	18,152	(99.8)	2.70×108	(99.8)
Periodontal Needs
Yes	3,394	(18.7)	5.32×107	(19.6)
No	14,791	(81.3)	2.18×108	(80.4)
LOA (mid-facial)
≥6mm	400	(2.3)	6.98×106	(2.5)
<6mm	17,785	(97.7)	2.64×108	(97.5)
LOA (mesial)
≥6mm	425	(2.2)	6.73×106	(2.6)
<6mm	17,760	(97.8)	2.64×108	(97.4)
Gingival/OHI Needs
Yes	5,381	(29.6)	7.73×107	(28.5)
No	12,804	(70.4)	1.94×108	(71.5)
Dental Implants
Yes	61	(0.4)	1.42×106	(0.6)
No	14,430	(99.6)	2.29×108	(99.4)
Root Restorations
Yes	541	(6.9)	1.08×107	(6.9)
No	7,324	(93.1)	1.47×108	(93.1)
Overall Recommendation of Care
Continue your regular routine care	8,323	(49.0)	1.33×108	(52.1)
See a dentist at your earliest convenience	7,844	(46.2)	1.11×108	(43.6)
See a dentist within the next 2 weeks	790	(4.7)	1.06×107	(4.2)
See a dentist immediately	29	(0.2)	4.38×105	(0.2)

By plotting clinical diagnosis of root caries/periodontal needs/gingival needs against self-reported responses to the question, “Do you limit foods because of teeth problems?” (Fig. 1), we observe a relationship between self-reported oral health and clinically determined oral health. As the responses between detected root caries and the perceived oral health variable “limitation of foods because of oral problems” changes from “never” and “seldom” to “always,” the percentage of diagnosed root caries increases monotonically from about 8% up to 34%. The percentage of diagnosed periodontal and gingival needs showed similar monotonically increasing trends, topping when the self-response variable “limitation of foods because of oral problems” is “(very) often,” and then it decreased.

Figure 1.

Table 4 presents the relationship between summarized oral health and the subjects’ characteristics. Age is negatively related to oral health and females had better oral health than males as is expected. More education is associated with better oral health, with means changing from 50% to 58 % to 66% as subjects’ education level increases from less than high school, to high school diploma, to more than high school education, respectively. For ethnicity, whites have the best oral health, followed by Other, Hispanics, and African Americans. For marital status, those who were married had the best oral health, whereas those who were widowed had the worst oral health; the others are in between.

Table 4.

Relationship between clinically summarized oral health and subjects’ characteristics

Characteristics	Mean Clinically Summarized Oral Health Index	Chi-square p-value
Age (years)
18–34	62.5	0.0008
35–49	60.8
50–64	59.2
65+	57.5
Gender
Male	55.5	<.0001
Female	59.3
Education
Less than high school	49.6	<.0001
High school diploma/GED	57.7
More than high school	66.3
Race
Hispanic	52.3	<.0001
White	60.2
Black	49.7
Other	55.2
Marital Status
Married	61.8	<.0001
Widowed	55.3
Divorced/separated	56.3
Never married/living with partner	56.8
Annual Family Income
$0 to $19,999	50.6	<.0001
$20,000 to $44,999	55.3
$45,000 to $74,999	61.0
$75,000 and higher	66.1
General Health Condition
Excellent	59.0	<.0001
Very good	61.1
Good	55.3
Fair	50.5
Poor	48.2
Times Received Health Care Past Year
None	54.2	0.0006
1–3	58.6
4–9	57.9
≥10	56.2
Seen Mental Health Professional Past Year
Yes	57.6	0.6198
No	58.1
Had Cancer
Yes	59.0	0.0087
No	60.4

It is clear that higher income is associated with better oral health. Those with better general health tend to have better oral health. For the number of times one received health care within the past year, those not receiving health care had the worst oral health and those who received health care 1 to 3 times had the best oral health. There was no difference in mean oral health by whether one saw a mental health professional within the past year.

Table 5 presents the relationship between subject characteristics and the differences of summarized clinical and self-reported oral health indices, defined as the latter subtracted from the former. All the means of the differences are negative, indicating, on average, that people in every category overestimate their oral health. Specifically, people over 65 years old tended to overestimate their oral health more than did other age groups. Males overestimated their oral health more than females. Less education is associated with larger difference with means ranging from –32, to –27, to –22 for education of less than high school, high school diploma, and more than high school education, respectively. Whites had the smallest mean difference between clinical and self-reported oral health index, followed by Other, Hispanics, and African American. There is a clear trend of the mean differences across the categories of income ranging from –29 down to –20 corresponding to income from less than $20,000 up to $75,000 or more, respectively. There is an association, but no clear trend, between general health and the mean differences: A lower number of visits to a health care professional in the past year is associated with a larger mean difference between clinical and self-reported oral health index. Those who saw a mental health professional within the past year had a smaller mean difference than those who did not.

Table 5.

Relationship between subjects’ characteristics and the differences of clinically and self-reported summarized oral health

Characteristics	Mean Differencea	Chi-square p-value
Age (years)
18–34	−25.25	<.0001
35–49	−24.93
50–64	−25.38
65+	−27.09
Gender
Male	−27.94	<.0001
Female	−23.39
Education
Less than high school	−32.00	<.0001
High school diploma/GED	−27.30
More than high school	−22.43
Race
Hispanic	−28.23	<.0001
White	−23.94
Black	−33.97
Other	−26.31
Marital Status
Married	−24.64	0.0003
Widowed	−27.50
Divorced/separated	−26.06
Never married/living with partner	−27.34
Annual Family Income
$0 to $19,999	−29.07	<.0001
$20,000 to $44,999	−27.34
$45,000 to $74,999	−25.38
$75,000 and higher	−19.80
General Health Condition
Excellent	−24.75	<.0001
Very good	−24.05
Good	−27.56
Fair	−26.76
Poor	−21.39
Times Received Health Care Past Year
None	−29.45	<.0001
1–3	−25.19
4–9	−24.72
≥10	−23.96
Seen Mental Health Professional Past Year
Yes	−21.24	0.0159
No	−25.89
Had Cancer
Yes	−23.93	<.0001
No	−25.71

^aThe differences are defined as summarized clinical oral health index minus self-reported oral health index.

Through model selection using an adjusted R², we found the best model (Table 6), including all the measures, with an adjusted R² of 0.21. The model showed that those in “very good” and “excellent” general health have significantly better summarized oral health than did those with “poor” general health, and those who received health care had significantly better oral health than did those who had not accessed health care. Females, regardless of education, income, age, etc., had better summarized oral health than males.

Table 6.

Calibration model of self-reported oral health score to summarized clinical oral health score by multiple regression using subject’s general health and characteristics as covariates

Measure	Calibration Coefficient	Adjusted
		Pr > |t|	CI	95%
Intercept	26.24	<.0001	16.04	36.43
Summarized Self-Reported Oral Health	0.32	<.0001	0.22	0.42
General Health (Reference category: Poor)
Fair	2.80	0.2486	-2.07	7.68
Good	3.02	0.2109	-1.81	7.84
Very good	6.70	0.0152	1.39	12.01
Excellent	7.32	0.0121	1.73	12.92
Number of Times Received Health Care (Reference category: None)
1–3	4.74	0.0040	1.64	7.84
4–9	6.24	0.0001	3.40	9.08
≥10	6.41	0.0012	2.75	10.07
Gender (Reference category:Male)
Female	5.91	<.0001	4.18	7.65
Age	0.01	0.7204	-0.05	0.07
Education (Reference category: < High School)
High School/GED	6.39	<.0001	3.91	8.86
>High School	10.21	<.0001	7.97	12.44
Income (Reference category: < $20K)
$20,000 to $44,999	4.10	0.0009	1.84	6.36
$45,000 to $74,999	7.22	<.0001	4.21	10.23
≥$75,000	12.55	<.0001	10.50	14.60

Note: Calibration model was selected by the best adjusted R², which is 0.21.

The ROC curves for the self-reported oral health and calibrated oral health were compared for their performance of predicting clinically determined oral health (Fig. 2). Clinically determined oral health is being dichotomized at <80% and ≥80% on the 0 to 100% scale, where 0 and 100% indicate worst and perfect oral health status, respectively. The Area-under-the-curve (AUC) values (95% confidence interval [CI]) for self-reported oral health and calibrated oral health were 0.681 (0.666–0.696) and 0.747 (0.733–0.761), respectively. These results are confirmed by estimates with 1,000 bootstrap samples. The difference between these AUC values was significant (P<0.0001). The diagnostic accuracy of the calibrated oral health to predict clinically determined oral health was superior to that of self-reported oral health, although both measures obtained relatively high values for AUC.

Figure 2.

The estimate of the five-fold validated AUC for calibrated oral health is 0.746 (95% CI: 0.732–0.760). Using 1,000 bootstrap samples, the optimism correction for over-fitting was found to be 0.0002 for the estimate of the AUC for calibrated oral health. It is ignorable when subtracted from the resubstitution estimate of 0.747. Therefore, the 1,000 bootstrap-validated estimate of the AUC for calibrated oral health is indistinguishable from resubstitution estimate. Note that the bootstrap-validated estimate is virtually identical to that of the five-fold cross-validation estimate.

Using the top selected calibration model, we have created a nomogram to visually calibrate self-reported oral health (Fig. 3). The mechanics of this tool are as follows. First, find the individual patient’s values on each axis and draw a line upward to determine how many points the patient receives for each variable value. Once all points have been determined, they are summed and located on the Total Points axis. Then, draw a line straight downward from the Total Points axis to determine the patient’s calibrated oral health.

Figure 3.

As an example, an 85-year-old male patient receives 0.85 points for his age (determined by drawing a line from 85 on the Age axis straight upward to the Points scale). He had less than a high school education, an annual income less than $20,000, poor general health, used health care 10 times in the past year, and had a self-reported oral health score of 40. We multiplied the values of each of these covariates X_i by their corresponding regression coefficient β_i to obtain the adjusted points (on the top bar):

$$X_{\mathit{adjusted}-\mathit{po}int-i}={\beta }_i{X}_i$$

His self-reported oral health score of 40 contributes 12.8 points; the 10 times he used health care in the past years contributes 6.41 points; all the other variables contributed 0 points apiece (because they are baseline values). Therefore, the total points are 12.8+0.85+6.41=20.06. Summation of the total points and the intercept β₀ of the regression equation yield the calibrated value: $({\displaystyle \sum _{i=1}^p}{X}_{\mathit{adjusted}-\mathit{po}int-i})+{\beta }_0$. Thus, the calibrated oral health score for this person is 20.06+26.24=46.3, which is larger than his self-reported oral health score, therefore, this patient could underestimate his oral health. The calibrated oral health score 46.3 is slightly less than half that of the 100 representing perfect oral health. In general, since all oral health items were rescaled within the range of 0 to 100, a calibrated oral health score is a score on the 0 to 100% scale, where 0 and 100% indicate worst and perfect oral health status, respectively.

Discussion

Dental caries and periodontal disease continue to be major concerns for dentists and their patients. Given the high prevalence of dental and oral health problems, most people are affected by one or both of these conditions. Even with the high prevalence of oral health problems among people who receive dental care, a large proportion of the population does not receive even routine dental checkups—a potential public health problem that needs proper intervention. We systematically examined the complex relationship between self-reported and clinically determined oral health and found that self-report is more predictive of dental caries, but less so for periodontal disease. This agrees with a previous study by Gilbert (36), which concluded that people are usually unable to report signs and symptoms related to their periodontal conditions. Age, gender, education, ethnicity, and income are significantly related to oral health, as well as to the discrepancy between clinical and self-reported oral health. For example, older people in general have worse oral conditions, and they tend to overestimate their own oral health. Many previous studies have established the strong link between oral health and general health and quality of life (9, 10, 37–38). In contrast, some other studies reported weak (36) or unrelated relationships (39, 40). Oral health is an important component of general community health and overall well-being. For individuals, problems in the mouth can signal trouble in other parts of the body (41, 42). Our analysis confirms such a positive association between general health and oral health.

We performed model selection by adjusted R², and the best model was thus selected with an adjusted R² of 0.21, which is not uncommon for calibrating behavioral measures such as self-reported oral health (43, 44). Nevertheless, the low R² obtained implies that there might be some other (unmeasured) factors associated with patients’ clinical oral health status. To assess the predictive accuracy of our calibration model, we used ROC curves to characterize the sensitivity and specificity of the calibrated oral health for their performance of predicting clinically determined oral health. We used the model built from the observed data and then resubstituted the model back into the data to obtain predictions. The ROC curve and its summary measures tend to be optimistic in self-prediction because they indicate better accuracy than the actual model allowed in practice. In order to overcome this over-fitting, we used cross-validation and bootstrap methods to validate our calibrated model. Since optimism correction for over-fitting in the bootstrap-validated estimate of the AUC for calibrated oral health is ignorable, the bootstrap-validated estimate of the AUC is essentially the same as resubstitution estimate. The bootstrap-validated estimate of the AUC for calibrated oral health is also virtually identical to the five-fold cross-validation estimate. This confirms that with reasonably large data sets such as NHANES, k-fold cross-validation (leave-one-out validation as a special case when n=k) and bootstrap methods produce similar results (34). The calibration model obtained was validated by the bootstrap-validated estimate as well as by the five-fold cross-validation estimate of the AUC for calibrated oral health. The internal validity of our calibrated model was thus established.

A simple and accurate self-reported oral health measure that can be efficiently used in clinical settings would provide a major advance for clinicians and investigators. Such a tool would be inexpensive and practical. In addition, it could be used in resource-poor settings, where more expensive and complicated clinical examinations are not affordable or unavailable. The models we developed can be used to calibrate and predict different clinically determined oral health conditions, such as dental caries, periodontal disease, and overall oral health based on simple self-reported oral health and a subject’s characteristics. As the United States moves toward a national insurance program for medical care, it seems that dental care will not be included in the discussion, thus the challenge facing dental insurers, cash-strapped public programs and dental service providers will be to find ways of maximizing the limited resources available. The concept of developing calibration models that can utilize self-reported data to provide clinical estimates of oral health will be needed to rationalize the delivery of scarce dollars for dental care.

Using the calibrated model, an individual’s oral health score is calibrated to the clinically determined oral health. For a population, a quick oral health assessment can be obtained without a complex clinical examination. By assessing sensitivity, specificity and the area under ROC curves, we can characterize predictive accuracy of the calibrated model for its performance of predicting clinically determined oral health. This will then enable us to perform oral screening and evaluation in large populations, thereby saving resources and may even yield significant policy implications for dental care. We can target to identify high risk groups who overestimated their clinically determined oral health status, and enable planners to devote additional resources to engaging these high risk groups to provide them with diagnostic, preventive and restorative care. On the other hand, for the low risk groups who underestimated their oral health status, such as the example given in the result section, the resources distributed to them can be reduced.

However, the NHANES dental data also has some limitations. First, it has limited self-reported oral health measures. For example, it has no self-reported dental caries measure. Second, it has cross-sectional data only. Although NHANES has been conducted over years, it does not have repeated measures for a given individual. Therefore, it is not practical for evaluating the change in oral health over time. The proposed model cannot be considered as a replacement for clinical assessments; however it could be used as a tool for identification of high risk groups who overestimated their oral health status.

In summary, we evaluated the distribution of self-reported and clinically determined oral health as well as general overall health marginally and across patient social demographic characteristics. Given that NHANES offers the best quality of oral health data available, we have used it to create summarized clinically determined oral health measures that can objectively measure oral health. Through our initial study, we evaluated the discrepancies between self-reported and clinically determined oral health status, created optimal models for calibrating the self-reported oral health to clinically determined standards, assessed predictive accuracy of the calibrated model for its performance of predicting clinically determined oral health, and validated the calibration model using cross-validation and bootstrap methods. The next step is to test the efficacy of the selected optimal models that can calibrate and correlate self-reported oral health status to that of clinically determined standards, especially within larger populations. To ensure such efficacy, the proposed calibration models need further evaluation before they are applied in dental-screening programs for these populations.

Once these models have been evaluated and determined to be effective, they will be useful for dental screening and evaluation with large populations. Based on these models, calibration tools, such as nomograms and calibration tables, can be created. With such tools, health care providers can simply obtain self-reported oral health and other factors, and then get a calibrated oral health status close to the clinically determined standard. These models and tools can be used in federal, state, or local programs for oral health screening and evaluation with large populations, particularly underserved populations, to identify high risk groups. These tools will enable the health care community to save substantial effort and resources and address the need for dental care more efficiently. In addition, these models and tools have great potential to be used in resource-limited countries and areas.