Quality Appraisal of Child Oral Health–Related Quality of Life Measures: A Scoping Review

Abstract

Background:

Children’s oral health–related quality of life (COHQoL) measures are well known and widely used. However, rigorous systematic reviews of these measures and analyses of their quality are in absence.

Objectives:

To systematically review and quantitatively assess the quality of COHQoL measures through a scoping review.

Data Sources:

Systematic literature search of PubMed, CINAHL (Cumulative Index to Nursing and Allied Health Literature), EMBASE (Excerpta Medica database), HaPI (Health and Psychosocial Instruments), and DOSS (Dentistry and Oral Sciences Source).

Study Eligibility:

The measure’s focus was COHQoL; the child age ranged from 5 to 14 years; the publication was either a research article or a systematic review and related to caries; and it was written in English or had an English abstract. Two authors independently selected the studies. Disagreements were reconciled by group discussions with a third author.

Appraisal:

The International Society for Quality of Life Research minimum standards for patient-reported outcome measures were used for quality appraisal.

Synthesis:

Descriptive analysis.

Results:

We identified 18 measures. Their quality scores ranged from 9.5 to 15.0 on a scale of 16. The quality appeared to bear no relationship to the citation and use of these measures. However, elements of these measures might be more useful than others, depending on the age-specific use and primary quality concerns.

Limitations:

Some of the information on the minimum standards of the 18 measures cannot be found in the existing literature. Measures published without English abstract were not searched.

Conclusions:

The quality of these measures is suboptimal. Researchers and practitioners in this field should exercise caution when choosing and using these measures. Efforts at improving the quality of the COHQoL measures, such as refining existing ones or developing new measures, are warranted.

Knowledge Transfer Statement:

Researchers, clinician scientists, and clinicians can use the results of this study when deciding which oral health–related quality of life measure they wish to use in children.

Introduction

More than 2 decades ago, the World Health Organization (1995) set a standard for quality of life, defining it as “individuals’ perceptions of their position in life in context of the culture and value system in which they live.” Oral health–related quality of life (OHRQoL) refers to the role of oral conditions or diseases on quality of life. It is a construct spanning multiple domains, including oral symptoms, functional well-being, and socioemotional well-being (Broder et al. 2014). It plays an important role in oral health research (Kragt et al. 2017). Yet, there are continuing concerns about the quality of these measures (Gill and Feinstein 1994). In addition, there have been debates on whether a claimed quality of life measure actually measures this construct (Gill and Feinstein 1994; Guyatt and Cook 1994). In dentistry, a classical commentary on this topic is from Locker and Allen (2007), and it is one of the motivations for this review.

At least 7 studies have examined OHRQoL measures. They checked 11 measures (Slade et al. 1998), 8 measures (Allen 2003), 5 measures (Locker and Allen 2007), 20 measures (Hebling and Pereira 2007), 4 measures (Jones et al. 2004), 3 measures (Brennan 2013), and 6 measures (Haag et al. 2017). The first 4 had OHRQoL measures as the primary focus; the remaining 3 had it as the secondary focus. However, none of these studies specifically examined children’s OHRQoL (COHQoL). Only Locker and Allen (2007) assessed the Child Perceptions Questionnaire 11-14 (CPQ11-14), with the other 4 OHRQoL measures for adults. None provided a quantitative quality assessment. Genderson et al. (2013) and Gilchrist et al. (2014) both reviewed 6 measures focusing on COHQoL. Yet, neither included 2 measures: the Michigan Oral Health–Related Quality of Life Scales (Filstrup et al. 2003) and the Scale of Oral Health Outcomes for 5-Year-Old Children (Tsakos et al. 2012). Furthermore, neither provided quantitative assessment of the measures’ quality.

Given the absence of quantitative assessment of the quality of COHQoL measures, we conducted a scoping review (Peters et al. 2015) on this topic. The quantitative assessment of the quality of COHQoL measures adds to previous qualitative assessments by providing an objective assessment. We used the International Society for Quality of Life Research (ISOQOL) minimum standards set for patient-reported outcome measures (Reeve et al. 2013). We systematically searched, selected, and appraised the quality of COHQoL measures and made recommendations for improving the quality of these measures. Since untreated caries is the most prevalent disease in the world, with an age-standardized prevalence of 34.1% (Kassebaum et al. 2017), and has important and negative impacts on COHQoL (Onoriobe et al. 2014; de Paula et al. 2015), we limited the focus to caries.

Methods

We adopted the 5 stages of conducting a scoping review (Peters et al. 2015) for this study: 1) identifying the research questions; 2) identifying the relevant studies and COHQoL measures; 3) selecting studies and COHQoL measures; 4) charting the data by assessing the quality of COHQoL measures based on the ISOQOL minimum standards; and 5) collating, summarizing, and reporting the results, which focused on the answers to the research questions identified in stage 1.

Identifying the Research Questions

We examined 2 related questions: What are the available COHQoL measures? What is the quality of these measures based on the ISOQOL minimum standards?

Identifying the Relevant Studies and COHQoL Measures

We searched the following 5 databases: the PubMed, the CINAHL (Cumulative Index to Nursing and Allied Health Literature), the EMBASE (Excerpta Medica database), the HaPI (Health and Psychosocial Instruments), and the DOSS (Dentistry and Oral Sciences Source). Search terms were platform-specific combinations of keywords and subject headings (MeSH/EMTREE). The search period was through January 1, 2018, for all databases, with no lower publication date limit. No restrictions were applied regarding publication type. A detailed PubMed search strategy is provided in Appendix 1. Conceptually, the PubMed search centered on surveys and questionnaires for OHRQoL specifically as they relate to children.

Selecting Studies and COHQoL Measures

The inclusion criteria were as follows: the measure’s focus was COHQoL; the child age ranged from 5 y (kindergarten) to 14 y (eighth grade); the publication was either a primary research article or a systematic review of COHQoL measures; the publication was related to caries; and the article was written in English or had an English abstract. The exclusion criteria were as follows: studies were reporting measures that examined concepts other than COHQoL, such as oral health status or utility, were focused on populations outside the included age range (e.g., <5 or >14), were disease or condition specific (e.g., malocclusion, cleft) or population specific (e.g., children with special needs), and were not in English or had no English abstract available. Two authors (C.Y. and Y.O.C.) screened the title, abstract, and then the full text of publications that met the inclusion criteria. Disagreements were reconciled by group discussions with a third author (R.N.).

Charting the Data

The ISOQOL minimum standards (Reeve et al. 2013) has 6 categories: 1) conceptual and measurement model, 2) reliability, 3) validity, 4) interpretability of scores, 5) translation of the patient-reported outcome measures, and 6) patient and investigator burden. Each category has subcategories, for a total of 16 criteria. We scored each identified COHQoL measure based on the ISOQOL minimum standards. For each criterion, we used the following scoring system: 1 = “Yes, the criterion was met,” 0.5 = “Cannot tell if the criterion was met or not or information missing,” and 0 = “No, the criterion was not met.” The summary scores of the 6 categories had different ranges (0 to 1, 0 to 5) due to the different number of items (1 to 5). The total ISOQOL score of a given measure ranged from 0 to 16. For all categories and criteria, higher scores indicated better quality.

An expanded explanation of the ISOQOL minimum standards is listed in Appendix 2. It shows that the ISOQOL criteria are more than measurement properties. For example, ISOQOL standard 5, translation, usually is not assessed in the published development studies. Therefore, additional studies are needed to find information about the ISOQOL criteria, instead of only the published development studies.

Collating, Summarizing, and Reporting the Results

Basic demographics from each study included name, acronym, original reference, target age, target population, completion method, development information (country, condition, and sample size), number of domains, number of items, and item allocation (how many items are allocated to each domain). We also counted how many times each measure was used in the included studies. In addition, Google Scholar (https://scholar.google.com/) citation information of the original article was retrieved and recorded for both the total number of citations and the average number of citations per year. Although not an indicator of quality, the Google Scholar citation can be a good indicator of a measure’s popularity. However, the popularity of COHQoL measures is not the emphasis of this review, because it is quite possible that investigators are using these measures without due regard to their quality. The emphasis of this review is the quality of the COHQoL measures, as indicated by their ISOQOL scores. We sorted the measures by their targeted ages, grouped their ISOQOL scores by the 6 categories, and calculated a sub-summary within each category. We used the total ISOQOL scores to assess the overall quality of a COHQoL measure and the sub-scores to assess the quality of subcategories. We explored the potential relationship between the ISOQOL scores and the sample size of the original sample for measure development. Finally, using scatter plots, we examined the relationship between the ISOQOL score of a COHQoL measure and the Google Scholar citation of its original article.

Results

Search Results

From the 5 databases, we identified 2,093 articles (Fig. 1). We removed 942 duplicates, leaving 1,151 for screening. We excluded 842, leaving 309 for COHQoL full-text inspection. From the 309 articles, we identified 18 measures that directly addressed COHQoL.

Figure 1.

Flowchart of article selection and measure identification. n = number of articles/measures. CINAHL, Cumulative Index to Nursing and Allied Health Literature; COHQoL, children’s oral health–related quality of life; DOSS, Dentistry and Oral Sciences Source; EMBASE, Excerpta Medica database; HaPI, Health and Psychosocial Instruments.

Demographics of the COHQoL Measures

Demographic information of the 18 COHQoL measures was summarized and sorted by target age (Table 1). These 18 measures targeted 5 age groups: preschool (5 measures), 8 to 10 y (2 measures), 11 to 14 y (4 measures), 13 to 18 y (1 measure), and broad ages (6 measures). Except for the ISCII-OHRQoL (Chen and Hunter 1996), all other 17 measures were developed after 2000: the CPQ11-14 (Jokovic et al. 2002), the PPQ (Jokovic et al. 2003), the MOHRQoL-C and the MOHRQoL-PG (Filstrup et al. 2003), the CPQ8-10 (Jokovic et al. 2004), the COIDP (Gherunpong et al. 2004), the CPQ11-14 SF (Jokovic et al. 2006), the ECOHIS (Pahel et al. 2007), the COHIP (Broder et al. 2007; Broder and Wilson-Genderson 2007), the POQL-P and the POQL-C (Huntington et al. 2011), the CDPQ (Barretto et al. 2011), the COHIP–SF 19 (Broder et al. 2012), the SOHO-5 (Tsakos et al. 2012), the COHRQoL-25 (Molek et al. 2016), the TOQOL (Wright et al. 2017), and the COHIP-PS (Ruff et al. 2017).

Table 1.

Demographics of the Identified Children’s Oral Health–Related Quality-of-Life Measures.

Target Age, y	Name	Original Reference: First Author and Year	Completion Method	Original Sample for Measure Development			Number of Domains	Number of Items	Item Distribution among Domains	Google Scholar Citation of the Original Reference
				Country	Conditions / Populations of Interest	Size, n				Total	Average per Year
2 to 5	COHIP-PS	Ruff 2017	Survey to parent	USA	Cleft lip/palate, speech therapy, routine pediatric dental care, and communities	327	4	10	3-2-2-3	0	0.0
5	ECOHIS	Pahel 2007	Survey to parent	USA	Children from high- and low-income counties	341	6	13	1-4-2-2-2-2	342	34.2
5	SOHO-5	Tsakos 2012	Interview with child	UK	Schoolchildren	296	1	7	7	62	12.4
2 to 6a	MOHRQoL-C	Filstrup 2003	Survey to child	USA	Children with or without ECC	112	4	9	3-2-2-2	384	27.4
2 to 6a	MOHRQoL-PG	Filstrup 2003	Survey to parent or guardian	USA	Same as MOHRQoL-C	112	2	10	5-5	384	27.4
8 to 9	CDPQ	Barretto 2011	Interview with child	Brazil	Caries	174	3	6	2-2-2	9	1.5
8 to 10	CPQ8-10	Jokovic 2004	Survey to child	Canada	Pediatric dentistry, orofacial	101	4	25	5-5-5-10	225	17.3
11 to 12	COIDP	Gherunpong 2004	Interview with child	Thailand	Schoolchildren	1,613	1	8	8	338	26.0
12 to 13	ISCII-OHRQoL	Chen 1996	Survey to child	New Zealand	Schoolchildren	1,074	3	14	8-2-4	195	9.3
11 to 14	CPQ11-14	Jokovic 2002	Survey to child	Canada	Pedodontic, orthodontic, and orofacial	206	4	37	6-9-9-13	593	39.5
11 to 14	CPQ11-14 SF	Jokovic 2006	Survey to child	Canada	Same as CPQ11-14	206	4	8 or 16	2-2-2-2 or 4-4-4-4	174	15.8
13 to 18	TOQOL	Wright 2017	Survey to child	USA	High schools, dental clinics, and community	363	5	16	3-3-3-3-4	1	1.0
6 to 12	COHRQoL-25	Molek 2016	Survey to child	Indonesia	Schoolchildren	434	5	25	5-5-5-5-5	0	0.0
6 to 14	PPQ	Jokovic 2003	Interview with parent	Canada	Pedodontic, orthodontic, orofacial	439	4	31	6-8-7-10	189	13.5
8 to 15	COHIP	Broder 2007	Survey to child	USA and Canada	Pediatric dental, orthodontic, craniofacial, community	1,380	5	34	10-6-8-4-6	147	14.7
2 to 16	POQL-P	Huntington 2011	Survey to parent	USA	Schoolchildren, dental clinics, and ECC intervention	1,158	4	10	2-2-3-3	42	7.0
8 to 16	POQL-C	Huntington 2011	Survey to child	USA	Same as POQL-P	1,821	4	10	2-2-3-3	42	7.0
7 to 18	COHIP-SF 19	Broder 2012	Survey to child	USA	Same as COHIP	1,175	3	19	5-4-10	26	5.2

CDPQ, Child Dental Pain Questionnaire; COHIP, Child Oral Health Impact Profile; COHIP-PS, Child Oral Health Impact Profile–Preschool; COHIP–SF 19, Child Oral Health Impact Profile Short Forms; COHRQoL, Child Oral Health–Related Quality of Life; COIDP, Child Oral Impacts on Daily Performances; CPQ8-10, Child Perceptions Questionnaire (8-10); CPQ11-14, Child Perceptions Questionnaire (11-14); CPQ11-14 SF, Child Perceptions Questionnaire (11-14) Short Forms; ECC, early childhood caries; ECOHIS, Early Childhood Oral Health Impact Scale; ISCII-OHRQoL, the Second International Collaborative Study OHRQoL; MOHRQoL-C, Michigan Oral Health–Related Quality of Life Scales–Child version; MOHRQoL-PG, Michigan Oral Health–Related Quality of Life Scales–Parent/Guardian version; POQL-C, Pediatric Oral Health–Related Quality of Life–Child Self-report; POQL-P, Pediatric Oral Health–Related Quality of Life–Parent Report on Child; PPQ, Parent Perception Questionnaire; SOHO-5, Scale of Oral Health Outcomes for 5-year-old children; TOQOL, Teen Oral Health–Related Quality of Life instrument.

^aMOHRQoL-C and MOHRQoL-PG have versions for children aged 4 to 16 y (Filstrup et al. 2003).

Of the 18 measures, 13 were designed for direct child responses, and 5 were designed for indirect caregiver responses (parent or guardian). Thirteen were developed in North America (United States and Canada) and 5 in other countries (United Kingdom, Indonesia, Brazil, Thailand, and New Zealand). The sample size ranged from 101 (CPQ8-10) to 1,821 (POQL-C). The number of items ranged from 7 (SOHO-5) to 37 (CPQ11-14). The number of domains ranged from 1 (SOHO-5, COIDP) to 6 (ECOHIS). Within each domain, the number of items ranged from 1 (the first domain of the ECOHIS) to 13 (the last domain of the CPQ11-14). As shown in Appendix 3, the 18 measures had different domains. The most common domains across the 18 measures were symptoms, functioning, physical, emotional, and social well-being. In addition, the CPQ11-14 was the most frequently used measure (84 times in the 309 included studies), while the COHIP-PS and the COHRQoL-25 were both used only once.

ISOQOL Scores of the COHQoL Measures

We scored each COHQoL measure according to the ISOQOL minimum standards (Tables 2), based on related information retrieved from the inspected articles.

Table 2.

Scoring COHQoL Measures Based on the ISOQOL Minimum Standards.

Namea	1. Conceptual and Measurement Model (0 to 5; Average 3.6, 72%)					2. Reliability (0 to 2; Average 0.9, 45%)		3. Validity (0 to 5; Average 3.9, 78%)					4. Interpret-ability (0 to 1; Average 0.8, 80%)	5. Trans-lation (0 to 1; Average 0.8, 80%)	6. Burden (0 to 2; Average 1.3, 65%)		Total Score (0 to 16; Average 11.4, 71.3%)
	1 a	1 b	1c	1 d	1 e	2a	2 b	3a1	3 a2	3 a3	3 b	3 c			6 a	6 b
COHIP_PS	1.0	1.0	1.0	0.5	0.5	0.5	0.5	1.0	0.5	0.0	1.0	0.5	1.0	0.5	1.0	1.0	11.5
ECOHIS	1.0	1.0	0.0	0.5	0.5	0.5	1.0	1.0	1.0	0.0	1.0	1.0	1.0	1.0	1.0	1.0	12.5
SOHO-5	1.0	1.0	1.0	1.0	0.5	0.5	1.0	1.0	1.0	1.0	1.0	1.0	1.0	1.0	1.0	1.0	15.0
MOHRQoL-Cb	1.0	1.0	0.5	0.5	0.5	0.5	0.5	1.0	1.0	0.5	1.0	1.0	1.0	0.5	1.0	1.0	12.5
MOHRQoL-PGb	1.0	1.0	1.0	0.5	0.5	0.5	1.0	0.5	1.0	0.5	1.0	1.0	1.0	0.5	1.0	1.0	13.0
CDPQ	1.0	1.0	0.5	0.5	0.5	1.0	1.0	0.0	0.5	0.0	1.0	0.5	1.0	0.5	0.5	0.5	10.0
CPQ8-10	1.0	1.0	0.0	0.5	0.5	0.0	0.0	1.0	1.0	1.0	1.0	1.0	0.5	1.0	0.5	1.0	11.0
COIDP	1.0	1.0	0.0	0.0	0.5	1.0	0.0	1.0	1.0	1.0	1.0	1.0	1.0	1.0	0.5	0.0	11.0
ISCII-OHRQoL	1.0	1.0	0.5	0.5	0.5	0.5	0.0	1.0	1.0	0.0	1.0	0.5	1.0	1.0	1.0	0.0	10.5
CPQ11-14	1.0	1.0	0.0	0.5	0.5	1.0	0.0	1.0	1.0	0.5	1.0	1.0	0.0	1.0	0.5	0.0	10.0
CPQ11-14 SF	1.0	1.0	0.0	1.0	0.5	1.0	0.5	1.0	1.0	0.5	1.0	0.5	0.0	1.0	0.5	0.0	10.5
TOQOL	1.0	1.0	1.0	1.0	0.5	0.5	1.0	1.0	1.0	0.0	1.0	0.5	1.0	0.5	0.5	0.0	11.5
COHRQoL-25	1.0	1.0	0.5	1.0	0.5	0.5	0.5	1.0	0.5	1.0	0.5	0.5	0.0	0.5	1.0	0.0	10.0
PPQ	1.0	1.0	0.0	0.5	0.5	0.0	0.0	1.0	1.0	0.5	1.0	1.0	0.0	1.0	1.0	0.0	9.5
COHIP	1.0	1.0	0.0	0.5	0.5	0.0	0.0	1.0	1.0	0.0	1.0	0.5	1.0	1.0	1.0	0.0	9.5
POQL-P	1.0	1.0	1.0	1.0	1.0	0.5	0.0	1.0	1.0	0.5	1.0	1.0	1.0	1.0	1.0	1.0	14.0
POQL-C	1.0	1.0	1.0	1.0	1.0	0.0	0.0	1.0	1.0	0.5	1.0	0.5	1.0	0.5	1.0	1.0	12.5
COHIP-SF 19	1.0	1.0	0.0	0.5	0.5	0.5	0.5	1.0	1.0	0.0	1.0	0.5	1.0	1.0	1.0	0.0	10.5
Average	1.0	1.0	0.4	0.6	0.6	0.5	0.4	0.9	0.9	0.4	.97	0.7	0.8	0.8	0.8	0.5	11.4

ISOQOL scores of the COHQoL measures at the item, category, and overall level of the ISOQOL standards. Scoring system: “1 = Yes, the criterion was met”; “0.5 = Cannot tell if the criterion was met or not / Information missing”; “0 = No, the criterion was not met.” The ISOQOL minimum standards: 1a, conceptual model; 1b, target population; 1c, dimensionality evidence; 1d, item-domain correlation; 1e, inter-domain correlation; 2a, test-retest reliability; 2b, internal consistency; 3a1, qualitative/quantitative methods; 3a2, empirical studies; 3a3, recall period justification; 3b, construct validity; 3c, responsiveness.

COHQoL, child oral health–related quality of life; ISOQOL, International Society for Quality of Life Research.

^aSee Table 1 for acronyms.

^bMOHRQoL-C and MOHRQoL-PG have versions for children aged 4 to 16 y (Filstrup et al. 2003).

For the 16 ISOQOL minimum standards at the criterion level, the standards of conceptual model (standard 1a) and target population (standard 1b) were met by all 18 COHQoL measures (all scored 1). Several other standards were met by the majority of the COHQoL measures. These included construct validity (standard 3b, averaged 0.97), qualitative/quantitative methods (standard 3a1, averaged 0.9), and empirical studies (standard 3a2, averaged 0.9). However, the standards of dimensionality evidence (standard 1c), internal consistency reliability (standard 2b), and recall period justification (standard 3a3) were not satisfactorily met (all averaged 0.4).

To compare the 6 categories of the ISOQOL minimum standards, we used the percentage of the average score out of the total score. For example, category 1, conceptual and measurement model, averaged 3.6 out 5.0; therefore, it scored 72% (3.6 of 5.0). Based on this approach, interpretability (80%), translation (80%), validity (78%), conceptual and measurement model (72%), and burden (65%) averaged at least half (50%) of the highest possible score. But reliability averaged only 45% of the highest possible score.

Among the 18 COHQoL measures, the total ISOQOL scores ranged from 9.5 (59%) to 15.0 (94%) out of 16, with an average of 11.4 (71%). The 3 measures that scored highest were the SOHO-5 (15.0), the POQL-P (14.0), and the MOHRQoL-PG (13.0). The measures that scored lowest were the PPQ (9.5) and the COHIP (9.5).

For the preschool age group, the SOHO-5 is the best (scored 15.0), followed by the MOHRQoL-PG (scored 13.0); for the 8- to 10-y group, the CPQ8-10 is the best (scored 11.0); for the 11- to 14-y group, the COIDP is the best (scored 11.0); for the 13- to 18-y group, only the TOQOL is available, and it scored 11.5; for the mixed age group, the POQL-P is the best (scored 14.0).

Table 3 reports the Spearman correlation coefficients between the ISOQOL scores and the size of the original sample for development of the 18 measures. The coefficients ranged from −0.462 to 0.257, but none of them was statistically significant (all with P values >0.05).

Table 3.

Spearman Correlation Coefficients between the ISOQOL Scores and the Size of the Original Sample for Development of the 18 Measures.

	Conceptual and Measurement Model	Reliability	Validity	Inter-pretability	Translation	Burden	Total ISOQOL Score
Spearman correlation coefficient with sample size	−0.008	−0.462	−0.133	0.257	0.231	−0.155	−0.148
P value	0.975	0.053	0.598	0.303	0.357	0.539	0.558

ISOQOL, International Society for Quality of Life Research.

Relationship between ISOQOL Score and Citations

The total Google Scholar citations of the original reference ranged from 0 (COHRQoL-25 and COHIP-PS) to 593 (CPQ11-14), with the average annual citation ranging from 0 to 39.5 (CPQ11-14). Through scatter plots, Figure 2 portrays the relationship between a measure’s ISOQOL score and the Google Scholar citations of its original reference. Although there seemed to be a positive relationship between these properties, clearly there were some outliers. For example, in the average citation plot, the POQL-P scored high on ISOQOL minimum standards (14.0) but received low citations (7.0/y). On the opposite, the CPQ11-14 scored low on the ISOQOL minimum standards (10.0) but received high citations (39.5/y).

Figure 2.

Scatter plot of ISOQOL score^a vs. citation^b of the COHQoL measures.

^aCOHQoL measure’s total score based on the ISOQOL standards. ^bGoogle Scholar citations of the original reference of the COHQoL measure. COHQoL, children’s oral health–related quality of life; ISOQOL, International Society for Quality of Life Research.

Discussion

Patient-centered research and patient-reported outcomes are central to understanding a patient’s perception of care (Frank et al. 2014). Consequently, numerous authors have spent significant time developing and validating COHQoL measures. OHRQoL is the most important patient-reported outcome in oral health research (Kragt et al. 2017). Moreover, an accurate quality-of-life measure is a vital component of quality-adjusted life-years (La Puma and Lawlor 1990), a health economic concept. The expectation, therefore, is that COHQoL measures will be of high quality (Teresi and Fleishman 2007). However, development of COHQoL measures is extremely difficult given the immature and ever-changing cognitive abilities of children (Broder et al. 2012). In part, as a consequence of this need, the National Institutes of Health supports 57 funded projects related to COHQoL.

Here we systematically address the quality of COHQoL measures. We identified 309 COHQoL-related articles and 18 COHQoL measures from 5 databases. It is now clear that although all were claimed as COHQoL measures, they are indeed measuring various domains (Appendix 3). This echoes the classical debates made by Gill and Feinstein and by Guyatt and Cook, both in 1994, on whether a claimed quality-of-life measure actually measures this construct. Further investigation on this fundamental point is needed.

We found that, in general, these COHQoL measures performed well on the ISOQOL standards, averaging 11.4 out of 16 (71%). However, some measures and subcategories are suboptimal. For example, reliability averaged 0.9 out of 2.0 (45%). It is surprising that the size of the original sample for development of the 18 measures does not correlate with their performance on the ISOQOL standards (Table 4). A potential reason for this discrepancy is the methodological issues in measure development. For example, principal component analysis, instead of factor analysis, was commonly used in the development of some measures. However, principal component analysis is conceptually inappropriate for measures (Zumbo 2006). Although, in general, measures that scored higher on ISOQOL minimum standards also received higher citations, a few highly cited measures scored relatively low.

It is noted that the 18 COHQoL measures were developed under different oral conditions or populations of interest (Table 1). This inconsistency may have an influence on the performances of the measures against the ISOQOL standards. Future studies toward this direction are needed.

Of the 18 measures, 15 originated from developed countries (United States, Canada, United Kingdom, and New Zealand). Only 3 were from developing countries (Thailand, Indonesia, and Brazil). This has implications to global generalizability. When translating and adapting the 15 measures from developed to developing countries, we must consider the influence of cultural aspects on quality of life.

Of critical importance is that suboptimal measures may have unwanted consequences. One classic example, from outside of oral health, is the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS; Randolph et al. 1998). Despite its popularity, the RBANS is suboptimal (Carlozzi et al. 2008; Yang et al. 2009). As a result, the RBANS led to false-negative outcomes (Yang and Vrana 2018). Similarly, in oral health, suboptimal measures can lead to false-negative and false-positive findings. This implies that researchers and policy makers in the field of COHQoL should be cautious when selecting these measures. If they adopt suboptimal COHQoL measures in their research or practices, they surely will be misled. They will face the “garbage in, garbage out” situation, where the flawed input data produce flawed output, results, and conclusions. Obviously, refining these suboptimal COHQoL measures or even developing new ones is greatly needed. Future work in this direction is now warranted.

There are some limitations in our study. Some of the information on the 16 ISOQOL minimum standards of the 18 COHQoL measures cannot be found in the existing literature. These missing values are coded as 0.5 in our scoring algorithm. We expect that the original authors and their colleagues will provide these data over time. This is especially true for ISOQOL minimum standard 3c, responsiveness. It usually requires a clinical trial or longitudinal study to assess responsiveness of measures. However, many COHQoL measures have been used in only cross-sectional studies. Second, COHQoL measures published in a language other than English were not searched. With additional efforts, these measures can be identified and appraised in the future.

Conclusions

Currently there are 18 COHQoL measures for children aged 2 to 18 y. These measures score 9.5 to 15.0 on the 16 ISOQOL minimum standards. Some highly cited COHQoL measures show suboptimal quality. Researchers and practitioners in the COHQoL field should exercise caution when choosing and using these measures. Efforts at improving the quality of the COHQoL measures, such as refining existing ones or developing new measures, are warranted.

Author Contributions

C. Yang, contributed to conception, design, data acquisition, analysis, and interpretation, drafted and critically revised the manuscript; Y.O. Crystal, contributed to data analysis and interpretation, critically revised the manuscript; R.R. Ruff, A. Veitz-Keenan, contributed to data interpretation, critically revised the manuscript; R.C. McGowan, contributed to data acquisition, drafted and critically revised the manuscript; R. Niederman, contributed to conception, data acquisition, and interpretation, critically revised the manuscript. All authors gave final approval and agree to be accountable for all aspects of the work.