Developing a Standard Set of Patient-centred Outcomes for Adult Oral Health – An International, Cross-disciplinary Consensus

Abstract

Objective

To develop a minimum Adult Oral Health Standard Set (AOHSS) for use in clinical practice, research, advocacy and population health.

Materials and methods

An international oral health working group (OHWG) was established, of patient advocates, researchers, clinicians and public health experts to develop an AOHSS. PubMed was searched for oral health clinical and patient-reported measures and case-mix variables related to caries and periodontal disease. The selected patient-reported outcome measures focused on general oral health, and oral health-related quality of life tools. A consensus was reached via Delphi with parallel consultation of subject matter content experts. Finally, comments and input were elicited from oral health stakeholders globally, including patients/consumers.

Results

The literature search yielded 1,453 results. After inclusion/exclusion criteria, 959 abstracts generated potential outcomes and case-mix variables. Delphi rounds resulted in a consensus-based selection of 80 individual items capturing 31 outcome and case-mix concepts. Global reviews generated 347 responses from 87 countries, and the patient/consumer validation survey elicited 129 responses. This AOHSS includes 25 items directed towards patients (including demographics, the impact of their oral health on oral function, a record of pain and oral hygiene practices, and financial implications of care) and items for clinicians to complete, including medical history, a record of caries and periodontal disease activity, and types of dental treatment delivered.

Conclusion

In conclusion, utilising a robust methodology, a standardised core set of oral health outcome measures for adults, with a particular emphasis on caries and periodontal disease, was developed.

INTRODUCTION

Oral diseases directly impact the lives of individuals by causing considerable pain and suffering, altering food choices, affecting speech, self-esteem, quality of life, and participation in everyday activities^1,2. Measuring the impact of oral diseases has traditionally been based upon the biomedical model that provides only limited insight into the impact of oral disease on people's lives³. The emerging patient-centred care model⁴ necessitates a focus on oral health rather than oral disease. Although measures of oral health and oral health-related quality of life have been developed, they have not been shown to be useful in all of the important domains of clinical practice, health services research, epidemiology and advocacy. This is reflected in recent definitions of oral health that now include physical, psychological, emotional and social domains, which are core to overall health and wellbeing5, 6, 7. From the patient-centred care perspective, oral healthcare providers should thus consider not only disease processes, but also the environmental, social and personal factors, overall quality of life and participation in all major life areas, including making decisions about and control over their health and the use of health services⁸. This approach is reflected in the FDI World Dental Federation (FDI) definition of oral health⁶ (Figure 1), and provides a theoretical framework for shared decision-making in clinical practice, as well as for health services research, epidemiology and advocacy.

Figure 1.

FDI oral health framework.

The ability to measure oral health outcomes from both the clinician and patient/consumer perspective in a simple reproducible manner is fundamental to the principle of value-based oral healthcare. This takes into account the achievement of health outcomes that matter to patients and the healthcare costs incurred in achieving those9, 10, 11, and is a central premise of the International Consortium for Health Outcomes Measurement (ICHOM). Defining a minimum standard or core set of health outcome measures that are meaningful to both patients and clinicians is a necessary first step towards assessment of the value of care provided. Accordingly, ICHOM has thus far developed 27 core outcome standard sets for a range of human diseases and conditions. Each standard set developed by ICHOM has the following components: (i) patient-centred outcomes; (ii) case-mix variables (which are factors that will affect the patient-centred outcomes that are not influenced by the management of the condition and are used to allow the construction of risk-adjustment models); (iii) validated instruments that can be used for the outcomes and case-mix variables; (iv) clinician- or patient-reported data sources; (v) specific time-points for data collection. Based on case studies of the implementation of these standard sets in clinical care, the benefits included fewer delays in clinical practice, greater efficiency in administrative processes and consultations with a greater focus on what mattered most to the patients. Patients reported that they were more engaged in their own care, it ‘gives us more of a role that makes the appointment more tailored to what is important to us’; while clinicians reported that the routine use of these standard sets resulted in the consultations being ‘far more focussed’ with a better structure to their patient inter-actions¹².

The key objective of the research reported here is the development of a minimum Adult Oral Health Standard Set (AOHSS) of outcome measures, produced through a collaboration between the FDI and ICHOM, for use in routine clinical practice, research, advocacy and population health. Reflecting their predominance in routine clinical practice, caries and periodontal disease are the principal focus among clinical conditions for this standard set.

METHODS

An international Oral Health Working Group (OHWG) of 22 members comprised of patient advocates, researchers, clinicians, policymakers and public health experts, representing 10 countries, was established to ensure diversity in subject matter expertise, sex and geographic location (Table 1, OHWG membership). This membership unified the discipline-specific expertise of FDI with the strong methodological expertise of ICHOM.

Table 1.

OHWG

Initials	Country	Affiliation	Dental speciality
RNR	(1) Ireland	(1) University College Cork	Oral Medicine
	(2) UK	(2) University College London
MG	USA	University at Buffalo, School of Dental Medicine	Oral Medicine
SSAA	UAE	Dubai Health Authority	Dental Public Health
KA	USA	American Dental Association
JB	USA	Harvard School of Dental Medicine	Oral Health Policy and Epidemiology
DC	Australia	Dental Health Services Victoria	Public Oral Health Service Provider
JJC	USA	University of California Los Angeles	Paediatric Dentistry
JEG	UK	Kings College London	Dental Public Health
JG	Australia	Patient Advocate
SH	Australia	Dental Health Services Victoria	Value-based Oral Healthcare
RK	Australia	Patient Advocate
EK	USA	University of California San Francisco	Preventive & Restorative Dentistry
AK	UK	Public Health Wales	Dental Public Health
RKC	Brazil	Federal University of Rio Grande do Sul	Epidemiology
SL	(1) Netherlands	(1) Radboud University Medical Centre	Quality Improvement, Health Economics,
	(2) Germany	(2) Heidelberg University Hospital	Preventive & Restorative Dentistry
SNM	USA	International Consortium for Health Outcome Measurement
RN	USA	New York University	Epidemiology and Health Promotion
TS	Switzerland	FDI World Dental Federation
MWS	Australia	Hospital Contribution Fund
WMT	New Zealand	University of Otago	Dental epidemiology and public health
GT	UK	University College London	Dental Public Health
MV	USA	Health Policy Institute - American Dental Association
RGW	UK	University College London	Dental Public Health
SW	USA	International Consortium for Health Outcome Measurement
DMW	UK	Bart's and The London School of Medicine and Dentistry, Queen Mary, University of London	Global Oral Health

The study included the following phases.

Phase 1: Selection of measures for AOHSS

According to the WHO, caries and periodontal diseases are considered the most important global oral health burdens¹³. The AOHSS focussed therefore on caries and periodontal disease in adults. A literature search was undertaken in PubMed to identify relevant oral health outcomes and case-mix variables published in the previous 10 years. Case-mix variables are used in value-based health to provide a contextual picture of a patient cohort.

The search terms and inclusion/exclusion criteria are presented in Table 2. Duplicate articles were excluded. Two reviewers, who had been calibrated to ensure consistency, reviewed all remaining titles and abstracts independently. After reviewing the first 50 abstracts, a Cohen's kappa coefficient was calculated to determine inter-rater agreement on the application of the inclusion and exclusion criteria. This iterative process was repeated until a kappa of > 0.7 was achieved; the remaining abstracts were then distributed between the two reviewers to determine inclusion and subsequent data extraction. Supplemental literature, including public health surveys and clinical guidelines, were also reviewed.

Table 2.

Search terms, inclusion/exclusion criteria and search results

PubMed Search Terms

(caries OR carious OR cavity OR decay OR demineralisation OR prophylaxis OR prevention OR restoration) (periodontal disease OR periodontitis OR gingivitis OR prophylaxis OR prevention OR scaling OR root planing OR curettage OR surgical flap) AND ("Quality of Life"[Mesh] OR "Quality Indicators, Health Care"[Mesh] OR "Patient Outcome Assessment"[Mesh] "Treatment Outcome"[Mesh])

Inclusion criteria

Adult population (> 18 years) with dental caries or periodontal disease, English language abstracts from 2006 onwards of studies including systematic reviews and meta analyses, randomised controlled trials, observational studies, case series and guidelines

Exclusion criteria

Studies focusing on other dental diseases, paediatric populations, study protocols, case reports, microbiological outcomes, non-English language and irretrievable material PubMed Search Results

Patient-Reported Outcome Measurement (PROM) instruments, including the Oral Health Impact Profile-14 (OHIP-14)^14,15, Oral Impacts on Dental Performance (OIDP)^16,17, General Oral Health Assessment Index (GOHAI)¹⁸ and OHQoL-UK¹⁹, were reviewed by the OHWG with respect to their coverage of patient-reported outcomes, psychometric properties, content, validity and reliability, availability of language translations, and applicability of copyright/licensing fees.

A patient advisory group was organised in the USA to record response frequency of previously identified domain concepts, as well as take notes on newly raised concepts in real time. This patient insight was gathered via a focus group held in a non-clinical setting in Boston Massachusetts, USA. Eight patients, with fluency in English and who had undergone dental treatment in the last year, participated in the focus group moderated by an experienced qualitative researcher with no expertise in oral health (Figure 2, Patient advisory group demographics).

Fig 2.

Patient advisory group demographics.

Phase 2: ICHOM consensus-driven modified Delphi technique to develop the AOHSS

A structured, consensus-driven modified Delphi technique^20,21 was used that included a combination of videoconferences and surveys to select the AOHSS measures and case-mix variables (Figure 3). Five face-to-face meetings took place in London (twice), San Francisco and Geneva (twice), and eight videoconferences were held (January–November 2017), five of which were followed by surveys to inform decisions.

Fig 3.

Delphi process in the development of Adult Oral Health Standard Set (AOHSS).

Videoconferences were preceded as needed by breakout group meetings with subject matter experts on specific areas, including PROMs, clinical measures for caries and periodontal disease, case-mix, and data collection. Following initial presentation of material for decision at each videoconference (Figure 2: Calls 1–8), OHWG members voted electronically on the importance of the proposed items. The consensus process required at least an 80% response rate from the OHWG on every survey before data could be analysed. OHWG members were instructed to vote only on items for inclusion that met all of the following criteria: were important to adult patients with caries and/or periodontal disease; represented end-results of care and not the process of care; were feasible to measure accurately, and were modifiable with quality improvement efforts. Items were ranked on a 1–9 scale: where 1 represented an item deemed non-essential; and 9 an item considered essential. Items ranked 1–3 by 80% of respondents were excluded; items ranked 7–9 by 80% of respondents were included. Items ranked 4–6 were re-voted after further discussion²². If the vote was still inconclusive, a third voting round was undertaken where a simple majority determined inclusion or exclusion.

To ensure that the terminology in relation to diagnosis and disease staging used in the clinical measures of disease was consistent with current concepts, subject matter experts in the fields of cariology and periodontal disease were consulted. Outcome measures for both of these were based on input from content experts involved in generating and publishing the most recent guidelines on management and classification of the respective diseases^23,24.

Phase 3: Global open review to determine the appropriateness and feasibility of implementing the AOHSS

The worldwide open review comprised an online structured survey, in English, pertaining to the AOHSS draft. It was distributed among oral health professionals, educators, policymakers, researchers, industry representatives and national dental association members, building on the established networks of both ICHOM and FDI. Health professionals were encouraged to circulate the AOHSS draft within their networks, which included clinicians, policy advisors, payers, educators and researchers, to facilitate snowball sampling. Online platforms including Twitter and LinkedIn were used to share the survey links. The original English language version of the AOHSS was translated into German, French, Spanish and Arabic, and distributed worldwide to determine its acceptability, coverage of the minimum relevant concepts, and appropriateness. Health professionals were additionally asked to consider utility and feasibility. The findings from the global open review were used to inform decisions by the OHWG on outcomes and case-mix factors for inclusion in the final AOHSS, as well as the corresponding measures recommended to capture them.

Phase 4: Patient/consumer survey to determine appropriateness of AOHSS from patient/consumer perspective

To confirm the acceptability, relevance, appropriateness of the questions selected in the draft AOHSS, and to provide an initial insight into their face validity, an online survey with patients/consumers was conducted in Australia and USA. These patients/consumers, recruited via ICOHM in USA and Dental Health Services Victoria in Australia, had adequate online and English language literacy, and had undergone dental treatment in the last year. Ethics approval and exemption determination were obtained for Australia (Austin Health HREC-EC00204) and USA, respectively.

RESULTS

Phase 1: Selection of measures for AOHSS

The literature search yielded 1,453 results, yielding 959 relevant outcome and case-mix abstracts.

Eight patients participated in a focus group moderated by an experienced qualitative researcher with no expertise in oral health (Figure 2, Patient advisory group demographics). The focus group was digitally recorded, transcribed and anonymised. The patient advisory group endorsed the concepts identified from the literature. However, they also identified the financial impact of oral healthcare on individuals as an impact to be considered for inclusion in the AOHSS.

Phase 2: Delphi voting outcome

The first modified Delphi voting survey 1 (DVS1) consisted of three voting rounds, resulting in the inclusion of the following outcome areas from the 10 outcomes identified from literature review: health-related quality of life; overall patient/consumer satisfaction; emotional well-being; oral health-related quality of life (functional and psychosocial aspects); and impact of oral health on personal relationships. An agreed definition for each outcome was then established by the OHWG in advance of mapping to established PROMs. Delphi voting survey 2 (DVS2) identified the question from the established PROM that best represented the defined outcome areas identified in DVS1. Input from the patient advisory group, specifically regarding the importance of each outcome area, was then combined with OHWG findings from DVS2. During this process, impact on personal relationships was excluded and financial impact was added.

Case-mix variables specific to patient demographics, lifestyle and general health, and caries and periodontal risk factors were voted upon. Aspects of clinical care, including baseline clinical data, treatment options and outcome of care, were also included in the clinical measures and were included in voting surveys. Examples of case-mix variables and clinical measures that were considered but not included were occupation, living status, salivary gland hypofunction, oral health literacy and household income.

Following the OHWG and subject matter expert consultations, it was decided that the clinically reported measures for outcomes pertaining to caries and periodontal disease staging be based on existing tools aligned with the International Caries Detection and Assessment System (ICDAS)²³ and the recently internationally agreed standards for measurement of periodontal conditions²⁴.

Phase 3: Global open review to determine the appropriateness and feasibility of implementing the AOHSS

The worldwide open review generated 347 responses from participants in 87 countries (Figure 4). The respondents were from diverse sectors involved in the provision of oral health, including clinicians (67%), researchers (13%), educators (8%), along with policymakers, those involved in advocacy and industry (11%). Following a review of the draft version of the AOHSS, the responses were uniformly positive, and 93% of participants expressed an interest in receiving the final AOHSS; 82% also indicated that they would implement the set in their day-to-day practice.

Fig 4.

Global open review.

Phase 4: Patient/consumer survey to determine appropriateness of AOHSS from patient/consumer perspective

The patient/consumer surveys generated 129 responses; 69% of respondents were female and 72% were aged between 36 and 65 years. Approximately 80% of patient respondents had received routine dental care, and for the overwhelming majority this had been a positive experience; 87% of respondents wanted their dentists to routinely use the AOHSS.

Patients/consumers from USA and Australia in Phase 3 and health professionals who participated in the worldwide open in Phase 4 review expressed similar views about simplifying the question language; revising the type of questions and their response options, and clarifying the concepts captured. Health professionals were concerned about the apparent exclusion of specific population groups such as those in rural areas or those with special needs. Patients/consumers cited a high literacy requirement and the length of the questionnaire as points of concern. Some questions were also perceived to be judgemental about their circumstances.

Feedback from the worldwide open review and patient/consumer surveys was taken into consideration by the OHWG, and the AOHSS was refined accordingly. In particular, draft questions were translated into plain language.

Final version of the AOHSS

After the last OHWG voting round, agreement was reached on the final version of the AOHSS. This resulted in the inclusion of 17 outcome concepts (13 patient-reported, four clinician-reported) and 14 case-mix concepts (eight patient-reported, six clinician- or administrator-reported). The 31 conceptual outcomes included in the AOHSS ultimately comprised a total of 80 measures (25-patient reported, 55-clinician/administrative reported). These 25 items were directed towards patients (including demographics, the impact of their oral health on oral function, a record of pain and oral hygiene practices, and financial implications of care) and items for clinicians to complete, including medical history, a record of caries and periodontal disease activity, and types of dental treatment delivered. A complete overview of the measures included in the AOHSS is presented in Figure 5, with Figure 6 demonstrating the mapping of the AOHSS to the FDI oral health definition framework.

Fig 5b.

Fig 5c.

Fig 5.

Adult Oral Health Standard Set (AOHSS).

Fig 6.

Mapping of Adult Oral Health Standard Set (AOHSS) to FDI oral health framework.

DISCUSSION

This collaboration between FDI and ICHOM used a rigorous, consensus-driven approach involving oral health experts and patients/consumers across the globe to arrive at a harmonised list of items to be included in an AOHSS. The methodology employed in the development of this outcome set followed ICHOM protocols and focused on what matters most to patients on a daily basis. This approach has resulted in the publication of 27 ICHOM sets that have been adopted in medical institutions across the globe, which are developing systems based on the principle of value-based healthcare. They range from the implementation of the ICHOM set for coronary artery disease in South Australia to the implementation of the cleft lip and palate ICHOM set at the Erasmus University Medical Centre in the Netherlands¹². Hence, they differ in their purpose and their methodology from, for example, the core outcome sets developed in both Core Outcome Measures in Effectiveness Trials (COMET), which focuses on standardising outcome measures for clinical trials²⁵, and Outcome Measures in Rheumatology (OMERACT), concentrating on rheumatological conditions²⁶. In terms of value-based healthcare, there is emerging debate about the relevance of defining ‘value’ more widely than purely monetary in the context of cost-effectiveness, and how such concepts can be applied to oral healthcare^27,28.

It is intended that the AOHSS will be used by clinicians and patients in a shared decision-making environment to co-produce care plans and track oral health outcome progress over time. From a clinical perspective, the focus of this AOHSS was on the most prevalent oral conditions, caries and periodontal disease, but the AOHSS can readily be adapted to include other oral diseases and conditions. The measurement of caries is broad, with the caries status of each tooth recorded as sound, restored, or with caries involving enamel, dentin or the pulp. This broad measurement can be considered as a foundational level of data collection with regard to dental caries. It does not preclude using other standard caries experience measures, such as the DMF (Decayed, Missing, Filled) classification, or even a more ‘granular’ level of information, such as the ICDAS classification^29,30. Similarly, with the clinical recording of periodontal disease staging, the periodontal recording per sextant as either healthy or three grades of pocketing (< 5 mm, 5–7 mm and > 7 mm) allows an overall assessment of periodontal status that is broadly in line with more detailed assessments undertaken for periodontal patients or in oral epidemiological studies of the condition³¹. From the patient perspective, the integration of PROMs is strongly linked to quality of life measures, similar to other health disciplines. Although the potential of PROMs for shifting the pattern of care provision towards addressing the concerns and priorities of the patients themselves³² is recognised, their use in routine practice is still uncommon³³, and there are a number of challenges related to their use and interpretation³⁴. Although of importance, the following groups were excluded from the scope of this project, but warrant targeted follow-up work: children; special needs populations; older people; institutionalised persons; and those otherwise dependent on others for day-to-day oral care.

Reference periods for the most widely used subjective outcome measures for oral health and quality of life range between 6 and 12 months for adults, and national studies have used the latter as a time reference point. However, reference intervals in clinical studies are usually narrower in order to reflect the trajectory of changes in oral health after an intervention. The guideline from the National Institute for Health and Care Excellence in the UK recommends that recall intervals should be tailored to each patient depending on their need, and for adults should not be longer than 24 months³⁵. Therefore, the recommendation on data collection time-points is flexible and could mean that, at a minimum, the AOHSS should be collected during the following stages of care: baseline, described as 0 months or prior to any new treatment; treatment and recovery, described as > 30 days after each intervention; and follow-up, described as 12 months after completion of a cycle of care or at least once every 24 months for individuals not requiring any interventions (Figure 7, Time-points). The full AOHSS does not need to be collected at each time-point. Only those data points relevant to each stage in the care cycle should be collected. Furthermore, these minimum data collection time-points should not preclude more frequent data collection, such as annually, if judged to be clinically appropriate.

Fig 7.

Time-points.

Following the determination of both the validity and reliability of this newly developed standard set, it is intended that the AOHSS be used in clinical, research, evaluation and community settings. Its flexibility in data collection will facilitate cross-sectional and longitudinal use within diverse global settings. Some of the selected measures have already been included in large national epidemiological surveys to document the impact of oral conditions on daily life³⁶. The applicability and adaptability of the AOHSS for the various aforementioned settings will inevitably lead to supplemental items or tools being added to this minimum standard set. As long as the integrity of the original AOHSS is maintained and the psychometric properties of this minimum set are demonstrated, following rigorous testing in a clinical, research and population health setting, the addition of subsequent variables will allow flexible data collection whilst facilitating a basic comparison. The adaptability of the AOHSS will, however, only truly become apparent following the phase of psychometric testing. At the patient level, using the AOHSS at the beginning and end of a treatment course will facilitate assessment of preventive and treatment interventions.

Although the development process for the AOHSS has been rigorous, it is not without limitations. Although the OHWG members were recruited from across the globe, there was limited representation from Asia (with one member from UAE). The initial OHWG membership included one member from Africa, who participated in the initial planning stages of the study. Unfortunately, due to the pressure of other commitments, he was unable to participate in a sufficient number of the Delphi voting rounds to meet the criteria for inclusion in the list of authors. The patients involved in its development were also not truly representative of the diverse global population for which AOHSS is intended, and were concentrated in USA and Australia. The broad scope of the data collected and the focus restricted to minimum measures for common preventable oral conditions limits the ability to collect comprehensive disease-specific information on all oral health conditions. For its effective implementation, a clear communication strategy explaining the value of data collected will be important. Clinicians will need to use these data in their clinical practice to understand what matters to patients, and share data with patients. Regular review of data generated by this set will facilitate an assessment of which interventions are more effective in achieving outcomes important to patients.

It is hoped that the FDI-ICHOM collaboration will be viewed as a model for future quality improvement in healthcare-related projects. This initiative complements other work in the field, such as the ADVOCATE project³⁷.

In conclusion, utilising a robust methodology and a standardised core set of oral health outcome measures for adults, with a particular emphasis on caries and periodontal disease, was developed. To inform additional refinement of the AOHSS, the next step is to conduct a validation and feasibility study in diverse clinical settings. The AOHSS will ultimately empower patients to become decision-makers and co-producers in their care, and this has the potential to result in improved patient outcomes. The AOHSS could be utilised globally and potentially facilitate international comparisons of oral health outcomes. It would also have the potential to be useful in communities with limited capacity for oral care.

Author contributions

M Glick, SN Myers, DM Williams contributed to conception, design, data analysis and interpretation, drafted the manuscript, critically revised the manuscript; R Ni Riordain, SSA AlMashhadani, J Barrow, D Cole, S Hegde, E Kalenderian, WM Thomson, G Tsakos, RG Watt contributed to data analysis and interpretation, drafted the manuscript, critically revised the manuscript; K Aravamudhan, JJ Crall, JE Gallagher, J Gibson, R Kaberry, A Karki, RK Celeste, S Listl, R Niederman, T Severin, MW Smith, M Vujicic, S Whittaker contributed to data analysis and interpretation, and critically revised the manuscript. All authors gave final approval and agree to be accountable for all aspects of the work.

Conflict of interest

The authors declare no potential conflicts of interest with respect to the authorship and/or publication of this article.