Table 1.

Characteristics of the included publications

Publications describing the development process of measures Author(s), title, year	Country	Study aim	Measure development method (Delphi, expert panel etc.)	Stakeholder involvement
1Dental quality alliance (DQA):Herndon et al (2015a)22 2American Dental Association (2016)23 3American dental Association (2018)24 4Hunt & Ohja (2017)25	United States	To develop and test paediatric oral health care quality measures	Literature review and RAND‐modified Delphi approach	Selected committee of the DQA, with engagement of multiple stakeholders, including clinicians, dental plans, State Medicaid agencies and federal agencies
5Herndon et al (2015b)26	United States	To validate three evidence‐based process of care quality measures related to dental caries prevention in children	The measures were previously developed by the DQA described in Herndon et al.22 Validity testing included the assessment of agreement between administrative data and dental records and assessment of variation in measures performance, using administrative / claims data from private or public dental benefits sources	Stakeholder involvement has been described above
6Herndon et al (2017)27	United States	To validate a DQA measure to determine whether children who have a caries‐related emergency department visit received follow‐up care and to formally validate a set of diagnostic codes in administrative claims data to identify caries‐related emergency department visit	The measures were previously developed by the DQA (Herndon et al).22 The validation process consisted of comparing the diagnosis codes in claims data with manual review of 300 records from a Florida hospital emergency department and by calculating the kappa statistic, sensitivity and specificity	Stakeholder involvement has been described above
Achmea Oral Health Project Hummel et al (2017)28 Projectteam Mondzorg (2015)29	Netherlands	To develop clinical outcome measures for oral health and explore their performance using health insurance claims records and clinical data from general dental practices	Literature review, meetings with an advisory board to select measures based on set criteria, and an implementation pilot to test the measures on feasibility and validity	Advisory board with four experts in quality of oral health care
European Global Oral Health Indicators Development (EGOHID) 1 Bourgeois et al (2008)30 Ottolenghi et al (2007)31 EGOHID catalogue, (2005)32	Europe	To develop a set of measures for monitoring and describing oral health morbidity and different facets of oral health care systems in Europe	Review of existing recommendations on oral health measures and a consensus process, including consultation meetings with a steering group and contributors, and a grading method to shortlist measures with highest importance from a long‐list	A steering group as part of the European health monitoring plan, and contributors, consisting of representatives from European clinical and scientific oral health organizations
Dental Quality and Outcomes Framework (DQOF) Department of Health (2011)33 Department of Health (2016)34	United Kingdom	To develop measures to measure dentists’ performance as part of the dental contract reform pilot of the National Health Services (NHS) in the United Kingdom	Consensus among members of a working group	Working group of 6 members, consisting of dental public health consultants and representatives of the British Dental Association and the Department of Health
National Oral Health Surveillance System (NOHSS) Malvitz et al (2009)35 Centers for Disease Control and Prevention (CDC) (2015)36 Chattopadhyay et al (2008)37	United States	To develop a public health surveillance system and to develop quality measures for adults and children which can be used by state health agencies and health programs to monitor oral health of the population	Limited information was found on the methods used to develop measures. A work group narrowed a list from 72 measures to 7, after which approval was sought by the Council of State and Territorial Epidemiologist (CSTE). All indicators were linked to the Healthy People 2010 objectives	Working group, consisting of content experts from the Center of Chronic Disease and Prevention Division of Oral Health (CDC DOH) and the Association of State and Territorial Dental Directors (ASTDD), forming the NOHSS
Nordic Project National Institute for Health and Welfare (2010)38 Ekornrud & Wilburg (2013)39	Denmark, Sweden, Norway, Finland, Iceland, Faroe Islands	To develop common quality measures for oral health care and improve for the Nordic countries	Oral health measures used in the Nordic countries were mapped. All measures were based on recommendations from the European community health indicators (ECHI), Organization for Economic Cooperation and Development (OECD) and EGOHID and specific Nordic interest. A working group agreed on the final measures	Representatives from all Nordics countries were included in the working group. Function of the representatives was not clear
Baâdoudi et al (2017)5	Europe	To establish measures of oral health for transparent and explicit reporting of routine data to facilitate more patient‐centred and prevention‐oriented oral health care	Four‐stage approach, including scoping of the literature and its appraisal, a meeting of experts, a two‐stage Delphi process and a World Café discussion	Stakeholders from 6 European Union countries, including dental practitioners, patients, insurers and policy makers from the public and private sector
Mattila et al (2002)40	Finland	To measure the quality of children's dental health care from the oral health records of 10‐year‐olds using five outcome measures	Measures were developed based on clinical experience, pedodontic expertise and scientific literature on technical (professional quality)	Not described
Mangione‐Smith et al (2011)41	United States	To describe the processes used to identify a recommended core set of quality measures for children's health care as mandated by the Children's Health Insurance Reauthorization Act of 2009 (CHIPRA) and provide an overview of the selected measures	Measures in use by Medicaid and Child Health Insurance programs (CHIP) were identified. A committee of experts developed criteria to evaluate the validity, feasibility and importance of quality measures. Subsequently a RAND‐UCLA‐modified Delphi process was used to process al measures and measures were assessed on legislative priorities	Stakeholders from the Agency for Healthcare Research (AHRQ) National Advisory Council for Healthcare Research and Quality Subcommittee for Medicaid and CHIP Programs, and members of the CHIPRA Federal Quality workgroup
Hussein et al (2017)42	Germany	To develop an external quality assurance procedure, examining the use of systemic antibiotics in periodontal, conservative and surgical treatment in ambulatory dental health care. The aim of the procedure was to increase patient safety through rational use of systemic antibiotics and increasing the use of first‐line medications	A systematic literature search, an analysis of dental claims data and antibiotic prescriptions. The proposed measures were evaluated by an expert panel using a RAND‐modified Delphi process	An expert panel consisting of dentists, maxillofacial surgeons and patient representatives

Publications describing the additional scientific properties or the implementation of measures Author(s), title, year	Country	Study aim	Methods to assess additional scientific properties or test the implementation of measures	Stakeholder involvement
Bhardwaj et al (2016)43	Unites States	To assess the feasibility and performance of a meaningful use dental clinical quality measure, which measures the percentage of children, aged 0‐20 years, who received a fluoride varnish application	The measure was previously developed as part of the Meaningful Use stage 2 by the centres of Medicare and Medicaid services (CMS). Feasibility was assessed by evaluation of the concordance between an automated query and a manual review (using dental electronic health records)	The measure was developed as part of the Meaningful Use stage 2 and 3 by Medicare and Medicaid and members of the DQA
Neumann et al (2017)44	United States	To adopt a DQA measure designed for administrative claims data to be used in electronic health records. To evaluate the feasibility and validity of implementing this measure to determine whether patients with diabetes received a comprehensive oral or periodontal examination	Adaptation of a dental quality measure, originally developed by the DQA, to be used in electronic health records. Development of an automated query to capture the oral healthcare received by patients with diabetes, and validation of this query by comparing the query with manual chart reviews	Original measure development; see DQA. Adaptation of the measure: researchers from dental Universities and dental professionals