Quality assessment of clinical practice guidelines for the management of paediatric dental emergencies applicable to the COVID-19 pandemic, using the AGREE II instrument. A systematic review

Abstract

Objective

To assess the quality of Clinical Practice Guidelines (CPG) related to the management of paediatric dental emergencies applicable to the COVID-19 pandemic, through the use of the measuring instrument AGREE II (Appraisal of Guidelines for Research and Evaluation in Europe).

Sources and data collection

A rigurous online search of CPG was accomplished among the main CPG compilers: National Institute for Health and Care Excellence (NICE), National Guideline Clearinghouse, Agency for Healthcare, Research and Quality (AHRQ), Andalusian Health Technology Assessment Department (AETSA), American Academy of Family Physicians, Tripdatabase. Furthermore, because of the need to identify CPG that meet the inclusion criteria, a manual search, among the main national and international dental organizations as well as recognized web sites, was also accomplished.

Selection of research studies

All of the guides focused on paediatric dental emergencies, available in the database and “gray” literature, and published between 2000 and 2020 (applicable to COVID-19 pandemic) were included without any language restrictions. The CPG that did not contain the full paper or were addressed to adults or children with special needs, were excluded from the selection. The evaluation of the CPG, independently included, were achieved by four (04) experts by using AGREE II.

Results

Five (05) out of twenty-three (23) selected CPG, were classified as “acceptable” according to AGREE II. These five guides were evaluated to determine their “Recommendation degree”. Only one (01) CPG “Guía Clínica AUGE de Urgencias Odontológicas Ambulatorias-Chile, 2011” reached a score of 75%, the highest among the other guides (5 domains with a score ≥ 60%, including the domain III “Rigour of Development”) to be considered as a “highly recommended” CPG.

Conclusions

According to the quality assessment and recommendation degrees criteria from AGREE II, high, middle and low quality CPG were identified. Only one CPG reached a score of 75%, to be classified as “highly recommended”. Therefore, it is suggested that the existing CPG updates and future CPG use the available tools and methodologies during their elaboration, in order to guarantee their quality.

Clinical significance

High quality CPG for the management of dental emergencies are designed to support dental health professionals in decision-making to adopt specific dental procedures in the current COVID-19 pandemic. As a matter of fact, these CPG might contribute to reduce the risk of transmission, in case of fresh outbreak of the illness. Likewise, they might help to determine which cases warrant medical attention in centres with special facilities for COVID-19.

Dentistry; Health profession; Emergency medicine; Pediatrics; Clinical practice guide; Quality; Guidelines; AGREE II; COVID19.

1. Introduction

Global public health is currently undergoing a significant crisis due to the outbreak and spread of the new SARS-Cov-2 (Severe Acute Respiratory Syndrome), originally reported in the city of Wuhan, China in December 2019 [1]. Renamed as COVID-19 by the World Health Organization (WHO) [2] and categorized as Pandemic on March 11^th, 2020 [3] is characterized by showing symptoms such as fever, cough, fatigue, myalgia, dyspnoea and in some cases diarrhoea. Patients with co-morbidities (hypertension, diabetes, obesity) and the elderly constitute the main population at risk. On the other hand, the majority of COVID-19 paediatric patients exhibit mild symptoms, no fever nor pneumoniae. During the first phase of the pandemic, there were not severe cases or deceases reported among paediatric patients [4]. As a matter of fact, a study that analysed 44,672 confirmed cases in China since February 2020, reported that only 416 cases (0.9%) were patients under 10 years old [5]. By June 2020, only two (02) deaths in children testing positive for COVID-19 were reported in China and no deaths, in Italy (the two countries with more confirmed cases). Nevertheless, with the progressive increase of confirmed cases in the adult population, the number of paediatric infections also increased concomitantly [6].

In general, any patient (either adult or paediatric) ought to be considered as potential COVID-19 carrier [7]. A large percentage of COVID-19 confirmed cases are asymptomatic or have mild symptoms [7, 8, 9]. Wang et. al. identified some risk factors associated with the virus transmission during dental treatments in paediatric patients i.e. the droplets emitted during sneezing and the aerosols generated by the high-speed piece [10]. The American Dental Association (ADA) and the Centres of MediCare and MedicAid Services (CMS), recommend that during the pandemic, dental procedures should be restricted only to emergencies so as to reduce the risk of virus spread among patients and dental staff [11, 12].

Life-threatening dental emergencies demand immediate treatment to stop continuous tissue bleeding, relieve pain or treat a severe infection. On the other hand, urgent dental care is focused on the management of conditions that require immediate attention to relieve moderate-severe pain, reduce the risk of infection and alleviate the patient burden in emergency centres [12].

The most frequent emergencies in children are: the reversible pulpitis, irreversible pulpitis [13], acute apical periodontitis, facial cellulitis, facial abscess and dental trauma [14]. Half of them are characterised for presenting sequelae related to dental caries [15].

The management of dental emergencies has become increasingly important due to the COVID-19 pandemic. The constant search for reliable scientific evidence, that allows solving clinical doubts and identifying suitable treatments, is more frequent in this context. As a result, it is necessary for the clinical dentists to have access to high quality CPG, which enable them to promote and recommend practical solutions to clinical doubts regarding efficient treatments in their daily routines. CPG can represent one part or the determining pillar in the elaboration of health policies. Therefore the preparation of CPG requires rigorous methodologies to ensure its quality. However, we need to take into consideration that not all CPG meet the basic requirements.

AGREE II is a reliable tool which assesses the methodological rigour and transparency used in the CPG preparation [16]. After having used this practical tool, it was shown that some CPG did not present an adequate structure, either due to a poor quality elaboration or a lack of updated scientific evidence [17]. The quality of a CPG is defined as the confidence that potential biases (in the development of the guide) have been adequately pointed out and that the recommendations are valid, both internally and externally [18].

The purpose of this systematic review is to evaluate the quality of the CPG for the management of paediatric dental emergencies, published in the period 2000–2020 and applicable to the context of pandemic, by using the AGREE II tool. Additionally, to provide relevant information to those researchers and/or institutions responsible for the development of CPG worldwide.

2. Methods

The present systematic review was registered in PROSPERO (registry number: CRD42020195678) and detailed methods are available in the published protocol [19]. The systematic review is reported according to PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) and to a checklist available as supplementary material [20]. The protocol was focused on the strategic search for published and available CPG. The questions, asked for the present review, were:

• •How many CPG for the management of paediatric dental urgencies and emergencies, are available and applicable to the COVID-19 pandemic?
• •Which high quality CPG could be recommended?

Details regarding the search are visualised in Figure 1.

Figure 1.

Guide identification and recovery flow chart.

2.1. Inclusion and exclusion criteria

Inclusion criteria:

• •CPG published in databases and gray literature, aimed at dental emergencies in children, and applicable to the current context of the COVID-19 pandemic.
• •CPG written in any language.∗
• •CPG published between 2000 and 2020.

∗ The native language of the evaluators is Spanish, with basic knowledge of English, Portuguese and Italian. For other languages, translations tools were used (https://www.enago.com/ar/y https://oxfordediting.com/). The CPG, included in this study, and the AGREE II instrument were translated into Spanish. Table 1.

Table 1.

CPG compiler agencies (Search and storage).

ORGANISMS		ELECTRONIC ADDRESS	KEYWORDS	CPG FOUND	CPG EXCLUDED	CPG INCLUDED FOR YOUR REVIEW	CPG EXCLUDED AFTER REVIEW	CPG INCLUDED TO EVALUATE WITH THE AGREE II INSTRUMENT	CPG EXCLUDED AFTER EVALUATION WITH AGREE II	CPG INCLUDED AFTER EVALUATION WITH AGREII
NICE	NICE	https://www.nice.org.uk/	Guide practice dental emergency children	324	320	4	1	3	0	3
National Guideline Clearinghouse	NGC	https://archive-it.org/collections/5265	Guidelines emergency dental	11	10	1	1	0	0	0
AHRQ Agency for Health Research and Quality	AHQR	https://www.ahrq.gov/	“Urgency dental”	4,441	4,441	0	0	0	0	0
AETSA	AETSA	http://www.juntadeandalucia.es	Guide practice dental emergency children	35	35	0	0	0	0	0
Primary Care Clinical Practice Guidelines	AAFP	https://www.aafp.org/about/policies/all/joint-development.html	Guidelines dental urgency emergency	63	63	0	0	0	0	0
TripDatabase	TRIP D	http://www.tripdatabase.com	Guidelines dental urgency emergency children	129	127	2	0	2	1	1
MANUAL SEARCH	Google Scholar	https://scholar.google.es/schhp?hl = es	Guidelines dental urgency emergency	67	30	37	19	18	17	1
TOTAL OF GUIDES				5,070	5,026	44	21	23	18	5

Exclusion criteria:

• •GPC that do not contain full text.
• •Previous versions of GPC
• •GPC aimed at children with special abilities.
• •CPG aimed at adults.

Ethical approval and informed consent were not necessary since no human beings were involved.

2.2. Search strategy

An online search was carried out among the main CPG compilers: National Institute for Health and Clinical Excellence (NICE), National Guidelines Clearinghouse, Agency for Health Research and Quality (AHRQ), Andalusian Health Technology Assessment Department (AETSA), American Academy of Family Physicians and Tripdatabase. The key terms used for this search were: (Guide practice dental emergency children), (guidelines emergency dental), “urgency dental”, (guidelines dental urgency emergency children); associated with the boolean operators: “AND” and “OR”. This search was carried out from 30th of April to 30th of July, 2020.

Additionally, a manual search was carried out for CPG that met the inclusion criteria and were available on the websites of various national and international dental organizations.

2.3. Screening and GPC selection

Initially, 5070 articles and CPG were collected. After the first filter, carried out by the reviewer “ASA”, and the subsequent examination carried out by the reviewers “JAM” and “GPS”, 5026 guides were excluded as they did not contain eligible aspects in the title and/or abstract. As a result, 44 CPG were selected for further content evaluation. Subsequently, a videoconference with all the reviewers (“JAM”, “ASA”, “GPS”, “GCHS”, “RLV” and “GTR”) was held to support the inclusion or exclusion of the assigned documents. Any disagreement among the reviewers was solved with further discussion and in cases where consensus was not reached, the judgement of an expert reviewer (“GTR”) was decisive. Eventually, only 23 papers met all the selection criteria. These were processed for data extraction and quality evaluation Table 1.

2.4. data extraction

The 23 selected CPG were assigned to the reviewer “JAM” to sort them according to their characteristics (year of publication, origin, type of guide – Expert opinion, Consensus or Based on evidence) and to classify them according to their specialty (dental emergencies) Table 2.

Table 2.

Characteristics of the Clinical Practice Guidelines included.

Characteristics of the Clinical Practice Guidelines included	Number	Percent
Year of publication
2000–2005	2	8,7%
2006–2010	1	4,3%
2011–2015	6	26,1%
2016–2020	14	60,9%
Continent of published guidelines
Europe	5	21,7%
Asia	1	4,3%
North America	1	4,3%
Center America	1	4,3%
South America	15	65,2%
Type of Guideline
Expert opinion	8	34,8%
Consensus	11	47,8%
Based on evidence	4	17,4%
Guideline specific to dental emergencies?
Yes	5	21.7%
No	18	78,3%

The evaluation of the quality and recommendations was carried out by using AGREE II.

2.4.1. AGREE II

AGREE II, an instrument for evaluating research guidelines, is commonly used to evaluate the quality of the information of the studies (components of the preparation and documentation of the process) and the recommendation degrees [18].

It is worth highlighting that this instrument does not contain specific criteria to assess the quality of clinical contents, nor the evidence that supports it.

Currently, this instrument is available in several languages and it also has a training manual which is aimed at guiding those who wish to critically evaluate any CPG [21]. https://www.agreetrust.org/resource-centre/

The AGREE II instrument consists of twenty-three [23] items, organized into six (06) domains, followed by two (02) global scoring items (General quality of the guide and Recommendations for its use). Each domain represents a unique dimension for quality evaluation, with a specific score that will determine whether the guide should be used (recommended) or not. All of the AGREE II items are ranked using a Likert-type scale from 1 (strongly disagree) to 7 (strongly agree). https://doi.org/10.1371/journal.pone.0174831.t001.

2.4.2. Analysis with AGREE II

The 23 selected guides were independently reviewed and five (05) CPG were classified as “acceptable”. After this result, all of the reviewers were calibrated (trained) in the use of this tool by an expert reviewer (“GTR”) via online (Cisco Webex and Zoom).

An instructive guide was used for this training. Subsequently, the evaluation of all of the five CPG was completed by each reviewer, who presented their data independently. The data were assessed for statistical analysis by using Cohen Kappa coefficient. The objective: to determine the degree of concordance among the reviewers.

During the calibration/training period, any discrepancy among the reviewers was discussed until consensus was reached and the Cohen Kappa coefficient (0.8) was obtained. All the data were compiled in a single table, in alphabetical order according to title, country of origin, organization that prepared it, year of publication etc. In addition, the evaluation scores achieved by all the CPG, according to the 6 domains of AGREE II, were included.

The Recommendation Degrees (RD) of the selected CPG were determined by using the following strategy: The guidelines could be classified as “Recommended” (R) (when at least 3 domains are ≥ 60%) “Recommended with Modification” (RM) (>30% to <60%) and “Not Recommended” (NR) (when at least 3 domains are =< 30%) [22]. Table 4.

Table 4.

Quality of the 5 dental guidelines report by domain.

Guideline characteristics			Domains Using AGREE II
			I	II	III	IV	V	VI	Global Average	Domain score		Recommendations
Title of Guideline	Year	Country	Scope and Purpose	Stakeholder Involvement	Rigour of Development	Clarity of Presentation	Applicability	Editorial Independence	Mean (SD)	≤30	≥60
Guía clínica AUGE urgencias odontológicas ambulatorias	2011	Chile	86	88	65	85	58	69	75% (12.8)	0	5	R
Management of acute dental problems. Guidance for healthcare professionals	2013	Scotland	82	76	40	44	17	40	50% (24.6)	1	2	RM
Manual de Referencia para Procedimientos Clínicos en Odontopediatría	2013	Brasil	57	56	39	67	28	25	45% (17.2)	2	1	RM
International Association of Dental Traumatology guidelines for the management of traumatic dental injuries: 3.Injuries in the primary dentition	2012	Suecia	94	67	38	39	0	25	44% (32.8)	2	2	RM
Linee guida nazionali per la prevenzione e la gestione clínica dei traumi dentali negli individui in eta’ evolutiva	2012	Italia	61	22	40	39	17	33	35% (15.6)	2	1	RM
	Mean (SD)	76.0 (16.2)	61.8 (25.2)	44.4 (11.5)	54.8 (20.5)	24.0 (21.5)	38.4 (18.2)	49.8 (19.6)	1.4	2.2
	Median	82	40	67	44	17	33
	Shapiro- Wilk p	0.392	0.001	0.639	0.142	0.528	0.104

Evaluation of the Global quality of the CPG according to the Domain score
High quality	3 domains> = 60% including domain III	Recommended (R)
Medium quality	> 30 < 60	Recommended with modifications (RM)
Low quality	3 domains ≤ 30% including domain III	Not Recommended (NR)

Overall evaluation of the guidelines according to domain score: High quality, when at least 3 domains are ≥60% (including domain III), it will be considered as Recommended (R). When the statistics are between> 30 < 60, they are determined as recommended with modifications (RM). Low quality, when at least 3 domains are ≤30 (including domain III), it will be considered Not recommended (NR).

The domain scores were calculated by adding the scores for each item in that domain and then scaling the total, as a percentage of the “Maximum Possible Score” for that domain. This was carried out by using the following mathematical operation:

Score obtained – Minimum Score = Percentage for that domain.

Maximum Possible Score – Minimum Possible Score x 100.

All this was weighted by the 4 reviewers, who qualified as concordant according to Cohen Kappa Coefficient Figure 2.

Figure 2.

Taken from: The AGREE Collaboration. 2001. “Evaluation of Clinical Practice Guidelines”. Appraisal of Guidelines, Research and Evaluation: St. George's Hospital Medica School, London, June.

2.5. Statistical analysis

The statistical analysis was carried out by using Stata V.15 software (Stata Corporation, College Station, Texas, USA).

The concordance degree among the reviewers for the eligibility of the guideline was calculated by using Cohen Kappa Coefficient, a qualitative assessment. (Cerda, L et al. 2008). [23]. The Kappa concordance coefficient among the 4 reviewers was k = 0.82. In addition to this, “Generalization of weighted Kappa coefficient” for more than two observers was necessary. After downloading the command Kappa2 (through the syntaxes “findit kappa2”) the expression “kappa2 OB01-OB04, wgt(w2)” was executed. All this, in order to obtain the global result.

The characteristics of the CPG were summarised by using descriptive statistics. The general scores of the included CPG are presented for each AGREE II domain through summary measures (mean, median and standard deviation) and Shapiro-Wilk p.

3. Results

From the analysis of the 23 CPG selected according to the inclusion criteria, the following results were obtained:

A gradual increase in the number of CPG publications was observed over the years. Of the total selected CPG, 13% of them were published between 2000 and 2010, 26.1% were published between 2011 and 2015; and 60.9%, between 2016 and 2020.

21.7% of the selected CPG came from Europe, 4.3% from Asia and 73.8% from America. On the other hand, five (05) were considered specific for paediatric dental emergencies, and applicable to this current context of pandemic. In addition, regarding the method of elaboration, 34.8% (08 CPG) of the selected guides were based on expert opinion, 47.8% (11 CPG) were created with consensus, and 17.4% (04 CPG), based on evidence Table 2.

The global evaluation of the 23 selected CPG, revealed that 78.3% (18 CPG) were “Not recommended” (NR) due to the lack of methodological rigour. Moreover, 21.7% (05 CPG) were identified to have acceptable quality and were categorised as “Recommended” (R) or “Recommended with Modification” (RM). On the other hand, regarding the Evaluation of the domains” in all the 23 CPG, it was shown that Domain I “Scope and Purpose” was the only one obtaining the highest average score (39.3%) and the Domains III and V (Rigour of Development and Applicability, respectively) obtained the lowest scores. The Shapiro-Wilk statistical test showed that Domains III, IV and VI (in all the 23 selected CPG) presented a statistically significant difference (p < 0.01) Table 3.

Table 3.

Quality of the 23 included guidelines on six domains using AGREE II tool.

Guideline characteristics				Domains Using AGREE II
				I	II	III	IV	V	VI	M	Domain Score		R
Title of Guideline	Organization/hipervínculo	Year	Country	Scope and Purpose (3–21)	Stakeholder Involvement (3–21)	Rigour of Development (8–56)	Clarity of Presentation (4–28)	Applicability (3–21)	Editorial Independence (2–14)	Mean Domain (SD)	≤30	≥60	Recommended for use?
Guidelines for dental care provision during the COVID-19 pandemic.	https://www.sciencedirect.com/science/article/pii/S1013905220303266#f0005	2020	Saudi-Arabia	44	39	4	17	0	8	19% (18.6)	4	0	NR
Covid-19 Recommendations in odontology.	http://www.msal.gob.ar/images/stories/bes/graficos/0000001881cnt-covid19-recomendaciones-en-odontologia.pdf	2020	Argentina	17	11	2	17	4	0	8% (7.6)	6	0	NR
Manual de Referencia para Procedimientos Clínicos en Odontopediatría.	https://www.revistaodontopediatria.org/publicaciones/manuales/referencia-para-procedimientos-en-odontopediatria-2da-edicion/Manual-de-Referencia-para-Procedimientos-en-Odontopediatria-2da-edicion.pdf	2013	Brasil	57	56	39	67	28	25	45% (17.3)	2	1	RM
Guía clínica AUGE urgencias odontológicas ambulatorias.	https://www.minsal.cl/portal/url/item/7222b6448161ecb1e04001011f013f94.pdf	2011	Chile	86	88	65	85	58	69	75% (12.8)	0	5	R
Guía de práctica clínica en salud oral Infancia y adolescencia.	http://www.saludcapital.gov.co/DSP/Documentos%20Salud%20Oral/Gu%C3%ADa%20de%20Pr%C3%A1ctica%20Cl%C3%ADnica%20en%20Salud%20Oral%20Infancia-Adolescencia.pdf C:∖Users∖Lenovo∖Downloads∖Alcaldía mayor de la ciudad http:∖www.saludcapital.gov.co∖DSP∖Documentos Salud Oral∖Gu%C3%ADa de Pr%C3%A1ctica Cl%C3%ADnica en Salud Oral Infancia-Adolescencia.pdf	2010	Colombia	72	39	15	17	4	67	36% (28.6)	3	2	NR
Diagnóstico y manejo de patología pulpar y periapical.	https://www.academia.edu/32275136/GPC_para_el_diagn%C3%B3stico_y_manejo_de_la_patolog%C3%ADa_pulpar_y_periapical	2016	Colombia	72	39	19	28	4	67	38% (26.9)	3	2	NR
Guía de manejo y atención en la clínica de urgencias.	http://odontologia.unicartagena.edu.co/programas-academicos/odontologia/guias-protocolos-y-manuales-de-atencion/file/10-manejo-y-atencion-clinica-urgencias	2020	Colombia	0	0	0	0	0	0	0%(0)	6	0	NR
Lineamiento técnico para la prevención y contención de COVID-19 para odontólogos y personal auxiliar de Costa Rica.	https://www.ministeriodesalud.go.cr/sobre_ministerio/prensa/docs/lineamientos_odontologos_v2_27032020.pdf	2020	Costa Rica	6	0	0	11	0	8	4% (4.83)	6	0	NR
Guías prácticas de estomatología.	https://www.academia.edu/36680221/Gu%C3%ADas_Pr%C3%A1cticas_de_Estomatolog%C3%ADa	2003	Cuba	56	22	6	39	63	8	32% (19)	3	1	NR
Protocolo para atención odontológica en emergencias y urgencias médicas durante la emergencia sanitaria por COVID – 19.	https://www.salud.gob.ec/wp-content/uploads/2020/04/PROTOCOLO-PARA-ATENCI%C3%93N-ODONTOL%C3%93GICA-EN-EMERGENCIAS-Y-URGENCIAS-ODONTOL%C3%93GICAS-DURANTE-LA-EMERGENCIA-SANITARIA-POR-COVID-19.pdf	2020	Ecuador	28	28	2	39	8	25	22% (13.9)	5	0	NR
Review of Urgent and Emergency Dental - Care in Wales.	https://www.ambulance.wales.nhs.uk/assets/documents/e5e029f8-5df2-49a6-a87e-3d765beb2db4636458390559076134.pdf	2016	Gales	28	28	13	28	13	25	22% (7.5)	6	0	NR
Linee guida nazionali per la prevenzione e la gestione clínica dei traumi dentali negli individui in eta’ evolutiva.	http://www.salute.gov.it/imgs/C_17_pubblicazioni_2755_allegato.pdf	2012	Italia	61	22	40	39	17	33	35% (15.6)	2	1	RM
Guía para el manejo odontológico de pacientes sospechosos o confirmados por covid-19 en las instalaciones de salud.	http://www.minsa.gob.pa/sites/default/files/publicacion-general/guia_para_el_manejo_odontologico_de_pacientes_sospechosos_o_confirmados_por_covid-19_en_las_instalaciones_de_salud_.def_.pdf	2020	Panamá	0	0	0	0	0	0	0% (0)	6	0	NR
Guía de práctica clínica: tratamiento de las enfermedades de la pulpa y de los tejidos periapicales en niños.	Ministerio de Salud - Hospital Santa Rosa http://190.102.131.45/transparencia/pdf/guiasclinicas/odonto/guia_pulpa_ninos.pdf	2017	Perú	0	0	0	0	0	0	0% (0)	6		NR
Manual de atención odontológica frente al covid-19.	ESSALUD-Red asistencial de Piura http://www.essalud.gob.pe/ietsi/pdfs/guias/Recomendaciones_OdontoestomatologIa_COVID.pdf	2020	Perú	39	17	8	17	4	0	14% (14)	5	0	NR
Guía de Prácticas Clínicas estomatológicas.	https://cdn.www.gob.pe/uploads/document/file/391344/Gu%C3%ADa_de_pr%C3%A1cticas_cl%C3%ADnicas_estomatol%C3%B3gicas20191017-26355-jichhz.pdf	2005	Perú	33	56	15	17	0	8	21% (20.1)	4	0	NR
Guía de Prácticas Clínicas Estomatológicas.	Hospital de apoyo NSM Carhuaz - departamento de odontología – MINSA https://www.academia.edu/40984334/Gu%C3%ADa_de_Practicas_estomatolo0gicas	2019	Perú	11	0	0	0	0	0	2% (4.5)	6	0	NR
Guía de atención odontológica para COVID 19.	Colegio Odontológico del Callao https://copcallao.org.pe/wp-content/uploads/2020/04/Guia-para-manejo-de-Covid-19-COP-Callao.pdf	2020	Perú	6	11	0	6	0	0	4% (4.6)	6	0	NR
Management of acute dental problems. Guidance for healthcare professionals.	https://www.sdcep.org.uk/wp-content/uploads/2013/03/SDCEP+MADP+Guidance+March+2013.pdf	2013	Scotland	82	76	40	44	17	40	50% (24.6)	1	2	RM
Management of Acute Dental Problems During COVID-19 Pandemic.	https://www.sdcep.org.uk/wp-content/uploads/2020/03/SDCEP-MADP-COVID-19-guide-300320.pdf	2020	Scotland	56	6	0	22	0	25	18% (21.5)	5	0	NR
International Association of Dental Traumatology guidelines for the management of traumatic dental injuries: 3.Injuries in the primary dentition.	https://pubmed.ncbi.nlm.nih.gov/22583659/	2012	Suecia	94	67	38	39	0	25	44% (32.8)	2	2	RM
Recomendaciones del Ministerio de Salúd Pública para profesionales odontólogos e higienistas dentales. Prevención y control de coronavirus COVID-19.	https://www.gub.uy/ministerio-salud-publica/sites/ministerio-salud-publica/files/documentos/noticias/MSP_RECOMENDACIONES_ODONTOLOGOS_HIGIENISTAS_DENTALES.pdf	2020	Uruguay	0	0	0	0	0	25	4% (10.2)	6	0	NR
ADA_Int_Guidance_Mgmt_Emerg-Urg_Dental_COVID19.pdf.	American Dental Association https://www.ada.org/∼/media/CPS/Files/COVID/ADA_Int_Guidance_Mgmt_Emerg-Urg_Dental_COVID19	2013	USA	56	11	0	0	63	8	23% (28.7)	4	1	NR
	Mean (SD)	39,3% (30.7)	26,8% (26.6)	13,3% (18.4)	23,1% (22.4)	12,3% (20.8)	20,3% (22.4)	23.6% (25.05)			NR = 78,3%
RM = 17,4%
R= 4.3%
	Median	39	22	4	17	4	8
	Shapiro- Wilk p	0.085	0.012	<.001	0.008	<.001	<.001

AGREE II scoring system: For each domain, scores are rated out on a 7-point scale (1 = strongly disagree, 7 = strongly agree) by individual appraisers. Individual appraiser scores are summed for an overall domain score, which is then scaled to a percentage of the maximum possible score for the domain, with higher scores indicating higher quality. The six domain scores are independent and are not aggregated into a single quality score.

Overall evaluation of the guidelines according to domain score: High quality, when at least 3 domains are ≥60% (including domain III), it will be considered as Recommended (R). When the statistics are between> 30 < 60, they are determined as recommended with modifications (RM). Low quality, when at least 3 domains are ≤30 (including domain III), it will be considered Not recommended (NR).

After evaluating all the domains in the 5 CPG classified as “acceptable” for this review, the following results were observed: The domain that achieved the highest average score was Domain I “Scope and Purpose” (76%) and the one with lowest score was Domain V “Applicability” (24%). On the other hand, Domain III, corresponding to “Rigour of development”, ranged from 38% to 65% with an average score of 44.4%. Furthermore, according to Shapiro-Wilk, it is observed that Domain III, presented p < 0.01, indicating that there is a statistically significant difference among the 5 CPG with respect to this domain. The summary measures (Mean, Median and Standard Deviation) were also obtained in each of the domains Table 4.

The quality evaluation of the 5 CPG using the AGREE II domains assessment, revealed that there was no specific CPG for the management of paediatric dental emergencies. However, when the objective of this topic was rigorously evaluated in these 5 CPG, it was observed that only one (AUGE Clinical Guide for Ambulatory dental emergencies – Chile, 2011) [24] reached the highest score (75%). This document exhibited 5 domains with a score ≥ 60%, including Domain III, and it was considered as “Recommended”, while the other four guides reached an average score of 43.5%. The results of the evaluation of these 4 guidelines were: Scotland, 2013 [25] that obtained 50%; Brazil, 2013 [26] that obtained 45%, Sweden, 2012 [27] that obtained 44% and Italy, 2012 [28], with 35%. All of them presented 1 to 2 domains with a score ≥60%. Furthermore, they all presented 1 to 2 domains with a score ≤ 30%. As a result, these four CPG were categorised as “Recommended with Modification” Table 4.

4. Discussion

These CPG for the management of paediatric dental emergencies have gained big importance during this COVID-19 pandemic since the American Dental Association (ADA) and the Centres for Medicare and Medicaid Services (CMS) [11, 12] recommended prioritising dental emergencies to avoid the spread of SARS COV-2 among patients and oral health professionals. Considering the current global situation we are undergoing, we planned to carry out this systematic review in order to find CPG, based on scientific evidence, with high methodological quality and applicable to this COVID-19 context.

Using the Agree II tool, we accomplished quality evaluations on all the CPG available online. The results of this review indicated that the general quality of the CPG for paediatric dental emergencies is mainly medium or high. These guides may be recommended with modification since the general scores are less than 50% for 3 out of the 6 AGREE II domains. As a matter of fact, we consider that it is still necessary to improve the presentations of the CPG, especially on the “Rigour of development”, “Applicability” and “Editorial independence”.

In addition, it is worth mentioning that although we used a search strategy for CPG, we did not find CPG, based of scientific evidence, including the title Paediatric Dental Emergency Management during COVID-19 pandemic. We strongly believe this is due to the recent SARS COV-2 outbreak. Consequently, we decided to include all the CPG that were related to the management of Dental Emergencies and with this, we were able to find provisional or preliminary CPG with special focus on the current context. One of these, was elaborated by the ADA though it was not prepared with the methodological rigour required for high quality CPG Table 3. Likewise, due to the little information regarding our objectives in this study, we decided to include the evaluation of protocols as they play an important role on CPG elaboration.

On the other hand, it is also relevant to clarify that, only one (01) of the five (05) selected CPG was exclusively made for children (“Reference Manual for Clinical Procedures in Paediatric Dentistry” ALOP, Brazil, 2013), while two (02) of these five, were aimed at primary dentition in relation to dentoalveolar trauma (“Linee Guida Nazionali per la Prevenzione e la Gestione Clinica dei Traumi Dentali negli individui in eta’ evolutiva”, Italy 2012) and (“International Association of Dental Traumatology Guidelines for the Management of Traumatic dental injuries: injuries in the primary dentitition”, Sweden 2012). Likewise, “Scotland 2013” guide is aimed at management of acute dental problems. Finally, the “Guideline for healthcare professionals” and the “Chilean CPG 2011” addressed to children and adolescents.

The CPG classification in this review showed that 17.4% of CPG are “Evidence-based”, while 34.8% are based on “Expert opinion”, and 47.8% are “Based on Consensus”. It should be noticed that a CPG prepared by consensus, represents the collective opinion or suggestions of a group [14]. In contrast, a CPG, made with scientific evidence, provides recommendations from a systematic review on a specific health issue and the possible benefits or disadvantages about the different treatment options [18]. Although both type of guidance documents contain suggestions for improving patients care and they both show their potential risks of bias [18, 19]. The CPG prepared by expert opinion were excluded since their methodological quality was poor (they did not have scientific rigour), the risk of bias was high and also the conflict of interest was considerable [18].

The online training, taken by the reviewers and directed by an expert (GTR), has also been reported by other authors [30, 31] and it has gained big importance for being an optimal way to guide, analyse and investigate through virtual platforms, during this pandemic.

4.1. About the limits

The specific criteria for stablishing the limits of evaluation for general quality in CPG, vary widely among the different studies [32]. In our study, it was considered to assess the CPG quality in a domain-specific way, with a limit of 60% to discern whether the CPG present high, medium or low quality. This strategy was adopted, based on previous studies.

In this regard, Hoffmann-Eber (2018) [22], reported that global or general evaluations of CPG, using AGREE II, are not frequently performed by CPG evaluators. This study recommends making more objective evaluations by weighing individual domains of AGREE II and considering Domains III and V as key factors on the results. Based on these studies, we established 60% as the cut-off point to discern high, medium or low quality guidelines (in the global evaluation and specific evaluation by domain). In addition, it is worth mentioning that the average score for domain III “Rigour of development” should be greater than or equal to 60%, to be considered as high quality.

In a parallel analysis, if our study only adopted the global evaluation strategy, also used by O'Donnell et al. (2020) [29], the result would show that 2 CPGs (Chile, 2011 (75%) and Scotland, 2013 (50%)) would be classified as high quality CPG. In contrast to this strategy, after the specific evaluation (domains) and assessment for recommendation degrees with the established limit of 60% including domain III, a more rigorous result was observed (only 1 CPG (Chile, 2011)) exhibited high quality and therefore, it was classified as recommended (R). Nevertheless, one of the main drawbacks in the application of this recommended CPG around the world, is the original language. As a matter of fact, its content is written in Spanish.

4.2. Global quality assessment

The evaluation of the global quality of the CPG is basically represented by the average of the 6 domains. In our review, a global average score of 49.8% was found, revealing a low global average quality in all the studied CPG (Table 4), while Mubenn et al (2019) [34] found approximate values of 51.9% (SD 13.3). Beckett et al. (2019) [35] applied the global mean score, by averaging the values of the 23 items of AGREE II and identifying the behaviour of each domain individually. In our review, we found that the only recommended guideline that met the criteria of the AGREE II tool, was the 2011 Chilean CPG, with an overall score of 75%; the other four CPG did not exceed 60% and therefore were considered recommended with modifications (RM), due to their low rigour of development.

Some authors question the global way of evaluating the CPG, without considering the weight of each domain. In fact, they mention that this global method is not “scientific” enough (30,33), and as a consequence, we decided to carry out the independent evaluation by domains in parallel, to obtain a more rigorous result.

4.3. Assessment of CPG quality by independent domains

Beckett et al. (2019) [35] used both forms of evaluation (Global and independent) in their study. In our study, an evaluation by independent domains was carried out, considering the importance of Domain III, similar to the studies by Jiao et al. (2019) [30] and Rabassa et al. (2018) [33].

Hoffmann-Eber (2018) [22] recommends prioritizing Domain III “Methodological rigour” and Domain II “Stakeholder involvement”. A complement to AGREE II, called AGREE REX (Recommendation of Excellence) was published in 2019. In this document, some guidelines are given on how users can classify guides as high, medium and low quality. In addition, the way in which the limits of scores can be determined by consensus among evaluators is explained, considerations about specific domains assessment are given for decision-making and limits are established based on each domain [36].

4.4. Comparison by domains

Domain I “Scope and Purpose”: In our study, the results of Domain I reached the highest global score (76%) compared to the other domains. Likewise, it was observed that each guide presented an average score between 57% and 94%, with the highest score belonging to Sweden 2013 (94%) and followed by Chile, 2011 (86%). The other three CPGs had lower scores, due to the lack of description of the CPG scope and objectives. Other authors also reported high scores in domain I [30, 31, 33, 35].

Domain II “Stakeholder Involvement”: In our study, the results of this domain reached the second highest average score among the 5 guides evaluated with [61.8% (22%–88%)]. The CPG from Chile, 2011 (88%) and Sweden, 2013 (76%) were the highest, compared to the other 3 guidelines, which did not consider the experiences of patients and their expectations about health care. Brosseau et al. 2014 [31] and Bhatt et al. (2018) [37] found similar mean score values in this domain, in contrast (Jiao et al. 2019), [30] found a low mean score for the CPG evaluated in their study.

Domain III “Rigour of Development”: This domain is considered one of the most important and influential indicators over the quality of a guide. The results of our study revealed a low global mean score [44.4% (38–65%)], only the Chilean CPG, 2011 showed a result of 65%, exceeding the limit of 60%. This categorised it as a “Recommended” guide, while the other 4 CPG did not exceed the 60% limit in this domain. As a matter of fact, our results coincided with the results from other studies such as Bhatt et al (2018) [37] and Jiao et al. (2019) [30] specifically in this regard. The variability of the results is due to the lack methodological rigour, limitations description, strengths, risks, benefits, among others. A good score in this domain guarantees the quality of the guide since the risk of bias can be controlled. However, according to Downell, et al. [29], it is important to keep in mind that none of the domains are more important than the others, as they are all related to each other.

Domain IV “Clarity of Presentation”: In our study, this domain obtained an average score of 54.8%, Chile presented 85% and Brazil 67% independently. The clarity of presentation, through key recommendations, algorithms and therapeutic options that facilitate decision-making, in most of the CPG were not explicit. Clarity in the presentation was only found in the CPG of Chile and partially in the CPG of Brazil and Scotland. Other authors [30, 31, 37], reported a global score greater than 60% in this domain.

Domain V “Applicability”: The lowest global score in our review was observed in this domain (24%). The CPG of Chile obtained 58% and the others showed scores lower than 28%. Similar and even lower results were reported by Broseau et al. (2014) [31], and Jiao et al. 2019 [30] whose studies showed 14% and 31.25%, respectively. While Bhatt et al. (2018) [37] showed values of 48% (10–96) and Beckett et al (2019) [35] 54.8% (±20.1). These results indicate that the majority of the CPGs did not present information on the facilitators, barriers and financial resources for the application of the recommendation; they only described the intention to carry it out without providing an implementation strategy. It is important that the CPG have an understandable format, including graphs and algorithms for decision-making. Regarding this, in our review, the Scottish CPG, 2013, presented an interactive electronic decision support tool, based on the information contained in this guide (http://madp.sdcep.org.uk/).

Domain VI “Editorial Independence”: Conflict of interest and Editorial independence have not been reported in detail in the majority of the CPG evaluated in this review. The average score that was obtained was 38.4%, independently. The Chilean CPG presented 69%, being different from the rest of the evaluated CPG. Other authors [29, 31, 37] also reported this domain as the lowest in their reviews. As a matter of fact, financial institutions seldom make an explicit declaration that their views or interests have not influenced the final recommendations. Reports of conflicts of interest and participation of financial entities in the development of guidelines are crucial for the assessment of this domain [29, 30].

In other aspects, Burgers JS [38] conducted a comparison study between North American and European CPG and found that European CPG exhibited a better quality. In our systematic review, the best quality CPG is from South America, Chile, 2011. This is due to the scientific progress this country has had in recent years and the large investment in public health development. Although Burgers JS mentioned that most of the high-quality guides have been developed by organizations in countries with more resources and funds for research (e.g. United Kingdom, United States, Canada etc.); in our review, we were able to verify that there are good quality CPG in developing countries, such as Chile, Mexico and Brazil. These countries present attractive proposals that could be modified and translated into the universal language for their application worldwide.

4.5. Implication in the development of new guidelines

The present lack of rigour in the development of CPG on dental emergencies, encourages us to develop new CPG based on high quality scientific evidence, to generate grades of recommendation aimed at the paediatric population.

The institutions in charge of elaborating CPG require a team of experts, internal and external, for the development of guides, complying with the methodological rigour.

A short-term measure is to update the high-quality CPG available and associated with the research topic of the guide to be developed. For this purpose, it is necessary to strengthen the cooperation of methodological experts, seek patients (opinions from the public) to improve the applicability of the CPG, solve financing problems and define conflicts of interest in a clear way.

4.6. Strengths and limitations

The strategy used for searching CPG constitutes one of the strengths of the present study. A meticulous investigation in the different guideline compilers and governmental entities from different countries, was carried out. The manual search of CPG applicable to the current context of the COVID 19 pandemic and the gray literature, provided additional value for obtaining eligible guidelines.

The world is undergoing a dreadful pandemic and this current context forces us to seek and provide quick solutions. The development of new knowledge on CPG is necessary. The management of emergencies in this context is relevant and so are high quality guidelines.

Nevertheless, this COVID-19 pandemic also represents one of the limitations, since in this context, the administrative processes that favour the adequate preparation of high-quality CPG for the management of paediatric dental emergencies are slowed down. In addition, specific CPG regarding this topic and written in the international language (English) are not available. As a matter of fact, this aspect represents a great limitation for our review.

4.7. Recommendations for future studies

For future research, it would be interesting to study the relationship between the quality of the guidelines and the effectiveness of the guidelines' recommendations in different countries because the economic, social and cultural realities of each country are different.

5. Conclusions

High, medium and low quality CPG for the management of paediatric dental emergencies were found. It is necessary to pay especial attention to the AGREE II domains so as to improve the CPG quality and apply them during the COVID19 pandemic.

According to the quality evaluation criteria and recommendation degree of the AGREEII instrument, only one CPG (AUGE clinical guide for ambulatory dental emergencies- Chile, 2011) was considered a “Recommended” CPG, but applicable only among Spanish-speaking countries. It would be advisable to work on this guide, using English as an international language.

Declarations

Author contribution statement

All authors listed have significantly contributed to the development and the writing of this article.

Funding statement

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Declaration of interests statement

The authors declare no conflict of interest.

Additional information

No additional information is available for this paper.

Appendix A. Supplementary data

The following is the supplementary data related to this article: