A critical review of recent clinical practice guidelines for pediatric urinary tract infection

Abstract

Introduction

Concerns regarding the quality, credibility, and applicability of recently published pediatric urinary tract infection (UTI) clinical practice guidelines have been raised due to the inconsistencies of recommendations between them. We aimed to determine the quality of the recent clinical practice guidelines on pediatric UTI by using the Appraisal of Guidelines Research and Evaluation (AGREE II) instrument, and summarize the standard of care in diagnosis and management of pediatric UTI from the top three clinical practice guidelines.

Methods

A systematic literature search was performed on medical literature electronic databases and international guideline repository websites. English language-based clinical practice guidelines from 2007–2016 endorsed by any international society or government organization providing recommendations for the management of pediatric UTI were considered. Eligible clinical practice guidelines were independently appraised by six reviewers using the AGREE II tool. Clinical practice guidelines were assessed for standardized domains and summarized for overall quality. Inter-rater reliability was assessed using inter-class coefficient (ICC).

Results

Thirteen clinical practice guidelines were critically reviewed. The Spanish clinical practice guidelines, American Academy of Pediatrics, and National Institute for Health and Clinical Excellence clinical practice guidelines consistently scored high on all AGREE domains (total averaged domain scores 90, 88, and 88, respectively). Among the six reviewers, there was a high degree of inter-rater reliability (average measure ICC 0.938; p<0.0001). There is reasonable consensus among the top three clinical practice guidelines in their major recommendations.

Conclusions

The clinical practice guidelines from Spain, American Academy of Pediatrics, and National Institute for Health and Clinical Excellence, with their major recommendations being similar, have scored highly on the AGREE II indicators of quality for the clinical practice guidelines development process.

Introduction

Urinary tract infection (UTI) represents one of the most common bacterial infections among infants and children.¹ If not managed appropriately, this condition may result in significant morbidity, with renal scarring, in particular, being the most worrisome long-term sequale.^1–3 According to the literature, the implementation of clinical practice guidelines (CPGs) for the management of UTI in children may be associated with significantly better outcomes.^4,5 However, multiple CPGs have been published, with significant variability and inconsistency in their recommendations. This leads to confusion and practical issues with regard to implementation strategies.^6,7 Likewise, concerns regarding the quality, credibility, and applicability of CPGs have been raised.^6–8 We hypothesized that these differences may be due to variation in the quality of the guideline development process.

The Appraisal of Guidelines, Research and Evaluation (AGREE) Collaboration, an international team of guideline developers and researchers, configured an instrument to assess the process and reporting of guideline development.⁹ The collaboration created a 23-item tool aimed at six quality-related domains to evaluate the quality standard of CPGs.¹⁰ In 2010, the AGREE II tool was introduced as the updated version and was recommended by the consortium as the preferred instrument for guideline development, reporting, and evaluation.^9,10

The aim of this project was to determine the quality of the CPGs on pediatric UTI using the AGREE II instrument. We also aimed to help clinicians by comparing, contrasting, and synthesizing the evidence-based recommendations for the diagnosis, assessment, and treatment of pediatric UTI across the highest-scoring guidelines.

Methods

This review complied with the standard reporting recommended by the PRISMA statement.¹¹ Prior to review, the study protocol was circulated among the reviewer group. Consensus was made to include only those documents identified as CPGs endorsed by any international society or government organization providing recommendations to guide clinical decision-making in diagnosing and treating pediatric UTI. Only English language-based CPGs were assessed. Publications such as narrative reviews, primary research, training manuals, patient and allied health professional guidelines, and technical guides were excluded. CPGs released prior to 2007 and or searches before 2006 were also excluded. If a CPG was already endorsed by a major umbrella professional organization, then the CPGs from its subsection or suborganization were not considered to reduce redundancy. CPGs for which development methods could not be verified due to the original documents being unavailable were likewise excluded. Only the latest version of the CPG was included.

CPG search, identification, and screening

The systematic literature search was performed by two reviewers independently in August 2016 for electronic databases (Pubmed, EMBASE, Scopus, US Agency for Healthcare Research and Quality [AHRQ], National Institute for Health and Care Excelence [NICE]-UK), the Australian Gov’t National Health and Medical Research Council (NHMRC), Scottish Intercollegiate Guideline Network (SIGN), Canadian Infobase Guideline, Guideline International Network (GIN), translate research into practice website (TRIP), Googlescholar, BMJ Best practice search, Wiley online library, and Cochrane online library. A complex search strategy included both “MeSH” (Medical Subject Heading) and “free-text” protocols. Specifically, the MeSH terms were: (“urinary tract infections” OR “infections” OR “pyelonephritis” OR “cystitis” OR “UTI”) AND (“pediatrics” OR “children”) AND (“practice guideline” OR “guideline” OR “CPG”). Multiple “free-text” searches were performed by applying the following terms through all fields: “cystistis,” “UTI,” “children,” and “CPG.” Experts of the field and regional professional organizations were contacted for any unpublished or draft guidelines. Relevant articles were also retrieved and cross-referenced to identify additional CPGs.

CPG appraisal and summary

A review team consisting of six physician representatives from different specialties, including general pediatrics, pediatric nephrology, pediatric urology, and general urology were involved in the CPG evaluation. All members were oriented with the AGREE II tool and underwent the online tutorial. To critically and effectively appraise CPGs and to reinforce consistency during appraisal, a clinical methodologist facilitator and field expert was involved to settle discrepancies or uncertainties. CPGs were independently appraised by each member and rated according to each domain according to the AGREE II tool.⁹ In order to avoid under-evaluation, all means were undertaken to assess the CPG documents; reviewers were requested to individually access the CPGs documents, as well as the websites, supplementary, and accompanied files associated with CPGs. Evaluation results from all appraisers were collected and tabulated. The appraisal score for each guideline was extrapolated for each AGREE domain and in overall total. Standardized domain scores were calculated according to AGREE II tool manual as follows:¹⁰

$$=\frac{\text{Obtained\hspace{0.17em}score}-\text{minimum\hspace{0.17em}possible\hspace{0.17em}score}}{\text{Maximum\hspace{0.17em}possible\hspace{0.17em}score}-\text{minimum\hspace{0.17em}possible\hspace{0.17em}score}}$$

Inter-rater reliability among the reviewers for each CPG was determined using interclass correlation coefficient (ICC) statistical analysis. The averaged total scores were standardized for each CPG by calculating the overall standardized domain scores. For the purpose of this critical appraisal, a domain score of <70 is considered low, an averaged total score of 80 for CPGs is considered satisfactory. The CPGs were then ranked according to the overall scores and assessed for each domain score.

One physician reviewer independently extracted the recommendation items from the top three CPGs using a standardized data extraction form, while another reviewer verified the extracted summary. The extracted data includes CPG source, scopes, objectives, and recommendations in pediatric UTI management.

Results

The result of systematic literature search from the online databases is summarized in Fig. 1. After removal of duplicate records, 1784 publications remained. Further screening of the records based on the document abstracts and synopsis left only 27 documents of CPGs for potential evaluation. Eleven guidelines were rejected as follows: five non-English guidelines (Chinese [2010], German [2007], Chile [Spanish, 2012], French Infection Disease Society and French Pediatric Society [2014], Portugal [2012]; three single-authored guidelines Sri Lankan, Korean, the AAFP, and one with methodology thatcould not be verified (Australian AFP). The Saudi (2010) guideline was excluded due to the document not being endorsed by any organization but rather more of a narrative review document. The 2010 AUA VUR guidelineswere also excluded, as this document does not discuss management of pediatric UTI specifically. A total of 16 documents with 13 CPGs remained and were critically reviewed.^12–27

Fig. 1.

PRISMA flow diagram of literature search process and results.

AGREE instrument scores

Among the six reviewers, there was a high degree of inter-rater reliability. The average measure ICC was 0.938, with a 95% confidence interval (CI) from 0.866–0.978 (F[12, 60]=16.141; p<0.0001). Table 1 summarizes the overall and individual domain scores of each of the included CPGs as assessed according to the AGREE II tool evaluation. The domains that scored highest among each CPG were the clarity of presentation and scope/objective. However, out of 13 CPGs, 10 had scores <70 for the domains of applicability, while the domains of stakeholder involvement and rigour of development were low in nine CPGs. The Spanish guideline for pediatric UTI, American Academy of Pediatrics (AAP), and NICE guidelines consistently scored high on all AGREE domains. These CPGs were also being ranked as the top three overall (Table 1).

Table 1.

AGREE II appraisal of pUTI CPGs: Summary of 6 domain mean standardized scores and total averaged domain scores of included guidelines

	Spain 2011	AAP 2011	NICE 2007	Australia 2015	ESPU 2015	Italian 2012	ACR 2012	Pakistan 2015	CPS 2015	AAUS 2016	Brazil 2015	Indian 2011	ESPR 2008
Total averaged domains score	90	88	88	71	63	59	50	42	45	35	33	33	30
Domain 1: Scope and purpose	97	93	97	81	73	87	58	44	87	45	61	69	44
Domain 2: Stakeholder involvement	86	82	95	79	48	57	51	30	50	34	16	38	31
Domain 3: Rigour of development	96	94	87	73	65	34	42	41	25	28	18	17	13
Domain 4: Clarity of presentation	94	95	93	74	85	81	67	47	79	73	42	53	46
Domain 5: Applicability	78	70	71	51	36	22	42	19	18	18	11	18	30
Domain 6: Editorial independence	89	94	82	67	74	74	38	74	8	11	50	1	15

AAP: American Academy of Pediatrics; AAUS: Asian Association of UTI and STI; ACR: American College of Radiology; CPG: clinical practice guidelines; CPS: Canadian Pediatric Society; ESPR: European Society of Pediatric Radiology; ESPU: European Society of Pediatric Urology; NICE: National Institute for Health and Clinical Excellence; pUTI: pediatric urinary tract infection.

Comparisons of scope, purpose, and content of top three ranked CPGs

Supplementary Table 1 (available at cuaj.ca) summarizes the similarities and differences of the top three CPGs. The objective of the top three CPGs were similar, where all aimed to improve clinical practice parameters in the management of pediatric UTI; however, there were differences in the scope among these CPGs. For the Spanish pediatric UTI CPG, the target population age ranges from one month to 18 year old, whereas in the NICE CPG, the target pediatric population is <16 years old. In the AAP CPG, they only targeted the pediatric population 2–24 months of age, where the authors indicated the evidence support was generated from studies of infants 2–24 months and did not believe it could be applied to children more than 24 months and less than two months old.

Review of key recommendations

Urine collection method

Due to age range difference, midstream clean catch urine specimen was recommended by the Spanish and NICE CPGs, while urethral catheterization and suprapubic aspiration (SPA) were preferred method of urine specimen collection by all three CPGs.

Urine specimen transport and preservation

All three guidelines agreed that the ideal window for urine examination is within four hours of collection of the specimen, whereas the AAP CPG is more strict, recommending that room temperature specimens should be processed <1 hour.

Urine microscopy

Both Spanish and NICE CPGs strongly recommend urine microscopy testing for patients less than three month old. Likewise, both CPGs have included leucocyte esterase, nitrites, and the presence of pyuria and bacteriuria as guides to patient management. However, the AAP guideline imposed a more strict recommendation that in order to establish the diagnosis of UTI, both positive urinalysis results (pyuria and/or bacteriuria) and the presence of at least 50 000 colony-forming units (CFUs) per mL of an uropathogen cultured should be required and specifically that the specimen is obtained through catheterization or SPA.

Blood testing to determine upper tract involvement of UTI

The Spanish CPG considers the test results of acute reactive protein, interleukin 6, C-reactive protein (CRP), and procal-citonin, while the NICE CPG does not recommend CRP to differentiate upper tract from lower tract involvement.

Diagnostic imaging

All three CPGs recommend ultrasound for the first febrile UTI in infants and older children and among patient with recurrent UTIs. All three CPGs recommend that routine radiological workup for vesicoureteral reflux (VUR) is not recommended after first UTI, except for cases where ultrasonography suggests either high-grade VUR or obstructive uropathy.

Prophylactic antibiotics

The Spanish CPG does not recommend giving prophylactic antibiotics for patient who will have a single catheterization, such as for voiding cystourethrogram (VCUG), while the NICE CPG recommends giving 3–4 days of prophylactic antibiotics with the procedure in the second day. Requesting dimercaptosuccinic acid scan (DMSA) after 4–6 months from the initial UTI is recommended by both Spanish and NICE CPG to determine renal parenchymal damage if upper tract involvement is likely. Furthermore, it is recommended as part of investigation if the patient experiences recurrent UTI.

Acute management

All three CPGs agreed that oral administration of antibiotic is the preferred route for the treatment of pediatric UTI; however, in circumstances that oral administration is not possible, intravenous (IV) antibiotics maybe considered. For the Spanish CPG, if the patient presents with suspected obstructive uropathy or high-grade VUR (4–5), signs of septicaemia, uncontrollable vomiting, or dehydration, then IV antibiotics should be initiated. The AAP CPG considered both oral and IV antibiotics equally efficacious. All three CPGs recommend starting empiric antibiotics of choice according to local antibiogram then adjusting based on urine culture sensitivity. Furthermore, it is also consistent among the three CPGs to reevaluate the patient’s clinical condition after 48 hours and advise parents to return for further evaluation if no improvement occurs.

Long-term management

For long term management of pediatric UTI, neither Spanish nor NICE CPGs recommend antibiotic prophylaxis for patients after the first UTI; however, prophylaxis may be considered for patients with recurrent UTI. The Spanish CPG recommends giving prophylactic antibiotics in the presence of urinary tract dilation and suspected obstruction until the diagnosis is confirmed and proper treatment is given, as well as for patients with high-grade VUR only (Grade 3–5 for girls and Grade 4–5 for boys) and re-evaluate after one year. The NICE CPG states that if a patient on prophylactic antibiotics develops a breakthrough UTI, the class of antibiotics should be changed and not to increase the dose of the same class of antibiotics. The AAP does not have any recommendation for antibiotic prophylaxis, but stated that although the effectiveness of anti-microbial prophylaxis for UTI prevention has not yet been demonstrated, the concept has biological plausibility.

Preventive measures

Both Spanish and NICE CPGs recommend preventive measures to reduce recurrences of UTI, particularly focusing on the pattern of urinary tract dysfunction, bowel habits of the patient, adequate fluid intake, and other behavioural modifications. Routine urine testing is not recommended as followup for asymptomatic patients with prior history of febrile UTI. According to both Spanish and NICE CPGs, infants and older children who have bilateral renal abnormalities should have regular monitoring to assess kidney function, blood pressure, and/or proteinuria, and if detected, they should be seen and managed by a pediatric nephrologist appropriately to prevent or slow the progression of chronic kidney disease.

Discussion

To date, there are a number of CPGs available for the management of pediatric UTI. It is strongly contemplated that an objective tool with high inter-rater reliability for the evaluation of the CPGs shall give a better sense of how the CPG may help practice among clinicians. The AGREE II tool is widely used to evaluate the quality standard of CPGs in different fields of medicine to assess methodological rigour and transparency of guideline development.²⁸ This CPG evaluation tool has been validated and tested for high reliability with detailed framework to assess the quality of guidelines in six standardized domains.^9,10,28 To the best of our knowledge, this is the first systematic review providing a critical appraisal of recent CPGs evaluating pediatric UTI.

Our study results showed that based on AGREE II tool evaluation, the quality of the available CPGs ranges. Using the AGREE II tool evaluation in this critical appraisal, our statistical analysis confirmed its high inter-rater reliability (average measure ICC=0.938), which according to accepted standard is more than ideal.^29,30 Furthermore, this high inter-rater reliability was obtained from a team of reviewers representing different specialties that are considered the stakeholders of these CPGs.

Quality of the CPGs and domain scores

Out of 13 CPGs assessed, only three had an overall score of >80 from the averaged domain scores. The AGREE recommends evaluating the CPGs according to the individual domains rather than tallying the overall score; however, taking that into considerations, our review result showed that the scores in each domains correlate well with the overall total. Specifically, the top three guidelines consistently rated high (>70) in all domains.

The recently published CPGs do not have higher scores compared to the earlier published CPGs. This implies that despite increased availability of high-quality evidence and awareness of evidence-based medicine through the years, there was no temporal relationship with the quality of the recently published/endorsed CPGs. The same findings were noted in prior reviews on pediatric CPGs,^31–33 which could be due to lack of awareness of AGREE II criteria leading to inadequate reporting. This also indicates that efforts are needed to increase recognition of the importance in improving the quality of the CPGs according to AGREE recommendations and engaging future CPG developers to adhere to a standard process in the development of CPGs.

Among the domains evaluated in the CPGs according to AGREE II tools, our assessment showed that clarity of presentation and scope/objective are the two domains that consistently rated higher than others among all the CPGs, while the domains on applicability, rigour of development, and stakeholder involvement consistently rated low across all CPGs. These findings were similar to prior critical appraisal of other pediatric and adult CPGs.^31–35 It is important for the CPGs to have undergone a rigourous process of development with involvement of stakeholders in the formulation of recommendations, and to provide means for facilitation in the implementation of the guideline with monitoring/auditing criteria. However, these are the domains that were consistently rated low among the CPGs. We strongly believe that these domains are the more important methodological quality standards that distinguish the credibility and usefulness of CPGs. The same proposition was made in a recent critical appraisal of adult non-neurogenic lower urinary tract management CPGs, where it was emphasized that more prominence and weight should be given to process development and to the means of facilitating implementation during evaluation of guidelines.³⁴

Recommendations from the top tier CPGs

The major difference among the top three CPGs is the target population age range. Compared to the Spanish and NICE CPGs, the AAP CPG has a more restricted population, targeting only 2–24-month-old children with UTI. Since the covered patient population was non-toilet-trained, the AAP also have a more strict definition for UTI diagnosis and stringent requirements for urine collection and preservation. The majority of recommendations for diagnostic criteria and acute management are consistent among the three CPGs; however, only the Spanish and NICE CPGs are able to give additional recommendations for long-term management. The Spanish CPG was developed following the recommendations of AGREE and also added de novo evidence for recommendation formulation.

Strengths and limitations of CPG critical appraisal

Having six physician reviewers representing different disciplines involved in this CPG critical appraisal using a validated tool (AGREE II) is the strength of this study. Furthermore, a high inter-rater reliability further added credibility to the assessment of each CPG.

In terms of the limitations of prior critical reviews of pediatric CPGs, AGREE II does not provide a cutoff in determining the adequacy of CPG quality; in this case, we decided on using an arbitrary priori cutoff of 70 as the basis to show adequacy of quality. A further limitation is the fact that the top three CPGs were published more than five years ago, with their respective literature searches and basis of recommendation not updated (although AAP had just recently reaffirmed their 2011 recommendation statements).³⁶ According to the statements of the endorsers of the top three CPGs, updated versions will soon be available.

Conclusion

The CPGs from Spain, AAP, and NICE scored highly on the AGREE II indicators of quality of the CPG development process. Domains of applicability, stakeholder involvement, and rigour of development were suboptimal quality-wise in the majority of the most recently available CPGs for pediatric UTI. It is recommended that clinicians consider these findings when selecting appropriate pediatric UTI guidelines for use in their practice.