Evaluation of the current guidelines for the management of haemorrhoidal disease using the Appraisal of Guidelines Research and Evaluation II instrument

Abstract

Background

Haemorrhoids are a very common disease and many professional societies have produced guidelines for their treatment. The aim of this study is to appraise the quality of the existing guidelines in the management of haemorrhoids.

Methods

A systematic search of the literature was conducted in the EMBASE, Google Scholar, Cochrane library, and PubMed databases. The quality of guidelines was independently appraised using the Appraisal of Guidelines Research and Evaluation II (AGREE II) instrument by five of the authors.

Results

Six guidelines of varying quality were identified and included in this study. The highest scoring guidelines were the SICCR (Società Italiana di Chirurgia Colorectale, which is Italian Society of Colorectal Surgery), ESCP (European Society of Coloproctology) and ASCRS (American Society of Colon and Rectal Surgeons) guidelines, scoring 86% each overall. There was considerable variability across not just the studies but across the different domains. The highest scoring domains were domain VI: editorial independence (median =95% across all studies) and domain I: Scope & Purpose (85%). The lowest scores were observed in domain V: Applicability (48%) and domain II: Stakeholder Involvement (41%). Only three of the six gained unanimous support for their use, whilst two of the guidelines were unanimously declared not suitable for clinical use.

Conclusions

With the notable exception of three guidelines (SICCR, ESCP and ASCRS), the general quality of haemorrhoid guidelines is poor. Stakeholder (especially patient) involvement and instructions on how to implement recommendations is lacking from the majority of guidelines. This is an area that requires urgent attention if we are to improve guidelines in haemorrhoid management.

Highlight box.

Key findings

• With the exception of three guidelines (SICCR, ESCP and ASCRS), the methodological quality of guidelines for haemorrhoids is poor.

What is known and what is new?

• To date, haemorrhoid guidelines have been constructed without being based on validated methodological tools.

• For the first time, this study assess the methodological quality for production of guidelines based on the AGREE II instrument

What is the implication, and what should change now?

• This study demonstrates that the main methodological limitations for the construction of haemorrhoid guidelines were lack of patient involvement and instructions on how to implement the recommendations of the guidelines. These areas require particular attention in future guidelines and it will be imperative to use of the AGREE II tool in the future.

Introduction

Haemorrhoids are the most common proctological condition in the Western world and their prevalence rate is around 4.5%, Furthermore, the most common affected cohort is those aged between 45 and 65 years (1,2). It has been reported that by the age of 50, half of the general population has experienced symptoms related to haemorrhoids (1-3). The debate over definition of symptomatic haemorrhoids is controversial and ongoing. Often, a variety of diverse symptoms are attributed to haemorrhoids (rightly or wrongly) either by physicians or by the patients themselves making it difficult for clinicians to define symptomatic haemorrhoids and has the consequential effect of either over or undertreatment of the symptoms (1-4). The most common anorectal conditions that overlap with haemorrhoids are fissures, skin tags, abscesses, inflammatory bowel diseases (IBD), and anorectal neoplasms (5,6). Painless rectal bleeding during or immediately after defecation is the most common presentation of haemorrhoids (1-6). Common associated symptoms are itching, soiling, prolapse, swelling and discomfort of the perianal area (1-6). Recurrent bleeding may cause secondary iron deficiency anaemia and rarely overwhelming bleeding may need urgent hospitalisation and blood transfusion (7,8).

In order to tailor an individualised treatment strategy, classifications and scoring systems for haemorrhoids have been developed. The Goligher classification was developed to classify internal haemorrhoids according to the presence and severity of prolapse (9). Principal limitations of the Goligher classification could be considered the lack of consideration of etiopathogenesis, and lack of consideration of associated symptoms and their impact on quality of life and from an anatomical point of view does not consider the difference between circumferential prolapse versus a single prolapsing haemorrhoid (10). To overcome the above limitations newer classification systems have been developed. In particular, the Nyström system is based on the frequency of discomfort, pain, itching, soiling, and need of manual reduction of prolapsed haemorrhoids (11). The system grades the frequency of the symptoms into four grades (I) never, (II) less than one-week, (III) 1–6 times per week and (IV) “every day”; it has been easily validated. However, it is limited by not considering the presence and frequency of cases with prolapse that do not need manual reduction. (11,12). In 2019, Rørvik et al., modified the Nyström scoring system considering how often the patient experiences prolapse of haemorrhoids but again not the need for manual reduction (13). Consequently, some other classifications have been proposed; however, they failed to gain widespread use due to their complexity (14,15). The most recent evidence based on the network meta-analysis by Jin et al. demonstrates that the treatment of haemorrhoids is not necessarily related to their symptomatic classification, but rather on their morphological presentation (16).

Realising the importance of the management of symptomatic haemorrhoids several professional societies have produced guidelines in order to aid effective management of patients with symptomatic haemorrhoids.

The aim of the present study was to evaluate the methodological quality of these guidelines using the Appraisal of Guidelines Research and Evaluation II (AGREE II) instrument. This study was presented in accordance with the PRISMA reporting checklist (available at https://atm.amegroups.com/article/view/10.21037/atm-22-4255/rc).

Methods

AGREE-II is a validated tool for assessment of the methodological quality of guidelines and moreover is supported by the World Health Organisation (WHO) advisory committee on health research and by many guidelines’ development teams (17,18). The AGREE II tool comprises 23 items divided into six domains: scope and purpose, stakeholder involvement, rigor of development, clarity of presentation, applicability, and editorial independence. For further details regarding the criteria used to describe and evaluate the 6 domains and the 23 consisted items, please see Figure S1. Detailed information can be found also at www.agreetrust.org (19).

Search strategy and guideline selection

PG and AA independently conducted a systematic literature search into Embase, PubMed, Cochrane library and Google Scholar using the following terms: clinical practice guidelines, hemorrhoidal disease, haemorrhoids or hemorrhoids. The search was limited to guidelines published in the English language. The search strategy was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) (20). After independent evaluation of guidelines by two authors (PG and AA) the following data were extracted: country of origin, year of publication, developers, funding resources, evaluation measures.

Appraisal of guidelines

The AGREE II tool was used to assess the quality of guidelines. Five appraisers (PG, AA, EG, NDA, SDS) evaluated the guidelines. Further details regarding the criteria used to describe and evaluate the 6 domains and the 23 items of the AGREE II tool are described in previous publications (21,22).

As per the AGREE II manual, discrepancies of more than 2 standard deviations (SDs) were resolved through dialogue. Authors had the ability to change their entry after group discussion. Domain scores were calculated with the following formula:

$$\frac{\left(\text{Obtained\hspace{0.17em}score}-\text{Minimum\hspace{0.17em}possible\hspace{0.17em}score}\right)}{\left(\text{Maximum\hspace{0.17em}possible\hspace{0.17em}score}-\text{Minimum\hspace{0.17em}possible\hspace{0.17em}score}\right)}\times 100\%$$ [1]

Results

AGREE II appraisal

A total of six guidelines were identified including: ACRSI (Association of Colon & Rectal Surgeons of India), ASCRS (American Society of Colon and Rectal Surgeons), ESCP (European Society of Coloproctology), SICCR (Società Italiana di Chirurgia Colorectale, which is Italian Society of Colorectal Surgery), PSG (Portuguese Society of Gastroenterology) and JPG (Japanese Practice Guidelines for anal disorders, chapter hemorrhoids) (23-28) (Figure S2).

Scores across the six domains varied widely; the overall median score across all the guidelines was 73% (Figures 1,2).

Figure 1.

Scores for each domain for the individual guidelines. ACRSI, Association of Colon & Rectal Surgeons of India; ASCRS, American Society of Colon and Rectal Surgeons; ESCP, European Society of Coloproctology; JPG, Japanese Practice Guidelines; PSG, Portuguese Society of Gastroenterology; SICCR, Società Italiana di Chirurgia Colorectale (Italian Society of Colorectal Surgery).

Figure 2.

Median scores for each domain. Error bars denote the lowest and highest scoring guidelines.

In domain I (Scope and purpose), the median score was 85%. The best score of 98% was observed in both SICCR and PSG and the lowest in ACRSI at 40%.

In domain II (Stakeholder involvement), the median score was 41%. ESCP scored the best at 89% and the lowest score was observed in JPG guidelines at 25%.

In domain III (Rigour of development), the median score was 66%, the highest scored was observed in ASCRS at 81% and the lowest in ACRSI at 19%.

In domain IV (Clarity of presentation), two guidelines ASCRS and ESCP shared the best score with 89% and the lowest score was observed in ACRSI at 68%. The median score of this domain was 81%.

In domain V (Applicability), ESCP scored 72% and the lowest score was observed in the JPG guidelines at 13%. The median score of the domain was 48%.

In domain VI (Editorial independence), the median score was 95%. The highest score was observed in ESCP at 100% and the lowest in the JPG guidelines at 4%.

Domain of overall assessment and recommendation for use of each guideline

Only the SICCR, ASCRS and ESCP guidelines were unanimously voted suitable for use in their present form, whilst the ACRSI guidelines received a score of ‘Yes with Modification’ (Figures 1,2).

Discussion

This study demonstrates that there is considerable variability with regards to the quality of haemorrhoid guidelines. The SICCR, ESCP and ASCRS guidelines scored the highest across almost all of the domains whilst the PSG and ACRSI scored reasonable scores across all the domains too. The JPG scored the lowest scores across all the domains.

Overall, domain VI (editorial independence) scored the highest out of all the domains. This domain assesses the editorial independence by checking whether the developers of the guidelines provided an explicit statement declaring whether they have any competing interests. In this domain, one guideline scored 95%. Five of the six guidelines scored above 95% (Figure 1).

These findings are similar to previous appraisals of guidelines for other colorectal conditions (20,21). This finding demonstrates that the developers explicitly describe the competing interests and guarantees that if there is a funding body this does not influence the content of the guideline and the development of recommendations.

In Domain I (scope and purpose) four (SICCR, PSG, ASCRS, ESCP) out of six guidelines scored above 94%. This finding demonstrates that the overall objectives regarding diagnosis, treatment, prevention, follow-up and the expected health benefits from the guidelines were specific to the health topic and reported clearly. Moreover, the target population, proposed interventions, comparisons and outcomes were reported clearly, particularly for the key recommendations.

Irrespective of healthcare issues or pathology, guidelines almost universally seem to score poorly in Domain II which pertains to the involvement of stakeholders. The Achilles’ heel of this domain is the limited representation of all relevant specialties. Moreover, contemporary patient centred healthcare systems require participation of patients within the target population. Despite the universally accepted modern approach to patient centred healthcare, repeated publications across a wide variety of subspecialties unfortunately indicate that patient groups are not being included in the formulation of guidelines. This is particularly concerning given that a considerable number of guidelines are produced by world renowned institutions and societies. In this domain ESCP, the PSG and SICCR guidelines achieved high scores. The remaining three scored very low and brought the overall score at 41%, the lowest of all the domains (Figure 1).

Scores for Domain III: Rigour of Development assesses the methodology of the guidelines and how robust the creation of the guidelines has been. Generally, the guidelines scored reasonably in this Domain. The highest score was obtained by the ASCRS (81%). Although three more guidelines namely ESCP, PSG and SICCR scored higher than 70%, the median score was 66%, because the lowest by ACRSI at 19% score had a negative impact on the median score. This finding demonstrates that the search of the literature was systematic, the electronic databases where the searched was performed was described and the search terms used precisely. In addition, evidence selection criteria, strengths and limitations of the evidence were described clearly. Moreover, the methods used to formulate the guidelines and the recommendations, and how final decisions were reached were described by the developers precisely.

The results of the domain IV (clarity of presentation) were quite satisfactory. Four (ASCRS, ESCP, PSG and SICCR) obtained scores above 88%. They achieved these results because their recommendations were specific and unambiguous. Moreover, the proposed management options and key recommendations were easily identifiable.

The haemorrhoid guidelines scored poorly in domain V (Applicability) (48%). The guidelines gave no indication or guidance as to how their recommendations should be implemented. This is another domain which has historically scored very poorly. Previous studies have reported this issue, namely that whilst guidelines may be good at presenting and clarifying their recommendations, they are often poor at reporting how clinicians should integrate these recommendations into clinical practice (21,22,29,30).

This historic tendency of guidelines to ignore applicability of their recommendations is very concerning as without a clear road map of how clinical practise could be improved, a unique opportunity may be missed to disseminate effective guideline implementation tools and strategies. Examples of how guideline recommendations should be implemented and not just what to implement, can go a long way in ensuring that hospitals and services learn from the best performing units around the world rather than each hospital trying to tackle the challenges of guideline implementation (of which there are many). Furthermore, a guideline that is able to consider the human and material resources required to implement its recommendations and how shortfalls in such resources could be overcome would undoubtedly be of greater value than one which simply lists recommendations.

There are several limitations to our study, most of which are inherent to the limitations of the AGREE II checklist itself. Whilst the AGREE II tool is a validated and universally accepted appraisal tool for the assessment of guidelines and is thought to be comprehensive in its six-domain assessment, it does have some restrictions. For example, AGREE II gives equal weighting to all of its Domains, implying that they are all equally as important as each other in formulating an overall impression of a guideline. However, this may not necessarily be the case. A 2017 study demonstrated that most assessors are heavily influenced by how guidelines perform in domain III (Rigour of Development) and domain V (Applicability) (31). In short, any guideline that scores well in these two areas is likely to have a favourable overall score. Arguably this phenomenon raises the question as to whether any future iteration of the AGREE checklist should have weighted rather than equivocal domains.

In any study such as this, there is also a concern for potential bias across the assessors. Although the assessors were all blinded to one another’s answers, loyalty to home/regional societies or familiarity with certain guidelines could have swayed an assessor’s score. Furthermore, despite the AGREE II website going to painstaking lengths to provide explanations as to what would constitute a certain grade/mark in each section, there is an element of subjectivity and interpretation that is hard to eliminate. In our study, the rate of concordance between the different assessors was high across the board and no particular Domain was highlighted as at ‘High Risk’ of discrepancy.

Conclusions

In line with previously published reports, guidelines in haemorrhoid surgery are generally good at outlining who they have formulated their recommendations for and what actions should be undertaken for particular clinical scenarios. Unfortunately, they also suffer from the same problems that many other guidelines experience and that is, they have been largely unable to instruct clinicians as to how they should implement these recommendations. The opinions of patients or their advocates have also sadly been lacking from the majority of the guidelines, highlighting a missed opportunity to elicit opinions from patients themselves as to what constitutes excellent care. During the updating of these guidelines, particular attention must be paid to certain Domains and the AGREE II checklist should be used as a template on how to improve future guidelines.

Supplementary

The article’s supplementary files as

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.