Impact of the Joint Advisory Group on Gastrointestinal Endoscopy (JAG) on endoscopy services in the UK and beyond

Abstract

The Joint Advisory Group on Gastrointestinal Endoscopy (JAG) was initially established in 1994 to standardise endoscopy training across specialties. Over the last two decades, the position of JAG has evolved to meet its current role of quality assuring all aspects of endoscopy in the UK to provide the highest quality, patient-centred care. Drivers such as changes to healthcare agenda, national audits, advances in research and technology and the advent of population-based cancer screening have underpinned this shift in priority. Over this period, JAG has spearheaded various quality assurance initiatives with support from national stakeholders. These have led to the achievement of notable milestones in endoscopy quality assurance, particularly in the three major areas of: (1) endoscopy training, (2) accreditation of endoscopy services (including the Global Rating Scale), and (3) accreditation of screening endoscopists. These developments have changed the landscape of UK practice, serving as a model to promote excellence in endoscopy. This review provides a summary of JAG initiatives and assesses the impact of JAG on training and endoscopy services within the UK and beyond.

Introduction

The role of Joint Advisory Group on Gastrointestinal Endoscopy (JAG) in quality assurance (QA)

QA is the process of monitoring and assessing a product, service or process to ensure that it is of sufficient quality.¹ In the 1990s, the expansion and multidisciplinary nature of endoscopy led to calls for a unified advisory body to quality assure endoscopy training.² Thus, the JAG was established in 1994 under the auspices of the Academy of Royal Medical Colleges with committee members from the Royal Colleges of Physicians, Royal Colleges of Surgeons, Royal Colleges of Radiologists and Royal Colleges of General Practitioners. Although the initial focus was on standardising training between specialties, the role of JAG has progressively evolved to meet its current vision of quality assuring all aspects of endoscopy to provide the highest quality, patient-centred care. Over the last two decades, drivers such as changes to healthcare agenda, national audits, advances in research and technology and the advent of population-based cancer screening have been key in this shift in priority.³ Over this period, JAG has spearheaded various QA initiatives with support from other national stakeholders including the Department of Health (DoH), British Society of Gastroenterology (BSG), Association of Coloproctology of Great Britain and Ireland (ACPGBI), Association of Upper GI Surgeons and Specialist Advisory Committees (SACs). These have led to the achievement of notable milestones in endoscopy QA (table 1), establishing its role in the three major areas of: (1) training, (2) accreditation of services and (3) accreditation of screening endoscopists (figure 1). These developments have changed the landscape of endoscopy practice in the UK and serve as a model to promote excellence in endoscopy. This review provides a summary of JAG initiatives and assesses the impact of JAG on training and endoscopy services within the UK and beyond.

Table 1.

Timeline of JAG achievements and corresponding drivers

Year	JAG milestone	Driver(s)
1994	JAG established.
1999	First JAG position statement: ‘Recommendations for training in GI endoscopy’.
2002	First JAG committee meeting.
2003	Development of endoscopy modernisation England – NHS endoscopy toolkit.	DoH support
2004	GRS piloted – part of NEP, aligned with JAG.	DoH support
	‘Guidelines for training, appraisal and assessment of trainees in GI endoscopy and for the Assessment of units for Registration and Re-Registration’. Endoscopy curriculum. DOPS in formative and summative assessment.	NCEPOD ‘Scoping Our Practice’ 1st National Colonoscopy Audit
	Early preparation for JAG QA visits.	DoH support
2005	National endoscopy training programme established.	DoH support, SACs
	Endoscopy GRS handover to JAG to underpin accreditation.
	Bowel Cancer Screening Programme (BCSP).
2006	Endoscopy service accreditation commenced to coincide with BCSP.
2007	BSG Quality and Safety Indicators for Endoscopy document released.
2008	Gastrointestinal Endoscopy for Nurses programme commenced.	NEP
2009	Formal handover of NEP work and all outputs to JAG.
	Release of JETS e-Portfolio.
	National Nurse Endoscopist project linked to training centres.	DoH support
	GRS introduced for private providers.
2011	JAG GRS released for use in New Zealand and Ireland.
	Electronic e-certification (diagnostic upper GI endoscopy, flexible sigmoidoscopy and colonoscopy).	2nd National Colonoscopy Audit, Full BCSP Roll-out.
	First DOPyS.
2013	National Endoscopy Database project started.
	JAG formally take on administration and governance of BCSP accreditation from Public Health England.
	Best Practice Tariff for JAG-accredited units.	DoH support
	BCSP Bowel Scope accreditation started.
2016	Updated Global Rating Scale census and JAG accreditation standards released.
	Updated DOPS forms and trainee certification criteria released.
2017	JAG Research Group formed.

BCSP, Bowel Cancer Screening Programme; BSG, British Society of Gastroenterology; DoH, Department of Health; DOPS, direct observation of procedural skills; DOPyS, direct observation of polypectomy skills; GRS, Global Rating Scale; JAG, Joint Advisory Group on Gastrointestinal Endoscopy; JETS, JAG Endoscopy Training System; QA, quality assurance; NCEPOD, National Confidential Enquiry into Patient Outcome and Death; NEP, National Endoscopy Programme; SAC, Specialist Advisory Committee.

Figure 1.

The role of JAG.¹² Courtesy of Dr John Anderson. BCSP, Bowel Cancer Screening Programme.

Training

The unacceptable standards of practice reported in the 1999 UK colonoscopy audit,⁴ in anticipation of national bowel cancer screening, raised questions over workforce competence. This catalysed a review of endoscopy training, which identified clear needs for defining standards for competent practice, methods for assessing competence and a structured endoscopy curriculum. In response, these elements were covered in a seminal 2004 JAG document,⁵ which also called for training units to have shared responsibility in ensuring trainee competence. The concept of certification was proposed, which relied on trainee adoption of a ‘JAG logbook of experience’, engagement in summative assessment and supervisor sign-off.⁵ Direct observation of procedural skills (DOPS) and direct observation of procedural polypectomy skills (DOPyS) were introduced to standardise assessment, and highly focused courses were developed for trainees and trainers. JAG-approved basic upper and lower GI endoscopy courses became compulsory for certification, while specific training-the-trainer courses evolved to improve training standards at base hospitals. The guideline formed the foundations for quality assurance of training (QA-T) and service accreditation (described below).

In 2009, the JAG Endoscopy Training System (JETS) was launched. It had four main purposes: (1) an electronic record of trainee procedural experience and assessment, (2) a portal for accessing training courses, (3) to provide evidence of trainees meeting JAG standards of competence and (4) to provide feedback to trainers and training course organisers.^{1 6} JETS enabled trainee competence to be monitored and determined centrally, paving the way for e-certification, which began in 2011 for upper and lower gastrointestinal (GI) endoscopy (figure 2). By January 2017, JETS had been adopted in >250 UK training centres, with 2857 instances of e-certification awarded.⁷ JAG training courses are now regularly delivered in 28 UK centres, comprising 10 types of basic skills courses, 9 skills improvement courses and 4 endoscopy trainer courses.⁸ JAG have also supported non-medical endoscopists and endoscopy nurses via the GI Endoscopy for Nurses programme.

Figure 2.

JAG trainee certification pathway.¹² Courtesy of Dr John Anderson. DOPS, direct observation of procedural skills; JAG, Joint Advisory Group on Gastrointestinal Endoscopy; KPIs, key performance indicators; QA, quality assurance.

Accreditation of services

Shortly after publication of the national colonoscopy audit,⁴ the 2004 National Confidential Enquiry into Patient Outcome and Death (NCEPOD) report: ‘Scoping our Practice’ also highlighted shortfalls in quality of care.⁹ The expectation for individual endoscopy units to meet quality standards of care led to centralised accreditation of endoscopy units being proposed in the 2004 JAG position statement.¹⁰ The Global Rating Scale (GRS) was developed in 2004 as a quality improvement tool for endoscopy units to self-assess against a number of measures associated with high-quality and safe patient-centred care.¹¹ The GRS assesses patient experience within four domains (table 2), each with corresponding items. Items are scored A–D, with level B indicating attainment of minimum requirements, with level A being an aspirational target for high performing services. To be accredited, units are required to achieve at least level B in all GRS domains and provide substantiating evidence. Service standards are inspected through a peer-review site visit by trained JAG assessors. During the site visit, the unit environment is also assessed to evaluate privacy, dignity and safe decontamination practices. Once awarded, accreditation is renewed annually, with services required to provide interim evidence of eligibility via the annual report card.

Table 2.

The four GRS domains and their corresponding items

Domains	Clinical quality	Quality of the patient experience	Workforce	Training of endoscopists
Items	Leadership and organisation. Safety. Comfort. Quality. Appropriateness. Results.	Respect and dignity. Consent process including patient information. Patient environment and equipment. Access and booking. Planning and productivity. Aftercare. Patient involvement.	Teamwork. Workforce delivery. Professional development.	Environment, training, opportunity and resources. Trainer allocation and skills. Assessment and appraisal.

By 2005, service accreditation achieved national roll-out and became required for services wishing to contribute to bowel cancer screening in England. A JAG subcommittee, now known as the Endoscopy Services Quality Assurance Group, administers and regulates the data collected. In 2013, the Best Practice Tariff was commissioned by the DoH in England, thereby enabling higher rates of reimbursement for accredited units. Additionally, service accreditation become a prerequisite for Trusts to receive trainees in endoscopy,¹² further incentivising units to participate in the QA process.

Today, the GRS has evolved into a web-based tool for unit-level quality improvement (figure 3). The GRS is in place in 485 UK units, with 228 units (47%) achieving full JAG accreditation in August 2017.¹¹

Figure 3.

JAG unit accreditation pathway. GRS, Global Rating Scale; JAG, Joint Advisory Group on Gastrointestinal Endoscopy; KMS, Knowledge Management System; QA, quality assurance.

Accreditation of screening endoscopists

The Bowel Cancer Screening Programme (BCSP) operates on the observation that population-based endoscopic screening reduces colorectal cancer (CRC) incidence and mortality.¹³ In England, the BCSP commenced roll-out in 2006 for colonoscopy and 2013 for flexible sigmoidoscopy (Bowel Scope) screening. At the outset of screening, the Screening Assessment and Accreditation System (SAAS) was launched to quality assure the endoscopists within BCSP.⁴ In order to achieve accreditation, endoscopists are required to provide evidence of locally verified key performance indicators (KPIs), complete knowledge-based assessments and demonstrate competence in summative assessment (figure 4).

Figure 4.

BCSP accreditation pathway.¹² Courtesy of Dr John Anderson. BCSP, Bowel Cancer Screening Programme; DOPS, direct observation of procedural skills; DOPyS, direct observation of polypectomy skills; JAG, Joint Advisory Group on Gastrointestinal Endoscopy; MCQ, Multiple Choice Questionnaire; NHS, National Health Service.

Aims

The primary aim of this review was to amalgamate published evidence supporting the impact of JAG on quality of care (ie, patient outcomes), services or training in endoscopy. Secondary aims included assessing the impact of JAG on (1) service implementation and (2) research, where JAG tools were integral to the design.

Methods

Search strategy

In order to assess the impact of JAG, a comprehensive literature search was conducted in July 2017 through Embase, Ovid and PubMed to identify relevant publications and conferences abstracts over the last 10 years. The search strategy involved the combination of the following terms: (‘Joint Advisory Group’ or ‘JAG’ or ‘Global Rating Scale’ or ‘JETS e-portfolio’ or ‘DOPS’ or ‘DOPyS’ or ‘bowel screening’) AND (‘endoscopy’ or ‘colonoscopy’ or ‘polypectomy’ or ‘accreditation’). Studies were limited to those in English, with accompanying abstracts, and those published after January 2007.

Inclusion and exclusion criteria

To enable summation of the literature search results, thematic analysis was used to summarise suitable publications into the following categories: (A) quality of care, for example, KPIs related to patient outcomes such as caecal intubation rate (CIR); (b) quality of service provision, for example, waiting times; and (C) quality of training, for example, trainee performance/satisfaction

For each category, studies were subdivided based on: (A) impact; and (B) implementation: these may demonstrate impact, but specifically include studies where JAG tools/recommendations have resulted in quality improvement of patient or trainee-centred services.

For duplicate abstracts, either the full paper was referenced or the earliest instance was selected.

Data extraction

Data from eligible articles were extracted into tables to summarise the literature review. Column headings included: (A) first author, (B) year of publication (full papers marked with an asterisk), (C) country, (D) JAG division (ie, training/service accreditation/SAAS), (E) study design, (F) outcomes, (G) results/conclusion and (H) impact of JAG.

Results

The search strategy yielded 887 results from full publications and conference proceedings. After removing 687 inappropriate results, 80 duplicate and 2 irrelevant studies, 118 publications (43 papers and 75 conference abstracts) were reviewed (figure 5). These were categorised according to the impact of JAG.

Figure 5.

Results from the literature search.

Impact on quality of care

Thirty-four studies were identified that related to quality of care (online supplementary appendix table 1),^14–47 with 31 relating to impact and three on implementation.

Key performance indicators

Evidence of variable practice between endoscopists has led to renewed focus on KPIs, which may be used as a surrogate marker for quality and safety in endoscopy. These may assess direct (eg, complication rates) or indirect effects on patient outcomes, for example, endoscopist’s adenoma detection rates as a surrogate for lower post colonoscopy CRC rates and mortality,⁴⁸ longer colonoscopy withdrawal times correlating with adenoma detection rate (ADR)⁴⁹ and higher CIR with reduced discomfort and sedation use.⁵⁰ As such, audits involving KPIs allow services to safeguard patient outcomes and benchmark performance.

From the literature review, 20 studies identified were audits of JAG standards, which are aligned with BSG recommendations. Fifteen were based on colonoscopy KPIs, four on endoscopic retrograde cholangiopancreatography (ERCP)16 20 38 39 and two on gastroscopy.^{15 37} Quality measures of published audits mainly comprised procedural completion rates14 16 21–23 28 29 33 34 38 39 and complications^{15 16 20} and also included comfort scores,^{37 45} gastric ulcer follow-up,^{31 42} antibiotic concordance in percutaneous endoscopic gastrostomy (PEG),¹⁸ quality of bowel preparation^{22 51} and colonic biopsies for diarrhoea.⁴¹ Seven studies pertained to Bowel Cancer Screening (SAAS), reporting higher quality of care in accredited endoscopists compared with non-accredited counterparts in terms of CIR,19 27 43 52polyp detection rates (PDR),^{17 19 43} ADRs,^{19 27 43} adherence to tattoo placement^{17 25} and polyp retrieval rate.⁴³

National improvements in patient outcomes

Several national studies have attributed improvements in care to JAG. In the second UK colonoscopy audit (n=20 085) performed in 2011,³⁰ Gavin et al reported significant improvements in KPIs since the previous audit (table 3), including improvements in CIR from 76.9% in 1999 to 92.3% in 2011. The authors credited improved performance to advances in quality of training and service accreditation. Similar findings were reproduced in a retrospective comparison of colonoscopy KPIs between 2004 and 2012.³² Valori et al³⁶ studied a composite measure coined the performance indicator of colonic intubation (PICI) which incorporated CIR with safe sedation use (midazolam <2 mg) and acceptable (mild–moderate) discomfort scores. On multivariate analysis, endoscopy performed within a JAG-accredited unit was an independent predictor of PICI (OR 1.26; 95% CI 1.16 to 1.35) and higher levels of JAG training were also associated with PICI. Britton et al³⁵ observed that in the UK, there was a lower postcolonoscopy colorectal carcinoma incidence compared with other countries. The authors suggested that advances in quality, driven by JAG and BCSP, may have contributed towards this finding.

Table 3.

Comparisons in colonoscopy KPIs between the two national colonoscopy audits

	Bowles4	Gavin30
Year performed	1999	2011
Procedures	9223	20 085
Caecal intubation rate (%)	76.9	92.3
Polyp detection rate (%)	22.5	32.1
Conscious sedation (%)	94.6	88.9

Evidence of implementation processes

Three studies described implementation processes related to quality of care. Dewi et al introduced root-cause analyses of perforations following screening colonoscopy, whereby suboptimal management was addressed with individual feedback and training.⁴⁶ Thompson et al assessed the robustness of the 90% CIR standard by factoring in the conversion of intended colonoscopy procedures to sigmoidoscopy.⁴⁷ Conversion to flexible sigmoidoscopy (FS) occurred in 4.7%, with the majority lacking valid reasons, which artificially boosted CIR. Falvey et al presented data regarding implementation of nurse-assisted reporting of comfort scores,⁴⁵ which was associated with significantly higher discomfort than endoscopist-reported scores. This enabled outlier performance to be identified and has since been adopted routinely as a QA measure.

Recently, two national recommendations have been released in partnership with JAG. These include the use of Buscopan in endoscopy procedures and the minimal and aspirational standards for colonoscopy practice and quality of service across the UK.^{44 53}

Impact on quality of service

Thirty-one studies (online supplementary appendix table 2) were relevant to the impact of JAG on quality of endoscopy services.^{2 54–83} Twenty-two were relevant to implementation of service, with the majority being GRS based.

Impact of the GRS

An overview of JAG unit accreditation has been described in detail by Stebbing,² who also reported improvements in quality of service. Valori et al⁶⁴ expanded on the national impact of the GRS by demonstrating a reduction in patients waiting >6 weeks for endoscopy from >250 000 in 2004 to <2000 in 2008. The components of the GRS have since been validated from a patient perspective.⁵⁷ Moreover, from a survey involving endoscopy staff ranging from administrative to nursing roles, >75% of staff felt that JAG recommendations had improved quality of service and care.⁶⁰

Other service evaluations

Service evaluations were reported in several UK studies. Challand et al⁵⁴ combined quality of care (CIR) and cost-effectiveness of a service (points per list) and found that this composite endpoint was met in 30% of endoscopists, with higher rates in trainer endoscopists and those with higher volumes. One study audited procedures that were overdue for surveillance,⁶¹ reporting high rates of procedures requested at inappropriate intervals and implemented vetting procedures to rationalise surveillance requests. Another reported implementation of a propofol-based service to meet increasing service demand.⁷² Two studies reported success with unit accreditation in the novel settings of community-based endoscopy⁷⁶ and within the independent sector,⁶³ both complemented by positive patient satisfaction. Three studies provided evidence to support the role of accreditation/GRS in paediatric services.^{70 82 83} Implementation of strategies to improve patient outcomes within BCSP have also been presented.^{68 71 77}

Impact on services abroad

Use of the GRS is not restricted to the UK. The JAG International subcommittee responds to international interest in JAG tools/services and facilitates implementation processes in those countries. The international impact of JAG is summarised in table 4. Adoption of the GRS has been described in studies from Ireland,⁷⁸ Canada,^{66 73 79} the Netherlands,^{69 84} New Zealand,⁸⁵ Iraq⁷⁵ and Malawi⁸⁰ following collaborative efforts with JAG International, which has been successfully used to benchmark quality of service and identify areas for improvement. In a service development initiative supported by BSG and JAG International in Malawi, Nyahoda et al⁸⁰ presented their experience of implementing a GI bleeding service, and Geraghty et al⁸⁶ described how a training network was established in three Malawian hospitals using JAG-based training, DOPS assessments, development of a local faculty and the application of a modified GRS for service evaluation. A similar international collaboration led to the introduction of a GRS in Iraq.⁷⁵ Carpentier et al provided evidence of test–retest reliability of the Canadian GRS (GRS-C), modified from the UK version, with evidence of quality improvement.⁷⁹ Use of the GRS-C has been reviewed in a 2013 publication,⁷⁴ which outlined its use in 39 Canadian units. Herein, 35% achieved improvements in 8/12 domains over a 2-year cycle, with 15% reporting improvements in wait times. GRS has also been referenced in international research, including the development of a novel comfort score⁵⁵ and the assessment of patient derived indicators of quality of care.⁵⁶

Table 4.

International use and involvement of JAG services (based on JAG International Committee minutes from July 2016) – courtesy of JAG office

Nation	Level of interaction with JAG
Australia	JAG are advising Queensland Nursing and Midwifery Office regarding roll-out of training programme.
Canada	Services have permission to use offline version of DOPS forms. Implementation of Canadian GRS.64
Hong Kong	Implementation of nurse endoscopist bowel cancer screeners trained via JAG curriculum.134
Iraq	Benchmarking of Iraqi endoscopy services using GRS.69
Ireland	Services completed GRS census (36 public and 5 private endoscopy services registered with JAG). Majority of services working towards accreditation. Eight services accredited.
Malawi	JAG supported training courses run in Malawi.70 71
Netherlands	Use of GRS tools.116
New Zealand	Services previously completed GRS census. On hold pending conversations with the Ministry of Health regarding future direction of work.68
Norway	Interest in GIN courses and e-Portfolio, with members of Norway screening programme attending a GIN training the nurse trainer course.
Poland	JAG-based Training Colonoscopy Leaders Course.38
Portugal	JAG supported colonoscopy upskilling and Training and Trainer courses (2015).
Saudi Arabia	King Abdullah Medical City Hospital leads approached JAG to ask about possibility of becoming JAG accredited. Conference call held to scope work and a proposal has been made to offer access to GRS and support via calls and documentation in the first instance.
Singapore	Services have permission to use offline version of DOPS forms.
South Africa	JAG supported colonoscopy upskilling and Training and Trainer courses (2015/2016).
Spain	A trial version of the GRS was requested by Madrid Hospital and set up.
USA	JAG setting up teleconference with representatives from the University of Colorado regarding EUS and ERCP training.

EUS, Endoscopic Ultrasound; ERCP, endoscopic retrograde cholangiopancreatography; DOPS, direct observation of procedural skills; GIN, GI Endoscopy for Nurses; GRS, Global Rating Scale; JAG, Joint Advisory Group on Gastrointestinal Endoscopy.

Impact on quality of training

Fifty-three articles relevant to the impact of JAG on training were identified (online supplementary appendix table 3),1 6 7 12 86–130 of which 29 assessed impact and 24 described training-related implementation. Two in-depth reviews of the role of JAG in endoscopy training are provided by Dunckley¹ and Anderson.^{1 12} The majority of studies relate to QA of training.

Trainee outcomes

Haycock et al surveyed changes in quality of training between 2002 and 2007 and reported significant improvements in standards of teaching, reduced trainee complication rates and increased trainee satisfaction, correlating with JAG’s impact.^{88 89} Dharmasiri et al⁹³ reported high rates of trainee satisfaction with the e-certification system. Other surveys at regional and national levels have exposed disparities in training satisfaction by specialty87 95 103 107 and training region.^{101 105} In the recently published 2016 BSG trainees survey,¹³⁰ 85% were satisfied with the level of supervision during endoscopy training, with 12.5% reporting no access to regular training lists. Conflicting on-call commitments,¹³⁰ competition for and absence of training lists are cited as contributory factors,^{95 104 130} despite the training domain of the GRS, which places onus on individual units to ensure sufficient training opportunities. To overcome these challenges, several studies have evaluated innovative approaches to improve trainee exposure. Walker et al¹¹⁵ described the successful implementation of a dedicated training e-booking system, which improved the uptake of dedicated training lists from 18% in 2007 to 61% in 2010. Similarly, by implementing generic training lists, Lamb et al¹¹⁷ reported increases in mean training lists from 7.8 to 13.6 lists per quarter, which was associated with improvements in trainee KPIs and DOPS counts.

Upskilling interventions

Two studies reported improvements in endoscopist KPIs after attendance on a JAG-based course. Hussain et al¹³¹ evaluated the performance of four certified endoscopists before and after attendance at a JAG-certified advanced colonoscopy course, specifying improvements in polyp retrieval and biopsy practice for chronic diarrhoea, although improvements in CIR (88%–93%) and minimal–mild discomfort scores (71%–82%) were not statistically significant. In an international study involving JAG faculty members, Kaminski et al¹³² identified endoscopy leaders from 40 Polish bowel cancer screening centres with suboptimal ADR and randomised them to a Train-Colonoscopy-Leaders (TCL) programme with a 2-day hands-on component or feedback only. The study analysed 24 582 colonoscopies performed by 38 leaders and 56 617 colonoscopies performed by 138 endoscopists at participating centres. The TCL arm had larger improvement in ADR than the feedback group in both early (OR 1.61; p<0.001) and late (OR 1.35; p=0.004) postintervention phases.

Non-medical endoscopists

In the UK, non-medical (nurse) endoscopists benefit from the flexibility of dedicated immersion training without competing medical and on-call commitments. The Health Education England sponsored clinical endoscopist programme has led to 31/40 non-medical endoscopists (78%) achieving gastroscopy or sigmoidoscopy certification within a 7-month timeframe.^{123 133} In a randomised trial from Hong Kong, non-medical endoscopists trained according to the JAG curriculum had superior ADR during screening colonoscopies compared with medical endoscopists (43.8% vs 32.7%). The authors concluded that proper training, that is, completion of well-established training programmes such as JAG, may equip nurses with the competencies for screening colonoscopy.¹³⁴

Competency-based certification

Effective training is key for competent unsupervised practice. The success of JETS implementation has been well characterised.6 7 114 116 The robustness of endoscopy certification has been evidenced by precertification KPIs during training¹⁰² and postcertification KPIs of independently performing specialty trainees.^{7 24} Based on a 2011 analysis of JETS entries, 28% of specialist trainee procedures were logged as service lists.⁹² Hence, supporting trainees to achieve certification enables effective contribution to an endoscopy service.

Competency assessment tools

The role of DOPS and DOPyS as competence-assessment tools have been evaluated since their introduction in 2004.⁵ DOPS were first assessed in the context of BCSP, which showed validity and reliability.¹²⁰ In order to standardise polypectomy assessment, DOPyS were developed,¹¹⁸ validated^{118 119} and integrated into colonoscopy certification criteria in 2011, with subsequent improvements in trainee polypectomy exposure and standards.¹²⁹ This has provided a much-needed framework for polypectomy assessment. A recent survey involving 610 colonoscopists from 19 countries unearthed significant variation in polypectomy training internationally.¹¹³ Only 4 of the 19 countries, including UK, had specific guidelines for polypectomy training and competency assessment. The impact of the post-July 2016 changes to DOPS has also been published. Implementation of novel DOPS for GI bleeding¹²¹ and percutaneous endoscopic gastrostomy (PEG) have been described.¹³⁵ The endoscopic non-technical skills (ENTSs) domain introduced into new DOPS/DOPyS has also been validated.¹²⁶ The changes in DOPS scoring from a performance-based to supervision-based scale have improved the quality and validity of assessment tools.¹²⁷ From a research perspective, DOPS and DOPyS have been integral to studies which appraise the impact of practical and simulator based training,96 99 100 106 108 110 112 136–138 thereby contributing to current understanding of optimal training methods in endoscopy. Similarly, interrogation of the JETS e-portfolio has enabled learning curve analyses for competence in gastroscopy,¹¹¹ colonoscopy⁹⁸ and polypectomy,¹⁰⁹ which inform trainees, trainers and SACs regarding length of training and variation in learning curves.

Direction of training

Several publications provide insights into the future direction of training. As gastroscopy certification does not ensure competence in managing GI bleeding, certification specific for endotherapy has been proposed.^{128 130} In response to trainee dissatisfaction regarding exposure to training,^{130 139} the JAG QA-T committee has outlined strategic measures,¹³⁰ including placing further increasing emphasis on GRS to improve unit-level training delivery and appraisal of measures to reduce time to competency.¹³⁰ A trial roll-out of accelerated training to specialty trainees has been effective and well received.¹²² ‘Immersion training’, where blocks of time are dedicated to endoscopy alone, is being considered.¹³⁰ These approaches may be paired with new e-learning tools to accelerate development of non-technical competencies such as lesion recognition.¹²⁴ However, the single innovation likely to have greatest impact on training is the impending National Endoscopy Database (NED).¹²⁵ NED has been designed to autopopulate KPIs from endoscopy reporting systems directly into future iterations of JETS and allow benchmarking of trainee learning curves nationwide.

Discussion

Summary

This literature review provides evidence that supports the impact of JAG on quality of care, service and training in UK endoscopy over the last decade. The majority demonstrate a positive impact of JAG. Of note, comparisons of performance metrics between the two national colonoscopy audits^{4 30} and the fall in national waiting times⁶⁴ are testament to advances in quality at both endoscopist and unit levels.

The promotion of the QA framework has underpinned the success of JAG. QA in endoscopy is reliant on: (1) definition of quality standards, (2) measuring quality by comparing against quality standards, (3) methods for improving quality and (4) providing incentives for participation.¹ Continuous audit and quality improvement is integral to the QA process and is supported by the centralised JAG infrastructure of JETS, GRS and SAAS. Importantly, a substantial proportion of publications (42%) in this review reported process implementation. Thus, it is encouraging that our review has uncovered the breadth of innovative approaches undertaken to accomplish and surpass minimum JAG standards.

Moreover, the international impact of JAG has also been recognised through involvement with at least 16 countries (table 4), with further involvement facilitated by international affiliations at individual institutions. Although the use of JAG tools, for example, DOPS, DOPyS and GRS, have supported research abroad, these have also benefited from international validation.

Strengths and limitations

This literature search was designed to be an objective and comprehensive summary of publications related to JAG. As the majority of publications were in abstract form, one limitation of this review is the lack of methodological rigour for selecting studies. The majority were retrospective, with only two abstracts excluded due to poor quality. However, some of these retrospective studies are well designed and have included patient numbers in excess of 100 000 (online supplementary appendix table 1). We acknowledge that there is crossover between JAG quality standards with other national guidelines. As some standards are ubiquitously recommended in guidelines, for example, CIR and ADR, it would not be appropriate to solely attribute these to the impact of JAG. Hence, the search strategy was limited to JAG-relevant terms and some studies may not have been captured. Next, search results were arbitrarily categorised into groupings of impact on care, services and training, which was intended to demonstrate outcomes on patients, units and trainees, respectively. We acknowledge there may be considerable overlap between the search results. This was also true for the subgrouping of studies according to impact and/or implementation. The relative lack of implementation-based data on patient outcomes (3/34 studies) may either indicate potential difficulties for units to implement change, which is a well-recognised barrier, or reflect publication bias, whereby unsuccessful interventions may be withheld. Sharing examples of successful (and failed) implementation measures are necessary to drive quality improvement, a strategy jointly promoted by JAG and the BSG Endoscopy Quality Improvement Project.

Future directions

In addition to future initiatives described above, upcoming agenda include a review of existing certification pathways to ensure these remain current, evidence based and supported by competency-based milestones. Plans to introduce certification in additional modalities of GI bleed haemostasis, ERCP, endoscopic ultrasound (EUS) and capsule endoscopy are also underway, supplemented with procedure-specific curricula, e-learning and JAG-approved courses. The demand for a JETS-like platform for endoscopy nurses to record competencies has prompted the development of JETS Workforce, an e-portfolio platform specific for endoscopy nurses for documenting assisting competencies that could be used to support revalidation. Pressures faced by endoscopy units,¹⁴⁰ notwithstanding the imminent plans to introduce faecal immunochemical testing¹⁴¹ and lower the bowel cancer screening age to 50 years,¹⁴² are likely to instigate a review of workforce requirements, with emphasis on recruitment and upskilling of existing Bowel Scope practitioners towards BCSP colonoscopy accreditation.

NED, which is hosted by JAG, went live in April 2018. The vision of NED is to autopopulate performance metrics of individual endoscopists to centrally benchmark performance, summate unit-level data pertaining to the Clinical Quality domain of the GRS and import trainee metrics directly into JETS. This will eliminate data entry bias and allow for reliable and detailed assessment of endoscopy performance and service activity. The NED promises a data-rich platform for research on endoscopy-based metrics. This is likely to boost the research impact of JAG and will further extend JAG’s influence as an international model for facilitating endoscopy QA.

Finally, in recognition that errors in endoscopy are common and under-reported,¹⁴³ JAG has announced a 5-year strategy to Improve Safety and Reduce Error in Endoscopy,^{144 145} which aims to improve training and the practice of error reporting, learning from error and implement system-level approaches for safety and performance improvement. This workstream will use the GRS and NED infrastructures and complement JAG’s aspirations to improve communication with endoscopy services to disseminate learning and support services in the UK, and in renewing commitments for placing patient safety and clinical quality at the forefront of endoscopy practice.

Conclusion

The UK experience shows that it is possible to achieve a transformation in quality, safety, patient experience and training with a strategic, centrally led, and modestly resourced approach.