Scope to improve: a multi-centre audit of 16 064 colonoscopies looking at caecal intubation rates, over a 2-year period

Abstract

Objective

Colonoscopy is the ‘gold standard’ assessment for large bowel mucosal pathology, but a complete examination is essential. The first national colonoscopy audit carried out in 1999 demonstrated caecal intubation rates (CIRs) of 56.9%. As a result, the Joint Advisory Group (JAG) on gastrointestinal endoscopy launched a programme of continuous quality improvement. JAG recommends that practitioners undertake 100+ procedures per annum with a target CIR of 90%. This current audit provides an assessment of performance against this quality standard.

Design

Data were collected from all procedures undertaken in 2008–2009 from six hospitals across three English regions.

Results

16064 colonoscopies performed: CIR = 90.57% (95% CI 90.11% to 91.01%). Operators doing 100+ procedures per annum, CIR=91.76% (91.24% to 92.25%). Operators doing <100 procedures per annum, CIR=87.77% (86.82% to 88.67%). Gastroenterologists, CIR=91.01% (90.32% to 91.70%). Surgeons, CIR=91.03% (90.27% to 91.79%). Other practitioners, CIR=81.51% (78.79% to 84.22%). Bowel cancer screening programme (BCSP) colonoscopies, CIR=97.71% (97.07% to 98.34%). Non-screening colonoscopies, CIR=88.31% (95% CI 87.68% to 88.94%).

Conclusion

This audit of 16064 colonoscopies across three regions demonstrates aggregated achievement of the CIR quality standard. However, there is a significant performance gap when comparing BCSP colonoscopists with non-screening colonoscopists and the overall CIR of >90% is supported by the volume of BCSP colonoscopy.

Endoscopists performing low volume colonoscopy (<100 per annum), have CIR of <90%. Endoscopists with low volume practice who do not meet the quality standards should engage in skills augmentation plus further training and increase volume of colonoscopy with local mentorship, or stop performing colonoscopy.

Introduction

Colonoscopy is the ‘gold standard’ examination for assessment of large bowel mucosal pathology and has superseded barium enema examinations as the modality of choice for detection of colorectal cancer (CRC).¹ Colonoscopy underpins the National Health Service Bowel Cancer Screening Programme (BCSP), which employs a faecal occult blood test strategy with the prevalent round rolling out between 2006 and 2010. Increased numbers of colonoscopy are associated with a reduction in mortality from CRC for some populations.²

The quality of colonoscopy is important. Analysis of a Canadian population diagnosed with colorectal cancer within 3 years of undergoing colonoscopy (termed as having postcolonoscopy colorectal cancer (PCCRC)) reveals that colonoscopy performed by practitioners with a higher caecal intubation rate (CIR) (a marker of complete colonoscopy) were less likely to have PCCRC.^{3 4} The cause of PCCRC is multi-faceted, poor bowel preparation being a factor.⁵ Poor bowel preparation is associated with lower CIRs. This emphasises the importance of CIR as both a marker of completion and a supportive quality measure of colonoscopy — reflecting good bowel preparation, high quality complete examinations with low rates of PCCRC.

The Joint Advisory Group (JAG) on gastrointestinal endoscopy, an inter-collegiate body responsible for standards, quality and training, has issued guidance for CIRs of 90% or higher on an intention-to-colonoscope basis.

In 2004 the outcomes of a prospective UK audit of 9223 colonoscopies performed over a four month period in 1999 demonstrated a completion rate of 76.9%.⁶ The only reliable markers for completion of colonoscopy are intubation of the terminal ileum or visualisation of the ileocaecal valve. When adjusted for one or both of these landmarks, the CIR was recorded as 56.9%. The authors of the paper described the performance standard as ‘unacceptably low’. This triggered a programme of continuous quality improvement in colonoscopy by standardising training, peer review and audit of standards at local and national level.

Over the last decade, JAG has presided over changes in training. All trainees are required to attend JAG approved endoscopy skills based training courses, maintain a detailed portfolio and undergo summative assessments, through direct observation of procedure, to gain accreditation in diagnostic colonoscopy. Accredited colonoscopists are audited regularly against these standards with guidance provided to address poor performance standards.⁷ In 2004 the Global Rating Scale (GRS) was introduced as a quality improvement tool for gastrointestinal endoscopy services. The BCSP is a further driver for the improvement of standards with aspiring screeners having to undergo a summative assessment of knowledge and skills to achieve accreditation to provide screening colonoscopy. Continuous Quality Assurance (QA) data is collected and monitored to ensure colonoscopists are meeting the defined BCSP quality standards including a minimum number of screening colonoscopies.⁸

In February 2011, the British Society of Gastroenterology undertook a prospective National Colonoscopy Audit to further assess the quality standards of metrics which included CIR, sedation practice, comfort scores, perforation rate and adenoma detection rate. It looked at over 20 000 procedures across the UK over a 2 week period.

The national audit may not accurately reflect actual practice due to the Hawthorne effect — the phenomenon of individuals changing their behaviour as a consequence of their performance being monitored.⁹ In addition, although the total numbers of procedures and operators are large, the number of procedures performed by individual colonoscopists is small.

This large longitudinal audit of 16 064 colonoscopies performed over a 2 year period from six centres in three regions provides a measure of performance against the quality standard of CIRs over a long period of time. This audit looks at a small number of operators over a 2-year period. The results will be free of the Hawthorne effect and give an accurate reflection of CIR and the impact of training and performance standards over time.

This audit aims to demonstrate colonoscopists' performance against JAG's 90% CIR quality standard analysis of operator subgroups. It was performed (by grade/speciality) to highlight those subgroups that would benefit from JAGs supportive recommendations for failing to meet the CIR quality standard.⁷

Methodology

Prospectively recorded data from all colonoscopies undertaken (symptomatic, surveillance and BCSP procedures) were retrospectively collected from six hospitals in three regions (North Northamptonshire, Leicestershire and Coventry/Warwickshire).

Data were collected by use of audit tools for Unisoft software (Unisoft Medical Systems, Middlesex UK) and Fujinon ADAM software (Fujinon (Europe) Germany).

The audit took place from 1 January 2008 to 31 December 2009. Data has been collated and analysed using Microsoft Excel software.

The data recorded included:

• ■Name and grade of operator
• ■Number of procedures performed by operator
• ■CIRs (on an intention to colonoscope and intubate the caecum basis)
• ■CIRs were unadjusted for preparation, obstructing lesion, colitis, etc.

Unisoft software allows a colonoscopy to be recorded as ‘completed’ or as reaching the following landmarks: terminal ileum/neo-terminal ileum/caecum/anastomosis. Fujinon ADAM software only records the landmark reached. For the purpose of this audit, only procedures recorded as reaching agreed landmarks were counted as representing true markers of completion and used in the calculation of a CIR (CIR%). Procedures only recorded as ‘complete’ were not seen as providing sufficient evidence and therefore recorded as incomplete for the purposes of this audit.

Data from BCSP were collated from one of the screening centres. The number of operators was recorded; totals do not necessarily tally for this data as some operators have performed colonoscopies at multiple sites (trainees on rotation around the region and BCSP colonoscopists who have lists at multiple sites).

Results

Between 1 January 2008 and 31 December 2009 there were 16 064 colonoscopies undertaken across the six sites. Table 1 shows a breakdown of the number of colonoscopies undertaken and the CIR by site and by year. For each CIR, 95% CI have been calculated and have been represented in a Forest plot (figure 1).

Table 1.

Summary data from all sites

	No. operators	No. procedures	TI/caecum	‘Completed’	Failed	CIR%	95% CI
SITE A 08	18	1169	1068	0	101	91.36%	89.61 to 92.84%
SITE A 09	20	1306	1192	0	114	91.27%	89.62 to 92.68%
SITE A total	24	2475	2260	0	215	91.31%	89.61 to 92.84%
SITE B 08	28	3430	3095	5	330	90.23%	89.19 to 91.18%
SITE B 09	28	4023	3706	0	317	92.12%	91.25 to 92.91%
SITE B total	36	7453	6801	5	647	91.25%	90.59 to 91.87%
SITE C 08	53	2965	2660	57	248	89.71%	88.57 to 90.76%
SITE C 09	58	3171	2828	96	247	89.18%	88.05 to 90.22%
SITE C total	68	6136	5488	153	495	89.44%	88.65 to 90.18%
All sites 08	96	7564	6823	62	679	90.20%	89.51 to 90.85%
All sites 09	102	8500	7726	96	247	90.89%	90.26 to 91.49%
All sites total	120	16064	14549	158	1357	90.57%	90.11 to 91.01%

CIR, caecum intubation rate; TI, Terminal ileum.

Figure 1.

Forest plot of data from Table 1

Analysis of the data represented on table 1 and figure 1 shows:

The overall CIR for 16 064 colonoscopies over 2 years was 90.57% (95% CI 90.11% to 91.01%).

CIRs were similar across the three regions. The 153 colonoscopies recorded as ‘complete’ at one centre reduces the CIR to 89.44% (95% CI 88.65% to 90.18%), which places their performance below other centres.

Analysis of the data by specialty, grade and operators performing <100 colonoscopies per annum compared with those doing 100 or more colonoscopies per annum (table 2).

Table 2.

Sub-analysis by operator and volume

	No. operators	No. procedures	TI/caecum	‘Completed’	Failed	CIR%	95% CI
Gastro trainees	40	4387	3966	7	414	90.46%	89.67 to 91.14%
Gastro Cons	18	4673	4253	143	277	91.01%	90.32 to 91.70%
Surgical trainees	17	558	494	4	61	88.37%	86.13 to 90.61%
Surgical Cons	22	3857	3511	4	342	91.03%	90.27 to 91.79%
Assoc Specialist	2	1673	1533	0	140	91.63%	90.51 to 92.75%
Nurse Endoscopist	7	358	338	0	20	94.41%	92.41 to 94.41%
Other	14	557	454	0	103	81.51%	78.79 to 84.22%
<100 pa		4793	4207	64	522	87.77%	86.99 to 88.55%
100+ pa		11271	10342	94	835	91.76%	91.33 to 92.19%

CIR, caecum intubation rate; TI, Terminal ileum

Table 2 (represented in the Forest plot – figure 2) shows the CIRs for gastroenterology trainees (90.46%), gastroenterology consultants (91.01%), surgical consultants (91.03%) and associate specialists (91.63%) are achieving the 90% threshold quality standard. Surgical trainees (88.37%) are below the 90% but the 95% CI cross the 90% threshold (95% CI 86.13% to 90.61%).

Figure 2.

Forest plot of data from Table 2

Nurse endoscopists accounted for a small number of procedures in the audit (358 colonoscopies/7 operators) but achieved a CIR of 94.41% (95% CI 92.41% to 94.41%) (median 51, mean 71.6, range 2–243).

An area of concern identified was the lack of attainment of the quality standard by non-gastrointestinal specialists (geriatricians, paediatricians, acute medical trainees and general practitioners) who typically performed small numbers of procedures (557 colonoscopies/14 operators) and have substandard CIR of 81.51% (95% CI 78.79% to 84.22%) (median 16, mean 39.8, range 1–127).

In this audit, 11 271 colonoscopies were performed by endoscopists who carry out more than 100 procedures per year (in line with JAG recommendations), they had a CIR of 91.76% (95% CI 91.33% to 91.19%). 4793 colonoscopies were performed by endoscopists who carried out less than 100 procedures per year. They did not meet the quality standard with CIR of 81.77% (95% CI 86.99% to 88.55%). When comparing these two groups, the difference is significant (p=<0.0001).

Table 3 shows comparison of BCSP colonoscopies versus non-BCSP colonoscopies (BCSP data only available for two regions) and a comparison of CIRs between gastroenterologists and gastrointestinal surgeons.

Table 3.

Sub-analysis – BCSP vs non-BCSP and GI surgeons vs gastroenterologists

	No. operators	No. procedures	TI/caecum	‘Completed’	Failed	CIR%	95% CI
BCSP	5	1526	1491	2	33	97.71%	97.07 to 98.34%
Non-BCSP	85	7085	6257	151	677	88.31%	87.68 to 88.94%
Gastro	58	9060	8219	150	691	90.72%	90.21 to 91.22%
Surgical	39	4416	4005	8	403	90.69%	89.97 to 91.41%

CIR, caecum intubation rate; Ti, Terminal Ileum

Specifically the data in table 3 (represented in the Forest plot – figure 3) shows:

Figure 3.

Forest plot of data from Table 3

When comparing the 1526 colonoscopies performed within the BCSP to the 7085 non-BCSP procedures performed in the symptomatic service at the same sites, there is a significant difference in CIRs with a difference approaching 10% (97.71% vs 88.31%) (p=<0.0001).

Gastroenterologists and GI surgeons had similar CIRs (90.72% vs 90.69%); however, gastroenterologists performed twice as many colonoscopies as gastrointestinal surgeons (9060 vs 4416).

Discussion

In the original 1999 audit, 9223 procedures were analysed with a CIR of 56.9%.⁶ This audit of 16 064 procedures from the latter part of the decade shows a marked improvement, with a CIR of 90.57% (95% CI 90.11% to 91.01%). This improvement in CIR indicates an improvement in the quality of colonoscopy. A possible explanation for this improvement is structured training, audit and the implementation of GRS standards, overseen by JAG. To fully assess this, an assessment of adenoma detection rate would be essential.

There remains scope for improvement. JAG's guidance that all independent practitioners should aim to undertake 100 procedures per annum is supported by our analysis that endoscopists who achieve this standard have higher CIR (which is statistically significant, 91.76% (95% CI 91.33% to 92.19%) versus 87.77% (95% CI 86.99% to 88.55%) (p=<0.0001). There is strong evidence supporting volume and outcome¹⁰ and this has been apparent in the reorganisation of vascular services and other complex specialities.

Within this audit a correlation between volume and CIR can be illustrated. The seven operators who carried out >500 procedures each (six of which were BCSP screening colonoscopists) performed 5380 procedures in 2 years (mean=768.6, median=623, range=545–1257). Their combined CIR% for 5380 procedures (screening and symptomatic procedures) was 94.61% (95% CI 93.97% to 95.18%, range=91.24–96.68%). This CIR% is much higher than that for the whole audit (90.57%).

In the BCSP, colonoscopies are only undertaken by operators who have passed a summative assessment test (known to some as ‘the driving test’). The patients are highly motivated and the procedures are performed on dedicated lists. BCSP colonoscopy standards could now be seen as the new gold standard.⁸ Comparison between BCSP colonoscopies and all other colonoscopies undertaken showed a 10% difference in CIR (97.71% (95% CI 97.07 to 98.34) vs 88.31% (95% CI 87.68 to 88.94) (p=<0.0001)). It is also notable that when removing BCS colonoscopist performance from the audit, the CIR drops below 90% suggesting that non-BCSP colonoscopists are not yet meeting the required standard and that BCSP is ‘upstaging’ the quality of colonoscopic practise as a whole.

Analysis of CIRs by grade and primary specialty of the endoscopist demonstrated the following CIR: consultant surgeons (91.03%), consultant gastroenterologists (91.01%), gastroenterology trainees (90.40%), associate specialists (91.63%) and nurse endoscopists (94.41%), all meet JAG's guidance of a CIR of 90%.

Surgical trainees are nearly meeting the standard (88.37%) with 95% CI crossing the 90% JAG standard (86.13–90.61).

A concern identified is the CIR of ‘other’ endoscopists (paediatricians, geriatricians, acute medical trainees and general practitioners), which is below the mandated quality standard set by JAG at 81.51% (95% CI 78.79 to 84.22). Typically the volume of procedures is low (mean of 19.4 per annum). A clear association between volume and improved outcomes has been demonstrated for a range of medical interventions, there is evidence supporting the position that non-GI endoscopists have higher rates of postcolonoscopy CRC and as discussed earlier, incomplete colonoscopy is associated with PCCRC.^{3 5 11} This compelling picture calls into question whether non-gastrointestinal operators should be providing colonoscopy at all.

In an effort to improve quality and outcomes for patients we would propose the following bold steps:

• ■JAG's guidance for operators to undertake 100+ procedures per annum should be adhered too, ultimately audit of CIRs to JAG standards should guide endoscopy unit leads on whether operators need re-training (or ultimately to stop undertaking colonoscopy if re-training is unsuccessful).⁷
• ■Practitioners who do not routinely work with patients who require colonoscopy (general practitioners, acute medical trainees, paediatricians and geriatricians) and are not maintaining minimum volume or CIR should follow JAG guidance on operators failing to meet standards or should stop undertaking colonoscopy.⁷
• ■The application of JAG's guidance on colonoscopists failing to meet CIR quality standards must be the responsibility of individual endoscopy units. This large study should serve to highlight which groups of operators may be struggling and that JAG provides supportive guidance to deal with these operators.

There are limitations to this audit:

As discussed, CIR is a quality marker of complete colonoscopy but it is not the only marker. Adenoma detection rate is extremely important in this respect and to comment fully on the quality of colonoscopy, CIR would need to be represented in conjunction with adenoma detection rate and comfort scores and sedation practice.

The software does not accurately allow the recording of caecal intubation for training procedures (who completed the procedure — trainee or supervisor?), Unisoft software does not record this and it is not a mandatory field to complete for Fujinon ADAM reports. This does limit the conclusions made about the performance of trainees as their CIR may be supported by the performance of the trainer supervising the procedure (and potentially completing the procedure if the trainee fails to do so).

The electronic reporting systems do not record if a photograph of the caecum or ileocaecal valve has been taken (though a photograph may be part of the report issued for the procedure). This omission results in the recording of reaching the caecum being reliant on the honesty and probity of the individual composing the report. As photographic evidence is a JAG requirement, software should be modified to allow this to be recorded and audited.

What is already know on this subject.

• ▶Caecal intubation rate (CIR) is a key performance measure of the quality of colonoscopic examination. In the first national colonoscopy audit in 1999, the CIR of 9223 colonoscopies was 56.9% —described as ‘unacceptably low’.
• ▶Low CIRs are associated with post colonoscopy colorectal cancer (colorectal cancer diagnosed in patients who have undergone a colonoscopy in the past).
• ▶Since the first national colonoscopy audit, the Joint Advisory Group (JAG) on gastrointestinal endoscopy launched a programme of continuous quality improvement, standardising training, peer review and audit. JAG recommends practitioners undertake at least 100 procedures per annum with target CIRs of 90%.
• ▶A second national colonoscopy audit was conducted over a 2-week period in spring 2011. It looked at over 20 000 colonoscopies performed by 2700+ colonoscopists from over 300 sites. Publications of results are awaited.

What this paper adds.

• ▶This audit of 16 064 colonoscopies of 120 colonoscopists from six sites has a CIR of 90.57% — meeting JAG's quality standard.
• ▶Bowel cancer screening colonoscopy metrics should be considered the new ‘gold standard’, these high quality examinations are supporting the CIR of all procedures to above 90%.
• ▶Non-gastrointestinal operators and those who are undertaking less than 100 colonoscopies per annum have CIRs of less than 90%.
• ▶For future practice : Non-gastrointestinal operators, those operators undertaking less than 100 procedures per annum and/or those failing to meet the 90% CIR quality standard should increase the volume of their practice, re-train or stop performing colonoscopy.