Ultrasound for the detection of full-thickness rotator cuff tears: the learning curve for an orthopaedic surgeon using a novel training method

Abstract

Background

The present study aims to add to the body of evidence delineating the learning curve for a shoulder surgeon to become proficient in focussed ultrasound for the detection of full-thickness rotator cuff tears, as well as to describe a learning method for this skill.

Methods

Consecutive patients who were scheduled to undergo an arthroscopy for rotator cuff disease were scanned immediately before surgery by a senior shoulder surgeon with limited previous experience of shoulder ultrasound. The presence or absence of a full-thickness rotator cuff tear on scan was compared with intra-operative findings as the gold standard.

Results

Two hundred and ten shoulders were scanned over three equal learning periods. Comparing predictive values from the first to the third training period, sensitivity improved from 0.86 to 0.95, specificity from 0.92 to 0.98, negative predictive value from 0.94 to 0.98, and positive predictive value from 0.82 to 0.95.

Conclusions

The high predictive values obtained in the present study for surgeon-led detection of cuff tears using ultrasound are comparable with those quoted for musculoskeletal radiologists in the literature. The present study adds evidence that a shoulder surgeon can achieve accelerated learning of this skill and offers some potentially time-saving and patient-friendly alternatives to existing guidelines.

Introduction

Disorders of the rotator cuff are a common cause of pain and impairment of upper limb function in patients presenting to orthopaedic surgeons.^1,2 Although clinical history and physical examination are normally accurate in localizing the source of symptoms to the rotator cuff region, differentiation between the common lesions of impingement syndrome, calcific tendonitis and rotator cuff tear may be difficult. The presence of a cuff tear is likely to influence a patient’s treatment pathway, especially following the failure of conservative modes of treatment. The use of imaging to provide accurate diagnosis can enable the clinician to provide specific patient counselling with respect to prognosis and potential outcomes of operative and non-operative management.

In a recent meta-analysis of the literature, a musculoskeletal radiologist who performed shoulder ultrasound was reported to have an equally high value of sensitivity and specificity of 0.95 (95% confidence interval = 0.93 to 0.96) for the detection of a cuff tear.³ Previous studies have indicated that it is possible for an orthopaedic specialist to achieve accuracy levels on a par with these values.^4,5 Therefore, an opportunity exists in selected cases for the treating clinician to streamline the patient pathway by performing ultrasound as an immediate adjunct to clinical examination, potentially avoiding a referral to the radiology department, without significant compromise of diagnostic accuracy.

Little information exists on the quantity and quality of deliberate practice required for a surgeon to develop this focussed musculoskeletal ultrasound skill. Several radiology societies have produced guidelines suggesting that between 150 and 300 supervised practice scans are necessary to gain proficiency in total musculoskeletal ultrasound.^6–8 Such a commitment may be unpalatable to the specialist shoulder surgeon.⁹ Previous studies have suggested that the shoulder surgeon’s unique three-dimensional knowledge of the normal and pathological shoulder, access to numerous patients with rotator cuff pathology, and ability to gain timely feedback from performing arthroscopic or open surgery could result in a faster learning curve.^9,10 Accordingly, the British Elbow and Shoulder Society (BESS) have produced guidelines for an accelerated method of learning to detect rotator cuff tears by ultrasound. These guidelines advise attendance at a shoulder ultrasound course, followed by 20 practice scans under the supervision of a musculoskeletal radiologist, with subsequent engagement and audit of practice in a training period of 50 scans on patients who are immediately about to undergo arthroscopic shoulder surgery.¹¹

The present study aimed to add to the body of evidence delineating the learning curve for a specialist shoulder surgeon to become proficient in focussed ultrasound for the detection of full-thickness rotator cuff tears, as well as to describe an individual learning method for this skill.

Materials and methods

All patients presenting for rotator cuff surgery over a 13-month period (May 2010 to June 2011) under the care of a surgeon with in excess of 10 years experience of specialist shoulder surgery were included in the present study. Patients had previously been assessed and consented for surgery in the outpatient clinic a minimum of 2 months before the day of procedure and listed for ‘shoulder arthroscopy plus subacromial decompression plus or minus rotator cuff repair’. The surgeon’s only previous experience of performing ultrasound was attendance at a 1-day course on shoulder ultrasonography and 20 practice cases without radiology supervision, partially in accordance with BESS guidelines.

On the day of surgery, consent for the surgical procedure was confirmed by the surgeon’s assistant. All patients consented to an ultrasound prior to the start of the operation. Following administration of an ultrasound-guided interscalene block and general anaesthesia, the surgeon performed an ultrasound of the shoulder using the same machine used to provide the nerve block [Digiprince 6600 (Mindray, Shenzhen, China) with 10-MHz linear transducer]. The sequence of ultrasound followed that described by Teefey et al. for the systematic and thorough examination of the rotator cuff and long head of biceps tendons.¹² The ultrasound criteria for the diagnosis of a full-thickness cuff tear, as previously described by Ziegler, included complete cuff nonvisualization, focal discontinuity in homogenous echogenicity without thinning, inversion of bursal contour and/or hyperechoic material in the location of the tendon that failed to move with the humeral head on passive movement.⁴

According the surgeon’s scan, the presence or absence of a full-thickness rotator cuff tear was recorded. When the theatre team positioned the patient, the surgeon consulted the patient’s notes prior to starting the procedure. At arthroscopy, the rotator cuff was always examined in entirety from both the articular and bursal sides (following bursectomy). Immediately after safe completion of surgery, the operative findings were recorded for comparison with ultrasonic findings.

For purposes of statistical analysis, paired scan and operative findings were divided into three equal groups to reflect three ‘training periods’ over time. Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and accuracy were collected for each training period. Statistical comparison of values for each training period was then carried out using PROC FREQ in SAS (SAS Institute Inc., Cary, NC, USA), setting p < 0.05 as the limit for statistical significance.

Results

Two-hundred and ten scans were performed on 210 shoulders in 210 patients over the study period. Therefore, the three training periods for analysis contained 70 patients each. All procedures were started and completed arthroscopically; none were converted to open. Using intra-operative findings as the gold standard, 63 of 210 patients were found to have a full-thickness cuff tear.

The sensitivity, specificity, PPV, NPV and accuracy for surgeon-performed ultrasound for each training period is shown in Table 1. Analysis of the third training period revealed high values of sensitivity (0.95) and specificity (0.98). A general trend for improvement in each value can be seen over time. Statistical comparison of values from the first and third training periods is included in Table 1 and reveals that this improvement reached statistical significance for PPV. Improvements in sensitivity, specificity and accuracy were very close to reaching statistical significance. Lack of statistical significance for NPV may be attributed to a very high value posted in the first training period.

Table 1.

Predictive values for the detection of full-thickness rotator cuff tears over three training periods of 70 scans each.

Training period	Sensitivity	Specificity	PPV	NPV	Accuracy
First	0.86	0.92	0.82	0.94	0.90
Second	0.91	0.96	0.91	0.96	0.94
Third	0.95	0.98	0.95	0.98	0.97
(p-value first versus third)	0.073	0.095	0.019	0.204	0.089

Statistical comparison of values attained in the first and third training period is represented in the bottom row. NPV, negative predictive value; PPV, positive predictive value.

On the basis of these results, a learning curve for sensitivity and specificity is presented in Figure 1. To graphically compare the findings of the present study with the published literature, a coloured box is overlain to delineate the quoted confidence intervals for sensitivity and specificity for specialist musculoskeletal radiologists in the previously mentioned meta-analysis.³ It appears the specialist shoulder surgeon is able to achieve comparable sensitivity and even potentially supersede the quoted specificity by the end of the third training period.

Figure 1.

Graphical representation of the learning curve for scan sensitivity (blue line) and specificity (red line) over the three training periods. The overlain green box represents the 95% confidence interval for the pooled sensitivity and specificity achieved by musculoskeletal radiologists in a recent meta-analysis.³

Discussion

The present study adds weight to the growing body of evidence that specialist shoulder surgeons are able to develop shoulder ultrasound skills for the detection of full-thickness rotator cuff tears rapidly, and that radiology guidelines produced for the training of musculoskeletal ultrasonographers are not applicable to this particular expert group of professionals wanting to perform shoulder-focussed ultrasound imaging.⁹ The results of the present study indicate that a shoulder surgeon can at least match quoted sensitivity and specificity of specialist musculoskeletal radiologists for the detection of full-thickness rotator cuff tears when using this learning method. Furthermore, it appears that the learning curve in the present study has still not reached a plateau, indicating that there is potential for the surgeon to improve accuracy beyond the best results quoted in the literature.

We consider that the reasons for the accelerated learning demonstrated in the present study include the surgeon’s deep understanding of the anatomy of the normal and diseased shoulder, exposure to multiple patients in a comparatively short period time, the immediate feedback provided from arthroscopy for personal reflection on performance, and the focus provided by the goal of improving the patient’s care pathway.

Very high values calculated for NPV from the first training period mean that a shoulder surgeon, using this method, could start using ultrasound to safely exclude a diagnosis of full-thickness cuff tear in the outpatient clinic early on. For example, a patient who has failed conservative measures is scanned after the first training period. The surgeon interprets this scan to be negative for a cuff tear. He may now advise the patient with significant certainty that they will not require a cuff repair at the time of surgery. Equally, a patient scanned after the third training period who is adjudged to have a full-thickness tear can be given this information with at least 94.7% certainty according to our measured PPV. This has enormous implications for the patient who may now plan their time off work and driving with greater certainty. Furthermore, the surgeon can approximate the time taken for surgery more accurately and make better use of the time allocated for his operating list.

Two recent studies defining the learning curve of an orthopaedic surgeon for shoulder ultrasound have been published.^9,10 The study by Murphy et al. demonstrated results extremely comparable to the present study amongst four participants (one senior shoulder surgeon).⁹ In that study, two participants performed 50 scans and the other two performed 100 scans each. Therefore, we hope that our study builds on that of Murphy et al.⁹ by demonstrating continued improvement in accuracy associated with a greater number of scans. The lower values of sensitivity and specificity that are reported in the study by McCulloch et al. may be attributable to differences in the learning protocol used.¹⁰ Specifically, feedback from performing arthroscopy on the scanned patients was received by the surgeon with much lower frequency. Furthermore, as scans were performed in the outpatient clinic weeks to months before surgery, feedback was also received on a much more temporally distant basis.

The learning method proposed in the present study is highly comparable to that proposed in the study by Murphy et al.,⁹ which in turn predicate BESS guidelines. However, subtle differences may mean that the method presented here may be slightly easier for surgeons to initiate without compromising results. The initial 20 practice cases in the present study were performed unsupervised using the same protocol as the rest of the study as opposed to enlisting the co-operation of a supervising musculoskeletal radiologist. Also, the ultrasound machine used was already spatially located in the vicinity of the patient after administration of regional anaesthesia, negating the need to move a machine to the patient or vice versa purely for practice. In addition, the anaesthetized patient is not exposed to potentially painful manoeuvres to accentuate cuff defects during the learning period. Although scanning an anaesthetized patient may be technically easier as a result of the obviation of patient discomfort, some studies have reported that probe-elicited tenderness and dynamic muscular contraction can improve sensitivity.^13,14 However, the ultrasound scans were not performed in a ‘real-world’ environment with an awake patient in the clinic and this is a weakness and limitation of this study.

The present study is potentially limited by bias. First, the patient cohort may display spectrum bias. All patients had failed conservative therapy and may represent the most extreme end of pathology more easily seen on scan. The incidence of cuff tears is likely higher in this population than those attending the radiology department or outpatient clinic. However, intra-operative diagnosis is the current gold standard in the detection of cuff tears.³ Thus, it would be difficult to represent true values for the accuracy of ultrasound in any other group. Furthermore, we consider that the feedback afforded through operating on these patients is vital to learning. Equally, other studies have a similar rate of cuff tears meaning that the results of the present study present a valid comparison.

Second, it is likely that the shoulder surgeon had clinically examined and seen the reports of any imaging prior to the day of surgery. However, this is offset by the substantial time that had passed between the last clinic visit and surgery, the confirmation of consent on the morning of surgery by the assistant rather than the surgeon and the routine practice of covering of patient’s eyes after completion of general anaesthesia, making recognition less likely.

It is important to note that this is a single surgeon study and the results may not be generalizable to other surgeons. Given the similarity in results to the comparable study by Murphy et al.,⁹ the present study adds to the evidence supporting the learning method published in the BESS guidelines. Until more studies utilizing this method are published firm recommendations on the minimum number of scans for the average shoulder surgeon required for a desirable accuracy cannot be made with real conviction, although it certainly provides guidance on what is achievable.

The size of cuff tear was not measured on ultrasound in the present study. It is the senior author’s (AI) practice to arthroscopically repair all small, moderate and large cuff tears, and oppose, as much as possible, the edges of massive cuff tears that are not completely repairable. Hence, the size of tear does not affect the surgical tactic on the day of operation. However, we acknowledge that practice differs between surgeons and knowing the size of tear pre-operatively can be of vital significance. Certainly, within the practice of the senior author, all patients are warned, at the time of consent, that if their cuff tear is found to be massive then the results of surgery are likely to be inferior. Further work is therefore required to determine whether surgeons can differentiate between cuff tear sizes accurately on ultrasound.

Finally, the present study does not obviate the need for dedicated musculoskeletal ultrasound by skilled ultrasonographers. The scans performed in the present study were very surgically directed and did not look for other common causes of pain such as partial cuff tears, bursitis, biceps tendonitis and calcific tendonitis.

Conclusions

The predictive values obtained in the present study for ultrasonic evaluation of the integrity of the rotator cuff are comparable with those quoted for musculoskeletal radiologists in the literature. The present study adds evidence for the capacity of shoulder surgeons to achieve accelerated learning and offers some potentially time-saving and patient-friendly additions to the BESS guidelines.

Declaration of conflicting interests

Research was previously presented at BESS Conference 2012, Torquay, under the title ‘Ultrasound for the detection of full-thickness rotator cuff tears: the learning curve for an orthopaedic surgeon using a novel training method’.

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.