Not all ultrasounds are created equal: general sonography versus musculoskeletal sonography in the detection of rotator cuff tears

Abstract

Background

This cross-sectional analytic diagnostic accuracy study was designed to compare the accuracy of ultrasound performed by general sonographers in local radiology practices with ultrasound performed by an experienced musculoskeletal sonographer for the detection of rotator cuff tears.

Methods

In total, 238 patients undergoing arthroscopy who had previously had an ultrasound performed by both a general sonographer and a specialist musculoskeletal sonographer made up the study cohort. Accuracy of diagnosis was compared with the findings at arthroscopy.

Results

When analyzed as all tears versus no tears, musculoskeletal sonography had an accuracy of 97%, a sensitivity of 97% and a specificity of 95%, whereas general sonography had an accuracy of 91%, a sensitivity of 91% and a specificity of 86%. When the partial tears were split with those ≥ 50% thickness in the tear group and those < 50% thickness in the no-tear group, musculoskeletal sonography had an accuracy of 97%, a sensitivity of 97% and a specificity of 100% and general sonography had an accuracy of 85%, a sensitivity of 84% and a specificity of 87%.

Conclusions

Ultrasound in the hands of an experienced musculoskeletal sonographer is highly accurate for the diagnosis of rotator cuff tears. General sonography has improved subsequent to earlier studies but remains inferior to an ultrasound performed by a musculoskeletal sonographer.

Introduction

Diagnostic imaging can be used as an adjunct to a thorough history and clinical examination when evaluating shoulder pain as a result of rotator cuff dysfunction and making a decision between conservative and surgical management.

Ultrasound can be used for the diagnosis of rotator cuff tears in primary care. There is less acceptance of its use in specialist care compared to other imaging modalities such as magnetic resonance imaging (MRI) as a result of accuracy concerns, despite many studies quoting high sensitivities and specificities.

Most studies evaluating the accuracy of ultrasound (US) in the diagnosis of rotator cuff tears use state of the art equipment and trained musculoskeletal radiologists with 10 or more years of experience to perform the scans. It is well known that US is operator and machine dependent. The use of experienced musculoskeletal radiologists to perform the study US may have lead to higher sensitivities and specificities compared to when scans are performed at the local radiology practice, where the sonographers may be generalists and not are specifically trained in musculoskeletal imaging.

The aim of the present study was therefore to compare the diagnostic accuracy and reliability of US performed at a local radiology practice (general sonography) with US performed in a surgical clinic by an experienced musculoskeletal sonographer (musculoskeletal sonography) in the diagnosis of supraspinatus tears with direct vision of the tendon under arthroscopy used as a gold standard.

Materials and methods

The study was a cross-sectional analytic diagnostic accuracy study. The study was approved by the local Ethics Committee.

The study population consisted of all patients undergoing elective shoulder surgery with a single surgeon at a day surgery facility between January 2013 and June 2015.

These patients had presented to the surgeon’s private rooms, having been referred there with shoulder pain by a general practitioner (GP). To be eligible for inclusion in the study, they needed to have presented having already had a shoulder US, MRI or magnetic resonance arthrography (MRA) organized by the GP and performed at a radiology facility prior to seeing the surgeon. They then had another US performed by the experienced musculoskeletal sonographer working in the surgeon’s rooms, who had trained specifically in musculoskeletal sonography and had over 10 years of experience in shoulder US working with this surgeon. The musculoskeletal sonographer did not view the scans or reports of the previous diagnostic imaging study prior to performing the second US and was not aware of the outcome of the clinical investigation of the surgeon. All imaging needed to be performed within 6 months of the date of surgery. It was considered that any longer than this might lead to inaccuracy as a result of a change in the status of the injury between imaging and surgery and any less time would exclude too many patients without necessarily changing the accuracy of the data. Exclusion criteria were revision surgery, surgery for calcific tendonitis, or if the imaging fell outside the 6-month timeframe.

All US conducted in the surgeon’s office was performed by an experienced musculoskeletal sonographer using a Logiq E9 machine (General Electric, Fairfield, CT, USA) with a 6–15 MHz linear transducer with a 50 mm × 10 mm footprint. The supraspinatus was visualized in both the transverse and longitudinal planes and a diagnosis of a full-thickness tear, partial thickness tear (bursal surface, intrasubstance or undersurface tear) or intact supraspinatus with or without tendinopathy was made. The mediolateral and anteroposterior dimensions of any full-thickness or partial thickness tears were measured and the percentage thickness of any partial thickness tears was determined. All tears were measured with standard machine calipers to the nearest millimetre.

To avoid compromising patient care, the orthopaedic surgeon was aware of the results of both imaging studies prior to the surgery. Arthroscopy was performed with the patient in a modified beachchair position under interscalene block with sedation. After insertion of the arthroscope at surgery, the presence or absence of a full- or partial thickness supraspinatus tear was noted. The supraspinatus was always examined from below and then from above, if needed. The dimensions of the tear size were measured by comparing the length and width of the tear with a 5-mm arthroscopic probe tip.

Statistical analysis

Cross-tabulations of arthroscopic assessments with the diagnosis based on general sonography and musculoskeletal sonography were made and diagnostic accuracy calculated. Calculations were performed to determine the sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, positive and negative predictive values and accuracy. These calculations were performed in two ways: (1) by considering both partial thickness and full-thickness tears as tears and comparing them with intact supraspinatus (no tear) and (2) by considering full-thickness tears and partial thickness tears ≥ 50% width of the tendon as tears and comparing them with intact tendon and partial thickness tears < 50% thickness (considered as no tears). Accuracy was defined as the proportion of correct diagnoses (full-thickness, partial thickness or no tear) out of the total number of shoulders in each group. Ninety-five percent confidence intervals were calculated for sensitivity, specificity, likelihood ratios and predictive values.

Results

There were 488 shoulder operations performed during the study time period. Of these, 189 (39%) patients were excluded for the specific reasons: they were undergoing revision surgery (n = 33); they had evidence of calcific tendonitis (n = 6); they had no general radiology imaging (n = 60); they had not had an office US performed on them (n = 30); there was more than 6 months between either imaging modality being performed and the date of surgery (n = 52); or they had no imaging of either type (n = 8). This left a final cohort of 299 patients. This cohort was split into two groups: a control group of 75 who had either MRI (n = 68) or MRA (n = 7) prior to seeing the surgeon and a study group of 238 who had an US prior to seeing the surgeon. Some patients were put into both groups because they came into clinic with both an US and an MRI or MRA (n = 14).

The time between US performed in the surgeon’s rooms (referred to as musculoskeletal sonography) and the arthroscopic examination ranged from 1 day to 176 days (median 27 days, mean 39 days). The time between US performed at a local radiology practice prior to seeing the surgeon (referred to as general sonography) and arthroscopic examination ranged from 6 days to 183 days (median 55 days, mean 67 days). The time between MRI and arthroscopic examination ranged from 4 days to 186 days (median 55 days, mean 70 days).

There were 238 subjects in the group who had both general sonography and musculoskeletal sonography and the findings on arthroscopy were used as the gold standard. During arthroscopy, there were 139 full-thickness tears, 77 partial thickness tears and 22 intact tendons identified.

Musculoskeletal US correctly identified 117 (84%) of the full-thickness tears in the cohort, 48 (62%) of the partial thickness tears and 21 (95%) of the intact cuffs (Table 1). This corresponded to an overall accuracy of musculoskeletal sonography of 78% (186 of 238). Of the 22 missed full-thickness tears, 21 were identified as partial thickness tears and one was a described as a no tear. There were 29 missed partial thickness tears: 24 were incorrectly identified by the musculoskeletal sonographer as full-thickness tears on US and five were incorrectly described as no tears. There was only one false positive result with an intact tendon at surgery, which was described as a 30% partial thickness tear by the musculoskeletal sonographer.

Table 1.

Comparison of diagnoses made with musculoskeletal sonography and general sonography with arthroscopic diagnoses of rotator cuff tears.

	Diagnoses with musculoskeletal sonography					Diagnosis with general sonography
	Full tear	Partial tear	No tear	Total	Accuracy	Full tear	Partial tear	No tear	Total	Accuracy
Arthroscopic diagnosis					186/238 = 78%					180/238 = 76%
Full tear	117	21	1	139		120	12	7	139
Partial tear	24	48	5	77		24	41	12	77
No tear	0	1	21	22		1	2	19	22

General sonography correctly identified 120 (86%) of the full-thickness tears, 41 (53%) of the partial thickness tears and 19 (86%) of the intact cuffs, corresponding to an overall accuracy of 76% (180 of 238). There were 19 full-thickness tears missed by general sonograpy: 12 were described as partial thickness tears and seven were described as no tears. There were 36 incorrectly identified partial thickness tears: 24 were described as full-thickness and 12 were found to have no tears on US. There were three false positive results: two were described as partial thickness tears and one was labelled a full-thickness tear.

The data were then analyzed in two ways to determine sensitivity and specificity. Initially, full-thickness tears and partial thickness tears were considered as tears and compared to intact cuffs where no tear was found at surgery (Table 2). When analyzed in this way, musculoskeletal sonography had an accuracy of 97% (231 of 238) and general sonography had an accuracy of 91% (216 of 238). Musculoskeletal sonography had a sensitivity of 97% and a specificity of 95% (Table 3). General sonography had a sensitivity of 91% and a specificity of 86%.

Table 2.

Comparison of diagnoses made with musculoskeletal sonography and general sonography with arthroscopic diagnoses of rotator cuff tears, with analyses performed in two ways.

	Diagnoses with musculoskeletal sonography				Diagnosis with general sonography
	Full and partial tears considered as tears		Full and > 50% partial tears considered as tears		Full and partial tears considered as tears		Full and > 50% partial tears considered as tears
	Tear	No tear	Tear	No tear	Tear	No tear	Tear	No tear
Arthroscopic diagnosis
Tear	210	6	208	7	197	19	181	34
No tear	1	21	0	23	3	19	3	20
Accuracy	231/238 = 97%		231/238 = 97%		216/238 = 91%		201/238 = 85%

Table 3.

Predictive values for musculoskeletal sonography and general sonography in the diagnosis of rotator cuff tears, with the analyses performed in two ways.

	Full and partial tears considered as tears		Full and > 50% partial tears considered as tears
	Musculoskeletal sonography	General sonography	Musculoskeletal sonography	General sonography
	210/216	197/216	208/215	181/215
Sensitivity	97% (CI: 95% to 99%)	91% (CI: 87% to 95%)	97% (CI: 94% to 100%)	84% (CI: 79% to 89%)
	21/22	19/22	23/23	20/23
Specificity	95% (CI: 87% to 100%)	86% (CI: 72% to 100%)	100% (CI: 100% to 100%)	87% (CI: 73% to 100%)
	(210/216)/(1/22)	(197/216)/(3/22)	(208/215)/(0.0001/23)	(181/215)/(3/23)
LR+	21.4 (CI: 3.2 to 145.2)	6.7 (CI: 2.3 to 19.2)	222512* (CI: 0 to infinity)	6.5 (CI: 2.2 to 18.6)
	(6/216)/(21/22)	(19/216)/(19/22)	(7/215)/(23/23)	(34/215)/(20/23)
LR–	0.03 (CI: 0.01 to 0.06)	0.1 (CI: 0.06 to 0.16)	0.03 (CI: 0.02 to 0.07)	0.2 (CI: 0.1 to 0.3)
PPV	99% (CI: 99% to 100%)	91% (CI: 97% to 100%)	100% (CI: 100% to 100%)	98% (CI: 97% to 100%)
NPV	78% (CI: 62% to 93%)	50% (CI: 34% to 66%)	77% (CI: 62% to 92%)	37% (CI: 24% to 50%)

^*The false positive rate was zero but a value of 0.0001 was used to allow calculation of a positive likelihood ratio.

CI, confidence interval; LR+, positive likelihoood ratio; LR–, negative likelihoood ratio; PPV, positive predictive value; NPV, negative predictive value.

Another analysis was conducted where partial tears ≥50% thickness and full-thickness tears were considered as tears, whereas intact cuffs and partial tears <50% were considered as no tears. When analyzed in this way, musculoskeletal sonography had an accuracy of 97% (231 of 238) and general sonography had an accuracy of 85% (201 of 238). Musculoskeletal sonography had a sensitivity of 97% and a specificity of 100%. General sonography had a sensitivity of 84% and a specificity of 87% (Table 3).

The 75 subjects in the MRI control group were analyzed using the same methods as described above for the US group to determine whether there was any additional benefit gained from MRI beyond that of musculoskeletal US. Overall accuracy was within 1% for both groups and musculoskeletal sonography performed better than MRI for accuracy, sensitivity and specificity in both sub-analyses. The full results and tables for the control group are provided in the Appendix (Tables A1 to A3).

Discussion

The present study investigated the reliability of US performed at a local radiology practice (general sonography) and US performed in a surgical clinic by an experienced musculoskeletal sonographer (musculoskeletal sonography). The accuracy of both general sonography and musculoskeletal sonography was above 90% when comparing tears with no tears, although musculoskeletal sonography was more accurate (97%) than general sonography (91%). Musculoskeletal sonography had higher sensitivities and specificities than general sonography in both analyses and the gap between the two increased in the second analysis where partial thickness tears ≥ 50% were considered as tears and partial thickness tears < 50% were considered as no tears.

We found a similar overall accuracy when comparing MRI with musculoskeletal US and both modalities were more accurate for diagnosing full-thickness tears than partial thickness tears. Musculoskeletal sonography had higher values than general MRI for both sensitivity and specificity in both analyses.

Previous diagnostic accuracy studies have consistently found a higher accuracy of both MRI and US for detecting full-thickness tears compared to partial thickness tears. These results were mirrored in our trial across all forms of imaging.

Cullen et al.¹ compared the US results of 68 shoulders performed by a single, fellowship trained musculoskeletal radiologist with 7 years of clinical experience with surgery performed by a single orthopaedic surgeon. Their results showed a sensitivity of 89% and a specificity of 100% for full-thickness tears and a sensitivity of 79% and a specificity of 94% for partial thickness tears.

Teefey et al.² compared ultrasonographic, MRI and arthroscopic findings in 71 consecutive cases. They reported a sensitivity of 97% to 98% for US and a specificity of 67% to 80% depending on whether they considered partial thickness tears as tears or no tears. There was no significant difference between these results and those of MRI, which were reported as a sensitivity of 100% and a specificity of 67% to 68%.

A Cochrane review by Lenza et al.³ compared the sensitivities and specificities of MRI and US using surgery as the gold standard in 20 studies (1147 shoulders). There was no significant difference in sensitivities and specificities of MRI and US for detecting any rotator cuff tears, full-thickness tears or partial thickness tears. Both modalities were more accurate in detecting full-thickness tears than partial thickness tears.

In the present study, tears incorrectly labelled as full-thickness tears mostly turned out to be partial thickness tears greater than 50% thickness of the tendon at arthroscopy. Missed partial thickness tears were incorrectly identified as both full-thickness tears or no tears, perhaps reflecting the difficulty of imaging with respect to identifying the extent and nature of the damage to the tendon if it is not fully ruptured. Over or underestimating the extent of partial thickness tears has clinical implications because it could potentially lead to the incorrect management of both small thickness tears labelled as full-thickness and larger thickness tears mislabelled as no tears. Patients with partial thickness tears are often more puzzling clinically, presenting without the easily identifiable supraspinatus weakness on clinical examination that often accompanies full-thickness tears. A patient with shoulder pain, weakness and dysfunction who has failed initial conservative management is more likely to be offered surgical management if a partial thickness tear > 50% tendon thickness is found on imaging compared to someone with a partial thickness tear < 50% thickness, who might persist with conservative management for longer, and therefore the accuracy of diagnosis of partial thickness tears is critical in this circumstance.

The results of the present study again illustrate the importance of operator experience on the accuracy of US findings in the diagnosis of rotator cuff tears. Musculoskeletal sonography consistently outperformed general sonography in all analyses and was equivalent and sometimes better than MRI in detecting rotator cuff tears. Goldberg et al.⁴ carried out a study looking at the accuracy of diagnostic US performed in general radiology practices, in contrast to other published studies that use experienced musculoskeletal sonographers to perform their US. Goldberg et al.⁴ found 155 false negatives and 51 false positives corresponding to a sensitivity of 24% and a specificity of 61%. Their study, published in 2003, was based on all US performed in 1996 and 1997. There has been much advancement in US imaging since then, in terms of both machine capabilities and the training of sonographers, and general sonography is now far more accurate than it used to be. In the present study, general sonography was able to correctly identify a tear in 197 cases with only 19 false negatives, corresponding to a sensitivity of 91%. A specificity of 86% had also improved to a much more acceptable value than the 61% in the study by Goldberg et al.⁴. The higher accuracy of the musculoskeletal sonographer compared to the general sonographer may reflect the extra training effect gained by the musculoskeletal sonographer, who is continually performing the same investigation when working in the shoulder clinic and has the additional benefit of having access to feedback on the accuracy of the results.

A strength of the present study is that a single cohort of patients had two US performed on them, allowing direct comparison of general and musculoskeletal sonography. To our knowledge, there are no other published studies directly comparing two US in the same patient. We also compared musculoskeletal US with MRI in a second cohort of patients and found a similar accuracy for both, although this was slightly in the favour of musculoskeletal US. Another strength unique to the present study is the separation of tear versus no tear by splitting the partial thickness tears into less than or greater than 50% of the tendon thickness to determine the second round of sensitivity and specificity analysis. This division may be more relevant to clinical practice when faced with treatment decision-making for a symptomatic patient without a full-thickness tear who is not responding to conservative management. Using this method of analysis, there is clear distinction between an experienced musculoskeletal sonographer performing the US versus an US performed in a local radiology practice by a general sonographer, highlighting both the difficulty in detecting partial thickness tears and the learning curve experienced by consistently performing musculoskeletal US.

There are several limitations to the present study. There was considerable time lag in some circumstances between the performances of one or both US and the final diagnosis at arthroscopy, with a maximum of 6 months as determined by our exclusion criteria. Rotator cuff pathology is not static in nature and may have changed over time between imaging and surgery, even in a 6-month period, leading to the recording of a diagnosis at imaging that was considered incorrect but may have actually been correct at the time. This is far more likely to affect the false negative results than the false positives (sensitivity more than specificity) because it is more likely that a damaged, degenerate tendon will go on to form a tear or that a partial thickness tear will progress to a full-thickness tear than it is that a partial or full-thickness tear will heal to an intact tendon during that time. There is also the limitation of selection bias, based on the fact that all cases were chosen at surgery. This meant that the negative control cases with intact tendons were having surgery performed for another reason and there was an absence of cases where the tendon was the source of the pain but without a tear present. This is because these cases are managed conservatively and not taken to the theatre. In an ideal situation for the present study, we would take everyone with shoulder pain for arthroscopy and work backwards from there, although it would be unethical to perform shoulder surgery on patients who did not need it clinically. A third limitation is that the surgeon was not blinded to the results of the imaging during surgery. This was to ensure patient safety so the surgeon would know what operation to perform prior to starting surgery. Once this was established and knife had been put to skin, the reports were not viewed again when looking at the tendon and determining the nature of the tear.

In conclusion, we have shown that US is accurate and reliable for the detection of rotator cuff tears when performed by a musculoskeletal sonographer and has a similar or better accuracy than MRI, with both being more accurate in detecting full-thickness tears than partial thickness tears. We have also shown that general US has improved significantly over the past 20 years but is still inferior to an US performed by a musculoskeletal sonographer, and it is important to consider the skill of both the sonographer and radiologist when interpreting imaging reports, especially when there is a partial thickness tear reported.

Appendix

Table A1.

Comparison of diagnoses made with musculoskeletal sonography and MRI with arthroscopic diagnoses of rotator cuff tears.

	Diagnoses with musculoskeletal sonography					Diagnosis with magnetic resonance imaging
	Full tear	Partial tear	No tear	Total	Accuracy	Full tear	Partial tear	No tear	Total	Accuracy
Arthroscopic diagnosis					54/75 = 72%					53/75 = 71%
Full tear	26	6	0	32		25	5	2	32
Partial tear	3	19	8	30		2	20	8	30
No tear	0	4	9	13		0	5	8	13

Table A2.

Comparison of diagnoses made with musculoskeletal sonography and magnetic resonance imaging (MRI) with arthroscopic diagnoses of rotator cuff tears, with analyses performed in two ways.

	Diagnoses with musculoskeletal sonography				Diagnosis with MRI
	Full and partial tears considered as tears		Full and > 50% partial tears considered as tears		Full and partial tears considered as tears		Full and > 50% partial tears considered as tears
	Tear	No tear	Tear	No tear	Tear	No tear	Tear	No tear
Arthroscopic diagnosis
Tear	54	8	51	11	52	10	44	18
No tear	4	9	1	12	5	8	2	11
Accuracy	63/75 = 84%		63/75 = 84%		60/75 = 80%		55/75 = 73%

Table A3.

Predictive values for musculoskeletal sonography and magnetic resonance imaging (MRI) in the diagnosis of rotator cuff tears, with the analyses performed in two ways.

	Full and partial tears considered as tears		Full and > 50% partial tears considered as tears
	Musculoskeletal sonography	MRI	Office ultrasound	MRI
	54/62	52/62	51/62	43/62
Sensitivity	87% (CI: 79% to 95%)	84% (CI: 75% to 93%)	82% (CI: 73% to 92%)	69% (CI: 60% to 82%)
	9/13	8/13	12/13	11/13
Specificity	69% (CI: 44% to 94%)	62% (CI: 35% to 88%)	92% (CI: 78% to 100%)	85% (CI: 65% to 100%)
	(54/62)/(4/13)	(52/62)/5/13)	(51/62)/(1/13)	(43/62)/(2/13)
LR+	2.8 (CI: 1.3 to 6.4)	2.2 (CI: 1.1 to 4.4)	10.7 (CI: 1.6 to 70.5)	4.5 (CI: 1.3 to 16.7)
	(8/62)/(9/13)	(10/62)/(8/13)	(11/62/(12/13)	(18/62)/(11/13)
LR–	0.2 (CI: 0.1 to 0.4)	0.3 (CI: 0.1 to 0.5)	0.2 (CI: 0.1 to 0.3)	0.3 (CI: 0.2 to 0.5)
PPV	93% (CI: 87% to 100%)	91% (CI: 84% to 99%)	98% (CI: 94% to 100%)	96% (CI: 90% to 100%)
NPV	53% (CI: 29% to 77%)	44% (CI: 21% to 67%)	52% (CI: 32% to 73%)	38% (CI: 20% to 56%)

CI, confidence interval; LR+, positive likelihoood ratio; LR–, negative likelihoood ratio; PPV, positive predictive value; NPV, negative predictive value.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Ethical Review and Patient Consent

The study was approved by the South Eastern Sydney and Illawarra Area Health Services Ethics Committee. All patients gave written informed consent prior to participation in the study.