A multicentre survey of the condition of ultrasound probes

Abstract

Background

The implementation of quality assurance for ultrasound scanners in the United Kingdom is patchy, but government appointed bodies require quality assurance and there are regulatory requirements for maintenance and inspection of equipment. Previous studies have shown high fault rates in ultrasound probes; some of these studies used electronic probe testers, but there is good evidence that over 90% of faults may be detected using simple methods. We aimed to conduct a multicentre survey of the condition of probes, using visual inspection and assessing the in-air reverberation.

Methods

Visitors to the stand run by Multi-Medix Ltd at the BMUS Annual Scientific Meeting in 2014 were invited to participate in the study. One or both of the authors visited participants, performing a visual inspection of probes for evidence of damage or wear and inspecting the in-air reverberation pattern for uniformity. Probes were classified using a risk-based traffic light system: green—no fault found; amber—fault found; further action required but the probe need not be removed from use; red—the probe should be removed from use due to physical or clinical diagnostic risks.

Results

Twelve sites and 219 probes were included in the survey. Sixty-three percent of probes were classified as green; 25% as amber and 13% as red.

Conclusion

More than one in three probes were faulty. Simple tests, at minimal cost, have the potential to demonstrate over 90% of probe faults, making it possible for employers to comply with their duties defined by regulations, national standards and professional guidelines.

Introduction

In the United Kingdom, there is currently no legislative requirement for quality assurance (QA) testing to be performed on ultrasound (US) scanning equipment, and implementation of QA is patchy. Obstacles to the implementation of US QA in the UK include the pressure of work on sonographers and the shortage of Medical Physics expertise.¹ However, there are several drivers for QA, including publications by the Medicines and Healthcare Products Regulatory Agency.² QA is mandatory for US equipment used in the UK National Health Service (NHS) Screening Programmes.^3–5 In the UK, the Imaging Services Accreditation Scheme developed by The Royal College of Radiologists and The College of Radiographers requires systems to manage equipment, including assurance of calibration and performance.⁶

Although there is no legislative requirement for US QA in the UK, the Provision and Use of Work Equipment Regulations 1998 (PUWER) require that employers ensure that work equipment is maintained in an efficient state, in efficient working order and in good repair, and that equipment is regularly inspected.⁷ The Health and Social Care Act 2008 (Regulated Activities) Regulations 2014 in England require that equipment be suitable for purpose and properly maintained and that risks are assessed and mitigated.⁸ The Care Quality Commission provides guidance on interpretation of the regulations, requiring that health care providers have operational policies and procedures and maintenance budgets to maintain their equipment to be sound, operationally safe and exhibiting only minor deterioration.⁹

The purpose of a QA programme should be to detect equipment faults before diagnostic quality or safety is compromised. Since only a small percentage of faults are detected during clinical use,¹⁰ then the lack of an effective QA programme introduces an unnecessary risk of diagnostic errors. Several previous studies have shown high fault rates in US probes that are in clinical use.^10–13

Hangiandreou et al.¹⁰ reviewed the results of a four-year quality control (QC) programme in a single large radiology department, including over 300 probes. They found that 165 probes failed (defined as a problem that required immediate repair or replacement) over the period, with an average annual failure rate of 13.9%. Probe failure represented 88.2% of total failures, the remainder being scanner component failures. The most frequent failures (probes and scanners) were image uniformity (66.3%) and mechanical integrity (25.1%). Only 7% of failures were detected in clinical use and only 1.6% was detected by measurement of depth of penetration in a tissue mimicking test object (TMTO). Image uniformity was assessed by looking for artefacts in images of a TMTO and the in-air reverberation pattern; acceptance criteria were subjective, taking into account the number, location and conspicuity of artefacts.

Martensson et al.¹¹ tested 676 probes using an electronic probe tester and found faults in 269 probes (40%), but did not state how many of these faults were visible by other means, e.g. inspection of the in-air reverberation pattern. The most frequent fault was delamination (179 probes, 67% of faults), with the remainder being cable faults (80 probes, 30%), and dead or weak elements (10 probes, 4%).

Martensson et al.¹² tested 299 probes a year after their earlier study¹¹ and found that 81 were defective (27.1%). These probes had either passed the test a year previously or had been purchased to replace defective probes. The distribution of fault types was similar to their previous study, with delamination (50%) and cable faults (35%) being the most common. They found the highest proportion of defective probes in radiology departments (36.0%) and that a factor in this was the way that probes were handled in different clinics, a higher failure rate being associated with probes being disconnected and stored away from the scanner when not in use, rather than being stored on the scanner.

Sipila et al.¹³ tested 135 probes using an electronic probe tester and with a TMTO to assess image uniformity and made a physical inspection of each probe, finding a total of 52 faulty probes (39%). A total of 21 faults (40% of total faulty probes) were detected by the electronic probe tester, 20 (38%) using the TMTO and 34 (65%) by physical inspection. Three faults (6% of total faulty probes) were demonstrated only by the electronic probe tester, eight (15%) only by the TMTO and 21 (40%) only by physical inspection.

Probe faults are therefore common and important to detect. There is evidence to show that electronic probe testers such as FirstCall (Unisyn, Golden, CO, USA) and ProbeHunter (BBS Medical AB, Stockholm, Sweden) provide comprehensive results that both demonstrate faults and indicate their likely origin.^11–13 Our experience using the FirstCall and comparing results with the in-air reverberation pattern is that the latter can show a single non-functioning element, with appropriate adjustment of scanner settings. Many functional probe faults can therefore be detected simply by inspection of the in-air reverberation pattern;^10,14 Figure 1 shows dropout and delamination (separation of layers within the probe). The ‘paperclip test’¹⁵ and imaging of a TMTO may then be used to assess the severity and inform management of faults, e.g. the paperclip test may or may not confirm dropout and a TMTO image may or may not show shadowing, but the physical origin of the fault may not be important unless considering a repair. All external probe faults can be detected by visual inspection.¹³

Figure 1.

Drop out (axial streak on the right) and delamination (disrupted reverberation pattern to left of centre) on a curvilinear array.

From the high incidence of image uniformity and mechanical integrity faults found by Hangiandreou et al.¹⁰ (91% of faults) and Sipila et al.¹³ (94% of faults) it is clear that performing a visual inspection and assessing the uniformity of the in-air reverberation pattern, with appropriate use of scanner settings, has the potential to detect over 90% of probe faults, with the caveat that there may be faults that are not detected by any of the current methods.

The aim of this study was to survey probes, using these simple methods, in a number of hospital US departments in England to assess the frequency of probe faults and the effect of different maintenance and QA practices.

Methods

Visitors to the stand run by Multi-Medix Ltd as part of the technical exhibition at the British Medical Ultrasound Society (BMUS) Annual Scientific Meeting in Manchester in December 2014 were invited to volunteer to participate in the study. Prospective participants were provided with a questionnaire asking for details of equipment to be included in the survey and their arrangements for maintenance and QA.

Participating hospitals were then visited during 2015 by one or both of the authors, who surveyed all accessible probes. Each probe, cable and connector was visually inspected for evidence of damage or wear. The in-air reverberation pattern was then inspected using a short image scale, a fundamental (non-harmonic) transmit frequency and single superficial focus with all compounding and advanced processing functions switched off.¹⁴ Any evidence of element dropout or delamination of the acoustic stack was recorded, adjusting frequency and gain for the optimal display of any fault seen.

Where dropout was seen or delamination suspected, the paperclip test¹⁵ was performed. For curved and linear arrays, the test was performed as recommended by the Institute of Physics and Engineering in Medicine (IPEM) as follows.¹⁴ Using the settings described above, the edge of a paperclip was gently moved across the affected area of the probe face in the longitudinal direction with the paperclip at 90° to the long axis of the array. With some probes there was more friction between the probe face and the paperclip and so a very thin film of water was used as coupling. In cases of dropout, typical findings would be a well-defined axial band in the in-air reverberation pattern, with a drop in paperclip reverberation amplitude at this point relative to other parts of the array. In cases of delamination, the in-air reverberation pattern is typically disrupted within the delaminated area, as shown in Figure 1.

For most of the phased arrays, a modified version of the paperclip test was used (when visiting site 3, the test had not yet been conceived). In order to ensure the full probe aperture was operating, a medium scale (perhaps 10 cm) and a mid-range focus were used; on some scanners, it may be necessary to use a deeper focus. M-mode was activated with the sampling line positioned down the centre of the image and the paperclip moved along the entire probe face as above. Any faulty elements were shown by vertical dark bands in the M-mode display, as shown in Figure 2. For each probe, the test was repeated several times to ensure reproducible demonstration of any fault.

Figure 2.

M-mode paperclip test showing dropout towards one end of the array (dark axial band towards the left of the image).

In all cases of element dropout, the probe connector was re-seated and, if necessary, moved between ports to exclude intermittent connector faults (perhaps due to dirty connections) or port faults. Where dropout was resolved by re-seating or moving between ports, a probe fault was not recorded but the equipment users were notified of the issue. Probe cables were flexed in an attempt to establish whether dropout was due to cable faults.

Probes were then classified using a risk-based traffic light system, summarised in Table 1. Green indicated that no fault had been found. Amber indicated that a fault had been found and further action was required but that the probe need not be removed from use, e.g. probe case requiring minor repair, single element failure towards the periphery of the probe. Red indicated that the probe should be removed from use due to physical or clinical diagnostic risks, e.g. cable faults, exposed electrical connections, multiple element failure or dropout near the centre of the array. Lens damage classified as red was either clearly visible wear, seen on the probe face and in non-uniformity of the reverberation pattern, or accidental damage, e.g. needle puncture. Several areas of dropout, one large area (>5% of array) or central dropout, where the paperclip test confirmed a fault, was classified as significant (red). Delamination was always classified as red. Where there was some axial shadowing in the reverberation pattern but the paperclip test showed no evidence of dropout, this was classified as minor non-uniformity (amber). Minor lens wear, seen only by small lateral variations (<5%) in the depth of the reverberation pattern, measured from the probe surface to the deepest visible reverberation line, was classified as amber. Figure 3 shows minor lens wear at the periphery and minor non-uniformity. Figure 4 shows significant lens non-uniformity, with >5% variation in the depth of reverberation along the probe, classified as red.

Table 1.

Summary of probe fault classification

Classification	Visual inspection	Reverberation	Action
Green	No fault	No significant non-uniformity	None
Amber	External physical damage, no functional consequence	Single peripheral element dropout; Minor non-uniformity or lens wear	Risk assess and repair or monitor
Red	Wear or damage to probe face	Cable fault (dropout); Large, multiple or central dropout; Delamination	Replace

Figure 3.

Minor lens wear shown by a change in depth of reverberations at the ends of the probe, and minor non-uniformity shown by faint axial streaking in the reverberation pattern.

Figure 4.

Significant lens non-uniformity (>5% variation in reverberation depth).

Participating organisations were provided with a detailed report for each scanner tested. An amber classification was accompanied by advice on further action. Some amber classifications required the users to carry out a risk assessment for infection control (split cases, damaged sealant) and a repair was advised. Where our judgement was that there was no immediate clinical diagnostic risk, but that the probe was not in perfect condition (minor lens wear, minor dropout, minor non-uniformity), monitoring for deterioration was advised. In the case of dropout, users were advised not to place a Doppler beam in the affected area, as there is evidence that dead elements affect Doppler spectra.^16,17 We did not follow up the survey to find out what actions had been taken by the organisations in response to the reports.

Statistical differences between fault rates were evaluated using the McNemar test for the difference between proportions (two-tailed test).¹⁸

Results

A total of 219 probes were tested in the survey, across 12 sites, as summarised in Table 2. A total of 137 (63%) probes were classified as green; 82 (37%) probes had at least one fault, 54 (25%) being classified as amber and 28 (13%) as red. Probe ratings on each site are shown in Table 3, together with an indication of whether maintenance and QA programmes were in place. On the three sites with a QA programme (1, 4 and 12; 55 probes) this was an annual QA visit to each scanner, with testing in accordance with IPEM guidelines.¹⁴

Table 2.

Summary of participating sites and probes

Site ID	Number of scanners	Number of probes
		Curved	Linear	Phased	Transvaginal	Total
1	10	9	2	6	4	21
2	1	1	1	0	1	3
3	14	21	12	6	7	46
4	6	6	11	0	2	19
5	6	4	5	4	3	16
6	1	2	2	0	1	5
7	1	2	2	0	1	5
8	15	14	13	5	6	38
9	2	1	3	0	1	5
10	17	15	14	6	8	43
11	1	1	1	0	1	3
12	6	7	4	2	2	15
Total	80	83	70	29	37	219

Table 3.

Maintenance/QA arrangements and probe ratings by site

Site ID	Number of probes	Maintenance/QA	Ratings			Total Faults (%)
			Green	Amber	Red
1	21	M/A	10	5	6	52
2	3	M	2	1	0	33
3	46	M	12	25	9	74
4	19	M/A	16	3	0	16
5	16	M	13	2	1	19
6	5	M	4	0	1	20
7	5	M	4	1	0	20
8	38	M	29	4	5	24
9	5	M	5	0	0	0
10	43	M	30	9	4	30
11	3	M	3	0	0	0
12	15	M/A	9	4	2	40
Total	219		137	54	28	37

QA: quality assurance; M: external maintenance contract; A: annual in-house QA.

For sites with and without an in-house QA programme, the proportions of red classifications were 15% and 12%, respectively (no significant difference, p > 0.05) and the proportions of amber classifications were 20% and 26%, respectively (no significant difference, p > 0.05). However, one of the hospitals (Site 1) with an in-house QA programme had a fault rate of 52% (24% amber; 29% red) and site 4 had a fault rate of 16% (all amber) suggesting possible differences in QA practice.

Table 3 shows a wide range of fault rates across sites, from 0% to 74%. Sites 2, 6, 7, 9 and 11 had only one or two scanners each so the number of probes is too small for meaningful comparison. The other sites had at least six scanners each and at least 15 probes with a range of fault rates from 16% to 74% suggesting differences in practice between centres. Tables 4 and 5 summarise the nature of the faults for probes categorised as red and amber, respectively. Some probes had more than one fault, and were categorised by the most severe. Twenty-three percent of faults were found by physical inspection.

Table 4.

Summary of faults classified as red

Fault category	Number of probes
Significant dropout	11
Lens damage	7
Lens non-uniformity (>5% thickness variation)	4
Cable fault	3
Delamination	3

Note: There were 22 curved arrays, five linear arrays and one transvaginal probe.

Table 5.

Summary of faults classified as amber

Fault category	Number of probes
Minor non-uniformity in reverberation patterna	20
Peripheral dropouta	14
Minor lens weara	8
Split caseb	9
Lens sealant brokenb	2
Damaged grommetb	1

Note: There were 21 curved arrays, 17 linear arrays, 9 phased arrays and 7 transvaginal probes.

^aMonitoring for deterioration recommended.

^bRepair recommended.

Five scanner suppliers were included in the study, with 118, 84, 13, 3 and 1 probes. There was no significant difference in probe fault rates between the two main suppliers (36% and 37%; p > 0.05). There were, however, differences in the nature of faults between these two suppliers. One supplier had a higher incidence of splits to the probe case or grommets (9.5% vs. 1.7%; p < 0.01), and a lower incidence of dropout (7.1% vs. 19%; p < 0.01), whilst the figures for lens wear/damage and minor non-uniformity were similar (respectively 8.3% vs. 7.6%; p > 0.05; 12% vs. 6.8%; p > 0.05).

Fifteen probes were used outside imaging departments including six on wards, five on Early Pregnancy Assessment Units, three in Accident and Emergency and one in Physiotherapy. A higher proportion (67%) of these probes were faulty (3 red, 7 amber) compared to those used in imaging departments (35%; p < 0.05).

Discussion

In this study, more than one in three (37%) probes were classified as being faulty. More than one in eight (13%) probes were not fit for clinical use and almost one in four (24%) probes were fit for purpose but required further action in the form of repair or monitoring. This is less than the fault or failure rates reported in other studies,^10,11,13 but in our study the failure rate varied widely between centres, suggesting that variations in local practice can influence the number of faulty probes in use. There was also a wide variation in fault rates between centres with an annual QA programme. Factors likely to affect fault rates at a given time include the age and use of equipment, user vigilance for faults, the nature of maintenance contracts (inclusive or exclusive of probe replacements), organisational culture, e.g. in reporting and remedying faults and the funding available for probe repairs and replacements. Where there is a QA programme in place, fault detection will depend on the range of tests performed and on the appropriate choice of scanner settings, which requires a good understanding of scanner controls. It is a concern that faults had not been reported and rectified at maintenance visits, although it is possible that faults occurred subsequent to the latest visit. The high fault rate suggests that employers are not fulfilling their duties under relevant legislation.^7,8

Fault rates outside imaging departments were significantly higher. This may be a result of use in a less well-controlled environment. Fault rates may be reduced by simple measures such as: careful cleaning with manufacturer approved materials using a wiping, rather than rubbing, action; careful stowage of probes and cables; avoiding tension on cables and grommets by stowing cables loosely; avoiding cables touching the floor or hanging near scanner wheels; storing probes in appropriate boxes when not connected to the scanner.

We demonstrated a lower fault rate than Martensson et al.¹¹ and Sipila et al.,¹³ who both used an electronic probe tester. The former group tested only with a probe tester. Only three faults (6% of total faulty probes) found by the latter group were demonstrated by the electronic probe tester alone, so it is unlikely that the difference between fault rates is due to our lack of a probe tester. However, we plan a further study to directly compare results from the electronic probe tester and the in-air reverberation pattern.

Fault rates depend on how faults and failures are defined and judgements regarding the need for immediate repair or replacement. We suggest defining every abnormal finding as a fault and subcategorising faults according to the level of risk and whether this risk can be managed, as we have done in this study. We did not use a TMTO in this survey, as non-uniformity may be reliably demonstrated by inspection of the in-air reverberation pattern but a TMTO may be useful in fault management by, for example, assessing whether a probe fault leads to shadowing in the image.

An important element of QA is acceptance testing. Some of the probes in our study could have been faulty at purchase, e.g. we would reject probes with reverberation that was non-uniform in comparison with other probes of a similar type; the four probes with non-uniform lens thickness fell into this category, as would a proportion of the probes with minor non-uniformity. Acceptance testing is also important in establishing a baseline for uniformity of the in-air reverberation pattern. Minor non-uniformity (Figure 3) may be evident and this should be monitored to ensure that it is not a precursor to delamination.

QA and maintenance programmes are important in demonstrating and addressing faults, but it is essential that tests are well designed and performed at appropriate intervals and that there is a process for managing faults. Martensson et al.¹² showed that annual testing is insufficient and Sipila et al.¹³ showed the importance of regular visual inspection of equipment. Using simple methods, we have demonstrated a high incidence of non-uniformity in the in-air reverberation pattern (29% of all probes) and a substantial incidence of external structural defects (9% of all probes). A well-designed user QA programme, e.g. that recommended by the BMUS,¹⁹ should enable detection of most probe faults. Further tests are needed to detect non-localised faults that affect the whole image, e.g. a general loss of sensitivity may be demonstrated by serial measurements of depth of penetration in a TMTO.

With frequent and carefully designed QA, it is therefore possible to detect the majority of faults; management of faults is another issue. External structural faults, where imaging is not affected, are relatively easily managed. These faults can make probe cleaning less effective and increase the risk of cross-infection, or increase the risk of further damage, but good quality repairs are possible. Where imaging may be affected, decisions can be more challenging. There is good evidence that both colour and spectral Doppler are affected by dropout.^16,17 If we advise users not to place a Doppler beam where there is dropout, how do we ensure that this advice is easy to follow, as minor dropout may not be visible with clinical settings? Where we know there is a fault, should we advise users to continue using the probe if we have no positive evidence that image quality is not affected? Our current approach to managing probe faults is reflected in our amber and red classification, as summarised in Tables 1, 4 and 5, and our advice to study participants above.

There are some limitations to these testing methods. There were six 4D probes in our survey (two phased arrays, one transvaginal probe and three low frequency curved arrays). 4D phased arrays are composed of a matrix of elements, so for dropout to be seen using our methods a large proportion of a single row of elements would have to fail, which is unlikely, but the impact of an isolated single element failure is likely to be insignificant. For 4D probes where a conventional array is enclosed in a housing, the in-air reverberation is from the transmission window rather than the lens and so dropout is less likely to be seen and the paperclip test is unlikely to work.

Conclusion

Using simple methods, we found that a high proportion of probes in this study (37%) were faulty, immediate replacement being recommended for 13%. This highlights the importance of frequent and regular QA by the equipment users, using the simple tests described in this study, with appropriate selection of scanner settings. Simple tests, at minimal cost, have the potential to demonstrate over 90% of probe faults, making it possible for employers to comply with their duties under regulations,^7,8 national standards^2–5 and professional guidelines.^6,14

Declaration of Conflicting Interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Multi-Medix is a company specialising in probe testing, repair, sales and system quality assurance.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Ethical approval

No ethical approval was required.

Guarantor

DJW

Contributorship

DJW conceived the project. NJD and DJW designed the survey and carried out the work. NJD analysed the data and wrote the first draft of the manuscript. Both authors reviewed the manuscript and approved the final version.