Assessment of PACS Display Systems

Abstract

This work describes our experience in reviewing the performance criteria for display systems and how we have implemented a practical approach to the assessment of the workstation environment in a large tertiary care hospital. The acceptance criteria contained in the draft report of Topic Group 18 of the American Association of Physicists in Medicine (AAPM) were used as a basis for assessment of primary and secondary displays. A telescopic photometer was used to measure the maximum luminance and the contrast ratio of the image for the displays used in our radiology department and in the operating and emergency rooms using the standard Society of Motion Picture and Television Engineers (SMPTE) pattern, in ambient light and with light decreased as much as possible. About half of the displays met the AAPM criteria for minimum luminance and contrast ratio in low light. None of the systems met the contrast ratio criteria in ambient light. The challenges in improving the performance and calibrating displays are discussed.

Introduction

Many hospitals and clinics are in the process of implementing picture archive and communication system (PACS). This usually starts with the connection of cross-sectional imaging systems and with the installation of dedicated reporting workstations in the radiology area. As soon as radiography is changed from film/screen systems to computed radiography (CR) or digital radiography (DR), facilities face the problem of the distribution of images throughout the facility and even beyond. In most cases, cost will preclude the widespread implementation of dedicated high-end clinical workstations typically used in radiology departments.

From many standpoints, the performance of PACS workstations is crucial to initial acceptance and ongoing performance of a PACS. Many papers have described the difficulty of establishing standards not only for the displays but also for the workspace, and, perhaps most difficult of all, for the quality of an image.^1–5 Even in medium-sized hospitals the number of primary or reporting workstation displays will likely be over 50 and the number of secondary displays will be several hundred. Like many centers, we have been introducing PACS displays in a phased approach over the last four years, and have various types of displays that are currently used for reporting and review.

This work describes our experience in reviewing the recommendations for display systems and how we have implemented a practical approach to the assessment of image display systems in a large tertiary care hospital.

Background

Luminance Measurement

Luminance is the amount of visible light emitted per unit projected area of the display. It is the brightness of a display we perceive. Normally, this is measured in units of lumens per steradian per square meter (commonly known as cd/m²) with a telescopic photometer with a defined angle of incidence, often 1°.

Performance Criteria and Standards

Some published standards or performance recommendations are many years old and do not reflect changes in technology.^6–10 The two most recent and relevant display documents come from the International Electrotechnical Commission (IEC) and the American Association of Physicists in Medicine (AAPM), but both are in draft form. The IEC document Evaluation and routine testing in medical imaging departments—Acceptance Tests—Image Display Devices (41 pages) is due to be finalized in 2005. The AAPM document Assessment of Display Performance for Medical Imaging Systems (Version 10; 156 pages)¹¹ comes from Task Group 18 (TG18) and may also be published in 2005. This TG18 draft is available on a public website http://www.deckard.duhs.duke.edu/tg18.

Both documents describe two types of display systems:

• Primary displays, which can be used for all primary diagnosis from all modalities including digital radiography and computed radiography, and

• Secondary displays, which can be used for all diagnoses except digital radiography and computed radiography.

IEC Proposed Standards

The least demanding standards come from the IEC. The main requirements are shown in Table 1.

Table 1.

IEC proposed standards

Parameter	Primary Display	Secondary Display
Contrast ratio (maximum/minimum luminance)	>100	>40
Minimum luminance ratio (maximum luminance/ambient luminance)	>100	>40
Luminance variation	<30% (<20% flat panel)	<35% (<20% flat panel)
SMPTE Resolution pattern resolution	All lines clear	All lines clear
	no diagonals	no diagonals
Contrast balance	5% and 95% levels clearly seen	5% and 95% levels clearly seen

AAPM TG18

The AAPM TG18 document describes many tools for the assessment of display systems. The document also gives suggested acceptance criteria for display systems. The main criteria are shown in Table 2.

Table 2.

AAPM TG18 acceptance criteria

Parameter	Primary display	Secondary display
Minimum luminance, Lmin	170 cd/m2	100 cd/m2
Contrast ratio (maximum/minimum luminance)	>250	>100
Reflection luminance ratio (minimum luminance/ambient luminance)	<4	<1
Luminance variation	<30%	<30%
SMPTE Resolution Pattern Resolution	All lines clear	All lines clear
	No diagonals	No diagonals
Contrast Balance	5% and 95% levels clearly seen	5% and 95% levels clearly seen

The American College of Radiology (ACR) also suggests 170 cd/m² as the minimum luminance for primary reporting.

The AAPM document is a very comprehensive treatise on display systems as well as their testing. It describes how all types of displays function and many tests for all facets of the image. However, most clinics do not have the staff or the time to perform the many routine tests that are described. This is especially true for testing secondary displays. Manufacturers often quote contrast ratios of >400:1 but this is often measured under ideal conditions. In the real clinical environment, the contrast ratio is often much lower than this, because of ambient scattered light and reflections. This work was started in order to ascertain how well the different types of display actually performed in our everyday clinical environments. To reduce the task to a manageable one, we decided to test only two parameters of the display—the maximum and minimum luminance, which provide us with the maximum luminance and the contrast ratio.

Materials and Methods

A six-month-old factory-calibrated telescopic photometer (Minolta Model LS-100) was used to measure the maximum luminance (L_max) and the contrast ratio of the image for the primary and secondary displays used in our radiology department and in the operating and emergency rooms. The LS-100 has a 1° acceptance angle. The photometer was used at a distance of 1 m and focused on the part of the screen to be measured. The 0% and the 100% intensity bars of the standard SMPTE pattern were used for this measurement. To investigate the local environment, the measurements were taken in ambient light and also with light decreased as much as possible. The measurement in dark conditions gives the maximum contrast ratio. In room light, one should strictly measure the reflected ambient light separately, but this is not easy to do when the units are in place in operating rooms or emergency areas. Although this way of performing the measurement of scattered ambient light by this method is not strictly correct according to the test methods of the AAPM, it does give a ratio that seems to reflect the practical situation for the radiologist. For the rest of this paper, this quantity measured in other than dark conditions should be read as an effective contrast ratio.

In our hospital, all the primary reporting workstations are used in custom-built reporting areas with low reflecting surfaces, ergonomically designed chairs, and recessed pot lighting with dimmer controls and climate control. The primary reporting stations are calibrated every three months by the manufacturer for luminance, contrast ratio, and gray scale according to DICOM part 3.14. In contrast, the secondary displays are used under a range of conditions throughout the hospital, often with the possibility of distracting reflections and high ambient lighting. The secondary displays are checked normally only during installation, and often the calibration factors can be changed by the operators.

Measurements were made on 43 primary and 41 secondary displays. The primary displays were of two types—Radiologists Reporting Stations and Clinical Workstations. The Radiologist Workstations comprised 34 high-intensity monochrome CRT systems specifically designed for PACS viewing (Siemens SMM; Dome MD2P 10bit video card). The Clinical Workstations were systems assembled in-house (NEC 2080UX; Matrox Med 2MP 10 bit video card). Most of the secondary displays were 17.2-in. flat panel color displays purchased in 2002 (Compaq 1720). The processing stations were 18-in. flat panel displays purchased in 2003 (NEC 1880SX) and one CRT system (HP7550).

The primary display measurements were made in the low light conditions normally used in the reporting areas, then with room lights on. The secondary displays measurements were made in the lighting conditions normally used in ORs and emergency departments—fairly bright. Measurements were then repeated after reducing the light as much as possible. All measurements were made when the areas were not being used. The researchers wore dark clothing to avoid reflection artifacts.

Results

Primary Displays

Figures 1 and 2 show maximum luminance and contrast ratio measurements for the Radiologist Workstations. The average maximum luminance was 291 cd/m²; the average contrast ratio was 251.

Fig 1.

The luminance measured on Radiologist Workstation displays.

Fig 2.

The contrast ratio measured on the Radiologist Workstation displays: solid bars—low light as used; shaded bars—with room lights on.

All the Radiologist Workstation displays met the acceptance criteria of the AAPM for maximum luminance (250 cd/m²), but 15 of the 34 primary displays did not have a contrast ratio >250 in the low light conditions normally used. The lower contrast ratio in low light was normally due to incorrect placement of the displays or high ambient light. The environment of all the primary displays could be changed to increase the contrast to more than 250, either by repositioning the display or removal of light sources. As shown in Figure 2, in room light the effective contrast ratio was dramatically reduced, as would be expected.

Clinical Workstations

The Clinical Workstations used the same PACS software as the primary stations (Agfa Impax 4.1), but the hardware was assembled in-house from standard PCs, a medical quality dual video card, and two high-quality flat panel displays. Nearly all the Clinical Workstation displays met the AAPM and ACR criteria for luminance (Fig 3). Only one had to be adjusted. However, the contrast ratios in low light were poor in general and none could be recalibrated to meet the AAPM contrast ratio standard (Fig 4).

Fig 3.

The luminance measured on Clinical Workstation displays.

Fig 4.

The contrast ratio measured on the Clinical Workstation displays: solid bars—low light as used; shaded bars—with room lights on.

Processing Stations

Processing Workstations are often stand-alone systems that can draw from the PACS archive to perform special processing such as 3D and dental reconstruction, and cardiac scoring. As they mainly use color, they are secondary displays. In our measurements, one of the systems (Processing Workstation Display # 5, using a CRT) failed to meet the luminance requirements for a secondary display system (Fig 5). Most met the contrast ratio requirement in low light, and some met the requirement in room light (Fig 6).

Fig 5.

The luminance measured on Processing Workstation displays.

Fig 6.

The contrast ratio measured on the Processing Workstations: solid bars—low light as used; shaded bars—with room lights on.

Operating Room and Emergency Department Displays

For these secondary displays, the average luminance was 98 cd/m² and the average contrast ratio was 176 in low light (Figs 7 and 8). All the secondary displays met the recommendation of the AAPM for minimum contrast ratio (100), but 19 of the 35 secondary displays did not have a maximum luminance >100, as suggested by AAPM. Unfortunately, these displays are used in a high light level environment, where the effective contrast ratio is decreased to very low levels.

Fig 7.

The luminance measured on OR and ER Workstations.

Fig 8.

The contrast ratio measured on the OR and ER Workstations: solid bars—low light as used; shaded bars—with room lights on.

Results

Tables 3 and 4 summarize the results. All the Radiologist Workstation displays met the luminance requirements and 52% met the contrast ratio requirement. Most of the Clinical Workstations met the luminance requirement, but none met the contrast ratio requirement. In the case of the Radiologist Workstation displays, it was possible to modify the environment to meet the contrast ratio requirements. In the case of the Clinical Workstations, it was possible to meet the luminance standards by recalibration, but it was not possible to meet the contrast ratio criterion.

Table 3.

Summary of measurements on primary display systems

Display type	Luminance >170 cd/m2	Contrast ratio >250:1 low light	Contrast ratio >250:1 room light
Primary displays
Radiologist	100%	52%	0%
Clinical	88%	0%	0%

Table 4.

Summary of measurements on secondary display systems

Display type	Luminance >100 cd/m2	Contrast ratio >100:1 low light	Contrast ratio >100:1 room light
Secondary Displays
OR/ER	54%	100%	0%
Processing	87%	66%	33%

For the secondary displays, only just over half of the systems deployed in the ORs and ERs met the luminance acceptance criterion, but all met the contrast ratio criteria in low light. None met the contrast ratio criterion in room light. Because they are all the same types of display purchased at the same time, it is probable that they can be recalibrated to the AAPM/ACR luminance recommendation for secondary displays. However, this may only be a theoretical consideration because the units are used in the high light environment of the ORs and ERs. For the Processing Workstation displays, all but the CRT display met the luminance criterion, and two-thirds met the contrast ratio criterion in low light conditions.

Discussion

It is clear from this work that ambient lighting is critical in a PACS environment, a fact that has been noted previously.¹² Our facility was fortunate in being able to design a new department for primary soft-copy reading of images, so that we could build in the appropriate recessed lighting, dimmer controls, climate control, and furniture. Even so, this survey showed the locations where changes were needed. This is perhaps surprising for the Radiologist Workstations, which already have a consistent Quality Control program. However, this QC consists of calibration to the DICOM part 3.14 using a surface mounted photometer, which perhaps takes less account of the ambient light. Use of a telescopic photometer gives a contrast ratio that includes ambient and reflected light from the environment. The measurement of the effective contrast ratio seems to be a simple way to identify problems associated with stray light.

Little mention has been made of the proposed IEC standards. In fact, all displays tested met the IEC standards except one—the Processing Station using a CRT display that did not meet the IEC standard for contrast ratio (40:1). We have had PACS in place for over four years, and it is the experience of our radiologists that the AAPM recommendations are necessary for primary reporting of CR and DR studies.

For secondary displays the problem is much more difficult. At the present time there are no displays bright enough to meet the AAPM contrast ratio criteria in normal room light, although displays with contrast ratios of over 3,000 and maximum luminance equivalent to room light have been described.¹³

Conclusion

The display systems used in PACS installations are vital in the diagnostic process. Despite this, there have been no recent published standards for the performance of these systems. To obtain an overview of the performance of all the displays used in our center, we measured the maximum luminance and contrast ratio in ambient and low light conditions. These results were compared to the AAPM TG18 acceptance criteria. For primary displays, almost all displays exceeded a luminance level of 170 cd/m², but fewer than half had a contrast ratio greater than 250:1 in low light conditions. For secondary displays, only just over half had a luminance level in excess of 100 cd/m², but the majority had a contrast ratio greater than 100:1 in low light conditions. None of the displays met the recommended contrast ratios in room light. Although the areas using primary displays in our center were specifically designed for PACS use and we have a consistent quality control program, these simple measurements indicated that some changes to ambient lighting or display location were still required.