PACS Experience as a Motivation for a Campus-wide Picture Network

INTRODUCTION

PICTURE ARCHIVING AND COMMUNICATION SYSTEMS (PACS) have so far been studied primarily as a tool to address the problems of electronic radiology. Certainly the trends toward digital imaging within radiology provide strong economic and medical incentives for the development of medical picture networks, but we believe that the applications for picture networks extend far beyond the field of radiology. Architects, engineers, biologists—practitioners of virtually every academic and commercial pursuit—deal with picture information every day, and increasingly these pictures are finding their way into digital form. We believe that industries and universities of the future will utilize sophisticated workstations serving a variety of scientific and commercial needs, and that these workstations will be linked by wideband networks capable of supporting not only text but high-resolution picture transmission as well.

The technical problems are similar both within and outside the field of radiology. In order to develop an electronic picture network, it is necessary to develop high-performance, low-cost components which will serve each of the three major elements of a PACS—archive, network, and display. The high data rates required for picture transmission will stretch performance requirements far beyond those of a typical local area network, and require state-of-the-art technology for many elements of the system design.

We continue to find that it is difficult to define and develop the basic PACS elements independently or without the application area context. Decisions regarding network configuration, for example, affect design decisions for the organization of a display. A comprehensive approach is required and, consequently, prototype PACS networks serve a useful purpose as a test bed for the evolution and evaluation of system concepts and component design.

Washington University has made a significant committment to the development of a prototype PACS network. Initially, in 1981, modeling experiments were carried out to define the requirements of a picture network suitable for radiology.¹ A prototype broadband network was designed, built, and installed in the medical center in 1983. Comprising approximately 1.5 miles of cable, the network served as an experimental laboratory for the transmission of video as well as low- and high-speed digital data.² In 1984, links were established within the Mallinckrodt Institute of Radiology (MIR is the radiology department of Washington University) between the prototype image management system and the comprehensive patient information system developed at MIR, making it possible, at prototype workstations throughout the department, to retrieve image data stored in a prototype central image archive.³

In 1985, the scope of the prototype broadband network was increased as the Institute for Biomedical Computing at Washington University established an image presentation, analysis, and quantification (IPAQ) facility. The goal of this major project is to provide additional network connectivity supported by a center capable of providing tools and computational assistance required for the processing of medical images.

In 1985, based on earlier PACS work at the medical center, a three-year project was begun to design and develop a campus-wide picture network capable of spanning the University from the medical center campus to the hilltop campus, three miles away. The objective of this fifteen-million-dollar project, undertaken in cooperation with Digital Equipment Corporation, was to develop a state-of-the-art wideband network capable of manipulating and transmitting pictures in addition to symbols and graphs. This picture network will cover all schools and departments of the University and will eventually extend outward into the community served by the University.

Throughout this period of aggressive expansion of PACS research at Washington University, the enduring objective has been to establish a “workbench” for PACS development—to provide the tools, the resources, and the environment where experiments can be carried out to evaluate various alternatives for the design of the components required for network, archive, and display. Our goal at this time is not to build the ideal PACS network, but rather, in an ambitious time frame, to design and construct, in cooperation with industry, a large scale PACS prototype that will serve as a proving ground for picture networks of the future.

PACS RESEARCH AND DEVELOPMENT ACTIVITIES

MIR PACS Workbench

A comprehensive radiology image management system requires the support of an equally comprehensive patient information system. The technical demands of a radiology information system (RIS) are significantly different from those of an image management system, and at the Mallinckrodt Institute of Radiology the two systems have been developed on separate computer systems. A loose coupling provides an electronic interchange of messages for coordination between the two systems, allowing them to be physically located in different portions of the department.

A major expansion and redesign of the radiology information system is underway. Currently, a network of PDP-11 and VAX computers supports about 200 terminals within the radiology department. Significant expansion of terminal support is anticipated in the coming year with office automation services being made available to doctors’ office areas. Until now, in order to support continued growth in a multiprocessor environment, a “back-end” database management computer has been linked to the other computers in the network. This philosophy will be preserved, but the new system will be based on a multiprocessor DEC-cluster configuration providing additional hardware and database redundancy. VAX computers will replace the PDP-11 systems which now support application programs. Terminal support will be provided through a number of Ethernet segments, both baseband and broadband, allowing users to have facile access to any computer and to any application supported within the network. The scope of the network will expand, although it already spans several buildings within the medical complex, and consists of an end-to-end length of several kilometers (Figure 1).

Figure 1.

MIR computer network.

Links have been established with the prototype image management system so that from any of the terminals in the department it is possible to request images from the image database. Methods for optimal linkage between patient management and image management are under study. A commercial information system (DECRAD) will be installed in a portion of the department and methods for linking this system with our own radiology information system and with our prototype image management system will be explored.

The presence of the existing computer network and its compatibility with some of the radiology equipment in the department, namely MRI and Nuclear Medicine, have promoted its use as an early test mechanism for acquiring digitized images on a routine basis. In the spirit of our PACS workbench approach, we find that many experiments for both image source-to-archive and archive-to-display can be carried out with moderate-speed digital transmission (average transfer rate less than 10 megabits per second). We are in the process of linking additional digital image sources (CT and Film Digitization) through Ethernet and IEEE-488 to a central archive. Interfaces to existing radiology imaging equipment, however, remain a significant problem in some cases. Although direct links to the two MRI systems have been established, floppy disk transfer is still required to gather information from CT scanners because of an operating system limitation. Digitization and acquisition of the Nuclear Medicine images have been accomplished with PDP-11 computers linked to each of the gamma cameras. A VAX computer collects all of the radionuclide studies and in turn transmits both demographic and image data to the MIR image network via Ethernet.

Ethernet, we believe, is suitable for transmission from source to archive, but will in many cases fall short in high-volume PACS use when transmitting images from archive to display. Higher speed links are under evaluation for future incorporation into the network to meet this requirement. Video distribution, in some situations, offers an alternative to high-speed digital transmission.³ Digitized images may be transferred in parallel at high speed from an archive to nearby digital buffers, allowing digital-to-analog conversion to take place centrally. The video-based image is then distributed to the viewing site. An early prototype configuration using this method is currently under evaluation. Three 1024 × 1024 image buffers within a Gould IP-8500 display system are timeshared among several users. These three high-resolution video outputs are then linked to a computer-controlled video switch, which can simultaneously route the three images to any of several viewing stations (Figure 2). In this scheme, the user requires only a standard computer terminal linked to an RIS and a high-resolution video display linked by coaxial cable to a central image display system. This expensive central system may be timeshared among several users and basic features such as zoom, pan, and intensity transformation are all controlled from the user’s terminal via low-speed communications.

Figure 2.

MIR video image distribution.

IPAQ: A Distributed Facility for Image Presentation, Analysis, and Quantification

Decentralized computing organized along departmental and research laboratory lines characterizes the evolving environment at most major medical research institutions. This is particularly true in quantitative imaging where many projects have diverse image-data sources, different data-acquisition requirements, and dissimilar methods for the extraction of quantitative information. Rather naturally, diverse computing styles and equipment choices have evolved. For example, major collaborative research groups at the Washington University Medical Center support installations tailored to their specific measurement and picture transformation needs, for which display peripherals from a variety of manufacturers (DeAnza, Ramtek, and Lexidata) are tightly coupled to different computers (Digital Equipment VAX 730, 750, 780, and Microvax II; Perkin Elmer 3230, 3242). Commonality is limited to little more than the popularity of VAX-class computers and a FORTRAN programming environment. Furthermore, the lack of common program-development tools and display-support software has minimized the opportunities for sharing results across research programs and has necessitated large host-specific investments.

We have begun a research and development program whose goals are the implementation of an environment for biomedical image presentation, analysis, and quantification (IPAQ). This activity is sponsored by the NIH Division of Research Resources Biotechnology Resource Program and is centered within the Institute for Biomedical Computing at Washington University. The implementation is based on a distributed approach, which provides connectivity to support access to data acquired by specialized imaging instruments and databases maintained within collaborating laboratories. Remote presentation of images can also facilitate timely interaction between the biomedical investigators and the computer technologists focused on algorithm and technology development. As algorithms mature, exportation of either software- or hardware-based implementations can be supported and maintained via the network.

The initial configuration for the IPAQ network will connect a modest number of sites (<10) that are located within several buildings at the Medical campus. Sites include Clinical Radiology, Cardiology PET, Neurology PET, Neuroanatomical Imaging, and EM Autoradiography (Figure 3). The picture transport required by IPAQ is similar to that evolving in MIR. The IPAQ network will capitalize on development at MIR of the PACS workbench. Approximately 10% of the bandwidth (6 standard TV channels) of the broadband CATV-based network is allocated for the use of IPAQ. In addition, the equivalent of 3 TV channels supports a broadband Ethernet on the dual cable system.

Figure 3.

IPAQ interconnect topology.

In contrast to the clinical radiologists using PACS, the users of the IPAQ facility are more likely to tolerate demanding set-up procedures and slower response times in exchange for increased flexibility. The small number of nodes and the modest traffic expected will permit careful measurement and evaluation of both transport facilities and selected communication protocols relevant to both IPAQ and radiology use of PACS.

A locally developed low-cost picture buffer of 512 × 512 pixels provides both a parallel interface for computer interfacing and a high-speed serial port for network experimentation. This activity addresses the problems associated with implementation of a cost-effective network organization. Previously reported studies⁴ conclude that segmentation of the message and picture service network and implementation of the picture service network using multiple shared-receive channels offers a favorable trade-off between cost and performance. The broadband network will host experiments with both shared channels and point-to-point channels at rates of greater than one megabit per second. The broadband Ethernet will initially support both a message network and a digital picture service network. Experimentation with low-cost, high-speed modems is continuing.

CAMPUS-WIDE NETWORK

The experience with the PACS workbench in MIR and with the IPAQ project in the Institute for Biomedical Computing provides a foreshadowing of the value of pictures as a method of interacting with computers. That this method will grow in importance is assured by the fact that memory and secondary storage costs continue to fall. A handful of microelectronic chips held the data for a screenful of alphanumeric characters in 1970. Similarly, by 1980, a handful of chips held a screenful of graphics information. Current projections suggest that by 1990, a million-pixel color picture can be stored in just a handful of chips. In the 90’s, displays capable of presenting high-quality pictures are likely to be as commonplace as alphanumeric displays were in the 70’s.

The ubiquity of picture displays will be an important factor in the promise of the electronic radiology department of the next decade. But beyond MIR and IPAQ, pictures will play an increasingly important role in computing throughout the university. In the Department of Neurology, the Laboratory of Neuro Imaging (LONI) applies image processing techniques to the study of the relationships between brain structure and function. The Department of Computer Science applies similar techniques to the process of abstracting cartographic information from satellite imagery. In Pharmacology, synthetically generated pictures of biologically-active molecules help researchers design new drugs. The School of Architecture, in its Urban Research and Design Center, generates synthetic images of a city’s buildings that can be viewed from any angle and can be changed to incorporate any proposed buildings. In the Department of Earth and Planetary Sciences, images from space vehicles are analyzed and stored in an image database for easy retrieval.

These samples are but a few of the many applications of the interaction of pictures with computers. As image presentation and analysis technology becomes more affordable, research and teaching activities throughout the University will make use of many similar examples of computational imaging.

It was the realization of this potential that suggested the expansion of the MIR and IPAQ network activities to a program for campus-wide picture transmission. Work toward this larger network has been underway, in cooperation with the Campus-Wide Program of Digital Equipment Corporation, since May, 1985. As of January 1, 1986, there are 72 nodes in our Campus-Wide network, and each corresponds to a host computer capable of supporting a number of terminals and workstations. The 72 nodes are distributed over 15 departments in five schools. We expect the number to grow over the three-year term of the program so that, by 1988, the network will serve every department in the University’s nine schools (Figure 4).

Figure 4.

Campus-Wide network.

The campus-wide network is based on Ethernet, but has an open architecture. Both DECNET and TCP/IP protocols now coexist on the network. Links to IBM and Apple-talk subnets have recently begun to function through protocol translator. Physically, the network has a variety of transmission media, including baseband, broadband, optical fiber, and microwave. Bridges connect subnets at the data link layer, isolating local traffic and forwarding non-local traffic independent of the network (or higher) layer protocols.

The development of the MIR and IPAQ subnets contemporaneously with a Campus-Wide network provides important synergism. Pictures and images are only part of the traffic in specialized networks such as the MIR and IPAQ subnets. These networks must be capable of carrying other information even though the major portion of their capacity is taken up by images. They can benefit by the Campus-Wide experience in this regard. On the other hand, experience gained with picture transmission in the MIR and IPAQ subnets will guide the development of Campus-Wide picture transmission. Furthermore, the wide variety of applications throughout the campus will give a base of experience from which a robust and flexible picture network can evolve.

DISCUSSION

Over the past four years, our ideas about PACS have matured, particularly as a result of experience with the PACS workbench installed in 1983. It is therefore an appropriate time to update some of the data previously reported,¹ to review the estimates of network response time and network loading, to evaluate alternatives to wideband networks, and to discuss these factors in the context of the Washington University Campus-Wide network.

Summary of Studies Performed at MIR in 1985

The studies performed at MIR in 1985 are categorized in Table 1 according to those presently digitized, chest x-rays, and other radiographic procedures. The number of images per study and array-size parameters are provided. Average study size in standard picture frames (1 spf = 512 × 512 × 8 bits) and cumulative average study sizes are also tabulated. Comparison to previously tabulated results (contained in parentheses) demonstrates a significant increase in both quantity of digitized studies (by a factor of 2.7) and an increase in the average study size (also a factor of 2.7). A small increase of approximately 8% is seen in the total number of studies.

Table 1.

Clinical Studies Performed in 1985

	1985 Studies	Cumulative Study Total	Images Per Study**	Array Size (bits)	Array Size (spf)	Average Study Size (spf)	Cumulative Average Study Size (spf)
Presently Digitized
CT head	9,855		27	512 × 512 × 11	1.375	37.0
CT body	12,041		22	512 × 512 × 11	1.375	30.0
MRI	2,163		24	512 × 512 × 11	1.375	33.0
Nuc Med	13,883		32	64 × 64 × 8	0.015	0.5
ICU chest (est)	2,750	40,692 (15,024)*	1	1024 × 1024 × 8	4.000	4.0	20.1 (7.5)*
Chest X-rays	105,226	145,918	1.92	1024 × 1024 × 12	6	11.5	13.9
Other Exams	125,025	270,943	3.92	1024 × 1024 × 12	6	23.5	18.3
Chest plus other exams	230,251	270,943(249,871)*	2.98	2048 × 2048 × 12	24	71.5	63.8***

*Numbers in parentheses correspond to results tabulated in 1981.

**Based on 1981 analysis of images per study.

***This entry is obtained by increasing the array size to 2048 × 2048 × 12 for both chest and other exams.

Network Response Time

The estimates of network activity on which previous response calculations were based changed only slightly as a result of the 8% growth in total studies observed for 1985. The primary influence on response time is seen to be increased study size. As an approximation, the lower bound on response time was found to be directly proportional to the average study size.¹ To achieve equivalent network response for transport of a complete study from archive to display console, the channel capacity would need to be increased by a factor of 2.7. For example, transmission of a 20 spf study in a time of two seconds or less requires a channel capacity of greater than 20 Mbits/sec. This lower bound does not include queuing delays imposed by channel sharing.

Picture sources have internal buffering which allows considerable leveling of the uneven generation rates, and these rates are expected to be an order of magnitude less than the retrieval rates.¹ Thus the requirements for wideband channels will continue to be dominated by retrievals.

Careful attention should also be given the possible limitations imposed by the technology chosen for the archive devices and the system constraints on record size. A brief comparison of two candidate storage technologies is provided in Table 2.

Table 2.

Characteristics of Storage Technologies for Radiology Archives

	Transfer Rate	Average Seek Time
Magnetic Disk	2.4 MB/sec	18 msec
Optical Disk	0.5 MB/sec	200 msec

Although precise calculations would have to be based on specific software/hardware implementations, some general conclusions are possible. A file structure tailored to support physical block sizes of the order of 1 spf can yield average transfer rates which are calculated to be in the order of 60% to 80% of the peak transfer rates. Studies of actual disk system performance of VAX/VMS support these calculations.⁵ Thus either magnetic or optical disk-based archives can support megabit output rates. If nominal block sizes currently found in file systems (order of 512 bytes/sector) and fragmented files are assumed, the seek time overhead reduces the effective transfer rates to the order of 0.5% to 2% of the peak transfer rates. This results in insufficient overall network bandwidth unless enhancements to typical file access procedures are employed.

Alternatives for Bounding Network Response Time

Although wideband digital networks are an important component of high-performance PACS, there are alternatives that can reduce the demand for bandwidth. In the following paragraphs, several of these alternatives are discussed.

Analog transmission of video signals is an attractive alternative to digital transmission of images in certain circumstances. CATV technology is relatively low cost and widely available, but transmission is limited to entertainment television standards (525 lines) if advantage is to be taken of the industry’s high volume components. We believe that electronic radiology and much imaging in scientific research calls for substantially higher resolution than 525 lines, except of course, where the nature of the source limits the image to no greater than entertainment quality.

Images with 1,000 line resolution require transmission components with up to 18 MHz. Baseband components are available with this bandwidth, but frequency multiplexing of such signals is costly and difficult to maintain. Thus a star topology of baseband video analog channels is required, connecting an image archive at the center of the star to high resolution monitors at the points of the star. This approach is limited to a small number of monitors in a relatively small geographic area.

If any of these conditions apply (limited resolution, limited number of monitors, limited geographic areas), analog transmission of video signals may be an approach to bounding network response time. Of course, the archive response time will always establish a lower bound on overall response time.

More generally applicable than the above approach is an alternative that couples limited analog transmission with a set of distributed archives. In this case, the traffic on the digital portion of the network is reduced to rates comparable to the generation of images rather than the request for images. In electronic radiology, this gives a reduction of at least an order of magnitude.¹ The analog portion of the network is a multiple star topology. Stars centered at each of the distributed archives support image transmission to monitors by baseband video transmission over limited distances.

This second alternative, distributed archives, can yield response time bounded by the archive response, but requires either substantial duplication of image storage or considerable locality of image use. Depending upon the evolution of storage costs and image usage patterns, this alternative may prove attractive in the future.

Data compression is a third alternative to wideband PACS networks. For radiological images, noise-free compression⁶ provides less than an order of magnitude compression, a factor insufficient to make a substantial decrease in required bandwidth. Noisy compression⁷ can give more than an order of magnitude reduction in required bandwidth, but has limitations if complete freedom to remap the gray scale or magnify the image is required by the user.

Progressive transmission schemes⁸ give the viewer the illusion of rapid network response without the undesirable properties of noisy compression. At present, it is difficult to predict whether wideband transmission facilities or special hardware for the reconstruction of images transmitted progressively will be more expensive. In any case, progressive transmission only provides the illusion of rapid network response; extensive trials would be necessary in each application area to verify that the illusion rather than the fact of rapid response meets the needs of the user.

Larger buffers can reduce network response time by allowing full advantage to be taken of peak transfer rates both over the network and along the bus connecting a disk controller to its buffer. As pointed out above, average transfer rates may be only a few percent of peak rates if a single sector is transferred for each rotation of the disk. Buffers large enough to hold an entire image are not available in typical operating systems and even a quarter of an spf requires a specially configured system. Nevertheless, it seems important to tailor operating systems to PACS service, particularly to avoid a bottleneck resulting from low average transfer rates from a disk archive.

Anticipating image requests by asking the operating system to prefetch data can be effective in reducing the apparent network response time. Of course, this technique is limited to circumstances in which the user’s requests are predictable, rare in a research environment, but applicable in some circumstances in electronic radiology. Prefetch must be coupled with large buffers both at the archive and at the viewing station to be effective.

Alternatives for Limiting Network Traffic

As discussed above, wideband digital networks satisfy the user’s requirements for heavy network traffic as well as rapid network response. There are alternatives that limit network traffic just as there were alternatives that bound network response time.

Subnets are the obvious approach to reducing global network traffic. Bridges between subnets carry global traffic, but prevent the transmission of local traffic. The subnet approach works well if substantial locality of traffic can be identified and designed into the network topology. If such locality cannot be identified or if it is dynamically varying, much less significant reductions in traffic can be anticipated.

Compression reduces traffic as well as improves response time. The same difficulties arise with this approach as were discussed above. Progressive transmission leads to no reduction in network traffic.

Prefetch of image data reduces peak traffic as well as response time. The prefetching may need to be carried out well in advance of use in order to have a significant load leveling effect, particularly if there is a peak of substantial duration in requests for viewing images. This circumstance will exacerbate the problem of anticipating requests discussed above.

CONCLUSIONS

None of the alternatives to wideband links for bounding network response time or limiting network traffic apply under all circumstances. Only compression and prefetch improve both response time and network loading, and even these approaches have serious limitations.

At the present time, bottlenecks in archive storage limit the overall bandwidth and response time that can be realized in a PACS, no matter how large the network bandwidth. Until these bottlenecks are eliminated by technological progress, it makes little sense to press for the most advanced transmission capabilities in the network.

The pace of technological progress and the demand for network bandwidth are trends that must be carefully studied in each of the many applications throughout our Campus-Wide network. In research environments such as IPAQ, expected response time and anticipated network load are much less demanding than in electronic radiology. Segmented Ethernets in these environments will provide sufficient bandwidth, we expect, for the near future.

Electronic radiology will undoubtedly lead the way in its requirements for additional network bandwidth. For limited projects, and for the near term, Ethernet coupled with analog transmission will be sufficient for anticipated network traffic. By the time demand has outstripped this interim network capability, we anticipate that a set of new technological opportunities will be available that will provide the necessary network bandwidth at reasonable cost.