Abstract
With the growing computing capability of mobile phones, a handy mobile controller is developed for accessing the picture archiving and communication system (PACS) to enhance image management for clinicians with nearly no restriction in time and location using various wireless communication modes. The PACS is an integrated system for the distribution and archival of medical images that are acquired by different imaging modalities such as CT (computed tomography) scanners, CR (computed radiography) units, DR (digital radiography) units, US (ultrasonography) scanners, and MR (magnetic resonance) scanners. The mobile controller allows image management of the PACS including display, worklisting, query and retrieval of medical images in DICOM format. In this mobile system, a server program is developed in a PACS Web server which serves as an interface for client programs in the mobile phone and the enterprise PACS for image distribution in hospitals. The application processing is performed on the server side to reduce computational loading in the mobile device. The communication method of mobile phones can be adapted to multiple wireless environments in Hong Kong. This allows greater feasibility to accommodate the rapidly changing communication technology. No complicated computer hardware or software is necessary. Using a mobile phone embedded with the mobile controller client program, this system would serve as a tool for heath care and medical professionals to improve the efficiency of the health care services by speedy delivery of image information. This is particularly important in case of urgent consultation, and it allows health care workers better use of the time for patient care.
Keywords: Mobile phone, PACS, PDA, filmless, medical images, health care
WITH THE ADVENT of the picture archiving and communication system (PACS) in a filmless hospital environment, there is an increasing demand for immediate access to image display.1 At present clinicians and radiologists can only retrieve relevant images and patient information through workstations with fixed locations and restricted local networks. With the growing computing capability of mobile phones, it is possible to develop a handy mobile-phone controller for accessing the PACS. This mobile-phone enhanced controller can be of use for enhancing image management for clinicians with nearly no restriction in time and location using various wireless communication modes. Clinicians and other health care professionals can remain in touch with medical images, image reports, and imaging workflow status both inside and outside the hospitals, without the need for designated computer platforms and facilities.
The Hong Kong Hospital Authority was founded in 1990 to manage all 44 public hospitals. The Authority has formed seven geographic clusters to manage the workload.2 This wide area image model has led to the need for efficient distribution of images and related patient information. As an adjunct solution to Internet-wide image distribution, the wireless mobile phone network provides a promising solution. There have been initiatives to use the personal digital assistant (PDA) to acquire images from PACS.3, 4, 5 Since 2002, the mobile phone has evolved to be image-enabled with 16-bit color display.6 This opens up a new solution for image display and access of patient information. The mobile phone may provide a better alternative than the PDA because it is handier, with better communication network coverage. Besides, there is no extra cost for computer facilities other than that of the mobile phone in order to access such a system.
METHODOLOGY
In the health care setting, the PACS is an integrated imaging system for image distribution. To streamline the operation of image data exchange among different workstations and modalities, DICOM (Digital Imaging and Communications in Medicine) is the standard for display, storage, and exchange.
The system presented here consists of a mobile controller, a Web server, a mobile application server, and a PACS.
The Mobile Controller
The mobile controller is a mobile phone embedded with mobile controller client software. It can receive a DICOM worklist and perform query and retrieval of images from the mobile application server. Images are displayed in miniature format. For serial images such as CT (computed tomography), MRI (magnetic resonance imaging), or US (ultrasonography) a cine-looping display function would allow an effective review of images. In addition there are pre-set image processing functions such as window-level and window-width. Basically, the mobile controller is composed of five main component modules, namely connection modules, XML (extensible markup language) parsing modules, image handling modules, SMS (short message service) modules, and user interface (UI) modules (Fig 1). The modules are developed in Java language using Java 2 Platform, MicroEdition (J2ME) standard (SUN, http://java.sun.com/j2me/). The mobile controller exists in the form of J2ME midlet suite program, which is compliant with Mobile Information Device Profile (MIDP) 2.0 and Connected Limited Device Configuration (CLDC) 1.1 standard.
Figure 1.
The component modules of the mobile controller.
The connection modules are mainly responsible for handling TCP/IP communication, over the GSM network, between the controller and the server side. The connections used in some submodules are synchronous, but in other submodules they are asynchronous. The use of the type of connections is determined by several factors, which include the interaction between the controller and servers, the responsiveness of other modules, the processing time required by the server modules, and the type of applications.
The XML parsing modules are mainly responsible for handling data exchange in XML format between the controller and the servers. There are two main reasons for using XML format in exchanging data between client side and server side. First XML serves to balance the complexity between the very efficient but non-structural ASCII Text protocol and the very structural but inefficient SOAP protocol (Fig 2). Another reason for choosing XML is to cope with the upcoming version of HL7 (Health Level 7)7, in which XML is being employed as the base data exchange format.
Figure 2.
Comparison chart of different communication protocols.
The image-handling modules are responsible for image reception display, and animation. Together with user interface modules, which are responsible for handling the display of data and image to the user, the image-handling module displays received image in full canvas mode. The UI modules are also responsible for the interaction between the user and the controller.
Finally, the SMS (Short Message Service) modules handle the communication between the controller (mobile user) and the server side client (user resides in hospital). The communication uses SMS as the base transmitting agent. The functions of SMS modules include SMS comment and SMS notification.
The Mobile Application Server
The mobile application server consists of five modules: The PACS application module, the DICOM image-processing module, the data accessing module, the Web servicing module, and the SMS servicing module (Fig 3).
Figure 3.
Modules of the Mobile Application Server.
The Web servicing module (WSM) handles the incoming HTTP requests from the Web server, which in turn is initiated from the mobile controller. The Web servicing module translates any worklist retrieving requests as well as queries to standard SQL (Structured Query Language), which will then be forwarded to the data accessing module for further processing. The Web servicing module will also translate the data returned from the data access module back into XML format, which will then be sent back to the mobile controller.
The SMS servicing module (SSM) accepts requests from the data accessing module (DAM) and sends notification messages in SMS format to the mobile controller. The SMS servicing module also listens to the communication port attached to the GSM terminal for any incoming SMS message from the mobile controller.
The DAM connects to the PACS application module for database data retrieval. The data processing module connects to the shadow database via JDBC and accepts standard SQL queries for accessing data in relational database fashion. This module is also responsible for passing the required requested data to the SMS and Web servicing modules.
The DICOM image processing module (DIPM) retrieves DICOM images from the PACS image repository and converts those images either to JPEG, if the requesting client device is a Pocket PC, or to PNG (portable network graphic) format if the request client device is a Java enabled mobile phone. The PNG format is the format that can be handled by all mobile phones.
The PACS application module (PAM) works as a mini PACS server for receiving and auto-routing of DICOM images as well as worklists from an enterprise PACS. The worklist and DICOM images retrieved also act as an intermediate image repository and data shadow for serving the other server modules. This intermediate layer also acts as a shield hiding the enterprise PACS server and preventing it from being hacked. Figure 4 illustrates the complete mobile controller system using various wireless communication modes.
Figure 4.
The mobile controller system.
The Whole System
The process can be initiated either actively from the mobile controller via user intervention or passively from a notification SMS sent from the remote site server. Before that, the mobile controller must be installed on the user mobile phone. To simplify the installation process, ordinary installation methods including installation from desktop via the IrDA-1.1 (Infrared Data Association) standard or an RS232 cable have been by passed. Instead, the deployment method we adopt is a more dynamic method, the OTA (Over-The-Air) Provisioning (Fig 5).
Figure 5.
Deployment via OTA provisioning.
For the OTA deployment, the user mobile phone will obtain the descriptor location from the SMS center (SMSC) via SMS. The user can then launch the built-in software application manager (SAM) to retrieve the descriptor file from the WAP server via either WAP service or General Packet Radio Services (GPRS). After downloading the descriptor, the SAM will read the descriptor for the mobile controller program files location as they may not be located in the same area of the WAP server. Once the program locations are confirmed, the SAM will automatically download the program files and store them in the phone memory. The user can then install and launch the mobile controller.
In general, the user connects to the remote Web (HTTP) server and browses for the patient’s general information such as patient name, identification number, date of birth, gender, and so on. The connection can be done via one of the following pathways. For mobile user outside the boundary of the hospital, the connection can be made via any of the following networks: GSM-WAP (Global System for Mobile communication) GPRS or EDGE (Enhanced Data GSM Environment), WCDMA (Wideband Code-Division Multiple Access), CDMA 2000, or any similar cellular phone data network. The ultimate connection will pass and convert through IP gateway(s) to the internet world. For mobile users inside the boundary of the hospital, the connection can be made via Wireless Local Area Network (WLAN) standards, such as IEEE 802.11b, IEEE 802.11a/g or via the latest Bluetooth (BT) piconet, using the Network Access Profile.
In browsing for patient information, the request will first be received and handled by the Web server. The Web server will then call up the required service module, In this case, the DAM will be called up (Fig 4). The data access service retrieves the required patient data by sending standard structured query language (SQL) query via one of the database drivers, which depend on the type and brand of the RDBMS (relational database management system) used, to the target data shadow RDBMS of the PACS application module. Once the data return from the PACS application module, the data accessing module will pass the retrieved data to the Web servicing module, where the data will then be converted into XML format and forwarded all the way back to the mobile controller.
In browsing for patient DICOM images, the Web servicing module will also call up DICOM image processing module apart from the data accessing module. The DIPM will retrieve the required DICOM image or a sequence of DICOM images, according to the user’s request. The DIPM will then decode the retrieved DICOM image(s) and convert it/them to a format, either JPEG or PNG, that can be handled by mobile devices (such as a PDA or a mobile phone). For security reasons, the converted image and tagged data will not be physically be saved into a file. Thus, after that, the binary data will then be passed to the Web servicing module and again pushed all the way back to the mobile user.
In retrieving the image reports from the RIS (Radiology Information System) the mobile controller allows two methods: SMS auto-routing and Web-based query/retrieve. For SMS auto-routing, once the RIS receives write-up reports from radiologists, it can route these reports automatically together with the respective patient ID, examination date/time, and phone numbers of corresponding clinicians to the RIS interfacing module at the mobile application server. The data accessing module can collect these data from the RIS interfacing module and encode the data to the SMS content. The formatted SMS content and the corresponding phone number will then be passed to the SMS servicing module and then to the mobile phones of the corresponding clinicians. For Web-based query/retrieve, the requests are initiated by the mobile users. The Web servicing module at the mobile application server will pass the query keys (patient ID and examination date and time) to the data accessing module where SQL statements based on these keys will be constructed. The RIS interfacing module will send these SQL statements to the RIS and then the responses, which are the image reports, will be sent back to the data accessing module. The SMS can be encoded by the data accessing module. The reports are then sent, along with the SMS, to the mobile users through the SMS servicing module.
The module controller is designed to have a number of functions and features, such as browsing the patient’s general information, as mentioned before. It allows mobile user to browse for the patient examination information, history, diagnosis, and so on. For image browsing, the DICOM image and its tag information can be retrieved and displayed in a way that supports a variety of image processing options. For these, most of the operations are done by the server side modules.
RESULTS
We have implemented a prototype mobile controller for connection with the PACS in the Hong Kong Polytechnic University. The PACS has been designed for research, teaching, and clinical use.8, 9, 10
Performance of the Mobile System
The patient general information is first obtained by the mobile phone client. In order to proceed to the next step, the user must select the patient we want to browse (Fig 6). After that, a list of patient examination history records will be retrieved and displayed. By scrolling and browsing for the list, the user can learn the patient’s clinical history. To view a particular DICOM image, the user must choose a particular study ID with examination date and time from the screen. Afterwards, a list of examination image information will be displayed, from which the user can choose to browse the image detail information, to preview a particular image, to view a sequence of images in cine-mode, or to view the report of that examination. To preview an image, the user needs to select “Get Image Preview” from the menu. Once selected a preview image of the same width as the mobile phone width will be obtained. To facilitate image viewing, several preset window width and window level options can be chosen from the mobile phone menu. The preset window/level settings are those for optimizing the viewing of different regions such as chest, bone, etc.
Figure 6.
Query of patient as it appears in the mobile client.
To allow efficient image viewing of serial images such as CT or MRI, we integrated the cine-mode for image display so that the user can review serial images without scrolling. A typical CT image display in a mobile phone is shown in Figure 7. We found that this function provides a useful preview function. However, we caution users that when handling image series with large volume, the retrieval time may be as long as 20 minutes. Certainly, the download time will greatly depend on the type of phone, the bandwidth of the cellular phone network, the operator services, the Internet traffic at a particular time, as well as the performance of the servers used.
Figure 7.
The CT image as it appears in the mobile phone.
Image Retrieval Time
At present image access by the mobile phone depends on GPRS transfer when the user is outside the hospital complex. The data transfer rate is around 56 to 114 kbps. Image access inside the hospital is mainly through a local network, using image viewing workstations. If a mobile phone is selected for image transfer, it can be accomplished through Bluetooth. At present the transfer rate of Bluetooth is 723 kbps. In our mobile phone system with GPRS, access time of a CR image is 8 seconds. For CT images of 24 slices, the access time is 191 seconds.
Image Quality of the Mobile Phone
The phone memory usually is less than 4 MB; the heap memory is even less. However, a typical single slice of CT image of resolution 512 × 512 pixels is around 0.5 MB. To allow most of the mobile phones to read the image as well as to greatly reduce the download time, we must resize both the preview image and the zoom-in image to a certain percentage of the original image size. This process will reduce the image detail to less than half of the image detail of the original DICOM image.
A typical mobile phone offers 128 × 160 pixels or 176 × 208 pixels resolution of display, with color depth of 65k to 260k colors. The display format is by RGB stripes arrangement. This display is not sufficient for diagnostic viewing of a typical 512 × 512 pixel CT at any single shot of view, let alone a typical CR image of the chest (2048 × 2048 pixels). To view the whole image, the user must move the image in the small viewing window.
DISCUSSION
A mobile phone is traditionally regarded as a means for voice communication. Early mobile phones were very limited in their ability to display medical images because of the hardware and software constraints related to both image resolution and image depth. Images from CT and MRI are typically presented by a 512 × 512 and 256 × 256 pixel matrix, respectively, with 12-bit image depth. With the sub sampling and dynamic window setting method, the mobile client is able to display CT, MRI, and even CR (computed radiography) image on the mobile phone for preliminary viewing.
The most common image format in PACS is DICOM,11 where textual information and image data are combined in the image file. The DICOM format is designed for imaging scanners, computed radiography, film printers, and display workstations. At present there is no display capability in mobile phone for DICOM images. Therefore, to allow display of a DICOM image in a mobile phone, a specially designed client-server program is required to convert the DICOM image to a format that can be handled by mobile phones.
This preliminary work is an integrated system used for health care and hospital environment. With it, the clinicians and health care professionals can access image and clinical information from the mobile controller. The system is linked with an enterprise integrated PACS where images and patient information are maintained. It would enhance a medical consultation if there were no restrictions in time and location. Accessing PACS through a mobile phone network provides that advantage. Such system may also allow patients to receive imaging and related reports by a short message through their mobile phones. The system enhances speedy delivery of imaging and health care information.
At present, image access through mobile phone is an emerging technology. There is no standard for access to medical images through a mobile phone. The interfacing of mobile environment with medical information system is a significant challenge for IT workers. The system reported here guarantees streamlined interfacing between these different systems. In addition to the conventional mobile communication methods such as wireless LAN (IEEE 802.11a/b/g) and Bluetooth (Class 2), the system we have developed can communicate through mobile phone (e.g. PDA phone, smartphone) by GSM-WAP (data transfer rate at 9.6 kbps) or GPRS (data transfer rate at 114 kbps) as the application layer is entirely separated from the network communication layer. The system is capable of making full use of the upcoming cellular phone network, such as EDGE, with a data transfer rate of 384 kbps, and 3G (e.g. WCDMA, CDMA200, etc.) at around 2 Mbps. The 3G network has speed similar to the current broadband network, but the mobility is far greater than that of the broadband network. Currently, 4G (broadband mobile communications) is being tested in some Japanese companies; the data transfer rate is as great as 20 to 40 Mbps.
The system would serve as a tool for heath care and medical professionals to improve the efficiency of the health care service by speedy delivery of imaging information from different imaging modalities such as CT (computed tomography), CR (computed radiography), DR (digital radiography), US (ultrasound) etc. This system needs only a mobile phone embedded with our client program. No complicated computer hardware or software is necessary. Availability of this system enables health care workers to monitor their patients from a distance, even when they are off-duty or at leisure. This is capability particularly important in the case of urgent consultation, yet allows professional workers greater flexibility in their use of time. Undoubtedly, the system could be of value for referring clinicians to obtain relevant images and accompanying diagnostic reports. However, what we have established to date is the feasibility for mobile communication for medical images. Further tests are required to evaluate its value in clinical context.
CONCLUSIONS
We have demonstrated the feasibility of using mobile phone, in addition to PDA, as a client workstation in accessing digital images through interfacing with PACS and RIS. Although the image detail is limited by its resolution and display intensity in the mobile display, the mobile controller can provide speedy delivery of images such that radiologists can preview images before they have access to an image workstation and clinicians can review reported images through mobile phone display.
Acknowledgment
The authors acknowledge with thanks the funding from Departmental General Research Grant (G-T915) of the Hong Kong Polytechnic University.
References
- 1.Huang HK. PACS: Basic Principles and Applications. New York: Wiley; 1999. [Google Scholar]
- 2.Inamura K, Kousaka S, Yammamoto Y., et al. PACS development in Asia. Comput Med Imaging Graph. 2003;27:121–128. doi: 10.1016/S0895-6111(02)00085-X. [DOI] [PubMed] [Google Scholar]
- 3.Andrade R, Wangenheim AV, Bortoluzzi MK. Wireless and PDA: a novel strategy to access DICOM-compliant medical data on mobile devices. Int J Med Informatics. 2003;71:157–161. doi: 10.1016/S1386-5056(03)00093-5. [DOI] [PubMed] [Google Scholar]
- 4.Engelmann U, Schroter A, Boralv E, et al. Mobile teleradiology: all images everywhere. Int Congress Series. 2001;1230:844–850. doi: 10.1016/S0531-5131(01)00143-1. [DOI] [Google Scholar]
- 5.Li M, Wilson D, Wong M. The evolution of display technologies in PACS applications. Comput Med Imaging Graph. 2003;27:175–184. doi: 10.1016/S0895-6111(02)00072-1. [DOI] [PubMed] [Google Scholar]
- 6.Nokia: Nokia 7600 imaging phone features, 2003. http://www.nokia.com
- 7.Health Level Seven: Health Level Seven Standards, in http://www.h17.org/.2003
- 8.Law MYY, Zhou Z. New directions in PACS training. J Comput Med Imaging Graph. 2003;27:147–156. doi: 10.1016/S0895-6111(02)00088-5. [DOI] [PubMed] [Google Scholar]
- 9.Law MY, Tang FH, Cao F. A PACS and image informatics training. Radiology. 2001;221:120–120. [Google Scholar]
- 10.Tang FH, et al. Implementation of a PACS for radiography training and clinical service in a university setting through a multi-national effort. In: Siegel EL, Huang H, et al., editors. Medical Imaging 2001: PACS and Integrated Medical Information Systems: Design and Evaluation, Proceeding of SPIE, 4232. San Diego: SPIE; 2001. pp. 67–72. [Google Scholar]
- 11.NEMA: Digital Imaging, Communications in Medicine (DICOM). Part 3.14: Grayscale Standard Display Function. Rosslyn, VA, National Electrical Manufacturers Association, 2000