The Empirical Foundations of Teleradiology and Related Applications: A Review of the Evidence

Abstract

Introduction: Radiology was founded on a technological discovery by Wilhelm Roentgen in 1895. Teleradiology also had its roots in technology dating back to 1947 with the successful transmission of radiographic images through telephone lines. Diagnostic radiology has become the eye of medicine in terms of diagnosing and treating injury and disease. This article documents the empirical foundations of teleradiology. Methods: A selective review of the credible literature during the past decade (2005–2015) was conducted, using robust research design and adequate sample size as criteria for inclusion. Findings: The evidence regarding feasibility of teleradiology and related information technology applications has been well documented for several decades. The majority of studies focused on intermediate outcomes, as indicated by comparability between teleradiology and conventional radiology. A consistent trend of concordance between the two modalities was observed in terms of diagnostic accuracy and reliability. Additional benefits include reductions in patient transfer, rehospitalization, and length of stay.

Introduction and Background

This is the sixth in a series of articles aimed at assessing the empirical foundations for telemedicine interventions as indicated in the scientific research literature. Each article in the series has been focused on a clinical application, a disease entity, or a coherent set of clinical applications. Collectively, the purpose of the series is intended to establish the scientific evidence regarding the feasibility and effects of telemedicine interventions from the published research literature. The evidence is organized on the basis of well-defined applications or disease entities. However, we expanded the scope of this review in the last article by focusing on primary care as a broad medical specialty that encompasses a broad range of disease entities and typically serves as a first point of entry in the health system and a coordinator of care for patients. In this article, we focus on a critical diagnostic specialty that constitutes an essential element in most, if not all, other clinical specialties and subspecialties. Indeed, much of modern medicine would not be possible without radiology. It is important to note that although our focus is on teleradiology as it applies in a typical department of radiology, radiographic images are often acquired and used in a variety of other clinical specialties. Hence, some of the evidence presented in this review highlights some of those applications when they are particularly compelling or present a unique use of teleradiology.

Interestingly, radiology is the only medical specialty that stemmed from serendipity. In 1895, Wilhelm Roentgen, Professor of Physics at the University of Wurzburg in Germany, discovered as he was experimenting with the conductivity of electricity through gases that a type of electromagnetic radiation (that he termed X-rays) could pass through various substances (like the human body) that are opaque to visible light. The first X-ray image of the human body was of his wife's hand exquisitely showing the bones and her wedding ring! He received the first Nobel Prize for physics in 1901. Roentgen's discovery led to developments in fluoroscopy (real-time moving images of the interior of an object), angiography/arteriography (radiography of an artery after injection of a radio-opaque substance), and computed tomography (CT).

At the risk of explaining the obvious, radiology is concerned with the acquisition and interpretation of images of the inside of the human body for the screening, diagnosis, and treatment of many diseases and abnormalities.^1,2 It utilizes a variety of electromagnetic and related modalities and techniques to acquire these images.^1–10 Teleradiology is based on the electronic capture, transmission, storage, and retrieval of images for remote viewing and interpretation.

Radiologic technologists are trained to perform imaging examinations and administer radiation therapy and thus are often directly in contact with patients during examinations.¹¹ Radiologists are medical doctors trained in diagnosing and treating injuries and diseases using the images acquired by various modalities and they also perform a variety of minimally invasive interventions such as biopsies and embolization to block blood flow. Radiographic images are also used in dentistry and teledentistry, but these are beyond the scope of this review, and will be touched upon only briefly.¹²

Radiology examinations are used for a wide variety of purposes, including (but not limited to) detection of injuries such as broken bones and ligament damage; detection of diseases such as cancer, arthritis, or Alzheimer's; measurement of disease response to therapy (e.g., tumor measurement before and after chemo or radiation therapy); screening for diseases such as breast, prostate, and lung cancer and tuberculosis; quantitative measurement of anatomic and functional processes (e.g., bone density, biomarkers for cancer and other diseases); image-guided minimally invasive interventions such as angiography, embolization, balloon angioplasty, needle biopsies, and vertebroplasty; assessing functional capacity and performance of organs such as the heart and lungs; and monitoring fetal growth and health. Radiation oncology¹³ is a clinical specialty that treats cancer by destroying cancer cells using high-energy radiation. Radiation oncology departments are generally independent of radiology departments, although they work closely and both use teleradiology, although for different purposes (some of which will be highlighted in this review).

Modalities of Teleradiology

Today, it is difficult to distinguish between regular radiology and teleradiology as the typical workflow, from the perspective of the radiologist, is very much the same. Nonetheless, for this review, it is important to define and clarify basic terms and elements that apply to both types of practices.

The most common type of radiology examination is the X-ray or plain film image. A generator creates a beam of X-rays that is transmitted through the part of the body under consideration or organ of interest. X-rays are absorbed by the tissues they pass through in different amounts, depending on tissue density and composition. Bones absorb more than soft tissues such as muscles. The beams that are not absorbed pass through the body and are recorded. Originally, X-rays were recorded on special radiographic film, but in the 1980s, digital detectors were developed, leading to the digital revolution in radiology. Today, nearly all X-ray images are acquired using digital detectors and are displayed on computer monitors rather than printed on film.

Before the advent of digital radiography (DR), images were acquired and viewed on sheets of film that were placed on a light box for viewing and interpretation. Digital images dramatically changed this practice and it paved the way for viewing images on computer monitors and manipulating them in ways that were not possible with film. Radiologists can employ window/level techniques to adjust the contrast of images, select points of interest, use zoom/pan (instead of a magnifying glass and hot light) to view fine image details at high resolution, and also employ a wide variety of image processing and image analysis tools to extract clinically relevant information from the images and render more accurate decisions.

Modern teleradiology had its origins in the development of DR. For over 80 years since the inception of radiology, patient care depended on film-based capture and subsequent viewing of the images on a light box. Film-based images had to be transported throughout the hospital, between hospitals, and between various locations, sometimes at appreciable distances from each other. This system was labor-intensive, time-consuming, inconvenient, and not consistently reliable due to loss or misplacement of images.

The plan for replacing all X-ray films into an electronic-based imaging system began in earnest in the 1970s. The work grew out of research with video-based and other digital detector systems.^14,15 The first digital subtraction angiogram was successfully performed and reported in 1977, followed by other digital imaging applications in radiology.^16–18 These developments paved the way for direct transmission of digitally acquired images for the practice of teleradiology.¹⁹

Computed radiography (CR) uses a photostimulable storage phosphor cassette to store digitally produced X-ray images. The data are subsequently read out using a stimulating laser beam to generate a digital image. It is often referred to as a passive detector system. DR is an active detector system that uses a flat-panel digital system to read the transmitted X-ray data right after they are created with the detector in place. Both systems have advantages and disadvantages, but both are used in nearly every hospital and clinic (as well as in dentistry and veterinary medicine) for inpatient, outpatient, and teleradiology examinations. Mammography, full-field digital mammography (FFDM), and digital breast tomosynthesis (DBT) are also X-ray-based examinations.²

Fluoroscopy is an X-ray technique that uses a continuous X-ray beam to scan through the body to create a real-time image in which motion within a given body part can be viewed such as the digestive system, urinary tract,⁸ respiratory tract, bone and muscle,³ and reproductive systems.⁶ Angiography uses special dyes to view blood vessels (arteries, veins, heart) and is based on similar techniques.

CT⁴—often referred to as CAT scans for computed axial tomography—was invented in 1972 by Hounsfield and Cormack²⁰ and was adopted widely soon thereafter. CT acquires slices or cross-sectional images through the body to provide a much more detailed view of the anatomy than can be captured by plain film. As the name implies, it is a digital-based X-ray-generating system that acquires digital signals, then reconstructs them to create an image of the body part scanned. Although early images were printed on film, just like other X-ray images, today both types can be viewed on computer monitors. Since its invention, CT has undergone numerous advancements to increase image resolution, decrease scanning times, and reduce radiation dose. Current systems can acquire up to 320 slices in a typical scan that can be viewed sequentially as a series of slices (or slabs of slices) or reconstructed to view volumetric images in 3D. These images are inherently gray scale, but pseudocoloring is often used to enhance or highlight clinically relevant features. Often, CT examinations use contrast materials (oral, rectal, injection) or dyes such as iodine to enhance viewing of specific organs, tissues, or blood vessels (as in vascular imaging).⁵

Magnetic resonance imaging (MRI)¹⁰ uses powerful magnets to align the nuclei of atoms inside the body and a variable magnetic field, causing the atoms to resonate. It is also referred to as nuclear magnetic resonance. These nuclei produce rotating magnetic fields that are detected by a digital scanner. The generated signals are then reconstructed to create images (again initially viewed on film, but now on a computer monitor). Much like CT, MRI creates slices of images through the body that reveal physiologic information in substantial detail. MRI is especially useful for brain and spinal cord imaging. Important variants include diffusion tensor imaging, which measures the movement of water molecules as they move through the body (useful for detecting stroke or tumors that restrict water diffusion). Functional MRI measures changes in blood in different parts of the brain. MRI examinations often use contrast (gadolinium) to enhance visibility of tissues or disease processes, especially those related to neurology.⁷ Although MRI is the most expensive radiological examination, it is increasingly used in a wide variety of applications.

Ultrasound (US) is another common type of nonionizing imaging. Like MRI, it relies on sound waves and echoes to create images. It is a relatively low-cost type of imaging often used in cardiac and fetal imaging,⁶ as well as other areas such as musculoskeletal imaging. In US imaging, a transducer emits high-frequency sound pulses into the body. These pulses travel until they hit a boundary between tissues such as bone or fluid. Once a boundary is encountered, some of the waves bounce back or are reflected back to the probe. The signals are then analyzed to calculate the distance the signals traveled and create images that reflect the distances and intensities of these echoes on a computer screen. US is rather unique, in that it captures moving images in real-time, making it possible, for example, to view a beating heart or a moving fetus. Blood flow can also be imaged with US (Doppler US) and is thus commonly used in vascular imaging studies. Like other images, US images are gray scale, but pseudocoloring is often used in vascular and cardiovascular⁹ studies to highlight and differentiate arterial from venous blood flow.

Nuclear medicine (NM) is unlike all other imaging modalities. It uses radioactive tracers that are injected into the bloodstream, inhaled, or swallowed. These tracers emit gamma rays that are then detected by digital cameras. Single-photon emission computed tomography (SPECT) is the most common type, although positron emission tomography (PET) is being used increasingly in larger medical centers. Both are primarily used in cardiac imaging,⁹ but increasingly are used in brain and oncologic imaging. PET is being used with CT and MRI (either sequentially with two separate devices or simultaneously with combined PET-CT or PET-MRI devices) to create fused images providing both structural and functional information.²¹

Interventional radiology (IR) uses many of these imaging techniques for minimally invasive surgeries and diagnostic or therapeutic procedures (e.g., cardiac stent placement, laser ablation of fibroids). IR procedures are increasingly being tested and introduced into teleradiology,⁵ paralleling the advances in minimally invasive procedures being used in telesurgery, although most applications to date focus on telementoring and tele-education.

Digital Imaging and Communications in Medicine (DICOM) was developed in 1993 through joint efforts by the American College of Radiology (ACR) and the National Electrical Manufacturers Association (NEMA). On a broad level, DICOM applies to a number of key aspects of the digital radiology enterprise. DICOM is an internationally accepted standard for medical images and metadata, with respect to handling, storing, printing, and transmitting images and other medical record information. It has standards for file format and network communications.

There are two DICOM standards with respect to displays, the Grayscale Presentation State Standard (GSPS) and the Grayscale Standard Display Function (GSDF). The GSDS is a data object associated with an image that specifies how the image should be presented on any compliant image viewer, including such features as customized look-up tables, text overlay, and manipulation functions such as zoom/pan. The GSDF was developed to address the problem of having multiple displays from several vendors, each with different luminance ranges, white points, and minimum and maximum luminance settings. In other words, standardization was needed because the same image would look very different depending on the display it was viewed on. The GSDF is a method to maximize the perceptibility of information and promote the consistency of image presentation across different displays. It is a calibration method based on the concept of perceptual linearity across grayscale values so that changes in image pixel values across a grayscale range are perceived as having similar contrast.²²

Computer-aided detection (CAD) and diagnosis (CADx) and image processing/analysis tools are becoming commonplace in many radiology applications, including breast, colon, and lung cancer screening.²³ Although initially developed to detect and classify abnormalities that radiologists tend to miss (or overcall), many of these techniques are integral to teleradiology as well. For example, CAD tools can be used by general radiologists in rural settings to improve their performance when access to subspecialty radiologists (even by tele) is not available. Image analysis and processing tools are critical for optimal display of digital images on electronic displays and for improving transmission, storage, and retrieval speeds, all of which are essential to teleradiology. A thorough review of these tools and their role in teleradiology is beyond the scope of this article, but we have included some.

Picture archiving and communication systems (PACSs) are at the heart of any radiology enterprise.^24,25 A typical PACS comprises the various digital acquisition imaging systems in a department (e.g., CR/DR, MRI, CT), a secure network for transmitting images from those devices to radiology workstations (onsite or offsite as with teleradiology), archives (onsite, offsite, or cloud-based) to store and retrieve images and related data, and viewing terminals (e.g., workstations, mobile devices). As noted, DICOM is an integral part of the PACS enterprise, facilitating interoperability and the acquisition, transmission, and archiving of images possible. The radiology information system or RIS is a component of the imaging enterprise that manages imaging workflow, images, data, billing information, and general record keeping. It is used in conjunction with the PACSs. Radiology reports are generally saved on the RIS or, in some cases, on the hospital information system (HIS).

Radiology maintained film-based practice even after DR was developed. Initially, images were printed onto film because the earlier electronic display monitors were not adequate for diagnostic viewing. Subsequently, significant developments in medical-grade display technology were started in the late 1980s and continue today. The early displays were cathode ray tube,^26–28 but today liquid crystal displays (LCDs) and variants such as organic light-emitting diodes are the norm in most radiology reading rooms.^29,30 Some of the key display parameters that have guided the development of these displays are directly related to the perceptual requirements of radiologists, the digital nature of the images, the complex nature of anatomic structures, and lesions in the images.³¹

Radiology Costs

Estimates of expenditures on imaging derive from various sources, including (but not limited to) the Health Care Cost Institute, National Expenditure Panel Survey, American College of Radiology, The Centers for Medicare and Medicaid Services, and the Harvey Neiman Health Policy Institute. However, none of these sources provided separate estimates of total national expenditures on either imaging or teleradiology as separate from radiology.

The cost of radiology consists of two components: (1) technical fees associated with the acquisition of images (i.e., costs to operate the devices and pay the radiology technologists) and (2) professional fees for reading and interpreting the images by radiologists. Charges vary as a function of modality (e.g., MRI, CT, plain film), whether or not contrast is used, body part/organ (e.g., head, abdomen, breast), and whether it is interventional or not (for an excellent resource on percentage of hospitals with access to a given modality (e.g., MRI, PET) and Medicare Part B total procedure spending by modality, see www.neimanhpi.org/almanac/series-select).

Policy changes in 2006 and 2007 (including the Multiple Procedure Payment Reduction [MPPR] and Deficit Reduction Act [DRA], respectively) were aimed at reducing both utilization of and payments for imaging, making it close to the bottom in terms of all service categories contributing to growth in Medicare spending.³² For example, in 2003, medical visits involving imaging accounted for 12% of Medicare expenditures. Currently, they account for about 10% (www.neimanhpi.org/infographics/trends-in-imaging). Average Medicare spending on imaging constitutes only about 14% of overall program spending. A recent analysis of Medicare Part B files from 2004 through 2012 revealed that the average per beneficiary spending on imaging has been declining over these years, averaging $298.63 from 2006 to 2012 down from its peak of $405.41 in 2006. However, these figures varied from state to state, with New York and Florida having the highest annual per beneficiary spending and Ohio and Vermont having the lowest.³³

It should be noted that payments for radiological service do not only go to radiologists. A number of clinical specialties also conduct and bill for imaging studies (e.g., cardiology, ob/gyn). In a recent analysis of data from 2011, average Medicare Physician Fee Schedule (MPFS) imaging compensation per Medicare beneficiary was $207.17 with $95.71 (46.2%) for radiologists and $111.46 (53.8%) for nonradiologists. On average, both technical and professional fees are higher for nonradiologists.³⁴ In many cases such as rendering interpretation services in emergency departments (EDs), radiologists receive no compensation nearly 30% of the time, especially when uninsured patients end up in the emergency room, and imaging is a front-line service in the majority of cases.³⁵

The costs of teleradiology vary according to the service model used and the range of services provided.³⁶ The most common payment system is based on a contractual arrangement with a teleradiology provider whereby a set amount is paid per case. The payment amount is a function of such variables as modality, case type, and whether it is urgent or routine. Typically, these fees are based on Medicare payment schedules. Otherwise, the site acquiring the images bills for technical fees (which they keep) and also bills for professional services through Medicare, Medicaid, or private insurers. These latter fees are passed on for the professional fees in the teleradiology contract. An alternative to the case-based contract is an annual set payment for a given site based on expected volume. If the actual workload exceeds the specified number of studies, additional charges are made at a set rate on a per case basis. Some contracts have bonus and/or penalty provisions based on quality metrics such as turnaround times. It is rare that a teleradiology service provider will bill the patient directly.

Total Part B Medicare image spending peaked at $13.8 B in 2006, reaching $9.51 G in 2014, somewhat below 2003. Imaging utilization among Medicare beneficiaries has declined by 5.1% since 2009, and advanced medical imaging has declined by 6.6%. Indeed, the Medicare Payment Commission 2012 report to Congress pointed out that imaging represents the slowest growing category in the fee-for-service Medicare program (See: Medical Imaging and Technology Alliance. www.medicalimaging.org/2012/09/20/new-mita--confirms-decline-in-medical-imaging-utilization-and-spending-within-medicare/). This decline has been attributed to several factors, including (1) recurring cuts in imaging reimbursement (some services by 60%, including bone density, arm and leg artery X-rays, and MRI of the brain); (2) reductions in inappropriate imaging (e.g., unnecessary scans for low-risk prostate cancer patients from 45% to 3%); (3) decline in duplicate imaging by direct access to electronic records; and (4) the implementation of the 2007 Deficit Reduction Act.

Teleradiology

Historically, the main driver for teleradiology was the need for after-hours coverage for urgent and emergent radiologic studies.^19,37 This remains true today. However, some organizations use teleradiology for routine interpretations as well. There is considerable debate in the radiology community regarding the merit of teleradiology, its regulation, and commoditization.^36,38–47 Nonetheless, it is clear that it has a significant role under the proper conditions, especially in situations where local radiology is either absent or inadequate and the quality of the remote provider is assured according to explicit criteria for qualifications and certification.³⁶ Some have suggested that patient satisfaction may be an incentive in some practices as a means for expediting service.⁴⁸ Overall, it was estimated that 40% of radiology practices in the United States (U.S.) performed outside readings in 2007, which accounted for 11% of their total workloads and 4% of the total workload of all radiologists.⁴⁹

The extent of integration of teleradiology into general radiology practice is exemplified by the history of the ACR teleradiology guidelines. The first ACR Standard for Teleradiology was issued in 1994 and it was subsequently revised in 1996, 1998, and 2002. The ACR had a sunset in 2007 since it was expected that the digital practice of general radiology would no longer be differentiated from teleradiology. The ACR and the European Society of Radiology recently released white papers on teleradiology practice, identifying the pros and cons as well as commenting on best practices.^50–52 The ACR stipulated that teleradiology equipment must receive approval from the Food and Drug Administration (FDA) and that image data integrity must be maintained at all system levels and times for both U.S. and international teleradiology.^53,54 With respect to licensing, the ACR has affirmed its position regarding state control at both ends, namely physicians who interpret images originating from another state must be licensed and credentialed at the site of origin of the images and in the state they are doing the interpretation.^53,54

Due to the high cost of image display equipment and the recurring costs of transmission media capable of speeds for remote consultations, the Federal High-Performance Computing and Communications (HPCC) program was established at the National Library of Medicine in 1994. The HPCC supports the development of high-speed network to enable the transmission of medical images to improve healthcare in the U.S.

The Review Process

We followed the same procedures and rules in this review that we employed in the previous articles in this series. As before, the two critical criteria in the choice of research articles were robust research design and sample size of 150 or more. Our initial literature search, using the term teleradiology from 2005 to 2015, yielded 2,271 articles. Of these, only 91 met the selection criteria for this review: (1) 52 focused on feasibility/acceptance; (2) 29 on intermediate outcomes; (3) 5 on outcomes; and (4) 5 on cost. In addition, as before, we organized the evidence into these four sets. The analysis of empirical findings was limited to the last three sets.

Feasibility and Acceptance

The history of teleradiology, and the associated evidence insofar as feasibility/acceptance is concerned, can be viewed as consisting of two distinct eras separated by several decades: (1) the predigital era of teleradiology during the 1940s and 1950s and (2) the modern digital era starting in the early 1990s. Accordingly, these are referred to here as old evidence and new evidence.

It is important to acknowledge here that the history of teleradiology is in many respects indistinguishable from that of digital radiology, PACSs, and the display technologies used to view these images in general. A formal review of this history is beyond the scope of this article (and excellent reviews already exist), but we highlight below some of the key early developments. It is also important to note that as in many fields, technologies and solutions tend to be developed, tried, and used, but then as the field progresses, they tend to be revived and revised to accommodate advances in technology and changes in practice. Thus, for example, we discuss below recent studies using modern smartphones and cameras to capture photographs of radiographic images for teleradiology use, but do not discuss some of the seminal studies conducted before 2005 that initially validated the use of digital cameras for this same purpose.

Old Evidence

The first documentation of the feasibility of teleradiology appeared in the literature in 1948 and it described a telephonic transmission of radiographs, using the earliest version of the facsimile machine.⁵⁵ Radiologist Joseph Gershon-Cohen and inventor Austin Cooley spent 2 years testing a system invented by Cooley to connect Chester County Hospital to Philadelphia, PA, 28 miles away through wire and radio circuits. Primitive by modern standards, the equipment consisted of a glass drum with a clamp on top to attach the film while the drum rotated at a uniform speed of 180 rpm. A beam of light illuminated tiny elemental areas of the film and picked up by a photo cell inside the cylinder and connected with a preamplifier to produce the full picture. The image was passed through an output amplifier before connecting it to a telephone line or radio transmitter. The two authors eloquently described the essence of their invention: “Consultation between the roentgenologist and surgeon, twenty-eight miles apart, took place over the same telephone circuit, with no more delay than a similar consultation would entail with the surgeon and roentgenologist present together in the hospital.” They were also cognizant of the primitive nature of their system and presciently predicted that engineering improvements are now being pursued to reduce the cost sufficiently to make this procedure economically practicable.

This landmark development in teleradiology was followed by a similar one in Montreal, CA, about one decade later. Radiologist Albert Jutras demonstrated the feasibility of transmitting radiographic images between Hotel Dieu and Jean-Talon hospitals (about 5 miles apart) through coaxial cable in 1957.⁵⁶ In 1959, Jutras pointed out that teleguided roentgen diagnosis has become a practical reality and may develop into a routine procedure. The system he tested was a point-to-point connection. However, like his predecessors, he saw the future in terms of building networks and pointed out that complex circuitry would be required to operate the networks.

A decade later, Kenneth Bird and his colleagues at Massachusetts General Hospital in Boston demonstrated and evaluated a comprehensive system for remote diagnosis and referred to it as telediagnosis.²⁶ Although considerable growth and phenomenal technological advances have occurred between these early stages and the present time, these technological advances are beyond the purview of this article.

This review of the evidence regarding feasibility/acceptance of teleradiology is limited to the latest decade, namely 2005–2015, despite the fact that considerable work was done in this field starting in the early 1990s. The basic rationale for limiting our scope to this recent era has to do with the relevance of the information to current practices and systems taking into account recent advances in technology as well as the rich myriad of clinical applications in teleradiology as well as the growth of radiology itself.

New Evidence

Our literature search for evidence regarding feasibility/acceptance of teleradiology yielded a total of 52 articles from 25 countries between 2005 and 2015 inclusive. Of these, 35% were from the U.S. It may be appreciated that today's technology bears little or no resemblance to what was available during the formative stage in the 1950s and early 1960s. Today's technology not only enables rapid and reliable capture, transmission, storage, and retrieval of large data files within secure networks, but it has also opened new vistas for building networks and collaboration between radiologists, surgeons, and other professionals. The following review is organized by year, starting in 2005.

Six studies investigated the feasibility/acceptance of teleradiology in the year 2005, three from the U.S. and one each from Switzerland, Ireland, and Germany. The first U.S. study was based on a 1999 national survey (n = 970 practices and a response rate of 77%) by the ACR.⁵⁷ The purpose of the survey was to establish a general portrait of radiology practices in the U.S. In 1999, 71% of multiradiologist (or group) practices had invested in teleradiology systems and used them for only 5% of their studies. This compared with only 30% of solo practices having teleradiology, but using them for 14% of their studies. The vast majority or 92% of multiradiologist practices used teleradiology for on-call interpretation, 95% to interpret CT, 84% for sonography, 47% for NM, and 43% for conventional radiographs. The true significance of the findings from this survey was pointed out by the authors, namely that teleradiology has already become a fixture in most practices by 1999. Its widespread presence positioned teleradiology to become a key element of radiology practice nationwide.

The second report was concerned with purported problems in the general radiology reporting system, which is not designed to be optimally safe, timely, and patient centered. This was not an empirical study, but it is included here because it proposed new channels of communication between radiologists and patients, which would likely increase satisfaction and patient-centered care by treating patients as equal partners with referring physicians.⁵⁸ This issue is now mostly resolved in health systems that established patient portals where patients can have access to their medical records, including diagnostic studies (virtually through the portal rather than requiring a CD to be burned during an in-person visit), as well as make appointments, order prescription refills, and communicate with their clinicians.

The third U.S. study examined the feasibility of wireless technology for enabling remote evaluation of renal colic (kidney stones) and renal trauma patients using CT.⁵⁹ The technology consisted of a cellular phone with a wireless modem link to a personal digital assistant (PDA). Diagnostic interpretations of images from 11 patients were initially reviewed by a chief resident in urology and subsequently by a staff urologist at an academic medical center. The sample was too small to draw any definitive conclusions, and the technology had its limits, such as the inability to detect intra-abdominal sources of pain such as appendicitis, cholecystitis, ruptured aortic aneurysm, pancreatitis, or pelvic processes. Hence, the authors suggested that telemedicine should be viewed as an additional source of information, and it is still dependent on a well-trained physician's ability to evaluate the patient, order appropriate tests, and consult a specialist as needed.

An Internet-based survey (n = 102, response rate = 22.7%) gathered information regarding use patterns, technical characteristics, and anticipated future of teleradiology in Switzerland.⁶⁰ All members of the Swiss Society of Radiology (n = 450) were contacted by e-mail and asked to complete the questionnaire online. Of those who responded, 41.2% were current users of teleradiology, 35.3% were planning to use it in the near future, and 14.5% had no plans to use it. Use was substantially higher in rural areas compared with urban areas (55% vs. 37%). Emergency service was the most common reason for using teleradiology, followed by expert consultation. CT was considered the most relevant type of imaging in teleradiology. The typical format for transmission was DICOM (66.7%), followed by bitmap/Joint Photographic Experts Group (JPEG) (38.1%). Different views were expressed regarding the future of teleradiology. Some were concerned about regulations and the legal aspects of the practice, followed by security issues. Current users preferred a standard interface, but nonusers seemed to prefer a nationwide medical net.

The feasibility of multimedia messaging for the referral of musculoskeletal limb injuries was investigated in a survey (n = 46) of emergency physicians and trauma surgeons in Northern Ireland.⁶¹ The respondents were asked to evaluate the quality of multimedia consults. Image quality was deemed acceptable in all, but one referral. Physicians reported that the multimedia imaging resulted in improved patient management in 35 of the 46 referrals, and 8 of 46 resulted in changing patient management. The authors concluded that teleradiology has potential to facilitate the rapid cost-effective management of musculoskeletal limb injuries.

A descriptive study by the German Hodgkin Study Group (GHSG) (500 participating centers) documented their experience with a central prospective radiation oncological review aimed at improving treatment quality.⁶² This involved a centralized review of all diagnostic imaging by expert radiation oncologists aimed at controlling the disease extension and to define the involved field treatment volume. This review process was used as a basis for determining the optimal extent of irradiation. The results of this group's experience were quite positive in terms of contributing to high [radiation therapy] RT quality for study patients.

In 2006, five studies/reports met the inclusion criteria for the feasibility/acceptance of teleradiology, two from the U.S. and one each from Canada, Greece, and Spain. All five were concerned with networks and the use of the Web for professional collaboration and/or clinical management.

The Canadian article focused on workflow within specially designed metropolitan networks, referred to as workflow-engaged networks (WENs).⁶³ The unique nature of WENs is the ability to interconnect several hospitals within an urban region to facilitate the operations of a comprehensive diagnostic imaging service. They are modeled on workflow priorities. However, when poorly designed, other network designs may reduce efficiency and cause delays when providers try to retrieve medical cases. Overall, WENs improved network performance by guaranteeing that critical image transfers experience minimal delay, with potential cost savings for leased line services measured in millions of dollars per year.

Two relevant U.S. reports were published in 2006. The first was a description of a work in progress. It is included here because it demonstrated the benefits of establishing a database system within the World Wide Web, which allows rapid and flexible communication of time-sensitive report information and interpretation for more expeditious clinical decision-making in intensive care settings.⁶⁴ The authors pointed out that such Web-based applications can extend the reach and efficiency of traditionally structured medical laboratories.

The second U.S. report described an intranet-based computerized physician order entry (CPOE) system in radiology for online ordering and scheduling at an academic medical center.⁶⁵ Although it did not deal with images per se, this system featured online scheduling, patient reminders, preparation instructions, and driving directions. Comparative scores for imaging examinations were displayed, and physician performance was tracked. Senior clinicians counseled colleagues with low performance scores. The use of the CPOE increased substantially during 1 year (2004), accounting for 75% of all outpatient studies (n = 72,000). After installation, the use of the system increased steadily from 2001 to 2004. At the end of 3 years, it accounted for 75% of all outpatient studies. The highest numbers of low utility examinations were CT and MRI of the spine (which declined from 6% to 2%, especially among primary care physicians). The authors concluded that CPOE can be widely accepted by clinicians and can have an impact on ordering practices.

The feasibility of clinical collaboration over the Web during interventional radiology procedures (IRPs) was investigated in Greece.⁶⁶ A hybrid Web-based IRP system was evaluated to ascertain its feasibility and effectiveness in terms of allowing subspecialists to support real-time mentoring, image manipulation, and education. Ten interventional radiologists, two vascular surgeons, and two medical physicists participated in 33 fully collaborative cases. In addition to mentoring clinicians during procedures, collaboration included open seminars and learner modules. The authors reported that the collaboration proved to be effective in enhancing outcomes especially in complex cases.

A descriptive analysis of a 6-year experience with radiological clinical telesessions was conducted in Spain.⁶⁷ The sessions consisted of clinical case presentations, discussions of clinical findings, differential diagnoses, and suggestions, ultimately leading to a definitive diagnosis. Initially, three groups of radiologists joined the cooperative working environment and shared their clinical experience over the Internet. Subsequently, two other groups joined the online cooperative. A total of 65 cases were reviewed. Of these, 55 were posted as cases of interest archive, thereby making them publicly available. A majority of the participants indicated a high degree of acceptance and no special difficulty in the use of the tools [was reported].

Three reports on the feasibility of teleradiology were published in 2007, two from the U.S. and one jointly from Greece and Germany. The Greece–Germany report was based on a collaborative project in interventional radiological procedures between the two countries.⁶⁸ A hybrid satellite–terrestrial network with multicasting (multiple video feeds) and wavelet capabilities was tested to determine its feasibility for international collaboration in IR. Eight interventional radiologists and one vascular surgeon participated in several clinical and educational sessions. The project demonstrated the feasibility of advanced collaboration among geographically dispersed international centers using this medium.

From the U.S., the first report described the use of grid computing to support interactive professional collaboration and computer-assisted diagnostic detection (CAD) of lung nodules in thoracic CT.⁶⁹ GridIMAGE was designed to (a) select images from several geographically dispersed DICOM servers, (b) send the images to a specified set of readers and CAD algorithms, and (c) compare interpretations from readers and the CAD. Using the National Cancer Institute (caGrid) infrastructure, the system allowed practitioners to obtain interpretations from one or more human readers or CAD algorithms. It also allowed cooperative imaging groups to perform systematic image interpretations.

The second report was focused on the implementation of an automated wireless network for the transmission of prehospital electrocardiograms (ECGs) as a means to reduce door-to-balloon times in acute ST-segment elevation myocardial infarctions (STEMIs).⁷⁰ STEMI is the most severe type of heart attack and constitutes a true medical emergency. The system consisted of Bluetooth devices, preprogrammed transmitting/receiving stations, dedicated e-mail servers, and smartphones. It connected emergency medical personnel and cardiologists to expedite triaging patients directly to the catheterization laboratory from the field when the patient arrives at the ED. This automated system resulted in the early evaluation and triage of patients with suspected STEMI and it reduced door-to-balloon times by more than 90 min. Although the study focused on ECGs, the implications for teleradiology include the pairing of these data with subsequent angiographic imaging data that can be reviewed by offsite cardiologists.

Three reports were published in 2008 concerning the feasibility of using the Internet for image transmission and retrieval in collaborative multisite systems, including one from the U.S., an Israeli project, and one from France. The U.S. study evaluated the use of the Internet to retrieve and integrate pertinent images into a radiology Web site.⁷¹ Thumbnail images were linked to the full-size image at the original site to permit viewing full images. A review of 20 randomly selected hypertext documents (containing 72 images) revealed that 96% of the displayed images were relevant in terms of demonstrating the feasibility of multisite collaboration for image sharing (although more for research and education than clinical interpretation).

The Israel report focused on a unique problem in developing countries, namely the over 50% idle (non-use) or disrepair rate of sophisticated imaging equipment.⁷² It is estimated that some three-quarters of the world's population have no access to medical imaging. Hence, it was ironic that about one-half of the imaging equipment is not optimally used. This study investigated the feasibility of centralized medical imaging systems based on mobile phone technology as a replacement for conventional stand-alone imaging devices. The system consisted of (a) data acquisition devices at the remote sites with limited on-site controls and (b) advanced image reconstruction and hardware control at a central site. A limited simulation test of breast cancer detection demonstrated the feasibility of this modality of medical imaging as a solution for the chronic shortage of imaging infrastructure in many parts of the world.

A French cross-sectional observational study (n = 21) investigated the role of retinal camera imaging as a substitute for in-person ophthalmologist examination of suspected abusive head injury among children.⁷³ Children were examined by standard ophthalmoscopy and by photography using the RetCam-120 Digital Retinal Camera. The images were stored and subsequently read remotely by an ophthalmologist located in a different place. The children also had radiographic skeletal series that were available electronically to identify bone fractures and CT or MRI scans of the head to ascertain intracranial hemorrhages. In this sample, the vast majority of the children (85.7%) had cerebral bleeding, and 66.7% had retinal hemorrhaging. The digital camera enabled the detection of all as well as one false-positive case. The sensitivity and specificity of the camera were 100% and 85.7%, respectively.

In 2009, six reports met the inclusion criteria for feasibility studies, two from the U.S. and one each from Sweden, Croatia, Belgium, and Germany. Each study focused on a unique clinical area. Collectively, they represent a rich mixture of telemedicine applications incorporating radiographic images. We start with studies from European countries and conclude with those from the U.S.

The Swedish study assessed the use of telemedicine for sharing information in pediatric radiotherapy (RT).⁷⁴ In 2003, survival rates of children with all forms of cancer were about 80%, which were attributed to surgery and chemotherapy, as well as RT. Typically, survivors have to cope with the effects of RT, which may include growth retardation for the rest of their lives. Hence, collaboration among radiation treatment specialists was expected to improve their respective knowledge and competence in treating children to maximize the benefits and minimize the side effects of this mode of therapy. After 1 year, the participant clinicians in the group indicated that the telemedicine conferences (which included remote viewing of radiographic and RT images) had a significant impact on their competence. While there was total agreement regarding these benefits, some clinicians complained about audio quality of the system, which was encountered in some instances. This problem was subsequently rectified by investing in dedicated systems and conference rooms.

The Croatian study reported on the use of telemedicine in neurosurgery.⁷⁵ The system connected CT, MRI, and digital subtraction angiography—a type of fluoroscopy in IR used to visualize blood vessels in a bony or dense soft tissue area—scanners in 29 hospitals with a referral center in Zagreb. Patient information, including demographics, medical history, and images, was archived automatically in a central workstation and made accessible on demand. The files were made available to neurosurgical departments in seven cities in Croatia. This report focused on access issues, notably obviated travel, and carbon footprint. The authors reported during the first 7 years of the project, and 25,366 expert opinions were rendered. A total of 7103 (28%) expert opinions were provided to the distant regional hospitals. Of these, 53% were neurotrauma, 22% cardiovascular disease, 19% brain tumors, 4% lumbar disc disease, and 2% neurosurgical disorders. The most valuable results from teleradiology were the decisions about proper and effective patient treatment. In terms of access, the network saved more than 400,000 km of patient transportation.

From Belgium, an observational study investigated the benefits and limits of an integrated 3D virtual approach to treatment planning for patients with a maxillofacial deformity (referred to as orthognathic surgery or corrective jaw or face surgery to realign the jaws and teeth to improve function and appearance).⁷⁶ The authors described the workflow processes in 3D planning for orthognathic surgery, including image acquisition, image processing to create a 3D image of the patient's head, virtual diagnosis and virtual treatment planning, communication, splint manufacturing, transfer to the operating room, and virtual treatment outcome evaluation. The authors pointed out that substantial laboratory and clinical research has been done worldwide on 3D virtual treatment planning of orthognathic surgery. However, the paradigm shift still has yet to demonstrate improvement in quality of care, workflow efficiency, and cost reduction. Whereas quality improvement is certain, efficiency and cost reduction are yet to be proven, mostly because 3D software packages are still expensive.

A somewhat similar application was reported in a technical note from a German project. It described an open platform for 3D cone beam reconstruction that can be applied in several clinical workflows.⁷⁷ Cone beam refers to an imaging technique using CT X-rays emanating from divergent sources that combine to form the shape of a cone. It can be used in implants, orthodontics, and IR. The authors proposed a fast 3D cone beam reconstruction in an open platform for worldwide comparison in back projection performance using high resolution to assure image quality. The authors commented that an open platform gives fair chance to all participants and should enable robust evaluation of the software and hardware. However, it may be appreciated that this is a technical rather than a clinical issue.

A report from the U.S. described a multilingual search engine (GoldMiner Global) to facilitate international (worldwide) access to an indexed collection of more than 200,000 radiological images.⁷⁸ The program uses Unicode standard to allow accurate representation of characters and ideographs in any language that supports writing from left to right and from right to left directions. The translated medical subject heading (MeSH) terms can be used to search for and retrieve images of interest. Explanatory text, pull-down menu choices, and navigational guides are displayed in the selected language. This is more of a research and education-based tool, but as it incorporates virtual access to images, it falls under teleradiology.

Finally, we include a special report on teledentistry aimed at improving access to care for underserved populations.⁷⁹ Teledentistry uses electronic health records, telecommunications technology, digital imaging, and the Internet to provide teleconsultation with specialists, supervision of collaborative hygienists in remote areas, and education. Similar to other telemedicine applications, it can be delivered synchronously or asynchronously. However, teledentistry is a relatively new area of dentistry. With further advances in technology, it is expected to be more widely used to allow specialists to provide diagnosis and treatment recommendations for patients who would otherwise not have access to specialists in person. Similarly, through collaborative hygienists in remote areas, patients [would] have improved access to preventive dental care.

In 2010, three reports met the inclusion criteria, one each from Poland, Scotland, and India. The common theme revolves around the benefits of networking.

The Polish study assessed image quality, software stability, constant availability, data transmission speed, and quality of real-time synchronized viewing of the images during the tele-DICOM teleconsultation.⁸⁰ Observational data were gathered from offline consultations of 918 patients who were referred for coronary artery bypass grafting (CABG). The tele-DICOM software was tested against measurements from echocardiography. The tele-DICOM consultations resulted in a CABG decision in 87.8% of the cases. The remaining 12% were recommended for medical therapy or percutaneous angioplasty. Measurements performed in the ECHO-tele-DICOM module were accurate compared with those performed on a standard echo machine (r = 0.98). The study demonstrated that the tele-DICOM system is suited for professional use in the field of cardiovascular disease.

The Scottish study was a prospective evaluation (n = 143) of quality of service in referrals to neurosurgery using PACSs and remote viewing.⁸¹ The authors explained that low caseload could not support subregional neurosurgery units in Scotland. Hence, nearly all acute neurosurgery referrals were made by unspecialized providers. Many of these referrals will be for advice on management and not all will require interhospital patient transfer. During the 4-year period of the study, remote image viewing facilities were gradually changed to dedicated image links by remote PACSs. Subsequently, 88% of the referrals were made through the new system. A minority of referrals (12%) required images (on CD-ROMS) to be physically transferred by courier. This added an average delay of 5.8 h to decision-making. The authors expressed concern over the slow adoption of PACSs in neuroscience, which could jeopardize patient safety.

An observational study in India (n = 962) documented the effects of teleradiology on accessibility.⁸² A network link was established between a teleradiology provider in Bangalore, South India, and a hospital in Arunachal Pradesh in the northeast over 3,000 km away. The hospital at the remote site is a referral site for the entire state with a population of over one million, and it had only one CT scanner. This project demonstrated that remote implementation of teleradiology is possible in rural India.

In 2011, a total of five studies investigated the feasibility of specific clinical applications in teleradiology, two from the U.S. and one each from Germany, New Zealand, and South Korea.

The feasibility of telefluoroscopic evaluation of oropharyngeal dysphagia (swallowing difficulty) was evaluated in a prospective observational study of 32 patients in Arkansas.⁸³ This medical condition can occur at any age and can result in respiratory complications, malnutrition, or death. In addition, it can limit social interactions and may lead to emotional distress. Patients with this problem participated in two separate fluoroscopic swallowing evaluations, one traditional and the other through a telemedicine system. The two sets were evaluated in terms of (1) severity of swallowing difficulty; (2) laryngeal penetration and aspiration; and (3) treatment recommendations. There was an overall agreement between the two modalities in severity ratings (κ = 0.64), penetration aspiration rate (difference of 1.1 points), and in treatment recommendations, ranging from 69% to 100%. Hence, the study demonstrated the feasibility and clinical utility of telefluoroscopy in evaluating swallowing difficulty.

Another U.S. study during 2011 investigated the utility of PACSs in terms of providing decision support from the perspective of radiology residents.⁸⁴ Decision support systems (DSSs) can provide patient-specific information during image interpretation that can improve diagnostic accuracy, and for residents doing rural rotations, access to DSSs through teleradiology can be useful. However, most DSSs require radiologists to exit PACSs to use them, thereby deterring some of them from using DSSs altogether. Forty-eight radiology residents were assigned randomly to a control group, which had Web access to DSSs, whereas the experimental group had the same access plus a PACS-integrated portal. Halfway through the 10-month study, the two groups were switched. The outcome measure was the number of decision support sessions initiated by the resident radiologist. The experimental group had much higher use scores than the control group by a factor of 3. When integrated access was removed from the original experimental group, their use declined by 52%, and when the control group had integrated access, their use rose by 20%. The authors concluded that integrated access is critical at the time of initial deployment. However, the reasons for that phenomenon are not clear.

From Germany, a retrospective analysis of clinical and imaging data was conducted pre- and postradioimmunotherapy for conventional lymphoma. The aim was to predict therapy response to radioimmunotherapy in a Web-based multicenter evaluation.⁸⁵ A total of 159 lesions (16 patients) were measured using the digital images. Lesion volume, total, and maximum volume were predictors of response. The Web-based multicenter archiving system proved to be technically feasible, and the analysis of the data suggested that image collection was feasible and allowed for a central analysis.

The development and use of an online system to locate radiologists in New Zealand were described in a 2011 publication.⁸⁶ Although not an image-based tool, this in-house system was designed to minimize the time spent looking for a radiologist from geographically separated radiology departments. The system allowed access to radiologists who are logged into the RIS. The information [name, contact information, and specialty] is displayed on a self-refreshing screen on the Intranet. Over a period of 1 month (April 2009), there were 2,798 hits on the locator page. Of these, 1,248 hits were made by radiologists and 1,550 by radiology administrative staff. Average time for reviewing the roster was 30 s, and a full review of the whole department was 195 s. Importantly, the use of this system realized savings of up to 16 h of radiologist reporting time per week.

From South Korea, a study assessed the usefulness of a mobile teleradiology system using the JPEG2000 compression scheme for emergency care imaging.⁸⁷ The feasibility of JPEG2000 compressed CT images transmitted over mobile networks was assessed in terms of image quality and transmission time. The system was able to query patient information and images instantly. Quality was judged as acceptable by the radiologists. As expected, higher compression levels provided quantitatively lower image quality, with ratios of up to 10:1 as clinically reasonable. The authors concluded that wireless transmission of JPEG2000 radiological images of emergency patients through mobile networks to remote specialists can help achieve proper first aid of emergency patients.

The feasibility of a multicenter Web-based system for predicting radioimmunotherapy response in patients with follicular lymphoma was assessed using a retrospective record review (n = 16 patients and 159 lesions).⁸⁵ Every measurable lesion was analyzed in terms of standardized uptake value, volume, and response. Lesion volumes were predictors of response. The important conclusion from this study affirmed the technical feasibility of a Web-based multicenter archiving system for identifying predictive factors in this form of lymphoma. This preliminary analysis suggested that imaging data can predict the likelihood of response to radioimmunotherapy.

A total of six feasibility studies were published in 2012, three from the U.S. and one each from Malawi, Korea, and United Kingdom, Serbia, Germany, and Ireland.

The first U.S. study evaluated the ability to diagnose acute appendicitis from abdominal CT using a mobile DICOM viewer.⁸⁸ The study used 25 abdominal CT studies for patients presenting with abdominal pain. They were interpreted by five radiologists on an iPhone using a DICOM viewer. The radiologists located the appendix and ascertained its diameter, appendicolith (calcified deposit), stranding (tissue thickening) and fluid, abscess, and acute appendicitis. Readings from an iPhone were compared with a PACS workstation. Acute appendicitis was correctly diagnosed in 98% of the cases with no false positives. Eighty-eight percent of appendicoliths were correctly identified as well as all three abscesses. The authors concluded that handheld device review of CT scans has the potential to aid in appropriate and expeditious triage of patients with acute appendicitis, ultimately improving patient outcomes.

An interactive, computer-based pediatric atlas for chest radiograph interpretation was developed to help in understanding normal variants in interpreting pediatric chest radiographs.⁸⁹ The atlas consisted of a review of 73 normal pediatric chest radiographs, 16 abnormal pediatric radiographs, and 4 normal adult radiographs. The images were originally DICOM formatted, then converted to JPEG format. The digital interactive format made it easy to manipulate the atlas and to compare cases, which is of benefit especially to general radiologists in remote settings with subspecialty training. The benefits of the atlas were demonstrated, and the authors concluded that improved interpretation of pediatric chest radiographs in the acute setting may be facilitated by a comprehensive, computer-based, pediatric chest atlas.

The third U.S. study in 2012 investigated the feasibility of a smartphone teleradiology application in a telestroke network.⁹⁰ It was based on the premise that a mobile application on smartphones affords neurologists access to radiological images of patients with stroke from remote sites in the context of a telemedicine evaluation. The objective was to assess concordance between a vascular neurosurgeon in a hub center and independent adjudicators. Data were gathered on a sample of 53 patients presenting with stroke in remote sites who participated in a telemedicine consultation. Their CT scans were evaluated by a hub vascular neurologist, a spoke (remote site) radiologist, and blinded telestroke adjudicators. CT head interpretations by vascular neurologists were in agreement (measured by κ values) with spoke radiologists and independent adjudicators at 1.0 for intracranial hemorrhage, 1.0 for neoplasms, contraindications to thrombolysis, 0.92 and 0.85, early ischemic changes 0.62, and hyper dense artery sign 0.40. Thus, the authors concluded that CT head interpretations of telestroke network patients by vascular neurologists using ResolutionMD [their software package] on smartphones were in excellent agreement with interpretations by spoke radiologists using a PACS and those of independent stroke adjudicators using a desktop viewer.

The feasibility of using teleradiology in tuberculosis (TB) screening and case management was evaluated in Malawi in southeast Africa.⁹¹ The authors reported that Malawi has one of the highest rates of HIV infections (10.6%) in the world and a higher rate of TB (14.5%) in a southern district (Thyolo) with about 600,000 inhabitants, mostly poor migrant farmers. A total of 159 images from 158 patients were reviewed by teleradiology. Nearly one in four cases (n = 36) resulted in changed management; two were undiagnosed TB; and 16 corrected a misdiagnosis of TB (i.e., false positives). The corrected cases of misdiagnosis averted inappropriate treatment. The authors concluded that teleradiology can improve tuberculosis diagnosis and case management. This would be particularly important in a resource-poor setting.

The use of mobile technology for remote viewing of CT images was assessed in an observational study (n = 100) in Korea.⁹² The project used ultramobile PC (UMPC) for Web-based viewing of abdominal CT scans for remote interpretation of appendicitis compared with using an LCD-based DICOM viewer. Receiver operating characteristic (ROC) area under the curve values for the UPMC (Az = 0.959) and the LCD monitor (Az = 0.976) were both high and there were no significant differences between the two display systems for interpreting abdominal CTs. Web-based mobile teleradiology is feasible for reading abdominal CTs for diagnosing appendicitis and may be valuable in emergency teleconsultation.

Finally, we diverge from our normal course, which is focused on feasibility/acceptance of teleradiology applications by including here a UK report on the quality of information available on the Internet search engines regarding IR, vascular surgery, and cardiology.⁹³ The article was accepted in 2012 and published in 2013. The leading search engines were compared using the Linguistic Assistant for Domain Analysis (LIDA)—an online tool for assessing health-related Web sites for usability testing, specifically to ascertain whether the information on the Web site is accessible, usable, and reliable. The overall results suggested that the three Web sites (or search engines) were generally well designed and easy to use. However, the majority lacked currency and reliability. There was also a disparity in knowledge of IR, which was attributed to low traffic on the Web site.

In 2013, seven studies met the selection criteria for feasibility analysis, emanating from six countries, two from Korea and one each from China, Turkey, Greece, Angola, and Hungary. These studies investigated the feasibility of teleradiology for remote diagnosis in several clinical areas, appendicitis, brain hemorrhage, and shoulder injury. A common thread among them is the use of mobile technology.

The first of the two Korean studies investigated the feasibility of teleconsultations in patients with inconclusive diagnosis of appendicitis, using CT scans and smartphones.⁹⁴ The article was accepted in 2013 and published in 2014. The teleconsultations were required when the on-call radiologists made inconclusive diagnoses. The sample consisted of 68 patients, 29 with confirmed appendicitis, and 25 were inconclusive. The smartphone readings were compared with preliminary reports by on-call radiologists and with the original final reports by in-house abdominal radiologists. Overall, the study results indicated that offsite smartphone readings did not differ significantly from that of the in-house preliminary report. However, the smartphone readings provided higher diagnostic confidence than the preliminary reports. This suggests that smartphone readings were made with more confidence, especially when the initial report of the on-call radiologist indicated an inconclusive diagnosis of appendicitis.

The second Korean study assessed the feasibility of iPad 2 for evaluating subtle brain hemorrhaging using CT under conventional lighting conditions, which were common in remote CT readings.⁹⁵ A sample of 100 brain CT scans for head trauma or headache was selected: 50 had subtle signs of intracranial hemorrhaging, and 50 had no significant abnormality. Five emergency physicians reviewed the CT scans in two modalities, IPad and LCD monitor. High sensitivities and specificities were observed when the two modalities were compared (ROCs Az values were 0.935 and 0.900 for IPad and LCD, respectively). Clinicians were able to diagnose subtle brain hemorrhage under suboptimal viewing conditions.

A Chinese study evaluated the feasibility of a robot-assisted vascular interventional surgery to enable surgeons to teleoperate a catheter in a safe mode (i.e., with reduced X-ray exposure).⁹⁶ The remote-controlled vascular intervention robot (RVIR) was designed on the basis of a master–slave structure and addressed C-arm calibration, distortion connection, catheter localization, and 3D vascular reconstruction. The experiment demonstrated that RVIR was feasible, and its accuracy would satisfy the surgeon's requirements.

A retrospective evaluation (n = 100) of the feasibility and accuracy of teleconsultations of coronary angiograms using smartphones was conducted in Turkey.⁹⁷ Images of single-vessel disease were reevaluated by consultants outside the hospital in two display modes: iPhone 4 screen and workstation monitor, reporting on localization and severity of angiographic lesions. There was high agreement between the core laboratory monitor readings and the readings on the iPhone (κ = 0.80). Accordingly, the authors concluded that smartphones may serve as a supplementary teleconsultation tool in both elective and emergency situations.

The feasibility and reliability of integrated teleradiological and telepathological evaluation of liver grafts were conducted in Greece using 15 MRI image sets.⁹⁸ The integrated analysis of both teleradiology and telepathology resulted in 100% specificity, 96.6% sensitivity, and 97.5% accuracy. The message from this simulation points to the feasibility and reliability of evaluating liver graft transplant when the two diagnostic methods are used.

A large observational study (n = 20,564 X-ray images) was conducted in Angola (seventh largest country in Africa) to ascertain the feasibility and quality of digital X-rays in general as well as teleradiology.⁹⁹ Teleradiology was used in 7.6% of all pediatric cases and provided an important contribution to case management. Overall, the implementation of a digital X-ray is feasible in low-resource settings with significant improvement in quality X-ray images.

The final feasibility study in 2013 comes from Hungary.¹⁰⁰ This short communication described the results of screening a total of 102 applications (in English) for smartphones and tablets in terms of diagnostic reading, decision support, medical books, integrative encyclopedias, and journal reading programs. Based on this broad analysis, the authors concluded that smartphones and tables offer new opportunities for diagnostic imaging practitioners with excellent display for diagnostic reading, reference, learning, consultations, and for communication with patients.

In 2014, three studies met the selection criteria, one each from Switzerland, Germany, and France. These studies reported on similar themes as previous ones, namely multidisciplinary collaboration, networking, and mobile technology.

From Switzerland, a study focused on multidisciplinary collaboration between radiology, surgery, RT, and pathology in head and neck tumor surgery.¹⁰¹ These fields were integrated by a common server, which provided remote access to the imaging data. Margins of resection and exact locations of biopsies were mapped intraoperatively. Pathologists provided numerical coordinates of the biopsies and traced each specimen to an anatomical field. RT was mapped according to this information. The model enabled the RT planning according to specific coordinates to minimize irradiation of adjacent structures. The navigation-assisted network described grants timely multidisciplinary feedback between all fields involved, attains meticulous pathological definition, and permits optimized coordinate-directed radiotherapy.

A single-center experience of off-hour IR service was evaluated to assess feasibility, frequency of use, and associated labor costs in a German study.¹⁰² The analysis was based on retrospective screening of the IR database for procedures performed outside regular working hours and manually selected cases where the interventionist was called from home. Three on-call interventionists performed 92 procedures during off-hours in 1 year. These consisted of angiography and hemorrhage control (39%), acute limb ischemia (27%), percutaneous biliary drainage (11%), nonocclusive ischemia (8%), and other (15%). The labor costs per procedure were €459.9. Based on these data, the authors concluded that a formal interventional [off-hour] is practicable in a university setting and the total costs appear to be moderate.

A nationwide cross-sectional survey (n = 131, response rate = 93.6%) of young radiation oncologists was conducted in France, April to June 2013.¹⁰³ The purpose was to assess how young French radiation oncologists use mobile technology and social media in clinical practice. Ownership of these devices was high, 93% owned a smartphone and 33% tablets. Overall, 83% had a social network account. Most of the residents used smartphones in their work for a variety of tasks. However, the majority did not check the validity of the apps on a consistent basis. The use of social network had a limited impact on their relationship with patients. According to the authors, this study highlights the irruption [or burst] and the risks of new technologies in the clinical practice and raises the question of a possible regulation for their use in the hospital.

Three studies met the selection criteria in 2015, all from the U.S. The first focused on the feasibility of the radiological image sharing network to obviate the need for physical media for patients and providers.¹⁰⁴ The National Institutes of Health (NIH) challenged the Radiological Society of North America (RSNA) to develop a transport method that could supersede the need for physical media (for patients and providers), replace point-to-point private networks among providers, and enable image exchange on an ad hoc basis between arbitrary health networks without long legal delays. This article constitutes a technical response to that challenge, incorporating the architecture for such a system.

A study assessed the use of social media sites among private radiology groups (PRGs) and academic radiology departments (ARDs).¹⁰⁵ The sample included the 50 largest PRGs (defined by the total number of full-time employee radiologists) and the 50 ARDs with the largest National Institutes of Health (NIH) funding in 2012. The use of each social media Web site, including Facebook, Twitter, Pinterest, Instagram, YouTube, and LinkedIn, was assessed during a 2-week period in September 2014. Measures of organizational activity and individual user activity were collected. Seventy-six percent of PRGs maintained more than one social media account compared with only 28% of ARDs who did. In terms of prevalence of social media among PRGs, Facebook was 66%, LinkedIn 56%, Twitter, 42%, YouTube 20%, Pinterest 4%, and Instagram 2%. The corresponding percentages among ARDs were 18%, 0%, 24%, 6%, 0%, and 0%, respectively. These findings indicate that the use of social media in health is emerging in the mainstream, especially among PRGs.

Finally, we include a special report by a global group (from four leading academic medical centers and cancer centers in the U.S. and three countries (Tanzania, Austria, and South Africa) aimed at providing an avenue for collaboration among radiation oncologists who are interested in participating in global health, but do not know how.¹⁰⁶ The report cites the growing worldwide burden of cancer (about 1.16 trillion US$) and global disparities, with more than 60% of 14 million new cases and 70% or 8.2 million deaths per year occurring in low- and middle-income countries. These disparities reflect the poignant underling disparities in radiation oncology. The authors suggest the establishment of a platform for facilitating participation for more effective and sustainable global collaboration. This would form a global health catalyst for cancer care, research, and education, involving institutions from developed and developing countries, while highlighting the potential of ICTs for catalyzing international collaboration in radiation oncology care, research, and education.

Teleradiology Intermediate Outcomes

As discussed earlier in this series of articles on empirical evidence, intermediate health outcomes constitute valid measures of quality, especially when they have a logical and necessary relationship with health outcomes or health effects. Because of this link, their use in quality assessment is appropriate and, in some instances, necessary. Furthermore, intermediate outcomes are (1) readily observable, (2) measurable in the short term, and (3) often of inherent interest as quality indicators. In teleradiology, the typical intermediate outcomes include accuracy and reliability of remote diagnosis compared with viewing images on PACSs or conventional film.

From 2005 through 2015, a total number of 29 studies met the eligibility criteria for our analysis of teleradiology intermediate outcomes. These were conducted in 22 counties, including eight from the U.S. The typical research design involved comparisons between two modalities of viewing images (Table 1).

Table 1.

Summary List of Empirical Evidence in Teleradiology: Intermediate Outcomes

REFERENCE	YEAR	COUNTRY	STUDY DESIGN	SAMPLE SIZE	MODALITY/INVERVENTION	FINDINGS	COMMENTS
Cone et al.107	2005	Ecuador	Comparative analysis: Observational	196	Digital camera CT and US	>90% agreement in CT; 56% in US	Higher agreement in CT than US
Eze et al.108	2005	United Kingdom	Prospective case control	154	Mobile phone/ENT	100% agreement	More confidence in X-ray box
Muller et al.109	2005	Germany	RCT	1,080	Centralized network for oncology review	Disease improvement observed: early stage = 49%; intermediate stage = 67%	Participants benefited from program; high RT quality
Leader et al.110	2006	United States	Descriptive/observational	1,003	Multisite mammography	66% agreement in telemammography	Telemammography reduces recall for additional procedures by 50%
Yang et al.111	2006	United States	Observational	3,744	Grid-enabled CAD	Algorithm accuracy of 86.02%	Atypical ductal hyperplasia had the highest false-positive rates
Javadi et al.112	2006	Thailand	Comparative analysis: observational	192	Digital camera/pneumonia surveillance	Digital camera: sensitivity = 89%; specificity = 75%; digitizer: 90% and 65%	Digital camera did not compromise pneumonia
Tangtrakulwanich et al.113	2006	Thailand	Comparative analysis: observational	150	Digital camera/fracture	Digital camera; sensitivity = 90%, specificity 80.1%	Digital camera cost $965 vs. $3000 for digitizer
Nitrosi et al.115	2007	Italy	Observational	180,000	Digital radiology	Productivity ↑12%; turnaround time ↓60%; imaging procedures ↑7%; Length of stay ↓12%	Waiting time for outpatients ↓90 to 40 days for CT and from 90–180 to 30–60 days for US
Wardlaw et al.116	2007	5 Countries	Comparative analysis: observational	207	Internet/ischemic changes	Sensitivity and specificity are comparable	Neuroradiologists saw more early ischemic diagnoses than did stroke physicians
Platts-Mills et al.117	2008	United States	Comparative analysis: observational	787	Teleradiology vs. in-house/CT	Discrepancies observed in 5.8% of interpretations	Only one discrepancy in 550 paired comparisons resulted in adverse event
Kennedy et al.118	2009	United States	Survey and case control	2,266	CT pulmonary angiography	Improvement in number of studies completed in 20 min = 95% vs. 13%; percentage of writing interpretations improved from 51% to 62%	Significant process improvements in interpreting inpatient imaging
Duka et al.119	2009	Serbia	Comparative analysis: Observational	432	Camera for radiography in impacted molars	Sensitivity = 99%; specificity = 99%	Diagnosis from photographs was equal to real-time assessment
Hurlen et al.121	2010	Norway	Retrospective record review	3,088	Diagnostic imaging	PACS and RIS reduced LOS by 25%	LOS reduced from 5.3 to 3.9 days
Ninos et al.122	2010	Greece	Observational	144	PDA/diagnostic performance	PDA provided rapid, secure, and convenient access	Experts had difficulty diagnosing microcalcification
Lakkis et al.124	2011	Lebanon	RCT	385	SMS/breast cancer screening	Enhanced SMS equivalent	Detailed SMS did not affect mammography intake
Rudat et al.125	2011	Saudi Arabia	Observational	148	Online verification	Setup accuracy the same	Daily online verification recommended
Shaligram et al.127	2012	Multiple countries	Retrospective record review	13,288	CT/appendicitis	Readmissions (1.8% vs. 5.13%)	Improved postoperative complications; lower cost
Hohmann et al.128	2012	Switzerland	Observational	1,028	Outsourced radiology	No disagreement = 79%; inconclusive, 4%; 1.3% omission	Outsource radiology supported
John et al.129	2012	Singapore	Comparative analysis: observational	264	IPad vs. workstation	3.4% major discrepancy; 5.6% minor discrepancy	Variability accounted for by interobserver interpretations
Angileri et al.130	2012	Italy	Retrospective record review	733	Remote second opinion	Correct interpretation: 96.5%	Average duration 38 min vs. 160 for in-person
Fruehwald-Pallamar et al.131	2012	Austria	Comparative analysis: observational	200	Teleradiology network	Automatic readout agreement: 90% and 97%	Only 2.5% and 9.5% rated as poor at two institutions
Chang et al.133	2013	Taiwan	Observational	3,770	Online/osteoporosis	Sensitivity = 75%; specificity = 75%	System had high reliability and validity
Puetz et al.134	2013	Germany	Retrospective observational	536	Stroke teleneurology	Discrepant findings 8%, only 1.7% clinically relevant	Interobserver agreement κ = 0.62
Freeman et al.135	2013	United Kingdom	Observational	253	Internet/shoulder injury	Interobserver reliability κ = 0.81	Labeling accuracy higher in educational sites
Franczak et al.136	2014	United States	Retrospective rater analysis	2,000	EHR/CT	Inter-rater reliability = 0.82	EHR necessary for radiology in the ER
Lee et al.137	2014	United States	Observational	444	Osteoporosis screening	Increased treatment from 4.8% to 35.2%	Improved rate of osteoporosis treatment in the VA
Schwartz et al.138	2014	Botswana	Comparative analysis: observational	150	e-consult/dx accuracy	Correct diagnosis in both modalities in 79%	Image quality expected to improve
Lyon et al.139	2015	United States	Retrospective record review	1,445	Telemedicine/referral	Reduce patient referrals in rural areas	One in four trauma patients used to be referred
Jacobs et al.140	2015	The Netherlands	Retrospective record review	806	Teleradiology/fractures	Improved fracture diagnosis from 9 to 2	Reduced unnecessary trips among patients with fractures

CAD, computer-aided detection; CT, computed tomography; EHR, electronic health records; ER, emergency room; US, ultrasound; VA, veterans administration.

Starting with 2005, three studies investigated various aspects related to intermediate effects of teleradiology, one each from Ecuador, United Kingdom, and Germany.

The first study was conducted in Ecuador by a U.S. Medical Informatics Consortium, and it was aimed at assessing the quality of low-cost digital imaging captured by handheld cameras in a remote location in Ecuador and transmitted through the Internet to an academic medical center in the U.S. for interpretation.¹⁰⁷ This was a comparative study based on 196 diagnostic studies, randomly selected to include a full range of diagnoses ranging from normal findings to obvious lesions, representing typical pathologies in the region. The images included plain film and CT scans. Corroborative diagnoses were obtained from the digital camera images for greater than 90% of the plain film and computed tomography studies. However, agreement on ultrasound studies was only 56%.

From the United Kingdom, a prospective case–control study (n = 154) investigated the accuracy of mobile phones in radiological investigations for diagnosing common ear, nose, and throat (ENT) problems presenting in the ED.¹⁰⁸ A total of 154 cases (CT scans and X-ray images) were assessed, comparing two modalities of viewing: mobile phone versus conventional X-ray film. All physicians made the correct diagnosis for every case examined, but they had slightly more confidence about making a diagnosis when using the X-ray film compared with the mobile phone. Moreover, the cost of mobile phones was much lower than conventional telemedicine technology (using e-mail and Integrated Service Digital Network (ISDN). The authors concluded that regular use of this technology may improve confidence with digital image, especially since many hospitals are now installing workstations to view radiological investigations on computer screens.

The third study in 2005 (RCT, n = 1,080) was conducted by the GHSG and included more than 500 participating centers.¹⁰⁹ A new RT program was established to (1) provide a centralized, prospective, radiation oncologic review to generate individual radiation treatment plans; (2) enable a retrospective analysis of the adequacy of the involved field; (3) conduct a multidisciplinary panel; and (4) integrate a teleradiology network into the trial. The purpose of the study was to evaluate the effectiveness of the RT program within the network in terms of adequacy of treatment fields, applied radiation doses, treatment time, and technical parameters. The review of diagnostic imaging showed corrections of disease involvement in 49% of patients with early stages [of disease] and 67% of patients with intermediate stages [of disease]. The introduction of electronic image transfer optimized and simplified the workflow of the quality assurance programs (QAP). “Rapid online consultation and real-time teleconferences regarding disease involvement, patient management, and communication of the radiotherapy (RT) prescription with connected hospitals proved to be extremely helpful.” The authors observed that radiation oncologists in their participating centers in Germany perform continuous and efficient quality assurance. This retrospective analysis of their experience with this program revealed that participants benefited from the educational program and that patients with poor initial response to treatment received additional RT as a result of the panel's recommendations. The introduction of teleradiotherapy into the GHSG trials improves the dialogue between the central RT reference center and the study participants and contributes to high RT quality for study patients.

A rich mixture of five studies, two from the U.S., two from Thailand, and one from Canada, was published in 2006, dealing with different measures of intermediate outcomes in teleradiology research, including diagnostic concordance, use of clinical decision support and computerized order entry, hospital stay, and detection rates. We start with the two U.S. studies.

The first was a descriptive study (five-part, total n = 1,003) of the effects of a telemammography system on use of service among women needing to return to the clinic for additional imaging procedures as part of a breast cancer screening protocol.¹¹⁰ A high-quality, multisite telemammography system enabled near-real-time remote patient management (from a central site), while the patients were located at the remote clinic. Images were cropped, compressed, and encrypted before their transmission to the central site, where high-resolution workstations displayed the images. Instant messaging (chat window) was provided for intersite communication. The technologists recommended additional procedures at 2.7 times the actual clinical recall rate for the same cases. Radiologists recommended additional procedures at 1.3 times the actual recall rate. The rate of agreement between the study and actual clinical interpretations was 66.1% (κ = 0.315). For every recall avoided in the telemammography system, about one unnecessary imaging procedure was recommended. Nonetheless, the study demonstrated that remote patient management can reduce the number of women who need to return physically to the clinic for additional procedures (for spot compression, magnifications views) by about 50% without performing an unreasonable number of unnecessary imaging procedures. It was concluded that this type of practice can reduce recall rates by about 50%.

The second U.S. study (observational, n = 3,744) was also concerned with the efficiency of breast cancer detection, using a grid-enabled computer-assisted diagnosis (CADx) system.¹¹¹ As background to the study, the authors pointed out the high prevalence rate of breast cancer in the U.S. for women (about one in eight), whereas death rates from breast cancer continue to decline due to early detection and improved treatment. They introduced an online grid-enabled system for scheduling, patient reminders, preparation instructions, and driving directions. Subsequently, they conducted a throughput analysis of breast cancer specimen on the grid. This method takes into account the total number of items processed in the system and their physical path over a specified period of time. The data allowed evaluation of use rates, examination of low utility scores, and changes in scores during the year. Grid technology leverages aggregated bandwidth, computational power, and secondary storage resources available at several sites. It is particularly suitable for research projects that require high levels of computation. The grid-enabled framework enabled a robust throughput analysis of imaged breast cancer specimens. In [these] experiments, atypical ductal hyperplasia (ADH) gave the highest false-positive rates.

Digital cameras were tested in Thailand to ascertain their accuracy in a pneumonia surveillance program.¹¹² This was also a comparative study of digital camera versus film digitizer for viewing chest radiographs in diagnosing pneumonia. The sample consisted of 192 radiographs from patients with pneumonia. Of these, 166 were diagnosed as having pneumonia on hard copy or film. Using hard copy readings as the gold standard, sensitivity and specificity for identifying pneumonia were 89% and 73% (using two cameras in two sets) for the camera and 90% and 65% for the digitizer. The digital camera set cost $965 compared with $3000 for the film digitizer. The study demonstrated that the use of digital cameras did not compromise the detection of pneumonia, whereas their cost was substantially lower than that of the film digitizer.

In addition, from Thailand, another study investigated the validity and reliability of teleconsultation for fracture care using digital camera images.¹¹³ As in the previous one, the investigators compared the reliability (as measured by κ), sensitivity, and specificity of digitized radiographs, digital clinical photographs, and conventional analog radiographs. The sample consisted of 100 patients with nondisplaced or minimally displaced fractures and 50 healthy age-matched adults (total n = 150). Overall reliability (kappa), sensitivity, and specificity of digitized radiographs were 0.57%, 83.2%, and 80.7%, respectively. Hence, the authors concluded here as did the others who investigated the same application for detecting pneumonia that teleconsultation using digital camera images was valid and reliable.

The fifth study in 2006 was conducted in Canada and it investigated detection rates in pediatric diagnostic imaging, comparing PACS and Web-based imaging system on a personal computer and a high-resolution monitor.¹¹⁴ The sample size for this study was only 100. Hence, the findings will be discussed here briefly, but will not be reported in Table 1 that summarizes the evidence. Of 100 chest or abdomen X-rays, 32 were normal, 33 obviously abnormal, and 35 subtle abnormal. Raters viewed these images on PACS and a Web-based imaging system. The two viewing methods produced similar results. Percentages of incorrect interpretations were 23.2% on the Web and 23.6% on PACS. Accordingly, it was suggested that high-resolution Web-based imaging through PC is an adequate alternative to PACS while potentially saving on cost.

In 2007, two studies met the selection criteria, one each from Italy and a consortium of researchers from five countries: Scotland, Germany, France, Italy, and Canada. These studies investigated the effects of teleradiology on several indicators of efficiency and quality.

The Italian study was a quantitative assessment (n = ∼180,000) of the impact of a digital radiology department in a regional hospital on the process of care in terms of quality and efficiency.¹¹⁵ The key indicators were turnaround time, number of procedures, and hospital length of stay (LOS), as well as workflow and productivity. After 1 year of becoming fully digital, the following improvements were observed: productivity increased by 12%; turnaround time reduced by 60%; and number of imaging procedures increased by 7%, whereas LOS for neurology patients decreased by 12%. Annual financial upsides have exceeded $1.9 million/year. The process enhancements had a significant impact on reducing wait time for outpatients from 90 to 40 days for nonurgent CT scans and from 90 to 180 days to 30–60 days for nonurgent ultrasound.

An international group of clinical neurologists from five countries (Scotland, Germany, Italy, France, and Canada) investigated factors that influence the detection of infarct signs, comparing neurologists’ scan readings with those of other specialists, all using the Internet.¹¹⁶ A total of 207 specialists (24 neuroradiologists, 25 general radiologists, 99 neurologists, 16 stroke physicians, 21 geriatricians, and 22 others) reviewed 63 scans (all acquired after 6 h from stroke) and their readings were compared. Neuroradiologists saw significantly more early ischemic changes than did stroke physicians, general radiologists, geriatricians, or neurologists (all p ≤ 0.0001), predominantly due to neurordiologists’ greater detection of mild hypoattenuation or swelling. However, neuroradiologists took 30 s longer to read each scan compared with other specialists. There were no significant differences in detecting severe hypoattenuation or swelling or hyperattenuated arteries between the various specialists, and years of experience did not account for any of the observed differences. Nonetheless, whereas this study did not investigate the sensitivity or specificity of electronic scan viewing versus cut film, the authors suggested that the two are comparable, provided viewing conditions (ambient light, etc.) are appropriate.

One U.S. study met the selection criteria for intermediate outcomes in 2008. This study (prospective observational, n = 787) investigated discrepancies between teleradiology and in-house radiology interpretations of CT scans in an ED at a level 1 trauma center.¹¹⁷ Over a period of 3 months, 550 CT scans of the head, cervical spine, chest, and pelvis were assessed. Minor discrepancies were identified in 116 (21.1%), and they involved renal or hepatic cysts, pulmonary nodules, or nonacute brain findings. Most discrepancies resulted from an in-house radiologist finding an abnormality not mentioned by the teleradiologist. Major discrepancies were identified in 32 (5.8%). Eight of the 32 were attributed to misinterpretation by teleradiology, with one case leading to an adverse event. Nine were attributed to misinterpretations by the in-house radiologists. Overall, discrepancies between the teleradiologists and in-house radiology occurred in 6% of CT scans. Discrepant interpretations were more common for certain types of pathologies (intraparenchymal cerebral changes, small bowel pathology, renal calculi, and pulmonary emboli).

In 2009, two studies met the criteria (one from the U.S. and one from Serbia). However, we will include in this review two additional studies, which do not meet the inclusion criteria because the topics explored are of interest.

The two studies published in 2009 focused on two different clinical applications. The first was a survey/case–control study (n = 75 radiologists and 31 emergency physicians) that completed 246 studies before the implementation of teleradiology and 382 after.¹¹⁸ A group of 1,638 CT brain studies over an identical period of time was used as a comparison group for a total n of 2,266. The median time for a preliminary interpretation of a CT pulmonary angiographic study by radiology department chairs was 60 min versus 20 min by emergency medicine physicians. The use of teleradiology resulted in a significant improvement in the percentage of preliminary reports for CT pulmonary angiographic studies (51% vs. 62%). Similar improvements were observed for number of studies completed within 20 min (95% vs. 13%). The authors concluded that the use of teleradiology to interpret off-hours inpatient imaging serves as an important process improvement tool in decreasing the time to a preliminary written report for CT pulmonary angiographic studies.

From Serbia, an RCT (n = 432) investigated the reliability of remote reading of radiographic images of the face and oral cavity over a Web server.¹¹⁹ Patients were of both genders, between the ages of 20 and 87, and had impacted molars. Both radiography and photographs of the face and mouth cavity were taken, and the images were uploaded to the Web server and transmitted to teledentists. Diagnostic agreement between the two modalities (telemedicine vs. in-person) was assessed in terms of sensitivity, specificity, and effectiveness. The results indicated almost complete diagnostic agreement between the two modalities (κ = 0.99, sensitivity = 99%, specificity = 99%, and effectiveness = 99%). Hence, it was concluded that diagnostic assessment of the clinical diagnosis of impacted or semi-impacted third molars assisted by the telemedicine approach was equal to the real-time assessment of clinical diagnosis.

Two other studies in 2009 (both U.S.) investigated the diagnostic accuracy of coronary CT angiography (CTA) on mobile phones and diagnostic ultrasound in retroperitoneal as well as pelvic imaging in space, respectively. Since none of these studies had the requisite n of 150, they will be briefly discussed here, but not presented in tabular form (Table 1).

Diagnostic accuracy of coronary CTA on a mobile handheld phone was assessed on a sample of 102 patients with stable chest pain.¹²⁰ Two blinded imagers interpreted the CTA in random order with a third to achieve consensus in two modalities: a dedicated three-dimensional imaging workstation and a mobile handheld device. Per-patient sensitivity, specificity, and positive and negative predictive values were 100%, 78%, 60%, and 100%, respectively; and per-artery 95%, 85%, 41%, and 99%. Thus, it was concluded that the interpretation of coronary CTA using a mobile handheld device with dedicated software for medical image evaluation possesses high diagnostic accuracy for detection and exclusion of significant artery stenosis.

A NASA study evaluated the diagnostic capability of ultrasound at MACH 20 as a means of diagnosing, monitoring, and treating medical or surgical conditions during spaceflight. Physiological changes during spaceflight can affect the genitourinary system and can cause urinary retention or nephrolithiasis. This study describes the use of ultrasound by a nonphysician crew to self-examine on the International Space Station, remotely guided by voice commands from experienced Earth-based sonographers. The results indicated that microgravity ultrasound imaging can provide diagnostic quality images of the retroperitoneum and pelvis, offering improved diagnosis and treatment for onboard medical emergencies. The reader may extrapolate the parallels between spaceflight and terrestrial applications.

In 2010, two studies met the selection criteria, one each from Norway and Greece. They investigated the effects of teleradiology on length of hospital stay and the detection of thyroid nodules.

From Norway, a retrospective record review (n = 3,088; 1,509 pre- and 1,579 postintervention) was conducted over a period of 1 year pre- and 1 year postimplementation of Information and Communication Technology (ICT) to support diagnostic imaging.¹²¹ The objective of the study was to determine whether the technological improvements were associated with a reduction in patients’ hospital LOS. After 1 year of ICT implementation, there was a general reduction in LOS from 5.3 to 3.9 days, but the difference was not statistically significant. However, there was a 25% reduction for one group—namely, patients with CT scans—after the introduction of PACS and RIS. This was true despite the heterogeneous nature of this group involving 445 different discharge diagnoses. The findings indicate documented improvement in clinical access to radiology results during that period.

From Greece, the performance of a wireless PDA teleradiology terminal was assessed in terms of diagnostic performance.¹²² An expert evaluated a total of 144 ultrasound images with thyroid nodules using a PDA. In addition, 10 members of the hospital medical staff completed a questionnaire that inquired about user assessment of mobility (how the user finds carrying, storing, and operating the equipment), usability (performance capacity), stability (technical acceptability), and performance (processing time, loading speed). The expert physician concluded that the ultrasound thyroid images were of similar quality to those displayed on a diagnostic visual display unit screen. On the other hand, the expert found difficulties in diagnosing microcalcification, internal echo texture, and vascularity. The PDA terminal provided rapid, secure, and convenient portable access to PACS images and the image quality was sufficient for diagnostic interpretation of ultrasound of the thyroid.

A novel methodology was employed for studying the biomechanics of ankle fractures using video recordings posted on YouTube.¹²³ This method studies injury mechanisms in vivo from YouTube, which enables viewing injuries as they occur and comparing them with subsequent radiographs. Over 1,000 video clips of potential ankle fractures were examined for assessing foot position and deforming force. Subsequently, X-rays were taken from the same individuals. The deforming mechanism in the video clips was classified as supination external rotation, supination abduction (SAD), pronation external rotation (PER), or pronation abduction. X-rays were classified the same way. Of the initial 1,000 video clips, 240 individuals posting such videos were contacted and 96 agreed to participate. Data analysis was made on a case-by-case basis for the first 15 using the Lauge-Hansen fracture classification system. The findings indicate that the Lauge-Hansen system was only 58% accurate overall in predicting fracture patterns and it varied substantially by type of injury. While the findings are inconclusive—and will not be reported in tabular form—the authors indicated that they have developed a flexible and valuable methodology for studying injury mechanisms, a methodology with a wide array of future applications.

Two studies met the selection criteria for intermediate outcomes in 2011, one from Lebanon and one from Saudi Arabia. The Lebanese study (RCT, n = 385) investigated the effects of two types of short message services (SMSs) on participation in breast cancer screening.¹²⁴ All participants had electronic medical records at a family medicine center and they were divided into two groups. The first group (n = 192) received SMS text invitation to do a mammogram, and the second group received an enhanced SMS containing information regarding the benefits of the mammogram. The total intervention cost was in U.S. dollars. The study subjects were observed for 6 months. No significant differences in response rates were observed between the two groups; the two modalities of SMS, 30.7% and 31.6%, of the two groups underwent a mammogram test during a 6-month follow-up period. Hence, the detailed information provided in the SMS did not make a difference in mammogram uptake.

A Saudi study (observational, n = 148, slightly less than the target of 150) evaluated the effects of different frequencies of online verifications on patient setup accuracy and margins among patients receiving radiotherapy in the head and neck region, chest, abdomen, and pelvis, using electronic portal imaging.¹²⁵ Online verification intervals varied from zero, once a week, and every other day. The data show an effective improvement of both the systematic and the random errors with increasing frequency of online verifications. However, some setup errors remained after online verification every other day. The authors concluded that in patients where high setup accuracy is desired, daily verification is highly recommended.

A Canadian retrospective study (n = 120) is discussed here, but not included in Table 1. It described a new teleradiology system that enabled ready access to 2D and 3D visualization on a smartphone device, without patient data being available on the device.¹²⁶ The study investigated the accuracy and interpretation times of acute stroke from noncontrast CT brain scans and CT angiogram head scan. The sensitivity, specificity, and accuracy of detecting acute parenchymal ischemic changes were 94.1%, 100%, and 98.09%, respectively, for reader one and 97.05, 100%, and 99.04% for reader two. Similar high rates were observed for detecting vessel occlusion, and no significant differences were observed in interpretation time.

In 2012, six studies met the selection criteria, one each from the U.S., United Kingdom/Switzerland, Singapore, Italy, Austria, and a group from Ireland, Brazil, and Canada. The first was a large-scale retrospective analysis of an administrative database of the University Health System Consortium.¹²⁷ A total of 13,288 young male patients (18–55 years of age) with suspected appendicitis were selected. Of these, 11,340 were assessed using a CT scan of the abdomen, and 1,888 did not receive the CT scan. Those undergoing the CT scan had less morbidity (0.86% vs. 2.2%) and fewer 30-day readmissions (1.8% vs. 5.13%). The authors concluded that the use of an abdominal CT scan is associated with improved immediate postoperative complications, lower readmission rates with observed higher length of stay, and increased cost of care.

Researchers from the United Kingdom and Switzerland evaluated the quality of outsourced CT teleradiology reports using a prospective analysis (n = 1,028: 437 females, 591 males, mean age = 51).¹²⁸ This study evaluated diagnostic agreement in an international teleradiology program based in the United Kingdom and Australia. Assessment of agreement on diagnosis was made by a panel of in-house radiologists using a five-point scale, ranging from (1) no disagreement to (5) unequivocal potential for serious morbidity or threat to life versus outsourced teleradiology. No disagreement was found in 79% of patients; 16% were rated as category 4—disagreement over style or presentation of report; 4% as category 3—clinical significance is debatable; 1.3% as category 2—omission of finding with moderate morbidity, but no threat to life; and no cases with category 1, which stood for unequivocal potential for serious morbidity or threat to life. The authors concluded, “The results of the present study support the use of an outsourced after-hours teleradiology reporting service ….”

From Singapore, the performance of an iPad tablet computer was compared with PACS workstation in terms of diagnostic discrepancies in CT and MRI reporting in emergency.¹²⁹ Three radiologists reviewed 264 readings (79 CT and 9 MRI studies). The results indicated 3.4% (15 of 264) having major discrepancies and 5.6% (5 of 264) having minor discrepancies. However, none of the errors were committed by all three readers. This suggested that discrepant readings were likely due to reader factors such as inherent variability in interobserver interpretations rather than hardware or software limitations. The authors concluded that diagnosis of emergency conditions commonly encountered in after-hours calls on CT and MRI using tablet computers such as the iPad can be made with good agreement to those reviewed on dedicated PACS workstations. They also suggested that shortcomings in software and application design [regarding stability and limitations in image manipulation] should be addressed if the potential of tablet computers for mobile teleradiology is to be fully realized.

A retrospective review of medical records (n = 733) was conducted in Italy to assess the effects of the telemedicine intervention (a remote neurosurgical second opinion in the management of intracerebral hemorrhage).¹³⁰ Neuroradiological and clinical data for patients were transmitted through a high-technology hub-and-spoke telemedicine network. Neurosurgical consultations were available to seven hospitals (spokes) serving a population of about 70,000, and data were gathered over a period of 8 years, 2003–2011. During this time period, about 2,800 patients were treated for stroke, 733 of them with intracerebral hemorrhage. This latter group received teleconsultations. The average duration of these teleconsultations was 38 min versus 160 min for in-person consultations; 24% were transferred to the hub; 13% had surgical treatment; and 11% underwent neurointensive care. A small percentage (1.4%) of patients who were treated at the spoke sites were subsequently transferred to the hub because of worsening symptoms or CT findings. However, the overall rate for correct interpretation of the data was 96.5%. Hence, the telemedicine intervention allowed on-demand rapid visualization of neurological and clinical data and appropriate treatment at peripheral hospitals. Furthermore, it allowed for prompt transfer of a small minority of patients with secondary deterioration, as necessary.

A clinical investigation (observational comparative analysis, n = 200) of feasibility, image quality, lesion detection, and breast imaging reporting and data system (BI-RADS) assessment was conducted in Austria.¹³¹ Two hundred digital mammograms (800 views) were sent to different institutions through a teleradiology network where three readers conducted the assessment. Automatic readout of image quality showed identical results, which were 90% and 97% before and after transmission. Only 2.5–9.5% were rated as poor. Congruence between the readers in terms of BI-RADS ranged from 90% to 91% at institution one versus two and from 86% to 92% at institution one versus three. Reader agreement on detection of masses and calcifications was high (κ = 0.78–0.89). Hence, it was concluded that transmission of uncompressed mammograms from one institution to another does not degrade image quality, lesion detection, or BI-RADS rating.

Finally, an international group of researchers from Ireland, Brazil, and Canada conducted a prospective review of the accuracy of virtual consultation in determining treatment plans for patients with malignant epidural spinal cord compression.¹³² They started with a sample of 146 patients, but 20 patients were excluded because they were referred directly to a spinal surgeon. The overall accuracy of the teleconsultation was estimated at 92%. Therefore, this process has the potential to shorten the time to reach a treatment decision and thereby enhance clinical outcomes. However, due to the small sample, this study will not be presented in Table 1.

Three studies met the inclusion criteria in 2013, one each from Taiwan, Germany, and the United Kingdom. An additional study from Turkey will be discussed, but not presented in tabular form.

The first was an observational cross-sectional study (n = 3,770) from Taiwan aimed at assessing an online system in terms of the detection and accuracy of osteoporosis among women over the age of 30.¹³³ The authors prefaced their study by describing osteoporosis as a silent killer and the importance of early detection and treatment. More than 50% of women over 50 years of age will develop some degree of osteoporosis. The study subjects underwent dual-energy X-ray absorptiometry and randomly assigned to 700 as the database for the system and 167 as the software verification testing to determine the accuracy of osteoporosis prediction. Survey data were gathered regarding demographics, height and weight, family history, smoking and drinking, wearing high heels, outdoor activity, perceived humpback, and knowledge about osteoporosis. In addition, information was gathered about calcium-rich diet, exercise, home safety, and compliance with physician orders. The study focused on the predictive power of an online osteoporosis detective system (OSLDS) based on these data as reported by the women participants. The findings indicated a high sensitivity of 75%, specificity of 75%, and positive predictive value of 75%. Hence, the authors concluded that the system had excellent reliability and validity especially among Asian women.

A German retrospective observational study (n = 536) investigated the reliability of standardized brain CT evaluation for early ischemic change by stroke teleneurologists.¹³⁴ Two neuroblinded radiologists were asked to reassess the earlier CT findings. Complete imaging data were available for 536 patients (corresponding to 851 cerebral ischemic events, 105 intracranial hemorrhages, and 80 stroke mimics). The earlier findings were categorized as false positive or false negative with regard to brain pathology. Discrepant findings were observed in 8% of the cases, and 1.7% were rated as clinically relevant. Stroke neurologists recommended thrombolytic treatment in eight patients, despite early ischemic changes, and one of them had symptomatic intracranial hemorrhage (sICH). Interobserver agreement between stroke neurologists and expert readers was substantial at κ = 0.62. The study demonstrated that stroke neurologists can reliably interpret the cerebral CT scan of patients with clinically suspected acute ischemic stroke in telemedicine in real time. At the same time, the authors indicate that decision-making remains the responsibility of neurologists: “… steady control of reading quality is recommended to provide the best medical care to individual stroke patients.”

A UK observational study (n = 253) investigated the accuracy of Internet images of injuries to the glenoid labrum (fibrocartilaginous rim around the margin of the shoulder socket) following shoulder separation from accident or injury.¹³⁵ Google and Bing search engines were used to find relevant images. Three reviewers were asked to assess the accuracy of image labeling. Of images labeled Bankart lesion, 30% (9/30) were incorrect, while Perthes lesion images were incorrect in 15% (9/60), and 4% of ALPSA lesion images were incorrect (2/46). These findings were corroborated by high interobserver reliability (κ = 0.81). However, labeling accuracy was higher on educational sites compared with commercial sites (6% vs. 25% inaccurate).

Another retrospective observational study (n = 100) was conducted in Turkey regarding the feasibility and accuracy of teleconsultation of coronary angiograms using IPhone 4 and FaceTime versus workstation monitor.⁹⁷ A consultant cardiologist reevaluated the images on iPhone using FaceTime and on the workstation monitor, including severity of angiographic lesions. There was high agreement between the core laboratory and the readings on the iPhone (κ = 0.80). Accordingly, the authors concluded that smartphones may serve as a supplementary teleconsultation tool in both elective and emergency situations. However, because of the small sample size, this article will not be included in Table 1.

Three studies met the selection criteria for 2014, two from the U.S. and one from Botswana. The first U.S. study (prospective rater analysis, n = 2,000) investigated the impact of radiologists’ access to electronic health records (EHR) in ED on radiological interpretations and medical management.¹³⁶ The investigators tried to ascertain the significance of access to the EHR when remote radiologists are asked to interpret head CT scans for ED patients. They compared medical information generation by ED physicians with information generated by interpreting radiologists who had access to EHR of patients. Three neuroradiologists conducted inter-rater reliability analysis of 2,000 consecutive head CT scans ordered by ED physicians. The results indicated strong agreement among the raters on the presence of additional medical content—0.82. The authors concluded, “Our results suggest that access to the EHR is necessary to radiologic decision-making in the ER.”

The second U.S. study (observational, n = 444) was conducted at the VHA concerning osteoporosis screening and treatment among veterans with recent fracture after implementation of an e-consult service.¹³⁷ To highlight the importance of the problem, the authors pointed out that fewer than 24% of veterans receive appropriate evaluation and/or treatment for osteoporosis within 6 months of an index fracture. Hence, the e-consult service was established to facilitate the identification of and recommend management for patients with recent fracture. Of the 444 who were not already on treatment, 129 (or 29.1%) of e-consults recommended bisphosphonate treatment and 258 (or 58.1%) bone density assessment. The primary care providers responded by prescribing bisphosphonate in 74 patients and bone density assessment in 183. Before the e-consult service, the rate of osteoporosis treatment following a fracture was 4.8% for bisphosphonate and 21.3% for calcium/vitamin D. These rates increased to 7.3% and 35.2%, respectively. Both differences in rates are statistically significant. Therefore, the e-consult service had a modest, yet significant, effect on improving the rates of osteoporosis treatment among patients with a recent fracture.

The third study (RCT, n = 150) in 2014 was conducted in Botswana, a resource-poor country with a population of about 200 million with only three government radiologists.¹³⁸ The purpose of the study was to assess the diagnostic accuracy of mobile phone teleradiology in the evaluation of chest X-rays. The images were selected from a database that contained typical pathologies in Botswana, including pneumonia, lung cancer, tuberculosis, pneumothorax, and normal. Seven radiologists from a U.S. academic medical center were blinded regarding patient medical histories and to the original clinical report. They reviewed 75 plain films on light boxes before viewing 75 digitized photographs on mobile phone, and then in reverse order. In both modalities of viewing, the correct diagnosis was made in 79% of the cases (82% and 76% for film viewing on a light box and digital photographs, respectively). A subgroup analysis revealed higher agreement for lung cancer and pneumonia compared with tuberculosis and pneumothorax. The authors commented that the quality of digital cameras on mobile phones is rapidly improving. Thus, the image quality obtained using mobile teleradiology can be expected to improve further.

Finally, two studies met the selection criteria in 2015, one from the U.S. and one from The Netherlands. The U.S. study (retrospective record review, n = 1,445) investigated the frequency of rural patient transfers to an ED and the factors associated with such transfers.¹³⁹ Over a 12-month period, a total of 1,445 patients were transferred to a tertiary ED and 1,066 were transferred from a rural ED. The analysis was focused on 685 patients after excluding 381 trauma and pediatric patients. One in four of this net group was transferred because of the lack of a radiology service at the rural site. These transfers place additional and social burdens on patients and their families. Accordingly, the authors suggest telemedicine as an alternative diagnostic option that is likely to reduce the number of patient transfers.

A somewhat complimentary study (retrospective, observational, before, and after design, n = 806) investigated the impact of introducing a teleradiology service on travel and treatment in a remote general practice located on the island of Ameland in the north of The Netherlands.¹⁴⁰ The study compared the accuracy of diagnosed fractures and the attendant unnecessary trips, treatment, and number of X-rays before and after the introduction of teleradiology. Of the 316 and 490 patients with trauma who were seen at the general practice in 2006 and 2009, 66 and 116 were found to have fractures or dislocations. Before the introduction of teleradiology (i.e., 2006), nine fractures were missed, whereas only two were missed after teleradiology in 2009. In 2006, 15 were treated at the general practice compared with 77 in 2009. In conclusion, since the introduction of teleradiology, the number of missed fractures in patients visiting the general practice with trauma and the number of unnecessary trips to a hospital are reduced. In addition, more patients with fractures and dislocations can be treated in the general practice, as opposed to the hospital.

Health Outcomes

Only five studies in teleradiology focused on health outcomes between 2005 and 2015, one each from South Africa, Italy/U.S., Portugal, Germany, and Japan. This is to be expected because radiology and teleradiology in particular are aimed at providing an accurate diagnosis for the most part and in monitoring/mentoring during IR (Table 2).

Table 2.

Summary List of Empirical Evidence in Teleradiology: Outcomes

REFERENCE	YEAR	COUNTRY	STUDY DESIGN	SAMPLE SIZE	MODALITY/INVERVENTION	FINDINGS	COMMENTS
Ichikawa et al.141	2005	Japan	Observational	Nonempirical	Grid portal	Grid aggregates computational capabilities for brain science	Portal enabled flexibility with large computational power
Zulu et al.142	2007	South Africa	Observational	316	Teleconsult/head injury	Reduced unnecessary travel; 84% treated locally; 16% required transfer	Telemedicine allows access to external resources and ensures optimal use of resources
Fabbri et al.143	2008	Italy	Observational	12,675	Telemedicine/head injury	Predicted unfavorable outcome: sensitivity 95.6%; specificity 86.0%	The scoring system for organ dysfunction is good predictor
Lefere et al.144	2013	Portugal	Prospective analysis	510	Teleradiology/colorectal cancer	Sensitivity = 98.11–100%; specificity = 90.97– 87.07%	Teleradiology CT accurate in screening colorectal cancer
Zerna et al.145	2015	Germany	Retrospective analysis	1,659	Stroke net/CT	Interobserver agreement	Underestimation of stroke not associated with thrombolysis-related ischemia

In 2005, a Japanese and U.S. group examined the clinical benefits of a grid portal for the analysis of brain function on the basis of region of interest.¹⁴¹ In vivo observation of brain function can lead to more efficient and effective diagnosis and treatment. Measurement devices that capture activities inside the brain are available today, including electroencephalography, electrocorticography, functional MRI, and magnetoencephalography. These devices can detect subtle changes in magnetic field secondarily generated by brain functions over time. However, these require the processing of large amounts of data that cannot be efficiently handled by single computers. The grid offers a solution by aggregating computational resources on the Internet in a transparent way. Consequently, turnaround time is substantially reduced and quality of service is also improved. The authors suggest that high-performance computation contributes greatly to the overall progress of brain science. The portal has thus made it possible for the users to flexibly include the large computational power in what they want to analyze.

In 2007, a South African study (prospective observational, total n = 316, initial 86 and subsequent 230) investigated the outcome of head injuries in general surgical units with an off-site neurosurgical consultation service.¹⁴² This publication was based on two sequential studies. The first was a pilot study in a single surgical unit over 18 months, followed by a larger study over 6 months in six surgical units. Patients in both studies had serious head injuries and lived in the KwaZulu-Natal province of South Africa, with a population of about 8 million. Typically, patients with serious head injuries are managed in peripheral hospitals in consultation with a neurosurgical specialist through telemedicine. Following teleconsultations, 84% were managed locally and 16% required transfer to the neurosurgical unit. Normally, head trauma is associated with high morbidity and mortality. The overall death rate was 13%. This corresponded to 11% in the general surgical unit and 22% in the neurosurgical unit. Thus, those who were referred to the neurosurgical unit experienced significantly higher mortality rates. Importantly, the delay before surgery caused by the transfer did not seem to affect outcome. The authors pointed out that advances in information technology (IT) have resulted in communication on a global scale and the transmission of images by telemedicine has obviated the need for unnecessary patient transfer, and the use of telemedicine may allow unlimited access to a limited area of expertise, thus ensuring optimal use of resources.

Another study (prospective review, n = 12,675) of patients presenting with head injury at the ED was conducted in Italy in 2008.¹⁴³ This study was focused on moderate head injury admitted to an ED of a general hospital connected to a regional neurosurgical center through telemedicine, and it was aimed at identifying early predictors of unfavorable outcomes (including death, permanent vegetative state, or permanent severe disability) at 6 months. Injuries were coded according to the Abbreviated Injury Scale, and the Injury Severity Score was used to calculate level of severity. The database derived from a general hospital with a telemedicine link to the regional neurosurgical center. Over a 6-year period (1999–2005), 309 cases were identified as having moderate head injury. Of these, 64.7% had a positive CT scan for intracranial injury, 16.5% needed a neurosurgical intervention, and 14.6% had an unfavorable outcome at 6 months. Predictor variables included basal skull fracture, subarachnoid hemorrhage, coagulopathy, subdural hematoma, a modified Marshall category (a scoring system for organ dysfunction), and the Glasgow Coma Scale. This group of variables predicted 6-month unfavorable outcomes with high sensitivity (95.6%) and specificity (86.0%).

In 2013, a Portuguese study (prospective analysis, n = 510) investigated the performance characteristics of a teleradiology-based CT colonoscopy in screening for colorectal cancer on a remote island.¹⁴⁴ Early diagnosis and treatment are necessary for a successful outcome. All patients underwent CT colonoscopy and optical colonography, and the data were sent to a radiologist at a remote center through telemedicine for interpretation. All colonographies were interpreted by an experienced radiologist using a 3D interpretation method, and all colonoscopies were reviewed by five colonoscopists who were blinded to the results of the colonoscopies. A total of 496 patients had complete information for assessing results. The prevalence of all lesions ≥6 mm and adenomas ≥6 mm was 13.9% and 10.7%, respectively. Advanced neoplasia was present in 6.5%, and 0.8% with adenocarcinoma. The results indicate high sensitivity, specificity, and positive and negative predictive values for CT colonography: adenomas ≥6 mm at 98.11%, 90.97%, 56.52%, and 99.75%, respectively. For advanced neoplasia, sensitivity, specificity, and positive and negative predictive values were 100%, 87.07%, 34.78%, and 100%, respectively. In brief, teleradiology-based CT colonography was accurate to screen a patient cohort of a remote island at average risk for colorectal cancer.

The final study in this series (retrospective analysis, n = 1,659, over a 5-year period—2007–2012) was concerned with the significance of misinterpretation of CT scans by stroke neurologists in a stroke regional network in Germany, The Stroke Eastern Saxony Network (SOS-NET).¹⁴⁵ The study investigated the association between CT misinterpretation by stroke teleneurologists and intracranial hemorrhage. The primary outcome was severe sICH, and the secondary outcome was unfavorable outcome at discharge. Of 1,659 patients with acute ischemic stroke, thrombolysis (t PA) was performed in 657 patients, but complete data were available for 432 patients. Of this latter group, 4.4% had sICH, and 59.95% had an unfavorable outcome at discharge. Interobserver agreement was fair (κ = 0.51). Underestimation was not associated with sICH (adjusted odds ratio = 1.32). Despite this modest level of inter-rater agreement between stroke neurologist and expert neuroradiologists, underestimation by the former was not associated with thrombolysis-related sICH in our stroke network.

Cost Implications

There is a large volume of studies that investigated the cost-effectiveness of specific modalities of radiology, such as CT colonography versus conventional colonoscopy for colorectal screening,^146,147 full-field digital mammography versus screen film mammography,^148,149 and digitized radiological department versus traditional radiology,¹⁵⁰ although this latter is no longer an issue because PACS is widely accepted and implemented in radiology departments. We focus here on economic studies that investigated the benefits and costs of teleradiology versus traditional radiology, as shown in Table 3.

Table 3.

Summary List of Empirical Evidence in Teleradiology: Cost

REFERENCE	YEAR	COUNTRY	STUDY DESIGN	SAMPLE SIZE	MODALITY/INVERVENTION	FINDINGS	COMMENTS
Pedersen et al.151	2005	Norway, Germany, Japan	Cost-saving analysis	149	Monitoring patients with stents	More efficient use of health facilities	Savings from obviated travel, spared accommodations, and reduced hospitalization
Plathow et al.152	2005	Germany	Comparative cost	500	Teleradiology cost/small hospital	Teleradiology profitable after 322 consultations per year	Most cost-effective at 500 CT scans per year
Flanagan, et al.153	2012	United States	Retrospective cohort analysis	491	Internet-based transfer	Internet and CD image transfer associated with lower repeat rates of imaging	PACS networks reduce cost and radiation exposure
Gray et al.154	2012	Ireland	Retrospective record review	145	CDU surveillance/EVAR	Sensitivity = 100%; specificity = 85%	CDU can substitute for EVAR without compromising accuracy and saving cost
Rosenberg et al.155	2013	Germany	Monte Carlo simulation	100–500	Teleradiology in mid-size hospital	Avoiding deficient pricing by 90% increased cost of cranial CT twofold	Important to consider pricing thresholds

CDU, color duplex ultrasound.

In 2005, two studies met the selection criteria for this analysis of evidence on teleradiology cost, one each from Norway and Germany. The Norwegian study is based on cost-saving analysis (n = 149), which estimated the cost savings that are accrued by local monitoring of patients treated with stent grafts for aortic disease.¹⁵¹ The components of cost savings included obviated travel, spared accommodations, obviated hospitalization (estimated at 34 of 149), and staffing, as well as necessary investment in software (in relation to cost savings). The cost analysis showed a potential for cost savings from local follow-ups, especially from moving from inpatient care at the university hospital to outpatient care locally. In addition, the use of teleradiology freed hospital beds at the university hospital, which can be used for new patients. The authors concluded that teleradiology could lead to more efficient use of healthcare facilities, which should be in the interest of the health authorities [referring to Norway].

The German study (comparative analysis, variable n) assessed the economic impact of teleradiology (CT scan examinations) in a small hospital setting.¹⁵² The premise of this analysis was that CT scan analysis in a local small hospital through teleradiology obviates travel costs. Hence, it would be important to estimate its total cost impact, including both fixed and variable costs under various scenarios, including (1) CT examination by an external institution, including patient transport; (2) external consultation through teleradiology; and (3) complete in-house radiology. Costs included both variable and fixed. Analysis revealed that scenario (1) was the most cost-effective for 500 CT scans per year, but also the most time-consuming. Scenario (2) is most cost-effective beyond 548 CT scans per year. Scenario (3) is economically feasible beyond 1,065 CT scans per year. On the basis of €30 per CT consultation, a teleradiology service providing system will be profitable from 322 consultations per year. The authors concluded that teleradiology applications are economically reasonable in a wide range in a small hospital.

In 2012, two studies met the selection criteria, one each from the U.S. and Ireland. The U.S. study (retrospective cohort, n = 491) evaluated an Internet-based and compact disc-based image transfer system with other available systems in terms of repeat imaging rate, cost, and radiation dose among patients transferred to a level I regional trauma center, involving four states, Alaska, Washington, Montana, and Idaho.¹⁵³ The sample consisted of 500 consecutive trauma patients who were transferred from an initial assessing hospital. The average Injury Severity Score (a standardized severity of major traumatic injury of six body systems) was 14.7; average age was 40.5 years, and 70% were men. All imaging data were identified as outside or local. A repeat study was defined as a local study after an equivalent outside study not meeting the criteria for a completion study. A total of 69 repeat CT scans were performed on 55 patients, equivalent to 17% rate. These tended to be older and more severely injured compared with those who did not have repeat imaging. The total value of imported CT studies was $244,373.69. Repeat imaging totaled $20,495.95 or $84.65 per patient with transferred CT studies. Based on this analysis, the authors concluded that a combination of Internet and compact disc image transfers in an interhospital transfer is associated with much lower repeat rates than those in the literature, suggesting that regional PACS networks may be useful for reducing cost and radiation exposure associated with trauma.

The Irish study (retrospective record review, n = 145) investigated the cost and efficacy of color duplex ultrasound (CDU) versus CT scans for the surveillance of patients after endovascular aneurysm repair (EVAR).¹⁵⁴ We decided to include this study, despite the fact that the sample size is slightly lower than the requisite 150. The use of CDUS as the first-line surveillance tool resulted in reducing expenditures from €117,500 to €34,915, a saving of €82,285 (or 70%). CDUS had a sensitivity of 100% and a specificity of 85% in the detection of endoleaks compared with CT. The positive predictive value was 28% and negative predictive value 100%. Hence, CDU can substitute for CT in postoperative surveillance following EVAR as the tool of choice without any compromise in accuracy of imaging and resulting in significant cost savings.

In 2013, a German study (Monte Carlo simulation, n = 100–500 repeated cost analysis) provided cost estimates for low-volume teleradiology service for a mid-size university hospital.¹⁵⁵ This study was aimed at demonstrating the feasibility of performing a cost analysis of teleradiology showing break-even points for a cost-effective practice. A Monte Carlo simulation was conducted to estimate cost amplitude (measure of the magnitude of differences between values) and to identify pricing thresholds for break-even points. The model showed cost distribution per category of service. Avoiding deficient pricing by a likelihood of 90% increased the cost of a cranial CT almost twofold compared with the lower limit cost. This methodology provided useful data for enhancing efficiency and in setting realistic reimbursement.

Summary and Conclusions

The practice of teleradiology is well established in the U.S. and elsewhere, relying on both high-end (e.g., CT and MRI scanners) and off-the-shelf technologies (e.g., digital cameras and smartphones) to acquire, transmit, and interpret images in a wide variety of clinical environments. In many ways, it resembles the general practice of radiology in terms of typical workflow, procedures, and patient contact. It has the added advantage of enabling the radiologist to manipulate the images in ways that would not be possible with film alone, thereby extracting more clinically relevant information from the images and rendering more accurate diagnosis and clinical decisions.

Our initial search for relevant research studies, using the terms teleradiology, radiology, and telemedicine, yielded a total of 2,271 journal articles. These articles were sorted by relevance. Those that were outside the purview of this analysis, such as editorials, commentaries, purely technological descriptions, and/or not empirical in nature, were discarded. In addition, those that did not address the topics of interest, namely feasibility/acceptance, intermediate effects, outcomes, and cost, were discarded. Accordingly, we ended up with 91 studies that met the selection criteria for evidence, namely credible research design and adequate sample size. As noted earlier, we expanded the scope of our review to incorporate other IT applications in radiology that support teleradiology or use transmitted images in other applications than clinical interpretation (e.g., education and research).

The evidence regarding the feasibility of teleradiology and related IT applications has been well established for over two decades. As Gitlin noted, “Thanks to the telecommunications and computer industries, by 1990, it appeared that both teleradiology and image management systems had all the hardware and software needed to proceed with addressing the issues of clinical acceptance and cost-effectiveness.”¹⁵⁶ In the ensuing years, the ACR established standards to assure quality and patient safety, including technical specifications regarding digital acquisition, transmission, image resolution, luminescence, and display.

Between 2005 and 2015, a total of 29 studies met the inclusion criteria for the empirical analysis of evidence in terms of intermediate outcomes, outcomes, and cost. The majority were based on a comparative analysis between teleradiology and conventional radiology, using either record review or prospective observational designs. Sample size varied from 144 to 13,288. Two studies that did not meet the requisite sample size of 150 (with n = 144 and 148) were included because they shed light on important developments in related applications that may be incorporated into teleradiology with further experimentation and innovation. There was near-unanimous evidence regarding the effectiveness of teleradiology and related applications compared with conventional radiology, as measures point to near-unanimous agreement regarding the high sensitivity (true positive rate, or the proportion of abnormalities correctly identified) and specificity (true negative rate, or the proportion of disease-free cases correctly identified). However, one study in 2005 suggested that radiologists tend to have more confidence in traditional radiology using the X-ray box rather than a computer screen. This study is, however, over 10 years old and display technology has improved significantly since then. The following is a summary of key findings regarding feasibility:

• • Participants in teleradiology benefited from improved quality and enhanced efficiency, including improved rate and type of treatment, improved image interpretation, and reduced postoperative complications.

• • Teleradiology is associated with reduced transfer of trauma patients, reduced unnecessary trips for patients, reduced acquisition (hence dose) of repeat imaging, and reduced waiting time for patients, as well as expedited image interpretation.

• • Diagnosis from digital photographs was equivalent to conventional image viewing in select image and lesion applications. The use of digital cameras did not compromise detection of lesions such as pneumonia and fractures.

The literature reported here uniformly affirmed the feasibility of teleradiology in a large variety of clinical and diagnostic applications, including screening programs for breast cancer and tuberculosis, diagnosis of various abnormalities and trauma, minimally invasive surgeries, remote consultations between specialists, and collaboration in multisite systems or networks. Indeed, it can be reasonably concluded that we are already in the postevaluation era of the feasibility of this modality of practice.

The evidence on the intermediate effects of teleradiology was heavily focused on accuracy and reliability of teleradiology, including the use of mobile devices, compared with conventional radiology and the use of the light box. The following is a summary of key findings:

• • The validity and reliability of teleradiology, including digital cameras, have been established in detecting various health problems, including (but not limited to) pneumonia, fractures, osteoporosis, breast cancer, intracranial hemorrhaging, stroke treatment, coronary bypass grafting, abdominal pain and appendicitis, and trauma.

• • The benefits of teleradiology have been confirmed in a variety of studies dealing with a variety of health problems and screening procedures, including breast cancer screening and early detection, return for additional imaging in breast cancer, detection of early ischemic changes, expedited door-to-balloon times in cardiac emergencies, prompt response in immunotherapy and cancer treatment, and overall accuracy and reliability.

• • Effects of teleradiology on use of service include reduced need for transfer among rural patients, reduced rehospitalization and LOS, increased frequency of online verification, and patient setup accuracy.

Only five studies investigated the effects of teleradiology on health outcomes. This is understandable in view of the fact that teleradiology is mostly diagnostic in nature. In fact, these findings are simply an extension of those reported for intermediate outcomes. Some of the interesting findings pertain to the unique benefits of grid computing in terms of expanding the capabilities of brain science—the observations that delay in surgery in head injury did not result in adverse outcomes and that underestimation of stroke was not associated with thrombolysis-related response.

Again, only five studies investigated the effects of teleradiology on cost. The findings suggest the following:

• • Remote monitoring of patients with stents improved the efficiency of health facilities and obviated the need for travel and the need for hospitalization.

• • From a return on investment standpoint, there is a minimal threshold for volume to make teleradiology profitable. Pricing thresholds are also important.

• • PACS networks can reduce cost as well as radiation exposure.

Finally, this review of the evidence regarding teleradiology and related applications seems to confirm previous findings regarding the feasibility/acceptance, as well as positive effects on intermediate and regular outcomes, and costs of telemedicine interventions. Early acceptance of teleradiology by the profession and the adoption of guidelines and standards for this modality of practice have resulted in its early acceptance as well as professional conformity to assure quality and patient safety. As the underlying technology continues to develop and flourish, so too will the practice of teleradiology.

Acknowledgments

The research leading to this publication was supported by the National Library of Medicine (The National Institutes of Health in the form of Intergovernmental Personnel Act #F037249 to R.L.B.), and is gratefully acknowledged. The authors wish to acknowledge the valuable assistance rendered by Whitney A. Townsend for the systematic search and organization of the literature. The authors of this article are responsible for the literature selection, analysis, accuracy, and interpretation of the data presented here.

Disclosure Statement

No competing financial interests exist.