Abstract
We aimed to evaluate the feasibility of an iPhone-based remote control system as a real-time remote computed tomography (CT) reading tool for suspected appendicitis using a third-generation (3G) network under suboptimal illumination. One hundred twenty abdominal CT scans were selected; 60 had no signs of appendicitis, whereas the remaining 60 had signs of appendicitis. The 16 raters reviewed the images using the liquid crystal display (LCD) monitor of a picture archiving and communication system (PACS) workstation, as well as using an iPhone connected to the PACS workstation via a remote control system. We graded the probability of the presence of acute appendicitis for each examination using a five-point Likert scale. The overall sensitivity and specificity for the diagnosis of suspected appendicitis using the iPhone and the LCD monitor were high, and they were not significantly different (sensitivity P = 1.00, specificity P = 0.14). The average areas under the receiver operating characteristic curves for all CT readings with the iPhone and LCD monitor were 0.978 (confidence interval 0.965–0.991) and 0.974 (0.960–0.988), respectively, and the two devices did not have significantly different diagnostic performances (P = 0.55). The inter-rater agreement for both devices was very good; the kappa value for the iPhone was 0.809 (0.793–0.826), and that for the LCD monitor was 0.817 (0.801–0.834). Each rater had moderate-to-very good intra-observer agreement between the two devices. We verified the feasibility of an iPhone-based remote control system as a real-time remote CT reading tool for identifying suspected appendicitis using a 3G network and suboptimal illumination.
Keywords: Teleradiology, Computed tomography, Clinical imaging viewing, Acute appendicitis
Introduction
Computed tomography (CT) examination for suspected acute appendicitis is commonly performed in the emergency department (ED). The accuracy for diagnosis of appendicitis on abdominal CT has been reported to be relatively high in several previous studies (sensitivity and specificity range from 93 to 100 and 93 to 99 %, respectively) [1–4]. However, on-site, real-time radiologic image interpretation by an expert (senior radiologist or senior emergency physician) is not readily available 24 h per day. Therefore, some patients’ treatments were decided based on the residents’ preliminary interpretations, which may result in false diagnosis (sensitivity of 80 % and specificity of 91 %) [5]. To compensate for this limitation, several teleradiology studies have been conducted. We have previously reported that the web-based mobile teleradiology system using an ultra-mobile PC (UMPC) with a wireless network was feasible for reading abdominal CTs to diagnose acute appendicitis [6]. However, considering that the smartphone is currently a worldwide, handheld display technology and that the third-generation (3G) mobile network is more accessible compared with wireless fidelity (Wi-Fi) or long-term evolution (LTE) in most countries, an examination of the feasibility of remote CT reading using a smartphone with 3G is necessary. Although a previously published study demonstrated the feasibility of the iPhone as a remote CT reading tool for acute appendicitis [7], it did not consider real-time remote image transmission with a wireless network or suboptimal illumination similar to the emergency room clinical setting. Furthermore, given the preliminary nature of that study because of the insufficient number of reviewers and radiologic images, the findings had weak power for generalization.
In this study, we aimed to evaluate the feasibility of a smartphone-based remote control system as a real-time remote CT reading tool for suspected appendicitis using a 3G network and suboptimal illumination.
Methods
Study Design and Settings
This study, which was conducted from April to May 2014, was approved by the Institutional Review Board of our institution. The design and settings of this study were based on our previous studies [6, 8]. The numbers of sample cases and readers were estimated with the multi- and single-reader sample size program for diagnostic studies (version 1.0) [9]. We obtained pilot data from the preliminary study with three readers and 100 cases; 50 cases had acute appendicitis, and the remaining 50 cases had normal appendix characteristics. Various combinations of readers and cases resulted in 0.80 power for the detection of a 0.025 difference in the area under the curve (AUC). These combinations were 7 readers and 181 cases, 12 readers and 132 cases, and 16 readers and 116 cases (unpublished data). We recruited 16 readers and collected 120 cases in this study [10].
Selection of Images
Abdominal CT images were obtained using a 16-channel multidetector row CT scanner (Somatom Sensation 16, Siemens, Germany) during the portal venous phase in an urban tertiary care teaching hospital.
An independent and blinded researcher, who was not an author in this study, searched the adult patients with suspected appendicitis who underwent CT scans in the ED from January to December 2013. The requirement of informed consent for using patient CT images was waived. The following search criteria were applied:
Patient age: >18 years
Department: emergency medicine
- Predesignated search codes:
-
Chief complaint codes: right lower quadrant (RLQ) pain/tenderness, abdominal pain/tenderness/rigidity, abdominal discomfort, umbilical pain, pelvic pain, and acute abdomenor
- Diagnostic codes: disease of the appendix (K35-38) [11]
-
Type of CT Examination: abdomen CT with contrast enhancement
Search period: January to December 2013
- Exclusion criteria
- Cases with other diseases diagnosed by the CT scan
- Cases without clinical follow-up
Based on the search results, we selected 60 consecutive patients with acute appendicitis confirmed by the pathology report from December 2013 in a retrograde order. An additional 60 patients without acute appendicitis were verified by clinical follow-up and were also selected in the same manner. Each case was randomly assigned a number from one to 120, regardless of the presence or absence of acute appendicitis. To download the CT examination of each case, a new folder was created on the PACS workstation, which functioned as a server computer in this study. Using a digital imaging and communications in medicine (DICOM) viewer, the 120 cases were downloaded from the central PACS server to this folder. The examinations were placed in order of the assigned number, and the patients’ identifying information was deleted. Therefore, the raters easily and rapidly opened and closed the CT examinations without revealing the patients’ identification through the DICOM viewer during rating.
Teleradiology System
Remote Viewer: Smartphone
The iPhone 5S (iPhone 5S, Apple Inc., USA), which is one of the most widely used handheld smartphones, was chosen as the remote viewing displayer. It is smaller and weighs less than the devices used in previous studies [6, 8] (dimensions of 123.8 × 58.6 × 7.4 mm, with a weight of 112 g); it also has a small screen with a diagonal dimension of 10.2 cm. The specifications of the iPhone display are shown in Table 1 [12]. The maximum luminance of the iPhone is 556 cd/m2. Although a luminance of 400 cd/m2 could satisfy the needs of most radiologists in a dimly lit office environment such as a radiology reading room [13], the brightness in this study was set to maximum. Furthermore, the auto-brightness feature was disabled because the CT image review using an iPhone was performed in a bright ED, which is similar to emergency clinical settings. Screen protectors were not applied because they may obscure the image data.
Table 1.
The specifications of the two displays used in this study
| iPhone 5S | Clinical LCD monitor | |
|---|---|---|
| Type of display | LED-backlit IPS LCD (Retina) | Color LCD |
| Screen shape | 16:9 | 4:3 |
| Number of pixels | 1136 × 640 (326 PPI) | 1600 × 1200 |
| Luminance (max) (cd/m2) | 556 | 300 |
| Luminance used (cd/m2) | 556 | 170a |
| Display size, diagonal (cm) | 10.2 | 54 |
| Contrast ratio | 800:1 | 1000:1 |
| DICOM calibration | None | Calibrated |
aThe provided luminance of the DICOM calibration mode was fixed at 170 cd/m2
DICOM digital imaging and communications in medicine, LCD liquid crystal display, LED light-emitting diode
PACS Workstation with High-Resolution LCD Monitor in the ED
A desktop computer (DB405T2A, SAMSUNG, Korea) with an LCD monitor (MX210, EIZO, Japan), which had previously been used to view radiologic images in the ED, was used as a server to remotely transmit the CT images to the iPhone. Windows 7 and a DICOM viewer (PiViewerSTAR, Infinitt Healthcare, Korea) were installed on this computer.
The high-resolution LCD monitor, with 1600 × 1200 pixels and a diagonal size of 54 cm, was used to view the CT examination during rating. The detailed specifications are shown in Table 1. This monitor was calibrated with DICOM Grayscale Standard Display Function and quality control software (RadiCS, EIZO) and an External Sensor (UX1 Sensor, EIZO) [14]. The luminance of the DICOM mode of this clinical monitor was set to 170 cd/m2 by the manufacturer, which the American College of Radiology (ACR) has recommended as the minimum level of luminance for a clinical monitor.
Remote Accessing System
The free remote control program (TeamViewer 9, TeamViewer GmbH, Germany) was installed on the server side computer [15], and the application of the TeamViewer Remote Control for the iPhone was downloaded onto the iPhone [16], which enabled the iPhone to remotely link with the PACS workstation of the ED via the TeamViewer relay server. Therefore, out-of-ED physicians could remotely control the PACS workstation and observe the CT images transmitted from the PACS workstation using their own handheld iPhones. The image data from the server to the iPhone were protected by supporting the coded network communication [17]. All DICOM viewer functions such as windowing, zooming, panning, and using calipers were available on the iPhone. In this study, we used the 3G mobile network for remote connection.
Review of Abdominal CT Images
Sixteen board-certified emergency physicians with more than 5 years of experience with CT reading for acute appendicitis (maximum experience of 9 years) were recruited as raters and asked to review the 120 preselected CT examinations. The raters were randomly divided into two groups. At their first visit, the raters in group one reviewed the CT examinations numbered from 1 to 60 using an LCD monitor; after a mandatory 20 min of rest, they reviewed the remaining CT examinations numbered from 61 to 120 using an iPhone with TeamViewer. The other group first reviewed the CT scans numbered from 1 to 60 using an iPhone with TeamViewer and subsequently reviewed scans 61 to 120 using the LCD monitor. There was no time limit for the rating examination. The raters visited two times within an interval of 4 weeks and reviewed the CT scans using the reverse devices at each session.
With the exception of sex, age, and their chief complaint, the patients’ clinical information was not disclosed. The raters were instructed to rate the likelihood of the presence or absence of appendicitis in each case using a five-point Likert scale (1 = absence of appendicitis, 2 = probable absence of appendicitis, 3 = inconclusive, 4 = probable presence of appendicitis, and 5 = presence of appendicitis).
All raters viewed both displays directly perpendicular to the surface of the display because the LCD could not display a uniform gradient of the intensity values on the viewing angle [18]. The distance between the display and the reviewer’s eyes was set to more than 25 cm, which is considered the minimum viewing distance wherein the screen is perfectly sharp for the eyes of an individual with 20/20 vision [12]. The screen of the iPhone was wiped using a lens cloth prior to and during use, if necessary, because the touch screen is susceptible to dirt, including oil, which would obscure the image and reduce the contrast. The PACS workstation (server side computer) and the LCD monitor were located in the ED under suboptimal illumination (300 lx), which was measured with photometry. The remote review using an iPhone was conducted in the office, which had similar illumination as the ED. The download rate of the 3G network was evaluated with the smartphone application BENCHBEE at the beginning and end of the CT review [16].
Data Analysis
The descriptive data are expressed as the mean and 95 % confidence interval (CI). McNemar’s test was used to compare the sensitivity and specificity between the iPhone display and the LCD monitor of the desktop PC [19]. The sensitivity and specificity for detecting the appendix were calculated; ratings of three or fewer were negative, and the gold standard was the pathology report. The areas under the receiver operating characteristic (ROC) curves for each display device for all reviewers and the differences in each reviewer’s performance between the two devices were calculated according to the Dorfman-Berbaum-Metz method (DBM), using Obuchowski-Rockette and Dorfman-Berbaum-Metz multi-reader, multi-case (OR-DBM-MRMC) software (version 2.4) [9]. The inter- and intra-rater agreements between the two devices were analyzed using the kappa coefficient. The kappa value was regarded as follows: poor (<0.20), fair (0.21–0.40), moderate (0.41–0.60), good (0.61–0.80), and very good (0.81–1.00).
Results
The characteristics of the selected cases are shown in Table 2. The age range of the 120 patients was 19–81 years (mean age 37.8, CI 35.3–40.3). The mean download rate of the 3G network was 5.3 Mbps (CI 4.8–5.8, minimum 3.1 Mbps, and maximum 7.8 Mbps). The mean durations of single session reviews (60 exams) with the iPhone and LCD monitor were 79.3 min (CI 73.7–85.0) and 62.4 min (CI 57.1–67.6), respectively.
Table 2.
The characteristics of the selected cases
| Cases with appendicitis | Cases without appendicitis | |
|---|---|---|
| Sex, male, n (%) | 32 (53.3) | 28 (46.7) |
| Age, year, mean (CI) | 38.7 (34.7–42.7) | 36.9 (33.9–40.0) |
| Diagnosis, n (%) | ||
| Acute appendicitis | 60 (100) | 0 (0) |
| Nonspecific (nonremarkable finding) | 46 (76.7) | |
| Diverticulitis | 4 (6.7) | |
| Enterocolitis | 5 (8.3) | |
| Mesenteric lymphadenitis | 2 (3.3) | |
| Gynecologic problem | 3 (5) | |
CI 95 % confidence interval
The accuracy of diagnosing suspected appendicitis using the iPhone display and LCD monitor was high, and they were not significantly different. The overall sensitivity of the iPhone and LCD monitor for the diagnosis of appendicitis was exactly identical (0.91, CI 0.89–0.93), and the overall specificities of each device were 0.95 (CI 0.93–0.96) and 0.96 (0.94–0.97), respectively (Table 3).
Table 3.
Diagnostic accuracies of the iPhone and LCD monitor
| Raters | Sensitivity | p valuea | Specificity | p valuea | Time to complete a single session (min) | Mobile internet speed (Mbps, mean) | |||
|---|---|---|---|---|---|---|---|---|---|
| iPhone | LCD | iPhone | LCD | iPhone | LCD | ||||
| 1 | 0.93 | 0.93 | 1.000 | 0.95 | 0.97 | 0.317 | 67 | 52 | 4.8 |
| 2 | 0.85 | 0.95 | 0.034 | 0.85 | 0.97 | 0.035 | 72 | 64 | 6.4 |
| 3 | 0.82 | 0.92 | 0.014 | 0.95 | 0.95 | 1.000 | 60 | 46 | 5.2 |
| 4 | 0.88 | 0.92 | 0.317 | 0.98 | 0.97 | 0.564 | 85 | 72 | 7.8 |
| 5 | 0.85 | 0.82 | 0.480 | 0.93 | 0.93 | 1.000 | 76 | 75 | 3.1b |
| 6 | 0.93 | 0.93 | 1.000 | 0.97 | 0.97 | 1.000 | 90 | 59 | 5.9 |
| 7 | 0.97 | 0.87 | 0.034 | 0.95 | 0.95 | 1.000 | 95 | 50 | 5.5 |
| 8 | 0.92 | 0.95 | 0.480 | 0.92 | 0.92 | 1.000 | 88 | 72 | 6.1 |
| 9 | 0.93 | 0.97 | 0.317 | 0.93 | 0.93 | 1.000 | 100 | 76 | 3.7b |
| 10 | 0.93 | 0.92 | 0.564 | 0.97 | 0.97 | 1.000 | 76 | 52 | 7.6 |
| 11 | 0.93 | 0.87 | 0.046 | 0.97 | 0.98 | 0.564 | 81 | 65 | 4.9 |
| 12 | 0.88 | 0.85 | 0.480 | 0.97 | 1.00 | 0.157 | 75 | 62 | 4.6 |
| 13 | 0.93 | 0.92 | 0.564 | 0.98 | 0.97 | 0.317 | 80 | 68 | 3.3b |
| 14 | 0.93 | 0.88 | 0.180 | 0.92 | 0.95 | 0.414 | 69 | 49 | 5.1 |
| 15 | 0.93 | 0.97 | 0.317 | 0.93 | 0.93 | 1.000 | 72 | 70 | 4.1 |
| 16 | 0.93 | 0.92 | 0.564 | 0.97 | 0.97 | 1.000 | 83 | 66 | 7 |
| Overall, mean (CI) | 0.91 (0.89–0.93 | 0.91 (0.89–0.93) | 1.000 | 0.95 (0.93–0.96) | 0.96 (0.94–0.97) | 0.138 | 79.3 (73.7–85.0) | 62.4 (57.1–67.6) | 5.3 (4.8–5.8) |
LCD liquid crystal display, CI 95 % confidence interval
aCalculated by McNemar’s test
bMobile Internet speed was slower than 4 Mbps
The average areas under the ROC curve for all readers’ CT readings with the iPhone and LCD monitor were 0.978 and 0.974, respectively; there was no significant difference in the diagnostic performance between the two devices (P = 0.55). Each reader also demonstrated a similar diagnostic performance between the two devices (Table 4).
Table 4.
Comparison of the diagnostic performances of the iPhone and LCD monitor
| Raters | Area under the ROC curve | Difference between areas | p value | 95 % CI | |
|---|---|---|---|---|---|
| iPhone 5 | LCD | ||||
| 1 | 0.956 | 0.976 | −0.020 | 0.475 | −0.074, 0.035 |
| 2 | 0.955 | 0.983 | −0.027 | 0.143 | −0.064, 0.009 |
| 3 | 0.962 | 0.983 | −0.021 | 0.150 | −0.051, 0.008 |
| 4 | 0.987 | 0.964 | 0.024 | 0.473 | −0.041, 0.088 |
| 5 | 0.949 | 0.957 | −0.009 | 0.726 | −0.057, 0.040 |
| 6 | 0.989 | 0.986 | 0.003 | 0.475 | −0.006, 0.012 |
| 7 | 0.989 | 0.954 | 0.035 | 0.081 | −0.004, 0.075 |
| 8 | 0.981 | 0.987 | −0.006 | 0.541 | −0.026, 0.014 |
| 9 | 0.985 | 0.993 | −0.008 | 0.326 | −0.025, 0.008 |
| 10 | 0.992 | 0.978 | 0.014 | 0.573 | −0.036, 0.064 |
| 11 | 0.992 | 0.993 | −0.001 | 0.746 | −0.008, 0.006 |
| 12 | 0.960 | 0.921 | 0.039 | 0.403 | −0.053, 0.130 |
| 13 | 0.997 | 0.988 | 0.009 | 0.344 | −0.010, 0.027 |
| 14 | 0.981 | 0.950 | 0.031 | 0.479 | −0.055, 0.117 |
| 15 | 0.985 | 0.993 | −0.008 | 0.326 | −0.025, 0.008 |
| 16 | 0.992 | 0.982 | 0.009 | 0.668 | −0.034, 0.052 |
| Overall, mean (CI) | 0.978 (0.965, 0.991) | 0.974 (0.960, 0.988) | 0.004 | 0.552 | −0.009, 0.017 |
ROC receiver operating characteristics, LCD liquid crystal display, CI 95 % confidence interval
The inter-rater agreement for each device was very good; the kappa value was 0.809 (CI 0.793–0.826) for the smartphone and 0.817 (CI 0.801–0.834) for the LCD monitor. Each rater had moderate-to-very good intra-observer agreement between the two devices (Table 5).
Table 5.
Intra-rater agreement for the 16 participants
| Raters | Weighted kappa iPhone with LCD | p value | 95 % CI |
|---|---|---|---|
| 1 | 0.883 | <0.001 | 0.799, 0.967 |
| 2 | 0.683 | <0.001 | 0.553, 0.814 |
| 3 | 0.833 | <0.001 | 0.734, 0.931 |
| 4 | 0.883 | <0.001 | 0.799, 0.967 |
| 5 | 0.764 | <0.001 | 0.649, 0.880 |
| 6 | 1.000 | <0.001 | 1.000, 1.000 |
| 7 | 0.800 | <0.001 | 0.694, 0.907 |
| 8 | 0.767 | <0.001 | 0.652, 0.882 |
| 9 | 0.867 | <0.001 | 0.778, 0.956 |
| 10 | 0.950 | <0.001 | 0.894, 1.000 |
| 11 | 0.883 | <0.001 | 0.799, 0.967 |
| 12 | 0.831 | <0.001 | 0.731, 0.931 |
| 13 | 0.933 | <0.001 | 0.869, 0.998 |
| 14 | 0.817 | <0.001 | 0.714, 0.920 |
| 15 | 0.867 | <0.001 | 0.778, 0.956 |
| 16 | 0.950 | <0.001 | 0.894, 1.000 |
LCD liquid crystal display, CI confidence interval
Discussion
In this study, we verified the feasibility of a smartphone-based remote control system as a real-time remote CT reading tool for suspected appendicitis. Several previous reports have demonstrated that there is no difference in the diagnostic accuracy for CT readings between smartphones and typical PACS workstations in various diseases [7, 18, 20–22]. However, to the best of our knowledge, this is the first study to demonstrate the diagnostic “performance” of remote CT reading for acute appendicitis using a smartphone, which has good concordance with the performance of the PACS workstation LCD monitor under bright ambient lighting in a “realistic ED setting” and using “real-time data transmission” with a 3G network. Other remote viewing systems used in previous studies have been predominately designed as a “thin client,” which should download all applications and data from the server. Therefore, smartphones that use their own versions of the programming language require specific programs to use the DICOM server, indicating that there are additional costs. Furthermore, the downloading process is time-consuming, which would restrict the active use of this system in an emergency situation. However, the televiewing based on the free remote control system used in this study does not require extra costs and enables access to the DICOM viewer in real time.
In this study, the areas under the ROC curves for the raters were high for both the iPhone with the mobile network and the LCD monitor of the PACS workstation; the differences between the two devices were not significant. The diagnostic performance of all raters using both devices was similar, which may also indicate that all cases included were very easily diagnosed. Therefore, an evaluation of the differences might be difficult. However, we selected 120 consecutive cases during the study period, and it is almost certain that equivocal cases were included. The raters encountered challenging cases during the rating examination, which is similar to the real ED situation.
At first, the raters faced difficulties controlling the DICOM viewer through the iPhone, especially in the process of measuring the diameter of the appendix with calipers and scrolling the screen. However, they easily became accustomed to this procedure within 10 min of practice prior to the rating examination, and they did not encounter any major difficulties in controlling the DICOM viewer with the iPhone during rating. However, the mean time for one session (rating 60 examinations) with the iPhone was significantly longer compared with the PACS workstation (79.3 vs. 62.4 min, respectively, P < 0.001), which could be problematic in a real clinical situation if the experts outside of the ED who are requested to review the remote CT readings are not familiar with the iPhone-based DICOM viewer control.
There are several considerations with remotely viewing the radiologic images using a smartphone outside of the hospital. First, the patients’ information should be protected. The remote viewing system used by TeamViewer as a remote control system in this study has high security standards. The connections between the server side computer and the remote viewing display are established by fully encrypted data channels that use a 2048-bit RSA key exchange (which is one of the first executable public cryptosystems described by Ron Rivest, Adi Shamir, and Leonard Adleman) and a 256-bit Advanced Encryption Standard (AES) session, with encoding established by the US National Institute of Standards and Technology. The two-factor authentication adds an additional security layer to protect the account against unauthorized access [17]. Furthermore, this remote control system does not require storage of the patients’ medical records, including radiologic images, in the remote viewing display; therefore, patient information can be protected from leakage via the remote viewing display outside of the ED.
Second, the remote viewer can be used behind firewalls, blocked ports or network address translation (NAT) routing because most hospitals utilize these systems. The remote control system, including TeamViewer, can always find a route between the server and the remote display [17].
Third, the available mobile Internet network should be supported. The average mobile Internet speed that we used for the rating examination (5.32 Mbps) was slower compared with the South Korean average in the first quarter of 2014 reported by Akamai (14.7 Mbps); this lower speed occurred because we only used the 3G network, which has the lowest connection speed in Korea [23]. Of the 56 countries included in the Akamai report, 21 countries had an average connection speed higher than 4 Mbps, which was regarded as the “broadband” level mobile connectivity. We expect that the same good concordance with the results of this study could be observed in these countries. However, the remaining 35 countries had an average connection speed under 4 Mbps, and even in South Korea, with the highest average connection speed in the world, 25 % of all connections had a connection speed under 4 Mbps. Therefore, further studies at this low mobile internet speed are needed. In this study, three raters conducted the examinations at a speed slower than 4 Mbps (3.1, 3.3, and 3.7 Mbps), and their results had good concordance with the final results of this study.
Fourth, this smartphone-based remote viewing system should be used under a very wide range of ambient lighting conditions that are frequently much brighter than medical monitors. The ACR recommends that conventional CT reading must be performed under dim ambient lighting (20–40 lx) in the case of PACS workstation reviewing because bright ambient lighting can significantly reduce the display contrast, thereby reducing the viewer’s ability to perceive subtle image details [24]. However, in real clinical situations, emergency physicians frequently perform the CT reading under the general residential indoor lighting; thus, the smartphone-based televiewing should also be performed under bright ambient lighting. Therefore, this study was designed to investigate whether emergency physicians could have similar diagnostic performance and accuracy for detecting the presence of appendicitis on a smartphone under general residential indoor lighting (i.e., 100–500 lx) compared to those using the high-resolution LCD monitor of the PACS workstation under similar lighting. In this study, the illumination was set to 300 lx for both locations, including the examination location for tele-interpretation using the smartphone and the location of the PACS workstation in the ED. This study demonstrated the feasibility of the smartphone-based real-time remote CT reading under general residential indoor lighting. However, further studies regarding the feasibility of this practice under brighter ambient lighting that exceeds 300 lx are required because smartphones can be used under this brighter lighting.
Finally, the images transmitted to the smartphone through the real-time mobile network must have sufficient quality to be interpreted against the images on the LCD monitor. The iPhone 5 provides a similar image viewing quality compared to the clinical LCD monitor [7, 18, 20–22]. The iPhone 5 has sufficient pixel pitch to support the radiology images at their original size and sufficient brightness and contrast to meet the ACR guideline recommendations for medical monitors [24]. The difference in the image quality between the radiology images transmitted to the iPhone under the low-speed 3G network in real time and the original images of the high-resolution LCD monitor was not significant, given that there was no significant difference in the interpretative performance in the CT reading for acute appendicitis, which requires the perception of complicated, subtle, and detailed findings.
Maluccio et al. reported that the predictive ability of CT for acute appendicitis depends on the interpreters’ level of interpretative skill. The authors suggested that the initial CT interpretation by the resident did not correlate with the pathology result [5, 19]. Even residents with a low level of experience can easily diagnose appendicitis in cases in which patients exhibit typical signs, but they often face challenging patients with equivocal CT signs of appendicitis. Residents are likely to miss the diagnosis in these patients. Many Korean EDs do not have sufficiently skilled CT interpreters available 24 h/day; this may be similar worldwide. Therefore, some patients are likely to be evaluated on the basis of the preliminary interpretation of the residents. In fact, the final interpretation of these patients is often changed by the experts’ interpretations the subsequent morning [19]. Some patients may undergo unnecessary surgery, whereas other patients may fail to be diagnosed with appendicitis. If these remote viewing systems are available, the number of missed patients would decrease. Because CT reading for acute appendicitis requires the perception of complicated, subtle, and detailed findings including mesenteric infiltration and wall thickening, the feasibility of iPhone-based CT reading for acute appendicitis supports the potential use of this teleradiology system for other diseases.
There are several limitations in this study. Although the overall diagnostic accuracy and performance of iPhone-based CT reading was concordant with the conventional CT reading with an LCD monitor, some raters had difficulty in reviewing the CT image using the iPhone. Furthermore, the cases included in this study were only acute appendicitis cases. Therefore, this study could not confirm that the iPhone would be generally applicable as a remote CT reading tool in every clinical situation. This study also could not show that the diagnostic accuracy and performance of the iPhone for abdominal CT reading are similar to the 3-megapixel diagnostic monitor (high specification) that is usually used in radiology room because we only used the clinical monitor with a medium level of specification in this study. However, the diagnostic accuracy and performance of the iPhone were very high; thus, the difference between the diagnostic monitor and the iPhone would not be significant.
Conclusion
The diagnostic accuracy and performance regarding the diagnosis of acute appendicitis using remote- and real-time-transmitted CT images on a smartphone and original CT images on a high-resolution LCD monitor, which were examined under residential indoor lighting, were not significantly different. These findings may be particularly helpful in many clinical settings where on call experts are not always available to access the PACS workstation.
Acknowledgments
This work was supported by funding from Hanyang University (HY-2013-MC).
Contributors
CK designed, analyzed, and drafted the manuscript; BSK conceived the project and assisted with manuscript preparation and revision. HJC assisted with the study design and revision of the manuscript. JBP collected and analyzed the data and contributed to the revision of the manuscript.
Conflict of Interest
The authors have no conflicts of interest to disclose.
Ethics Approval
This study was approved by the Institutional Review Board of our institution (GURI 2014–02).
Registration
Clinical trials.gov NCT02108717
References
- 1.Cuschieri J, Florence M, Flum DR, et al. Negative appendectomy and imaging accuracy in the Washington State Surgical Care and Outcomes Assessment Program. Ann Surg. 2008;248:557–63. doi: 10.1097/SLA.0b013e318187aeca. [DOI] [PubMed] [Google Scholar]
- 2.Raman SS, Osuagwu FC, Kadell B, et al. Effect of CT on false positive diagnosis of appendicitis and perforation. N Engl J Med. 2008;358:972–3. doi: 10.1056/NEJMc0707000. [DOI] [PubMed] [Google Scholar]
- 3.Gaitini D, Beck-Razi N, Mor-Yosef D, Fischer D, Ben Itzhak O, Krausz MM, Engel A. Diagnosing acute appendicitis in adults: accuracy of color Doppler sonography and MDCT compared with surgery and clinical follow-up. AJR Am J Roentgenol. 2008;190(5):1300–6. doi: 10.2214/AJR.07.2955. [DOI] [PubMed] [Google Scholar]
- 4.Wijetunga R, Tan BS, Rouse JC, Bigg-Wither GW, Doust BD. Diagnostic accuracy of focused appendiceal CT in clinically equivocal cases of acute appendicitis. Radiology. 2001;221(3):747–53. doi: 10.1148/radiol.2213001581. [DOI] [PubMed] [Google Scholar]
- 5.Maluccio MA, Covey AM, Weyant MJ, et al. A prospective evaluation of the use of emergency department computed tomography for suspected acute appendicitis. Surg Infect (Larchmt) 2001;2(3):205–11. doi: 10.1089/109629601317202687. [DOI] [PubMed] [Google Scholar]
- 6.Choi HJ, Lee JH, Kang BS. Remote CT reading using an ultramobile PC and web-based remote viewing over a wireless network. J Telemed Telecare. 2012;18(1):26–31. doi: 10.1258/jtt.2011.110412. [DOI] [PubMed] [Google Scholar]
- 7.Choudhri AF, Carr TM, 3rd, Ho CP, et al. Handheld device review of abdominal CT for the evaluation of acute appendicitis. J Digit Imaging. 2012;25(4):492–6. doi: 10.1007/s10278-011-9431-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Park JB, Choi HJ, Lee JH, et al. An assessment of the iPad 2 as a CT teleradiology tool using brain CT with subtle intracranial hemorrhage under conventional illumination. J Digit Imaging. 2013;26(4):683–90. doi: 10.1007/s10278-013-9580-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Medical Image Perception Laboratory. Available at: http://perception.radiology.uiowa.edu. Accessed 21 May 2014
- 10.Hillis SL, Obuchowski NA, Berbaum KS. Power estimation for multireader ROC methods an updated and unified approach. Acad Radiol. 2011;18(2):129–42. doi: 10.1016/j.acra.2010.09.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.The Web’s Free 2014 ICD-10-CM and ICD-10-PCS Medical coding reference. Available at: http://www.icd10data.com/. Accessed 21 May 2014
- 12.Soneira RM. Flagship Smartphone Display Technology Shoot-Out; Apple iPhone 5 – Samsung Galaxy S III – Apple iPhone 4. Available at: http://www.displaymate.com/Smartphone_ShootOut_2.htm. Accessed 21 May 2014
- 13.Hayes JC. Monitor’s megapixels do not affect image interpretation. Available at http://www.diagnosticimaging.com/conference-reports/scar2005/article/113619/1198917. Accessed 21 May 2014
- 14.Digital imaging and communications in medicine (DICOM). Part 14. Grayscale display standard function. NEMA PS 3.14-2007. Rosslyn, VA: National Electrical Manufacturers Association (NEMA), 2006
- 15.Teamviewer. http://www.teamviewer.com. Accessed 21 May 2014
- 16.iTunes. http://www.apple.com/itunes. Accessed 21 May 2014
- 17.TeamViewer information for remote support and remote access. Available at http://www.teamviewer.com/en/products/security.aspx. Accessed 21 May 2014
- 18.Mitchell JR, Sharma P, Modi J, et al. A smartphone client–server teleradiology system for primary diagnosis of acute stroke. J Med Internet Res. 2011;13(2):e31. doi: 10.2196/jmir.1732. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Hawass NE. Comparing the sensitivities and specificities of two diagnostic procedures performed on the same group of patients. Br Radiol. 1997;70(832):360–366. doi: 10.1259/bjr.70.832.9166071. [DOI] [PubMed] [Google Scholar]
- 20.Modi J, Sharma P, Earl A, et al. iPhone-based teleradiology for the diagnosis of acute cervico-dorsal spine trauma. Can J Neurol Sci. 2010;37(6):849–54. doi: 10.1017/s0317167100051556. [DOI] [PubMed] [Google Scholar]
- 21.Choudhri AF, Radvany MG. Initial experience with a handheld device digital imaging and communications in medicine viewer: OsiriX mobile on the iPhone. J Digit Imaging. 2011;24(2):184–9. doi: 10.1007/s10278-010-9312-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Choudhri AF, Norton PT, Carr TM, 3rd, et al. Diagnosis and treatment planning of acute aortic emergencies using a handheld DICOM viewer. Emerg Radiol. 2013;20(4):267–72. doi: 10.1007/s10140-013-1118-8. [DOI] [PubMed] [Google Scholar]
- 23.The State of the Internet, 1st Quarter, 2014, Vol. 7, No. 1. Available at http://www.akamai.com/dl/akamai/akamai-soti-q114.pdf?WT.mc_id=soti_Q114. Accessed 21 May 2014
- 24.Norweck JT, Seibert JA, Andriole KP, et al. ACR-AAPM-SIIM technical standard for electronic practice of medical imaging. J Digit Imaging. 2013;26(1):38–52. doi: 10.1007/s10278-012-9522-2. [DOI] [PMC free article] [PubMed] [Google Scholar]