Abstract
Our aim was to prove the feasibility of the remote interpretation of real-time transmitted ultrasound videos of dynamic and static organs using a smartphone with control of the image quality given a limited internet connection speed. For this study, 100 cases of echocardiography videos (dynamic organ)—50 with an ejection fraction (EF) of ≥50 s and 50 with EF <50 %—and 100 cases of suspected pediatric appendicitis (static organ)—50 with signs of acute appendicitis and 50 with no findings of appendicitis—were consecutively selected. Twelve reviewers reviewed the original videos using the liquid crystal display (LCD) monitor of an ultrasound machine and using a smartphone, to which the images were transmitted from the ultrasound machine. The resolution of the transmitted echocardiography videos was reduced by approximately 20 % to increase the frame rate of transmission given the limited internet speed. The differences in diagnostic performance between the two devices when evaluating left ventricular (LV) systolic function by measuring the EF and when evaluating the presence of acute appendicitis were investigated using a five-point Likert scale. The average areas under the receiver operating characteristic curves for each reviewer’s interpretations using the LCD monitor and smartphone were respectively 0.968 (0.949–0.986) and 0.963 (0.945–0.982) (P = 0.548) for echocardiography and 0.972 (0.954–0.989) and 0.966 (0.947–0.984) (P = 0.175) for abdominal ultrasonography. We confirmed the feasibility of remotely interpreting ultrasound images using smartphones, specifically for evaluating LV function and diagnosing pediatric acute appendicitis; the images were transferred from the ultrasound machine using image quality-controlled telesonography.
Keywords: Telesonography, Clinical imaging viewing, Smartphone
Introduction
Ultrasonography is a widely used examination tool, and its use in the emergency department (ED) is becoming increasingly widespread, especially for evaluating unstable, time-sensitive patients with suspected acute coronary syndrome, cardiac tamponade, heart failure, etc. It is also widely used for diagnosing acute abdominal conditions such as acute appendicitis in radiation-sensitive patients [1, 2]. However, the use of this technique is limited by the availability of a trained sonographer.
Several studies have investigated the use of telesonography, in which ultrasound images are transferred to a remote display in real time to remedy a lack of on-site expert interpreters in under-resourced settings [3–15]. The term “telesonography” was first used in 1992 by Mendlowitz [3]. The authors of that study transformed ultrasound images into analog video signals using a broadcast-quality switched video service and transmitted them to networked locations via the telephone company network. However, in the process of analog-to-digital conversion, the data must be compressed, and thus, the resolution of both video and still images is degraded. Today, although images remain digital during every step of image transmission, systems that transmit compressed data at a degraded resolution have been actively investigated because the available bandwidth is often limited [4–13]. These studies have demonstrated that the perception of the image quality is not significantly affected by the degraded resolution because the resolution required for the perception of image quality and smoothness is low [4]. However, the perception of image quality may not be correlated with diagnostic accuracy.
Several recent studies have reported that the diagnostic accuracy also does not significantly differ between conventional ultrasound examinations and telesonography [7–13]; however, these investigations have been limited to simple cases, such as fluid collection and pneumothorax in trauma patients, which do not require high resolution or a high frame rate of transmission. Thus, these studies cannot validate the use of telesonography for evaluations of other organs that require the ability to perceive subtle, complex and detailed findings. Furthermore, previous studies have been predominantly limited to static organs or slowly moving organs [7–11]. Therefore, the general suitability of telesonography for use in evaluating dynamic organs has not been confirmed.
In the present study, we investigated the feasibility of telesonography in the management of patients undergoing echocardiography and abdominal ultrasound examinations, specifically for evaluating left ventricular function (involving a dynamic organ) and suspected pediatric acute appendicitis (involving a static organ), tasks that are frequently performed in the ED. We hypothesized that in the diagnosis of static organs such as an inflamed appendix, the imaging resolution should be more important than the frame rate, and thus, a slight lag might be acceptable during the interpretation of static organs, whereas the frame rate should be more important than the image quality for evaluations of dynamic functions, such as the determination of the cardiac ejection fraction (EF) [16]. Therefore, if a telesonographic system is used to evaluate the wall motion of the heart given a limited mobile internet speed, then the frames per second (FPS) can be increased by reducing the image quality (file size).
As high-speed broadband mobile internet networks, such as Long-Term Evolution (LTE) networks, have increasingly been established in many countries, including South Korea, mass data transmission using smartphones has become feasible. High-resolution videos with high frame rates of transmission can be freely transferred over LTE networks. However, because third generation (3G) mobile networks are more widely used than LTE networks in most countries and LTE network use is expensive, we investigated whether telesonography is feasible in a limited-bandwidth internet scenario such as a 3G network.
For this study, we developed a system to control the loss rate of single frames of ultrasound video while they are being transferred.
Methods and Materials
Study Design and Settings
This study was designed based on our previous studies [17–19]. The reviewers reviewed preselected original ultrasound videos using the liquid crystal display (LCD) monitor of an ultrasound machine and also reviewed the same videos on a smartphone display as they were transmitted in real time from the ultrasound machine using the telesystem described below in a crossover design.
This study was approved by our institutional review board.
Telesonographic System
The image transmission system (CubeView, Alpinion Medical Systems, Korea) was developed and implemented on a high-performance ultrasound machine (E-Cube 15, Alpinion Medical Systems, Korea) with a light-emitting diode (LED)-backlit thin-film transistor LCD monitor with 1366 × 768 pixels, a luminance of 250 candela (cd)/m2, and a diagonal size of 47.0 cm (LC185EXN, LG Display Co., Ltd., Korea). The CubeView system can continuously transfer the images displayed on the LCD monitor of the ultrasound machine to a smartphone display. It captures ultrasound images at a rate higher than 25 FPS. In this study, the captured images were transmitted to the server computer via a broadband fixed-line internet network, which had an internet speed of more than 100 Mbps, and were subsequently transferred to the smartphone over a 3G mobile network (Fig. 1). The quality of a single frame of video (resolution) could be adjusted between 60 % (lossy, JPEG file) and 100 % (lossless, PNG file). The transmitted data were encrypted, and their access was restricted via password protection.
Fig. 1.
Remote viewing system using a smartphone: ultrasound videos on the smartphone display transmitted from the ultrasound machine in real time
Remote Viewer
The iPhone 5S (iPhone 5S, Apple Inc., USA) was used as the remote viewer in this study. It has an LED-backlit in-plane switching LCD display with a small diagonal dimension of 10.2 cm and a high resolution (1136 × 640 pixels). Its maximum luminance is 556 cd/m2, and its luminance was fixed to 250 cd/m2 during the study. The smartphone application CubeView was downloaded to the smartphone from iTunes [20].
Study Participants and Case Selection
Based on our preliminary study [21], the numbers of image cases and readers were calculated using the Multi- and Single-Reader Sample Size Program for Diagnostic Studies (version 1.0) [22]. At least 12 readers and 100 videos in the case of echocardiography and 12 readers and 90 images in the case of abdominal ultrasound were found to be required for the detection of a 0.03 difference in the area under the curve (AUC) with a power of 0.80. For this study, 12 board-certified emergency physicians with more than 5 years of clinical experience in emergency echocardiography and abdominal ultrasonography and 20/20 corrected vision were recruited as readers (Table 1). The physicians who originally performed the scans were not included in the study. One hundred echocardiography videos for cases of suspected heart failure (dynamic organ) and 100 abdominal ultrasound static images and full-motion sonographic examinations (ultrasound sequence videos) for cases of suspected pediatric appendicitis (static organ), which were acquired between November 2013 and December 2014, were consecutively collected from the ultrasound machine (E-Cube 15, Alpinion Medical Systems, Korea). The full-motion sonographic examinations showing the appendix had been recorded during abdominal ultrasonography in the cases of suspected pediatric appendicitis.
Table 1.
Characteristics of reviewers
| Sex | Age | Certified board | Experience of ultrasonography, years | Corrective vision | Presence of astigmatism | Smartphone use, years | |
|---|---|---|---|---|---|---|---|
| Raters | |||||||
| 1 | Male | 36 | Emergency medicine | 5 | 20/20 | No | 5 |
| 2 | Male | 37 | Emergency medicine | 6 | 20/20 | No | 6 |
| 3 | Male | 34 | Emergency medicine | 5 | 20/20 | No | 4 |
| 4 | Male | 40 | Emergency medicine | 8 | 20/20 | No | 4 |
| 5 | Male | 41 | Emergency medicine | 6 | 20/20 | No | 4 |
| 6 | Male | 39 | Emergency medicine | 5 | 20/20 | No | 1 |
| 7 | Male | 32 | Emergency medicine | 5 | 20/20 | No | 3 |
| 8 | Male | 33 | Emergency medicine | 6 | 20/20 | No | 4 |
| 9 | Male | 38 | Emergency medicine | 5 | 20/20 | No | 4 |
| 10 | Male | 37 | Emergency medicine | 5 | 20/20 | No | 5 |
| 11 | Male | 40 | Emergency medicine | 6 | 20/20 | No | 6 |
| 12 | Male | 36 | Emergency medicine | 7 | 20/20 | No | 5 |
Of all echocardiography videos recorded for each case, only one video (parasternal long-axis view) was chosen, and the 8-s full-motion sonographic examinations and all still images related to the appendix (usually three to five images) were considered in the abdominal ultrasound cases. Based on the visual estimation of two blinded echocardiography experts, the echocardiography videos were categorized into two groups: cases with EF ≥ 50 % and cases with EF < 50 %. Cases with an EF less than 20 % and cases with specific ultrasonographic findings identifying them were excluded from this study. Cases without agreement (ambiguous cases) were categorized by consensus between the two experts. The inter-rater agreement was calculated using Cohen’s kappa statistic (k = 0.88). Based on the consensus of the two experts, we consecutively selected 50 cases from each group.
For the abdominal ultrasonography cases, 50 cases in which acute appendicitis was confirmed by the pathology report were consecutively selected, and an additional 50 cases without appendicitis, as verified by clinical follow-up (patients without a follow-up visit were verified via a telephone survey in January 2015), were also consecutively chosen.
Each case was randomly assigned a number between 1 and 100, and they were then arranged in numerical order. The patients’ information, except for sex and age, was deleted.
Intervention
The diagnostic performances in the evaluation of left ventricular systolic function for the echocardiography cases and in the evaluation of acute appendicitis for the abdominal ultrasound cases were investigated by the 12 reviewers. They were randomly divided into two groups and requested to visit twice (Fig. 2). On the first visit, six reviewers (group I) reviewed the original echocardiography videos numbered from 1 to 50 using the LCD monitor of an ultrasound machine and then reviewed the videos numbered from 51 to 100 using a smartphone, to which the videos were transmitted from the ultrasound machine. The resolutions of the transmitted echocardiography videos were reduced by approximately 20 % to increase the frame rate of transmission (Fig. 3). The remaining reviewers (group II) reviewed the videos numbered from 1 to 50 as transferred to the smartphone and also reviewed the remaining videos, 51 to 100, using the LCD monitor. After 30 min of mandatory rest, they reviewed the abdominal ultrasound images and videos in the same manner. We did not reduce the resolution of the abdominal ultrasound videos and images. Four weeks later, they visited again and reviewed the echocardiography videos and abdominal ultrasound images and videos using the opposite devices in each session. No time limit was placed on their review of the images.
Fig. 2.
Flow diagram of the study design. EC echocardiography, US ultrasonography, LCD liquid crystal display
Fig. 3.
Screenshot of original image on LCD monitor of ultrasound machine and the transmitted image on smartphone display: The resolution of a single frame of echocardiography video is reduced by approximately 20 %
They graded whether the EF was decreased (EF < 50 %) or the probability of the presence of acute appendicitis in each examination using a five-point Likert scale; for echocardiography, the scale was defined as 1 = normal EF, 2 = probably normal EF, 3 = inconclusive, 4 = probably decreased EF, and 5 = decreased EF, and for abdominal ultrasound, the scale was defined as 1 = absence of appendicitis, 2 = probable absence of appendicitis, 3 = inconclusive, 4 = probable presence of appendicitis, and 5 = presence of appendicitis. Before the first review, they met to review and discuss several sample images and videos that were not included in this study to enhance the diagnostic performance, accuracy, and inter-reviewer agreement.
After the review, the subjective image quality in both examinations was also measured using the double stimulus impairment scale (DSIS). The original videos on the LCD monitor (reference videos) and the videos transmitted to the smartphone were sequentially presented to the reviewers for comparison. The reference video was always presented before the transmitted video. Each video was presented for 8 s, with the original and transmitted videos separated by 3 s of inter-video blackout, followed by the next case after a 5-s interval (Fig. 4). The reviewers rated their level of annoyance while interpreting each transmitted video in comparison with the reference videos using a five-point Likert scale (1 = obviously annoying, 2 = annoying, 3 = slightly annoying, 4 = perceptible but not annoying, and 5 = imperceptible) before the next case was presented.
Fig. 4.
The process of video presentation for the double stimulus impairment scale (DSIS) evaluation; the original videos on the LCD monitor and the transmitted images on the smartphone were sequentially presented to the reviewers for 8 s, separated by 3-s intervals
The reviews were conducted in the ultrasound room of the ED under slightly dim ambient light (75–150 lux) in accordance with the recommendation of the American Association of Physicists in Medicine [23, 24]. The luminances of both devices (LCD and smartphone display) were set to 250 cd/m2.
The mobile internet speed of the 3G network was measured twice using the smartphone application BENCHBEE at the beginning and end of each examination [20].
Main Outcome and Data Analysis
The areas under the receiver operating characteristic (ROC) curves (diagnostic performance) for all reviewers for each device and each examination and the differences between the two devices were analyzed using the Obuchowski-Rockette and Dorfman-Berbaum-Metz methods for multi-reader, multi-case diagnostic studies (OR-DBM-MRMC software, version 2.4) [22].
The sensitivity and specificity for identifying decreased wall motion (EF < 50 %) and for identifying the presence of appendicitis were measured and compared between the LCD and smartphone displays for each examination using McNemar’s test [25]. A score of 3 or below was considered normal, and the gold standards for the echocardiography and abdominal ultrasound examinations were the experts’ determinations and the pathology reports, respectively.
The kappa coefficient value was used to measure the inter- and intra-rater agreements between the two devices on the following scale: poor (<0.20), fair (0.21–0.40), moderate (0.41–0.60), good (0.61–0.80), and very good (0.81–1.00).
The mean DSIS score was calculated. The video transmitted to the smartphone was considered to be not significantly different from the original video on the LCD monitor if the mean score was greater than four [16, 26].
Results
Of the 592 total ultrasound examinations performed in our ED from November 2013 to December 2014, 234 were echocardiographic examinations and 282 were abdominal ultrasound examinations. The characteristics of the selected cases are summarized in Table 2. Echocardiography was typically performed for aged individuals; the mean age was 58.5 (56.3–60.6). By contrast, abdominal ultrasound for the diagnosis of acute appendicitis was typically performed for pediatric patients (mean age 10.6, CI 10.0–11.2). The mean EF values were 55.7 % (54.3–57.1 %) in the group with normal EF and 37.2 % (35.5–38.9 %) in the group with impaired EF. The mean heart rates per minute were 90.7 (84.5–96.9) and 93.1 (87.7–95.5), respectively. The mean mobile connection speed was 5.54 Mbps (5.17–5.91) during the reviews.
Table 2.
The characteristics of the selected cases
| Echocardiography | Abdominal ultrasonography | |||
|---|---|---|---|---|
| Cases with normal EF (EF ≥ 50) | Cases with impaired EF (EF < 50) | Cases with appendicitis | Cases without appendicitis | |
| Sex, male, n (%) | 26 (52.0) | 24 (48.0) | 27 (54.0) | 26 (52.0) |
| Age, year, mean (CI) | 56.5 (53.5–59.5) | 60.4 (57.4–63.4) | 10.1 (9.3–10.8) | 11.1 (10.3–12.0) |
| EF, % (CI) | 55.7 (54.3–57.1) | 37.2 (35.5–38.9) | N/A | N/A |
| Heart rate, beat/min (CI) | 90.7 (84.5–96.9) | 93.1 (87.7–98.5) | N/A | N/A |
CI 95 % confidence interval, EF ejection fraction
The multi-reader, multi-case diagnostic tests were performed between January and March 2015 in a tertiary urban hospital. The average areas under the ROC curves for each reviewer’s interpretations using an LCD monitor and smartphone were respectively 0.968 (0.949–0.986) and 0.963 (0.945–0.982) for evaluations of dynamic cardiac function (echocardiography) and 0.972 (0.954–0.989) and 0.966 (0.947–0.984) for diagnoses of acute appendicitis (static organ) (Table 3). The diagnostic performance did not significantly differ between the two displays for either dynamic or static organs (p = 0.55 and p = 0.18, respectively).
Table 3.
Diagnostic performance comparison between the LCD monitor and the smartphone for evaluations of echocardiography and abdominal ultrasonography
| Rater | Area under the ROC curve | Difference in area (CI) | p | |
|---|---|---|---|---|
| LCD | Smartphone | |||
| Echocardiography | ||||
| 1 | 0.975 | 0.969 | 0.006 (−0.042–0.053) | 0.81 |
| 2 | 0.982 | 0.940 | 0.042 (−0.003–0.087) | 0.07 |
| 3 | 0.961 | 0.933 | 0.028 (−0.025–0.082) | 0.30 |
| 4 | 0.964 | 0.976 | −0.012 (−0.054–0.028) | 0.54 |
| 5 | 0.952 | 0.937 | 0.015 (−0.033–0.063) | 0.54 |
| 6 | 0.975 | 0.960 | 0.015 (0.000–0.030) | 0.06 |
| 7 | 0.936 | 0.974 | −0.038 (−0.09–0.018) | 0.18 |
| 8 | 0.984 | 0.974 | 0.010 (−0.011–0.032) | 0.32 |
| 9 | 0.990 | 0.972 | 0.018 (−0.007–0.042) | 0.16 |
| 10 | 0.957 | 0.987 | −0.030 (−0.063–0.004) | 0.09 |
| 11 | 0.987 | 0.986 | 0.001 (−0.006–0.008) | 0.78 |
| 12 | 0.947 | 0.952 | −0.005 (−0.061–0.051) | 0.87 |
| Overall, mean (CI) | 0.968 (0.949–0.986) | 0.963 (0.945–0.982) | 0.004 (−0.009–0.017) | 0.55 |
| Ultrasonography | ||||
| 1 | 0.983 | 0.973 | 0.010 (−0.025–0.044) | 0.59 |
| 2 | 0.982 | 0.985 | −0.003 (−0.031–0.024) | 0.82 |
| 3 | 0.980 | 0.948 | 0.032 (−0.001–0.064) | 0.06 |
| 4 | 0.965 | 0.976 | −0.011 (−0.050–0.029) | 0.61 |
| 5 | 0.979 | 0.975 | 0.004 (−0.016–0.025) | 0.64 |
| 6 | 0.990 | 0.969 | 0.021 (0.001–0.042) | 0.045 |
| 7 | 0.962 | 0.973 | −0.011 (−0.044–0.021) | 0.49 |
| 8 | 0.980 | 0.971 | 0.009 (−0.011–0.028) | 0.38 |
| 9 | 0.975 | 0.950 | 0.025 (−0.006–0.055) | 0.11 |
| 10 | 0.941 | 0939 | 0.002 (−0.028–0.032) | 0.92 |
| 11 | 0.985 | 0.960 | 0.025 (−0.004–0.054) | 0.09 |
| 12 | 0.942 | 0.968 | −0.026 (−0.067–0.015) | 0.21 |
| Overall, mean (CI) | 0.972 (0.954–0.989) | 0.966 (0.947–0.984) | 0.006 (−0.003–0.016) | 0.18 |
ROC receiver operating characteristics, LCD liquid crystal display, CI 95 % confidence interval
The sensitivities and specificities for each rater using the LCD monitor and the smartphone are presented in Table 4. In both evaluations (the cardiac dynamic function evaluations and the diagnoses of acute appendicitis), the diagnostic accuracies (sensitivities and specificities) of all raters were not significantly different between the two devices (all p values >0.05).
Table 4.
Diagnostic accuracies of echocardiography and ultrasonography using an LCD monitor and a smartphone
| Raters | Sensitivity | Specificity | p a | Mobile internet speed, Mbps, mean | DSIS, mean | ||
|---|---|---|---|---|---|---|---|
| LCD | Smartphone | LCD | Smartphone | ||||
| Echocardiography | |||||||
| 1 | 0.92 | 0.94 | 0.98 | 0.96 | 0.72 | 6.38 | 4.10 |
| 2 | 0.91 | 0.82 | 1.00 | 0.98 | 0.18 | 7.71 | 4.38 |
| 3 | 0.92 | 0.84 | 0.94 | 0.93 | 0.18 | 5.01 | 4.40 |
| 4 | 0.92 | 0.89 | 0.96 | 0.98 | 0.45 | 4.95 | 4.34 |
| 5 | 0.85 | 0.81 | 1.00 | 0.93 | 1.00 | 5.25 | 4.38 |
| 6 | 0.94 | 0.90 | 0.96 | 0.96 | 1.00 | 5.02 | 4.50 |
| 7 | 0.86 | 0.89 | 0.95 | 0.96 | 0.72 | 5.16 | 4.50 |
| 8 | 0.94 | 0.92 | 0.92 | 0.92 | 1.00 | 5.24 | 4.50 |
| 9 | 0.96 | 0.89 | 1.00 | 0.98 | 0.45 | 5.62 | 4.32 |
| 10 | 0.92 | 0.94 | 0.98 | 0.98 | 1.00 | 5.11 | 4.40 |
| 11 | 0.88 | 0.92 | 0.98 | 0.98 | 0.37 | 4.86 | 4.34 |
| 12 | 0.88 | 0.91 | 1.00 | 0.96 | 0.34 | 5.01 | 4.24 |
| Overall, mean (CI) | 5.44 (4.92–5.97) | 4.37 (4.31–4.43) | |||||
| Ultrasonography | |||||||
| 1 | 0.94 | 0.92 | 0.96 | 0.96 | 1.00 | 6.82 | 4.38 |
| 2 | 0.94 | 0.91 | 1.00 | 1.00 | 0.48 | 7.92 | 4.62 |
| 3 | 0.94 | 0.86 | 0.96 | 0.95 | 0.13 | 4.32 | 4.50 |
| 4 | 0.90 | 0.89 | 0.94 | 0.96 | 0.68 | 5.26 | 4.48 |
| 5 | 0.94 | 0.90 | 0.90 | 0.92 | 0.45 | 5.12 | 4.52 |
| 6 | 0.92 | 0.91 | 0.98 | 0.96 | 1.00 | 5.31 | 4.60 |
| 7 | 0.91 | 0.92 | 0.96 | 0.96 | 1.00 | 5.36 | 4.66 |
| 8 | 0.89 | 0.91 | 0.96 | 0.96 | 1.00 | 5.55 | 4.70 |
| 9 | 0.89 | 0.89 | 0.98 | 0.96 | 1.00 | 6.15 | 4.48 |
| 10 | 0.89 | 0.92 | 0.94 | 0.92 | 0.25 | 5.45 | 4.56 |
| 11 | 0.94 | 0.89 | 0.96 | 0.96 | 0.25 | 5.12 | 4.54 |
| 12 | 0.91 | 0.87 | 0.96 | 0.96 | 0.48 | 5.24 | 4.36 |
| Overall, mean (CI) | 5.64 (5.04–6.23) | 4.53 (4.49–4.58) | |||||
LCD liquid crystal display, DSIS double stimulus impairment scale, CI 95 % confidence interval
aCalculated using McNemar’s test
The inter-rater agreements for both devices in the echocardiographic evaluations were very good; the kappa values were 0.81 (0.71–0.90) for the LCD monitor and 0.80 (0.69–0.89) for the smartphone. In the diagnoses of acute appendicitis, they were 0.84 (0.75–0.93) for the LCD monitor and 0.82 (0.73–0.92) for the smartphone. Each rater exhibited good to very good intra-observer agreement between the two devices in both examinations (Table 5).
Table 5.
Intra-rater agreement for each of the 12 reviewers
| Raters | Echocardiography | Abdominal ultrasonography | ||||
|---|---|---|---|---|---|---|
| Weighted kappa, smartphone, and LCD | 95 % CI | p | Weighted kappa, smartphone, and LCD | 95 % CI | p | |
| 1 | 0.840 | 0.733–0.946 | <0.001 | 0.900 | 0.814–0.985 | <0.001 |
| 2 | 0816 | 0.702–0.931 | <0.001 | 0.960 | 0.905–1.000 | <0.001 |
| 3 | 0.820 | 0.707–0.932 | <0.001 | 0.860 | 0.759–0.960 | <0.001 |
| 4 | 0.859 | 0.759–0.960 | <0.001 | 0.880 | 0.786–0.973 | <0.001 |
| 5 | 0.773 | 0.647–0.900 | <0.001 | 0.860 | 0.760–0.960 | <0.001 |
| 6 | 0.960 | 0.905–1.000 | <0.001 | 0.920 | 0.843–0.997 | <0.001 |
| 7 | 0.838 | 0.731–0.946 | <0.001 | 0.940 | 0.873–1.000 | <0.001 |
| 8 | 0.820 | 0.708–0.932 | <0.001 | 0.900 | 0.814–0.985 | <0.001 |
| 9 | 0.859 | 0.759–0.960 | <0.001 | 0.859 | 0.758–0.960 | <0.001 |
| 10 | 0.940 | 0.873–1.007 | <0.001 | 0.940 | 0.873–1.000 | <0.001 |
| 11 | 0.899 | 0.813–0.985 | <0.001 | 0.940 | 0.873–1.000 | <0.001 |
| 12 | 0.798 | 0.680–0.917 | <0.001 | 0.960 | 0.905–1.000 | <0.001 |
LCD liquid crystal display, CI confidence interval
The mean DSIS value was 4.37 (4.31–4.43) for the image quality comparison of the transmitted echocardiography videos on the smartphone (resolution reduced by approximately 20 %) with the original videos on the LCD. The mean DSIS value for the comparison between the transmitted abdominal ultrasonography videos on the smartphone and the original videos on the LCD was 4.53 (4.49–4.58) (Table 4).
Discussion
We confirmed the feasibility of the real-time tele-interpretation of ultrasonography, specifically echocardiography for the evaluation of cardiac wall motion and abdominal ultrasonography for the diagnosis of pediatric appendicitis, using an image quality-controlled telesonographic system over a 3G mobile network. The diagnostic performance achieved using the Dorfman-Berbaum-Metz or Obuchowski-Rockette method for multi-reader ROC studies, the diagnostic accuracy, and the image quality were all assessed in this study [27]. The cited approach is a well-established method for evaluating performance in diagnostic tests in radiological imaging studies by analyzing the ROC curves. In this method, the readers assign disease-likelihood ratings using either a discrete or a quasi-continuous scale. In this study, the readers assigned their ratings using a five-point Likert scale. The areas under the ROC curves were high for both echocardiography and ultrasonography and for both smartphone-based telesonography and conventional ultrasonography using an LCD monitor. The overall differences between the two methods were not significant, indicating that the diagnostic performances achieved by the readers using both methods were similar. However, the diagnostic performance achieved by reader 6 using the smartphone was significantly lower than that achieved by the same reader using the LCD monitor in the abdominal ultrasonography cases (p = 0.045). Given that his performance in the echocardiography cases using the smartphone was also lower than that achieved using the LCD monitor, although this difference was not statistically significant (p = 0.06), and that every other reader demonstrated similar performance between the two devices, we assume that he was the only reader who was slightly unfamiliar with operating a smartphone and interpreting radiological images using a smartphone display. Indeed, his length of experience with smartphone use was quite short (less than 1 year).
Although several studies have evaluated the feasibility of telesonography [3–15], to our knowledge, this is the first study of the use of telesonography to evaluate cardiac dynamic function or to diagnose acute appendicitis with deliberate control of the image quality. When the speed of the mobile network is limited, one must optimize for the tradeoff between image quality and frame rate.
The diagnosis of acute appendicitis sometimes requires the ability to perceive subtle findings, such as the presence of periappendiceal fat infiltration. The most commonly used traditional criterion for diagnosing acute appendicitis on ultrasound is a dilated diameter of the appendix, with an outer diameter greater than 6 mm [28]. However, when this criterion is used, the negative appendectomy rate is relatively high (5 to 10 %) in pediatric patients because there are several normal variants of the appendix with an outer diameter greater than 6 mm in this population [29, 30]. The presence of periappendiceal fat infiltration is a more specific criterion for acute appendicitis [31–33]. In certain cases, the finding of echogenic fat infiltration could be ambiguous; therefore, images with high spatial resolution and without pixelation may be needed to perceive this subtle finding.
The heart is the most dynamic organ in the human body; therefore, a high frame rate of transmission may be necessary for evaluating the dynamic function of the heart. Humans typically perceive continuous images at 15 FPS as a continuous video without interruptions in motion. Agboma et al. have suggested that at least 12 FPS is required for action movies [16]. Although a higher frame rate may be necessary to display the highly dynamic motion in media such as sports games, videos at approximately 15 FPS may be sufficient to be perceived as continuous video in ultrasound examinations of dynamic organs such as the heart.
In this study, a single captured image typically required 150 kilobytes (KB) (110–140). An internet speed of 8 bits per second (BPS) is needed to transmit 1 KB of data within 1 s. Thus, theoretically, an internet speed of at least 18 Mbps should be required to transmit 150 KB of data at a frame rate greater than 15 FPS. The Ministry of Science, ICT and Future Planning of South Korea has reported that the average LTE mobile connection speed in Korea was 77.8 Mbps during the fourth quarter of 2014 [34]. Thus, such an LTE network is sufficient to transfer continuous high-resolution videos to handheld devices without lag, pixilation, or any other degradation. However, this study was conducted on a 3G mobile network because 3G is more widely used than LTE in most countries. The average 3G connection speed was approximately 5.1 Mbps in South Korea in January 2015 [34] and was 5.54 Mbps in this study.
The frame rate of transmission can reach approximately 4 FPS when 150 KB of high-resolution image data is transmitted losslessly over a connection with a speed of approximately 5 Mbps, and in this study, this speed was found to be sufficient for the interpretation of sequence videos for static organs. Although the transmitted images of static organs such as the appendix are typically static, real-time observations of full-motion ultrasound examinations (ultrasound sequence videos) might aid in their interpretation. These sequence videos for static organs may also not require high frame rates of transmission.
In the case of echocardiography, the image quality of each single frame of transmitted video was reduced to achieve a frame rate of at least 15 FPS. Reducing the spatial resolution of a single frame by 20 % decreased the file size of that frame from approximately 150 KB to approximately 40 KB, which could be transmitted at a frame rate of 15 FPS over a connection with a speed of approximately 5 Mbps. The echocardiography videos with this reduced image quality (80 % of the quality of the original image) could be effectively interpreted for evaluations of cardiac dynamic function when they were transmitted at a frame rate of approximately 15 FPS. Although the DSIS values for echocardiography were slightly lower than those for abdominal ultrasonography (probably because of the reduction in spatial resolution applied in the echocardiography cases), the mean DSIS value for echocardiography remained greater than 4, indicating that the image quality of the echocardiography videos transmitted to the smartphone was not significantly different from that of the original videos on the LCD monitor. The fastest heart rate among the patients included in this study was 135 beats per minute (beats/m). Therefore, this study cannot confirm that echocardiography videos can be transmitted without any significant break or lag at a frame rate of 15 FPS if the heart rate is faster than 135 beats/m.
Our tele-ultrasonography system continuously captures the images displayed on the monitor of an ultrasound machine and transmits them to a server computer. High-speed mobile internet has already been established in several countries, such as South Korea, Turkey, and the UK. Since the telecom company began LTE service in South Korea in July 2011, the use of the LTE network has been rapidly increasing, and approximately 90 % of mobile internet access was estimated to occur over the LTE network in January 2015 [34]. Therefore, high-quality ultrasound sequence videos can be transmitted at high frame rates using this telesonographic system in South Korea. Furthermore, background video with a voice calling capability could be transmitted simultaneously over this high-speed mobile network. In this study, we have already confirmed the feasibility of remote interpretation for left ventricular EF evaluations and diagnoses of the presence of pediatric acute appendicitis using a smartphone. Further studies of remote experts guiding inexperienced or novice practitioners in examination tasks, such as finding an appendix, using this telesonographic system are being conducted. If such complex and detailed guidance is feasible, then this system could be used in several under-resourced clinical settings.
Of the 50 countries considered in the Akamai report [35], 30 countries had an average connection speed higher than 4 Mbps (4 Mbps is regarded as the “broadband” level of mobile connectivity). It is believed that similarly good performance commensurate with the results of this study could be achieved in those countries.
Acknowledgments
We thank Alpinion Medical Systems, which developed the tele-ultrasonography system in collaboration with us. Author CK and HC contributed equally.
Abbreviations
- EF
Ejection fraction
- DSIS
Double stimulation impairment scale
- ROC
Receiver operating characteristic
- FPS
Frames per second
- OR-DBM-MRMC
Obuchowski-Rockette and Dorfman-Berbaum-Metz software for diagnostic studies of multiple readers and multiple cases
- KB
Kilobytes
- BPS
Bits per second
Compliance with Ethical Standards
Conflicts of Interest
The authors have no conflicts of interest to disclose.
Study Approval
This study was approved by the institutional review board of our institution.
Funding Source
No external funding was secured for this study.
Financial Disclosure
The authors have no financial relationships relevant to this article to disclose.
Footnotes
Trial registration: Clinical trials.gov NCT02271048
References
- 1.Lee HK, Kang BS, Kim C, Choi HJ. Ultrasound-guided regional anesthesia for the pain management of elderly patients with hip fractures in the emergency department. Clin Exp Emerg Med. 2014;1(1):49–55. doi: 10.15441/ceem.14.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Ahn C, Kim C, Kang B, Choi HJ, Cho JH. Variation of availability and frequency of emergency physician-performed ultrasonography between adult and pediatric patients in the academic emergency department in Korea. Clin Exp Emerg Med. 2015;2(1):16–23. doi: 10.15441/ceem.14.020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Mendlowitz AD, Young DK. A system for full-motion real-time telesonography. AJR Am J Roentgenol. 1992;159(5):1123–1124. doi: 10.2214/ajr.159.5.1414788. [DOI] [PubMed] [Google Scholar]
- 4.Paulus YM, Thompson NP. Inexpensive, realtime tele-ultrasound using a commercial, web-based video streaming device. J Telemed Telecare. 2012;18(4):185–188. doi: 10.1258/jtt.2012.110112. [DOI] [PubMed] [Google Scholar]
- 5.Popov V, Popov D, Kacar I, Harris RD. The feasibility of real-time transmission of sonographic images from a remote location over low-bandwidth Internet links: a pilot study. AJR Am J Roentgenol. 2007;188(3):W219–W222. doi: 10.2214/AJR.05.2148. [DOI] [PubMed] [Google Scholar]
- 6.Adambounou K, Farin F, Boucher A, Adjenou KV, Gbeassor M, N’dakena K, Vincent N, Arbeille P. System of telesonography with synchronous teleconsultations and asynchronous telediagnoses (Togo) Med Sante Trop. 2012;22(1):54–60. doi: 10.1684/mst.2012.0002. [DOI] [PubMed] [Google Scholar]
- 7.Ogedegbe C, Morchel H, Hazelwood V, Chaplin WF, Feldman J. Development and evaluation of a novel, real time mobile telesonography system in management of patients with abdominal trauma: study protocol. BMC Emerg Med. 2012;12:19. doi: 10.1186/1471-227X-12-19. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.McBeth P, Crawford I, Tiruta C, Xiao Z, Zhu GQ, Shuster M, Sewell L, Panebianco N, Lautner D, Nicolaou S, Ball CG, Blaivas M, Dente CJ, Wyrzykowski AD, Kirkpatrick AW. Help is in your pocket: The potential accuracy of smartphone- and laptop-based remotely guided resuscitative telesonography. Telemed J E Health. 2013;19(12):924–930. doi: 10.1089/tmj.2013.0034. [DOI] [PubMed] [Google Scholar]
- 9.Biegler N, McBeth PB, Tiruta C, Hamilton DR, Xiao Z, Crawford I, Tevez-Molina M, Miletic N, Ball CG, Pian L, Kirkpatrick AW. The feasibility of nurse practitioner-performed, telementored lung telesonography with remote physician guidance—‘a remote virtual mentor’. Crit Ultrasound J. 2013;5(1):5. doi: 10.1186/2036-7902-5-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.McBeth PB, Crawford I, Blaivas M, Hamilton T, Musselwhite K, Panebianco N, Melniker L, Ball CG, Gargani L, Gherdovich C, Kirkpatrick AW. Simple, almost anywhere, with almost anyone: remote low-cost telementored resuscitative lung ultrasound. J Trauma. 2011;71(6):1528–1535. doi: 10.1097/TA.0b013e318232cca7. [DOI] [PubMed] [Google Scholar]
- 11.McBeth PB, Hamilton T, Kirkpatrick AW. Cost-effective remote iPhone-teathered telementored trauma telesonography. J Trauma. 2010;69(6):1597–1599. doi: 10.1097/TA.0b013e3181e61ea9. [DOI] [PubMed] [Google Scholar]
- 12.Martinov D, Popov V, Ignjatov Z, Harris RD. Image quality in real-time teleultrasound of infant hip exam over low-bandwidth internet links: a transatlantic feasibility study. J Digit Imaging. 2013;26(2):209–216. doi: 10.1007/s10278-012-9512-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Arbeille P, Fornage B, Boucher A, Ruiz J, Georgescu M, Blouin J, Cristea J, Carles G, Farin F, Vincent N. Telesonography: virtual 3D image processing of remotely acquired abdominal, vascular, and fetal sonograms. J Clin Ultrasound. 2014;42(2):67–73. doi: 10.1002/jcu.22093. [DOI] [PubMed] [Google Scholar]
- 14.Sable CA, Cummings SD, Pearson GD, Schratz LM, Cross RC, Quivers ES, Rudra H, Martin GR. Impact of telemedicine on the practice of pediatric cardiology in community hospitals. Pediatrics. 2002;109(1):E3. doi: 10.1542/peds.109.1.e3. [DOI] [PubMed] [Google Scholar]
- 15.Costa C, Oliveira JL. Telecardiology through ubiquitous Internet services. Int J Med Inform. 2012;81(9):612–621. doi: 10.1016/j.ijmedinf.2012.05.011. [DOI] [PubMed] [Google Scholar]
- 16.Agboma F, Liotta A. Quality of experience management in mobile content delivery system. Telecommun Syst. 2012;49(1):85–98. doi: 10.1007/s11235-010-9355-6. [DOI] [Google Scholar]
- 17.Choi HJ, Lee JH, Kang BS. Remote CT reading using an ultramobile PC and web-based remote viewing over a wireless network. J Telemed Telecare. 2012;18(1):26–31. doi: 10.1258/jtt.2011.110412. [DOI] [PubMed] [Google Scholar]
- 18.Park JB, Choi HJ, Lee JH, Kang BS. An assessment of the iPad 2 as a CT teleradiology tool using brain CT with subtle intracranial hemorrhage under conventional illumination. J Digit Imaging. 2013;26(4):683–690. doi: 10.1007/s10278-013-9580-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Kim C, Kang B, Choi HJ, Park JB: A feasibility study of real-time remote CT reading for suspected acute appendicitis using an iPhone. J Digit Imaging. 2015 Feb 21 [DOI] [PMC free article] [PubMed]
- 20.iTunes. http://www.apple.com/itunes. Accessed April 7 2015. Archived by WebCite® at http://www.webcitation.org/6XcqJOUMs
- 21.Kim C, Kang B, LEE J, Choi HJ: The feasibility and effectiveness study for clinical application of realtime emergency tele-ultrasonography using the LTE smartphone—Pilot study. Presented at: Pan-Pacific Emerg Med Congress. 2014 Oct 14
- 22.Medical Image Perception Laboratory. http://perception.radiology.uiowa.edu. Accessed April 7 2015. Archived by WebCite® at http://www.webcitation.org/6Xcs81hxF
- 23.Chawla AS, Samei E. Ambient illumination revisited: a new adaptation-based approach for optimizing medical imaging reading environments. Med Phys. 2007;34(1):81–90. doi: 10.1118/1.2402583. [DOI] [PubMed] [Google Scholar]
- 24.Norweck JT, Seibert JA, Andriole KP, Clunie DA, Curran BH, Flynn MJ, Krupinski E, Lieto RP, Peck DJ, Mian TA. ACR-AAPM-SIIM technical standard for electronic practice of medical imaging. J Digit Imaging. 2013;26(1):38–52. doi: 10.1007/s10278-012-9522-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Hawass NE. Comparing the sensitivities and specificities of two diagnostic procedures performed on the same group of patients. Br Radiol. 1997;70(832):360–366. doi: 10.1259/bjr.70.832.9166071. [DOI] [PubMed] [Google Scholar]
- 26.Huynh-Thu Q, Ghanbari M: Temporal aspect of perceived quality in mobile video broadcasting. IEEE Tran. Broadcast. 2008 Sep;54(3)
- 27.Hillis SL, Obuchowski NA, Berbaum KS. Power estimation for multireader ROC methods an updated and unified approach. Acad Radiol. 2011;18(2):129–142. doi: 10.1016/j.acra.2010.09.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Jeffrey RB, Jr, Laing FC, Townsend RR. Acute appendicitis: sonographic criteria based on 250 cases. Radiology. 1988;167:327–329. doi: 10.1148/radiology.167.2.3282253. [DOI] [PubMed] [Google Scholar]
- 29.Krishnamoorthi R, Ramarajan N, Wang NE, et al. Effectiveness of a staged US and CT protocol for the diagnosis of pediatric appendicitis: reducing radiation exposure in the age of ALARA. Radiology. 2011;259(1):231–239. doi: 10.1148/radiol.10100984. [DOI] [PubMed] [Google Scholar]
- 30.Karakas SP, Guelfguat M, Leonidas JC, et al. Acute appendicitis in children: comparison of clinical diagnosis with ultrasound and CT imaging. Pediatr Radiol. 2000;30(2):94–98. doi: 10.1007/s002470050023. [DOI] [PubMed] [Google Scholar]
- 31.Kim C, Kang B, Park JB, Ha YR: The use of clinician-performed ultrasonography to determine the treatment method for suspected paediatric appendicitis. Hong Kong J Emerg Med. 2015 Jan;22(1):31–40. Availability: http://www.hkjem.com/sites/default/files/p31-40.pdf. Accessed April 7 2015. Archived by WebCite® at http://www.webcitation.org/6XcsGbO6G
- 32.Wiersma F, Toorenvliet BR, Bloem JL, et al. US examination of the appendix in children with suspected appendicitis: the additional value of secondary signs. Eur Radiol. 2009;19(2):455–461. doi: 10.1007/s00330-008-1176-6. [DOI] [PubMed] [Google Scholar]
- 33.Trout AT, Sanchez R, Ladino-Torres MF. Reevaluating the sonographic criteria for acute appendicitis in children: a review of the literature and a retrospective analysis of 246 cases. Acad Radiol. 2012;19(11):1382–1394. doi: 10.1016/j.acra.2012.06.014. [DOI] [PubMed] [Google Scholar]
- 34.The ministry of science, ICT and future planning. http://www.smartchoice.or.kr/smc/smartreport/evaluateDownload.do. Accessed April 7 2015. Archived by WebCite® at http://www.webcitation.org/6XcsNPQ1r
- 35.The State of the Internet, 4th Quarter, 2014; 7(4). http://www.akamai.com/dl/content/q4-2014-soti-report.pdf. Accessed April 7 2015. Archived by WebCite® at http://www.webcitation.org/6XcspGrey