Diagnostic accuracy and reliability of smartphone captured radiologic images communicated via WhatsApp®

Abstract

Introduction

Sending radiographic images as instant messages have become a common means of communication between physicians, aiding in triaging and transfer decision-making in emergencies. While use of technology is increasing, this is not the case for the underserved or rural areas of South Africa with no picture archiving and communications system or advanced hardware in place. In these areas, the medical staffing population have nearly universal access to smartphones and could benefit from the ability to share images quickly and easily with trained radiologists. South African data on diagnostic reliability of smartphone captured radiology images is lacking. The objective of the study was to determine the accuracy and reliability of diagnoses made on radiologic images captured with smartphone compared to radiologic images on picture archiving and communication system(PACS).

Methods

A cross-sectional study was conducted with radiographs from June 2018 to July 2019 selected from the PACS system at Pelonomi Tertiary Hospital. Images were displayed on PACS computer screen and captured by principal researcher using a smartphone. Five radiology registrars received the images via WhatsApp® and reviewed them on smartphones. After three weeks, registrars viewed images in random order on PACS stations. McNemar's test was used to compare the diagnostic accuracy of smartphone vs PACS and Kappa values calculated for agreement. Reliability was assessed by analysing the results of different registrars and diagnoses separately.

Results

135 X-rays, representative of common emergency conditions, were selected. For all registrars, PACS accuracy was generally higher than smartphone accuracy. The Kappa values all indicated fair to moderate agreement between smartphone and PACS diagnosis.

Conclusion

Capturing radiographic images using at least 12-megapixel smartphone and sharing them via WhatsApp® is a reliable method that can be used with a high degree of confidence in emergencies to aid clinical decision making. This method of viewing medical imaging is however not a substitution for images viewed on PACS.

African relevance

• •Sending radiographic images as instant messages have become a common means of communication between physicians
• •Radiology support in especially smaller district hospitals in low- and middle income countries is lacking
• •The use of smartphone-based WhatsApp® imaging for sending images can be used with high degree of confidence in diagnosis, decision making and management

Introduction

Smartphones are inexpensive, user-friendly, readily available and the camera quality has improved dramatically 1., 2.. Smartphones are rapidly becoming an essential part of modern society, changing the portability of information with functionalities such as high-quality phone cameras, internet access and access to third-party services 3. They also have the potential to positively impact healthcare with easier access to telemedicine 3. Smartphones have the potential to change the availability and access to specialist healthcare, especially in rural and remote communities 3., 4..

Teleradiology is a branch of telemedicine where telecommunication systems are used for the transmission of radiological images from one location to another for the purpose of sharing studies with other healthcare professionals including radiologists or physicians. Teleradiology is a highly evaluated and widely used method despite its high costs and technical complexity 5. Worldwide, teleradiology is increasingly recognised as an invaluable tool, due to the lack of adequate staff to provide radiological coverage and lack of expertise in the specialty 5. However, teleradiology has its limitations; for example, the system may require digital imaging and communications in medicine (DICOM) images to be transferred to a remote device for viewing before interpretation and installation of the required hardware and software can be costly. Space, insufficient infrastructure, hospital financial constraints, image storage capacity, system maturity and vendor-related concerns among others are the most pertinent challenges of picture archiving and communications system (PACS) implementation in South Africa 6.

While the use of technology in radiology is on the increase, especially with the availability of PACS, this is not the case for the underserved or rural areas of South Africa with no PACS or advanced hardware in place. These areas also lack trained radiologists. In South Africa, a country with a population of approximately 59 million, according to the Radiology Society of South Africa (RSSA) there are only approximately 60 qualified radiologists in academic institutions while a large number of qualified radiologists are in the private sector. These are rough estimates because not all qualified radiologists are RSSA members. As a result, the interpretation of images relies on the clinician who might not have sufficient training for diagnostic interpretation. In these underserved or rural areas, however, the medical population generally all have access to smartphones 1 and would benefit from the ability to quickly and easily share images with trained radiologists across the world especially when the diagnosis may be essential to altering patient management 1. The WhatsApp® application is a free cross-platform application that can be used on most smartphones and, as such, no additional equipment is required. With this application, one can exchange messages easily with a specific team member or the entire group with the additional benefit of being notified if the message has been received and read 7. Capturing radiographic scans and video clips from computer screens and sending them as instant messages have become a common means of communication between physicians, aiding in the triage and transfer decision-making in orthopaedic and neurosurgical emergencies 2.

Several studies, in various disciplines, have investigated the use of smartphones for medical image capture. A study published by Bullard in 2013, demonstrated that mobile-phone images of computed tomography (CT) scans appear to provide adequate images for triaging patients and assist with transfer decisions of neurosurgical cases 3. Orthopaedic studies conducted by Giordano (2015) found an excellent inter-and intra-observer agreement in the imaging assessment of tibia plateau fractures sent via WhatsApp® Messenger 8. In 2012, Padmasekara demonstrated that multi-media messaging service (MMS) with smartphones is a useful tool when assessing radiology images to work out management plans in distal radius fractures 3.

However, no South African studies have been conducted on the diagnostic reliability of smartphone captured and viewed radiology images. With this in mind the researchers set out to determine whether smartphone captured radiographs transmitted via WhatsApp® instant messenger can be used to make an accurate and reliable diagnosis. The study aimed to evaluate the reliability of radiographs captured and viewed using a smartphone, compared to a full-featured diagnostic PACS station for diagnosis of emergency life-threatening conditions.

Methods

This was a cross-sectional study conducted at Pelonomi Tertiary Hospital, Bloemfontein, Free State. The radiology unit within Pelonomi Tertiary Hospital serves the population of the Free State province, as well as occasional out of province and private patients.

The study population consisted of radiographs obtained from Pelonomi Tertiary Hospital from 1 June 2018 to 1 July 2019. The principal investigator, who was a 4th year radiology registrar at the time of data collection and interpretation, identified 135 plain radiographs from PACS, which are representative of emergency life-threatening conditions.

The cases were not selected randomly or as they occurred, but rather with the specific intent of developing a spectrum representative of conditions that require immediate intervention and/or referral to a senior hospital for further management and investigation. The researchers aimed at selecting common emergency conditions, with at least 20 radiographs per condition; this was achieved for all the identified medical conditions to be included in the study, with the exception of pneumomediastinum for which we were only able to acquire seven radiographs.

The selected medical conditions included pneumothorax, pneumomediastinum, pneumoperitoneum, bowel obstruction, spinal fractures of different morphology which consisted mostly cervical spine fractures and normal X-rays. The selected conditions are common for the region and require urgent management, sometimes further imaging and transfer of patient to a higher level of care. These conditions could be easily missed and overlooked especially if the clinicians are not competent in image interpretation. The images were displayed on a 2megapixel PACS computer screen and captured by the principal investigator using a 12-megapixel iPhone 7 camera, positioned 25 cm from the screen. To simulate what would occur in an emergency -centre, the x-rays were captured without any additional settings or equipment. The camera has a resolution of 1334 × 750 pixels at 326 pixels per image according to product specifications. The captured images were subsequently sent to five radiology registrars via the WhatsApp® instant messaging application (version 2.12.5).

Antero-posterior/postero-anterior and lateral views of chest, abdomen and musculoskeletal radiographs were selected. Additional views were also used when needed. The selected images were viewed by the principal investigator and a board-certified radiologist; this was deemed as the gold standard.

Although in the clinical setting the medical history will be provided, this was not provided to participants in the study to limit recall bias. To ensure confidentiality, each registrar was allocated an identification number to be used throughout the study. The participating registrars reviewed the images on their phones. The participants' smartphones were from different companies but a minimal screen resolution of at least 1334 × 750 pixels was required for the study. The participants completed a data form, stating the location of pathology and final diagnosis.

A period of 3 weeks was allowed to limit the possible recall of previous images. Thereafter the registrars viewed the exact same images, but in random order, on a departmental full-featured PACS station; again the location of the pathology and their final diagnosis was captured on a data form.

The registrars were required to delete the images from their phones within 2 weeks of receiving them. While it is not possible for the researchers to delete the images on the recipients' smartphones, WhatsApp® instant messenger uses end-to-end encryption for data protection. This means that only the sender and the recipients have access to the image, WhatsApp® and third parties do not have the means to decrypt the data and therefore cannot read the data.

The researchers coded the data and entered it into a Microsoft Excel spreadsheet. The data was analysed by the Department of Biostatistics at the University of the Free State. McNemar's test was used to compare the diagnostic accuracy of smartphone and PACS. Kappa values were calculated for agreement. Reliability was assessed by analysing the results of different registrars and diagnoses separately. In the case of normal scans, the percentage accurately diagnosed reflect specificity whereas for all other diagnoses the percentage accurately diagnosed reflect sensitivity.

Ethical approval was obtained from the Health Sciences Research Ethics committee of the University of Free State (Ethical clearance number UFS-HSD2019/0236/3007). Permission to perform the study was obtained from the Free State Department of Health.

Personal information on images, including patient age, sex, and hospital number were hidden before the image transfer to protect patient confidentiality. Verbal consent was obtained from the participating registrars. No identifying details regarding the participating registrars were made known to protect confidentiality.

Results

A total of 135 radiographs were included in the study and five radiology registrars assisted with the interpretation of the images. One registrar had completed the final written fellowship examination, three registrars were in their 3rd year of residency and one registrar was in the second year of residency.

Most images were of the chest (70/135 [51.9%] images), cervical spine (31/135 [23.0%] images) and abdomen (29/135 [21.5%] images). Image pathologies were distributed among pneumothorax (24.2%), spinal fractures (20.5%), normal (18.0%), bowel obstruction (16.1%), pneumoperitoneum (15.4%) and pneumomediastinum (5.2%) (Table 1).

Table 1.

Images per body part and pathology.

	Frequency	Percentage (%)
Images per body part (N = 135)
Abdomen	29	21.5
Cervical spine	31	23.0
Chest	70	51.9
Chest and abdomen	4	3.0
Lumbar spine	1	0.7
Images per pathology
Pneumothorax	32	24.2
Spinal fractures	28	20.5
Normal	25	18.0
Bowel obstruction	22	16.1
Pneumoperitoneum	21	15.4
Pneumomediastinum	7	5.2

The overall diagnostic accuracy (percentage of diagnoses agreeing with gold standard diagnosis) of all registrars on smartphones vs PACS is presented in Table 2.

Table 2.

Overall diagnostic accuracy on smartphone vs PACS.

	Smartphone accuracy (%) (N = 135)		PACS accuracy (%) (N = 135)
Registrar	Frequency	Percentage (%)	Frequency	Percentage (%)	p-value	Kappa
1	114	84.4	124	91.9	0.04	0.51
2	115	85.2	124	91.9	0.01	0.47
3	95	70.4	116	85.9	<0.01	0.37
4	96	71.1	118	87.4	<0.01	0.43
5	112	83.0	123	91.1	0.1	0.39

The diagnostic accuracy of all registrars using smartphones vs PACS, per image pathology, is presented in Table 3. For all registrars, the PACS accuracy was generally higher than the smartphone accuracy. The diagnostic accuracy for pneumothorax and pneumoperitoneum was high on smartphones and PACS, while spinal fractures had a low accuracy with an average of 63% on smartphones and 72% on PACS.

Table 3.

Diagnostic accuracy per image pathology on smartphone vs PACS.

Registrar:	Smartphone accuracy (%)					PACS accuracy (%)
	1	2	3	4	5	1	2	3	4	5
Diagnosis
Spinal fractures (n = 28)	78	71	61	42	64	75	71	75	64	75
Bowel obstruction (n = 22)	86	90	81	86	95	86	100	95	95	95
Pneumoperitoneum (n = 21)	100	95	95	95	90	100	100	100	100	90
Pneumothorax (n = 32)	96	90	93	90	84	100	100	96	100	96
Pneumomediastinum (n = 7)	85	100	100	85	85	100	100	100	85	100
Normal (n = 25)	72	84	56	44	96	100	96	72	76	100

Discussion

Capturing images with a smartphone and sharing them with colleagues via WhatsApp®, has become one of the main communication channels, especially among the on-call teams. In remote areas, where there are no PACS in place and therefore this practice can assist in prompt diagnosis and timeous transfer of patients to tertiary institutions. It is known that WhatsApp® reduces the size of the image before sending it to the recipient and, therefore, there is a reduction in image quality and resolution compared to the original images. Other disadvantages include the inability to manipulate the image as well as having to view the medical images on a small smartphone screen 9.

This study was designed to evaluate the accuracy of smartphone captured radiographic images communicated via WhatsApp® and viewed on a smartphone in comparison to viewing images on a full-featured PACS system. The study sample comprised of radiographs representing common accident and emergency conditions. The majority of diagnoses were made without difficulty on PACS, while a higher number of diagnostic abnormalities were misdiagnosed on smartphone captured images, as would be expected, given the greater degree of technical limitations.

This was however not the case with spinal fractures, where a higher number of fractures were missed on both PACS and smartphones. This may be due to the complexity of the fractures and the fact that clinical history and examination findings were withheld from the participating reviewers. Conditions such as pneumothorax, pneumomediastinum, and pneumoperitoneum were included, which could potentially be missed with inadequate access to radiologic expertise. It is of concern that two of the participants over diagnosed a high percentage normal images on smartphones leading to low accuracy on smartphones, and this accuracy significantly increased on PACS viewing.

The intraobserver variation is indicated in Table 2. To our knowledge, no tested method has achieved 100%. Diagnostic accuracy of pneumothorax on a smartphone and PACS were both high (average 90% accuracy on smartphone and 98% on PACS), as was the case for pneumoperitoneum (average 95% accuracy on smartphone and 98% on PACS). Although pneumomediastinum accuracy was also high, these results cannot be conclusive because of the number of selected cases.

Our results are consistent with published literature that also showed good accuracy of smartphone captured images 1., 10., 11., 12.. A non-radiology study by Handelman (2018) concluded that chest x-ray transmission via WhatsApp® results in a comparable ability to identify clinical findings as viewing the same image on a workstation 11. A paediatric study published by Westberg et al. in 2016 found no significant differences in the accuracy of diagnosing pneumothorax on a smartphone versus PACS 12. In this study, 40 paediatric chest x-rays were viewed by 20 participants; the accuracy on a smartphone was 81% and 80% on PACS.

There are some limitations to this study that are worth mentioning. Recall bias is inevitable in intra-observer studies. The researchers tried to minimise this by waiting at least three weeks between evaluating the cases in both methods. Another limitation was the relatively small number of cases, due to lack of representative radiographs on PACS system as most patients arrive as referral cases from local hospitals with printed films. The duration of the study also contributed to this limitation. Further larger studies with more representative cases and reviewers are advised.

A final limitation is taking into account the differences in quality of smartphone screen characteristics, which may be pertinent to the variations in smartphone quality. We attempted to minimise this bias by utilising minimal resolution requirements of at least 1334 × 750 pixels.

Our findings indicate that the use of smartphone-based WhatsApp® imaging for sending images can be used with high degree of confidence in diagnosis, decision making and management, especially for the detection of pneumothorax and pneumoperitoneum (accuracy of 90% and 95% respectively on a smartphone). This will in turn reduce waiting times in the emergency centre and prevent unnecessary interfacility transfers.

Overall, the findings suggest that identifying major diagnostic abnormalities on smartphone-captured images utilising at least a 12-megapixel camera can be done with a high degree of confidence. This method, however, should only be used in emergency settings to aid in timeous patient management/transfer to the next level of care. We do not recommend routine usage of smartphones as a substitute for PACS.

Dissemination of results

Results from this study was shared with staff members at the data collection site through an informal presentation.

Authorship contribution statement

Authors have contributed as follows to the conception and design of the study; data acquisition; analysis; interpretation of data; and drafting of work and revising it critically for important intellectual content: UN contributed 70%, JVR 15% and GJ 15%. All authors approved the version to be published and agreed to be accountable for all aspects of the work.

Declaration of competing interest

The authors declare that they have no financial or personal relationships that may have inappropriately influenced them in writing this article. The authors received no financial support for research, authorship, and publication of this article. The views and the opinions expressed in this article are those of the authors and do not necessarily reflect those of their affiliated institution.