Diagnostic Agreement Between Telemedicine on Social Networks and Teledermatology Centers

Abstract

PURPOSE

With increasing delays in obtaining a dermatological consultation, general practitioners (GPs) are using social networks for telemedicine to obtain advice on dermatological images. The objective was to analyze diagnostic agreement between telemedicine on social networks (Twitter and MedPics) and standard teledermatology services (TDS).

METHODS

This retrospective observational study included images published on Twitter and MedPics by GPs in 2016. The contextualized images were evaluated by 2 teledermatology services in Paris, France and an expert committee. Diagnoses obtained from telemedicine on social networks, TDS, and the expert committee were collected for each image. The agreement between the diagnoses made on social networks and by TDS was measured using Cohen κ statistic. The number of correct diagnoses obtained using social networks and TDS as determined by agreement with the expert’s diagnoses were compared with χ² tests.

RESULTS

Two hundred and seventy health professionals responded to the 60 selected images from social networks. The main diagnoses, according to the experts were: purpura (8.3%), eczema (6.7%), mycosis (6.7%), and viral infections (6.7%). Diagnostic agreement between telemedicine on social networks and TDS was moderate over the entire set of images (κ = 0.55; 95% CI, 0.42-0.68) and good for images containing dermatologist’s answers (κ = 0.63; 95% CI, 0.45-0.85). The number of correct diagnoses was not statistically different between telemedicine on social networks and TDS on all images (60% vs 55%; P = .28) but was higher on social networks when a dermatologist answered (65% vs 55%; P <.01).

CONCLUSIONS

Diagnostic agreement using social network images showed that use of this telemedicine tool could be a reliable means to alleviate the difficulties of accessing dermatology consultations although data safety probably needs to be improved.

INTRODUCTION

Studies have shown that the dermatological diagnostic competence of general practitioners (GPs) is suboptimal, contributing to an increased use of dermatologists.¹ In fact, GPs feel insecure about their diagnostic abilities in dermatology, especially for cases of suspected malignant skin tumors.¹ In 2014, a review found that no tools or diagnostic aids for managing pigmented lesions were available to GPs.² Faced with increased delays (up to 2 months) for dermatologist consultations, the development and use of a photographic telemedicine tool could help minimize errors for patients managed in general practice.^3,4 Thus, for patients requiring a dermatology teleconsultation, a study found a decreased delay (4 days) in the teledermatology group, compared with 55.5 days in the control group. In addition, dermatologists estimated that 39% of dermatologist consultations were potentially avoidable, mainly because the patients recovered or the pathology was within the scope of general practice.⁵

Few studies have assessed the reliability of this process. They have reported moderate agreement between dermatological diagnoses made with teleconsultation and those made by consultating dermatologists or emergency physicians.^6,7 Teledermatology networks remain rare or relatively unknown in general practice despite the interest that primary care physicians have in telemedicine,⁸ and the use of social networks has emerged as an alternative tele-expertise tool.⁹ Actually, social networks are used by about 10% of health care professionals to share their experiences and questions. The impact of social network use on medical practices remains limited, however, partly due to concerns about the accuracy of answers obtained.¹⁰ Health care applications based on sharing medical images such as MedPics, could address this issue because they use a secure network only accessible to identifiable health care professionals.

Social networks used as teledermatology services have not been studied for diagnostic quality and reliability of the messages published in tele-expertise consultation. We hypothesize that diagnoses using 2 social networks (Twitter and MedPics) used as a teledermatology tool are concordant with those of standard teledermatology services (TDS). The objective of this study was to assess the diagnostic agreement between telemedicine on social networks and TDS using images of dermatological issues published on Twitter and MedPics by GPs.

METHODS

Design and Inclusion

A retrospective observational study was conducted on images published by GPs on Twitter (a social network open to the public allowing the exchange of short messages and images) and MedPics (a French-language application for sharing medical images and collaborative clinical cases reserved for health care professionals) (Figure 1). On Twitter included images were published from January through June 2016 by a GP and tagged with the hashtags #dermatotoctoc, #doctoctoc or #docstoctoc (toc toc is French for the English phrase knock knock). These hashtags have been used by French health care professionals to ask medical questions of the community since 2012. On MedPics, the last 50 images published by a GP in the Dermatology section prior to and through June 1, 2016, were included. The exclusion criteria were publication by a user who was not identified as a GP in their profile and the absence of a response from a graduate doctor based on their profile. Since the same image could be tagged with several hashtags, duplicate images on Twitter were excluded. These criteria were established, after discussion with the teledermatologists participating in the study, to ensure it’s feasibilty and enable the inclusion and analysis of 50 to 60 images.

Figure 1.

Sources and distribution of study images.

Outcomes

The social network diagnosis was used as the leading diagnosis for each image and defined as the most frequently suggested diagnosis among graduate physicians on Twitter and MedPics. The lead diagnosis was unique and selected by criteria in the following order: the dominant diagnosis suggested by dermatologists, then the dominant diagnosis among other responding doctors, and excluding responses given by the GP who published the image.

The TDS diagnosis was the diagnosis given by the responses of the TDS of 1 of 2 teaching hospitals. The diagnosis selected had to respond to the question asked by the requesting GP on Twitter and MedPics and was the main diagnosis (in case of concomitant pathologies).

The reference diagnosis of the image was established by an expert committee of 12 dermatologists from a teaching hospital (Saint Louis Hospital, Paris, France). This diagnosis had to be unique. It was used to define the percentage of exact diagnoses for Twitter, MedPics, and TDS relative to the reference diagnoses.

Specific management recommendations such as the need for dermatological consultations to perform a biopsy or for a dermatological follow-up, were collected.

Procedure

Included images were stratified on the social networks (Twitter or MedPics), then randomized before being sent to the TDS of 2 teaching hospitals (Henri Mondor Hospital, Créteil, France and Tenon Hospital, Paris, France) (Figure 1).

Diagnoses collection in TDS was carried out under the usual operating conditions, which meant sending an e-mail to the dedicated address of the TDS, including the image with all the clinical information available on Twitter and MedPics, between 9:00 AM to 6:00 PM in accordance with the schedules of the teledermatologists who had agreed to participate in the study.

For each image, the questions asked of the TDS and the expert committee were the same questions asked on Twitter and MedPics.

Statistical Analyses

The diagnoses were grouped by type of pathology to enable analyses to be carried out. They were performed with an α risk of 5% using R, Version 3.1.1 (R Project for Statistical Computing). Agreement analyses were conducted using Cohen’s κ method and the 95% CIs were determined by a bootstrap.¹¹ Chi square and Student tests were used to compare categorical and quantitative variables, respectively. McNemar tests were used to compare paired categorical data.

The primary analysis consisted of studying diagnostic agreement between social network diagnoses and TDS diagnoses using Cohen’s κ coefficient. Secondary analyses included studying (1) the agreement between social networks (Twitter and MedPics) and the expert committee, and (2) the number of correct diagnoses in social networks (Twitter and MedPics) and in TDS. Analyses were performed on all images, then images from Twitter and MedPics separately, and finally among the subgroup of images with at least 1 response by a dermatologist on Twitter and MedPics, and among the subgroup with no dermatologist’s response.

Ethics

The analyses were based on public anonymized data, complying with terms of use and privacy policies for both Twitter and MedPics. No exact quotations were included in this article. A registration to the French Commission for Data Protection and Liberties (Commission Nationale de l’Informatique et des Libertés) was made before the completion of this study (CNIL; #1990661). As the data published on social networks are public and the study was carried out retrospectively, approval by an ethics committee was not necessary.

RESULTS

Characteristics and Diagnoses

This study collected the responses of 270 health care professionals on social networks concerning 60 images: 39 images on Twitter and 21 on MedPics (Figure 1). Responses analyzed that contributed to social network diagnoses were submitted by 178 graduate doctors including 124 GPs and 25 dermatologists. A dermatologist response was present for 23 images (38.2%).

The most frequent diagnosis from social networks was mycosis (10.0%), from TDS was eczema (11.7%), and from the expert committee was purpura (8.3%). Viruses and viral rashes were among the 4 most frequent diagnoses in each group of evaluators (Table 1; Supplemental Table 1, available at https://www.AnnFamMed.org/content/19/1/24/suppl/DC1/). There was a recommendation for dermatologic consultation for 15 of the images (25.0%).

Table 1.

Diagnoses Based on the Images (N = 60)

Diagnosis	Expert Committee No. (%)	Twitter and MedPics No. (%)	Teledermatology Services No. (%)
Mycosis	4 (6.7)	6 (10.0)	6 (10.0)
Eczema	4 (6.7)	5 (8.3)	7 (11.7)
Viruses and viral rashes	4 (6.7)	4 (6.7)	5 (8.3)
Purpura	5 (8.3)	4 (6.7)	4 (6.7)
Lice and insect bites	2 (3.3)	2 (3.3)	3 (5.0)
Other	41 (68.3)	40 (66.7)	35 (58.3)

Diagnostic Agreement

According to Altman’s classification,¹² the diagnostic agreement between social network diagnoses and TDS diagnoses was moderate for all images and on images from Twitter and MedPics. The agreement was good when a dermatologist’s response was present on social networks but moderate when no dermatologist answered (Table 2).

Table 2.

Diagnostic Agreement Between Groups

Comparisons	All Images κ (95% CI) (n = 60)	Images With Dermatologist Responses κ (95% CI) (n = 23)	Images Without Dermatologist Response κ (95% CI) (n = 37)
Social networks vs teledermatology services	0.55 (0.42-0.68)	0.63 (0.45-0.85)	0.49 (0.33-0.68)
Twitter vs teledermatology services	0.57 (0.40-0.73)	0.61 (0.42-0.88)	0.52 (0.32-0.78)
MedPics vs teledermatology services	0.50 (0.30-0.74)	0.69 (0.20-1.0)	0.44 (0.24-0.72)
Social networks vs expert committee	0.58 (0.46-0.72)	0.63 (0.45-0.81)	0.55 (0.39-0.71)
Twitter vs expert committee	0.57 (0.43-0.75)	0.61 (0.43-0.83)	0.53 (0.32-0.78)
MedPics vs expert committee	0.60 (0.41-0.84)	0.69 (0.20-1.0)	0.56 (0.36-0.80)

Diagnostic agreement between social networks and the expert committee was moderate for all images and the Twitter images. It was good when a dermatologist answered in the social networks and in the MedPics images. In the absence of a dermatologist response, agreement was moderate (Table 2).

Correct Diagnoses

Based on the reference diagnoses made by the expert committee, 36 diagnoses (60%) were correct on Twitter and MedPics and 33 diagnoses (55%) were correct for TDS. There was no significant difference between the number of correct diagnoses made on social networks (Twitter and MedPics) and by TDS when all images were analyzed (P = .29).

The presence of a dermatologist response on Twitter and MedPics increased the probability of a correct diagnosis (65% vs 55%; P <.01), but no difference was found when analyzing the images without dermatologist responses (58% vs 65%; P = .8).

DISCUSSION

Summary

Our study found an acceptable agreement between the diagnoses made using social networks (Twitter and MedPics) and those made by TDS. Agreement was improved when a dermatologist answered posted queries. In addition, compared with reference diagnoses established by an expert committee, the number of correct diagnoses made on Twitter and MedPics were similar to those made by TDS.

Strengths and Limitations

Our search did not find any other study evaluating the diagnostic agreement between telemedermatology on the 2 social networks and TDS. This study allowed us to analyze a sufficient number of images and the responses of 270 health professionals over 6 months to limit selection bias. The cases in this study corresponded to issues seen in family medicine practices, as they were obtained from data published by GPs on Twitter and MedPics. The expert committee made it possible to use reference diagnoses and compare them with the tele-expertise of TDS.

Our study had some limitations. It was a retrospective study involving physicians that we could not directly question so no follow-up of patients was possible. In particular, there was no direct follow-up or dermatological monitoring implemented for patients. However, in order to avoid a selection bias of the cases and to have real situations, we judged it was not necessary to modify this aspect. Also, there was no histological analysis carried out to make a definitive diagnosis. Since the diagnoses were classic pathologies in general practice, however, it did not seem ethically necessary to systematically perform biopsies for diagnoses that are often determined by clinical observations. The committee of experts was therefore a good option for providing reference diagnoses in this situation lacking the gold standard.¹³

Comparison With Existing Literature

The dermatological pathologies diagnosed in our study were consistent with other French outpatient studies, finding mycoses and eczema to be the most frequently observed dermatological conditions.^14-16 The diagnosis of these frequent conditions is often clinically determined in general practice which probably explains why the number of correct diagnoses on Twitter and MedPics, without dermatologist’s answer, was close to that of teledermatologists. There were few skin lesions suspicious for cancer in our study. This may be due to GPs not wanting to take a risk with an opinion from a social network for these lesions, and these patients were probably directly referred to a dermatologist.

Some studies have investigated the agreement between diagnoses by a physician in-person and tele-expertise services, but they did not evaluate this for the general practice setting. A study conducted in France between emergency physicians and TDS found diagnostic agreement coefficients ranging from 0.6 to 1.0.⁶ In another study, conducted in California, dermatological tele-expertise was used to investigate suspected skin cancers. It compared dermatologists’ diagnoses with those of TDS, by sending images by smartphone or e-mail. The agreement coefficients were between 0.57 and 0.62, which is similar to our results.⁷

Implications for Practice

Our study provides information on the expertise of GPs diagnosing benign dermatological pathologies in primary care. GPs who are less familiar with dermatology seem to benefit from the experience of other GPs. However, the involvement of dermatologists as secondary care physicians seems essential in the case of more unusual pathologies or uncertain diagnosis as Eminović et al has also stated.⁵

Twitter and MedPics seem to be reliable telemedicine tools in the face of dermatological diagnostic challenges in general medicine and limited accessibility to dermatologists. However, from an ethical point of view, some constraints must be taken into account. Sharing medical data through communication technologies that support telemedicine requires attention to the conditions for hosting data in a way that can guarantee patient privacy. In 2011, the National Council of the French Medical Order (Conseil National de l’Ordre des Médecins) established recommendations for doctors using social networks. These mention that social media users must “know and implement confidentiality settings in order to control the disclosure of personal information.” Therefore, obtaining the patient’s consent is an essential preliminary step. It is not specified that this consent is mandatory on MedPics or Twitter, but it is required in another mobile app called Figure 1. Then, the anonymization of images must be implemented. The MedPics app integrates this step using a face recognition algorithm that automatically masks faces when images are uploaded. Elements that could provide information about the identity of patients (names, references, tattoos, etc) must also be removed. On Twitter, no faces or distinctive signs were present on the posted images, but there is no automatic procedure on this social network. Despite these precautions, some dermatological lesions could appear as a distinctive sign and compromise patient privacy.

Regarding data security, MedPics states that images are stored on a secured server, while publication on Twitter is public. It is possible for a user to restrict the access to his messages by keeping his account private, therefore the possibility of responses by other users is limited.

Finally, it should be noted that the information about profession on the physician’s Twitter profile is not subject to verification, in contrast to MedPics. However, the National Council of the French Medical Order does allow the use of pseudonyms, which implies the same ethical conditions as when the identity is known and which must be declared to the Council.¹⁷ In this context, dermatologists have reservations about the use of social networks, particularly concerning data protection, patients’ and physicians’ anonymity, and medical liability.¹⁸

In conclusion, the results of this study show that some social networks could provide good diagnoses for dermatological lesions in general practice. The establishment of dermatological tele-expertise using social networks with more dermatologists would be necessary to enhance the GP’s quality of diagnosis. These exchanges between colleagues would lead to a collective elaboration of the diagnosis, which would also make it possible to improve the accuracy of diagnoses and limit errors.¹⁹ In addition, the use of these communication networks could improve physicians’ professional satisfaction²⁰ and contribute to their professional development.²¹ It seems essential to organize and secure social networks in order to ensure the protection of health data in the area of tele-expertise. There is potential danger in the development of unofficial telemedicine due to potential risks of using non-secure networks or even using social networks as a teleconsultation service.²² The use of social networks for mutual assistance between physicians has been investigated in other studies,²³ but larger projects would be necessary to do this more precisely. In view of the good agreement with TDS and the experts committee, it should be possible to extend telemedicine tools with secure social networks to other medical specialties, encouraging exchanges between different specialists and promoting the continuity of patient care within the scope of GP’s competencies.