Skip to main content
PMC home page
BMJ Open logoLink to BMJ Open
. 2023 Aug 11;13(8):e068207. doi: 10.1136/bmjopen-2022-068207

Diagnostic reliability in teledermatology: a systematic review and a meta-analysis

Adrienn N Bourkas 1,#, Natasha Barone 2,#, Matthew E C Bourkas 3, Matthew Mannarino 4, Robert D J Fraser 5,6, Amy Lorincz 7, Sheila C Wang 6,8, Jose Luis Ramirez-GarciaLuna 4,✉
PMCID: PMC10423833  PMID: 37567745

Abstract

Objectives

To compare teledermatology and face-to-face (F2F) agreement in primary diagnoses of dermatological conditions.

Design

Systematic review and meta-analysis

Methods

MEDLINE, Embase, Cochrane Library (Wiley), CINAHL and medRxiv were searched between January 2010 and May 2022. Observational studies and randomised clinical trials that reported percentage agreement or kappa concordance for primary diagnoses between teledermatology and F2F physicians were included. Titles, abstracts and full-text articles were screened in duplicate. From 7173 citations, 44 articles were included. A random-effects meta-analysis was conducted to estimate pooled estimates. Primary outcome measures were mean percentage and kappa concordance for assessing diagnostic matches between teledermatology and F2F physicians. Secondary outcome measures included the agreement between teledermatologists, F2F dermatologists, and teledermatology and histopathology results.

Results

44 studies were extracted and reviewed. The pooled agreement rate was 68.9%, and kappa concordance was 0.67. When dermatologists conducted F2F and teledermatology consults, the overall diagnostic agreement was significantly higher at 71% compared with 44% for non-specialists. Kappa concordance was 0.69 for teledermatologist versus specialist and 0.52 for non-specialists. Higher diagnostic agreements were also noted with image acquisition training and digital photography. The agreement rate was 76.4% between teledermatologists, 82.4% between F2F physicians and 55.7% between teledermatology and histopathology.

Conclusions and relevance

Teledermatology can be an attractive option particularly in resource-poor settings. Future efforts should be placed on incorporating image acquisition training and access to high-quality imaging technologies.

Trial registration number

10.17605/OSF.IO/FJDVG

Keywords: Telemedicine, DERMATOLOGY, PRIMARY CARE, Information technology, STATISTICS & RESEARCH METHODS

STRENGTHS AND LIMITATIONS OF THIS STUDY.

  • This is the most comprehensive systematic review and meta-analysis of the topic to date without language restrictions applied.

  • Inclusion criteria were broad, including all types of dermatological diseases, imaging technologies, in-person physician specialisations (GPs, hospitalists and dermatologists) and the presence or absence of image acquisition training.

  • The article search was limited to 2010 and later due to the recent incorporation of smartphones in teledermatology practices.

  • Due to considerable heterogeneity between studies, meta-analysis and synthesis of predictors for accurate diagnoses remotely were limited even after subgrouping.

Introduction

With the emergence of COVID-19, the introduction of virtual consults in healthcare settings, especially dermatology, has been expanded to allow many patients the opportunity for equitable access to care when in-person appointments pose a challenge and risk to patients.1 Different modalities were introduced to support teledermatology. This involves remote sharing of patient data, including synchronous video-streaming teledermatology and asynchronous sharing of still images via emails, or text messages, or store-and-forward teledermatology (SFTD).

Although both synchronous and asynchronous approaches have been shown to be cost-effective, SFTD is particularly popular as it requires fewer resources and less coordination than synchronous teledermatology.2 3 With the advent of higher resolution smartphone cameras, relatively minimal training is required to capture data for remote dermatologists correctly; multiple SFTD studies opted to provide no training in image capture and still found value in teledermatology.4 5

There is valid concern over the reliability of teledermatology given the significant variability in diagnostic accuracy predicted across pre-pandemic research.6 This is expected given the lack of standardisation across studies and the potential for confounders across teledermatology methodologies and applications, for example, level of training or skin lesion type. This variability in approach may benefit from an increased demand, which could provide greater impetus to optimise and standardise teledermatology.

To our knowledge, this is the first and most inclusive meta-analysis (MA) that compares teledermatology consults to face-to-face (F2F) that looked at all relevant studies without overly exclusive inclusion criteria. The primary objective of this study was to compare the reliability of teledermatology diagnoses to F2F consults, as determined by Cohen’s kappa interrater agreement and total agreement rates. Teledermatology can assume important roles as a routine complement to primary care and an alternate route to the typical in-person referrals. Consequently, we wanted to determine agreement for teledermatology and all F2F consults, teledermatology and F2F primary care consults, and finally teledermatology and F2F dermatologist consults, which would arguably best capture the limitations introduced by the change in medium from F2F to teledermatology.

Additional subset analyses were performed to control for potential confounders (eg, inflammatory vs malignant, staff training for image acquisition, teledermoscopy, and smartphone vs digital cameras) introduced by the heterogeneous methodology. The secondary objectives sought to determine the agreement rate within teledermatology diagnoses and F2F consults to provide an idea of each medium’s consistency and provide the best estimate of accuracy for the agreement rate between teledermatology and histopathology.

Methods

This study was reported in accordance with the Preferred Reported Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.

Protocol registration

Prior to the conduct of this review, a protocol which adhered to the PRISMA-protocols (ie, PRISMA-P) guidelines was developed and then registered on Open Science Framework (OSF). Access: https://osf.io/fjdvg.7

Search strategy

A comprehensive search of major bibliographic databases, MEDLINE, Embase, Cochrane Library (Wiley), CINAHL and medRxiv was performed in August 2021. MEDLINE was searched again between August 2021 and May 2022 to screen any new articles published after our protocol was registered. The search strategy was developed by a medical librarian at Queen’s University (Kingston, Ontario). Please see the online supplemental appendix for additional information on the search strategy.

Supplementary data

bmjopen-2022-068207supp001.pdf (1.6MB, pdf)

No restrictions were placed on the language or status of the publications. Search results were limited to studies published between January 2010 and May 2022 due to the novelty of incorporating smartphones in teledermatology remote consultations.8 The International Prospective Register of Systematic Reviews (PROSPERO) and OSF were searched up to May 2022 for relevant ongoing systematic reviews using the terms ‘telemedicine’, ‘teledermatology’, ‘dermatology’, ‘diagnostic accuracy’ and ‘diagnostic concordance’. Reference lists of included studies were screened to identify additional studies not captured in the search.

Eligibility criteria

Studies evaluating the diagnostic reliability of teledermatology that reported on patients with dermatological conditions assessed by a clinician using asynchronous or synchronous telemedicine systems were included. All articles were required to compare telediagnosis to F2F diagnosis conducted by a physician. In this context, an ‘F2F physician’ refers to healthcare professionals, such as dermatologists, general practitioners or emergency department physicians, who conducted in-person assessments only. This term is used to represent the comparison group in our analyses, and these assessments may occur concurrently or sequentially with teledermatology consultations, depending on the case. Exclusion criteria encompassed survey articles, feasibility studies, non-dermatological telemedicine studies, cost-effectiveness studies, editorials, review articles, studies using teledermatology as the reference standard, studies comparing only dermatoscopic images without clinical images and studies where patients captured their own photographs. The latter was excluded to ensure consistent image quality, enabling a more accurate comparison of diagnostic reliability between tele methods and F2F methods. Included articles are summarised in online supplemental table S1 in the online supplemental appendix. Inclusion and exclusion criteria are summarised in online supplemental table S2, available in the online supplemental appendix.

Data selection and extraction

Following the removal of duplicated citations, the titles and abstracts were screened. Following this step, a full-text assessment was conducted. At both stages, two reviewers performed screening independently (ANB and NB). Any disagreements were resolved through consensus by the two reviewers and when necessary, through discussion with a third reviewer (JLR-G).

A data collection form was created on the Covidence website and piloted by two reviewers (ANB, NB). Three additional reviewers assisted with data extraction (JLR-G, MECB, MM). Two reviewers were assigned to each paper. One reviewer extracted all characteristics of the included literature, and the second reviewer validated the characteristics for accuracy. Any disagreements were resolved by consensus. In the online supplemental appendix, online supplemental table S3 summarises the information extracted from full-text articles.

Data synthesis

This meta-analysis assessed the effectiveness of SFTD technologies and live video conferencing in diagnosing skin conditions. Outcomes regarding complete diagnostic percentage agreement rates and Cohen’s kappa concordance were evaluated separately, with some studies being part of both analyses if they reported both variables. The patient, intervention type, lesion and geographic characteristics were summarised qualitatively. Please see the online supplemental appendix and online supplemental table S4 for more details on data synthesis and nomenclature for each study grouping.

Risk of bias

Three reviewers (ANB, NB, MECB) completed the risk of bias (RoB) assessment; all studies were independently reviewed. Version 2 of the Cochrane RoB tool for randomised trials (RoB 2) was used to assess the RoB in three randomised trials.9–11 RoB 2 is structured into a fixed set of domains of bias, focusing on different aspects of trial design, conduct and reporting.12 The Quality Assessment of Diagnostic Assessment of Diagnostic Accuracy (second Edition, QUADAS-2) was used to assess the RoB. Uncertain RoB was assigned to studies with insufficient information except for studies that were likely to be biased due to missing data. In the latter case, a high RoB was assigned.

Synthesis of results

Statistical analysis was performed using the dmetar package in R V.4.0.1 (R Foundation for Statistical Computing, 2022). Agreement rates and Cohen’s kappa concordances for unique study groupings were treated as individual and independent values. For the percentage of agreement, meta-analyses were conducted using the aggregated data, and proportions were calculated with the corresponding 95% CIs. Point-biserial correlations were utilised to calculate pooled kappa values. Statistical heterogeneity was investigated using the I² index and the τ² statistic, leading to the use of a random-effects model for overall complications with a logit transformation due to the high degree of heterogeneity. Possible sources of heterogeneity were explored through subgroup analysis, and confounding factors were controlled using meta-regression. A random-effects model, as proposed by DerSimonian and Laird, was chosen as the primary method to estimate all pooled estimates. Further details on the statistical analysis can be found in the online supplemental appendix.

Patient and public involvement

Patients or the public were not involved in our research’s design, conduct, reporting or dissemination plans.

Results

A total of 7173 studies were screened for eligibility of which 44 were included in this study. Of these, 40 studies reported diagnostic agreement rates4 5 9–11 13–47 and 21 studies reported kappa concordance.5 9 13 14 19 22 25 28–33 35–37 48–52 Further details are provided in the PRISMA diagram in figure 1. The complete list of excluded studies can be found in the online supplemental appendix, online supplemental table S5.

Figure 1.

Figure 1

Open in a new tab

Preferred Reported Items for Systematic Reviews and Meta-Analyses (PRISMA flow diagram of study selection.

Study and patient characteristics

Online supplemental table S1 summarises the study and participant characteristics for the 44 included papers. Forty-one of the included studies were observational, of which 32 were prospective, 8 were retrospective. One study was ambispective. Two studies were randomised controlled trials and one study was a quasi-randomised trial. Studies selected for the review included a total of 52 075 patients (range: 26–24 210 patients). Some patients had multiple lesions, and the total number of lesions included in the study was 57 222 (range: 26–27 519 lesions).

The mean age reported in 27 (61%) studies was 54.78±15.69 years (range: 0–100 years old). Thirty-four (77%) studies reported participant gender, with a mean of 57% women (range: 3.2–74%). Only 13 (29%) studies reported information on Fitzpatrick skin types, ethnicity or race. Twenty-eight studies (64%) included in this analysis were inclusive of all types of dermatoses, 13 (29%) studies looked specifically at suspicious lesions and 3 (7%) studies excluded skin cancers completely.

Diagnostic reliability of teledermatology when compared with F2F (specialist and non-specialist) evaluation

We assessed the diagnostic reliability of teledermatology compared with F2F evaluations by analysing diagnostic agreement rates and concordance. The overall diagnostic agreement rate ranged from 13.9% to 98.0% (mean 68.9%, CI 64.4% to 73.1%), with a concordance that ranged from 0.21 to 0.96 (mean 0.67, CI 0.60 to 0.74). See online supplemental figure S1 and the online supplemental appendix for further details.

Sub-group analyses

Diagnostic agreement between teledermatologist and teledermatologist, F2F and F2F physicians, and teledermatology and histopathology

See online supplemental appendix and online supplemental figure S2 for further details.

Diagnostic reliability of teledermatologist versus F2F specialist and non-specialist

Teledermatologists’ 70.96% agreement rate with F2F dermatologists significantly exceeded the 44.1% rate from non-specialists (p<0.001). Non-specialists consistently showed lower diagnostic concordance across studies; see online supplemental appendix and online supplemental figure S3 for further details.

Diagnostic reliability of teledermatology versus F2F by training provided for image acquisition

Twenty studies with 37 unique comparisons explicitly provided training to those in charge of image acquisition shown in figure 2.9–11 14–16 19 20 23 26 29 32 35–41 43 44 The mean agreement rate between teledermatology and F2F physicians in these studies was 75.9% (CI 74.4% to 77.27%), significantly higher than the 62.1% (CI 60.5% to 63.7%) observed when no training was provided (p=0.033, heterogeneity: Iˆ2=98%). Concordance values were also higher when training was provided (mean 0.77, CI 0.66 to 0.84) compared with when no training was provided (mean 0.60, CI 0.49 to 0.69) (p=0.01, Iˆ2=98%).

Figure 2.

Figure 2

Open in a new tab

Forest plot representing F2F and teledermatology primary diagnostic agreement by whether imaging acquisition training was indicated by the study. Forest plot representing F2F and teledermatology primary diagnostic agreement when image acquisition training is involved. Studies were sorted into two groups, (a) did not conduct or did not report training personnel on image acquisition; (b) stated that person in charge of image acquisition was trained. (Left) Forest plot representing percentage agreement and 95% CI for overall concordance across 40 studies with a total of 72 unique number of comparisons, N of events and total included participants. (Right) Forest plot representing kappa concordance and 95% CI for overall concordance across 21 studies with a total of 45 unique number of comparisons, N of total included participants. F2F, face-to-face; PCP, primary care provider; TD, teledermatology or teledermatologist.

Diagnostic reliability of teledermatology versus F2F by type of technology used for image acquisition

Approximately half of the studies with 41 unique comparisons that compared teledermatologists with F2F physicians used digital cameras for image acquisition. Eighteen studies comparing F2F and teledermatology agreement rates with 26 unique comparisons reported the use of smartphones and tablets for image acquisition. Figure 3 shows that the mean percentage agreement rate for digital cameras was 71.7% (CI 70.3% to 73.1%) compared with 59.8% (CI 57.2% to 62.3%) for smartphones or tablets. The higher agreement rate with digital photography was statistically significant (p=0.029, heterogeneity: Iˆ2=98%). The concordance values for digital photography were reported for 12 studies with a mean of 0.70 (CI 0.61 to 0.76). Concordance values for smartphone or tablet technologies were reported for eight studies with a mean of 0.62 (CI 0.38 to 0.78). The higher concordance with digital photography was statistically significant (p=0.003, heterogeneity: Iˆ2=100%).

Figure 3.

Figure 3

Open in a new tab

Forest plot representing F2F and teledermatology primary diagnostic agreement by device type used to capture clinical photographs. Forest plot representing F2F and teledermatology primary diagnostic agreement by imaging technology used. Studies were sorted into three groups, (i) digital photography; (ii) imaging technology not mentioned, (iii) smartphone or tablet. (Left) Forest plot representing percentage agreement and 95% CI for overall concordance across 40 studies with a total of 72 unique number of comparisons, N of events and total included participants. (Right) Forest plot representing kappa concordance and 95% CI for overall concordance across 21 studies with a total of 45 unique number of comparisons, N of total included participants. F2F, face-to-face; PCP, primary care provider; TD, teledermatology or teledermatologist.

Other sub-group analyses

No statistically significant patterns could be identified with the inclusion of teledermoscopy in addition to clinical images (online supplemental figure S4), lesion type (online supplemental figure S5), grouping studies as pre-pandemic or post-pandemic (figure not shown), or RoB (figure not shown). Please see the online supplemental appendix for further details.

Quality assessment

The quality assessment results for RoB and applicability in individual studies are displayed in the online supplemental appendix and online supplemental table S6.

Discussion

To our knowledge, this study constitutes the most extensive systematic review and meta-analysis on teledermatology, including 44 studies across four languages.

Our subgroup analyses revealed that agreement rates between teledermatology consultations and F2F physicians were significantly higher when dermatologists conducted in-person assessments compared with non-specialists. This finding suggests that teledermatology may be more beneficial in supplementing primary care than specialist care, as lower concordance with non-specialists indicates reduced reference test accuracy. Although we did not directly assess the impact of consulting teledermatologists on non-specialist accuracy, the included studies report high levels of non-specialist satisfaction with the teleconsultation process. In fact, 96% of non-specialists agreed that they learnt about the dermatologic diagnosis, and 100% agreed that it helped patient care.23 These results are consistent with prior research attributing high provider satisfaction to streamlined workflows, effective communication and fast turnaround times in teledermatology.2 53

The study emphasises the importance of standardised training on image acquisition in improving agreement rates between in-person and remote care. Additionally, digital photography was linked to increased agreement rates, potentially due to enhanced image resolution and experienced staff conducting virtual consultations using standardised procedures. This suggests a crucial need for comprehensive training in image acquisition, highlighting the importance of equipping primary care providers supporting telehealth delivery with high-quality cameras and the latest smartphone models.24 54 55

Assessing agreement on the management plan is crucial in teledermatology as it serves as a triage tool for distinguishing mild/benign cases from severe/malignant/uncertain cases. Ensuring concordance in the management plan between telemedicine and F2F consultations is vital for optimising patient care. Future research should explore the consistency of treatment recommendations and interventions between telemedicine and in-person consultations to further enhance the evaluation of telemedicine’s effectiveness in guiding appropriate patient management.

Pathological assessment of skin lesions is the cornerstone of skin cancer diagnosis. This meta-analysis found a 55.7% (CI 53.0% to 58.4%) agreement rate between teledermatology and histopathology. This low agreement rate reflects all skin biopsies, and specific diagnostic accuracy rates could not be calculated by lesion type due to the small number of studies that reported this value. Through subgroup analyses, we were able to compare cancerous and non-cancerous lesions; slightly higher concordance was seen with skin cancers compared with studies that also included non-suspicious lesions like dermatitis and psoriasis. However, the data was too heterogeneous for any significant conclusions. We also looked at the use of teledermoscopy, another technique that could help improve the diagnostic accuracy of teledermatology for suspicious lesions, but no significant trends could be identified. These findings reflected the results of a 2016 systematic review on teledermatology.6

Many teledermoscopy studies grouped statistics from lesions analysed with and without dermoscopy, preventing the assessment of the dermatoscope’s incremental contributions without the influence of potentially less accurate, dermatoscope-free analysis. Supporting this explanation, the three teledermoscopy studies that focused on cancer lesions demonstrated greater concordance rates than the teledermoscopy studies targeting broader lesions. One study identified agreement rates between teledermatology and F2F dermatology of 92.3% (24/26) and between teledermatology and histopathology of 66.7% (17/26), both above our identified median.45 Another study found an agreement rate of 90% (37/41) when targeting pigmented lesions, although the rate may have been inflated due to recall bias introduced by having the same dermatologist perform teledermatology and F2F consults.16 Finally, one study diagnosed keratotic lesions in sun-exposed areas, finding a high agreement rate of 92% (915/1000).37 However, this study also risked bias from its experimental design, which excluded lesions with poor image quality. This fails to recapitulate the complexities of practical teledermatology, which must contend with potentially difficult image acquisition.

The 68.9% (CI 64.4% to 73.05%) combined agreement rate between teledermatology and F2F is lower than the agreement rates outlined in a recent review.56 This suggests that our greater sample size introduces more studies with poor agreement, which may better reflect the reality of adopting teledermatology at a larger scale and signal risk from a lack of standardisation.55 Our date cut-off of 2010 means our dataset has little overlap with existing reviews, and more heavily features new relevant technologies like smartphone apps for image acquisition.6 57 The most recent MA57 on teledermatology limited its dataset to studies with multiple teledermatology and F2F consults and variably choosing to filter low-frequency diagnoses from certain studies.46

We acknowledge several potential limitations. The heterogeneity of the data, though at first glance might limit generalisability, enhances the adaptability and applicability of teledermatology across diverse real-world contexts. Challenges exist due to the absence of stratification by study design and a limited number of randomised controlled trials. Nevertheless, our findings emphasise the critical importance of standardised processes for effective teledermatology, such as training in image acquisition, reporting guidelines and addressing privacy concerns. Our study reveals a greater degree of heterogeneity compared with previous meta-analyses, reflecting real-world application and clinical practice, bolstering the robustness of our conclusions. We advocate for a nuanced interpretation when generalising these findings across all settings, recognising the demographic and technological diversity in our sample as an asset. While our attempts to filter biased studies did not yield significant improvements to our meta-analysis model, we are mindful of the potential risk of publication bias in our review.

Furthermore, our study only included a limited number of live video conferencing studies,11 24 46 and our ability to draw meaningful conclusions regarding the differences between live video conferencing and SFTD methods is therefore limited. A recent study by Duong et al24 demonstrated that live video conferencing can significantly contribute to diagnosis in teledermatology by improving the quality of collected information and accuracy of the patient’s status evaluation. The study found that videoconferencing significantly improved the diagnostic performance in 68.7% of cases. While these results are promising, further research is needed to explore the potential differences between clinical images and live video conferencing.

In addition, our search was limited to published literature and may have missed relevant studies in the grey literature and reports from low-income and middle-income countries. Nonetheless, the variability across providers and settings underlines the need for a standardised framework to employ and assess teledermatologists. Future research is needed to explore the differences between these methods and other potential factors that may impact the efficacy of teledermatology, particularly in low-income and middle-income countries. We acknowledge these limitations and encourage further research to address these gaps in the literature.

Current trends suggest that teledermatology will continue to expand; there have been many recent studies examining its accuracy without the design considerations necessary to allow comparisons beyond siloed investigations.1 The implementation of evidence-informed processes is critical to the success of teledermatology services, and the accurate assessment of teledermatology will be required to assess which contexts it should be employed in, for example, suspected malignancy vs erythema.

While acknowledging the significant potential of artificial intelligence (AI) in enhancing teledermatology, particularly in areas like image recognition and diagnosis, it is crucial to note that our current study does not incorporate these aspects. The impact of AI on teledermatology, while promising, introduces an additional layer of complexity, necessitating a dedicated, separate investigation beyond the scope of our current study.

The factors targeted by our subanalysis are undoubtedly important to standardise with best practices requiring the inclusion of primary care provider training in image acquisition, explicitly outlined conditions where dermatoscope attachments are required, and standardised reporting with a lesion’s anatomical site, size, distribution, morphology and colour. Additional guidelines for data reporting could be designed with a mind to future research goals, for example, the inclusion of Fitzpatrick grading to identify gaps in medical care. Finally, both clinical and research guidelines must address privacy concerns, as integrating to the electronic medical record (EMR) and sharing of patient images or videos presents potential vulnerabilities.

Conclusion

This meta-analysis indicates that diagnostic agreement between remote and in-person dermatologists is acceptable in select conditions (ie, when training for image acquisition is provided and technologies for high-quality images are used). Telemedicine adoption rates are accelerating globally, and teledermatology must be considered for enhanced accessibility, flexibility, reduced costs and safer environments it can provide for patients.

The results of this meta-analysis represent significant evidence to indicate the suitability of teledermatology for remote care, particularly as a complement to primary care, where it can serve as an intermediate step before F2F specialist consultations. Furthermore, the categorisation of diagnostic concordance highlights important factors to further improve diagnostic accuracy. Additionally, it highlights the lack of standardisation in teledermatology studies, calling for greater structure in clinical practice and conducting primary research.

Supplementary Material

Reviewer comments
bmjopen-2022-068207.reviewer_comments.pdf (238.9KB, pdf)
Author's manuscript
bmjopen-2022-068207.draft_revisions.pdf (14.4MB, pdf)

Acknowledgments

The authors would like to acknowledge Heba Tallah Mohammed affiliated with Swift Medical for reviewing the manuscript and adding their valuable insight. The search strategies were developed with Abdul Kareem Pullattayil (AKP), a health science librarian at Bracken Health Sciences Library Botterell Hall at Queen’s University.

Footnotes

Twitter: @rdjfraser, @ramirezgl

ANB and NB contributed equally.

Contributors: JLR-G is the guarantor of the review and supervised study design. JLR-G also contributed to data analysis and provided statistical expertise. ANB and NB oversaw study design, data collection, data analysis and original draft preparation. ANB designed the search strategy with the guidance of the medical librarian, AKP. ANB, NB, MECB and MM participated in the abstract and full-text screen, data extraction and risk of bias assessment. RDJF, AL and SCW contributed to the draft review and editing. All authors read, provided feedback and approved the final manuscript.

Funding: This work was supported by the Digital Technology Supercluster as part of the Telewound Care Canada project. The funder was not involved in the design or analysis of this study.

Competing interests: RDJF is an employee, and SCW is a co-founder, chief medical officer, and shareholder of Swift Medical. JRGL and AL were formerly employees of Swift. No funding bodies have any role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. All other authors declare no conflict of interest

Patient and public involvement: Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

Provenance and peer review: Not commissioned; externally peer reviewed.

Supplemental material: This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Data availability statement

Data are available in a public, open access repository. All data relevant to the study are included in the article, uploaded as supplementary information, or deposited on Open Science Framework: https://osf.io/fjdvg. Data are available under the terms of the Creative Commons Zero 'No rights reserved' data waiver (CC0 1.0 Public domain dedication). Our systematic review produced a large amount of information, and the arising database is available for future collaboration on additional analyses. Please contact the corresponding author with any inquiries.

Ethics statements

Patient consent for publication

Not applicable.

Ethics approval

Not applicable.

References

  • 1.Mohammed HT, Hyseni L, Bui V, et al. Exploring the use and challenges of implementing virtual visits during COVID-19 in primary care and lessons for sustained use. PLoS One 2021;16:e0253665. 10.1371/journal.pone.0253665 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Armstrong AW, Chambers CJ, Maverakis E, et al. Effectiveness of online vs in-person care for adults with psoriasis: a randomized clinical trial. JAMA Netw Open 2018;1:e183062. 10.1001/jamanetworkopen.2018.3062 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Wang RH, Barbieri JS, Nguyen HP, et al. Clinical effectiveness and cost-effectiveness of teledermatology: where are we now, and what are the barriers to adoption? J Am Acad Dermatol 2020;83:299–307. 10.1016/j.jaad.2020.01.065 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Okita AL, Molina Tinoco LJ, Patatas OHG, et al. Use of smartphones in telemedicine: comparative study between standard and teledermatological evaluation of high-complex care hospital Inpatients. Telemed J E Health 2016;22:755–60. 10.1089/tmj.2015.0086 [DOI] [PubMed] [Google Scholar]
  • 5.Saleh N, Abdel Hay R, Hegazy R, et al. Can teledermatology be a useful diagnostic tool in dermatology practice in remote areas? An Egyptian experience with 600 patients. J Telemed Telecare 2017;23:233–8. 10.1177/1357633X16633944 [DOI] [PubMed] [Google Scholar]
  • 6.Finnane A, Dallest K, Janda M, et al. Teledermatology for the diagnosis and management of skin cancer: a systematic review. JAMA Dermatol 2017;153:319–27. 10.1001/jamadermatol.2016.4361 [DOI] [PubMed] [Google Scholar]
  • 7.Moher D, Shamseer L, Clarke M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev 2015;4:1. 10.1186/2046-4053-4-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Brewer AC, Endly DC, Henley J, et al. Mobile applications in dermatology. JAMA Dermatol 2013;149:1300–4. 10.1001/jamadermatol.2013.5517 [DOI] [PubMed] [Google Scholar]
  • 9.Ríos-Yuil JM. Correlation between face-to-face assessment and telemedicine for the diagnosis of skin disease in case conferences. Actas Dermosifiliogr 2012;103:138–43. 10.1016/j.ad.2011.05.011 [DOI] [PubMed] [Google Scholar]
  • 10.Romero Aguilera G, Cortina de la Calle P, Vera Iglesias E, et al. Interobserver reliability of store-and-forward teledermatology in a clinical practice setting. Actas Dermosifiliogr 2014;105:605–13. 10.1016/j.ad.2013.12.007 [DOI] [PubMed] [Google Scholar]
  • 11.Romero G, Sánchez P, García M, et al. Randomized controlled trial comparing store-and-forward teledermatology alone and in combination with web-camera videoconferencing. Clin Exp Dermatol 2010;35:311–7. 10.1111/j.1365-2230.2009.03503.x [DOI] [PubMed] [Google Scholar]
  • 12.Sterne JAC, Savović J, Page MJ, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ 2019;366:l4898. 10.1136/bmj.l4898 [DOI] [PubMed] [Google Scholar]
  • 13.Altieri L, Hu J, Nguyen A, et al. Interobserver reliability of teledermatology across all fitzpatrick skin types. J Telemed Telecare 2017;23:68–73. 10.1177/1357633X15621226 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Azfar RS, Lee RA, Castelo-Soccio L, et al. Reliability and validity of mobile teledermatology in human immunodeficiency virus-positive patients in Botswana a pilot study. JAMA Dermatol 2014;150:601–7. 10.1001/jamadermatol.2013.7321 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Barbieri JS, Nelson CA, James WD, et al. The reliability of teledermatology to triage inpatient dermatology consultations. JAMA Dermatol 2014;150:419–24. 10.1001/jamadermatol.2013.9517 [DOI] [PubMed] [Google Scholar]
  • 16.Barcaui CB, Lima PMO. Application of teledermoscopy in the diagnosis of pigmented lesions. Int J Telemed Appl 2018;2018:1624073. 10.1155/2018/1624073 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Batalla A, Suh-Oh HJ, Abalde T, et al. Teledermatology in Paediatrics. In: Observations in daily clinical practice. Anales de pediatria. Barcelona, Spain, 2003: 324–30. 10.1016/j.anpede.2015.07.036 [DOI] [PubMed] [Google Scholar]
  • 18.Börve A, Holst A, Gente-Lidholm A, et al. Use of the mobile phone multimedia messaging service for teledermatology. J Telemed Telecare 2012;18:292–6. 10.1258/jtt.2012.120206 [DOI] [PubMed] [Google Scholar]
  • 19.Börve A, Terstappen K, Sandberg C, et al. Mobile teledermoscopy-there’s an app for that DPC 2013;3:41–8. 10.5826/dpc.0302a05 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Carter ZA, Goldman S, Anderson K, et al. Creation of an internal teledermatology store-and-forward system in an existing electronic health record: a pilot study in a safety-net public health and hospital system. JAMA Dermatol 2017;153:644–50. 10.1001/jamadermatol.2017.0204 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Chen TS, Goldyne ME, Mathes EFD, et al. Pediatric teledermatology: observations based on 429 consults. J Am Acad Dermatol 2010;62:61–6. 10.1016/j.jaad.2009.05.039 [DOI] [PubMed] [Google Scholar]
  • 22.Clarke EL, Reichenberg JS, Ahmed AM, et al. The utility of teledermatology in the evaluation of skin lesions. J Telemed Telecare 2023;29:382–9. 10.1177/1357633X20987423 [DOI] [PubMed] [Google Scholar]
  • 23.Costello CM, Cumsky HJL, Maly CJ, et al. Improving access to care through the establishment of a local, teledermatology network. Telemed J E Health 2020;26:935–40. 10.1089/tmj.2019.0051 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Duong TA, Cordoliani F, Julliard C, et al. Emergency department diagnosis and management of skin diseases with real-time teledermatologic expertise. JAMA Dermatol 2014;150:743–7. 10.1001/jamadermatol.2013.7792 [DOI] [PubMed] [Google Scholar]
  • 25.Gabel CK, Nguyen E, Karmouta R, et al. Use of Teledermatology by dermatology hospitalists is effective in the diagnosis and management of inpatient disease. J Am Acad Dermatol 2021;84:1547–53. 10.1016/j.jaad.2020.04.171 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Gatica JL, Bertolo S, Morales E, et al. Store-and-forward Teledermatogy in Chile: a contribution to primary health care. Piel 2015;30:148–54. 10.1016/j.piel.2014.07.007 [DOI] [Google Scholar]
  • 27.Gerhardt CA, Foels R, Grewe S, et al. Assessing the diagnostic accuracy of teledermatology consultations at a local veterans affairs dermatology clinic. Cureus 2021;13:e15406. 10.7759/cureus.15406 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Giavina-Bianchi M, Azevedo MFD, Sousa RM, et al. Part II: accuracy of teledermatology in skin neoplasms. Front Med (Lausanne) 2020;7:598903. 10.3389/fmed.2020.598903 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Giavina-Bianchi M, Sousa R, Cordioli E. Part I: accuracy of Teledermatology in inflammatory dermatoses. Front Med (Lausanne) 2020;7:585792. 10.3389/fmed.2020.585792 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Keller JJ, Johnson JP, Latour E. Inpatient teledermatology: diagnostic and therapeutic concordance among a hospitalist, dermatologist, and teledermatologist using store-and-forward teledermatology. J Am Acad Dermatol 2020;82:1262–7. 10.1016/j.jaad.2020.01.030 [DOI] [PubMed] [Google Scholar]
  • 31.Lamel SA, Haldeman KM, Ely H, et al. Application of mobile teledermatology for skin cancer screening. J Am Acad Dermatol 2012;67:576–81. 10.1016/j.jaad.2011.11.957 [DOI] [PubMed] [Google Scholar]
  • 32.Muir J, Xu C, Paul S, et al. Incorporating teledermatology into emergency medicine. Emerg Med Australas 2011;23:562–8. 10.1111/j.1742-6723.2011.01443.x [DOI] [PubMed] [Google Scholar]
  • 33.Nami N, Massone C, Rubegni P, et al. Concordance and time estimation of store-and-forward mobile teledermatology compared to classical face-to-face consultation. Acta Derm Venereol 2015;95:35–9. 10.2340/00015555-1876 [DOI] [PubMed] [Google Scholar]
  • 34.Patro BK, Tripathy JP, De D, et al. Diagnostic agreement between a primary care physician and a teledermatologist for common dermatological conditions in North India. Indian Dermatol Online J 2015;6:21–6. 10.4103/2229-5178.148927 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Ribas J, Cunha M da G, Schettini APM, et al. Agreement between dermatological diagnoses made by live examination compared to analysis of digital images. An Bras Dermatol 2010;85:441–7. 10.1590/s0365-05962010000400004 [DOI] [PubMed] [Google Scholar]
  • 36.Rubegni P, Nami N, Cevenini G, et al. Geriatric Teledermatology: store-and-forward vs. face-to-face examination. J Eur Acad Dermatol Venereol 2011;25:1334–9. 10.1111/j.1468-3083.2011.03986.x [DOI] [PubMed] [Google Scholar]
  • 37.Sola-Ortigosa J, Muñoz-Santos C, Masat-Ticó T, et al. The role of teledermatology and teledermoscopy in the diagnosis of actinic keratosis and field cancerization. J Invest Dermatol 2020;140:1976–84. 10.1016/j.jid.2020.02.013 [DOI] [PubMed] [Google Scholar]
  • 38.Tan E, Yung A, Jameson M, et al. Successful triage of patients referred to a skin lesion clinic using teledermoscopy (IMAGE IT trial). Br J Dermatol 2010;162:803–11. 10.1111/j.1365-2133.2010.09673.x [DOI] [PubMed] [Google Scholar]
  • 39.Tran K, Ayad M, Weinberg J, et al. Mobile teledermatology in the developing world: implications of a feasibility study on 30 Egyptian patients with common skin diseases. J Am Acad Dermatol 2011;64:302–9. 10.1016/j.jaad.2010.01.010 [DOI] [PubMed] [Google Scholar]
  • 40.Vañó-Galván S, Hidalgo A, Aguayo-Leiva I, et al. Store-and-forward teledermatology: assessment of validity in a series of 2000 observations. Actas Dermosifiliogr 2011;102:277–83. 10.1016/j.ad.2010.11.006 [DOI] [PubMed] [Google Scholar]
  • 41.Vestergaard T, Prasad SC, Schuster A, et al. Diagnostic accuracy and interobserver concordance: teledermoscopy of 600 suspicious skin lesions in Southern Denmark. J Eur Acad Dermatol Venereol 2020;34:1601–8. 10.1111/jdv.16275 [DOI] [PubMed] [Google Scholar]
  • 42.Warshaw EM, Gravely AA, Nelson DB. Reliability of store and forward teledermatology for skin neoplasms. J Am Acad Dermatol 2015;72:426–35. 10.1016/j.jaad.2014.11.001 [DOI] [PubMed] [Google Scholar]
  • 43.Zanini M. Analyze of diagnostic concordance between face-to-face and teledermatology diagnosis. Med Cutan Ibero Lat Am 2013;41:60–2. [Google Scholar]
  • 44.Zink A, Kolbinger A, Leibl M, et al. The value of teledermatology using a mobile app compared to conventional dermatology. Eur J Dermatol 2017;27:429–31. 10.1684/ejd.2017.3044 [DOI] [PubMed] [Google Scholar]
  • 45.Zink A, Kolbinger A, Leibl M, et al. Teledermoscopy by mobile phones: reliable help in the diagnosis of skin lesions Hautarzt 2017;68:890–5. 10.1007/s00105-017-4042-0 [DOI] [PubMed] [Google Scholar]
  • 46.Marchell R, Locatis C, Burges G, et al. Comparing high definition live interactive and store-and-forward consultations to in-person examinations. Telemed J E Health 2017;23:213–8. 10.1089/tmj.2016.0093 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Jones L, Jameson M, Oakley A. n.d. Remote skin cancer diagnosis: adding images to electronic referrals is more efficient than wait-listing for a nurse-led imaging clinic. Cancers;13:5828. 10.3390/cancers13225828 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.González Coloma F, Sandoval Garcés M, Gedda Quiroga V, et al. Teledermatology in remote parts of Chile: experience in 4 isolated rural areas. Actas Dermosifiliogr (Engl Ed) 2019;110:653–8. 10.1016/j.ad.2019.01.009 [DOI] [PubMed] [Google Scholar]
  • 49.Silveira CEG, Carcano C, Mauad E, et al. Call phone usefulness to improve the skin cancer screening: preliminary results and critical analysis of mobile app development. Rural Remote Health 2019;19:1–7. 10.22605/RRH4895 [DOI] [PubMed] [Google Scholar]
  • 50.Piccoli MF, Amorim BDB, Wagner HM, et al. Teledermatology protocol for screening of skin cancer. An Bras Dermatol 2015;90:202–10. 10.1590/abd1806-4841.20153163 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.Şenel E, Baba M, Durdu M. The contribution of teledermatoscopy to the diagnosis and management of non-melanocytic skin tumours. J Telemed Telecare 2013;19:60–3. 10.1177/1357633X12474961 [DOI] [PubMed] [Google Scholar]
  • 52.Warshaw EM, Gravely AA, Nelson DB. Accuracy of teledermatology/teledermoscopy and clinic-based dermatology for specific categories of skin Neoplasms. Journal of the American Academy of Dermatology 2010;63:348–52. 10.1016/j.jaad.2009.10.037 [DOI] [PubMed] [Google Scholar]
  • 53.McFarland LV, Raugi GJ, Reiber GE. Primary care provider and imaging technician satisfaction with a teledermatology project in rural veterans health administration clinics. Telemed J E Health 2013;19:815–25. 10.1089/tmj.2012.0327 [DOI] [PubMed] [Google Scholar]
  • 54.Massone C, Javor S, Amato I, et al. Training of primary care physicians enhances performance of mobile Teledermatology. An Bras Dermatol 2021;96:514–6. 10.1016/j.abd.2020.07.017 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 55.Finnane A, Curiel-Lewandrowski C, Wimberley G, et al. Proposed technical guidelines for the acquisition of clinical images of skin-related conditions. JAMA Dermatol 2017;153:453. 10.1001/jamadermatol.2016.6214 [DOI] [PubMed] [Google Scholar]
  • 56.Lee JJ, English JC. Teledermatology: a review and update. Am J Clin Dermatol 2018;19:253–60. 10.1007/s40257-017-0317-6 [DOI] [PubMed] [Google Scholar]
  • 57.Bastola M, Locatis C, Fontelo P. Diagnostic reliability of in-person versus remote dermatology: a meta-analysis. Telemed J E Health 2021;27:247–50. 10.1089/tmj.2020.0043 [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary data

bmjopen-2022-068207supp001.pdf (1.6MB, pdf)

Reviewer comments
bmjopen-2022-068207.reviewer_comments.pdf (238.9KB, pdf)
Author's manuscript
bmjopen-2022-068207.draft_revisions.pdf (14.4MB, pdf)

Data Availability Statement

Data are available in a public, open access repository. All data relevant to the study are included in the article, uploaded as supplementary information, or deposited on Open Science Framework: https://osf.io/fjdvg. Data are available under the terms of the Creative Commons Zero 'No rights reserved' data waiver (CC0 1.0 Public domain dedication). Our systematic review produced a large amount of information, and the arising database is available for future collaboration on additional analyses. Please contact the corresponding author with any inquiries.

Articles from BMJ Open are provided here courtesy of BMJ Publishing Group

RESOURCES