Abstract
Introduction:
Demand for dermatologic services in safety net hospitals, which disproportionately serve patients with darker coloured skin, is growing. Teledermatology has the potential to increase access and improve outcomes, but studies have yet to demonstrate the reliability of teledermatology for all Fitzpatrick skin types.
Methods:
We assessed the reliability of teledermatologists’ diagnoses and management recommendations for store- and-forward teledermatology in patients with lightly pigmented (Fitzpatrick skin types I–III) versus darkly pigmented (Fitzpatrick skin types IV–VI) skin, when compared to in-person diagnosis and management decisions. This prospective study enrolled 232 adult patients, presenting with new, visible skin complaints in a Los Angeles county dermatology clinic. Forty-seven percent of patients were Fitzpatrick skin types I–III, and 53% were Fitzpatrick skin types IV–VI.
Results:
Percent concordance for the identical primary diagnosis was 53.2% in lighter (Fitzpatrick I–III) skin types and 56.0% in darker (Fitzpatrick IV–VI) skin types. There was no statistically significant difference in concordance rates between lighter and darker skin types for primary diagnosis. Concordance rates for diagnostic testing, clinic-based therapy, and treatments were similar in both groups of Fitzpatrick skin types.
Discussion:
These results suggest that teledermatology is reliable for the diagnosis and management of patients with all Fitzpatrick skin types.
Keywords: Telecare, teledermatology, telehealth, telemedicine, ehealth
Introduction
Teledermatology has been explored as a technology to help improve access to dermatology services for patients who otherwise have limited access to healthcare. Access to care is an ongoing issue for the uninsured and those with Medicaid, the majority of whom are people of colour.1 By 2017, 30% of the 26 million Americans who will have gained Medicaid coverage through the Affordable Care Act (ACA) will be seen at safety net community health centres.2 The newly insured are more likely to be nonwhite, less likely to rank themselves in ‘excellent’ or ‘very good’ health, and are more likely to speak a language other than English.3 One in four is Latino, black, Asian, Native American, or multi-racial.3 Even though the newly insured will need specialty medical care, fewer than one third of the safety net clinics offer specialty services on site.4 Considering that the wait to see a dermatologist can be upwards of several months in safety net clinics,5 and that rates of fatal skin diseases like melanoma are increasing among Latinos in California,6 teledermatology has the potential to increase access and improve outcomes in the uninsured and Medicaid populations.
Two main types of teledermatology are used: live interactive (LI) and store-and-forward (SF). LI teledermatology allows the patient and dermatologist to see each other and converse in real time, while SF teledermatology allows a primary care clinician to take pictures of a rash or lesion, submit them along with the patient’s clinical information to a teledermatologist, and receive feedback regarding diagnosis and management in a matter of hours to days. SF teledermatology has been shown to be cost-effective,7-9 reduce time to treatment,10 and have similar patient satisfaction ratings when compared to traditional in-person dermatology.11-13
Since teledermatology has emerged as a tool to increase access to dermatology services in safety net hospitals, it is necessary to evaluate its effectiveness in ethnic minorities. Only a handful of studies regarding SF teledermatology have reported racial or ethnic characteristics. While a study by Taylor et al.14 reported in the subanalysis that there was no difference in concordance rates between pigmented skin and non-pigmented skin, no studies have examined, as the primary outcome measure, the influence of varying degrees of skin pigmentation on concordance rates.
Warshaw et al.10 reviewed 78 studies on diagnosis, management, outcomes, and costs of teledermatology, and found three studies15-17 that reported racial or ethnic characteristics. The overall majority of patients (72%) in these three studies was white. One of the LI teledermatology studies enrolled 60 patients, of whom 73% were white, and did not find a significant difference in diagnostic concordance between two physicians evaluating white versus nonwhite patients in the subanalysis.17 Another LI study enrolled a higher percentage of nonwhite (40%) patients but did not evaluate skin pigmentation as a variable.15 The third study, in which 85% of the patients were white, compared LI, SF, and in-person examination diagnostic and management concordance, but did not report any results for how these outcomes differed according to varying levels of skin pigmentation.16
While most teledermatology concordance studies examine diagnostic concordance as a primary outcome, clinical management recommendations can also be assessed. The three main categories of clinical management include diagnostic testing recommendations (e.g. biopsy, bacterial culture, labs), medical therapy recommendations (e.g. recommendations of application of topical creams or oral prescriptions), and clinic-based therapy recommendations (e.g. cryotherapy, excision, intralesional steroid injection). Assessing these variables may be equally important as diagnostic concordance because the decision of how to manage a medical issue may have a greater impact on clinical outcome than accuracy of the clinical diagnosis. No studies to date have examined the concordance of teledermatologists’ clinical management recommendations for all Fitzpatrick skin types. In an effort to determine any variability in teledermatology concordance rates across all Fitzpatrick skin types, this clinical study evaluated SF teledermatology in Fitzpatrick skin types IV through VI compared to types I through III.
Methods
Study design and patients
To assess whether SF teledermatology would be useful in safety net facilities that regularly care for a large proportion of uninsured and Medicaid patients in an urban setting, this prospective clinical study was designed to evaluate the accuracy of this technology in patients with darkly pigmented versus lightly pigmented skin, as a primary outcome. The study protocol was approved by the institutional review board at the University of Southern California and the Veterans Affairs Greater Los Angeles Healthcare System. All patients provided written informed consent.
The study was conducted at the Roybal Comprehensive Health Center’s dermatology clinic, a Los Angeles County Department of Health Services outpatient clinic affiliated with the University of Southern California (USC) Medical Center. All patients over 18 years of age presenting with new, visible skin complaints at the dermatology clinic were considered for the study, which was conducted from February 2007 to June 2007. A board-certified dermatologist served as the in-person clinician. Three board-certified dermatologists (authors JC, MC, JK) at the Veterans Affairs Greater Los Angeles Healthcare System served as teledermatologists. For each clinical case, there was one in-person dermatologist and two teledermatologists involved.
Teledermatology procedure
Prior to being seen by the dermatologist, consenting patients had digital images of their skin lesions or rashes taken by the same designated investigator. Fitzpatrick skin type and ethnicity were assessed for each patient by the same designated investigator who took the digital images. Participants’ skin lesions were imaged with a Canon Powershot SD400 5MP Digital Elph camera (Canon Inc., Tokyo, Japan). The digital images consisted of three standard images: (1) an image obtained a medium distance away from the lesion to show perspective; (2) a close-up macro image, with a standard ruler placed next to the lesion or rash, to show detail and scale; and (3) a side view to show if the lesion or rash was raised. Additional images were taken if deemed appropriate. Dermoscopic photos were not taken. The images were stored in a compressed Joint Photographic Experts Group (JPEG) format for later review by the teledermatologists.
After the digital images were taken, the patients proceeded with their regularly scheduled dermatology appointment with the in-person dermatologist. After the clinic appointment, the in-person dermatologist filled out a standardized form consisting of his or her top three diagnoses, in order of most to least likely diagnosis, as well as recommendations regarding the next course of action, including medical therapy diagnostic testing and clinic-based therapy. The digital images for each case were then reviewed by two of the three teledermatologists, who independently filled out the same standardized form that the in-person dermatologist had completed.
Outcome measures
The study outcomes measured the degree of concordance of diagnoses for primary diagnosis, any matching diagnosis (i.e. any one of three diagnoses from the in-person dermatologist matching any one of three diagnoses from a teledermatologist), medical therapy recommendations, diagnostic testing recommendations, and clinic-based therapy recommendations given by an in-clinic dermatologist and two teledermatologists. Concordance for these categories was calculated for Fitzpatrick skin types I–III and IV–VI. Overall concordance rates and Cohen’s kappa for these categories was also calculated.
Statistical analysis
Complete agreement was defined as the primary diagnoses matching between the in-person dermatologist and teledermatologist. Any agreement was defined as an overlap between any of the top three diagnoses listed by the in-person dermatologist and teledermatologist. Semantic differences in the dermatologic terminology were taken into consideration by two dermatologists (authors JH and AC). An interrater reliability analysis using the kappa statistic was performed to determine consistency among raters for complete diagnostic agreement and was analysed with 95% confidence intervals (95% CI). Statistical calculations were done with SPSS for Windows version 20.0 (SPSS Inc., Chicago, IL, USA). Concordance rates were also stratified by whether the dermatologic problem was a rash or a lesion.
Results
Demographic and diagnostic characteristics
Our study population included 232 patients. In our study population, 70.7% were Latino. 14.7% were white, 9.9% were Asian, 3.4% were black, and 1.3% were Middle Eastern; 108/232 (47%) were Fitzpatrick skin types I through III, and 124/232 (53%) were Fitzpatrick skin types IV through VI (Table 1). There were approximately equal numbers of growths and rashes seen (48.3% and 45.7%, respectively), and a smaller number of dermatologic problems (6.0%), such as hair and pigmentation problems, were classified as ‘other’. When classified by type of dermatosis (i.e. autoimmune, dermatitis, benign neoplasm, premalignant neoplasm, malignant neoplasm, alopecia, other, or acneiform eruption), the most common dermatologic problems were benign neoplasms (34.9%), dermatitis (25.0%), papulosquamous dermatoses (11.6%), and acneiform eruptions (7.8%). Lesions were mostly located on the head and neck (37.1%) and extremities (29.7%). A smaller proportion of dermatologic problems were located on the trunk (19.4%) or were diffusely distributed (13.8%). The most common diagnoses included seborrheic keratosis (11.6%), nevus (9.5%), and basal cell carcinoma (8.1%).
Table 1.
Demographic and diagnostic characteristics of 232 enrolled patients.
| Fitzpatrick skin type | N (%) |
|---|---|
| I–III | 108 (47%) |
| IV–VI | 124 (53%) |
| Race/ethnicity | |
| Latino | 164 (70.7%) |
| White | 34 (14.7%) |
| Asian | 23 (9.9%) |
| Black | 8 (3.4%) |
| Middle Eastern | 3 (1.3%) |
| Type of dermatologic problem | |
| Growth | 112 (48.3%) |
| Rash | 106 (45.7%) |
| Other (e.g. hair, pigment) | 14 (6.0%) |
| Lesion location | |
| Head and neck | 86 (37.1%) |
| Extremities | 69 (29.7%) |
| Trunk | 45 (19.4%) |
| Diffuse | 32 (13.8%) |
| Diagnostic classification | |
| Acneiform eruption | 18 (7.8%) |
| Alopecia | 4 (1.7%) |
| Autoimmune | 18 (7.8%) |
| Benign neoplasm | 81 (34.9%) |
| Dermatitis | 58 (25.0%) |
| Malignant neoplasm | 16 (6.9%) |
| Papulosquamous | 17 (11.6%) |
| Premalignant neoplasm | 6 (2.6%) |
| Other | 4 (1.7%) |
Agreement, diagnosis, and management
Our primary outcome of interest was the level of diagnostic agreement found among in-person dermatologists versus teledermatologists. During the study, 464 diagnostic and management observations were generated for the 232 cases. Of these observations, agreement for primary diagnosis was achieved 254 times (54.7%). The concordance was 86.6% for any matching diagnosis (Table 2). The concordances for identical primary management recommendations were 68.3% for medical treatment, 59.3% for diagnostic testing, and 79.5% for clinic-based therapy. Interrater reliability between the in-person dermatologist and the teledermatologists was determined for each teledermatologist. Kappa values for primary diagnosis were found to be kappa=0.51, 95% CI (0.43, 0.59); kappa=0.51, 95% CI (0.43, 0.59); and kappa=0.57, 95% CI (0.49, 0.64) for each of the three teledermatologists, respectively. The concordances among diagnosis, diagnostic testing, and clinic-based therapy were similar for Fitzpatrick skin types I–III and IV–VI (Table 3).
Table 2.
Diagnostic and management concordance between three teledermatologists and one in-person dermatologist.
| Primary diagnosis (%) |
Any diagnosis (%) |
Medical treatment (%) |
Diagnostic treatment (%) |
Clinic-based therapy (%) |
|
|---|---|---|---|---|---|
| Teledermatologist 1 | 58.1 (93/160) | 91.3 (146/160) | 72.5 (116/160) | 75.0 (120/160) | 82.5 (132/160) |
| Teledermatologist 2 | 53.3 (81/152) | 86.2 (131/152) | 66.4 (101/152) | 56.6 (86/152) | 78.9 (120/152) |
| Teledermatologist 3 | 52.6 (80/152) | 82.2 (125/152) | 65.8 (100/152) | 45.4 (69/152) | 77.0 (117/152) |
| Overall | 54.7 (254/464) | 86.6 (402/464) | 68.3 (317/464) | 59.3 (275/464) | 79.5 (369/464) |
Table 3.
Diagnostic and management concordance between three teledermatologists and one in-person dermatologist, grouped by Fitzpatrick skin types.
| Percentage agreement |
||
|---|---|---|
| Lighter skin (Fitzpatrick I–III) |
Darker skin (Fitzpatrick IV–VI) |
|
| Primary diagnosis | ||
| TD 1 | 57.5% (42/73) | 58.6% (51/87) |
| TD 2 | 52.2% (36/69) | 54.2% (45/83) |
| TD 3 | 50.0% (37/74) | 55.1% (43/78) |
| Overall | 53.2% (115/216) | 56.0% (139/248) |
| Any diagnosis | ||
| TD 1 | 93.2% (68/73) | 89.7% (78/87) |
| TD 2 | 84.1% (58/69) | 88.0% (73/83) |
| TD 3 | 85.1% (63/74) | 79.5% (62/78) |
| Overall | 87.5% (189/216) | 85.9% (213/248) |
| Diagnostic testing | ||
| TD 1 | 72.6% (53/73) | 77.0% (67/87) |
| TD 2 | 55.1% (38/69) | 57.8% (48/83) |
| TD 3 | 48.6% (36/74) | 42.3% (33/78) |
| Overall | 58.8% (127/216) | 59.7% (148/248) |
| Clinic-based therapy | ||
| TD 1 | 82.2% (60/73) | 82.8% (72/87) |
| TD 2 | 78.3% (54/69) | 79.5% (66/83) |
| TD 3 | 74.3% (55/74) | 79.5% (62/78) |
| Overall | 78.2% (169/216) | 80.6% (200/248) |
| Medical treatment | ||
| TD 1 | 75.3% (55/73) | 70.1% (61/87) |
| TD 2 | 63.8% (44/69) | 68.7% (57/83) |
| TD 3 | 68.9% (51/74) | 62.8% (49/78) |
| Overall | 69.4% (150/216) | 67.3% (167/248) |
TD: teledermatologist
Discussion
Given that non-urgent dermatology appointments at safety net clinics commonly take months to obtain, even prior to the onset of the ACA,5 the newly insured are expected to be faced with even longer wait times and a shortage of appointment availabilities. As the population becomes more diverse, with people of colour projected to account for over half of the United States population by 2050,18 teledermatology’s reliability in diagnosing patients with darker skin types will become even more pertinent. In order to establish teledermatology as a viable solution to these issues, we must first establish that it works equally well in patients of all Fitzpatrick skin types, so as to not create further disparities in healthcare amongst vulnerable uninsured or underinsured people.
Concordance for lighter versus darker skin
In our study, which assessed interobserver reliability between an in-person dermatologist and two teledermatologists, no disparities between diagnostic and therapeutic/management decisions were found when Fitzpatrick skin types were controlled for. Based on the similar concordance rates in darker and lighter skin for diagnostic concordance, diagnostic testing recommendations, clinic-based therapy recommendations, and medical treatment recommendations, we can conclude that Fitzpatrick skin type does not appear to have an effect on diagnostic reliability or management recommendations in teledermatology.
Diagnostic concordance
Our study had a concordance of 54.7% for primary diagnosis and 86.6% for any matching diagnosis. The diagnostic concordance values obtained in this study are lower than, yet still comparable to, values reported in similar store-and-forward teledermatology studies. In a recent review of 15 store-and-forward general teledermatology studies by Warshaw et al.,10 concordance rates varied widely between studies based on the complexity and variety of diagnoses rendered. The authors reported a 66.5% concordance rate for primary diagnosis (range: 46–88%) and a 65.3% concordance rate for any matching diagnosis (range: 60–95%).10 Of the 15 studies included in the Warshaw review, a study by Whited et al.19 reported diagnostic concordance rates strikingly similar to ours: 54% concordance for primary diagnosis and 92% for any matching diagnosis. This study was similar to ours because their in-person dermatologists and teledermatologists provided diagnoses for lesions and rashes, as opposed to classifying a single lesion as benign or malignant (which is likely to yield higher concordance rates).
Kappa coefficients for our study ranged from 0.48 to 0.55 (moderate agreement, according to the categorization schema described by Landis and Koch20) for types I through III and 0.53 to 0.57 (moderate agreement) for types IV through VI. Similar studies have reported kappa coefficients ranging from 0.65 to 0.71.10 The lower than expected kappa coefficients could have likely been improved with many approaches. Some disagreement among the physicians may be attributed to the technology itself, as the digital camera used did not offer optics as advanced as many cameras currently used in teledermatology studies. Although there is evidence in the literature to suggest that digital images with a resolution of 1.2MP are sufficient for diagnosis,21 it is likely that a more advanced camera could have improved the concordance. Aside from the quality of images, it is important to note that digital images in our study were not accompanied by medical history, making it impossible to directly compare the in-person visit with the teledermatology consultations. Dermoscopic images were also not used, which may have led to decreased diagnostic confidence and poorer concordance among the dermatologists. Additionally, our study was conducted prior to the routine use of teledermatology in dermatology training programmes and concordance may have improved if our dermatologists had been exposed to the use of teledermatology during their training.
Management concordance
Our study examined concordance for identical top matching management recommendations and found concordance rates of 59.3% for diagnostic testing recommendations, 68.3% for medical therapy, and 79.5% for clinic-based therapy. These values fall within the range of management concordance rates that have been previously reported (55% to 84%).10 Concordance rates varied by the number of management options available to the teledermatologists and the complexity of the skin lesions. Management concordance rates reported in a study similar to ours reported diagnostic agreement to be between 67% to 68% for diagnostic testing, 67% to 69% for medical therapy, and 64% to 74% for clinic-based therapy.19
Limitations
We acknowledge several limitations in our study. Overall kappas were unable to be calculated for the lighter versus darker skin diagnostic concordance because all three teledermatologists did not provide diagnoses for the same patients. Each clinical case was assessed by one in-person dermatologist and two teledermatologists. Although this method increased consistency, better external validity could have been achieved if more cases had been assessed, albeit by fewer teledermatologists per case. Also, skin pigmentation on the palmoplantar regions and on the mucosae is fairly similar between Fitzpatrick skin types, but lesions in these locations were not excluded from the study. Because biopsies were not performed, definitive histologic accuracy assessments were not available. Although this method may be less exact, reliance on biopsy-rendered diagnoses is not necessary in all cases because many dermatologic diagnoses are made clinically and do not rely on histological diagnosis.
Future research
In upcoming years, previously uninsured minorities will be increasingly seeking care in safety net clinics after gaining Medicaid coverage. Store-and-forward teledermatology can deliver a cost-effective and useful service to low-resource settings, such as the safety net clinic in which this study was performed. Adequate diagnostic and management concordance suggests that common diagnoses may be readily made via teledermatology. It may be beneficial in the future to conduct studies that include a higher percentage of complex diagnoses. This could help determine if teledermatology can be used to appropriately address high morbidity/mortality cutaneous-related health issues. Future studies are needed to further evaluate the use of teledermatology in large, underserved communities and safety net clinics. In particular, patient satisfaction, provider satisfaction, and the ability to appropriately address high morbidity/mortality health issues within this context are necessary. Furthermore, the development of high quality, accessible teledermatology programmes in underserved clinics relies on training healthcare providers that are well versed in the use of telemedicine. Professionals looking to establish telemedicine programmes in safety net clinics should consider continued education in the use of telemedicine.
Conclusion
With a growing demand for dermatologic services in safety net hospitals, which disproportionately serve patients with darker coloured skin, teledermatology has the potential to increase access and improve outcomes. This study evaluated a store-and-forward model for delivering care to patients with various Fitzpatrick skin types. In this study, which assessed interobserver reliability between an in-person dermatologist and two teledermatologists, no disparity between diagnostic and therapeutic/management decisions was found when Fitzpatrick skin type was controlled for. These results suggest that teledermatology is reliable for the diagnosis and management of patients with all Fitzpatrick skin types.
Acknowledgements
The authors have not published or submitted any related papers from the same study. Acquisition, analysis, and interpretation of the data: Hu, Peng, Kim, Chiu, Cotliar. Statistical analysis: Nguyen, Cockburn, Altieri. Dr. Cockburn and Mr. Nguyen had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Hu, Peng. Drafting of the manuscript: Altieri, Crew, Cockburn, Nguyen. Critical revision of the manuscript for important intellectual content: Crew, Hu, Kim, Chiu, Cotliar, Peng, Cockburn.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
References
- 1.Henry J Kaiser Family Foundation. Distribution of the nonelderly with Medicaid by race/ethnicity, 18 September, http://kff.org/medicaid/state-indicator/distribution-by-raceethnicity-4/ (2015, accessed 23 October 2015).
- 2.Blumenthal D and Collins SR. Health care coverage under the Affordable Care Act–a progress report. New Engl J Med 2014; 371(3): 275–281. [DOI] [PubMed] [Google Scholar]
- 3.Health Research Institute. Health insurance exchanges: long on options, short on time, October, www.pwc.com/us/en/health-industries/health-insurance-exchanges/assets/pwc-health-insurance-exchanges-impact-and-options.pdf (2012, accessed 23 October 2015).
- 4.Gusmano MK, Fairbrother G and Park H. Exploring the limits of the safety net: community health centers and care for the uninsured. Health Aff 2002; 21(6): 188–194. [DOI] [PubMed] [Google Scholar]
- 5.Felt-Lisk S, McHugh M and Howell E. Monitoring local safety-net providers: do they have adequate capacity? Health Aff 2002; 21(5): 277–283. [DOI] [PubMed] [Google Scholar]
- 6.Cockburn MG, Zadnick J and Deapen D. Developing epidemic of melanoma in the Hispanic population of California. Cancer 2006; 106(5): 1162–1168. [DOI] [PubMed] [Google Scholar]
- 7.Wootton R, Bloomer SE, Corbett R, et al. Multicentre randomised control trial comparing real time teledermatology with conventional outpatient dermatological care: societal cost-benefit analysis. Br Med J 2000; 320(7244): 1252–1256. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Whited JD, Datta S, Hall RP, et al. An economic analysis of a store and forward teledermatology consult system. Telemed J E Health 2003; 9(4): 351–360. [DOI] [PubMed] [Google Scholar]
- 9.Loane MA, Bloomer SE, Corbett R, et al. A comparison of real-time and store-and-forward teledermatology: a cost–benefit study. Br J Dermatol 2000; 143(6): 1241–1247. [DOI] [PubMed] [Google Scholar]
- 10.Warshaw EM, Hillman YJ, Greer NL, et al. Teledermatology for diagnosis and management of skin conditions: a systematic review. J Am Acad Dermatol 2011; 64(4): 759–772. [DOI] [PubMed] [Google Scholar]
- 11.Eminović N, de Keizer NF, Wyatt JC, et al. Teledermatologic consultation and reduction in referrals to dermatologists: a cluster randomized controlled trial. Arch Dermatol 2009; 145(5): 558–564. [DOI] [PubMed] [Google Scholar]
- 12.Moreno-Ramirez D, Ferrandiz L, Bernal AP, et al. Teledermatology as a filtering system in pigmented lesion clinics. J Telemed Telecare 2005; 11(6): 298–303. [DOI] [PubMed] [Google Scholar]
- 13.Whited JD, Hall RP, Foy ME, et al. Patient and clinician satisfaction with a store-and-forward teledermatology consult system. Telemed J E Health 2004; 10(4): 422–431. [DOI] [PubMed] [Google Scholar]
- 14.Taylor P, Goldsmith P, Murray K, et al. Evaluating a telemedicine system to assist in the management of dermatology referrals. Br J Dermatol 2001; 144(2): 328–333. [DOI] [PubMed] [Google Scholar]
- 15.Lowitt MH, Kessler II, Kauffman CL, et al. Teledermatology and in-person examinations: a comparison of patient and physician perceptions and diagnostic agreement. Arch Dermatol 1998; 134(4): 471–476. [DOI] [PubMed] [Google Scholar]
- 16.Edison KE, Ward DS, Dyer JA, et al. Diagnosis, diagnostic confidence, and management concordance in live-interactive and store-and-forward teledermatology compared to in-person examination. Telemed J E Health 2008; 14(9): 889–895. [DOI] [PubMed] [Google Scholar]
- 17.Phillips CM, Burke WA, Shechter A, et al. Reliability of dermatology teleconsultations with the use of teleconferencing technology. J Am Acad Dermatol 1997; 37(3 Pt 1): 398–402. [DOI] [PubMed] [Google Scholar]
- 18.DeNavas-Walt C, Proctor BD and Smith JC. Current population reports In: US Census Bureau (ed.) Income, Poverty, and Health Insurance Coverage in the United States. Washington, DC: U.S. Government Printing Office, 2013, pp.60–245. [Google Scholar]
- 19.Whited JD, Hall RP, Simel DL, et al. Reliability and accuracy of dermatologists’ clinic-based and digital image consultations. J Am Acad Dermatol 1999; 41(5 Pt 1): 693–702. [DOI] [PubMed] [Google Scholar]
- 20.Landis JR and Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33(1): 159–174. [PubMed] [Google Scholar]
- 21.van der Heijden JP, Spuls PI, Voorbraak FP, et al. Tertiary teledermatology: a systematic review. Telemed J E Health 2010; 16(1): 56–62. [DOI] [PubMed] [Google Scholar]