Long-running telemedicine networks delivering humanitarian services: experience, performance and scientific output

Abstract

Objective

To summarize the experience, performance and scientific output of long-running telemedicine networks delivering humanitarian services.

Methods

Nine long-running networks – those operating for five years or more– were identified and seven provided detailed information about their activities, including performance and scientific output. Information was extracted from peer-reviewed papers describing the networks’ study design, effectiveness, quality, economics, provision of access to care and sustainability. The strength of the evidence was scored as none, poor, average or good.

Findings

The seven networks had been operating for a median of 11 years (range: 5–15). All networks provided clinical tele-consultations for humanitarian purposes using store-and-forward methods and five were also involved in some form of education. The smallest network had 15 experts and the largest had more than 500. The clinical caseload was 50 to 500 cases a year. A total of 59 papers had been published by the networks, and 44 were listed in Medline. Based on study design, the strength of the evidence was generally poor by conventional standards (e.g. 29 papers described non-controlled clinical series). Over half of the papers provided evidence of sustainability and improved access to care. Uncertain funding was a common risk factor.

Conclusion

Improved collaboration between networks could help attenuate the lack of resources reported by some networks and improve sustainability. Although the evidence base is weak, the networks appear to offer sustainable and clinically useful services. These findings may interest decision-makers in developing countries considering starting, supporting or joining similar telemedicine networks.

Introduction

Telemedicine (i.e. medicine practised at a distance) has been used to improve health care delivery in a wide range of applications. To date, most of the work has taken place in industrialized countries and there is relatively little experience in the developing world.¹ Telemedicine’s fundamental benefit is in improving access to care, and in the developing world such access is often poor.¹ Thus, telemedicine may provide a useful way to reduce inequities and strengthen health systems in developing countries.

In 2005, the World Health Organization (WHO) established a global observatory for e-health to monitor the development of information and communications technologies (ICT) for health care – including telemedicine – and to provide reliable information and guidance on best practices, policies and standards. According to a recent survey, telemedicine has progressed far less in lower-income countries than in high-income countries both in terms of the proportion of countries with established services and the proportion offering pilot telemedicine services.² Nonetheless, several telemedicine networks around the world deliver humanitarian services on a routine basis, many to low-income countries. These networks provide tele-consultations for physicians and other health professionals needing advice about the clinical management of difficult cases, and some also provide education. Store-and-forward methods (e.g. e-mail) are often used for communication because they are generally cheaper and more convenient, but real-time methods (e.g. video links) are also used when required. Telemedicine networks delivering humanitarian services may be of interest to decision-makers considering wider implementation. Existing networks employ different organizational models and provide different kinds of services, and what represents best practice is unclear. Furthermore, data about network activities and performance are lacking.

To ensure effective and appropriate use of telemedicine in resource-limited settings, implementation must be guided by more and better evidence.³ The objective of the present paper is to summarize the experience gained so far with long-running telemedicine networks delivering humanitarian services; it looks at general information, network performance and scientific output. Long-running networks were selected for study because lack of programme sustainability is a commonly reported problem in telemedicine.

Methods

The work was conducted in three stages: (i) identifying relevant telemedicine networks; (ii) collecting information about their activities, and (iii) summarizing the resulting data.

Network selection

Long-running telemedicine networks delivering humanitarian services were identified. In this context, “long-running” was defined as having existed for 5 years or more. “Telemedicine” was defined as clinical and educational work at a distance. “Humanitarian services” were defined as actions designed to save lives, alleviate suffering and maintain human dignity delivered unconditionally (i.e. without seeking payment from their recipients). These networks were initially identified by one of the authors based on personal knowledge of the field.

A contact person at each telemedicine network was approached and asked if they knew of any other long-running telemedicine networks. This produced a list of nine networks, all of which were contacted. Two of the nine telemedicine networks were excluded from further consideration: one because it had apparently ceased operating when contacted and another because it never replied. The seven long-running telemedicine networks covered by this paper are shown in Table 1. Respondents, who were always the network coordinators, were invited to co-author this paper in recognition of their contribution to data collection, subsequent data analysis and the preparation of this paper.

Table 1. Main purpose of long-running telemedicine networks delivering humanitarian services.

Network	Main purpose
Africa Teledermatology Project	Teledermatology in Africa
ITM Telemedicine	Advice for health-care workers treating HIV infection in Africa
Pacific Island Health Care Project (PIHCP)	Second opinions for Pacific Islanders health-care providers
Partners Online Specialty Consultations	Second opinions for rural clinics in Cambodia
RAFT	Health staff education in French-speaking African countries
Swinfen Charitable Trust	Second opinions for physicians in developing countries
Teletrauma	Advice on trauma care advice in the Ukraine

HIV, human immunodeficiency virus; ITM, Institute of Tropical Medicine; RAFT, Réseau en Afrique Francophone pour la Télémédecine.

Data collection

In September 2011, a questionnaire was sent to each contact person to collect basic information about the networks. This was followed in October 2011 by a second questionnaire designed to collect more detailed information regarding network performance and the evidence base. Each questionnaire contained 20 questions based on the framework for network performance evaluation proposed by Wootton et al.⁴ This framework takes into account the perspectives of the three main user groups – requesters (i.e. physicians requesting advice on the management of difficult cases), coordinators (i.e. network managers) and experts (i.e. those who respond to the electronic referrals) – and it identifies five performance measures of relevance to each of the three user groups. A third questionnaire was sent in February 2012 to obtain information about governance, liability and confidentiality.

Data summary

From the responses to the questionnaire, a list of the scientific papers describing the work of each network was compiled. Only papers listed in Medline and dealing with telemedicine services were included in the analysis. To summarize their contents, one author extracted information from each published paper on the following: (i) study design; (ii) effectiveness; (iii) quality; (iv) economics; (v) provision of access to care, and (vi) sustainability.

A second author, working independently, then graded each point. Study design was assessed using the nine-point scale traditionally employed to measure the strength of the evidence.⁵ For convenience, the nine levels were converted into a quality score (1 = poor, 2 = average, 3 = good), where levels I–III were considered good, IV–VI average, and VII–IX poor. The other five topics were graded subjectively and assigned a quality score based on the amount of information provided about them (0 = none, 1 = poor, 2 = average, 3 = good). Finally, the results were reviewed by all authors and changed by consensus where necessary. Hence, the papers from each network were reviewed by all the other networks, and this increased the consistency of the results.

Results

General information

The seven networks studied had been in operation from 5 to 15 years (median: 11 years), as shown in Table 2, available at: http://www.who.int/bulletin/volumes/90/5/11-099143. All the networks provide clinical tele-consultations for humanitarian purposes, and five of them were also involved in some form of education.

Table 2. General information about the telemedicine networks delivering humanitarian services.

Information	Africa Teledermatology Project	ITM Telemedicine	Pacific lsland Health Care Project	Partners Online Specialty Consultations	RAFT	Swinfen Charitable Trust	Teletrauma
First year of operation	2007	2003–2004	1997	2001	2001	1999	2000
Activities provided	Clinical/educational	Clinical/educational	Clinical/educational	Clinical	Clinical/educational	Clinical	Clinical/educational
Clinical activities
Modality for tele-consultations	Store-and-forward	Store-and-forward	Store-and-forward	Store-and-forward	Store-and-forward	Store-and-forward	Store-and-forward and videoconferencing
Clinical specialties offered	Dermatology	HIV/AIDS	All specialties	All specialties	All specialties	All specialties	Trauma, orthopaedics, neurosurgery
Equipment purchase and maintenance	Support from the American Academy of Dermatology and the Commission for Development Studies, Austrian Academy of Sciences	Belgian Development Cooperation for web site maintenance and education of participants based in resource-limited settings	US$ 250 000 start-up grant. Continued funding as part of core budget in jurisdictions and the TAMC	Revenue from other sites, block grant from Partners HealthCare	Financed by the network for 2 years, then by local funds	Charitable funds	State budget for the hospitals
Educational activities
Technology used in tele-education	Computer-based learning	Computer-based learning	–	–	Audio computer-based learning	–	Audio, video
Modality used in tele-education	Asynchronous	Asynchronous	–	–	Both synchronous and asynchronous	–	Synchronous
Clinical specialties offered	Dermatology	HIV/AIDS	–	–	All specialties	–	Trauma
Requesters
No. of requesters	Hundreds registered; approximately 50 actively submitting	Approximately 400	More than 300	10	More than 500	403	50
No. of requesting sites	15	About 80	11	4	More than 50	399 (not all are active)	7
No. of countries	13	42	9	1	15	58	5
Accreditation of requesters	Someone in the network knows them personally	Alumni of the SCART/eSCART course; physicians working in resource-limited settings for international organizations; manual approval of a membership	Approval by the medical director based on recommendations of ministers/secretaries of health and local laws and regulations of jurisdictions	None	Verification of credentials by local coordinator	Known personally to someone on the board or to a third party known by them	Have to be physicians (service not for the patients directly)
Experts
No. of experts	25	Approximately 20	100	30	50	513	15
No. of expert sites	6	6	1	14	20	502 (not all are active)	5
No. of countries	3	5	1	1	15	22	3
Location of experts	Same country that the requests are from; other industrialized countries	Other developing countries; other industrialized countries	Other industrialized countries	Other industrialized countries	Same country where the requests are from; other developing countries	Other industrialized countries	Same country where the requests are from; other industrialized countries
Accreditation of experts	Experience in dermatology in the developing world. Internal review of experts. Training	Linked with institutional collaborations; relevant work experience in resource-limited settings	Selected and approved by the medical director, vetted by the Surgeon General of the US Army, credentialed by TAMC, certified by the American Boards of Medical Specialists, and licensed by at least one of the 50 states in the USA	Practising physicians within the network subject to rules and accreditation requirements by the State of Massachusetts	Verification of credentials by local coordinator	Known personally to someone on the board or to a third party	No special requirements; clinical experience in special questions and scientific degree
Funding of experts	The consultants are volunteers	The consultants are volunteers	The consultants are volunteers	The consultants are volunteers	The consultants are volunteers; the network pays for their time	The consultants are volunteers	The network pays for their time
Coordinators
Management of requests and selection of experts	Made by requesters	Made by a coordinator	Made by the medical director	Made by both requesters and a coordinator	Made by a coordinator	Made by a coordinator	Made by a coordinator
Funding of coordinators	The coordinators are volunteers	The network pays for their time	Medical director is a volunteer	The network pays for their time	The coordinators are volunteers; the network pays for their time	The coordinators are volunteers	The network pays for their time

AIDS, acquired immunodeficiency syndrome; HIV, human immunodeficiency virus; ITM, Institute of Tropical Medicine; RAFT, Réseau en Afrique Francophone pour la Télémédecine; SCART, short course on antiretroviral treatment; TAMC, Tripler Army Medical Center; US$, United States dollars.

All networks used store-and-forward (asynchronous) methods for delivering tele-consultations, which confirms that real-time techniques are probably not appropriate in the present context. In addition, one telemedicine network used videoconferencing for consultations pertaining to trauma cases, which usually require an immediate response. Four of the networks offered tele-consultations in all clinical specialties, whereas the other three networks focused on delivering specialist services in areas such as traumatology, orthopaedics, neurosurgery, dermatology and management of patients with human immunodeficiency virus (HIV) infection or acquired immunodeficiency syndrome (AIDS). The source of funding for equipment acquisition and maintenance varied between telemedicine networks but was generally a mix of charitable funds and national or local support.

Four of the five networks delivering tele-education provided detailed information about their activities. Three networks delivered tele-education via asynchronous methods such as computer-based learning or web-based discussion forums. Two networks also used synchronous tele-education delivered through videoconferencing. Educational activities were offered by all the telemedicine networks delivering tele-consultations in specialty areas. Consistent with the clinical activities, tele-education was offered for trauma, dermatology and HIV/AIDS case management.

Organizational models were investigated through questions regarding the three main user groups. The number of requesters gives an idea of the size of each network and ranged from 10 to over 500. The number of requesters was loosely associated with the range of clinical services provided but not with the duration of network operation. The number of referring sites ranged from 4 to 399, and the number of countries ranged from 1 to 58. The number of sites and countries where requesters were based was roughly proportional to the number of requesters. Methods of requester accreditation – the licensing necessary to perform a clinical consultation – were different between the seven telemedicine networks, as detailed in Table 2. Three of the networks had a formal requester accreditation process; others used a more informal mechanism, consisting, for example, of personally knowing the physicians involved. One network claimed to have no requester accreditation process at all.

The smallest network had a total of 15 experts and the largest had 513. The number of sites where these were located varied greatly, from 1 to 502, and the number of countries ranged from 1 to 22. Differences in organizational models were evidenced by the large differences in the number of physicians using the telemedicine networks. One network indicated that not all requesters and experts were active because of constant staff turnover. This was not surprising, especially for big networks registering hundreds of physicians. All seven telemedicine networks had experts based in other industrialized countries. Two networks also had experts located in the countries where the requesters were based, and one network had experts from other developing countries. All the telemedicine networks had a process for accrediting the experts, as detailed in Table 2. Methods of accreditation included state licensure, formal credentials, clinical experience and training. Finally, in six telemedicine networks the experts were working as volunteers. Only two networks paid experts for the time they spent delivering tele-consultations.

The management of requests and the selection of the experts responsible for answering them were done by a coordinator in six of the seven networks. Thus, the coordinator appears to be a key element of the networks’ organizational model. In one network, this activity was performed entirely by the requesters themselves, whereas in another network the requesters were supported in the process by a coordinator. The coordinators and the experts were funded differently; in three of the seven networks coordinators were volunteers, whereas in the other four networks they were paid for their time.

Network performance

Network activity ranged from 50–400 tele-consultations per year. In 2010, the networks managed an average of 209 cases. In 2006–2011, three networks showed a positive trend over the 5-year period, three showed a negative trend, and one showed stable activity. On average, network activity increased by about 10% each year.

The average time to first reply to a request is an important performance measure affecting the value of the clinical tele-consultations to the referrer. Estimates from the last 12 months of operation showed that most networks took an average of 24 hours (range: 5.6–72 hours) to provide a first reply to a request. Unanswered replies to requests in 2010 were very few, particularly compared with network activity. Moreover, some requests could not be processed; in some cases, for instance, inadequate images were submitted and the referrer was unable to provide satisfactory alternatives. Overall, almost all requests were answered promptly, i.e. within 48 hours. Another performance measure is the possibility of a dialogue between requesters and experts. This feature was available for all seven telemedicine networks. On the other hand, the experts were not always informed of individual patient outcomes. In 2010, the number of cases managed by a network whose individual outcomes were fed back to experts varied from none in some networks to all in others.

Little quantitative data were available on the educational activities conducted by the telemedicine networks since their establishment. Data from the RAFT [Réseau en Afrique Francophone pour la Télémédecine] network in sub-Saharan Africa showed an increase in the number of hours of tele-education delivered each year. The telemedicine network of the Institute of Tropical Medicine (ITM) in Antwerp, Belgium, offered online learning and web-based discussion forums, but contact hours could not be quantified.

Scientific output

By the time of the survey, the seven networks had published a total of 59 papers; 44 that dealt with telemedicine and that were indexed in Medline were included in the present study.^6–49 The scientific output represented by the 44 papers is summarized in Table 3. Information about study design was available for all papers. In particular, 29 papers represented non-controlled clinical series, with the number of patients observed ranging from a dozen to about 2000. The remaining 15 papers were anecdotes or case reports. Thus, all papers provided evidence classified as poor in strength. Evidence pertaining to programme sustainability and improved access to care was provided by more than half of the papers. One fourth of them also covered quality and effectiveness, while only a few provided any evidence on network economics. Overall, the quality of the scientific output was poor to average.

Table 3. Quality of the scientific output of the telemedicine networks delivering humanitarian services, based on 44 papers.

Measure	Study design	Effectiveness	Quality	Economics	Access to care	Sustainability
No. of papers providing evidence	44	11	14	6	28	25
Median quality scorea	1	1	1	1	1	2
Range of quality scores	1–1	1–2	1–2	1–2	1–2	1–2

^a 0 = none; 1 = poor; 2 = average, 3 = good.

Other factors

Other factors relevant to the operation of each telemedicine network are summarized in Table 4, available at: http://www.who.int/bulletin/volumes/90/5/11-099143. Although governance varied across the seven networks as a function of differences in organizational models and contexts, medical liability and patient confidentiality were handled in similar ways. The factors concerning sustainability were generally different between networks, and included institutional anchoring, organizational models, technical and clinical solutions, clinical quality and benefits to patients, exit strategy, technology and funding. Uncertainties in funding were a common risk factor. Others had to do with the availability of coordinators, the training of experts, a lack of infrastructure and equipment and similar problems. Questionnaire respondents gave their views about the future plans of their telemedicine networks. These were similar across networks and had to do primarily with expansion to other countries and with the engagement of additional experts. Collaboration with other telemedicine networks was also listed as a feasible and useful plan for the future.

Table 4. Other features of the telemedicine networks delivering humanitarian services .

Factor	Africa Teledermatology Project	ITM Telemedicine	Pacific lsland Health Care Project	Partners Online Specialty Consultations	RAFT	Swinfen Charitable Trust	Teletrauma
Governance	Site managed by a core group. IT support from the Medical University of Graz; clinical work from Carrie Kovarik	Funding for the project was obtained through a grant from the Directorate General for Development Cooperation	The project was established in 1990 by TAMC and is sponsored by the US Congress. It is codified in Federal Law and governed by a medical director	The network is managed by the Center for Connected Health, a non-profit teaching hospital. There are two dedicated coordinators and one corporate manager	The project is hosted at the division for e-health and telemedicine at Geneva University Hospitals. The focal points in each participating country are responsible for the national governance	The organization is constituted as a registered charity under United Kingdom law. The management board meets every 3 months. Financial accounts and reports are filed every year	General management is by the Regional Trauma Hospital. The methods are according to the Ukrainian Telemedicine and E-Health Development NGO
Medical liability	Reliance on the “Good Samaritan” clause; liability not completely eliminated	Experts belong to the ITM or partner institutions	Consultants are located at TAMC, and are licensed, credentialed and vetted. Requesters have liability until patients arrive at TAMC	Physicians sign a terms and conditions statement when referring patients to the network	It is the referring physician’s responsibility to apply the expert’s advice. The identity and credentials of experts are verified by the national coordination team	It is a not-for-profit telemedicine service. United Kingdom medical insurers cover doctor-to-doctor advice according to the so-called “Good Samaritan” clause	Teletrauma works within the health- care system in Ukraine. Consultations are part of the physicians’ duties and are free for patients
Patient confidentiality	Data held on a secure server; access to cases only permitted by requesters and experts. Use of full-face photos, names or dates of birth is discouraged. Patient consent is obtained	Referrers sign a “policies agreement” and a disclaimer is made available on the web site. The second-opinion advice is free of charge	Patient Referral Form; informed consent document. Data are stored in a secure, password protected database. HIPAA rules, privacy impact statements and Privacy Act System of Record Notice	Data are uploaded to a secure web site. An encryption service is used for e-mails	The latest version of the tele-expertise tool uses public–private keys to encrypt information and ensure traceability of access	Data are stored on a secure, password-protected server, and can only be accessed via encrypted connections	Patient consent is obtained. Anonymized data are stored in a secure server
Sustainability factors	Motivation, personal relationship and trust of requesters and experts	Institutional project	Provision of travel and definitive medical care for indigenous persons at no cost to the patient or to the jurisdiction; funding included in core budget to support graduate medical education	Teleconsultations are also commercially available to patients in other developed countries. This produces the financial margin necessary to sustain this programme in Cambodia	Institutional anchoring; clear exit strategy	Core group of retired/semi-retired board members	Clear technical and organization solutions; clear methodology of clinical usage; quality of recommendations
Risk factors and challenges	Work on minimal funding but need of some continued funding for web site maintenance	Linkage to other institutions in the field	Lack of infrastructure, technical and medical expertise, deteriorating equipment. Inadequate financial resources; inadequate administrative, logistical, and ancillary support	Market demand in developed countries; capacity to offer free consultations to patients in Cambodia; lack of training of local experts in Cambodia	Cost of Internet connectivity	How to increase the pool of coordinators	Introduction of telemedicine into clinical protocols in trauma and orthopaedics
Future plans	Reach new locations and countries; engage the few dermatologists in the African countries where consults are provided to become experts; expand the educational activities	Collaboration with other networks, under an international umbrella	Continue to improve access to care, expedite referrals/consultations, and continue to mine the PIHCP database for education and training	Train local experts to take over tiers 1 and 2 of severity of incoming requests; expand to other countries (China, India), with local non-profit partners	Expand within countries to reach district hospitals throughout Africa (as well as pilot projects in Latin America)	Become part of a network of networks	Telemedicine has to be introduced into clinical protocols in trauma and orthopaedics

HIPAA, Health Insurance Portability and Accountability Act; ITM, Institute of Tropical Medicine; NGO, nongovernmental organization; PIHCP, Pacific Island Health Care Project; RAFT, Réseau en Afrique Francophone pour la Télémedecine; TAMC, Tripler Army Medical Center.

Discussion

The present study summarizes the experience gained to date with existing telemedicine networks that deliver humanitarian services. All of the seven well established, long-standing networks studied provided reasonable evidence that they were improving access to care in the developing world. However, the overall quality of the scientific output emanating from these networks is still rather weak. This applies to study design and to the evaluation of other important parameters, including effectiveness, service quality and economics. Stronger evidence is therefore needed to increase the appropriate use, scale and impact of telemedicine in resource-limited settings.⁵⁰ This need for stronger evidence underscores that more and better evaluations need to be conducted.³ Given their size and relative success, long-running telemedicine networks should be the subject of controlled evaluations in future.

Another relevant finding from the study pertains to network performance. Measuring the performance of a telemedicine network is essential for understanding whether the network is working as intended or having the desired effect. By adapting a recently developed framework for network performance evaluation,⁴ we documented the seven telemedicine networks’ performance and the services they provided. Differences in services and performance could be explained by different organizational models.

Notwithstanding the use of different organizational models, clinical case load was strikingly similar across networks: all seven networks were providing only a few hundred tele-consultations annually. This activity level may stem from the fact that the networks are run by a single individual or a small number of committed enthusiasts or “clinical champions”. Since the present networks collectively appear to meet only a tiny fraction of the potential demand from the developing world,⁵¹ one may wonder why their activity levels are not increasing rapidly. Although the reasons could be many, small-scale organizational models may be one. Future work might therefore be directed at investigating new organizational models that would facilitate large-scale network operation. Improved collaboration between existing networks may prove beneficial as well, since it would attenuate the lack of resources reported by some networks and improve sustainability.

The findings of the present study have two main implications. First, telemedicine networks delivering humanitarian services appear to be sustainable – at least as operated to date – and they deliver clinically useful services. Second, the evidence summarized in this paper, albeit weak, may be useful to decision-makers. It may, for instance, encourage ministries of health in developing countries to establish, support or join similar telemedicine networks.

The present study has several limitations. For one thing, the list of networks studied may not be exhaustive; other long-running telemedicine networks around the world may also be delivering humanitarian services. However, we believe that we have covered the main active networks. Furthermore, the study was not a systematic review and the assessment of the quality of the scientific studies emanating from the networks was necessarily subjective. Moreover, we examined only successful networks (and arguably, experience from unsuccessful networks may be equally informative). Finally, the experience of the telemedicine networks was reviewed by people responding on behalf of the networks and may reflect reporting bias.

The present study emphasizes the need to generate stronger evidence and more and better evaluations of telemedicine networks and their effectiveness in improving outcomes and access to health care. Future research should address these topics. Nonetheless, the present study provides reasonable grounds for supporting the future expansion of telemedicine networks offering humanitarian services in developing countries.