Abstract
Introduction: The effectiveness of any new technology is typically measured in order to determine whether it successfully achieves equal or superior objectives over what is currently offered. Research in telemental health—in this article mainly referring to telepsychiatry and psychological services—has advanced rapidly since 2003, and a new effectiveness review is needed. Materials and Methods: The authors reviewed the published literature to synthesize information on what is and what is not effective related to telemental health. Terms for the search included, but were not limited to, telepsychiatry, effectiveness, mental health, e-health, videoconferencing, telemedicine, cost, access, and international. Results: Telemental health is effective for diagnosis and assessment across many populations (adult, child, geriatric, and ethnic) and for disorders in many settings (emergency, home health) and appears to be comparable to in-person care. In addition, this review has identified new models of care (i.e., collaborative care, asynchronous, mobile) with equally positive outcomes. Conclusions: Telemental health is effective and increases access to care. Future directions suggest the need for more research on service models, specific disorders, the issues relevant to culture and language, and cost.
Key words: telepsychiatry, effectiveness, telemental health, videoconferencing, telemedicine
Introduction
Telemental health, a use of telemedicine to provide mental health assessment and treatment at a distance, enters its sixth decade as a well-known practice in the medical field—it has increased access to care, and patients and providers are very satisfied with it for a wide variety of services.1 In this article, we used the term “telemental health” to refer to telepsychiatry and other psychological services, as the term has been used in social science and other fields as well. The American Telemedicine Association (ATA) has published telemental health practice guidelines,2 as has the American Association of Child and Adolescent Psychiatry.3 A new generation of studies on telemedicine has replaced the “primary” view of telemental health as a new and different way of providing health services to a contemporary view that it is a vehicle for providing care that is here to stay. The studies supporting this contemporary view have examined the effectiveness of telemental health to answer the question “Is telemental health ‘effective’ to do ‘what’ for ‘whom’ and ‘when’ at this point in time, based on its evolution?”
Effectiveness implies that telemental health works. In telemedicine and telemental health, few authors have explicitly addressed effectiveness4; however, research appears to be changing this.5 The underlying premise of being “effective” is the assurance that the chosen technology is specific to the objective of the service being offered.6
Effectiveness needs to be considered from the perspective of the patient, provider, program, community, and society as a whole. The only previous review of telemental health's effectiveness considered it effective in terms of providing access, improving basic outcomes, and being well-accepted.4 Telemental health was judged to have broad utility for clinical disorders, facilitated empowerment of patients, and had good educational outcomes. Today, its effectiveness is better described in terms of the model of telepsychiatry used7,8 and the population being served (e.g., rural, underserved, children).
This article discusses telemental health's effectiveness related to clinical care. There is a review of diagnostic (reliability/validity) or assessment processes, populations (child, geriatric, and ethnic), new models, settings (e.g., collaborative care, asynchronous, emergency, home health), mental health disorders, and cost-related and other outcomes. Recommendations for further effectiveness studies will be offered, and future directions for telemental health services will be discussed.
Materials and Methods
A comprehensive review of the telepsychiatric literature was conducted in the MEDLINE, PubMed, PsychInfo, Embase, Science Citation Index, Social Sciences Citation Index, Telemedicine Information Exchange databases, Centre for Reviews and Dissemination, and The Cochrane Library Controlled Trial Registry databases for the period of July 2003 to March 2013. (The previous review4 covered 1965 to June 2003). The Journal of Telehealth and Telecare was also manually searched for those years when it was not on MEDLINE. Key words included telepsychiatry, telemental, health, telecare, telemedicine, e-health, videoconferencing, effectiveness, efficacy, access, outcomes, satisfaction, quality of care, rural, mental health, cost, children/child, cultural/culture, geriatric, population, home health, medical home, emergency, face-to-face, in-person, reliability/validity, and international; the term “in-person” will be used rather than “face-to-face” in this article.
Article titles and abstracts were reviewed by the authors to see if they were applicable to the theme of effectiveness. Data on effectiveness appear in a wide range of range of case studies, case series, project descriptions, and program evaluations to more formal research trials.5 Selected articles were pulled, and their references were reviewed to identify additional articles that may have been missed by the keyword search. In total, 755 articles were initially reviewed for this article, with 670 excluded because of little information/data on effectiveness. Although more reviews of the topic or related topics would have been interesting, 15 were chosen as most salient; this left 70 actual studies. Interventions like education, medication management, and most of the therapies were excluded by the words searched.
Effectiveness, overall, was determined on the basis of clinical parameters, the beneficial effects of a program or policy under optimal conditions of delivery, and other data under more real-world conditions.9 This differs from evidence of efficacy ratings, which is traditionally organized as A (best) to F (least). A key component of effectiveness is feasibility and/or replicability or adaptation to other settings (also known as disseminability). Clinical research trials usually assess effectiveness compared with in-person service, preferably with a design that is randomized. Tips for program effectiveness4 have been updated and are summarized in Table 1; this compilation, however, is not based on research or analysis of studies.
Table 1.
Measures |
Starting points |
Case report, series, or mix of patients |
Project or program description |
Qualitative analysis: impressions, perceptions, or information to form additional questions |
Cost, cost comparison, or cost offset, often of “direct” costs |
Project or program evaluation, sometimes retrospective |
Small(er) total n |
Micro- (e.g., one party) analysis |
Some control of variance or limited interplay of variables |
Cross-sectional analysis |
Goals |
Prospective, question-based |
Comparison group |
Study design “same as” or “equal to” |
Noninferiority trials |
Study design randomized controlled trial |
Cost-effectiveness, -benefit analysis with computations of direct and indirect costs |
Evaluation that “drives” the objectives and prospectively collected |
Generally, large(r) total n (but not always guarantee “good” study) |
Micro- (all parties individually) and macro- (system-wide=patient, provider, clinic, health system, community, and other parties) analyses |
Analysis of variance |
Longitudinal analysis |
Access |
Increased access to care |
Improved level of, or quality of, existing care |
Specific to the need (e.g., consultation–liaison rather than management [only] to primary care) |
Complements or integrates service delivery (or prevents use of more intensive or costly service) |
Quality of care |
Reliable/valid |
Diagnosis and assessment |
Detection of limitations and process to “control” for them is delineated |
Improved level of, or quality of, existing care |
Specific to the need (e.g., consultation–liaison rather than management [onl] to primary care) |
Complements or integrates service delivery (or prevents use of more intensive or costly service) |
Population |
Setting |
Satisfaction and related intangibles (e.g., empowerment) |
Costs |
Technology |
Adequate description of equipment, bandwidth, frames per second, and other parameters |
Data on failures, problems (i.e., reliability) |
Time, effort, and other “hidden” costs of “new” technologies (e.g., asynchronous telepsychiatry) |
Administration |
Feasibility |
Level of coordination to initiate, maintain, and financially support |
Measures of Effectiveness
Diagnosis (Validity and Reliability) and Assessment (Tables 2 and 3)
Table 2.
TOPIC, STUDY | N | PATIENT POPULATION | KBS | LOCATION | COMMENT(S) |
---|---|---|---|---|---|
Geriatric | |||||
Lyketsos et al. (2001) | NAP | Geriatric outpatients | NS | United States | Video reduced “unneeded” hospitalizations. |
Poon et al.33 (2005) | 22 | Geriatric dementia patients | 1.5 Mb | China | Significant, comparable cognitive improvement in video and in-person; high satisfaction; feasible assessment, intervention, and outcomes |
Rabinowitz et al.32 (2010) | 106 | Nursing home residents | 384 | United States | Reduced travel time, fuel costs, physician travel time, personnel costs |
Weiner et al.35 (2011) | 85 | Adult and geriatric dementia patients | NS | United States | Feasible alternative to face-to-face care in patients with cognitive disorders who live in remote areas |
Adult | |||||
Graham et al. (1996) | 39 | Adult outpatients | 768 | United States | Video reduced “unneeded” hospitalizations. |
Zaylor et al. (1999) | 49 | Adult depressed or schizoaffective outpatients | 128 | United States | Video equals in-person in GAF scores at 6-month follow-up. |
Hunkeler et al. (2000) | 302 | Adult primary care outpatients | NS | United States | Video by nurses improved depressive symptoms and functioning and had high satisfaction versus in-person. |
Ruskin et al.16 (2004) | 119 | Adult Veterans | 384 | United States | Depression outcomes video and in-person equal, as were adherence, satisfaction, cost |
Manfredi et al.74 (2005) | 15 | Adult inmates | 384 | United States | Feasibility from an urban university to rural jail; less need for inmate transport |
Sorvaniemi et al.59 (2005) | 60 | Adult emergency patients | 384 | Finland | Minor technical problems occurred; assessment and satisfaction fine |
Modai et al.76 (2006) | 24/15 | Adult outpatients | NS | Israel | Video greater than in-person cost per service and more hospitalization cost (less available per usual care) |
Urness et al.75 (2006) | 39 | Adult outpatients | 384 | Canada | Video less than in-person for encouragement; improved outcomes for both |
O'Reilly et al.13 (2007) | 495 | Adult outpatients | 384 | Canada | Video equal to in-person in outcomes, satisfaction; 10% less expensive per video |
Yellowlees et al.53 (2010) | 60 | Non-emergency adult patients | NAP | United States | First ATP to demonstrate feasibility |
Pediatric | |||||
Nelson et al.27 (2003) | 28 | Children | 128 | United States | Video equals in-person in reducing depression over 8 weeks; satisfaction high, but 15/100 consultations had an issue with technology. |
Greenberg et al.77 (2006) | NS | Children | NS | Canada | Video experiences positive; family caretakers and service providers frustrated with limitations of the video |
Myers et al.78 (2006) | 115 | Adolescents, incarcerated | 384 | United States | 80% of youth successfully prescribed medications, and they expressed confidence with the psychiatrist's recommendations; youth expressed concerns about privacy. |
Myers et al.23 (2010) | 172 | Children and adolescents | 384 | United States | Parents' satisfaction higher with school-aged children and lower with adolescents; adherence high for return appointments |
Pakyurek et al.12 (2010) | NAV | Children/adolescents in primary care | NS | United States | Video might actually be superior to in-person for consultation. |
Lau et al.79 (2011) | 45 | Children and adolescents | NS | United States | Video reaches a variety of children, with consultants providing diagnostic clarification and modifying treatment |
Jacob et al.80 (2012) | 15 | Child outpatients | NS | United States | Patient satisfaction was high, and PCPs found recommendations helpful; outcomes pending on follow-up |
All ages | |||||
De Las Cuevas et al.14 (2006) | 130 | All ages—outpatients | 384–768 | Spain | Video equals in-person, including those in remote areas with limited resources |
Depression | |||||
Ruskin et al.16 (2004) | 119 | Adult Veterans | 384 | United States | Video equals in-person for adherence, patient satisfaction, and cost. |
Fortney et al.15 (2007) | 177 | Adult outpatients | NS | United States | Video can help adapt collaborative care model in small PC clinics, and symptoms improved more rapidly in intervention group versus usual-care group. |
Moreno et al.37 (2012) | 167 | Adult patients | NS | United States | Video may close gap in access to culturally and linguistically congruent specialists; improves depression severity, functional ability, and quality of life |
Fortney et al.9 (2013) | 364 | Adult patients | NS | United States | Video collaborative care group had greater reductions in severity than usual-care group. |
PTSD | |||||
Frueh et al.18 (2007) | 38 | Adult male Veterans | 384/NS | United States | Video equals in-person in clinical outcomes and satisfaction at 3-month follow-up; video less comfort versus in-person in talking with therapist post-treatment and had worse treatment adherence |
Morland et al.17 (2010) | 125 | Adult male Veterans | 384/NS | United States | Video CBGT for PTSD-related anger is feasible for rural/remote Veterans, with reduced anger. |
Germain et al.81 (2009) | 48 | Adult patients | NS | Canada | Video equals in-person in reducing PTSD over 16–25 weeks |
Substance abuse | |||||
Frueh et al.46 (2005) | 14 | Adult male outpatients | 384/NS | United States | Video had good attendance, comparable attrition, and high satisfaction. |
Developmental disability | |||||
Szeftel et al.30 (2012) | 45 | Adolescents | NS | United States | Video led to changed Axis I psychiatric diagnosis (excluding developmental disorders) 70%, and changed medication 82% of patients initially, 41% at 1 year, and 46% at 3 years; video helped PCPs with recommendations for developmental disabilities. |
Panic disorder | |||||
Bouchard et al.82 (2004) | 21 | Adults | 384/NS | Canada | Video 81% of patients panic-free post-treatment and 91% at 6-month follow up via CBT |
Hispanic | |||||
Moreno et al.37 (2012) | 167 | Adult patients | NS | United States | Video lessens depression severity, raises functional ability and quality of life, and improves access to culturally and linguistically congruent specialists. |
Chong et al.40 (2012) | 167 | Adult patients | NS | United States | Video is acceptable to low-income depressed Hispanic patients, but its feasibility is questionable. |
Yellowlees et al.55 (2013) | 127 | English- and Spanish-speaking patients | NS | United States | ATP equal for English- and Spanish-speaking patients |
American Indian | |||||
Shore et al.43 (2008) | 53 | Male adult patients | NS | United States | Video equals in-person assessment, interaction, and satisfaction; comfort level high and culturally accepted |
European | |||||
Mucic44 (2010) | 61 | Adult outpatients | 2Mbit (Denmark)10Mbit (Sweden) | Denmark | Video improved access, reduced waiting time, and reduced travel to see bilingual psychiatrists; high satisfaction video preferred via “mother tongue' rather than interpreter-assisted care |
Asian | |||||
Ye et al.41 (2012) | 19 | Adult outpatients | NS | United States | Primary language facilitates expression of feelings, emotional discomfort, or social stressors. |
Sign language | |||||
Lopez et al.45 (2004) | 1 | Adult female, deaf since birth | NS | United States | Video communication was fine with ASL interpreter, and psychiatric symptoms improved. |
Those studies before 2003 are not referenced in this regular article since it is not a review; name and year of those not referenced are given in Hilty et al.4 (2003).
ASL, American Sign Language; ATP, asynchronous telepsychiatry; CBT, cognitive behavioral treatment; NAP, not applicable; NAV, not available; NS, not specified; PC, primary care; PCP, primary care provider; PTSD, posttraumatic stress disorder.
Table 3.
COST | N | PATIENTS | KBPS/FRAMES | LOCALE | COMMENTS |
---|---|---|---|---|---|
Mielonen (1998) | 14 | Adult inpatients | NS | Finland | Savings in healthcare costs, reduction in travel, and ease and speed of consultation |
Trott (1998) | 50 | Adult and child outpatients | NS | Australia | Substantial savings in healthcare costs from reduction in traveling and patient transfers |
Alessi (1999) | NAV | Adult forensic inpatients | NAV/NAV | United States | Video is cost-effective. |
Doze (1999) | 90 | Adults | 336–384/NS | Alberta | Costs break even at 7.6 consultations. |
Simpson (2001) | 379 | Adult outpatients | 128–384 | Canada | Costs break even at 224 consultations/year; less if used for administration, too |
Elford (2001) | 30 | Children and parents | 336 | Newfoundland, Canada | Cost $400 per consultation via video or by patient traveling |
Hailey (2002) | NAP | Adults | NAP/NAP | United States | Reduced costs to rural patients |
Edwards et al.83 (2003) | 518 | Adults and children | United States | Video saved $400/consultation versus in-person | |
Jong68 (2004) | 71 | NS | NS | Canada | Video saved $2,000/consultation versus in-person and saved government $140,088 in 2003. |
Ruskin et al.16 (2004) | 119 | Adult Veterans | 384/NS | United States | Video greater than in-person unless psychiatrist traveled >22 miles away, and the productivity (increasing number of patients/day) minimized costs. |
Cluver et al.63 (2005) | 10 | Adult outpatients | NS | United States | In-home portable video works but costly for the average person |
Persaud et al.69 (2005) | 215 | Adults | NS | Canada | Video versus in-person of equal cost, overall, as patient costs more for in-person literal consultation $240–$1,048 (Canadian $) versus telehealth $17–$70, but from societal perspective, video costs more at $1,736–$28,084 versus in-person $325–$1,133. |
Harley84 (2006) | 11 | Adults | 128 | United Kingdom | Video in rural areas costs 4 times less than in-person, once a threshold of 5–6 episodes per year is completed. |
Modai et al.76 (2006) | 24/15 | Adult outpatients | NS | Israel | Operational video costs greater than in-person, particularly if resulted in hospitalization (223.7% higher); video costs of sessions 32% higher unless travel included (then only 10.6% higher) |
O'Reilly et al.13 (2007) | 495 | Adult outpatients | 384/NS | Canada | Video 10% less expensive per patient than service provided in-person |
Shore et al.42 (2007) | 53 | Adults | 384/NAP | United States | Video costs lower than in-person |
Smith et al.66 (2007) | 1,499 | Children | NS | Australia | Video cost about $600/consultation versus $1,000+in-person |
Spaulding et al.67 (2010) | 257 | Children/adolescents | 384+? | United States | Video cost $168/consult more but only $31 if travel costs included |
Rabinowitz et al.32 (2010) | 106 | Nursing home residents | 384/NS | United States | Reduced travel time, fuel costs, physician travel time, personnel costs |
Pyne et al.85 (2010) | 395 | Adult outpatients | NS | United States | Video $85,634/QALY for collaborative care |
Butler and Yellowlees54 (2012) | 125 | Adult primary care patients | ATP5 | United States | ATP and video fixed costs $7,000 and $20,000, respectively, and per consultation ATP was $68.18, video was $107.50, and in-person $96.36; this means ATP is most cost-effective at 249 consultations/year. |
Those studies before 2003 are not referenced in this regular article since it is not a review; name and year of those not referenced are given in Hilty et al.4 (2003).
ATP, asynchronous telepsychiatry; KBPS, kilobits per second; NAP, not applicable; NAV, not available; NS, not specified; QALY, quality-adjusted life years.
Studies of telemental health's reliability and validity started with 128–384 kilobits per second (Kbps) and now occur at 384+Kbps; these of course do not apply to asynchronous and other telephonic options. Diagnoses have been made reliably, with good inter-rater reliability, for a wide range of psychiatric disorders in children, adolescents, and adults; less information is available on geriatric patients, but preliminary results are positive. Limitations have been largely overcome, including patients' difficulties in hearing, concentration, and attention; some rural areas that lack access because of line or satellite technology are more restrictive, but patients often travel to a nearby site.
A wide range of scales has been studied for adults and children/adolescents via videoconferencing, as reviewed by Yellowlees et al.2 for the ATA and Richardson et al.5 These include the Brief Psychiatric Rating Scale (BPRS), Scales for the Assessment of Negative and Positive Symptoms (SANS and SAPS, respectively), the Structured Clinical Interview for the Diagnostic and Statistical Manual (DSM) (SCID), Hamilton Depression Rating Scale (HDRS), Diagnostic Interview Schedule (DIS) (initially by telephone), the Abnormal Involuntary Movement Scale (AIMS), and the Yale Brown Obsessive Compulsive Scale (semistructured) (YBOCS). For children/adolescents, the DSM-IV, Schedule for the Assessment of Depression and Schizophrenia (K-SADS), and DIS (DISC) have been used. The Geriatric Depression Scale (GDS) and many neuropsychiatric scales like the Mini-Mental Status Examination (MMSE), CAMCOG (neuropsychiatric test, computerized), National Adult Reading Test, Quick Test, and Adult Memory and Information Processing Battery are effective. The reliability and validity of asynchronous telepsychiatry has been shown using English and Spanish versions of the SCID and Mini-International Neuropsychiatric Interview (MINI).
Comparison With In-Person Care
Since the last review,4 studies have compared many parameters using traditional comparison and noninferiority studies.5,10 Some have noted that with some populations (i.e., children and adolescents), telepsychiatry may be better than in-person services because of the novelty of the interaction, direction of the technology, the psychological and physical distance, and the authenticity of the family interaction.11 Reports have also included reduced length of hospitalization,12,13 better medication adherence,12,14 symptom reduction of disorders,12–15 and effective therapy such as using evidence-based treatments for posttraumatic stress disorder, including group cognitive processing.16–18
Specific Populations: Child, Geriatric, and Those of Culture
The feasibility, acceptability, and sustainability of telemental health for children and adolescents have now been shown,19,20 and it has been hypothesized that this approach may be better for some disorders, such as autism-spectrum patients, than in-person care.11 A qualitative study of young people's perspectives on receiving telepsychiatric services revealed that the sessions were helpful, they felt a sense of personal choice during the consultation, and they generally liked the technology.21 Attention-deficit hyperactivity disorder treatment by telepsychiatry3,22–24 has been actively studied, and, once again, satisfaction is high among all parties in a variety of settings.22,23
Child research in telemental health has progressed into new areas20 like randomized trials and Web-based data systems, with work from adults being replicated. Diagnosis appeared to be reliable in early studies,25,26 demonstration of clinical improvement with the use of cognitive behavioral treatment for depression followed,27 and then primary care patients treated by telemental health showed improvement in terms of depression and subscales of the Child Behavioral Check List.28,29 Psychiatric consultation leads to newly diagnosed anxiety or mood disorders in almost one-third of patients seen and a change in the patient's medication for 82% of patients at initial assessment, 41% at Year 1, and 46% at Year 3.30 Collaborative care for adolescent depression is under evaluation.31
In terms of geriatric services, the benefits of telepsychiatry are emerging from neuropsychiatric studies (see above) and a few clinical studies. Preliminary studies in nursing homes have mainly focused on depression or dementia, with telemental health evaluation judged as more facile and efficient in terms of the use of consultant time.32 Assessment, cognitive intervention, and outcomes appear to be similar to in-person results.32 Telepsychiatry to a rural geropsychiatric inpatient unit yielded positive results in terms of satisfaction compared with in-person care34 and in a 5-year study of patients referred for evaluation of potential cognitive impairment, 55%, 14%, and 12% had Alzheimer's disease, another psychiatric illness, or mild cognitive impairment, respectively.35
Ethnicity, culture, and language issues affect health,36 and there is often inadequate access to specialists37—inroads to patient needs and preferences that can be met by telemental health are progressing. A recent study of nearly 40 rural health clinics compared impressions of 25 primary care providers (PCPs) and 32 staff impressions of factors important to care: using providers who value differences (5.4 and 7.0, respectively), quality of the provider's care (4.9 and 7.0, respectively), access to care in general (4.5 and 7.0, respectively), and availability of trained interpreters for use with patients (4.4 and 7.0, respectively).38 Others are studying the specific needs of Hispanics/Latinos,37,39,40 Asians,41 Native Americans,35,43,44 Eastern Europeans,44 and those using sign language45—all using telepsychiatry for service provision. With patients of different cultural backgrounds, using the patients' primary language allows for a more comfortable atmosphere where they may express their genuine feelings and emotions.
Summary of Outcomes for Age/Population and Specific Disorders (Table 2)
Results are encouraging, overall. Videoconferencing appears to be as effective as in-person care for most parameters, such as feasibility, outcomes, age, and satisfaction with a single assessment and consultation or follow-up use. Illnesses studied have been depression,9,15,31 posttraumatic stress disorder,16–18 substance use,46 and developmental disabilities.30
Models and Settings of Telemental Services
Consultation to primary care versus management
Past studies showed positive outcomes for patients when using a consultation model of care into primary care sites. Specialists changed the diagnosis and medications in 91% and 57% of cases, respectively, with primary care interventions led to clinical improvements in 56% of cases.1 Provider knowledge, skills, and complexity of questions improve over time,47 particularly in rural PCPs.48 The most intensive model of consultation to primary care is collaborative care, which has now been more formally applied to telemedicine9,15,31 with encouraging results. The virtual collaborative care team was able to produce better outcomes than the traditional “gold standard” methodology of primary care psychiatry.9
Models of care have been thoroughly studied and well articulated.7 Examples are:
1. Randomized controlled trial for depression in adults, using disease management and telepsychiatric consultation versus usual care over 12 months.49
2. Phone and e-mail physician-to-provider consultation system for adults and children with developmental disabilities, using a 24-h warm-line.50
3. An integrated program of mental health screening, therapy on site, and telepsychiatric consultation (phone, e-mail, or video), with continuing medical education and training on screening questionnaires.28,29
4. Cultural consultation to rural primary care using telemedicine.38
5. Disaster response to a bioterrorism attack was evaluated as feasible in terms of training and consultations.51
6. Collaborative care via telepsychiatry is co-provision of medication for primary care patients by the telepsychiatrist and PCP in rural communities, based on the earlier models of in-person care to achieve national standards of antidepressant prescriptions.9,52 This model is often integrated with stepped models of care, which, similar to the above, use “less intensive or expensive interventions” first; then if patients fail to improve, “step it up” to more intensive services.
7. Asynchronous telepsychiatry (ATP) to primary care (described below) is feasible and helpful (described below).53,54
ATP
ATP services, formerly known as store-and forward services, have been demonstrated to be feasible, valid, and reliable in English- and Spanish-speaking patients in primary care.53–55 Asynchronous telemedicine is used in radiology, dermatology, ophthalmology, cardiology, and pathology, and it is now available in psychiatry, where it may also facilitate the “medical home,” a patient-centered approach that supports the PCP to improve patient care and health.56
ATP works at the patient end via taping a videorecording of local providers and patients, use of a basic questionnaire, and uploading of videos and patient histories for a remote psychiatrist for review in a Health Insurance Portability and Accountability Act (HIPAA)-adherent manner.57 He or she evaluates the information, diagnoses the patient, and makes two or three treatment recommendations in a report. ATP is specifically designed for patients who can be primarily managed in primary care, but could offer the opportunity for PCPs to collaborate with psychiatrists to provide specialized care, and it is less costly than video and usual care.53
Emergency room telepsychiatry
Telepsychiatry emergency services have been slow to develop in psychiatry compared with neurology (e.g., stroke), obstetrics (e.g., fetal monitoring), and other clinical areas. This is surprising despite the consultation models used and the long delays before mental health evaluation may occur on site. The effectiveness of emergency telepsychiatric consultations has rarely been studied; however, one study of patients with mainly depression, bipolar disorder, and schizophrenia revealed that 65% were discharged, 16% were admitted, and 19% were transferred.58 This study, which examined eight programs, found that most rated themselves as moderately successful (3/5 or 4/5, with 5 best) and patients and emergency physicians rated services at 4.4/5. The same was found in another study.59 Guidelines on how to be effective in providing emergency telepsychiatry need to be evaluated.60,61
Medical home, home health, and other mobile technology methods
These services are in development and need to be better studied, although costs are dramatically decreasing. The patient-centered medical home is a concept founded on the presence of inadequate treatment in primary care and/or an inability to access needed services.56 The patient-centered medical home allows telepsychiatric input at home, still under the general purview of the PCP, and it has been shown to improve patient care and health.62 Desk-mounted video systems offer great convenience for therapy to cancer patients to avoid travel, but the cost used to be prohibitive for most consumers.63 Internet-based video technology via personal computers and mobile devices must be HIPAA-adherent. Use of these technologies is increasingly becoming available and will support the move of telepsychiatry to the home, such as programs that are now being implemented by the Veterans Health Administration.64
Access to Care
Access appears to have been greatly increased, based on the recent decade's research—with a few exceptions. Patients may have less travel, absence from work, and time waiting, more clinical choice and control, and better outcomes, as summarized previously.4 Satisfaction, generally, with services is so high that it de facto precludes study. Rarely do patients report a less satisfactory interaction by videoconferencing than in-person. A few access-to-care issues remain unresolved for patients: (1) privacy and confidentiality where some patients prefer services delivered from elsewhere (e.g., living on a reservation or wanting total anonymity for personal reasons), (2) cultural and language nuances related to telemental healthcare, and (3) inadequate payment for indigent, rural, and other underserved patients. PCPs and communities are generally happy to “keep” their patients locally for continuity of care.
Cost Issues and Implications
Ideally, costs should be considered for patients, clinics, providers, and society at large—with both direct and indirect costs accounted for. Direct costs include equipment, installation of lines, and other supplies. Fixed costs also include the rental cost of lines, salary and wages, and administrative expenses. Variable costs include data transmission costs, fees for service, and maintenance and upgrades of equipment. Costs may also include projections for travel, transfers in emergencies, waiting times, and more “appropriate” use of other services or, more globally, by rural towns retaining dollars that would have been otherwise lost to suburban centers upon referral.
With regard to cost, there is benefit to delineate between differing types of cost analyses.65 The cost-offset model, which implies treating mental conditions may reduce other health costs, is widely used. Cost-minimization analysis implies the same effectiveness model, but different (lower) costs. Cost-effectiveness assesses intervention costs versus alternative expenditures; a subtype is cost-utility analysis, which includes data on health-related quality-of-life measures (i.e., quality-adjusted life-years). Cost-benefit analysis values all outcomes by translating them into economic terms to the degree possible and is particularly important when an intervention appears far too expensive at face value (or cross section) but not longitudinally (e.g., a transplant helps someone live and work an additional 50 years; this calculation gets into quality of life-years analysis).
Cost studies have differences in data sought, their collection, and how they are analyzed (Table 3). Videoconferencing may be cost-effective if someone does not have to travel or transfers as “expensive” services are avoided. Savings may be shown versus in-person with high consultation rates (e.g., 1,500 consultations total),66 “break-even” or other thresholds used (e.g., number of consultations/year), or when the patient's travel, time, and food are included.66,67 A break-even analysis is highly specific to a program, with a range of consultations needed, from 7 to 774 depending on methods of calculation.66 A comparison of ATP, video, and in-person showed fixed costs were $7,000 for ATP and $20,000 for regular video, and the cost per consultation was $68.18 for ATP, $107.50 for videoconferencing, and $96.36 in-person; this means ATP is most cost-effective at 249 consultations/year.54 Governments have been tabulating savings, too,68,69 and an economic evaluation of telehealth data collection with rural populations has been completed.70
Conclusions, Implications, and Recommendations for Future Research
Today, telemental health services are unquestionably effective in most regards, although more analysis is needed. They are effective for diagnosis and assessment, across many populations (adult, child, geriatric, and ethnic), and in disorders in many settings (emergency, home health), are comparable to in-person care, and complement other services in primary care. Overall, better evaluation with formal measures (i.e., randomized trials, lack of inferiority designs) and analysis of variance to predictors of outcomes are necessary. Studies need to be focused on areas where there is currently a relative paucity of information, such as anxiety, substance use, and psychotic and other disorders.
A key area the integration of telepsychiatric models like collaborative care into services in primary care settings. The fact that it worked better than usual models is a key step—it may change our decision-making about how to best do things in the future. Web-based data management37,71,72 will facilitate services, as can stepped models of care. For example, a new stepped model might have low tier physician-to-provider phone or e-mail consultation followed by ATP, then therapies, and finally videoconferencing.
A plan for assessment and care for patients with ethnic, cultural, and language issues is essential.54,73 Scientific and policy questions from this discussion include:
• What tools, methods, and measures are needed to assess the patients, providers, and health systems?
• What are the intersections of culture, sociodemographic, geography, and technology in health?
• Will patients' disorder, racial or ethnic identity, or other factors determine whether e-mental health or in-person care is more effective?
• What is the most cost-effective and feasible way to provide language/interpreting support?
Limitations of this article include that it is not a systematic, fully longitudinal review of the literature. Second, the scope was limited to exclude specifics on medication management, the therapies (largely), and other treatments. Furthermore, not all findings apply to all locales or settings therein. Fourth, the landscape of healthcare is rapidly changing, with consumer/patient use of technology—the field will be hard pressed to keep up. Finally, plans that offer the most “adaptability” or “flexibility” of telemental services—beyond this article's scope—will afford the most opportunities for improvement.
Acknowledgments
We thank the American Telemedicine Association Mental Health Interest Group; the University of California, Davis Center for Health and Technology, Department of Psychiatry and Behavioral Sciences and Health Informatics Graduate Group; telemedicine and telepsychiatry pioneers, authors, organizational leaders, and innovators; and the American Telemedicine Association.
Disclosure Statement
No competing financial interests exist.
References
- 1.Hilty DM. Marks SL. Urness D, et al. Clinical and educational applications of telepsychiatry: A review. Can J Psychiatry. 2004;49:12–23. doi: 10.1177/070674370404900103. [DOI] [PubMed] [Google Scholar]
- 2.Yellowlees PM. Shore J. Roberts L. Practice guidelines for videoconferencing-based telemental health, American Telemedicine Association. Telemed J E Health. 2010;16:1074–1089. doi: 10.1089/tmj.2010.0148. [DOI] [PubMed] [Google Scholar]
- 3.AACAP practice parameter for telepsychiatry with children and adolescents. J Am Acad Child Adolesc Psychiatry. 2007;47:1468–1483. doi: 10.1097/CHI.0b013e31818b4e13. [DOI] [PubMed] [Google Scholar]
- 4.Hilty DM. Liu W. Marks SL, et al. Effectiveness of telepsychiatry: A brief review. Can Psychiatry Assoc Bull. 2003;(Oct):10–17. [Google Scholar]
- 5.Richardson LK. Frueh BC. Grubaugh AL, et al. Current directions in videoconferencing tele-mental health research. Clin Psychol (New York) 2009;16:323–328. doi: 10.1111/j.1468-2850.2009.01170.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Morland LA. Greene CJ. Rosen C, et al. Issues in the design of a randomized noninferiority clinical trial of telemental health psychotherapy for rural combat veterans with PTSD. Contemp Clin Trials. 2009;30:513–522. doi: 10.1016/j.cct.2009.06.006. [DOI] [PubMed] [Google Scholar]
- 7.World Health Organization. Geneva: WHO Press; 2011. Telemedicine opportunities and developments in member states. Results of the second global survey on eHealth. [Google Scholar]
- 8.Hilty DM. Yellowlees PM. Cobb HC, et al. Models of telepsychiatric consultation-liaison service to rural primary care. Psychosomatics. 2006;47:152–157. doi: 10.1176/appi.psy.47.2.152. [DOI] [PubMed] [Google Scholar]
- 9.Fortney JC. Pyne JM. Mouden SP, et al. Practice-based versus telemedicine-based collaborative care for depression in rural federally qualified health centers: A pragmatic randomized comparative effectiveness trial. Am J Psychiatry. 2013;170:414–425. doi: 10.1176/appi.ajp.2012.12050696. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Flay BR. Biglan A. Boruch RF, et al. Standards of evidence: Criteria for efficacy, effectiveness and dissemination. Prev Sci. 2005. www.ebppc.hawaii.edu/Resources/Standards%20of%20Evidence%5B1%5D.pdf. [Feb 1;2013 ]. www.ebppc.hawaii.edu/Resources/Standards%20of%20Evidence%5B1%5D.pdf [DOI] [PubMed]
- 11.Morland LA. Greene CJ. Rosen C, et al. Issues in the design of a randomized noninferiority clinical trial of telemental health psychotherapy for rural combat veterans with PTSD. Contemp Clin Trials. 2009;30:513–252. doi: 10.1016/j.cct.2009.06.006. [DOI] [PubMed] [Google Scholar]
- 12.Pakyurek M. Yellowlees PM. Hilty DM. The child and adolescent telepsychiatry consultation: Can it be a more effective clinical process for certain patients than conventional practice? Telemed J E Health. 2010;16:289–292. doi: 10.1089/tmj.2009.0130. [DOI] [PubMed] [Google Scholar]
- 13.O'Reilly R. Bishop J. Maddox K, et al. Is telepsychiatry equivalent to face to face psychiatry: Results from a randomized controlled equivalence trial. Psychiatr Serv. 2007;58:836–843. doi: 10.1176/ps.2007.58.6.836. [DOI] [PubMed] [Google Scholar]
- 14.De Las Cuevas C. Arrendondo MT. Cabrera MF, et al. Randomized controlled trial of telepsychiatry through videoconference versus face-to-face conventional psychiatric treatment. Telemed J E Health. 2006;12:341–350. doi: 10.1089/tmj.2006.12.341. [DOI] [PubMed] [Google Scholar]
- 15.Fortney JC. Pyne JM. Edlund MJ, et al. A randomized trial of telemedicine-based collaborative care for depression. J Gen Intern Med. 2007;22:1086–1093. doi: 10.1007/s11606-007-0201-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Ruskin PE. Silver-Aylaian M. Kling MA, et al. Treatment outcomes in depression: Comparison of remote treatment through telepsychiatry to in-person treatment. Am J Psychiatry. 2004;161:1471–1476. doi: 10.1176/appi.ajp.161.8.1471. [DOI] [PubMed] [Google Scholar]
- 17.Morland LA. Greene CJ. Rosen CS, et al. Telemedicine for anger management therapy in a rural population of combat veterans with posttraumatic stress disorder: A randomized noninferiority trial. J Clin Psychiatry. 2010;71:855–863. doi: 10.4088/JCP.09m05604blu. [DOI] [PubMed] [Google Scholar]
- 18.Frueh BC. Monnier J. Yim E, et al. Randomized trial for post-traumatic stress disorder. J Telemed Telecare. 2007;13:142–147. doi: 10.1258/135763307780677604. [DOI] [PubMed] [Google Scholar]
- 19.Morland LA. Hynes AK. Mackintosh MA, et al. Group cognitive processing therapy delivered to veterans via telehealth: A pilot cohort. J Trauma Stress. 2011;24:465–469. doi: 10.1002/jts.20661. [DOI] [PubMed] [Google Scholar]
- 20.Myers KM. Valentine JM. Melzer SM. Feasibility, acceptability, and sustainability of telepsychiatry for children and adolescents. Psychiatr Serv. 2007;58:1493–1496. doi: 10.1176/ps.2007.58.11.1493. [DOI] [PubMed] [Google Scholar]
- 21.Myers KM. Palmer NB. Geyer JR. Research in child and adolescent telemental health. Child Adolesc Psychiatr Clin North Am. 2011;20:155–171. doi: 10.1016/j.chc.2010.08.007. [DOI] [PubMed] [Google Scholar]
- 22.Boydell K. Volpe T. Pignatello A. A qualitative study of young people's perspectives on receiving psychiatric services via televideo. J Can Assoc Child Adolesc Psychiatry. 2010;19:5–11. [PMC free article] [PubMed] [Google Scholar]
- 23.Myers KM. Vander Stoep A. McCarty CA, et al. Child and adolescent telepsychiatry: Variations in utilization, referral patterns and practice trends. J Telemed Telecare. 2010;16:128–133. doi: 10.1258/jtt.2009.090712. [DOI] [PubMed] [Google Scholar]
- 24.Palmer NB. Myers KM. Vander Stoep A, et al. Attention-deficit/hyperactivity disorder and telemental health. Curr Psychiatry Rep. 2010;12:409–417. doi: 10.1007/s11920-010-0132-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Pesamaa L. Ebeling H. Kuusimaki ML, et al. Videoconferencing in child and adolescent telepsychiatry: A systematic review of the literature. J Telemed Telecare. 2004;10:187–192. doi: 10.1258/1357633041424458. [DOI] [PubMed] [Google Scholar]
- 26.Elford R. White H. Bowering R, et al. A randomized, controlled trial of child psychiatric assessments conducted using videoconferencing. J Telemed Telecare. 2000;6:73–82. doi: 10.1258/1357633001935086. [DOI] [PubMed] [Google Scholar]
- 27.Nelson EL. Barnard M. Cain S. Treating childhood depression over videoconferencing. Telemed J E Health. 2003;9:49–55. doi: 10.1089/153056203763317648. [DOI] [PubMed] [Google Scholar]
- 28.Neufeld JD. Bourgeois JA. Hilty DM, et al. The e-Mental Health Consult Service: Providing enhanced primary care mental health services through telemedicine. Psychosomatics. 2007;48:135–141. doi: 10.1176/appi.psy.48.2.135. [DOI] [PubMed] [Google Scholar]
- 29.Yellowlees PM. Hilty DM. Marks SL, et al. A retrospective analysis of child and adolescent e-mental health. J Am Acad Child Adolesc Psychiatry. 2008;47:1–5. doi: 10.1097/chi.0b013e31815a56a7. [DOI] [PubMed] [Google Scholar]
- 30.Szeftel R. Federico C. Hakak R, et al. Improved access to mental health evaluation for patients with developmental disabilities using telepsychiatry. J Telemed Telecare. 2012;18:317–321. doi: 10.1258/jtt.2012.111113. [DOI] [PubMed] [Google Scholar]
- 31.Richardson L. McCauley E. Katon W. Collaborative care for adolescent depression: A pilot study. Gen Hosp Psychiatry. 2009;31:36–45. doi: 10.1016/j.genhosppsych.2008.09.019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Rabinowitz T. Murphy K. Amour JL, et al. Benefits of a telepsychiatry consultation service for rural nursing home residents. Telemed J E Health. 2010;16:34–40. doi: 10.1089/tmj.2009.0088. [DOI] [PubMed] [Google Scholar]
- 33.Poon P. Hui E. Dai D, et al. Cognitive intervention for community-dwelling older persons with memory problems: Telemedicine versus face-to-face treatment. Int J Geriatr Psychiatry. 2005;20:285–286. doi: 10.1002/gps.1282. [DOI] [PubMed] [Google Scholar]
- 34.Holden D. Dew E. Telemedicine in a rural gero-psychiatric inpatient unit: Comparison of perception/satisfaction to onsite psychiatric care. Telemed J E Health. 2008;14:381–384. doi: 10.1089/tmj.2007.0054. [DOI] [PubMed] [Google Scholar]
- 35.Weiner M. Rossetti H. Harrah K. Videoconference diagnosis and management of Choctaw Indian dementia patients. Alzheimers Dement. 2011;7:562–566. doi: 10.1016/j.jalz.2011.02.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Chapter 4. Mental health care for American Indians and Alaska Natives. Rockville, MD: Substance Abuse and Mental Health Services Administration (US); 2001. [Feb 1;2013 ]. Office of the Surgeon General (US); Center for Mental Health Services (US); National Institute of Mental Health (US). Mental Health: Culture, Race, and Ethnicity: A Supplement to Mental Health: A Report of the Surgeon General. [PubMed] [Google Scholar]
- 37.Moreno FA. Chong J. Dumbauld J, et al. Use of standard webcam and internet equipment for telepsychiatry treatment of depression among underserved Hispanics. Psychiatr Serv. 2012;63:1213–1217. doi: 10.1176/appi.ps.201100274. [DOI] [PubMed] [Google Scholar]
- 38.Hilty DM. Lim RF. Koike AK, et al. Planning for telepsychiatric consultation: A needs assessment for cultural and language services at rural sites in California. J Rural Mental Health. 2013 (submitted for publication). [Google Scholar]
- 39.Nieves J. Stack KM. Hispanics and telepsychiatry. Psychiatr Serv. 2007;58:877–878. doi: 10.1176/ps.2007.58.6.877. [DOI] [PubMed] [Google Scholar]
- 40.Chong J. Moreno FA. Feasibility and acceptability of clinic-based telepsychiatry for low-income Hispanic primary care patients. Telemed J E Health. 2012;18:297–304. doi: 10.1089/tmj.2011.0126. [DOI] [PubMed] [Google Scholar]
- 41.Ye J. Shim R. Lukaszewski T. Yun K, et al. Telepsychiatry services for Korean immigrants. Telemed J E Health. 2012;18:797–802. doi: 10.1089/tmj.2012.0041. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Shore JH. Savin D. Orton H, et al. Diagnostic reliability of telepsychiatry in American Indian veterans. Am J Psychiatry. 2007;164:115–118. doi: 10.1176/ajp.2007.164.1.115. [DOI] [PubMed] [Google Scholar]
- 43.Shore JH. Brooks E. Savin D. Orton H. Grigsby J. Manson S. Acceptability of telepsychiatry in American Indians. Telemed J E Health. 2008;14:461–466. doi: 10.1089/tmj.2007.0077. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Mucic D. Transcultural telepsychiatry and its impact on patient satisfaction. J Telemed Telecare. 2010;16:237–242. doi: 10.1258/jtt.2009.090811. [DOI] [PubMed] [Google Scholar]
- 45.Lopez AM. Cruz M. Lazarus S, et al. Use of American sign language in telepsychiatry consultation. Telemed J E Health. 2004;10:389–391. doi: 10.1089/tmj.2004.10.389. [DOI] [PubMed] [Google Scholar]
- 46.Frueh BC. Henderson S. Myrick H. Telehealth service delivery for persons with alcoholism. J Telemed Telecare. 2005;11:372–375. doi: 10.1258/135763305774472060. [DOI] [PubMed] [Google Scholar]
- 47.Hilty DM. Yellowlees PM. Nesbitt TS. Evolution of telepsychiatry to rural sites: Change over time in types of referral and PCP knowledge, skill, and satisfaction. Gen Hosp Psychiatry. 2006;28:367–373. doi: 10.1016/j.genhosppsych.2006.05.009. [DOI] [PubMed] [Google Scholar]
- 48.Hilty DM. Nesbitt TS. Kuenneth TA, et al. Telepsychiatric consultation to primary care: Rural vs. suburban needs, utilization and provider satisfaction. J Rural Health. 2007;23:163–165. doi: 10.1111/j.1748-0361.2007.00084.x. [DOI] [PubMed] [Google Scholar]
- 49.Hilty DM. Marks SL. Wegeland JE, et al. A randomized controlled trial of disease management modules, including telepsychiatric care, for depression in rural primary care. Psychiatry. 2007;4:58–65. [PMC free article] [PubMed] [Google Scholar]
- 50.Hilty DM. Ingraham RL. Yang RP, et al. Multispecialty phone and email consultation to primary care providers for patients with developmental disabilities in rural California. Telemed J E Health. 2004;10:413–421. doi: 10.1089/tmj.2004.10.413. [DOI] [PubMed] [Google Scholar]
- 51.Ayers K. Yellowlees P. Mental health considerations during a pandemic influenza outbreak. Internet J Rescue Disaster Med. 2009;9(1) doi: 10.5580/1481. [DOI] [Google Scholar]
- 52.Katon W. Von Korff M. Lin E, et al. Collaborative management to achieve depression treatment guidelines. J Clin Psychiatry. 1997;58(Suppl 1):20–24. [PubMed] [Google Scholar]
- 53.Yellowlees PM. Odor A. Burke MM, et al. A feasibility study of asynchronous telepsychiatry for psychiatric consultations. Psychiatr Serv. 2010;61:838–840. doi: 10.1176/ps.2010.61.8.838. [DOI] [PubMed] [Google Scholar]
- 54.Butler TN. Yellowlees P. Cost analysis of store-and-forward telepsychiatry as a consultation model for primary care. Telemed J E Health. 2012;18:74–77. doi: 10.1089/tmj.2011.0086. [DOI] [PubMed] [Google Scholar]
- 55.Yellowlees PM. Odor A. Patrice K, et al. Transcultural psychiatry made simple—Asynchronous telepsychiatry as an approach to providing culturally relevant care. Telemed J E Health. 2013;19:259–264. doi: 10.1089/tmj.2012.0077. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56.Rosenthal TC. The medical home: Growing evidence to support a new approach to primary care. J Am Board Fam Med. 2008;21:427–440. doi: 10.3122/jabfm.2008.05.070287. [DOI] [PubMed] [Google Scholar]
- 57.Yellowlees PM. Odor A. Patrice K, et al. PsychVACS: A system for asynchronous telepsychiatry. Telemed J E Health. 2011;17:299–303. doi: 10.1089/tmj.2010.0159. [DOI] [PubMed] [Google Scholar]
- 58.Williams M. Pfeffee M. Boyle , et al. Telepsychiatry in the emergency department: Overview and case studies. California HealthCare Foundation. 2010. www.chcf.org/publications/2009/12/telepsychiatry-in-the-emergency-department-overview-and-case-studies. [Feb 1;2013 ]. www.chcf.org/publications/2009/12/telepsychiatry-in-the-emergency-department-overview-and-case-studies
- 59.Sorvaniemi M. Ojanen E. Santamäki O. Telepsychiatry in emergency consultations: A follow-up study of sixty patients. Telemed J E Health. 2005;11:439–441. doi: 10.1089/tmj.2005.11.439. [DOI] [PubMed] [Google Scholar]
- 60.Shore JH. Hilty DM. Yellowlees PM. Emergency management guidelines for telepsychiatry. Gen Hosp Psychiatry. 2007;29:199–206. doi: 10.1016/j.genhosppsych.2007.01.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 61.Yellowlees PM. Burke MM. Marks SL, et al. Emergency telepsychiatry. J Telemed Telecare. 2008;14:277–281. doi: 10.1258/jtt.2008.080419. [DOI] [PubMed] [Google Scholar]
- 62.Hollingsworth JM. Saint S. Hayward RA, et al. Specialty care and the patient-centered medical home. Med Care. 2011;49:4–9. doi: 10.1097/MLR.0b013e3181f537b0. [DOI] [PubMed] [Google Scholar]
- 63.Cluver JS. Schuyler D. Frueh BC, et al. Remote psychotherapy for terminally ill cancer patients. J Telemed Telecare. 2005;11:157–159. doi: 10.1258/1357633053688741. [DOI] [PubMed] [Google Scholar]
- 64.Shore P. VISN 21 Home Based Telemental Health Pilot Project Program. Veteran's Health Administration. Portland, OR. 2011. http://conference.avapl.org/pubs/2012%20Conference%20Presentations/Home%20Based%20Telemental%20Health.pdf. [Feb 1;2013 ]. http://conference.avapl.org/pubs/2012%20Conference%20Presentations/Home%20Based%20Telemental%20Health.pdf
- 65.Weinstein MC. Stason WB. Foundations of cost-effectiveness analysis for health and medical practices. N Engl J Med. 1977;296:716–721. doi: 10.1056/NEJM197703312961304. [DOI] [PubMed] [Google Scholar]
- 66.Smith AC. Scuffham P. Wootten R. The costs and potential savings of a novel telepaediatric service in Queensland. BMC Health Serv Res. 2007;7:35. doi: 10.1186/1472-6963-7-35. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 67.Spaulding R. Belz N. DeLurgio S, et al. Cost savings of telemedicine utilization for child psychiatry in a rural Kansas community. Telemed J E Health. 2010;16:867–871. doi: 10.1089/tmj.2010.0054. [DOI] [PubMed] [Google Scholar]
- 68.Jong M. Managing suicides via videoconferencing in a remote northern community in Canada. Int J Circumpolar Health. 2004;63:422–428. doi: 10.3402/ijch.v63i4.17759. [DOI] [PubMed] [Google Scholar]
- 69.Persaud DD. Jreige S. Skedgel C, et al. An incremental cost analysis of telehealth in Nova Scotia from a societal perspective. J Telemed Telecare. 2005;11:77–84. doi: 10.1258/1357633053499877. [DOI] [PubMed] [Google Scholar]
- 70.Shore JH. Brooks E. Savin DM, et al. An economic evaluation of telehealth data collection with rural populations. Psychiatr Serv. 2007;58:830–835. doi: 10.1176/ps.2007.58.6.830. [DOI] [PubMed] [Google Scholar]
- 71.Unutzer J. Choi Y. Cook IA, et al. A web-based data management system to improve care for depression in a multicenter clinical trial. Psychiatr Serv. 2002;53:671–678. doi: 10.1176/ps.53.6.671. [DOI] [PubMed] [Google Scholar]
- 72.Geyer J. Myers K. Vander Stoep A, et al. Implementing a low-cost web-based clinical trial management system for community studies: A case study. Clin Trials. 2011;8:634–644. doi: 10.1177/1740774511416384. [DOI] [PubMed] [Google Scholar]
- 73.Yellowlees PM. Marks SL. Hilty DM, et al. Using e-health to enable culturally appropriate mental health care in rural areas. Telemed J E Health. 2008;14:486–492. doi: 10.1089/tmj.2007.0070. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 74.Manfredi L. Shupe J. Batki S. Rural jail telepsychiatry: A pilot feasibility study. Telemed J E Health. 2005;11:574–577. doi: 10.1089/tmj.2005.11.574. [DOI] [PubMed] [Google Scholar]
- 75.Urness D. Wass M. Gordon A, et al. Client acceptability and quality of life—Telepsychiatry compared to in-person consultation. J Telemed Telecare. 2006;12:251–254. doi: 10.1258/135763306777889028. [DOI] [PubMed] [Google Scholar]
- 76.Modai I. Jabarin M. Kurs R, et al. Cost effectiveness, safety, and satisfaction with video telepsychiatry versus face-to-face care in ambulatory settings. Telemed J E Health. 2006;12:515–520. doi: 10.1089/tmj.2006.12.515. [DOI] [PubMed] [Google Scholar]
- 77.Greenberg N. Boydell K. Volpe T. Pediatric telepsychiatry in Ontario: Caregiver and service provider perspectives. J Behav Health Sci Res. 2006;33:105–111. doi: 10.1007/s11414-005-9001-3. [DOI] [PubMed] [Google Scholar]
- 78.Myers K. Valentine J. Morganthaler R. Melzer S. Telepsychiatry with incarcerated youth. J Adolesc Health. 2006;38:643–648. doi: 10.1016/j.jadohealth.2005.07.015. [DOI] [PubMed] [Google Scholar]
- 79.Lau ME. Way BB. Fremont WP. Assessment of SUNY Upstate Medical University's child telepsychiatry consultation program. Int J Psychiatry Med. 2011;42:93–104. doi: 10.2190/PM.42.1.g. [DOI] [PubMed] [Google Scholar]
- 80.Jacob MK. Larson JC. Craighead WE. Establishing a telepsychiatry consultation practice in rural Georgia for primary care physicians: A feasibility report. Clin Pediatr (Phila) 2012;51:1041–1047. doi: 10.1177/0009922812441671. [DOI] [PubMed] [Google Scholar]
- 81.Germain V. Marchand A. Bouchard , et al. Effectiveness of cognitive behavioural therapy administered by videoconference for posttraumatic stress disorder. Cogn Behav Ther. 2009;38:42–53. doi: 10.1080/16506070802473494. [DOI] [PubMed] [Google Scholar]
- 82.Bouchard S. Paquin B. Payeur R, et al. Delivering cognitive-behavior therapy for panic disorder with agoraphobia in videoconference. Telemed J E Health. 2004;10:13–25. doi: 10.1089/153056204773644535. [DOI] [PubMed] [Google Scholar]
- 83.Edwards MA. Patel AC. Telemedicine in the state of Maine: A model for growth driven by rural needs. Telemed J E Health. 2003;9:25–39. doi: 10.1089/153056203763317620. [DOI] [PubMed] [Google Scholar]
- 84.Harley J. Economic evaluation of a tertiary telepsychiatry service to an island. J Telemed Telecare. 2006;12:354–357. doi: 10.1258/135763306778682378. [DOI] [PubMed] [Google Scholar]
- 85.Pyne JM. Fortney JC. Tripathi SP. Maciejewski ML. Edlund MJ. Williams K. Cost-effectiveness analysis of a rural telemedicine collaborative care intervention for depression. Arch Gen Psychiatry. 2010;67:812–821. doi: 10.1001/archgenpsychiatry.2010.82. [DOI] [PubMed] [Google Scholar]