Abstract
Background: Although effective psychotherapies for posttraumatic stress disorder (PTSD) exist, high percentages of Veterans in need of services are unable to access them. One particular challenge to providing cost-effective psychological treatments to Veterans with PTSD involves the difficulty and high cost of delivering in-person, specialized psychotherapy to Veterans residing in geographically remote locations. The delivery of these services via clinical videoteleconferencing (CVT) has been presented as a potential solution to this access to care problem. Materials and Methods: This study is a retrospective cost analysis of a randomized controlled trial investigating telemedicine service delivery of an anger management therapy for Veterans with PTSD. The parent trial found that the CVT condition provided clinical results that were comparable to the in-person condition. Several cost outcomes were calculated in order to investigate the clinical and cost outcomes associated with the CVT delivery modality relative to in-person delivery. Results: The CVT condition was significantly associated with lower total costs compared with the in-person delivery condition. The delivery of mental health services via CVT enables Veterans who would not normally receive these services access to empirically based treatments. Additional studies addressing long-term healthcare system costs, indirect cost factors at the patient and societal levels, and the use of CVT in other geographic regions of the United States are needed. Conclusions: The results of this study provide evidence that CVT is a cost-reducing mode of service delivery to Veterans with PTSD relative to in-person delivery.
Key words: medicine, military medicine, telehealth, telepsychiatry
Introduction
Posttraumatic stress disorder (PTSD) is one of the most costly psychiatric conditions to society.1 PTSD is a common mental health disorder experienced by combat Veterans and is associated with high levels of depression,2 substance abuse,3 medical comorbidity,4 mortality,5 healthcare service use,6,7 and impaired functioning.8–10 Apart from the distress and reduced quality of life associated with this disorder, military Veterans with PTSD and their families are often confronted with job loss,11 poor family functioning,12 homelessness,13 anger, violence,14,15 and incarceration.16 Approximately 11% of patients served by the Department of Veterans Affairs (VA) healthcare system have been diagnosed with PTSD.17 Despite this, estimates suggest that up to 40–50% of the Veterans who need and are eligible for mental health services in the United States do not access these services, and many of those who initiate mental healthcare drop out of treatment prematurely.18,19 Currently, 3.3 million of the 8 million Veterans who use the VA in the United States reside in rural areas and are geographically remote.20 Thus, access to healthcare services is a significant challenge for many Veterans, including many returning Operation Enduring Freedom/Operation Iraqi Freedom/Operation New Dawn Veterans.21,22
To solve this access to care problem, the VA system has dramatically expanded its use of telemedicine to deliver treatment services.23 Clinical videoteleconferencing (CVT), which allows for synchronous face-to-face communication between patient and provider, is one example of a telemedicine modality that has become increasingly available in a wide variety of institutions and service systems. Research supports the acceptability,24 general feasibility, and clinical efficacy relative to traditional in-person mental health services of CVT applications for several underserved and difficult-to-access populations,25–29 including combat Veterans with PTSD.30–33
One aspect of the promise of telemedicine is that it represents not only a clinically effective way to surmount access to care challenges, but also a cost-reducing means of doing so. That is, the telemedicine modality should allow for the treatment to be delivered at a lower cost than in-person delivery without sacrificing quality of care.27,34 However, despite the common assumption that telemedicine is cost reducing, at this point the accumulation of data on cost lags well behind that of data supporting clinical efficacy.28,35,36 There is currently a lack of coherent methodology for the cost evaluation of services delivered via telemedicine. Researchers have noted the need for studies that measure quantifiable clinical and cost outcomes, including measures of efficacy, efficiency, and opportunity cost,37 when conducting studies to determine if the potential benefits outweigh the costs of providing services via telemedicine. Investigators further recommend that studies on the cost-effectiveness of telemedicine take into account the total investment for the equipment, including the software, maintenance, repairs, and depreciation when conducting cost-benefit analyses.38
The few studies that have been conducted have generally supported the cost-effectiveness of telemedicine. Research on the costs of conducting neuropsychological assessments via CVT has indicated that the cost of telemedicine is significantly lower than the cost of delivering these interviews in-person.39 In Veteran populations, the cost-effectiveness of telemedicine has been demonstrated for the delivery of a care coordination/home telehealth program for Veterans with diabetes40 and structured clinical interviews with American Indian Veterans in rural locations.41 However, evidence that telemedicine may reduce costs of mental healthcare is not a uniform finding. A study investigating telehealth mental health screening with military populations post-deployment found that using telehealth to screen large numbers of Veterans was more expensive than in-person screening.42 Given the current lack of studies addressing the cost-effectiveness of telemedicine and discrepancies in the findings in this area of research, future investigations of the cost of this delivery modality are warranted.
The use of telemedicine improves access to care by allowing the VA to reach Veterans who would not otherwise receive treatment (e.g., rural residents) or screening for mental health disorders. However, there are important considerations to take into account when conducting services via a telemedicine modality. Providers of mental health treatment via telemedicine must establish specific plans for dealing with patients who are at risk.42 Another prominent concern is the lack of reliable, long-term reimbursement mechanisms,37 which can be especially problematic if the provider is delivering the treatment remotely from a location outside of a hospital.38 The cost-effectiveness of telemedicine may also be dependent on the providers' workload.43 Furthermore, providing services via telemedicine requires additional training, administrative support, and technological expenses beyond those necessary for traditional clinical care.42 In the future, payers, providers, and policy makers need to work together in order for telehealth to be adopted by a larger number of health service providers.44
The current study represents a retrospective analysis of data from a randomized controlled trial that demonstrated non-inferiority of clinical effectiveness and comparable process outcomes for a group-administered cognitive behavioral treatment for anger management delivered via CVT compared with in-person delivery.31,45 We examined the direct cost of the evidence-based psychotherapy delivered via CVT relative to traditional in-person care in a setting where the specially trained therapist would otherwise have to fly to a rural VA satellite clinic to deliver treatment.
Materials and Methods
Participants
Participants were 74 male Veterans with PTSD and moderate to severe anger problems recruited across three VA clinical sites and three Veterans Centers on the Hawaiian Islands of Maui and the Big Island of Hawaii. In the parent study (n=125), participants were recruited in 10 cohorts and then randomized into either in-person (control) or CVT (experimental) anger management treatment46 groups. The current study uses post-treatment data from the six cohorts that were conducted on the remote islands of Maui and the Big Island of Hawaii; therefore, the data from the cohorts conducted on Oahu were excluded. Statistical analysis was completed at the cohort level and revealed that participants in the two treatment conditions did not differ (were non-inferior) on any of the baseline characteristics or treatment outcomes31; therefore, Veterans were collapsed across cohorts for the current analysis. Ethnoracial demographics for this study were as follows: Native Hawaiian/Pacific Islander, 31.1%; white, 39.2%; Asian American, 21.6%; and other ethnicity, 6.8%. Most Veterans were married (66%) and the average age was 56.4 years (standard deviation, 8.21 years). The Veterans included in this study did not differ significantly on ethnicity, marital status, completer status, or clinical outcomes from Veterans in the parent study. Significant differences were found on age at treatment and war era between Veterans in the current study and those in the parent study.
Participants had a diagnosis of lifetime PTSD as assessed by the Clinician-Administered PTSD Scale.47 Veterans were required to have been on a stable medication regimen for at least 2 months prior to study entry. Veterans with active psychosis, active homicidal or suicidal ideation, significant cognitive impairment, a history of organic mental disorder, current substance dependence, or unwillingness to refrain from substance abuse during treatment were excluded. The VA Pacific Islands Healthcare System's Institutional Review Board approved the study protocol.
Measures
Clinical measures
Post-treatment clinical data were used from the total score for the Novaco Anger Scale (NAS-T)48 and Trait Anger (T-ANG) and Anger Expression Index (AEI), which are two subscales of the State-Trait Anger Expression Inventory-249 that measure trait anger and anger expression, respectively. The Cronbach's alpha values for these measures across the five assessment points in the parent study were as follows: NAS-T, 0.89–0.94; T-ANG, 0.83–0.93; and AEI, 0.73–0.83.
Cost measures
Various cost outcomes were calculated in order to examine the association of CVT condition relative to in-person anger management treatment with clinical and cost outcomes.50 Each cohort was assigned a cost depending on the year that the treatment of the cohort occurred. The travel costs in 2007 were lower than 2005 and 2006 because there was a ticket-price competition among airlines in Hawaii during 2007. All costs were converted to 2012 dollars using the Consumer Price Index (www.bls.gov/data/inflation_calculator.htm) as of November 2012. Connectivity costs were not included in the cost measures as these expenses are a normal fixed facility cost, which would not be affected by the intervention being delivered via CVT.
The personnel per session costs were calculated using the per-minute cost and activities associated with each condition. We estimated the total employment costs per minute for a clerk, an information technician, and a psychologist. Per-minute costs were estimated using the annual salary based on the Federal General Schedule for salaries for the years 2005, 2006, and 2007 with the addition of the cost-of-living adjustment for a person federally employed in Honolulu (25% for 2005–2007) and the cost of benefits (base salary + cost-of-living adjustment at 25% rate + 35% for benefits). In order to obtain a therapy per session cost, the employment cost of the psychologist for a 90-min session was calculated using the same formula as the personnel per session cost.
For the CVT condition, the cost of the telehealth equipment and costs for the clinician to travel to the remote islands were also calculated. Travel costs were calculated using the modal cost per trip to represent the typical cost of travel for each year because using the mean cost gave too much weight to outlier costs. These travel costs included the price of the airline ticket, parking fees at the originating airport, and taxi fees between the remote airport and the VA clinic. The equipment costs were calculated by dividing the cost of the equipment by the expected years it will be in use to derive a per-time unit cost. We used the “straight-line method” to amortize, or allocate the total sum of the costs to different time periods.51 For the two Tandberg (Cisco, San Jose, CA) units, we used the purchase cost of the units over 3 years52 to obtain a per-minute cost of the videoconference equipment. To amortize the T1 line and based on the annual maintenance costs of the line, we calculated per-minute cost for the 512 kilobytes of bandwidth we used per session.
Participant per session costs were calculated, using the number of participants in each condition of each cohort, on the following variables: (1) personnel per session, (2) travel, and (3) equipment costs. The cost per person for the in-person condition included the total cost of the salary for a supervisory-level licensed clinical psychologist for the time spent traveling from the large medical center in Honolulu to the satellite clinics on remote Hawaiian Islands, total travel expenses (e.g., airfare, ground transportation) for those trips, and cost of the psychologist for the time spent providing the anger management therapy. The cost per person for the CVT condition was the cost of the equipment used during the therapy session and cost of the psychologist for the time spent providing the anger management therapy. The total cost per participant was the product of the per session cost and the total number of treatment sessions that each participant attended.
Statistical Analysis
Because of a non-negligible percentage of missing values in 14.9% of the cases for the NAS-T variable, 12.2% of cases for the T-ANG variable, and 12.2% of cases for the AEI variable, a missing data analysis was conducted. A multiple imputation method using the linear regression model was found to be appropriate and was subsequently used for this analysis. All statistical analyses are based on the pooled imputed values for missing post-treatment variable values of the NAS-T and T-ANG and AEI subscales of the State-Trait Anger Expression Inventory-2.
Unadjusted means of the cost differences and percentage change in clinical outcomes (from baseline to last recorded outcome) by delivery mode were tested using Student's t tests. In order to examine the cost of telemedicine delivery of treatment relative to in-person delivery while not sacrificing quality of care or incurring inferior clinical outcomes, we estimated three regression models to examine the association of telemedicine delivery relative to in-person care while adjusting for the percentage change from baseline to last reported value for each of the three clinical outcomes; therefore, in addition to adjusting the cost models for delivery modality, we adjusted for the change in the clinical outcome in each model. With the non-normally distributed total cost per participant outcome, generalized linear model regression with a gamma distribution and log link was used for all three models. All analyses were performed using STATA version 9.0 (StataCorp, College Station, TX).
Results
Means for treatment costs and the three clinical outcome measures are reported in Table 1 by mode of delivery. For in-person delivery, the percentage change from baseline to last reported value of outcome was fairly stable, ranging from a decline of 5% for the NAS-T to a decline of 4% for the T-ANG and a decline of 4% for the AEI. For the CVT delivery, the percentage change from baseline to last reported value of outcome was much higher relative to in-person delivery for all three outcomes, with a 10% decline in the NAS-T, a 19% decline in the T-ANG, and a 22% decline in the AEI. There were statistically significant differences in percentage change from baseline to last reported value by delivery mode for the T-ANG (p=0.03) and AEI (p=0.01). Unadjusted mean cost of CVT was significantly (p=0.00) lower by $713 relative to the mean cost of in-person delivery.
Table 1.
|
CVTa |
IN-PERSONa |
||
---|---|---|---|---|
OUTCOMES | MEAN | 95% CI | MEAN | 95% CI |
Total cost in 2012b | $79 | $73, $84 | $792 | $727, $856 |
Clinical variables | ||||
NAS-Tc,d | −0.10 | −0.16, −0.05 | −0.05 | −0.13, 0.04 |
T-ANGc–e | −0.19 | −0.29, −0.10 | −0.04 | −0.18, −0.09 |
AEIc–e | −0.22 | −0.32, −0.13 | −0.04 | −0.32, −0.13 |
The sample size for both conditions was 37.
Unadjusted mean cost.
Percentage of change from pretreatment to post-treatment in clinical scale scores.
Pooled data from multiple imputation for missing values used.
Significant difference by delivery mode based on Student's t tests at p<0.05.
AEI, Anger Expression Index subscale of State-Trait Anger Expression Inventory-2; CI, confidence interval; CVT, clinical videoteleconferencing; NAS-T, Novaco Anger Scale Total; T-ANG, Trait Anger subscale of State-Trait Anger Expression Inventory-2.
Adjusted cost of CVT relative to in-person delivery for each clinical outcome is reported in Table 2. We estimated similar models for each change in clinical outcome in order to examine the sensitivity of the estimate of CVT relative to in-person association with cost for different clinical outcomes. The significant (p=0.00) cost reduction associated with CVT was found to be relatively stable across the three clinical outcomes. Cost reduction was very stable, ranging from $703 (β=25.17) for the T-ANG to $708 (β=10.21) for the AEI and $710 (β=17.98) for the NAS-T. We found that regardless of the clinical outcome, the estimate on CVT was relatively stable, which indicates that our estimate on CVT is robust to model specification, therefore increasing our confidence that the estimate on CVT is reliable.
Table 2.
|
MODELS |
|||||
---|---|---|---|---|---|---|
OUTCOMES | NAS-T | 95% CI | T-ANG | 95% CI | AEI | 95% CI |
COSTa | −$710b | −$773, −$648 | −$703b | −$765, −$641 | −$708b | −$772, −$644 |
NAS-Tc,d | 17.98 | −42.65, 78.61 | ||||
T-ANGc,d | 25.17 | −10.56, 60.91 | ||||
AEIcd | 10.21 | −27.24, 47.67 |
Telemedicine (clinical videoteleconferencing) relative to in-person. The sample size for both conditions was 37.
p=0.00.
β coefficient of the cost of 1% change in clinical outcomes and the confidence intervals of the β coefficients.
Pooled data from multiple imputation for missing values used.
AEI, Anger Expression Index subscale of State-Trait Anger Expression Inventory-2; CI, confidence interval; NAS-T, Novaco Anger Scale Total; T-ANG, Trait Anger subscale of State-Trait Anger Expression Inventory-2.
Discussion
In this retrospective analysis of cost data for a randomized controlled trial of telemedicine service delivery for Veterans with PTSD,31,45 we examined both the unadjusted and adjusted total cost by delivery method based on the estimated direct costs to the VA healthcare system (i.e., provider-level costs). Veterans in both the in-person and CVT groups displayed reductions on anger outcomes after treatment, with the CVT group showing statistically significant reductions on two of the three clinical anger outcomes relative to the in-person group. Results show that the mental health services provided via telemedicine were vastly less expensive than services provided via the traditional in-person mode, which required therapists to fly from the VA medical center to outlying VA satellite clinics, with no drop-off in clinical efficacy.
The current study compared the cost of providing mental health treatment via CVT delivery with that for in-person delivery. It should be noted that, in many cases, the alternative to delivering specialized mental health services via CVT is not in-person delivery, but for the Veterans to receive no mental health services. Although providing empirically based treatments to Veterans who would not normally receive services because of limited access to healthcare is clearly more expensive than providing no care in the short term, the benefits produced at the systems level make CVT a cost-reducing mode of mental health services delivery.
In this relatively simple study, the cost-reducing value of telemedicine for providing mental health services to Veterans with PTSD was strongly supported. By estimating separate models for changes in three different clinical outcomes, we were able to test the sensitivity of our cost findings relative to clinical outcome and control for changes in clinical outcome, thereby ensuring that cost reductions were not obtained by reducing quality of care and clinical outcomes.
The potential cost impact of CVT is likely to vary depending on alternative modalities and methods of service delivery. In the present study, conducted in the context of the unique geographic realities of the state of Hawaii, the alternative to CVT involved the clinician traveling to deliver care in-person; therefore, CVT was clearly more cost reducing relative to the alternative in-person delivery modality. The remoteness of rural clinics was not merely a result of distance between VA clinics (which ranged up to 220 miles), but also by the fact that the Hawaiian Islands are separated by the ocean, which means that air travel is required to get from one island to another in a reasonably efficient manner. Despite the unique geography of the Hawaiian Islands, the current findings should still be relevant to many rural areas of the United States and other countries where vast distances separate patients from traditional healthcare services.
These findings, while encouraging, leave many important economic questions unanswered. They provide only an estimate of the potential cost-reducing value of telemedicine at the provider level relative to traditional in-person services. They do not tell us anything about larger system costs, especially in the absence of the in-person services provided in this study. At least in the short term, providing greater access to mental health services should actually result in greater use of services as patients access care they had previously been foregoing. On the other hand, in the longer term it is conceivable that a comprehensive healthcare system like the U.S. VA might realize significant cost savings if efficacious mental healthcare successfully improves the overall health of patients and results in lower use of other medical services and benefits. The current analyses are also limited by not including the indirect cost factors at the patient (e.g., quality of life, travel costs, lost wages, time, family burden) or societal (e.g., productivity, lost taxes, early mortality, out-of-network service provider costs, insurance) levels. For all of these reasons, the current study does not represent a full examination of the cost-effectiveness of telemedicine. Additional prospective randomized clinical trials, several of which several are currently under way,53–55 are needed to address these limitations. Specific trials would be useful to assess the cost differential of CVT versus in-person psychotherapy in geographic regions where clinicians can drive rather than fly to outlying clinics.
Acknowledgments
This work was partially supported by grant TEL 03-080-3 from the Veterans Affairs Health Services and Development. This work was also supported by the Office of Research and Development, Medical Research Service, Department of Veterans Affairs. L.A.M., Principal Investigator of this study, had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Disclosure Statement
The authors have no affiliations with or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript.
References
- 1.Kessler RC. Heeringa S. Lakoma MD. Petukhova M. Rupp AE. Schoenbaum M. Wang PS. Zaslavsky AM. Individual and societal effects of mental disorders on earnings in the United States: Results from the National Comorbidity Survey Replication. Am J Psychiatry. 2008;165:703–711. doi: 10.1176/appi.ajp.2008.08010126. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Elhai JD. Contractor AA. Palmieri PA. Forbes D. Richardson JD. Exploring the relationship between underlying dimensions of posttraumatic stress disorder and depression in a national, trauma-exposed military sample. J Affect Disord. 2011;133:477–480. doi: 10.1016/j.jad.2011.04.035. [DOI] [PubMed] [Google Scholar]
- 3.Bremner JD. Southwick SM. Darnell A. Charney DS. Chronic PTSD in Vietnam combat veterans: Course of illness and substance abuse. Am J Psychiatry. 1996;153:369–375. doi: 10.1176/ajp.153.3.369. [DOI] [PubMed] [Google Scholar]
- 4.Kang HK. Natelson BH. Mahan CM. Lee KY. Murphy FM. Post-traumatic stress disorder and chronic fatigue syndrome-like illness among Gulf War veterans: A population-based survey of 30,000 veterans. Am J Epidemiol. 2003;157:141–148. doi: 10.1093/aje/kwf187. [DOI] [PubMed] [Google Scholar]
- 5.Johnson DR. Fontana A. Lubin H. Corn B. Rosenheck R. Long-term course of treatment-seeking Vietnam veterans with posttraumatic stress disorder: Mortality, clinical condition, and life satisfaction. J Nerv Ment Dis. 2004;192:35–41. doi: 10.1097/01.nmd.0000105998.90425.6a. [DOI] [PubMed] [Google Scholar]
- 6.Hoge CW. Lesikar SE. Guevara R. Lange J. Brundage JF. Engel CC., Jr Messer SC. Orman DT. Mental disorders among U.S. military personnel in the 1990s: Association with high levels of health care utilization and early military attrition. Am J Psychiatry. 2002;159:1576–1583. doi: 10.1176/appi.ajp.159.9.1576. [DOI] [PubMed] [Google Scholar]
- 7.Schnurr PP. Friedman MJ. Sengupta A. Jankowski MK. Holmes T. PTSD and utilization of medical treatment services among male Vietnam veterans. J Nerv Ment Dis. 2000;188:496–504. doi: 10.1097/00005053-200008000-00004. [DOI] [PubMed] [Google Scholar]
- 8.Frueh BC. Turner SM. Beidel DC. Cahill SP. Assessment of social functioning in combat veterans with PTSD. Aggress Violent Behav. 2001;6:79–90. [Google Scholar]
- 9.Magruder KM. Frueh BC. Knapp RG. Johnson MR. Vaughan JA., III Carson TC. Powell DA. Hebert R. PTSD symptoms, demographic characteristics, and functional status among veterans treated in VA primary care clinics. J Trauma Stress. 2004;17:293–301. doi: 10.1023/B:JOTS.0000038477.47249.c8. [DOI] [PubMed] [Google Scholar]
- 10.Thorp SR. Stein MB. Posttraumatic stress disorder and functioning. Natl Center Post-traumatic Stress Disord PTSD Res Q. 2005;16:1–7. [Google Scholar]
- 11.Thorp SR. Stein MB. Occupational disability. In: Reyes G, editor; Elhai JD, editor; Ford JD, editor. Encyclopedia of psychological trauma. Hoboken, NJ: John Wiley & Sons; 2008. p. 453. [Google Scholar]
- 12.Evans L. McHugh T. Hopwood M. Watt C. Chronic posttraumatic stress disorder and family functioning of Vietnam veterans and their partners. Aust NZ J Psychiatry. 2003;37:765–772. doi: 10.1080/j.1440-1614.2003.01267.x. [DOI] [PubMed] [Google Scholar]
- 13.Kasprow WJ. Rosenheck RA. Frisman L. DiLella D. Referral and housing processes in a long-term supported housing program for homeless veterans. Psychiatr Serv. 2000;51:1017–1023. doi: 10.1176/appi.ps.51.8.1017. [DOI] [PubMed] [Google Scholar]
- 14.McHugh T. Forbes D. Bates G. Hopwood M. Creamer M. Anger in PTSD: Is there a need for a concept of PTSD-related posttraumatic anger? Clin Psychol Rev. 2012;32:93–104. doi: 10.1016/j.cpr.2011.07.013. [DOI] [PubMed] [Google Scholar]
- 15.Morland LA. Love AR. Mackintosh M-A. Greene CJ. Rosen CS. Treating anger and aggression in military populations: Research updates and clinical implications. Clin Psychol Sci Pract. 2012;19:305–322. [Google Scholar]
- 16.Greenberg GA. Rosenheck RA. Mental health and other risk factors for jail incareration among male veterans. Psychiatr Q. 2009;80:41–53. doi: 10.1007/s11126-009-9092-8. [DOI] [PubMed] [Google Scholar]
- 17.Magruder KM. Frueh BC. Knapp RG. Davis L. Hamner MB. Martin RH. Gold PB. Arana GW. Prevalence of posttraumatic stress disorder in VA primary care clinics. Gen Hosp Psychiatry. 2005;27:169–179. doi: 10.1016/j.genhosppsych.2004.11.001. [DOI] [PubMed] [Google Scholar]
- 18.Hoge CW. Interventions for war-related posttraumatic stress disorder: Meeting veterans where they are. JAMA. 2011;306:549–551. doi: 10.1001/jama.2011.1096. [DOI] [PubMed] [Google Scholar]
- 19.Tanielian T. Jaycox LH. Invisible wounds of war: Psychological and cognitive injuries, their consequences, and services to assist recovery. Santa Monica, CA: RAND Center for Military Health Policy Research; 2008. [Google Scholar]
- 20.Office of Rural Health, U.S. Department of Veteran Affairs. About the Office of Rural Health. 2012. Apr 4, 2012. www.ruralhealth.va.gov/about/index.asp. [Jul 2;2012 ]. www.ruralhealth.va.gov/about/index.asp Updated.
- 21.Hoge CW. Auchterlonie JL. Milliken CS. Mental health problems, use of mental health services, and attrition from military service after returning from deployment to Iraq or Afghanistan. JAMA. 2006;295:1023–1032. doi: 10.1001/jama.295.9.1023. [DOI] [PubMed] [Google Scholar]
- 22.Seal KH. Maguen S. Cohen B. Gima KS. Metzler TJ. Ren L. Bertenthal D. Marmar CR. VA mental health services utilization in Iraq and Afghanistan veterans in the first year of receiving new mental health diagnoses. J Trauma Stress. 2010;23:5–16. doi: 10.1002/jts.20493. [DOI] [PubMed] [Google Scholar]
- 23.Tuerk PW. Fortney J. Bosworth HB. Wakefield B. Ruggiero KJ. Acierno R. Frueh BC. Toward the development of national telehealth services: The role of Veteran's Health Administration and future directions for research. Telemed J E Health. 2010;6:1–3. doi: 10.1089/tmj.2009.0144. [DOI] [PubMed] [Google Scholar]
- 24.Grubaugh AL. Cain GD. Elhai JD. Patrick SL. Frueh BC. Attitudes toward medical and mental health care delivered via telehealth application among rural and urban primary care patients. J Nerv Ment Dis. 2008;196:166–170. doi: 10.1097/NMD.0b013e318162aa2d. [DOI] [PubMed] [Google Scholar]
- 25.Frueh BC. Deitsch SE. Santos AB. Gold PB. Johnson MR. Meisler N. Magruder KM. Ballenger JC. Procedural and methodological issues in telepsychiatry research and program development. Psychiatr Serv. 2000;51:1522–1527. doi: 10.1176/appi.ps.51.12.1522. [DOI] [PubMed] [Google Scholar]
- 26.Hilty DM. Marks SL. Urness D. Yellowlees PM. Nesbitt TS. Clinical and educational telepsychiatry applications: A review. Can J Psychiatry. 2004;49:2–23. doi: 10.1177/070674370404900103. [DOI] [PubMed] [Google Scholar]
- 27.Monnier J. Knapp RG. Frueh BC. Recent advances in telepsychiatry: An updated review. Psychiatr Serv. 2003;53:1604–1609. doi: 10.1176/appi.ps.54.12.1604. [DOI] [PubMed] [Google Scholar]
- 28.Morland LA. Frueh BC. Pierce K. Miyahira S. PTSD & telemental health: Updates & future directions. PTSD Clin Q. 2003;12:1–5. [Google Scholar]
- 29.Richardson LK. Frueh BC. Grubaugh AL. Egede L. Elhai JD. Current directions in videoconferencing tele-mental health research. Clin Psychol Sci Pract. 2009;16:323–338. doi: 10.1111/j.1468-2850.2009.01170.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Frueh BC. Monnier J. Yim E. Grubaugh AL. Knapp R. A randomized trial of telepsychiatry for post-traumatic stress disorder. J Telemed Telecare. 2007;13:142–147. doi: 10.1258/135763307780677604. [DOI] [PubMed] [Google Scholar]
- 31.Godleski L. Darkins A. Peters J. Outcomes of 98,609 U.S. Department of Veterans Affairs patients enrolled in telemental health services, 2006–2010. Psychiatr Serv. 2012;63:383–385. doi: 10.1176/appi.ps.201100206. [DOI] [PubMed] [Google Scholar]
- 32.Morland LA. Greene CJ. Rosen CS. Foy D. Reilly P. Shore J. He Q. Frueh BC. Telemedicine for anger management therapy in a rural population of combat veterans with posttraumatic stress disorder: A randomized noninferiority trial. J Clin Psychiatry. 2010;71:855–863. doi: 10.4088/JCP.09m05604blu. [DOI] [PubMed] [Google Scholar]
- 33.Tuerk PW. Yoder M. Grubaugh A. Myrick H. Hamner M. Acierno R. Prolonged exposure therapy for combat-related posttraumatic stress disorder: An examination of treatment effectiveness for veterans of the wars in Afghanistan and Iraq. J Anxiety Disord. 2011;25:397–403. doi: 10.1016/j.janxdis.2010.11.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Simpson J. Doze S. Urness D. Hailey D. Jacobs P. Evaluation of a routine telepsychiatry service. J Telemed Telecare. 2001;7:90–98. doi: 10.1258/1357633011936219. [DOI] [PubMed] [Google Scholar]
- 35.Bergmo TS. Can economic evaluation in telemedicine be trusted? A systematic review of the literature. Cost Effectiveness Resource Allocation. 2009;7:18. doi: 10.1186/1478-7547-7-18. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Wade VA. Karnon J. Elshaug AG. Hiller JE. A systematic review of economic analyses of telehealth services using real time videocommunication. BMC Health Serv Res. 2010;10:233. doi: 10.1186/1472-6963-10-233. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Lacroix A. Lareng L. Padeken D. Nerlich M. Bracale M. Ogushi Y. Okada Y. Orlov OI. McGee J. Sanders JH. Doarn CR. Prerost S. McDonald I. International concerted action on collaboration in telemedicine: Recommendations of the G-8 Global Healthcare Applications Subproject-4. Telemedicine J E Health. 2002;8:149–157. doi: 10.1089/15305620260008084. [DOI] [PubMed] [Google Scholar]
- 38.Devereaux J. Evaluation methods in telehealth: Getting to outcomes—A physician's insight. Telemed J E Health. 2010;16:984–985. doi: 10.1089/tmj.2010.9941. [DOI] [PubMed] [Google Scholar]
- 39.Schopp L. Johnstone B. Merrell D. Telehealth and neuropsychological assessment: New opportunities for psychologists. Prof Psychol Res Pract. 2000;31:179–183. [Google Scholar]
- 40.Barnett TE. Chumbler NR. Vogel WB. Beyth RJ. Ryan P. Figueroa S. The cost-utility of a care coordination/home telehealth programme for veterans with diabetes. J Telemed Telecare. 2007;13:318–321. doi: 10.1258/135763307781644843. [DOI] [PubMed] [Google Scholar]
- 41.Shore JH. Brooks E. Savin DM. Manson SM. Libby AM. An economic evaluation of telehealth data collection with rural populations. Psychiatr Serv. 2007;58:830–835. doi: 10.1176/ps.2007.58.6.830. [DOI] [PubMed] [Google Scholar]
- 42.Jones MD. Etherage JR. Harmon SC. Okiishi JC. Acceptability and cost-effectiveness of military telehealth mental health screening. Psychol Serv. 2012;9:132–143. doi: 10.1037/a0026709. [DOI] [PubMed] [Google Scholar]
- 43.Persaud DD. Jreige S. Skedgel C. Finley J. Sargeant J. Hanlon N. An incremental cost analysis of telehealth in Nova Scotia from a societal perspective. J Telemed Telecare. 2005;11:77–84. doi: 10.1258/1357633053499877. [DOI] [PubMed] [Google Scholar]
- 44.Pan E. Cusack C. Hook J. Vincent A. Kaelber DC. Bates DW. Middleton B. The value of provider-to-provider telehealth. Telemed J E Health. 2008;14:446–453. doi: 10.1089/tmj.2008.0017. [DOI] [PubMed] [Google Scholar]
- 45.Greene CJ. Morland LA. Macdonald A. Frueh BC. Grubbs KM. Rosen CS. How does tele-mental health affect group therapy process? Secondary analysis of a noninferiority trial. J Consult Clin Psychol. 2010;78:746–750. doi: 10.1037/a0020158. [DOI] [PubMed] [Google Scholar]
- 46.Reilly PM. Shopshire MS. Anger management group treatment for cocaine dependence: Preliminary outcomes. Am J Drug Alcohol Abuse. 2000;26:161–177. doi: 10.1081/ada-100100598. [DOI] [PubMed] [Google Scholar]
- 47.Blake DD. Weathers F. Nagy LM. Kaloupek DG. Gusman FD. Charney DS. Keane TM. The development of a clinician-administered PTSD scale. J Trauma Stress. 1995;8:75–90. doi: 10.1007/BF02105408. [DOI] [PubMed] [Google Scholar]
- 48.Novaco RW. Anger as a risk factor for violence among the mentally disordered. In: Monahan J, editor; Steadman H, editor. Violence and mental disorder. Chicago: University of Chicago Press; 1994. pp. 21–59. [Google Scholar]
- 49.Speilberger CD. Manual for the State Trait Anger Expression Inventory. Odesa, FL: Psychological Assessment Resources; 1988. [Google Scholar]
- 50.Chumney E. Simpson K. Methods and designs for outcomes research. Bethesda, MD: American Society of Health-System Pharmacists; 2006. [Google Scholar]
- 51.Shore JH. Brooks E. Savin DM. Manson SM. Libby AM. An economic evaluation of telehealth data collection with rural populations. Psychiatr Serv. 2007;58:830–835. doi: 10.1176/ps.2007.58.6.830. [DOI] [PubMed] [Google Scholar]
- 52.Grady BJ. A comparative cost analysis of an integrated military telemental health-care service. Telemed J E Health. 2002;8:293–300. doi: 10.1089/15305620260353180. [DOI] [PubMed] [Google Scholar]
- 53.Morland LA. Greene CJ. Rosen C. Mauldin PD. Frueh BC. Issues in the design of a randomized noninferiority clinical trial of telemental health psychotherapy for rural combat veterans with PTSD. Contemp Clin Trials. 2009;30:513–522. doi: 10.1016/j.cct.2009.06.006. [DOI] [PubMed] [Google Scholar]
- 54.Morland LA. Hynes AK. Mackintosh M. Resick PA. Chard K. Group cognitive processing therapy for PTSD delivered to rural combat veterans via telemental health: Lessons learned from a pilot cohort. J Trauma Stress. 2011;24:465–469. doi: 10.1002/jts.20661. [DOI] [PubMed] [Google Scholar]
- 55.Richardson LK. Frueh BC. Grubaugh AL. Egede L. Elhai JD. Current directions in videoconferencing tele-mental health research. Clin Psychol Sci Pract. 2009;16:323–338. doi: 10.1111/j.1468-2850.2009.01170.x. [DOI] [PMC free article] [PubMed] [Google Scholar]