Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 May 30.
Published in final edited form as: Psychiatry Res. 2016 Feb 24;239:111–116. doi: 10.1016/j.psychres.2016.02.049

Using telehealth to augment an intensive case monitoring program in veterans with schizophrenia and suicidal ideation: A pilot trial

John Kasckow a,b,c,*, Susan Zickmund c,d, John Gurklis a, James Luther a, Lauren Fox a, Melissa Taylor e, Ira Richmond e, Gretchen L Haas a,b
PMCID: PMC4882604  NIHMSID: NIHMS777791  PMID: 27137970

Abstract

Veterans with schizophrenia admitted for suicidal ideation were recruited into a post-discharge program consisting of Intensive Case Monitoring (ICM) with daily monitoring with the Health Buddy (HB; experimental group) or ICM alone (control group). This study tested the feasibility of the telehealth monitoring intervention in this population. Secondly, we determined whether augmentation of ICM with our intervention for 3 months would result in a reduction in suicidal ideation. Twenty of 25 telehealth participants could set up the device. Monthly adherence for telehealth participants was > 80%. A qualitative analysis of endpoint surveys revealed that the majority of participants had positive responses. In both groups, there were improvements in Beck Scale for Suicidal Ideation (BSS) scores at endpoint relative to baseline. No group differences were present with survival analysis when using remission (i.e., BSS score = 0) as the outcome; however, in a subgroup with a history of suicide attempt, there was a trend (p = .093) for a higher rate of remission for those in the HB condition. In conclusion, telehealth monitoring for this population appears to be feasible for those who are able to start using the system. The pilot data obtained should help investigators design better telehealth interventions for this population.

Keywords: Schizophrenia, Telehealth, Suicide

1. Introduction

Suicide is a leading cause of premature death among people with schizophrenia (Kasckow et al., 2011). Many patients with schizophrenia who are hospitalized for suicidal ideation or attempt require post-hospitalization follow-up (While et al., 2012). This is particularly important during the first 3 months post-discharge when patients are at highest risk for suicide (e.g., While et al., 2012). To have the greatest impact on suicide prevention during this high-risk time, the American Psychiatric Association has recommended intensive monitoring (American Psychiatric Association, 2006). Most monitoring strategies for suicide prevention rely on in-person or telephone contact with a clinician or health care provider (Valenstein et al., 2009).

Veterans are a select U.S. subpopulation experiencing greater than average increases in suicide rates. Suicide is a leading cause of death in Veterans (Bruce 2010). In 2010, it was estimated that up to 22 Veterans commit suicide each day. The VA has responded to escalating suicide rates since 2008 by including enhanced mandated monitoring, the creation of high risk suicide lists and a 24 h hot line along with placement of suicide prevention coordinators. When inpatient Veterans are placed on the high risk list, they undergo intensive monitoring. Upon discharge following a hospitalization for suicidal behavior, they are monitored for at least once a week for the first month and then at least monthly for the remaining 2 months after hospital discharge for suicidal behavior. Since this monitoring policy was implemented, suicides among Veterans who receive VHA services have decreased somewhat. However, they still remain elevated at an unacceptable level. For instance, Hoffmire et al. (2015) indicated that the number of observed Veteran suicides is still approximately 20% higher than that which occurred in 2000. Thus, additional strategies for intervention are warranted.

Our research team has developed a telehealth monitoring system for suicidal patients with schizophrenia using the Health Buddy©, a telephone device that facilitates symptom assessment and patient-staff communication between visits. This telehealth intervention involves augmenting an Intensive Case Monitoring (ICM) program with daily use of the Health Buddy© system. ICM included weekly face-to-face meetings at the hospital clinic plus twice weekly phone calls in addition to standard VA monitoring. These visits and phone calls included assessment using the Patient Health Questionnaire (PHQ9; Kroenke et al., 2001) and the Beck Scale for Suicidal Ideation (BSS; Beck et al., 1979).

The purpose of the current study was to first test the feasibility of the telehealth monitoring intervention for suicidal behavior in this population of Veterans with schizophrenia or schizoaffective disorder. To address this, we asked whether participants would be willing to engage with the technology-based intervention, and continue to use it over a 3 month period. The secondary purpose was to assess with a random assignment trial, whether augmentation of ICM with our intervention would result in a significant reduction in suicidal ideation relative to a group that received only ICM.

We had previously reported that when augmenting the Health Buddy telehealth system with VA Treatment as Usual, there were improvements in suicidal ideation in at-risk Veterans with schizophrenia and suicidal ideation (Kasckow et al., 2015). There have been very few studies examining the use of telehealth for monitoring Veterans at risk for suicide. With the current study, our control condition, i.e., intensive case monitoring (ICM) was different and we obtained different outcomes compared to what we noted previously. Research in this area is expected to grow in the future and investigators will need guidance in the design of such approaches. Thus, it is important that we report our findings with this different control group to help provide guidance.

2. Methods

2.1. Recruitment and screening

All procedures were approved by the Institutional Review Board of the VA Pittsburgh Health Care System. The research team assessed recently admitted inpatients ≥18 years old for a diagnosis of schizophrenia/schizoaffective disorder and recent suicidal ideation; if an eligible patient was found, the clinician would be asked to refer the patient and ask the patient to sign a HIPAA form to provide permission for further screening and contact by the research team. If signed, the research team would discuss the protocol with the patient. If the patient was interested, s/he was invited to participate in an informed consent process wherein written informed consent was obtained. Following the consent procedures, the patients were further screened for the following inclusion criteria: a Mini Mental Status score ≥20 (Folstein et al., 1975); lack of a medical disorder which could influence diagnostic decisions, safety, and/or anticipated adherence. An example of a participant who had an exclusionary medical illness was an individual with recurrent hematemesis due to esophageal varices. Another example was an individual with motor dexterity problems due to neurologic diseases which prevented them from effectively using the telehealth device.

Other inclusionary criteria included a score >0 on item 4 and/or 5 of the BSS which respectively assesses active suicidal ideation and passive suicidal ideation, and a score ≥8 on the 17-item Hamilton Depression Rating Scale (Hamilton, 1960). Patients also needed to have a land-line telephone. At baseline, we also obtained demographic and other clinical data using the Calgary Scale for Depressive Symptoms (Addington et al., 1992), the Scales for Assessment of Positive Symptoms (Andreasen et al., 1995) and Negative Symptoms (Andreasen, 1992) [see Tables 1 and 2]. Recruitment occurred from 2/2008 to 9/2011.

Table 1.

Sociodemographic measures by treatment.

Measure Total (N = 51) HB (N = 25) ICM (N = 26) p-Value
Age (years) 51.1 (11.3) 51.0 (11.7) 51.2 (11.1) 0.838a
Race 0.136b
 Black    17 (34.0)      6 (24.0)    11 (44.0)
 White    33 (66.0)    19 (76.0)    14 (56.0)
Education (in years) 12.6 (1.80) 12.7 (2.29) 12.5 (1.12) 0.734a
Marital status 0.287c
 Married/living with a partner      8 (15.7)      4 (16.0)      4 (15.4)
 Separated/divorced    15 (29.4)    10 (40.0)      5 (19.2)
 Never married    23 (45.1)      8 (32.0)    14 (57.7)
 Widowed      5 (9.8)      3 (12.0)      2 (7.7)
Open in a new tab

Note: ICM = Intensive Case Monitoring group; HB = Telehealth + ICM group.

a

Mann-Whitney test.

b

Chi-squared test.

c

Fisher’s exact. Values represent means (standard deviations) for continuous variables or N (percentages) for categorical measures.

Table 2.

Baseline clinical measures by treatment.

Measure Total (N = 51) HB (N = 25) ICM (N = 26) p-Value
Beck scale for suicidal ideation (BSS) score   10.3 (7.23)   9.80 (6.15)   10.7 (8.24) 0.477
17 item Hamilton depression rating scale   16.0 (6.53)   17.0 (5.47)   15.2 (7.31) 0.347
Calgary depression rating scale   12.2 (5.13)   12.9 (4.74)   11.6 (5.48) 0.381
Mini mental status exam   26.7 (2.25)   27.1 (2.44)   26.2 (2.01) 0.092
Scale for assessment of positive symptoms (global)   5.36 (3.06)   5.00 (3.37)   5.71 (2.76) 0.596
Scale for assessment of negative symptoms (global) 11.40 (2.30) 11.40 (2.30) 11.20 (2.30) 0.728
Open in a new tab

Note: All comparisons were made by Mann-Whitney test. Values represent means (standard deviations) for all measures.

ICM = Intensive Case Monitoring Group; HB = Telehealth + ICM Group; BSS = Beck Scale for Suicidal Ideation.

2.2. Study description

Participants were randomized to ICM alone (control condition) or ICM with daily Health Buddy© monitoring (experimental condition). All participants received usual mental health care. This included weekly assessments with the BSS (Beck et al., 1979) and Personal Health Questionnaire 9 (PHQ9; Kroenke et al., 2001), which were administered face-to-face and also twice weekly on the phone by nurses; nurses’ inter-rater reliability intra class correlation was >0.9. The Beck scale was the standard scale with 19 items and a range of 0-38 (Beck et al. 1979). Face-to-face assessments decreased to every other week if BSS scores were 0 for 4 weeks and then to monthly with once weekly phone assessments if BSS scores were 0 for another 6 weeks. Face to face assessments also included the Calgary Scale for Depressive Symptom (Addington et al. 1992), Hamilton Depression Rating Scale (Hamilton, 1960) and the Scales for Assessment of Positive Symptoms and Negative Symptoms (Andreasen et al. 1995).

Participants randomized to the telehealth group were provided the Health Buddy device upon discharge from the inpatient psychiatric unit. They were provided instructions on how to use the device. This involved standard procedures (provided by Bosch HealthCare) to ensure that participants were able to understand what power connections were needed and that participants were able to press the appropriate buttons to start the dialogues. In addition, staff ensured that participants could answer questions appropriately by pushing the buttons on the device, end the daily sessions and understand how information is sent to the care providers. They would also explain to participants that they should contact support staff about any equipment/power failures or any other questions.

Daily telehealth monitoring included queries for participants about suicide, depressive symptoms and medication adherence utilizing dialogues provided originally by Health Hero, the company which originally marketed the Health Buddy©. During the study, Bosch Healthcare administered the Health Buddy system in the Department of Veterans Affairs. The Health Buddy© connected to the participant’s land-line telephone; each day, for 10–15 min, a participant answered questions by pushing one of the buttons on the device. Responses were then electronically transferred to the hospital daily and read by staff within 4 h of transfer. The reason for the four hour time frame was to balance 2 factors: (1) minimizing the time in which a potentially high risk response would be transmitted by the patient and subsequently read by a clinician and (2) practical issues with regard to how frequently clinical staff could monitor the website.

The HB dialogues provided daily psychoeducational support and would assist participants in deciding whether they should contact their clinician with worsening symptoms. Participants would also be provided with the phone number for the crisis line if they stated they had suicidal intent and/or plan. Copies of the dialogues are available upon request. Furthermore, participants returned the telehealth devices at the end of the study.

If participants did not download responses within 24 h since the last time this was done, they would be contacted by staff to ensure that they were safe and to remind them to continue to use the device. In addition, staff would immediately contact patients if they responded ‘yes’ to a question that inquired about suicidal behavior. In this case, staff members would assess the situation and decide whether (1) no action was needed; (2) whether the participant needed to come in soon to see their clinician (if there was not an appointment scheduled in the near future); (3) whether participants needed to come to the emergency room or; (4) if urgent assessment was needed and if the participant was unwilling to come in, whether the police needed to go to the participants’ home for further evaluation.

2.3. Qualitative data analysis

After they completed the study, participants completed a structured survey with the option of including open-ended responses. The survey asked them to write their assessment (i.e., judgement) of the telehealth intervention, including its strengths and how it could be improved. For the analysis of the participants’ statements (i.e., actual words used), all comments by participants were linked together based on their study ID. Two trained qualitative coders judged whether each participant was: (1) generally positive about the program, (2) generally negative, or (3) whether no assessment of the program could be made. Each coder judged the patients’ statements independently and then compared the results. There was a single disagreement between the coders (one selecting a ‘positive’ rating while the other assessed it as ‘cannot judge’), which was resolved through discussion. The ‘positive’ rating was chosen as the final judgment. The overall inter-coder reliability kappa statistic was 0.857 which is what Landis and Koch (1977) has described as “near perfect’ reliability. The senior qualitative analyst [SZ] then used qualitative content analysis based on Miller and Crabtree (1992) known as the “editing style” to highlight reasons for the positive or negative statement. This approach involved developing an iterative codebook based on a close reading of the text. The purpose of this was to better understand the reasons participants provided their judgements which are referred to as ‘assessments’ in the text.

2.4. Quantitative data analysis

Continuous measures were expressed as means and standard deviations. Tests of association included Student’s t or Mann-Whitney tests. Categorical measures were expressed by frequency and percentage distributions; tests of association included chi-square or the Fisher’s exact test if cell frequencies were small. We examined changes in suicidal ideation between groups by survival analysis with either time to remission, i.e., BSS score = 0 or as % response (i.e., ≥50% change in BSS scores) as the endpoint. We also examined whether there were differences in the other outcome measures, i.e., scores from the Calgary Depression Rating Scale, the 17 item Hamilton Depression Rating Scale, Scale for Positive Symptoms and Scale for Negative Symptoms. With each scale, scores were fit with a repeated measures mixed model which included random intercepts and an unstructured covariance matrix. To account for the tendency of scores to increase toward the end of the study, a quadratic term was added to each model.

3. Results

3.1. Recruitment

A total of 1628 inpatients had been screened for the trial. Twenty five patients were randomized to the telehealth experimental group and 26 were randomized to the ICM group. Fig. 1 displays the recruitment flow chart for the study. Tables 1 and 2 show baseline demographic and clinical characteristics and indicate that there were no significant group differences. Not shown are the gender differences; there were 2 female control participants and 1 female experimental participant. Furthermore, 24/25 telehealth and 21/26 control participants had a lifetime history of substance abuse/dependence; and 7/25 telehealth and 9/26 control participants had substance abuse/dependence a month prior to screening. The two groups did not differ in terms of frequency of substance abuse/dependence conditions.

Fig. 1.

Fig. 1

Open in a new tab

Recruitment Flow Chart (No HIPAA implies patient not willing to sign a consent form allowing the research staff to screen their records and to contact them; HIPAA implies the opposite). F2F means ‘face to face interview’.

3.2. Feasibility

To address feasibility, 5 out of 25 telehealth participants never set up the system. One patient’s landlord did not allow him to use the technology on the premises. The second patient was too disorganized and cognitively impaired (MMSE score = 21) and the third found out upon returning home that he had a phone company debt which he could not pay. The fourth participant relapsed to substance dependence and the fifth participant realized that he had transportation problems and would not be able to make it in for the face-to-face assessments and decided to withdraw soon after randomization. In addition, of the 20 who were able to start the system, 4 required assistance, i.e., staff visited patients’ residences to help with set-up.

Monthly adherence for participants was as follows: month 1: 83% (n = 20); month 2: 92% (n = 19) and month 3: 89% (n = 15). Rates were calculated monthly by adding for each participant the number of days they filled out the questions divided by the number of days they were in the study that month. In month 1, 1 participant dropped out at week 3. In month 2, 4 more dropped out at weeks 6, 6, 7 and 7 and in month 3, 1 more dropped out at week 10. To further address feasibility, we identified two adherence patterns among the 20 telehealth participants who started using the system over the 3 months: We defined a “HIGHLY adherent” group (n = 11) as those who exhibited average daily adherence rates >80%. We defined a “MODERATELY adherent” group (n = 9) as participants who exhibited at least 1 month of daily adherence which was less than 80%. Interestingly, among the moderately adherent participants, post-hoc review of data from face-to-face interviews revealed that factors associated with moderate adherence included: exacerbation of depression (n = 2), technical problems (n = 4), a disruptive home environment (n = 2), or unknown (n = 1).

3.3. Analysis of qualitative data

3.3.1. Open-ended statements

Fourteen of the telehealth participants completed open-ended responses at the end of the written survey aimed at evaluating the telehealth intervention. A total of 44 statements were written by participants. From the 14 surveys, 17 of the responses were judged to be positive, 7 negative, and 20 provided no information to judge either way (e.g., statements such as “no comment”).

3.3.2. Negative statements

For those with negative statements, concerns over the telehealth intervention focused on the limitations perceived in symptom management, as well as its impersonal nature. For example, one person expressed great concern over the impersonalized treatment that came from communicating with a computer. This participant wanted “[t]he human equation…. More emphasis on the whole person… not a subject of a study to be analyzed, not automation but as a real live man with deep psychological problems.” The BSS scores had improved in this individual over the 3 month intervention period. Another participant expressed frustration about the lack of symptom abatement: “Some days when I got confronted with suicidal questions, it would make me feel frustrated since the symptoms did not get better.” For this individual the BSS scores got worse over the 3 month intervention period.

3.3.3. Positive statements

For those with positive statements, the telehealth intervention was praised for its ability to instill hope. One participant noted: “It really helped me a lot when I had bad days, it gave me hope.” Patients described having a sense of being listened to and they noted that the program was akin to discussing problems with a medical provider. As one participant wrote about the program: “It was helpful and straightforward. It was like talking to a doctor on a daily basis; the 1st month I did not think it would help but I changed my mind.” Participants also described the program as effective in decreasing their suicidal thoughts. One participant concluded that it made him “aware of the undesirability of taking my own life…drummed into my mind and psyche the desire and motivation to keep on living and trying, praying and believing in God as my source.” Other participants described the program as effective in terms of improving their medication adherence, and symptom reduction for anxiety and depression.

3.4. Analysis of quantitative outcomes

Both groups exhibited improvements in suicidal ideation. At baseline, the mean (+/− standard deviation) HB BSS score was 9.8 (±6.15) and, at endpoint, the mean score was 2.44 (±5.52). For the control group, the mean baseline BSS score was 10.7 (±8.24) and, at endpoint, the mean score was 2.88 (±6.71). Using survival analysis, we did not detect any group differences when examining time to remission, defined as having a BSS score = 0 nor when examining % response (i.e. those with ≥50% change in BSS scores). For the subgroup of participants who had a lifetime history of suicide attempt (i.e., excluding those who had experienced ideation only), we found a trend for a higher rate of remission at 3 months for those in the HB condition (16/18) as compared with those in the ICM condition (14/19; log rank = 2.82; df = 1; p = 0.093; see Fig. 2).

Fig. 2.

Fig. 2

Open in a new tab

This figure shows the cumulative probability of remission in participants with a history of suicidal attempts receiving either the telehealth intervention (HB; n = 18) or the control (ICM; n = 19) intervention; log rank = 2.82; df = 1; p = 0.093.

In addition, we used repeated measures regression analysis to examine whether there were changes in the clinical measures in the ICM + HB group vs the ICM group; this included scores in Calgary Depression Rating Scale, Hamilton Depression Rating Scale, Scale for Positive Symptoms and Scale for Negative Symptoms. No differences were detected between groups.

4. Discussion

Our telehealth intervention has as its primary purpose daily monitoring with early detection of suicide risk so that clinicians know when to intervene. It combines engagement, monitoring and early intervention. Secondarily, the intervention provides supportive coaching and psychoeducation. Our pilot findings suggest that the use of our telehealth monitoring system is feasible in monitoring post-discharge suicide risk in this population. The population we studied is considered to be at high risk for non-adherence (Daniels et al., 2014).

Twenty out of 25 participants could set up the system. Of those who set up the system, the majority of participants in the ICM + HB group completed the 3 month protocol. Furthermore of the 20 who started using the system, only 4 dropped out within the 3 month protocol. One dropped out because of a lack of interest. Another one moved to another town. The other 2 dropped out because of circumstances beyond their control: 1) transportation problems and 2) incarceration. The overall adherence for the remaining 20 participants for logging in daily to the telehealth system exceeded 80% for each of the 3 months while they were in the study. This is consistent with what was reported previously for the same population of Veterans (Kasckow et al. 2015).

Telehealth monitoring has been shown to be acceptable for patients with schizophrenia using telehealth modalities other than the Health Buddy system (Kasckow et al. 2014). The current study also assessed acceptability; acceptability has not been reported previously in studies involving suicide monitoring with this population (Kasckow et al. 2015). Negative statements indicated that patients at times became frustrated when communicating with a machine rather than with a human. Despite there being some negative statements, the analysis of our qualitative data determined that the majority of the participants who responded to the survey had positive responses. The positive statements indicated that the system helped instill hope and that the system was helpful and easy to use. These statements have also been useful for further improving the telehealth system.

Although each group exhibited substantial improvement in endpoint vs baseline BSS scores, there were no statistically significant group differences. This may be due to the relatively intensive nature of the control condition which both groups received (i.e., two calls from the research nurse per week and one face-to-face visit per week); patients assigned to the control condition exhibited a marked treatment effect by itself—an outcome which may have left little room for further improvement when telehealth was used as an adjunct to this treatment. We implemented these intensive measures into our design because of safety concerns. In retrospect, while we realize that this control condition helped to enhance safety, it also consisted of a level of intervention considerably more intensive than typically provided in outpatient mental health settings. This issue reflects a common problem with research involving suicidal participants, i.e., finding the correct balance between achieving sufficient scientific rigor vs maintaining high ethical standards in order to maximize safety with a high risk population (Reynolds et al., 2001). As investigators design more studies in the future involving telehealth and populations at risk for suicide, establishing the appropriate control will be important. Human Subjects oversight boards tend to err on the side of caution and act in a conservative manner when deciding which control condition would be appropriate. Our findings thus provide important guidance in this regard.

In the current trial, a weak signal was detected when assessing suicidal ideation in participants with higher risk, i.e., those with a history of suicide attempt. This suggests that the intervention may have a stronger impact on those with a history of a suicide attempt. We also determined that the ‘number needed to treat’ for reaching a BSS score of 0 by week 28 with telehealth was 6 in the subgroup of participants with a history of suicide attempts. Future trials with a sufficiently powered sample size may help verify that significant improvements in suicidal ideation with telehealth can be demonstrated in participants with a history of a suicide attempt.

Study limitations included lack of double blinding; lack of double blinding may have caused information bias. The high adherence rates could have been due to selection bias since participants were willing to consent to a trial; this may have affected external validity. The sample size was small and included only Veterans from a single urban eastern US site. There were 37 patients who screened as eligible who did not sign the HIPAA form; this could have caused a selection bias. The baseline scores were not marked. This is due to the fact that our inclusion criteria only required participants to have a screening score of only 1 on item 4 and/or 5 on the BSS. The purpose of requiring a lower score was to maximize recruitment. It is not known whether having a sample of higher risk participants with higher BSS scores would have been associated with significant differences in BSS scores. Future studies will be aimed at answering that question.

Telehealth technology allows patients and clinical staff ease and efficiency of communication. This is important given that patients with schizophrenia often become isolated and, if non-adherent to appointments, can become out of reach of the outpatient treatment team. Our findings add to the literature which supports that self monitoring is an important component of disease management. For instance, self monitoring has been shown to be important in improving outcomes with weight loss (Burke et al., 2011) and cardiovascular diseases (Heneghan et al., 2006). Application of a telehealth clinical monitoring system holds promise in efforts to monitor suicide risk. Progress has been made with the use of telephone-, video- and internet-based modalities in the treatment of patients with schizophrenia (Kasckow et al., 2014). Our current telehealth system represents another approach.

Acknowledgments

Funded by the VISN 4 MIRECC and a VISN 4 CPPF award. The contents do not represent the views of the US Government or the Department of Veterans Affairs of the US Government. Dr. Kasckow has received assistance from Bosch Health Care for the software and transmission costs associated with this project.

References

  1. Addington D, Addington J, Maticka-Tyndale E, Joyce J. Reliability and validity of a depression rating scale for schizophrenics. Schizophr Res. 1992;6:201–208. doi: 10.1016/0920-9964(92)90003-n. [DOI] [PubMed] [Google Scholar]
  2. American Psychiatric Association. Psychiatry Online. Arlington, VA: 2006. Practice Guidelines for the Treatment of Psychotic Disorders. [Google Scholar]
  3. Andreasen NC. Negative symptoms in schizophrenia Definition and reliability. Arch Gen Psychiatry. 1982;39(7):784–788. doi: 10.1001/archpsyc.1982.04290070020005. [DOI] [PubMed] [Google Scholar]
  4. Andreasen NC, Arndt S, Miller D, Flaum M, Nopoulos P. Correlational studies of the scale for the assessment of negative symptoms and the scale for the assessment of positive symptoms: an overview and update. Psychopathology. 1995;28:7–17. doi: 10.1159/000284894. [DOI] [PubMed] [Google Scholar]
  5. Beck AT, Kovacs M, Weissman A. Assessment of suicidal intention: the scale for suicide ideation. J Consult Clin Psychol. 1979;47:343–352. doi: 10.1037//0022-006x.47.2.343. [DOI] [PubMed] [Google Scholar]
  6. Burke LE, Wang J, Sevick MA. Self-monitoring in weight loss: a systematic review of the literature. J Am Diet Assoc. 2011;111:92–102. doi: 10.1016/j.jada.2010.10.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Bruce ML. Suicide risk and prevention in veteran populations. Ann NY Acad Sci. 2010;1208:98–103. doi: 10.1111/j.1749-6632.2010.05697.x. [DOI] [PubMed] [Google Scholar]
  8. Daniels K, Loganathan M, Wilson R, Kasckow JW. Appointment adherence in patients with schizophrenia. Clin Care. 2014;11:467–482. [Google Scholar]
  9. Folstein MF, Folstein SE, McHugh PR. Mini-mental state. J Psychiatr Res. 1975;12:189–198. doi: 10.1016/0022-3956(75)90026-6. [DOI] [PubMed] [Google Scholar]
  10. Hamilton M. A rating scale for depression. J Neurol Neurosurg Psychiatry. 1960;23:56–62. doi: 10.1136/jnnp.23.1.56. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Heneghan C, Alonso-Coello P, Garcia-Alamino J, Perera R, Meats E, Glasziou P. Self-monitoring of oral anticoagulation: a systematic review and meta-analysis. Lancet. 2006;367:404–411. doi: 10.1016/S0140-6736(06)68139-7. [DOI] [PubMed] [Google Scholar]
  12. Hoffmire CA, Kemp JE, Bossarte RM. Changes in suicide mortality for veterans and Nonveterans by gender and history of VHA service use, 2000–2010. Psychiatric Services. 2015 May 1; doi: 10.1176/appi.ps.201400031. 2015. appips201400031 2015. (Epub ahead of print) [DOI] [PubMed] [Google Scholar]
  13. Kasckow J, Felmet K, Appelt C, Thompson R, Rotondi A, Haas G. Telepsychiatry in the assessment and treatment of schizophrenia. Clin Schizophr Relat Psychoses. 2014;8:21–27A. doi: 10.3371/CSRP.KAFE.021513. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Kasckow J, Felmet K, Zisook S. Managing suicide risk in patients with schizophrenia. CNS Drugs. 2011;25:129–143. doi: 10.2165/11586450-000000000-00000. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Kasckow J, Gao S, Hanusa B, Rotondi A, Chinman M, Zickmund S, Gurklis J, Fox L, Cornelius J, Richmond I, Haas GL. Telehealth monitoring of patients with schizophrenia and suicidal ideation. Suicide Life Threat Behav. 2015:17. doi: 10.1111/sltb.12154. http://dx.doi.org/10.1111/sltb.12154 (Epub ahead of print) [DOI] [PubMed]
  16. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression measure. J Gen Intern Med. 2001;16:606–613. doi: 10.1046/j.1525-1497.2001.016009606.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Landis J, Koch G. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–174. [PubMed] [Google Scholar]
  18. Miller W, Crabtree BF. Primary care research: a multi typology and qualitative road map. In: Crabtree BF, Miller WL, editors. Doing Qualitative Research. Sage Press; London: 1992. [Google Scholar]
  19. Reynolds CF, 3rd, Degenholtz H, Parker LS, Schulberg HC, Mulsant BH, Post E, Rollman B. Treatment as usual (TAU) control practices in the PROSPECT study: managing the interaction and tension between research design and ethics. Int J Geriatr Psychiatry. 2001;16:602–608. doi: 10.1002/gps.466. [DOI] [PubMed] [Google Scholar]
  20. Valenstein M, Eisenberg D, McCarthy JF, Austin KL, Ganoczy D, Kim HM, Ganoczy D, Kim HM, Zivin K, Piette JD, Olfson M, Blow FC. Service implications of providing intensive monitoring during High-risk periods for suicide among VA patients with depression. Psychiatr Serv. 2009;60:439–444. doi: 10.1176/appi.ps.60.4.439. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. While D, Bickley H, Roscoe A, Windfuhr K, Rahman S, Shaw J, Appleby L, Kapur N. Implementation of mental health service recommendations in England and Wales and suicide before-and-after observational study. Lancet. 2012;379:1005–1012. doi: 10.1016/S0140-6736(11)61712-1. [DOI] [PubMed] [Google Scholar]

RESOURCES