Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2020 Jul 1.
Published in final edited form as: Psychiatr Serv. 2019 May 14;70(7):569–577. doi: 10.1176/appi.ps.201800511

Predictors of Hospitalization with Individuals With First-Episode Psychosis: Data From a Two-Year Follow-Up in the RAISE-ETP Study

Delbert G Robinson a,b, Nina R Schooler c, Robert A Rosenheck d, Haiqun Lin e, Kyaw J Sint e, Patricia Marcy f, John M Kane a,b
PMCID: PMC6602852  NIHMSID: NIHMS1525105  PMID: 31084291

Abstract

Objective:

Despite treatment advances in other domains, inpatient psychiatric hospitalization rates for individuals with first-episode psychosis remain high. Even with early intervention services, a third or more of individuals are hospitalized over the first 2 years of treatment. Reducing hospitalization is desirable from individuals’ perspectives and for public health reasons as hospitalization costs are a major component of treatment costs.

Method:

Univariate and multivariate baseline and time-varying covariate analyses were conducted to identify predictors of hospitalization in the RAISE-ETP study, a two-year cluster randomized trial for participants experiencing a first episode of psychosis who were outpatients at study entry. The trial compared an early intervention treatment model (NAVIGATE) to usual community care at 34 clinics across the United States.

Results:

34% of NAVIGATE and 37% of usual care participants were hospitalized during the trial. Risk analyses revealed significant predictors of hospitalization to be: the number of hospitalizations before study entry, duration of untreated psychosis, and time-varying days of substance misuse, presence of Positive and Negative Syndrome Scale positive symptoms, and beliefs about the value of medication.

Conclusions:

These results indicate that hospital use may be decreased by reducing the duration of untreated psychosis and prior hospitalizations, minimizing residual symptoms, preventing substance misuse and facilitating adherence in medication taking. Addressing these factors could enhance the impact of first-episode early intervention treatment models as well as enhance outcomes of first-episode psychosis treated with other models.

ClinicalTrials.gov registration: NCT01321177

Introduction

Inpatient hospitalization can be very disruptive to the goals (e.g. schooling) of young people with first-episode psychosis (FEP) and is often experienced as traumatic (1,2). Caregivers frequently experience distress and negative outcomes (e.g. stigma, changes in relationships) associated with their family members’ hospitalization (3,4) in mixture with the positive changes that hospitalizations can foster (4). From a services perspective, hospitalization costs are a major component of early intervention services (EIS) first-episode care cost. Mean cumulative cost for psychiatric inpatient treatment over 5 years for the OPUS intervention in Denmark was €58,502 of the total treatment cost of €123,683 (5) and over 18 months £6103 of the total of £11,685 in the British LEO trial (6). In the United States, average costs every 6 months in the RAISE-ETP study were $9018 with EIS NAVIGATE treatment of which $4709 were hospitalization costs (7). Decreasing hospitalization costs further could bolster the cost effectiveness and thus sustainability of EIS.

A meta-analysis (8) of EIS trials (917) found that EIS compared with usual care was associated with a reduced risk of hospitalization (rates presented in Table 1). Hospitalization utilization varied by follow-up duration and health system. Over a two-year period, the lowest percentage of participants hospitalized even with EIS is a third. Reports (e.g. (18,19)) outside randomized trials from EIS present similar hospitalization rates.

Table 1:

Hospitalization Rates in Early Intervention Controlled Trial Treatment Studies

Study Country Number of Participants Length of Treatment Hospitalization Rates During Follow-Up Interval
Experimental Intervention Treatment As Usual
Sample Size > 100
OPUS (during year 2 of trial1) (11) Denmark 243 intervention 193 control2 13–24 months 26% 39%
PIANO (15) Italy 272 intervention 172 control 9 months 17% 16%
Valencia et al Study 1 (14) Mexico 60 intervention 60 control 6 months 5.6% 10%
LEO (10) United Kingdom 71 intervention 73 control 15 months 33% 51%
RAISE-ETP (16) United States 223 intervention 181 control 24 months 34% 37%
STEP (17) United States 60 intervention 57 control 12 months 23% 44%
Sample Size < 100
Grawe et al (12) Norway 30 intervention 20 control 24 months 33% 50%
Valencia et al Study 2 (13) Mexico 44 intervention 44 control 12 month 5.1% 10.7%
COAST (9) United Kingdom 32 intervention 27 control 12 months 7 total admissions for 32 participants 11 total admissions for 27 participants
1

rates of hospitalization in OPUS for year 1 were 59% with the intervention and 71% with treatment as usual. These rates include the hospitalization at the time of recruitment for participants recruited as inpatients.

2

participants with 2 year follow-up; baseline sample included 263 participants assigned to the intervention and 244 assigned to treatment as usual

To identify targets for development of interventions to decrease hospitalization risk of individuals with FEP, we examined data from the Recovery After an Initial Schizophrenia Episode-Early Treatment Program (RAISE-ETP) (ClinicalTrials.gov registration NCT01321177). RAISE-ETP compared a multi-element treatment model (20) for FEP to Community Care - usual care treatment. The background, rationale, and design have been published (21) as have the CONSORT flow diagram, participant characteristics, and two-year outcomes (16). For examining hospitalization predictors with a population with an already relatively low rate of hospitalization, the RAISE-ETP study had the advantage of having data covering a 2 year follow-up and low hospitalization rates for studies with 2 years of follow-up.

Methods

Participants

RAISE-ETP enrolled English-speaking individuals between the ages of 15–40 years with DSM-IV diagnoses of schizophrenia, schizoaffective disorder, schizophreniform disorder, brief psychotic disorder, or psychotic disorder not otherwise specified. Individuals with affective psychosis, substance-induced psychotic disorders, psychosis due to general medical conditions, clinically significant head trauma, or serious medical conditions were excluded. All participants had experienced only one episode of psychosis (though this episode might have resulted in multiple hospitalizations) and had taken ≤six months of lifetime antipsychotics.

Written informed consent was obtained from adult participants and legal guardians of those under 18 years old, who provided written assent. The Institutional Review Boards of the coordinating center and the participating sites approved the study. The National Institute of Mental Health Data and Safety Monitoring Board provided study oversight.

Clinical Sites

Thirty-four outpatient community mental health centers in 21 states were selected via a national search. Site eligibility criteria included (1) experience treating individuals with schizophrenia; (2) interest in offering EIS for FEP; (3) sufficient staff to implement the experimental intervention; (4) ability to recruit an adequate number of participants; and (5) institutional assurance that research assessments would be completed. Academic centers or sites with existing first-episode programs were excluded. All participants were outpatients at the time of their baseline assessment.

RAISE-ETP used cluster randomization; i.e., randomization by clinic rather than individual participant (22). The study statisticians randomly assigned 17 of the clinics to the experimental intervention and 17 to standard care.

Interventions

NAVIGATE (20), the experimental EIS, is team-based and includes four interventions: personalized medication management; family psychoeducation; resilience-focused individual therapy; and supported education and employment (manuals available at www.raiseetp.org). The primary outcome measure and therefore goal of RAISE-ETP was improving quality of life and not preventing hospitalization per se. These goals are not mutually exclusive as hospitalization inhibits improving quality of life. Of relevance to factors that might influence hospitalization risk, personalized medication management included assessment of symptoms, side effects, adherence and substance use at each visit. The psychosocial interventions included illness management strategies and modules about adherence and making decisions about substance use. The control condition, “Community Care”, was psychosis treatment determined by individual and clinician choice and service availability.

Trial Duration

Enrollment occurred between July 2010 and July 2012. Minimum potential trial duration for each participant was two years (longer for early enrollees) and is the focus of this report. Study assessments were suspended during periods of incarceration/hospitalization, but resumed after release/discharge. Research assessments continued even if participants discontinued NAVIGATE or Community Care treatment.

Assessment Strategy and Measures

Centralized assessors, masked both to individual treatment assignments and to the overall study design, administered over live, two-way, video conferencing the Structured Clinical Interview for DSM-IV (SCID) (23) for diagnosis and for obtaining the information required to determine duration of untreated psychosis (DUP); the Positive and Negative Syndrome Scale (PANSS) (24); the Clinical Global Impressions Severity Scale (25); the Calgary Depression Scale for Schizophrenia (26); and the Heinrichs-Carpenter Quality of Life Scale (27), the primary outcome measure. Remote assessment via two-way video conferencing is comparable to face-to-face assessments in patient acceptability and reliability (28). The SCID was completed at baseline; other measures every six months.

Site Research Assistants interviewed participants with the Service Use and Resource Form (SURF) (29,30) monthly to capture psychiatric inpatient and emergency services and self-reported days of alcohol/drug use. Emergency room visits lasting over 24 hours were considered as hospitalizations. Participant-reported assessments allowed us to not only obtain information about treatment participants received at their study site but also treatment received outside the site (e.g. inpatient admission at another agency). Participant self- report has proven to be generally accurate (29). The outcome of interest for this report was mental health hospitalization after the study entry that occurred in an outpatient context. Data on hospitalizations prior to study entry were obtained through individual and family interview and medical record search and examined as predictors of hospitalization during the study.

At baseline, 3 months, 6 months and every 6 months thereafter, participants completed the Intent to Attend measure (31), the Adherence Estimator (32), The Brief Evaluation of Medication Influence and Beliefs (BEMIB) scale (33), seven items from the Stigma Scale (34), a subset of the Perceived Well-Being Scale (35), the 6 item Autonomy Support Scale short-form version of the Health Care Climate Questionnaire (36), and an abbreviated version of the Mental Health Recovery Measure (37). They also rated their current state of mental/emotional health on a scale with a range of 1 (worst possible) to 100 (perfect health).

Statistical Analysis

We used time-to-event analysis for hospitalization. Cox proportional hazard model was used with site included as a frailty term to account for clustering of individuals within site. Clustered randomized trials typically have a limited number of clusters that could cause an imbalance between randomized treatment conditions on baseline measures, and this imbalance may confound the relationship between treatments and individual-level outcomes. Therefore, significant baseline covariates for treatment conditions were included as adjustment variables. We assessed whether the two treatment conditions differed in hospitalization adjusted for the baseline covariates of gender, student status at entry and total PANSS score that were found to be significantly associated with the treatment conditions in previously reported analyses (16).

For analysis of the longitudinal assessments, time-varying predictor variables were constructed consisting of the results of the assessment concurrent with, or if not concurrent the prior assessment closest in time to, an individual’s first hospitalization. Severity/intensity of a factor often changes over time. The time-varying predictor variables allowed us to examine the effects of a variable of interest at the time when it might have the greatest impact on hospitalization based upon being assessed closest to the time of hospitalization. For example, if an individual had a hospitalization at month 18, the results of the month 18 assessment was chosen and if that was not available the closest assessment before 18 month was chosen. For individuals with no hospitalizations, results from their last assessment were chosen.

To determine hospitalization predictors, we first performed univariate analyses using Cox proportional model with frailty of site for each candidate of the baseline (Table 2) and time-varying covariates (Table 4). For developing the multivariate model for baseline predictors, variables screened for entry into the analysis were those with significant or trend level associations in the univariate analyses of baseline predictors presented in Table 3. Since inclusion of correlated variables can result in unstable multivariate correlations, we used the following criteria to determine which correlated variables to enter. Preference was given to variables that would provide more clinically meaningful information if associations with hospitalization were found (e.g. factor scores were preferred over total scores as these describe more circumscribed symptoms than a total score and could provide more precise targets for intervention development). The final set of baseline variables for model entry included DUP greater than 74 weeks, number of prior hospitalizations, PANSS Positive and Excited factors, CDSS and the Well-being Scale.

Table 2.

Univariate Associations Between Baseline Variables and Later Hospitalization

Hazard Ratio 95% Confidence Interval p-value
Categorical Variables
Duration of Untreated Psychosis > 74 weeks 1.45 .99–2.11 .055
Male sex .80 .54–1.2 .284
Race
(reference: White)
.983
African American .98 .66–1.45
Other .94 .48–1.84
Hispanic ethnicity .89 .54–1.46 .649
Marital status
(reference: never married)
.623
Presently married .91 .42–1.96
Widowed/divorced/separated .62 .23–1.67
Current residence
(reference: independent living)
.641
Supported or structured .80 .24–2.66
Family, parents, grandparents, sibling .77 .49–1.22
Homeless, shelter, or other 1.02 .49–2.14
Patient’s education (reference: some or completed grade school) .903
Some college or higher .87 .36–2.06
Completed high school 1.03 .44–2.44
Some high school .94 .40–2.24
Mother’s education (reference: no school or unknown) .290
Some college or higher .76 .47–1.24
Completed high school .70 .41–1.21
Some high school or grade school .51 .25–1.04
Current student .77 .46–1.27 .304
Currently working .54 .29–1.01 .055
Student or working .71 .46–1.09 .116
Type of insurance
(reference: private)
.453
Public .95 .58–1.55
Uninsured .76 .48–1.22
SCID diagnoses
(reference: schizophrenia)
.586
Schizoaffective bipolar 1.30 .62–2.71
Schizoaffective depressive .85 .47–1.51
Schizophreniform provisional or definite .67 .38–1.20
Brief psychotic disorder or psychotic disorder NOS .96 .51–1.82
Lifetime alcohol use disorder (reference: does not meet criteria) .117
Met abuse criteria .71 .37–1.39
Met dependence criteria 1.39 .92–2.1
Lifetime cannabis use disorder (reference: does not meet criteria) .395
Met abuse criteria 1.01 .59–1.74
Met dependence criteria 1.35 .87–2.10
Number of prior hospitalizations (reference: no prior hospitalization) .002
1 1.30 .75–2.26
2 1.92 1.04–3.54
3 or more 2.77 1.52–5.06
Prescribed one or more antipsychotics at consent 1.10 .65–1.87 .724
Medication compliance by SURF interview
 Days not taking first antipsychotic among those prescribed antipsychotics (reference: few if any, <7) .874
7 to 13 .88 .32–2.41
14 to 20 1.17 .43–3.18
Most, >20 .69 .25–1.88
 Days taking less than the prescribed number of pills among those prescribed antipsychotics (reference: never or almost never, 0 to 25%) .851
Always/almost always, 76 to 100% .78 .32–1.92
Usually, 51 to 75% .95 .23–3.89
Sometimes, 26 to 50% 1.33 .58–3.04
 Days not taking first antipsychotic (reference: few if any, <7) .949
7 to 13 .88 .32–2.41
14 to 20 1.17 .43–3.19
Most, >20 .69 .25–1.88
Not prescribed antipsychotic .95 .59–1.55
 Days taking less than prescribed number of pills (reference: never or almost never, 0 to 25%) .938
Always/almost always, 76 to 100% .78 .31–1.92
Usually, 51 to 75% .96 .24–3.91
Sometimes, 26 to 50% 1.32 .57–3.03
Not prescribed antipsychotic .97 .6–1.58
Adherence Estimator Risk Category (reference: low risk) .143
Medium .89 .49–1.64
High .54 .28–1.04
Continuous Variables
Age .98 .94–1.01 .188
Duration of untreated psychosis (weeks) 1.00 1.00–1.00 .102
Heinrichs Carpenter Quality of Life Scale
Total score .99 .98–1.00 .116
Interpersonal relations .99 .97–1.01 .394
Instrumental role .97 .94–1.00 .069
Intrapsychic foundations .98 .96–1.01 .244
Common objects and activities .96 .88–1.04 .270
Positive and Negative Syndrome Scale
Total score 1.01 1.00–1.03 .067
Wallwork Factor scores
Positive 1.07 1.02–1.12 .007
Negative .97 .94–1.01 .169
Disorganized/concrete .97 .91–1.04 .388
Excited 1.10 1.03–1.17 .005
Depressed 1.09 1.02–1.15 .006
Calgary Depression Scale for Schizophrenia 1.05 1.01–1.1 .014
Clinical Global Impressions - Severity Scale 1.41 1.11–1.78 .004
Autonomy support scale mean score 1.02 .87–1.19 .828
Brief Evaluation of Medication Influences and Beliefs mean score .97 .81–1.16 .726
Mental Health Recovery Measure mean score .88 .77–1.01 .079
Stigma Scale mean score 1.04 .89–1.22 .633
Well Being Scale mean score .79 .63–.99 .044
Current State of Mental Health (1=worst possible emotional health; 100=perfect emotional health) .99 .98–1.00 .032
Life as a Whole (1=Terrible; 7=Delighted) .94 .83–1.08 .382
Number of days of alcohol intoxication past month .98 .89–1.07 .584
Number of days of illegal drugs past month 1.00 .98–1.03 .940
Duration of lifetime anti-psychotic medication at consent (days) 1.00 1.00–1.00 .669
How likely to complete study (0=not at all; 9=extremely) 1.06 .95–1.18 .326
How likely to attend next visit (0=not at all; 9=extremely) 1.12 .98–1.28 .110
Adherence Estimator Risk Numeric Ordinal (0=low,1=med,2=high) .96 .69–1.35 .829

Table 4.

Univariate Associations Between Time-Varying Variables and Hospitalization

Hazard Ratio 95% Confidence Interval p-value
Categorical Variables
Current residence (reference: independent living) .093
Supported or structured .44 .11–1.86
Family, parents, grandparents, sibling 1.02 .65–1.59
Homeless, shelter, or other 2.2 1.02–4.72
Current student .93 .59–1.47 .757
Currently working .42 .24–.73 .002
Student or working .62 .42–.93 .022
Type of insurance (reference: private insurance) .873
Public .88 .54–1.44
Uninsured .90 .54–1.48
Days not taking an antipsychotic (reference: few if any, <7) .254
7 to 13 1.33 .61–2.91
14 to 20 2.47 1.07–5.72
Most, >20 1.06 .49–2.32
Not prescribed antipsychotic .92 .59–1.45
Days taking less than prescribed antipsychotic (reference: always/almost always, 76 to 100%) .058
Usually, 51 to 75% 1.50 .72–3.12
Sometimes, 26 to 50% 3.04 1.39–6.65
Never or almost never, 0 to 25% 1.24 .50–3.09
Not prescribed antipsychotic .95 .61–1.50
Adherence Estimator Risk Category (reference: low risk) .070
Medium 1.87 1.08–3.22
High 1.30 .73–2.31
Continuous Variables
Heinrichs Carpenter Quality of Life Scale
Total score .99 .98–1.00 .014
Interpersonal relations .98 .96–1.00 .116
Instrumental role .95 .92–.98 <.001
Intrapsychic foundations .98 .96–1.01 .116
Common objects and activities .94 .87–1.01 .09
Positive and Negative Syndrome Scale
Total score 1.02 1.01–1.03 .001
Wallwork Factor scores
Positive 1.09 1.04–1.14 <.001
Negative .98 .94–1.01 .186
Disorganized/concrete 1.04 .97–1.11 .294
Excited 1.14 1.07–1.22 <.001
Depressed 1.13 1.07–1.20 <.001
Calgary Depression Scale for Schizophrenia 1.07 1.02–1.11 .003
Clinical Global Impressions - Severity Scale 1.54 1.24–1.90 <.001
Autonomy Support Scale mean score .89 .77–1.02 .100
Brief Evaluation of Medication Influences and Beliefs mean score .81 .68–.96 .017
Mental Health Recovery Measure mean score .81 .70–.93 .003
Stigma scale mean score 1.07 .92–1.25 .365
Well Being Scale mean score .73 .58–91 .005
Current State of Mental Health (1=worst possible emotional health; 100=perfect emotional health) .99 .98–1.00 .009
Life as a Whole (1=Terrible; 7=Delighted) .86 .76–.98 .027
Number of days of alcohol intoxication 1.03 .95–1.10 .518
Number of days of illegal drugs 1.02 1.00–1.04 .029
How likely to complete study (0=not at all; 9=extremely) 1.05 .93–1.19 .402
How likely to attend next visit (0=not at all; 9=extremely) 1.02 .91–1.15 .708
Adherence Estimator Risk Numeric Ordinal (0=low,1=med,2=high) 1.37 1.05–1.80 .023

Table 3.

Multivariate Model of Associations Between Baseline Variables and Hospitalization1

Parameter Hazard Ratio 95% Confidence Interval Chi-Square p-value
Duration of untreated psychosis greater than 74 weeks 1.51 1.02–2.23 4.13 .042
One prior hospitalization versus none 1.73 .97–3.08 3.46 .063
Two prior hospitalizations versus none 2.43 1.29–4.58 7.57 .006
Three or more prior hospitalizations versus none 3.78 2.00–7.15 16.67 <.001
Positive and Negative Syndrome Scale Excited Factor 1.11 1.03–1.18 8.19 .004
Well-being scale mean score .79 .63–1.00 3.85 .050
1

Model from backward selection of variables (frailty model with site). Variables entered into the analysis were: duration of untreated psychosis greater than 74 weeks, number of prior hospitalizations, Positive and Negative Syndrome Scale Positive and Excited factors, Calgary Depression Scale for Schizophrenia and the Well-being scale

The strategy for developing the final multivariate models integrating both baseline and time-varying variables was to consider for entry baseline variables with significant association with hospitalization in the multivariate baseline analysis shown in Table 3 and time-varying variables from the univariate analyses of time-varying variables and hospitalization shown in Table 4. Examining the correlations among variables and our strategy of considering clinical meaningfulness in variable selection, we developed two groups of variables for entry into the analyses. Both groups included the baseline predictors DUP and number of prior hospitalizations and the time-varying predictors of days of illegal drug use and being a student or working. In addition, Analysis 1 added time-varying variables rated by the central assessors, the PANSS Positive and Excited factors, the CDSS, and the participant rated Adherence Estimator. Analysis 2 added time-varying variables rated by the participant, the BEMIB and the Well Being Scale total score. The Adherence Estimator and BEMIB are participant rated measures and were highly correlated (r = −0.43). Thus, only one could be included in Analysis 2 that focused on participant assessment. We chose to include the BEMIB in Analysis 2 because it taps a participant’s beliefs about the value of medication for her or himself. The Adherence Estimator taps general attitudes toward medication and was not significantly correlated with any of the central assessor rated variables; it was included in Analysis 1.

For each analysis, we checked proportional hazard assumptions by dividing time into six month intervals and assessing whether the coefficients were statistically different across time intervals.

Results

Participants

Characteristics of the full RAISE-ETP sample of 404 individuals have been published (16). Some participants did not have any post-baseline assessments. Supplemental Table 1 presents the characteristics of the 382 participants who had at least one post-baseline assessment and thus have post-baseline hospitalization data. Overall, the 382 participants were young (mean age 23 years), mostly male (73%) and of diverse racial backgrounds. Outpatient community center sites typically receive most of their FEP referrals from inpatient units. Consistent with this pattern, only 84 participants had never had an inpatient psychiatric hospitalization.

Psychiatric hospitalization

One hundred twelve participants had at least one psychiatric hospitalizations during the 2-year observation period. Based upon a survival analysis, 34% of NAVIGATE and 37% of Community Care participants had a hospitalization (this estimate is that same as previously reported (16) for the sample of 404 individuals due to censoring effects with survival analysis with individuals who did not have post baseline assessments). Hospitalization rates did not differ between NAVIGATE and Community Care treatment (hazard ratio = 0.892, chi-square = 0.3448, p= 0.5571).

Factors associated with hospitalization

Baseline Variables:

Table 2 presents the associations between baseline characteristics and psychiatric hospitalization during the follow-up based upon univariate analyses. Significant associations were found for having had a hospitalization before study entry; scores on the Wallwork (38) positive, excited and depressed factors of the PANSS, the CDSS total score, CGI-Severity, and participants’ ratings of the Well-being Scale and their current state of mental health. Other variables with trend level associations (p<0.1) were DUP (dichotomized at the median value of 74 weeks (16,39)), working at study entry, QLS Instrumental role, PANSS total score and the Mental Health Recovery scale.

Table 3 presents the results of multivariate analyses of the association between baseline variables and subsequent hospitalization. DUP, prior hospitalizations, the PANSS excited factor and the Well-being Scale score were all significant predictors of subsequent hospitalization.

Time-varying variables:

Table 4 presents univariate analyses of the associations between the time-varying variables and hospitalization. There were significant associations between current working, being a student or worker, QLS total and Instrumental role scores, PANSS total and positive, excited and depressed factor scores, CDSS, CGI-severity, BEMIB, Mental Health Recovery scores, Well-being Scale scores, current state of mental health, life as a whole, number of days of illegal drugs and Adherence Estimator Risk scores and subsequent hospitalization.

Multivariate models integrating baseline and time-varying variables:

As described in the Statistical Analysis section, we tested two analysis models. As presented in Table 5, both analyses found significant associations between hospitalization during the study and number of hospitalizations before study entry and with time-varying days of illegal drug use. Additional significant associations with PANSS positive symptoms were found in Analysis 1 and in Analysis 2 with DUP >74 weeks and BEMIB scores.

Table 5.

Multivariate Models of Associations Between Baseline and Time-Varying Variables and Hospitalization

Model 11
Hazard Ratio 95% Confidence Interval Chi-Square2 p-value
One prior hospitalization before baseline versus none 2.02 .97–4.22 3.51 .061
Two prior hospitalizations before baseline versus none 2.55 1.11–5.86 4.84 .028
Three or more prior hospitalizations before baseline versus none 4.42 2.03–9.59 14.09 <.001
Time-varying Positive and Negative Syndrome Scale Positive Factor 1.08 1.02–1.14 7.87 .005
Time-varying days of illegal drug use 1.03 1.00–1.05 4.74 .029
Model 23
Hazard Ratio 95% Confidence Interval Chi-Square2 p-value
Duration of Untreated Psychosis > 74 weeks 1.78 1.14–2.79 6.41 .011
One prior hospitalization before baseline versus none 2.59 1.18–5.67 5.67 .017
Two prior hospitalizations before baseline versus none 3.42 1.42–8.21 7.53 .006
Three or more prior hospitalizations before baseline versus none 5.67 2.51–12.83 17.35 <.001
Time-varying days of illegal drug use 1.03 1.01–1.05 5.96 .015
Time-varying Brief Evaluation of Medication Influences and Beliefs .82 .67–.99 4.15 .042
1

Model from backward selection of variables (frailty model with site). Variables entered into the analysis: baseline variables were duration of untreated psychosis greater than 74 weeks and number of prior hospitalizations; time-varying variables were Positive and Negative Syndrome Scale Positive and Excited factors, Calgary Depression Scale for Schizophrenia, days of illegal drug use, Adherence Estimator Risk scores and being a student or working

2

df for all Chi-Square analyses =1

3

Model from backward selection of variables (frailty model with site). Variables entered into the analysis: baseline variables were duration of untreated psychosis greater than 74 weeks and number of prior hospitalizations; time-varying variables were days of illegal drug use, Longitudinal Brief Evaluation of Medication Influences and Beliefs, Well Being Scale total score and being a student or working

Discussion

Even though RAISE-ETP participants experienced a relatively low hospitalization rate, we were able to identify predictors of hospitalization. At study baseline, those with longer DUP, who had experienced more hospitalizations before study entry, had symptoms of excitement and reported lower well-being were more likely to be hospitalized during the two-year treatment period. When we added information gathered across the trial, longer DUP and history of hospitalization before study entry continued to influence risk of hospitalization but now positive psychosis symptoms closer to the time of hospitalization, use of illegal drugs, and beliefs about medication were predictive in our multivariate analyses.

Our results are generally consistent with the predictors of hospitalization from other first-episode trials or longitudinal follow-up studies with the exception of the findings about DUP. Our finding of individuals who had prior hospitalizations being at increased risk for hospitalization during the trial is consistent with other studies of first-episode populations over the first years of treatment (18,40). This vulnerability may persist for longer periods as Mortensen and Eaton (41) found that time to readmission over the first 10 years following a first admission for schizophrenia become shorter with greater number of admissions. As with our results, other first-episode studies have identified hospitalization risk factors of the presence of psychosis (18,4244) and excitement symptoms (45), the use of illegal drugs (11,4650) and poor medication adherence (48,5154). In our study, individual self-report of adherence over time predicted hospitalization at a trend level in univariate analyses and the association with beliefs about medication was significant in the multivariate analyses.

DUP is a predictor of several outcome domains of FEP (reviewed (55,56)). In contrast, no association between DUP and hospitalization risk has been found with several first-episode populations (18,43,44,57) although we and an analysis by Sipos and colleagues (45) did find an association. The studies come from a variety of countries with different health systems and pathways to care that may have contributed to the variability of results. Further, comparison across studies is complicated by the often skewed distribution of DUP. For example, although the median duration in RAISE-ETP was 74 weeks, 23.8% of participants had a duration of 3 months or less, the target duration of untreated psychosis in the Consensus statement (58) of the World Health Organization and the International Early Psychosis Association. Nevertheless, the DUP in all of the studies that did not find an association with hospitalization risk was shorter than the median 74 weeks in RAISE-ETP. It is possible that once DUP is shortened to a particular degree further DUP shortening does not decrease hospitalization risk. Research is needed to clarify the effect of DUP on first-episode hospitalization risk and determine what, if any, is the minimal DUP that is associated with increased hospitalization risk.

Our findings have implications for future efforts to enhance EIS. Individuals enter outpatient treatment with an already fixed number of prior hospitalizations and duration of untreated psychosis. Changing these factors will require public health initiatives and innovative outreach strategies (e.g. (59)) to facilitate earlier entry into treatment. These baseline characteristics can also be used to identify individuals at increased hospitalization risk who might be candidates for interventions specifically targeted to decrease hospitalization risk such as individualized relapse prevention plans. Current EIS models include interventions to help individuals decrease substance misuse, achieve reduction of symptoms and have an understanding of medications and adherence. Some of these interventions have low participation by individuals who would benefit from them (e.g. substance misuse interventions (60)) suggesting more effort may be needed to motivate individuals to use available services. Further direct development/refinement of the interventions such as innovative strategies to support medication adherence (6163) should also be considered.

To be a RAISE-ETP site, facilities had to have an interest in participating in such a study and to have the clinical and administrative infrastructure capable of providing NAVIGATE treatment if the site was randomly assigned to provide NAVIGATE. A limitation of generalizing our finding to the entire range of community clinics is that the site inclusion criteria may have selected clinics with above average motivation and resources to serve individuals with FEP. Our study sites were outpatient facilities. Our data do not address predictors of hospitalization for first-episode individuals who never come to outpatient treatment (e.g. those whose only treatment occurs on inpatient units).

In summary, current treatment practices can reduce the risk of hospitalization of individuals with FEP but further efforts at reducing hospitalization risk are needed. Potential targets for further intervention development include reducing the length of DUP and the number of hospitalizations before EIS care commences, decreasing substance misuse, decreasing symptoms and enhancing adherence. Better intervention could enhance the impact of first-episode EIS treatment models as well as enhance outcomes of FEP treated with other models.

Supplementary Material

supplement

Highlights:

  • Current treatment practices for first episode psychosis decrease hospitalization risk but even with integrated early intervention services a third or more of individuals with first episode psychosis will be hospitalized during the first two years of treatment

  • Targets for further intervention development based upon significant predictors of hospitalization in the RAISE-ETP study include baseline characteristics of the number of hospitalizations before study entry and the duration of untreated psychosis and, over the course of an individual’s participation in the study, longitudinal days of substance misuse, presence of Positive and Negative Syndrome Scale positive symptoms, and beliefs about the value of medication.

  • Although longer duration of untreated psychosis has been associated with poor outcomes in other domains, it has not been previously associated with hospitalization risk in many first episode studies.

  • Although early intervention models include interventions for our intervention targets, a challenge for the field is to develop more effective interventions or enhance utilization of existing interventions for earlier treatment engagement, minimizing symptoms, decreasing substance use and facilitating medication adherence to further reduce hospitalization risk.

Acknowledgments:

We wish to acknowledge the contributions of the RAISE-ETP study participants without whose participation the study would not have been possible. We also wish to acknowledge the contributions of the research personnel at the study sites and at the central research center and affiliated institutions.

This work has been funded in whole or in part with funds from the American Recovery and Reinvestment Act and from the National Institute of Mental Health (NIMH) under contract HHSN271200900019C. Additional support for these analyses was provided by an NIMH Advanced Centers for Intervention and/or Services Research award (P30MH090590).

Disclosures:

Dr. Robinson has been a consultant to Costello Medical Consulting, Innovative Science Solutions, Janssen, Neurocrine, Otsuka and US WorldMeds. Dr. Schooler has received grant support from Otsuka and has provided consultation or participated in advisory boards for Allergan, Alkermes and Roche. Ms. Marcy is shareholder in Pfizer and is the executive director of the Vanguard Research Group. Dr. Kane has been a consultant for or received honoraria from Alkermes, Eli Lilly, EnVivo Pharmaceuticals (Forum), Forest (Allergan), Genentech, H. Lundbeck. Intracellular Therapies, Janssen Pharmaceutica, Johnson and Johnson, Merck, Neurocrine, Otsuka, Pierre Fabre, Reviva, Roche, Sunovion, Takeda and Teva. He has has received grant support from Otsuka, Lundbeck and Janssen. Dr. Kane is a Shareholder in Vanguard Research Group and LB Pharmaceuticals, Inc. Drs. Rosenheck and Lin and Mr. Sint have no interests to report.

References

  • 1.Zubi Y, Connolly P: Recollected experiences of first hospitalisation for acute psychosis among persons diagnosed with schizophrenia in South Africa. South African Journal of Psychiatry 19: 81–85, 2013. [Google Scholar]
  • 2.Rodrigues R, Anderson KK: The traumatic experience of first-episode psychosis: a systematic review and meta-analysis. Schizophrenia research 189: 27–36, 2017. [DOI] [PubMed] [Google Scholar]
  • 3.Corcoran C, Gerson R, Sills-Shahar R, et al. : Trajectory to a first episode of psychosis: a qualitative research study with families. Early Intervention in Psychiatry 1: 308–315, 2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Weller BE, Faulkner M, Doyle O, et al. : Impact of Patients’ Psychiatric Hospitalization on Caregivers: A Systematic Review. Psychiatric Services 66: 527–535, 2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Hastrup LH, Kronborg C, Bertelsen M, et al. : Cost-effectiveness of early intervention in first-episode psychosis: economic evaluation of a randomised controlled trial (the OPUS study). British Journal of Psychiatry 202: 35–41, 2013. [DOI] [PubMed] [Google Scholar]
  • 6.McCrone P, Craig TKJ, Power P, et al. : Cost-effectiveness of an early intervention service for people with psychosis. British Journal of Psychiatry 196: 377–382, 2010. [DOI] [PubMed] [Google Scholar]
  • 7.Rosenheck R, Leslie D, Sint K, et al. : Cost-effectiveness of comprehensive, integrated care for first episode psychosis in the NIMH RAISE Early Treatment Program. Schizophrenia bulletin 42: 896–906, 2016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Correll CU, Galling B, Pawar A, et al. : Comparison of Early Intervention Services vs Treatment as Usual for Early-Phase Psychosis: A Systematic Review, Meta-analysis, and Meta-regression. JAMA Psychiatry, 2018Available from: https://jamanetwork.com/journals/jamapsychiatry/fullarticle/2679768 [DOI] [PMC free article] [PubMed]
  • 9.Kuipers E, Holloway F, Rabe-Hesketh S, et al. : An RCT of early intervention in psychosis: Croydon Outreach and Assertive Support Team (COAST). Social Psychiatry and Psychiatric Epidemiology 39: 358–363, 2004. [DOI] [PubMed] [Google Scholar]
  • 10.Craig TKJ, Garety P, Power P, et al. : The Lambeth Early Onset (LEO) Team: randomised controlled trial of the effectiveness of specialised care for early psychosis. BMJ (Clinical research ed.) 329: 1067, 2004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Petersen L, Jeppesen P, Thorup A, et al. : A randomised multicentre trial of integrated versus standard treatment for patients with a first episode of psychotic illness. BMJ (Clinical research ed.) 331: 602, 2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Grawe RW, Falloon IRH, Widen JH, et al. : Two years of continued early treatment for recent-onset schizophrenia: a randomised controlled study. Acta Psychiatrica Scandinavica 114: 328–336, 2006. [DOI] [PubMed] [Google Scholar]
  • 13.Valencia M, Juarez F, Ortega H: Integrated treatment to achieve functional recovery for first-episode psychosis. Schizophrenia Research and Treatment 2012: 962371, 2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Valencia M, Juarez F, Delgado M, et al. : Early Intervention to Improve Clinical and Functional Outcome in Patients with First Episode-Psychosis [Internet]; in Mental Disorder. iConcept Press, 2014. [cited 2018 May 24]. Available from: https://www.iconceptpress.com/book/mental-disorder/11000123/1305000979/
  • 15.Ruggeri M, Bonetto C, Lasalvia A, et al. : Feasibility and Effectiveness of a Multi-Element Psychosocial Intervention for First-Episode Psychosis: Results From the Cluster-Randomized Controlled GET UP PIANO Trial in a Catchment Area of 10 Million Inhabitants. Schizophrenia Bulletin 41: 1192–1203, 2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Kane JM, Robinson DG, Schooler NR, et al. : Comprehensive versus usual community care for first-episode psychosis: 2-year outcomes from the NIMH RAISE early treatment program. American Journal of Psychiatry 173: 362–372, 2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Srihari VH, Tek C, Kucukgoncu S, et al. : First-Episode Services for Psychotic Disorders in the U.S. Public Sector: A Pragmatic Randomized Controlled Trial. Psychiatric Services (Washington, D.C.) 66: 705–712, 2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Addington DE, Beck C, Wang J, et al. : Predictors of Admission in First-Episode Psychosis: Developing a Risk Adjustment Model for Service Comparisons. Psychiatric Services 61: 483–488, 2010. [DOI] [PubMed] [Google Scholar]
  • 19.Dixon LB, Goldman HH, Bennett M, et al. : Implementing Coordinated Specialty Care for Early Psychosis: The RAISE Connection Program. Psychiatric services (Washington, D.C.) 66: 691–698, 2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Mueser KT, Penn DL, Addington J, et al. : The NAVIGATE program for first-episode psychosis: rationale, overview, and description of psychosocial components. Psychiatric Services 66: 680–690, 2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Kane JM, Schooler NR, Patricia Marcy BSN, et al. : The RAISE early treatment program for first-episode psychosis: Background, rationale, and study design. Journal of Clinical Psychiatry 76: 240–246, 2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Donner A, Klar N: Design and analysis of cluster randomization trials in health research. Chichester, Wiley & Sons, 2000 [Google Scholar]
  • 23.First M, Spitzer R, Gibbon M, et al. : Structured Clinical Interview for Axis I DSM-IV Disorders (SCID-I/P), 1994. [Google Scholar]
  • 24.Kay SR, Fiszbein A, Opler LA: The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophrenia Bulletin 13: 261–276, 1987. [DOI] [PubMed] [Google Scholar]
  • 25.Guy W, Bonato RR: CGI: Clinical global impressions. ECDEU Assessment Manual for Psychopharmacology, Revised 217–222, 1976. [Google Scholar]
  • 26.Addington D, Addington J, Maticka-Tyndale E: Assessing depression in schizophrenia: the Calgary Depression Scale. The British Journal of Psychiatry. Supplement 39–44, 1993. [PubMed] [Google Scholar]
  • 27.Heinrichs DW, Hanlon TE, Carpenter WT: The Quality of Life Scale: an instrument for rating the schizophrenic deficit syndrome. Schizophrenia Bulletin 10: 388–398, 1984. [DOI] [PubMed] [Google Scholar]
  • 28.Zarate CA, Weinstock L, Cukor P, et al. : Applicability of telemedicine for assessing patients with schizophrenia: acceptance and reliability. The Journal of Clinical Psychiatry 58: 22–25, 1997. [DOI] [PubMed] [Google Scholar]
  • 29.Rosenheck R, Kasprow W, Frisman L, et al. : Cost-effectiveness of supported housing for homeless persons with mental illness. Archives of General Psychiatry 60: 940–951, 2003. [DOI] [PubMed] [Google Scholar]
  • 30.Rosenheck RA, Leslie DL, Sindelar J, et al. : Cost-effectiveness of second-generation antipsychotics and perphenazine in a randomized trial of treatment for chronic schizophrenia. The American Journal of Psychiatry 163: 2080–2089, 2006. [DOI] [PubMed] [Google Scholar]
  • 31.Leon AC, Demirtas H, Hedeker D: Bias reduction with an adjustment for participants’ intent to dropout of a randomized controlled clinical trial. Clinical Trials 4: 540–547, 2007. [DOI] [PubMed] [Google Scholar]
  • 32.McHorney CA: The Adherence Estimator: a brief, proximal screener for patient propensity to adhere to prescription medications for chronic disease. Current Medical Research and Opinion 25: 215–238, 2009. [DOI] [PubMed] [Google Scholar]
  • 33.Dolder CR, Lacro JP, Warren KA, et al. : Brief evaluation of medication influences and beliefs: development and testing of a brief scale for medication adherence. Journal of Clinical Psychopharmacology 24: 404–409, 2004. [DOI] [PubMed] [Google Scholar]
  • 34.King M, Dinos S, Shaw J, et al. : The Stigma Scale: development of a standardised measure of the stigma of mental illness. The British Journal of Psychiatry: The Journal of Mental Science 190: 248–254, 2007. [DOI] [PubMed] [Google Scholar]
  • 35.Ryff CD: Happiness is everything, or is it? Explorations on the meaning of psychological well-being. Journal of personality and social psychology 57: 1069, 1989. [Google Scholar]
  • 36.Williams GC, Rodin GC, Ryan RM, et al. : Autonomous regulation and long-term medication adherence in adult outpatients. Health Psychology 17: 269, 1998. [DOI] [PubMed] [Google Scholar]
  • 37.Young S, Bullock W: The Mental Health Recovery Measure. Toledo, Ohio, University of Toledo, 2003 [Google Scholar]
  • 38.Wallwork RS, Fortgang R, Hashimoto R, et al. : Searching for a consensus five-factor model of the Positive and Negative Syndrome Scale for schizophrenia. Schizophrenia Research 137: 246–250, 2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Addington J, Heinssen RK, Robinson DG, et al. : Duration of Untreated Psychosis in Community Treatment Settings in the United States. Psychiatric Services (Washington, D.C.) 66: 753–756, 2015. [DOI] [PubMed] [Google Scholar]
  • 40.Turner MA, Boden JM, Mulder RT: Predictors of hospitalization two years after treatment for first-episode psychosis. Psychiatric Services (Washington, D.C.) 64: 1230–1235, 2013. [DOI] [PubMed] [Google Scholar]
  • 41.Mortensen PB, Eaton WW: Predictors for readmission risk in schizophrenia. Psychological Medicine 24: 223–232, 1994. [DOI] [PubMed] [Google Scholar]
  • 42.Craig TJ, Fennig S, Tanenberg-Karant M, et al. : Rapid versus delayed readmission in first-admission psychosis: quality indicators for managed care? Annals of Clinical Psychiatry 12: 233–238, 2000. [DOI] [PubMed] [Google Scholar]
  • 43.Cougnard A, Parrot M, Grolleau S, et al. : Pattern of health service utilization and predictors of readmission after a first admission for psychosis: a 2-year follow-up study. Acta Psychiatrica Scandinavica 113: 340–349, 2005. [DOI] [PubMed] [Google Scholar]
  • 44.Wade D, Harrigan S, Harris MG, et al. : Pattern and correlates of inpatient admission during the initial acute phase of first-episode psychosis. Australian & New Zealand Journal of Psychiatry 40: 429–436, 2006. [DOI] [PubMed] [Google Scholar]
  • 45.Sipos A, Harrison G, Gunnell D, et al. : Patterns and predictors of hospitalisation in first-episode psychosis. Prospective cohort study. The British Journal of Psychiatry: The Journal of Mental Science 178: 518–523, 2001. [DOI] [PubMed] [Google Scholar]
  • 46.Patel R, Wilson R, Jackson R, et al. : Association of cannabis use with hospital admission and antipsychotic treatment failure in first episode psychosis: an observational study. BMJ open 6: e009888, 2016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Schoeler T, Petros N, Di Forti M, et al. : Effects of continuation, frequency, and type of cannabis use on relapse in the first 2 years after onset of psychosis: an observational study. The Lancet. Psychiatry 3: 947–953, 2016. [DOI] [PubMed] [Google Scholar]
  • 48.Schoeler T, Petros N, Di Forti M, et al. : Association Between Continued Cannabis Use and Risk of Relapse in First-Episode Psychosis: A Quasi-Experimental Investigation Within an Observational Study. JAMA psychiatry 73: 1173–1179, 2016. [DOI] [PubMed] [Google Scholar]
  • 49.Schoeler T, Petros N, Di Forti M, et al. : Effect of continued cannabis use on medication adherence in the first two years following onset of psychosis. Psychiatry Research 255: 36–41, 2017. [DOI] [PubMed] [Google Scholar]
  • 50.Schoeler T, Petros N, Di Forti M, et al. : Poor medication adherence and risk of relapse associated with continued cannabis use in patients with first-episode psychosis: a prospective analysis. The Lancet. Psychiatry 4: 627–633, 2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.Verdoux H, Lengronne J, Liraud F, et al. : Medication adherence in psychosis: predictors and impact on outcome. A 2-year follow-up of first-admitted subjects. Acta Psychiatrica Scandinavica 102: 203–210, 2000. [DOI] [PubMed] [Google Scholar]
  • 52.Morken G, Widen JH, Grawe RW: Non-adherence to antipsychotic medication, relapse and rehospitalisation in recent-onset schizophrenia. BMC Psychiatry 8: 32, 2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.Gearing RE, Mian I, Sholonsky A, et al. : Developing a Risk-Model of Time to First-Relapse for Children and Adolescents With a Psychotic Disorder: The Journal of Nervous and Mental Disease 197: 6–14, 2009. [DOI] [PubMed] [Google Scholar]
  • 54.Bodén R, Brandt L, Kieler H, et al. : Early non-adherence to medication and other risk factors for rehospitalization in schizophrenia and schizoaffective disorder. Schizophrenia Research 133: 36–41, 2011. [DOI] [PubMed] [Google Scholar]
  • 55.Perkins DO, Gu H, Boteva K, et al. : Relationship between duration of untreated psychosis and outcome in first-episode schizophrenia: a critical review and meta-analysis. American Journal of Psychiatry 162: 1785–1804, 2005. [DOI] [PubMed] [Google Scholar]
  • 56.Penttilä M, Jääskeläinen E, Hirvonen N, et al. : Duration of untreated psychosis as predictor of long-term outcome in schizophrenia: systematic review and meta-analysis. The British Journal of Psychiatry: The Journal of Mental Science 205: 88–94, 2014. [DOI] [PubMed] [Google Scholar]
  • 57.Craig TJ, Bromet EJ, Fennig S, et al. : Is there an association between duration of untreated psychosis and 24-month clinical outcome in a first-admission series? American Journal of Psychiatry 157: 60–66, 2000. [DOI] [PubMed] [Google Scholar]
  • 58.Bertolote J, Mcgorry P: Early intervention and recovery for young people with early psychosis: consensus statement. The British Journal of Psychiatry 187: s116–s119, 2005. [DOI] [PubMed] [Google Scholar]
  • 59.Birnbaum ML, Garrett C, Baumel A, et al. : Using Digital Media Advertising in Early Psychosis Intervention. Psychiatric Services (Washington, D.C.) 68: 1144–1149, 2017. [DOI] [PubMed] [Google Scholar]
  • 60.Cather C, Brunette MF, Mueser KT, et al. : Impact of comprehensive treatment for first episode psychosis on substance use outcomes: A randomized controlled trial. Psychiatry Research 268: 303–311, 2018. [DOI] [PubMed] [Google Scholar]
  • 61.Velligan D, Mintz J, Maples N, et al. : A randomized trial comparing in person and electronic interventions for improving adherence to oral medications in schizophrenia. Schizophrenia Bulletin 39: 999–1007, 2013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 62.Kane JM, Perlis RH, DiCarlo LA, et al. : First experience with a wireless system incorporating physiologic assessments and direct confirmation of digital tablet ingestions in ambulatory patients with schizophrenia or bipolar disorder. The Journal of Clinical Psychiatry 74: e533–540, 2013. [DOI] [PubMed] [Google Scholar]
  • 63.Labovitz DL, Shafner L, Reyes Gil M, et al. : Using Artificial Intelligence to Reduce the Risk of Nonadherence in Patients on Anticoagulation Therapy. Stroke 48: 1416–1419, 2017. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

supplement

RESOURCES