Abstract
Objective
To assess the impact of a New York City supportive housing program on housing stability and preventable emergency department (ED) visits/hospitalizations among heads of homeless families with mental and physical health conditions or substance use disorders.
Data Sources
Multiple administrative data from New York City and New York State for 966 heads of families eligible for the program during 2007–12.
Study Design
We captured housing events and health care service utilization during 2 years prior to the first program eligibility date (baseline) and 2 years postbaseline. We performed sequence analysis to measure housing stability and compared housing stability and preventable ED visits and hospitalizations between program participants (treatment group) and eligible applicants not placed in the program (comparison group) via marginal structural modeling.
Data Collection/Extraction Methods
We matched electronically collected data.
Principal Findings
Eighty‐seven percent of supportive housing tenants experienced housing stability in 2 years postbaseline. Compared with unstably housed heads of families in the comparison group, those in the treatment group were 0.60 times as likely to make preventable ED visits postbaseline (95% CI = 0.38, 0.96).
Conclusions
Supportive housing placement was associated with improved housing stability and reduced preventable health care visits among homeless families.
Keywords: Administrative data uses, homeless populations, Medicaid
Homelessness is associated with increased risk of emergency department (ED) visits and hospitalizations that might have been avoided with proper management of chronic conditions (Kushel et al. 2006; Bharel et al. 2013). Homeless individuals may experience a financial barrier to primary care due to urgent needs for survival such as food and stable housing (Kushel et al. 2006; White and Newman 2015). Additionally, distrust in public health systems, social isolation, and discrimination might be nonfinancial barriers to primary care among homeless individuals (Baggett et al. 2010; Hwang and Henderson 2010).
Despite recent efforts to increase access to health care via the Affordable Care Act, urban homelessness in the United States has been associated with preventable ED visits and hospitalizations even when controlling for health insurance coverage (Lin et al. 2015). A similar result was observed in Canada, where the universal health insurance system guarantees access to care for all citizens (Hwang et al. 2013). These studies highlight the importance of efforts to address the social welfare and primary care needs of homeless individuals to reduce risk of preventable ED visits and hospitalizations.
Several studies of homeless, single adults have found that supportive housing effectively reduced homelessness, health care costs, ED visits, and hospitalizations (Larimer et al. 2009; Sadowski et al. 2009; Parker 2010). Yet evidence is lacking for homeless families. Between 2009 and 2017, the number of families with children living in New York City (NYC) shelters increased from 8,081 to 12,761 and families currently represent two‐thirds of the homeless population in NYC (New York City Department of Homeless Services 2017). The association between supportive housing and residential stability is inconclusive among homeless families in the United States (Bassuk et al. 2014), indicating that the impact of supportive housing for homeless single adults may be different from that of homeless families.
The purpose of this evaluation was to assess the impact of a NYC supportive housing program, known as New York/New York III (NY/NY III), on housing stability among heads of chronically homeless families with serious mental illness, substance use disorders, physical disabilities, or HIV/AIDS. In addition, we tested whether housing stability influences the association between supportive housing and preventable ED visits and hospitalizations.
Methods
Setting and Samples
The population evaluated herein included heads of chronically homeless families with mental illness, substance use disorders, physical disabilities, or HIV/AIDS who were eligible applicants to the NY/NY III supportive housing program. NY/NY III is funded by NYC and New York State and provides permanent, independent housing to individuals who are chronically homeless or have high risk of homelessness. The NY/NY III program defines a chronically homeless family if (1) a family has lived in a homeless shelter for at least 365 days of the last 2 years (not necessarily consecutively), or (2) a head of household with mental illness or substance use disorders has spent at least one of the last 2 years in a shelter or lived on the street and will reunify with their child(ren) via the NY/NY III supportive housing program. Homeless families are placed in congregate housing, and contracted agencies provide case management and other services such as primary medical care, counseling, and childcare, either directly or through referral.
The NY/NY III housing application collected baseline (i.e., on the date a person was determined to be eligible for NY/NY III) and prebaseline information about demographics, substance use, and physical and mental health conditions. Administrative records about eligible applicants for 2 years prior to baseline and 2 years postbaseline were also pulled from matches with a number of other data sources: NYC health registries (mortality, HIV, and STI), NYC subsidized housing programs, jails, homeless shelters, New York State psychiatric facilities, Medicaid, cash assistance, and food stamps. Detailed information about the NY/NY III program has been published elsewhere (Levanon Seligson et al. 2013).
There were 966 heads of families eligible for NY/NY III during 2007–12. Applicants were categorized into two groups: (1) placed in NY/NY III housing for >7 days (“treatment group”); and (2) not placed in NY/NY III or placed in NY/NY III for ≤7 days (“comparison group”). Housing placement was not random, and persons were selected by housing agencies after interviewing a maximum of three eligible applicants for each vacant unit. Eight treated persons who resided in other government‐subsidized housing within 1 day prior to their move‐in to NY/NY III were excluded because they were considered to have received treatment prior to baseline. In addition, 32 persons placed in other government‐subsidized housing in the first 6 months after eligibility determination were excluded because they were considered to be treated. The final sample in this evaluation had 926 heads of families. Treated persons remained in the analysis as part of the treatment group even if they left the NY/NY III housing, and comparison persons remained in the comparison group even if they were placed in other government‐subsidized housing after the first 6 months. The NYC Department of Health and Mental Hygiene Institutional Review Board determined this study to be program evaluation not human subjects research and therefore not under its purview.
Variables
The primary outcome of interest was housing stability in the 2 years after baseline. This outcome was measured using sequence analysis, which allowed us to characterize the pattern of occupancy in supportive housing, jails, homeless shelters, and hospitals (i.e., “housing patterns”). In this evaluation, we defined housing stability as a pattern of a continuous placement in supportive housing, rarely disrupted by incarceration, homelessness, or hospitalization. We did not consider consistent placement in other types of institutions as housing stability because supportive housing placement, as opposed to other institutional placement, was designed to result in permanent, independent living.
The secondary outcome was ambulatory care sensitive (ACS or preventable) ED visits and hospitalizations postbaseline. ACS conditions included in the analysis were angina, asthma, bacterial pneumonia, chronic obstructive pulmonary disorder, chronic ulcer of skin, congestive heart failure, convulsion, dehydration, diabetes, gastroenteritis, hypoglycemia, hypertension, kidney/urinary infection, severe ENT infections, and tuberculosis (Billings et al. 1993). We calculated ACS rates as the total number of ACS ED visits or hospitalizations per individual divided by the total Medicaid eligible days per individual. A decrease in preventable ED visits and hospitalizations might lead to an increase in outpatient care visits for ACS conditions; therefore, we calculated the rate of outpatient care visits for ACS conditions postbaseline.
Along with the outcomes, we included covariates that captured demographic, clinical, and public service use characteristics to describe the sample and adjust for baseline differences between the treatment and comparison groups. See Appendix SA2 for the full list of covariates.
Statistical Analysis
Sequence Analysis
We performed sequence analysis to identify distinct housing patterns for 1 year before program eligibility and 2 years after eligibility. Sequence analysis is a statistical method that generates nonoverlapping groups of individuals who share similar duration and sequencing of events (Abbott and Tsay 2000; McAllister, Lennon, and Kuang 2011). We used five types of events for this analysis: jail incarceration, medical and psychiatric hospitalization, unstable government housing (homeless shelter system, supported or transitional single room occupancy), stable government housing (public housing, NY/NY III, other government‐subsidized housing programs), and noninstitutional stay (e.g., independent housing, street homelessness, and residence outside of NYC). Each person's records were divided into time periods of 30 days, and a single housing event was assigned to each 30‐day period. For example, if a person was incarcerated during a given 30‐day period, that period was marked as an incarceration event. If there was no record of stable housing or an institutional stay during a 30‐day period (i.e., no record of where a family was housed), that period was categorized as a noninstitutional stay. If more than one type of event occurred within a 30‐day period, which was less than 1 percent of total housing events, the number of days associated with each was counted and the event with the higher numbers of days was assigned to that period. If the events were of equal duration, the event less prevalent in the overall individual‐level sequence was assigned. We assessed the degree of dissimilarity among all possible pairs of sequences using an algorithm known as Levenshtein distance (Abbott and Tsay 2000). This was repeated over all possible pairs and summarized as a distance matrix. We then performed a hierarchical cluster analysis with the Ward method, resulting in nonoverlapping clusters representing housing patterns. In other words, individual‐level sequences of monthly housing events (represented as horizontal lines) for 1 year before program eligibility and 2 years after eligibility were stacked together and divided into distinct clusters based on their similarities. A full description of sequence analysis to describe housing patterns has been previously published (Lim, Singh, and Gwynn 2017).
IPTW and Marginal Structural Modeling
We used inverse probability of treatment weights (IPTW) to control for observed differences in baseline and prebaseline characteristics between treatment and comparison groups. To obtain an individual's probability of being in the treatment group (i.e., propensity score), we developed a propensity score model using logistic regression with placement in NY/NY III as the dependent variable and covariates listed in Appendix SA2 as the independent variables. A categorical variable for the prebaseline housing pattern was also included in the propensity score model. IPTW were calculated as the inverse of the propensity score. To minimize influences from large weights, we created stabilized IPTW by multiplying the IPTW by the marginal probability of treatment (Robins, Hernan, and Brumback 2000; Cole and Hernan 2008).
A separate marginal structural model for each of three outcomes (preventable ED visits, preventable hospitalizations, and outpatient visits due to ACS conditions) was used to estimate an average treatment effect of NY/NY III placement while adjusting for confounding via stabilized IPTW (Robins, Hernan, and Brumback 2000). The covariates listed in Appendix SA2 were included as independent variables to make point estimates robust against model misspecification (i.e., doubly robust estimation; Funk et al. 2011). The total number of ED visits, hospitalizations, or outpatient visits prior to baseline were included as an additional covariate in each marginal structural model because health service use prebaseline was strongly associated with use postbaseline. The number of days of Medicaid eligibility postbaseline was included as an offset term to account for the greater probability of health service use for persons with health insurance. A sandwich estimator was used to calculate standard errors that were robust against model misspecification. Note that we did not use IPTW when performing sequence analysis. The purpose of sequence analysis was to create a housing stability outcome for all participants, and we believed that this measurement, similar to health outcomes, was independent of treatment status. Lastly, because housing placement was not random, we conducted a sensitivity analysis to quantify bias due to unobserved confounding using the bias equation of VanderWeele and Arah. Specifically, we assessed the impact of unobserved confounding on the rate ratios of preventable ED visits by supportive housing placement (VanderWeele and Arah 2011).
Statistical significance was determined using two‐sided p value <.05. Sequence analysis was performed using TraMineR and cluster packages in R 2.14.2 software (R Foundation for Statistical Computing, Vienna, Austria). All other analyses were performed using SAS 9.2 software (SAS Institute, Inc., Cary, NC).
Results
Of 926 eligible heads of families experiencing chronic homelessness who applied for the NY/NY III supportive housing program, most were women and persons of color (Table 1). Their average age was 35 years. Compared with unplaced applicants (comparison group), those placed in NY/NY III (treatment group) were less likely to be mentally ill, have a history of substance use, and have a history of violence‐related symptoms or behaviors. Treated persons were more likely to need assistance with activities of daily living. There were no significant differences in health care utilization between the treatment and comparison group for the 2 years before baseline. Three housing patterns were identified in the year before baseline: (1) unstable housing; (2) increasing housing instability; and (3) noninstitutional stay (Appendix SA3). Those in the treatment group, as opposed to the comparison group, were more likely to have the unstable housing pattern (75 vs. 62 percent), reflecting more homeless shelter stays, and less likely to have the noninstitutional stay pattern (13 vs. 27 percent) prior to being eligible for NY/NY III.
Table 1.
Selected Baseline Demographic Characteristics among Heads of Homeless Families Who Were Eligible for the NY/NY III Program
| Total | Treatment | Comparison | p‐Value | |
|---|---|---|---|---|
| N | 926 | 300 | 626 | |
| Sex | ||||
| Female | 79% | 78% | 80% | .51 |
| Male | 21% | 22% | 20% | |
| Race/ethnicity | ||||
| Non‐Latino white | 7% | 7% | 7% | .99 |
| Non‐Latino black | 47% | 48% | 47% | |
| Latino | 44% | 44% | 44% | |
| Others | 2% | 1% | 2% | |
| Any substance use historiesa: yes | 33% | 23% | 38% | <.01 |
| Any mental illness at the time of applicationb: yes | 77% | 65% | 83% | <.01 |
| Severe physical illnessc: yes | 48% | 50% | 47% | .36 |
| Any diagnoses of physical illness: yes | 67% | 65% | 68% | .28 |
| Education | ||||
| <High school diploma | 53% | 51% | 54% | .63 |
| High school diploma or higher | 45% | 47% | 44% | |
| Others | 2% | 2% | 2% | |
| No. of assistance required for activities of daily livingd | ||||
| 0 | 28% | 20% | 32% | <.01 |
| 1 | 16% | 19% | 15% | |
| 2–3 | 28% | 31% | 26% | |
| 4–10 | 28% | 29% | 27% | |
| Mean age on 1/1/2007 (SD) | 35 (11) | 35 (10) | 34 (11) | .18 |
| Mean outpatient visits for 2 years prior to baseline (SD) | 71 (82) | 76 (74) | 69 (86) | .18 |
| Mean hospitalizations for 2 years prior to baseline (SD) | 1 (3) | 1 (3) | 1 (2) | .97 |
| Mean ED visits for 2 years prior to baseline (SD) | 3 (5) | 3 (4) | 3 (6) | .13 |
Based on use of substances (alcohol, amphetamines, cocaine, crack, hallucinogens, opiates, marijuana, PCP, sedative/hypnotics, others) in the past.
Determined according to Axis I or Axis II codes of the Diagnostic and Statistical Manual of Mental Disorders, 4th edition.
Determined according to Charlson comorbidity index (Charlson, M. E., P. Pompei, K. L. Ales, and C. R. MacKenzie. 1987. A New Method of Classifying Prognostic Comorbidity in Longitudinal Studies: Development and Validation. Journal of Chronic Diseases 40 (5): 373–83.)
Activities for daily living include walking and climbing, traveling, hearing, vision, feeding and meal preparation, housekeeping, cognitive functions, managing finances, toileting, and personal hygiene.
SD, standard deviation.
Sequence analysis identified four housing patterns during the 2 years postbaseline: (1) stable housing; (2) noninstitutional stay; (3) unstable housing; and (4) transition to noninstitutional stay (Figure 1). Twenty‐nine percent of the cohort followed the stable housing pattern, which was characterized by consistent placement in supportive housing (568 days on average, Table 2) after a brief period of unstable government‐subsidized housing. Seventeen percent followed the noninstitutional stay pattern as they spent most of the follow‐up period in a noninstitutional setting. Twenty percent had consistent placement (585 days on average) in unstable government housing, which was labeled as the unstable housing pattern. Lastly, the one‐third of the cohort (34 percent) experienced a transition from unstable government housing to a noninstitutional setting (transition to noninstitutional stay).
Figure 1.
- Notes: Each horizontal line in the y‐axis represents an individual‐level sequence of monthly housing events during 2 years postbaseline. The x‐axis represents each month during 2 years postbaseline. Change in color indicates transition between different housing events. For example, if a blue line up to the first six months is switched to a red line afterwards, it represents an individual who initially stayed in unstable government housing for 6 months and moved out of NYC institutions for the next 18 months. These individual‐level sequences are stacked together and divided into four distinct clusters (or plots) based on their similarities. The height of the original plot is proportional to the number of individuals in each cluster, but then adjusted to the same size to more clearly show color patterns.
Table 2.
Frequencies and Duration of Hospitalization, Incarceration, Unstable Government Housing, Stable Government Housing, and Noninstitutional Stay by Four Housing Stability Patterns for 2 years after the First NY/NY III Eligibility Dates among Heads of Homeless Families Who Were Eligible for the NY/NY III Program
| Total | Stable Housing | Noninstitutional Stay | Unstable Housing | Transition to Noninstitutional Stay | |
|---|---|---|---|---|---|
| N | 926 | 268 | 155 | 187 | 316 |
| Mean number of hospitalization (SD) | 0.7 (1.8) | 0.6 (1.5) | 0.5 (1.0) | 0.7 (1.3) | 0.9 (2.4) |
| Mean number of incarceration (SD) | 0.1 (0.5) | 0.0 (0.2) | 0.1 (0.3) | 0.2 (0.8) | 0.1 (0.5) |
| Mean number of stable government housing (SD) | 0.3 (0.5) | 1.0 (0.1) | 0.0 (0.1) | 0.2 (0.4) | 0.0 (0.1) |
| Mean number of unstable government housing (SD) | 0.5 (2.1) | 0.1 (0.3) | 0.2 (0.6) | 1.1 (3.6) | 0.7 (2.2) |
| Mean number of noninstitutional stay (SD) | 1.3 (2.6) | 0.2 (0.5) | 1.0 (1.1) | 1.6 (3.6) | 2.1 (3.1) |
| Mean length of hospitalization (SD) | 4.9 (21.4) | 4.3 (23.8) | 2.9 (8.3) | 4.1 (14.4) | 6.8 (26.6) |
| Mean length of incarceration (SD) | 3.7 (26.6) | 1.1 (13.2) | 1.0 (8.0) | 4.3 (23.1) | 6.8 (39.5) |
| Mean length of stable government housing (SD) | 173.5 (263.1) | 567.7 (109.8) | 4.6 (51.9) | 38.8 (91.4) | 1.9 (16.1) |
| Mean length of unstable government housing (SD) | 231.7 (211.0) | 137.1 (97.8) | 16.3 (24.7) | 585.3 (116.0) | 208.2 (95.5) |
| Mean length of noninstitutional stay (SD) | 308.1 (279.6) | 37.4 (110.4) | 628.3 (222.7) | 99.8 (106.1) | 503.8 (106.9) |
SD, standard deviation.
After controlling for differences in baseline characteristics between the treatment and comparison groups via stabilized IPTW, 87 percent of the treatment group experienced housing stability, and the remaining 13 percent had the three other housing patterns. In contrast, only 1 percent of the comparison group exhibited the stable housing pattern. The remaining persons in the comparison group followed the transition to noninstitutional stay (53 percent), unstable housing (24 percent), and noninstitutional stay (22 percent) patterns.
On average, the treatment and comparison group persons were eligible for Medicaid for 1.8 years and 1.6 years of the 2‐year follow‐up period, respectively. The weighted rate of preventable ED visits among the treatment group was 0.78 times that among the comparison group (95% CI = 0.58, 1.05; 309 vs. 408 per 1,000 person‐years), but this difference was not statistically significant (Table 3). When stratified by postbaseline housing pattern—where patterns with a small number of persons were combined with the stable housing pattern (treatment) and noninstitutional stay pattern (comparison)—the association between the NY/NY III and preventable ED visits was statistically significant. The treatment group had a lower rate of preventable ED visits (Relative Risk [RR] = 0.60, 95% Confidence Interval [CI] = 0.38, 0.96) than the comparison group persons with unstable housing pattern. For the treatment group, the rate of preventable ED visits decreased each year postbaseline (Figure 2). There was no decreasing trend for the comparison group as a whole or for comparison group members in the unstable housing pattern. The weighted rate of preventable hospitalizations among the treatment group was 0.69 times (95% CI = 0.42, 1.14; 100 vs. 120 per 1,000 person‐years) that of the comparison group. After stratification by housing pattern, the association between NY/NY III and preventable hospitalizations remained nonsignificant. Lastly, NY/NY III program participation was not associated with the rate of outpatient visits due to ACS conditions and this null finding remained after stratification by housing pattern (data not shown).
Table 3.
Rates of Preventable ED Visits and Hospitalizations per 1,000 Person‐years and Relative Risks for Rates of Preventable ED Visits and Hospitalizations after the First NY/NY III Eligibility Dates by Treatment versus Comparison Groups and Housing Stability Patterns among Heads of Homeless Families Who Were Eligible for the NY/NY III Program
| Preventable ED Visits | Preventable Hospitalizations | |||
|---|---|---|---|---|
| Rate per 1,000 Person‐years (95% CI) | Relative Risk (95% CI)a | Rate per 1,000 Person‐years (95% CI) | Relative Risk (95% CI)a | |
| Treatment group | 309 (260, 358) | 0.78 (0.58, 1.05) | 100 (73, 128) | 0.69 (0.42, 1.14) |
| Comparison group | 408 (368, 447) | Referent | 120 (99, 141) | Referent |
| Treatment groupb | 309 (260, 358) | 0.60 (0.38, 0.96) | 100 (73, 128) | 0.52 (0.26, 1.03) |
| Comparison groupb | ||||
| Noninstitutional stay | 290 (222, 359) | 0.71 (0.42,1.20) | 101 (60, 141) | 1.06 (0.41, 2.74) |
| Unstable housing | 456 (375, 537) | Referent | 85 (50, 120) | Referent |
| Transition to noninstitutional stay | 437 (379, 494) | 0.70 (0.45, 1.10) | 147 (114, 181) | 0.60 (0.31, 1.16) |
Adjusted for confouding via stabilized inverse probability of treatment weights and covariates.
Because of a small number of treatment persons with patterns other than stable housing (n = 41 out of 300), we collapsed these patterns into the stable housing pattern. Similary, because of a small number of comparision persons with the stable housing pattern (n = 9 of 626), we collapsed this pattern with the pattern of noninstitutional stay.
CI, confidence interval; ED, emergency department.
Figure 2.
- Note. Treatment (solid line); comparision (dahsed line); comparision with unstable housing pattern (dotted line). ED, emergency department.
As a sensitivity analysis, we tested the extent to which the relative risk of preventable ED visits was biased due to unobserved confounding (Appendix SA4). If an unobserved risk factor was more prevalent among the comparison versus treatment groups (e.g., substance use behaviors), an unbiased estimate of the upper bound of the 95% CI would have been greater than one, and the relative risk by the NY/NY III placement would no longer have been statistically significant. On the other hand, if an unobserved risk factor among the comparison persons were less prevalent (e.g., assistance for independent living), the 95% CI upper bound would have contained below one, and the relative risk remained statistically significant.
Comment
In this evaluation, we found that the NY/NY III supportive housing program was associated with improved housing stability among heads of homeless families who had underlying mental and physical health conditions or substance use disorders. Nearly 90 percent of the program participants experienced housing stability in the 2 years after baseline, whereas only 1 percent of applicants not placed in the program achieved housing stability through government‐subsidized housing. Seventy‐five percent of nonplaced applicants were in (or transitioned to) noninstitutional settings, while nearly one in four (24 percent) were consistently in unstable housing—primarily homeless shelter. Placement in the housing program was also associated with reduced preventable ED visits when stratified by housing pattern. Compared with homeless families who spent most of the postbaseline period in unstable government housing such as homeless family shelters, those placed in the NY/NY III program were less likely to make ED visits for chronic conditions that may have been avoided with proper primary care.
Unlike previous studies which found mixed results for housing stability for families in supportive housing, the current evaluation found a positive and statistically significant association (Bassuk et al. 2014). This might be because we used a housing stability measure that incorporated order and duration of multiple housing events from administrative data, which might be more valid than self‐reported measures or single housing events as reported in previousstudies (McAllister, Lennon, and Kuang 2011; Lim, Singh, and Gwynn 2017).
The current evaluation advances existing knowledge by establishing a relationship between supportive housing and reduced preventable medical care among heads of homeless families. Although there was little difference in the prevalence of chronic conditions between the treatment and comparison groups at baseline, when 2‐year housing stability was factored in, the NY/NY III participants had a lower rate of preventable ED visits postbaseline than persons who remained unstably housed. Similar to a previous study that reported positive associations between HIV treatment adherence and housing stability, our finding demonstrates that addressing housing, a fundamental social determinant of health, can lead to significant reductions in avoidable health care utilization as well as to improved health outcomes (Lim et al. 2015). On the other hand, a positive impact of NY/NY III on preventable hospitalizations was not observed. Total counts of preventable hospitalization events were small, suggesting that more follow‐up time might be needed to detect a trend influenced by supportive housing. In this population, the decrease in preventable ED visits was not associated with an increase in outpatient visits for ACS conditions. It might be that stable housing allows formerly homeless families to meet their urgent need and thus manage their chronic conditions via medication and social services (White and Newman 2015). Further studies using detailed service and program data are warranted to test an exact mechanism for the pathway between supportive housing and reduced preventable care.
This evaluation finding also provides evidence for a noneconomic benefit of supportive housing programs. In a recent policy debate, researchers have argued that a conventional approach to assess short‐term cost‐effectiveness of the housing program might not be adequate because it fails to capture the complex life challenges and trauma experienced by homeless persons. For example, traditional analyses focused on cost‐effectiveness do not capture urgent needs such as the burden of chronic diseases or unmet needs for childcare (Kertesz et al. 2016). Homeless families derive great value from stable housing assistance, resulting in reduced anxiety and the ability to use resources to restart stable and independent lives (Fisher et al. 2014). As we found in this analysis, supportive housing provides societal benefits beyond costs savings by reducing the use of the ED for preventable conditions. In addition to the health care benefits, we believe that consistent and stable supportive housing creates a setting where families can maintain routines and activities, develop family identities, and achieve physical and mental well‐being.
This evaluation had several limitations. First, we were not able to track housing events outside of NYC institutions (e.g., street homelessness, residence outside of NYC). It results in possible misclassification of housing stability and biasing our findings toward the null (i.e., weaker associations between the housing program and outcomes) because we coded these events as noninstitutional stays and the comparison group had more persons with noninstitutional stays. Among noninstitutional events, we also could not separate stable housing experiences from unstable ones due to lack of relevant administrative data. Second, supportive housing placement was not randomly assigned. Although a large number of covariates were used to account for this selection process, we could not rule out residual confounding due to unobserved covariates that could bias findings. As seen in the sensitivity analysis, depending on the prevalence ratio of unobserved risk factors among the treatment versus comparison persons, the RR could be biased toward or away from the null. Third, the definition of housing stability did not incorporate a threshold to quantify occurrence of incarceration, homelessness, or hospitalization while experiencing continuous placement in supportive housing. Instead, we let sequence analysis identify groups of individuals whose sequences of housing events met our definition of housing stability. Despite these limitations, use of sequence analysis allowed us to incorporate order and duration of multiple housing events from administrative data, which greatly improved validity of the housing stability measure compared with traditional mean or frequency‐based measures (McAllister, Lennon, and Kuang 2011). Another strength of the evaluation was the use of the matched administrative data, which improved measurement (e.g., reduced recall bias) and provided more complete data to capture potential confounders and various service use outcomes.
In conclusion, NY/NY III had a positive impact on housing stability among heads of homeless families having serious mental illness, substance use disorders, physical disabilities, or HIV/AIDS. This improved housing stability was associated with a reduction in preventable ED visits among those placed in the program. These findings provide important evidence for the continuation and expansion of supportive housing programs for homeless families.
Supporting information
Appendix SA1: Author Matrix.
Appendix SA2: A List of Covariates Included in the Propensity Score Model.
Appendix SA3: Pre‐Baseline Housing Stablity Patterns.
Appendix SA4: Estimated Upper Bound of 95% Confidence Intervals of Rate Ratio of Preventable ED Visits by the Supportive Housing Placement Adjusted for an Unobserved Confounder (U).
Acknowledgments
Joint Acknowledgment/Disclosure Statement: We appreciate multiple agencies that provided data and/or conducted data linkage for this program evaluation, or provided input about the evaluation, including the New York City Human Resources Administration and within it Customized Assistance Services and the HIV/AIDS Services Administration, the New York State Office of Mental Health, the New York City Department of Correction, the New York City Department of Health and Mental Hygiene Office of IT Informatics, the New York City Department of Health and Mental Hygiene Bureau of Vital Statistics, the New York City Department of Health and Mental Hygiene Bureau of HIV/AIDS Prevention and Control, the New York City Department of Health and Mental Hygiene Bureau of Mental Health, and the New York City Department of Homeless Services. Some results in this paper were presented at the 2017 Society for Epidemiologic Research Annual Meeting on June 21, 2017. All authors are not engaged in any financial or other contractual agreements with potential conflicts of interest. All authors work for the New York City Department of Health and Mental Hygiene, which, in conjunction with New York State, funds and oversees the supportive housing being evaluated.
Disclosures: None.
Disclaimer: None.
References
- Abbott, A. , and Tsay A.. 2000. “Sequence Analysis and Optimal Matching Methods in Sociology.” Sociological Methods & Research 29 (1): 3–33. [Google Scholar]
- Baggett, T. P. , O'Connell J. J., Singer D. E., and Rigotti N. A.. 2010. “The Unmet Health Care Needs of Homeless Adults: A National Study.” American Journal of Public Health 100 (7): 1326–33. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bassuk, E. L. , DeCandia C. J., Tsertsvadze A., and Richard M. K.. 2014. “The Effectiveness of Housing Interventions and Housing and Service Interventions on Ending Family Homelessness: A Systematic Review.” American Journal of Orthopsychiatry 84 (5): 457–74. [DOI] [PubMed] [Google Scholar]
- Bharel, M. , Lin W.‐C., Zhang J., O'Connell E., Taube R., and Clark R. E.. 2013. “Health Care Utilization Patterns of Homeless Individuals in Boston: Preparing for Medicaid Expansion under the Affordable Care Act.” American Journal of Public Health 103 (suppl 2): S311–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Billings, J. , Zietel L., Lukomnik J., Carey T. S., Blank A. E., and Newman L.. 1993. “Impact of Socioeconomic Status on Hospital Use in New York City.” Health Affairs (Millwood) 12: 162–73. [DOI] [PubMed] [Google Scholar]
- Cole, S. R. , and Hernan M. A.. 2008. “Constructing Inverse Probability Weights for Marginal Structural Models.” American Journal of Epidemiology 168 (6): 656–64. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fisher, B. W. , Mayberry L. S., Shinn M., and Khadduri J.. 2014. “Leaving Homelessness Behind: Housing Decisions among Families Existing Shelter.” Housing Policy Debate 24 (2): 364–86. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Funk, M. J. , Westreich D., Wiesen C., Stürmer T., Brookhart M. A., and Davidian M.. 2011. “Doubly Robust Estimation of Causal Effects.” American Journal of Epidemiology 173 (7): 761–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hwang, S. W. , and Henderson M. J.. 2010. Health Care Utilization in Homeless People: Translating Research into Policy and Practice. Working Paper No 10002. Agency for Healthcare Research and Quality.
- Hwang, S. W. , Chambers C., Chiu S., Katic M., Kiss A., Redelmeier D. A., and Levinson W.. 2013. “A Comprehensive Assessment of Health Care Utilization Among Homeless Adults under a System of Universal Health Insurance.” American Journal of Public Health 103 (suppl 2): S294–301. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kertesz, S. G. , Bagett T. P., O'Connell J. J., Buck D. S., and Kushel M. B.. 2016. “Permanent Supportive Housing for Homeless People – Reframing the Debate.” New England Journal of Medicine 375: 2115–7. [DOI] [PubMed] [Google Scholar]
- Kushel, M. B. , Gupta R., Gee L., and Hass J. S.. 2006. “Housing Instability and Food Insecurity as Barriers to Health Care among Low‐Income Americans.” Journal of General Internal Medicine 21: 71–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Larimer, M. E. , Malone D. K., Garner M. D., Atkins D. C., Burlingham B., Lonczak H. S., Tanzer K., Ginzler J., Clifasefi S. L., Hobson W. G., and Marlatt G. A.. 2009. “Health Care and Public Service Use and Costs before and after Provision of Housing for Chronically Homeless Persons with Severe Alcohol Problems.” Journal of American Medical Association 301 (13): 1349–57. [DOI] [PubMed] [Google Scholar]
- Levanon Seligson, A. , Lim S., Singh T., Laganis E., Stazesky E., Donahue S., Lanzara C., Harris T. G., Marsik T., Greene C. M., Lipton F. R., Myers R., and Karpati A. M.. 2013. “New York/New York III Supportive Housing Evaluation: Interim Utilization and Cost Analysis. A Report from the New York City Department of Health and Mental Hygiene in Collaboration with the New York City Human Resources Administration and the New York State Office of Mental Health” [accessed on February 21, 2017]. Available at http://www1.nyc.gov/assets/doh/downloads/pdf/mental/housing-interim-report.pdf
- Lim, S. , Singh T. P., and Gwynn R. C.. 2017. “Impact of a Supportive Housing Program on Housing Stability and Sexually Transmitted Infections among New York City Young Adults Aging out of Foster Care.” American Journal of Epidemiology 186: 297–304. [DOI] [PubMed] [Google Scholar]
- Lim, S. , Nash D., Hollod L., Harris T. G., Lennon M. C., and Thorpe L. E.. 2015. “Influence of Jail Incarceration and Homelessness Patterns on Engagement in HIV Care and HIV Viral Suppression among New York City Adults Living With HIV/AIDS.” PLoS ONE 10 (11): e0141912. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lin, W.‐C. , Bharel M., Zhang J., O'Connell E., and Clark R. E.. 2015. “Frequent Emergency Department Visits and Hospitalizations among Homeless People with Medicaid: Implications for Medicaid Expansion.” American Journal of Public Health 105 (suppl 5): S716–22. [DOI] [PMC free article] [PubMed] [Google Scholar]
- McAllister, W. , Lennon M. C., and Kuang L.. 2011. “Rethinking Research on Forming Typologies of Homelessness.” American Journal of Public Health 101 (4): 596–601. [DOI] [PMC free article] [PubMed] [Google Scholar]
- New York City Department of Homeless Services . 2017. “DHS Average Monthly Census Counts” [accessed on May 2, 2017]. Available at https://www1.nyc.gov/assets/dhs/downloads/pdf/dailyreport.pdf
- Parker, D. 2010. “Housing as an Intervention on Hospital Use: Access among Chronically Homeless Persons With Disabilities.” Journal of Urban Health 87 (6): 912–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Robins, J. M. , Hernan M. A., and Brumback B.. 2000. “Marginal Structural Models and Causal Inference in Epidemiology.” Epidemiology 11 (5): 550–60. [DOI] [PubMed] [Google Scholar]
- Sadowski, L. S. , Kee R. A., VanderWeele T. J., and Buchanan D.. 2009. “Effect of a Housing and Case Management Program on Emergency Department Visits and Hospitalizations among Chronically Ill Homeless Adults.” Journal of American Medical Association 301 (17): 1171–8. [DOI] [PubMed] [Google Scholar]
- VanderWeele, T. J. , and Arah O. A.. 2011. “Bias Formulas for Sensitivity Analysis of Unmeasured Confounding for General Outcomes, Treatments, and Confounders.” Epidemiology 22 (1): 42–52. [DOI] [PMC free article] [PubMed] [Google Scholar]
- White, B. M. , and Newman S. D.. 2015. “Access to Primary Care Services Among the Homeless: A Synthesis of the Literature Using the Equity of Access to Medical Care Framework.” Journal of Primary Care & Community Health 6 (2): 77–87. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Appendix SA1: Author Matrix.
Appendix SA2: A List of Covariates Included in the Propensity Score Model.
Appendix SA3: Pre‐Baseline Housing Stablity Patterns.
Appendix SA4: Estimated Upper Bound of 95% Confidence Intervals of Rate Ratio of Preventable ED Visits by the Supportive Housing Placement Adjusted for an Unobserved Confounder (U).