Abstract
Objective
This study investigated whether a move to public housing affects people’s use of healthcare services.
Method
Using administrative data from Manitoba, the number of hospitalizations, general practitioner (GP), specialist and emergency department (ED) visits, and prescription drugs dispensed in the years before and after the housing move-in date (2012/2013) were measured for a public housing and matched cohort. Generalized linear models with generalized estimating equations tested for differences between the cohorts in utilization trends. The data were modeled using Poisson (rate ratio, RR), negative binomial (incident rate ratio, IRR), and binomial (odds ratio, OR) distributions.
Results
GP visits (IRR = 1.04, 95% CI 1.01–1.06) and prescriptions (IRR = 1.04, 95% CI 1.02–1.05) increased, while ED visits (RR = 0.90, 95% CI 0.82–1.00) and hospitalizations (OR = 0.95, 95% CI 0.93–0.96) decreased over time. The public housing cohort had a significantly higher rate of GP visits (IRR = 1.08, 95% CI 1.04–1.13), ED visits (RR = 1.18, 95% CI 1.01–1.37), and prescriptions (IRR = 1.09, 95% CI 1.05–1.13), and was more likely to be hospitalized (OR = 1.39, 95% CI 1.21–1.61) compared to the matched cohort. The rate of inpatient days significantly decreased for the public housing cohort, but did not change for the matched cohort.
Conclusion
Healthcare use changed similarly over time (except inpatient days) for the two cohorts. Public housing provides a basic need to a population who has a high burden of disease and who may not be able to obtain and maintain housing in the private market.
Electronic supplementary material
The online version of this article (10.17269/s41997-018-0162-2) contains supplementary material, which is available to authorized users.
Keywords: Public housing, Healthcare use, Health services, Health status, Record linkage, Administrative data
Résumé
Objectif
Déterminer si l’installation dans un logement social a un effet sur l’utilisation des services de soins.
Méthode
À l’aide des données administratives du Manitoba, nous avons mesuré le nombre d’hospitalisations, de consultations d’omnipraticiens ou de spécialistes, de visites à l’urgence et de médicaments sur ordonnance délivrés dans les années qui précèdent et qui suivent la date d’installation dans un logement (2012–2013) pour une cohorte vivant dans des logements sociaux et une cohorte témoin. Les différences dans les tendances d’utilisation de chaque cohorte ont été testées par des modèles linéaires généralisés avec équations d’estimation généralisées. Les données ont été modélisées à l’aide d’analyses de régression de Poisson (rapport de taux, RT), de régression binomiale négative (rapport de taux d’incidence, RTI) et de régression binomiale (rapport de cotes, RC).
Résultats
Dans l’ensemble, les consultations d’omnipraticiens (RTI = 1,04, IC de 95% 1,01–1,06) et les médicaments sur ordonnance (RTI = 1,04, IC de 95% 1,02–1,05) ont augmenté, tandis que les visites à l’urgence (RT = 0,90, IC de 95% 0,82–1,00) et les hospitalisations (RC = 0,95, IC de 95% 0,93–0,96) ont diminué avec le temps. La cohorte des logements sociaux a présenté des taux sensiblement plus élevés de consultations d’omnipraticiens (RTI = 1,08, IC de 95% 1,04–1,13), de visites à l’urgence (RT = 1,18, IC de 95% 1,01–1,37) et de médicaments sur ordonnance (RTI = 1,09, IC de 95% 1,05–1,13) et une plus grande probabilité d’hospitalisation (RC = 1,39, IC de 95% 1,21–1,61) que la cohorte témoin. Le nombre de jours de traitement en établissement a sensiblement diminué dans la cohorte des logements sociaux, mais n’a pas changé dans la cohorte témoin.
Conclusion
L’utilisation des soins de santé a évolué de façon similaire dans les deux cohortes au fil du temps (sauf pour les jours de traitement en établissement). Le logement social comble un besoin fondamental pour une population qui présente une charge de morbidité élevée et qui n’a pas toujours les moyens de trouver et de garder un logement sur le marché privé.
Mots-clés: Logement social, Utilisation des soins de santé, Services de santé, État de santé, Couplage des dossiers médicaux, Données administratives
Introduction
Public housing is a form of long-term subsidized housing owned and/or managed by government. To be eligible, prospective tenants must meet certain requirements, including that their household income cannot exceed the program limit. Rent is typically calculated as a percentage of before-tax household income (often < 30%) (Government Assisted Housing n.d.).
A move to public housing may represent an improved housing situation if it is more affordable (i.e., cost of rent is < 30% of the total before-tax income), adequate (i.e., not in need of major repairs), suitable (i.e., enough bedrooms according to the National Housing Standards) and accessible (e.g., wheelchair access, close to transit and services) than where people lived before (i.e., not in core housing need). It seems reasonable to expect that if a person’s housing situation changed (for the better), other aspects of their lives may change as well, such as their health and healthcare use (see Fertig and Reingold 2007). However, only a few researchers have investigated whether public housing functions like an intervention, changing people’s trajectories of different outcomes such as health, healthcare use, education, employment, justice involvement, etc., since public housing is not designed/implemented to target specific outcomes (Dockery et al. 2008).
Other researchers have examined changes in healthcare use among special populations who moved into public housing (Smith et al. 1997; Wood et al. 2016) and changes in use when people moved out of public housing (Cooper et al. 2012; Cummings et al. 2016). To our knowledge, no one has investigated changes in healthcare use among a general population of residents new to public housing compared to a matched cohort. Our purpose was to determine whether public housing functions as an intervention in the use of healthcare services. The objectives were to test for changes in the use of different types of healthcare services over time (i.e., before and after people moved into public housing) and test for differences in use between public housing residents and individuals from the general population who were similar in terms of their socio-economic characteristics but who did not live in public housing. We hypothesized that public housing residents would be high users of healthcare services, because previous research has shown that public housing residents are more likely to be in poorer health (Bazargan et al. 2005a; Digenis-Bury et al. 2008; Hinds et al. 2016; Simning et al. 2011; Wiggers et al. 2001). We hypothesized that use of acute care services (i.e., hospitalizations, emergency department (ED) visits) and general practitioner (GP) visits would decrease when people moved into public housing (Hinds et al. 2018), but would remain above the general population level. We suspected that a move to public housing would have a stabilizing effect, and thus health would improve (at least temporarily) and consequently, healthcare use would decrease. Since pharmaceutical use tends to increase over time, we expected that it would continue to increase when people moved into public housing. We focused on the year before and year after people moved into public housing to assess the immediate (i.e., short-term) impact of public housing on healthcare use (Srebnik et al. 2013).
Methods
Study cohorts
Manitoba is an ethnically diverse Canadian prairie province with a population of approximately 1.3 million, with large newcomer and Indigenous populations. The public housing cohort included all adults (18+ years) who moved into Manitoba Housing between January 1, 2012, and December 31, 2013 who were listed as the primary applicant, resided at least 1 year (i.e., the move-out date was at least 365 days after the move-in date), and were registered with the Manitoba Health Services Insurance Plan (MHSIP). Tenancy duration was determined from the move-in and move-out dates. Applicants residing in public housing within 2 years of their 2012/2013 move-in date and residents of Churchill, a remote northern community, were excluded. It was not possible to distinguish those residing in subsidized units from those in the private market units in Churchill (Finlayson et al. 2013).
To create the matched cohort, we began with the population of Manitoba who had not applied to or were not residents of public housing in the years 2010 to 2014. A “move-in” date between January 1, 2012, and December 31, 2013 was then randomly assigned; each day in this 2-year period had an equal chance of being assigned the “move-in” date. Individuals not covered under the MHSIP in the year before and after the “move-in” date, individuals less than 18 years old, and Churchill residents were excluded. Individuals who moved in the year after the “move-in” date were also excluded to match the requirement that the public housing cohort were residents of public housing for at least 1 year. The general population was then matched one-to-one based on age at the move-in date, sex, receipt of income assistance in the year prior to the move-in date, and region of residence (defined as the regional health authority) at or before the move-in date to the public housing cohort.
Data sources
We used health and social administrative databases in the Population Research Data Repository at the Manitoba Centre for Health Policy. These databases can be linked at the individual level via a unique de-identified health number.
The Tenant Management System contains information on applicants and residents of public housing owned and managed by the provincial government (approximately 2300 buildings and 13,000 units) (Finlayson et al. 2013). The Population Registry contains demographic and health coverage information (e.g., start and end of coverage dates). The Social Assistance Management Information Network database contains information on households receiving financial support from the provincial Employment and Income Assistance program.
The Hospital Discharge Abstracts database contains information on discharges from all acute and chronic care facilities. Up to 25 diagnostic codes are recorded using the International Classification of Diseases (ICD-10-CA) coding system, 10th revision, Canadian version. The Physician Billing Claims database contains information about the majority of ambulatory physician visits; specifically, all visits to fee-for-service physicians and some visits to salaried physicians (i.e., shadow billed) are captured. Most physicians in the province are reimbursed on a fee-for-services basis (estimates range from 84% to 93%) (Lix et al. 2016; Lix et al. 2015; Watson et al. 2004). Only the diagnosis the physician deems most responsible for the visit is recorded, using a three-digit International Classification of Diseases, 9th revision, Clinical Modification (ICD-9-CM) code. The Emergency Department Information System contains information on visits to all EDs in Winnipeg, the capital of Manitoba. No data are available on visits to EDs outside of Winnipeg. The Drug Program Information Network (DPIN) database contains information about prescription drugs dispensed from community pharmacies; prescriptions dispensed from hospital pharmacies, nursing stations, and the provincial cancer agency are not included in the DPIN database.
Study variables
Use of public housing was the exposure variable. The matched cohort did not use public housing (i.e., unexposed), while the public housing cohort used public housing (i.e., exposed).
Healthcare use was measured in the 365 days before and 365 days after the move-in date. The number of GP visits and specialist visits were measured using the date of service and physician type. The number of inpatient hospitalizations and length of hospital stay were determined using admission and discharge dates. The number of different prescription drugs at the fourth level of the Anatomical Therapeutic Chemical (ATC) Classification System, which denotes the chemical/therapeutic/pharmacological subgroup of a drug, was measured based on the dispensation date. The number of ED visits was calculated using the admission and discharge dates. Hospitalizations, prescription drugs use (based on days supply), and ED visits that spanned both periods were counted in each period.
Our choice of covariates was guided by the Behavioral Model for Vulnerable Populations (Bazargan et al. 2005b; Gelberg et al. 2000), an update of Andersen and Newman’s Behavioral Model of Health Service Use (Andersen 1995), which proposes that predisposing, enabling, and need factors predict and explain healthcare use. This model was used by other researchers to study the determinants of healthcare use (i.e., alternative healthcare, vision care) for public housing residents (Baker et al. 2005; Bazargan et al. 2005b). In Andersen’s (1995) model, predisposing factors include demographic characteristics, social factors, and health beliefs that incline certain people to seek healthcare more readily than others. The updated model includes a “Predisposing Vulnerable” domain, which includes social structure, childhood characteristics, living conditions, and residential mobility, to name a few. We could only include some of the variables from these models; we were limited by what data were captured in the databases. The predisposing factors in our study included demographic variables, namely sex and age group (18–24, 25–39, 40–64, 65+ years) at the move-in date, and residential mobility. Residential mobility (i.e., moved/did not move) was determined from changes in the six-digit postal code in the 365 days prior to the move-in date. A maximum of two moves can be detected within a year timeframe since the Population Registry is updated twice a year (June and December).
Enabling factors are financial and organizational factors that facilitate/impede healthcare use (Andersen 1995). In our study, the enabling factors included geographic (i.e., region) and economic (i.e., receipt of income assistance and income quintile) characteristics. Region of residence (i.e., urban/Winnipeg or rural/non-Winnipeg) prior to the move-in date was determined from the six-digit postal code. Individuals were classified as recipients of income assistance (IA) if they or a member of their household received IA in the 365 days prior to the move-in date (Heaman et al. 2012). Income quintile (IQ), an area-level measure based on the average household income for a dissemination area (DA), was obtained from the Canadian Census. The DAs are sorted from poorest to wealthiest and grouped into quintiles (approximately 20% of the population).
Need factors are reasons why someone may seek healthcare, such as having a health condition (Andersen 1995). The need factors in our study included measures of health status assessed in the 365 days prior to the move-in date. Selected conditions were determined by the presence of specific ICD codes (Appendix). The mental health conditions (de Boer et al. 1997; Kushel et al. 2002) included schizophrenia, affective (mood and anxiety) disorders, and substance abuse disorders (Kushel et al. 2002). The physical health conditions included respiratory illness (i.e., asthma chronic obstructive pulmonary disease, bronchitis, emphysema), diabetes, hypertension, cancer, arthritis, and injuries (de Boer et al. 1997; Duchon et al. 1999; Kushel et al. 2002; O’Toole et al. 1999). Health status was also assessed using Aggregated Diagnostic Groups (ADGs) (Austin et al. 2011; Roos et al. 2014). ADGs are groups of ICD-9-CM/ICD-10-CA codes that represent diagnoses that are clinically similar and for which the expected or actual use of healthcare services is similar. The Johns Hopkins Adjusted Clinical Groups® (ACG®) Case-Mix System (version 9) clusters the ICD codes into 32 mutually exclusive ADGs.
Statistical analysis
The cohorts were characterized using descriptive statistics. A chi-square test was used to assess differences between the cohorts in the frequency distributions. We used generalized estimating equations to determine whether there was a cohort (i.e., public housing versus general population) by time period (i.e., before versus after the move-in date) interaction effect for each healthcare measure. A statistically significant cohort by time period interaction indicates the cohorts differ in how healthcare use changed over time. For example, if public housing has an effect on healthcare use, use of healthcare services should change (more dramatically) for the public housing cohort and less so (or not at all) for the matched cohort. The main effects model was interpreted if the interaction was not statistically significant. We adopted an unstructured correlation structure, the least restrictive structure, to account for the within-subject correlation over the two periods. The economic, residential mobility, and health status variables were confounding covariates in the adjusted models. The unadjusted models did not include these confounding covariates.
Hospitalizations were modeled as a dichotomous variable (odds ratios, ORs). GP visits, specialist visits, ED visits, inpatient days, and prescription drug use were modeled as count variables. ED visits were modeled using the Poisson distribution (rate ratios, RRs). GP visits, specialist visits, inpatient days, and prescription drug use were modeled using a negative binomial distribution (incident rate ratios, IRRs). The corresponding 95% confidence intervals were calculated. The distribution (e.g., negative binomial, Poisson) for the outcome measure was selected based on the quasi-likelihood information criterion (QIC) statistic (SAS Institute Inc 2013). All models included the full cohorts, except for inpatient days, which only included individuals who were hospitalized and their matches. All analyses were conducted using SAS version 9.4 (SAS Institute Inc., Cary, NC, USA).
Results
A total of 2619 individuals (52.4%) were retained in the public housing cohort (Fig. 1). Almost all individuals (2612; 99.7%) were matched to individuals from the general population.
Socio-economic and health characteristics
The frequency distributions for sex, age, region of residence, and receipt of IA (see Table 1) were identical for the public housing and matched cohorts, as expected due to the matching. Almost three quarters of the cohorts were female, the majority (59.8%) resided outside of Winnipeg, and almost two thirds received IA. The public housing cohort was more likely to reside in lower income areas and be residentially mobile. Affective disorders were the most common health condition in the cohorts (public housing cohort, 30.1%; matched cohort, 24.8%). The public housing cohort was significantly more likely to have each of the health conditions (p < 0.01), except cancer and inflammatory bowel disease (p > 0.01). The cohorts differed significantly on the distribution of the ADG categories; the public housing cohort was more likely to have co-occurring health conditions.
Table 1.
Variables | Categories | Public housing cohort (n = 2612) | Matched cohort (n = 2612) |
---|---|---|---|
Sex | Males | 27.1 | 27.1 |
Females | 72.9 | 72.9 | |
Age (years) at move-in date | 18–24 | 15.9 | 15.9 |
25–39 | 32.2 | 32.2 | |
40–64 | 37.4 | 37.4 | |
65+ | 14.4 | 14.4 | |
Region | Winnipeg | 40.2 | 40.2 |
Non-Winnipeg | 59.8 | 59.8 | |
Income assistance | Yes | 65.1 | 65.1 |
Income quintile† | Q1 (poorest) | 43.0 | 32.0 |
Q2 | 22.0 | 21.8 | |
Q3 | 15.6 | 19.6 | |
Q4 | 11.6 | 13.5 | |
Q5 (most affluent) | 6.4 | 11.1 | |
NF* | 1.4 | 2.0 | |
Change in postal code† | Yes | 25.8 | 15.5 |
Physical health conditions | Arthritis‡ | 24.2 | 20.7 |
Injury† | 21.6 | 16.9 | |
Respiratory disease† | 18.5 | 14.4 | |
Hypertension‡ | 18.3 | 15.2 | |
Diabetes† | 14.6 | 9.8 | |
Ischemic heart disease‡ | 1.9 | 1.0 | |
Cancer | 3.0 | 2.9 | |
Inflammatory bowel disease | 0.5 | 0.7 | |
Mental health conditions | Affective disorders† | 30.1 | 24.8 |
Substance abuse disorders‡ | 6.1 | 4.1 | |
Schizophrenia | 2.1 | 2.6 | |
ADGs† | Low | 17.4 | 24.5 |
Medium | 25.3 | 28.9 | |
High | 57.4 | 46.6 |
ADGs, Aggregated Diagnostic Groups; *NF, not found; ‡p < 0.01 †p < 0.0001
Healthcare utilization
A summary of healthcare use in the 365 days before and after the move-in date for both cohorts is presented in Table 2. In general, the public housing cohort was more likely to use healthcare services compared to the matched cohort and use tended to be higher in the year before the move-in date compared to the year after the move-in date (except for ED visits and the number of prescription drugs dispensed).
Table 2.
Public housing cohort (n = 2612) | Matched cohort (n = 2612) | |||
---|---|---|---|---|
Healthcare utilization | Year before | Year in housing | Year before | Year in housing |
Hospitalization (%) | ||||
Yes | 16.4 | 13.3 | 10.2 | 8.9 |
No | 83.7 | 86.7 | 89.9 | 91.1 |
Median # of inpatient days* (IQR) | 6 (20) | 6 (14) | 4 (9) | 4 (9) |
Mean # of physician visits (SD) | 8.7 (7.8) | 8.2 (7.5) | 7.1 (7.4) | 6.8 (7.1) |
Mean # of GP visits (SD) | 7.0 (6.8) | 6.6 (6.4) | 5.6 (6.2) | 5.3 (5.9) |
Mean # of specialist visits (SD) | 1.8 (3.4) | 1.6 (3.1) | 1.5 (3.3) | 1.4 (3.2) |
Mean # of prescriptions (SD) | 7.1 (5.8) | 7.2 (5.7) | 5.7 (5.2) | 5.6 (5.2) |
Number of ED visits (%) | ||||
0 | 75.3 | 66.0 | 83.4 | 75.2 |
1 | 12.1 | 13.3 | 8.7 | 11.8 |
2+ | 12.6 | 20.8 | 8.0 | 13.0 |
GP, general practitioner; ED, emergency department; IQR, interquartile range; SD, standard deviation. *Calculated for those who were hospitalized
The results from the models are presented in Table 3 and are described below. Additionally, we describe the change in the estimates over time for both cohorts.
Table 3.
Model | Effect | Category | Estimate | Significance 95% CI | QIC |
---|---|---|---|---|---|
Hospitalization | |||||
Adjusted model | Time period | Year in housing | 0.79 | 0.70, 0.89 | 6711.98 |
Year before | Ref | – | |||
Cohort | Public housing | 1.39 | 1.21, 1.61 | ||
Matched | Ref | – | |||
Unadjusted model | Time period | Year in housing | 0.82 | 0.74, 0.90 | 7661.38 |
Year before | Ref | – | |||
Cohort | Public housing | 1.66 | 1.45, 1.89 | ||
Matched | Ref | – | |||
GP visits | |||||
Adjusted model | Time period | Year in housing | 1.04 | 1.01, 1.06 | − 120,990.85 |
Year before | Ref | – | |||
Cohort | Public housing | 1.08 | 1.04, 1.13 | ||
Matched | Ref | – | |||
Unadjusted model | Time period | Year in housing | 0.95 | 0.93, 0.97 | − 129,531.57 |
Year before | Ref | – | |||
Cohort | Public housing | 1.25 | 1.18, 1.31 | ||
Matched | Ref | – | |||
Specialist visits | |||||
Adjusted model | Time period | Year in housing | 1.00 | 0.94, 1.06 | − 3265.49 |
Year before | Ref | – | |||
Cohort | Public housing | 0.98 | 0.88, 1.09 | ||
Matched | Ref | – | |||
Unadjusted model | Time period | Year in housing | 0.91 | 0.87, 0.96 | − 3078.40 |
Year before | Ref | – | |||
Cohort | Public housing | 1.11 | 1.01, 1.23 | ||
Matched | Ref | – | |||
Prescriptions | |||||
Adjusted model | Time period | Year in housing | 1.04 | 1.02, 1.05 | − 132,628.77 |
Year before | Ref | – | |||
Cohort | Public housing | 1.09 | 1.05, 1.13 | ||
Matched | Ref | – | |||
Unadjusted model | Time period | Year in housing | 1.00 | 0.99, 1.02 | − 155,558.88 |
Year before | Ref | – | |||
Cohort | Public housing | 1.27 | 1.21, 1.33 | ||
Matched | Ref | – | |||
ED visits | |||||
Adjusted model | Time period | Year in housing | 0.90 | 0.82, 1.00 | 3998.54 |
Year before | Ref | – | |||
Cohort | Public housing | 1.18 | 1.01, 1.37 | ||
Matched | Ref | – | |||
Unadjusted model | Time period | Year in housing | 0.90 | 0.82, 1.00 | 3180.39 |
Year before | Ref | – | |||
Cohort | Public housing | 1.59 | 1.36, 1.86 | ||
Matched | Ref | – |
Bold values denote significant results. Adjusted model covariates include residential mobility, income quintile, injury, diabetes, arthritis, hypertension, respiratory disease, cancer, affective disorders, schizophrenia, substance abuse disorders, and ADGs
Hospitalizations: For hospitalizations, the cohort by time period interaction was not statistically significant (p = 0.33), but the main effects were significant. The public housing cohort had a higher odds of being hospitalized (OR = 1.39; 95% CI 1.21, 1.61) than the matched cohort (main effect of cohort). The odds of hospitalization were 21% lower in the year after the move-in date than in the year before the move-in date (OR = 0.79; 95% CI 0.70, 0.89) (main effect of time period). The odds in the post-move-in period relative to the pre-move-in period was 0.75 (95% CI 0.65, 0.88) for the public housing cohort and 0.85 (95% CI 0.70, 1.02) for the matched cohort; thus, the change in the odds between the two time periods was statistically significant for the public housing cohort, but not for the matched cohort.
Inpatient hospital days: There were 629 individuals in the public housing cohort and 433 in the matched cohort who had at least one hospitalization during the 2 years. In the year before the move-in date, the average length of stay was 15.8 days (SD = 38.6) and 7.6 days (SD = 23.1) for the public housing and matched cohorts, respectively. In the year after the move-in date, the average length of stay was 7.7 days (SD = 20.3) for the public housing cohort and 7.9 days (SD = 23.2) for the matched cohort. The cohort by time period interaction was statistically significant (p < 0.01). For the public housing cohort, the rate of inpatient days was significantly lower in the post-move-in period compared to the pre-move-in period (IRR = 0.60; 95% CI 0.46, 0.79). There was no difference in the rate of inpatient days between the two periods for the matched cohort (IRR = 1.25; 95% CI 0.87, 1.82). The public housing cohort had a significantly higher rate of inpatient days in the pre-move-in period (IRR = 1.64; 95% 1.12, 2.41) compared to the matched cohort. In the post-move-in period, there was no difference between the cohorts (IRR = 0.79; 95% 0.55, 1.12).
GP visits: The cohort by period interaction for GP visits was not statistically significant (p = 0.78). Compared to the matched cohort, the public housing cohort had an incident rate 1.08 times greater (95% CI 1.04, 1.13) (main effect of cohort). The incident rate during the year in public housing was significantly higher than the year before the move-in date (IRR = 1.04; 95% CI 1.01, 1.06) (main effect of time period). The incident rate in the year after the move-in date relative to the year before the move-in date was not statistically significant (IRR = 1.03, 95% CI 1.00, 1.07) for the public housing cohort, but was significant (IRR = 1.04; 95% CI 1.01, 1.08) for the matched cohort.
Specialist visits: The cohort by period interaction for specialist visits was not statistically significant (p = 0.26). The incident rates did not differ between the cohorts (IRR = 0.98; 95% CI 0.88, 1.09) (main effect of cohort). There was no time main effect (IRR = 1.00; 95% CI 0.94, 1.06). The incident rate in the year after the move-in date relative to the year before was 0.96 (95% CI 0.88, 1.05) for the public housing cohort and 1.03 (95% CI 0.95, 1.12) for the matched cohort.
Prescription drug use: The period by cohort interaction was not statistically significant for prescription drug use (p = 0.10). The public housing cohort filled prescriptions at a rate significantly higher (IRR = 1.09; 95% CI 1.05, 1.13) (main effect of cohort) and the rate at which prescriptions were filled increased over time (IRR = 1.04; 95% CI 1.02, 1.05) (main effect of time period). The incident rate in the year after the move-in date relative to the year before the move-in date was statistically significant (IRR = 1.05; 95% CI 1.03, 1.07) for the public housing cohort but not for the matched cohort (IRR = 1.02; 95% CI 1.00, 1.04).
ED visits: The cohort by time period interaction for ED visits was not statistically significant (p = 0.53), but both main effects were significant. The public housing cohort had an ED visit rate 1.18 times that of the matched cohort (95% CI 1.01, 1.37) (main effect of cohort). The ED visit rate in the post-move-in period was significantly lower than the rate in the pre-move-in period (RR = 0.90; 95% CI 0.82, 1.00) (main effect of time). The ED visit rate in the year after the move-in date relative to the year before the move-in date was 0.92 (95% CI 0.81, 1.05) for the public housing cohort and 0.87 (95% CI 0.74, 1.01) for the matched cohort; thus, neither cohort had a significant change in the ED visit rate over time.
Discussion
We identified a cohort of new public housing residents and successfully matched them to a cohort similar in terms of their socio-economic characteristics but who did not apply to or move into public housing. The socio-economic and health characteristics of the public housing cohort are consistent with cohorts from other studies (Aratani 2010; Finlayson et al. 2013; Hinds et al. 2016).
Although the two cohorts were similar in terms of their socio-economic characteristics (due to matching), the public housing cohort was more likely to have health conditions. Occasionally, public housing is blamed for contributing to poor health; however, this study and others have shown that individuals are often in poor health prior to applying (Carder et al. 2016; Hinds et al. 2016), which suggests that public housing provides affordable housing to a population which may have difficulty obtaining and maintaining housing in the private market. Since many individuals move into public housing with physical and mental health challenges, it is important that supports are available to ensure they have successful tenancies (also suggested by the Housing First literature (Tsemberis and Eisenberg 2000; Tsemberis et al. 2004)). Successful tenancies mean that people are stably housed (i.e., moving is minimized, particularly when moving is unwanted, such as due to eviction) (Distasio and McCullough 2014). One advantage of residing in public housing (over other forms of housing) is that public housing clusters people together, which should make it easier to support people and deliver services. As well, public housing tends to be located in areas where services are geographically concentrated.
Research has shown that applicants to and residents of public housing use healthcare services frequently (Black et al. 1997; Black et al. 1998; Carder et al. 2016; Hinds et al. 2016; Smith et al. 2013), though this is not always a consistent finding (Digenis-Bury et al. 2008). In the years before and after the move-in date, a higher percentage of the public housing cohort used healthcare services and had higher average use compared to the matched cohort. Given that many public housing residents had health challenges, it is not surprising that the public housing cohort was significantly more likely to use all forms of healthcare (except specialist visits).
Almost three quarters (73%) of the public housing cohort was female. We addressed the potential confounding effect of sex on healthcare use by matching on sex. However, this sex imbalance is noteworthy as it may also partly explain the overall high healthcare use we observed, as females tend to seek healthcare more readily than males (Iron and Goel 1998; Mackenzie et al. 2006; Sibley et al. 2010).
There was a significant time period by cohort interaction for inpatient days, indicating the rate of inpatient days decreased more for the public housing cohort over time compared to the matched cohort. Hospital staff may have been concerned that people lacked stable housing and community supports in the year before they moved into public housing. This finding may reflect that time spent in hospital is a mechanism to provide supports for patients beyond just access to healthcare. In fact, discharge planning should take housing into account in any release plan. This is a noteworthy finding as it may represent a substantial reduction in healthcare costs (Latimer et al. 2017). Since the time period by cohort interaction was not significant for the other healthcare measures, we found little evidence that public housing affects use of healthcare services. This finding should not be misinterpreted as public housing serving no function; it is the opposite in fact. Our findings of no interaction effect are reasonable given that public housing residents tend to have a high burden of co-occurring chronic physical and mental health conditions which may require ongoing care and monitoring, and thus healthcare use may not change much over a 2-year timespan. In fact, it is even reasonable to expect that healthcare use may increase when people move into public housing. Often, clinics are strategically located in or nearby to public housing (Apparicio et al. 2006; Cooper et al. 2012; Culhane-Pera et al. 2007; Fertig and Reingold 2007), so residents may experience improved healthcare access when they move in. It is also possible that healthcare use may increase because people are now in a stable housing situation and are able to attend to their health.
Even with receipt of housing, public housing residents may face challenges to meet their needs (e.g., safety, food, recreation) because they still have low incomes, which negatively influences health and healthcare use (Battaglia et al. 2012) and housing stability. Therefore, the other social determinants of health—income, early child development, education, (un)employment, food security, and social exclusion—must be addressed by making investments in these areas as well. For example, while providing individuals with a minimum level of income would be ideal, ensuring that public housing residents receive income assistance and the other benefits they are eligible for is a feasible goal.
Study strengths
We linked longitudinal, population-based, health and social administrative databases at an individual level. We identified individuals new to public housing and matched them on socio-economic characteristics to a cohort from the general population who did not apply to or move into public housing. We had the advantage (over primary data collection methods) of being able to include everyone who resided in public housing regardless of where they lived in the province during a period of time in the past, and we were able to access demographic, geographic, economic, housing, health, and healthcare use information over a period of time for people who resided and for people who did not reside in public housing. Additionally, we were able to examine changes in use of different types of healthcare services.
Study limitations and future directions
Our cohort only included residents of public housing owned and directly managed by the province; no individual-level administrative data exist in Manitoba for other forms of social housing. Moving may be a disruptor, and the matched cohort may not have experienced it (we tried to limit this in the post-period by excluding people who had a residential start date during this period). A future study could compare people who moved into public housing with people who moved but not into public housing. Additionally, a comparison group who applied to public housing but did not move into public housing could be added because the application cancelation date and cancelation reasons are available.
There are limitations to using administrative data. There may be measurement error in the covariates. Only one diagnosis code is recorded for each physician visit, so the number of people with health conditions may be underestimated; however, this should have affected both cohorts similarly. Residential mobility may be underestimated if address changes were not reported to Manitoba Health; however, this is likely minimal given that healthcare use was high and administrators often request accurate address information at every healthcare contact. The matched cohort may have been more likely to underreport address changes since they had less contact with the healthcare system; however, to our knowledge, no study has compared the reliability of reporting address changes between these two populations. As well, information about other determinants of healthcare use suggested by Andersen’s (1995) model (e.g., people’s attitudes towards seeking healthcare) was not available.
Ideally, we would have matched the cohorts on individual or household income, but this information was not available; instead, income quintile, an area-level measure of income, was a covariate in our statistical models. Differences between the cohorts on income therefore were accounted for in the analysis and this is a well-accepted method to address potential confounding in observational studies. Additionally, it is an accepted practice to use neighbourhood-level income when household-level income is not available. While there may be heterogeneity within neighbourhoods with respect to household income (Diez Roux et al. 2001; Hanley and Morgan 2008; Marra et al. 2011), we used the smallest level geographic area, DA, for which census data are reported and there is evidence that area-level measures based on small geographic areas result in less bias than area-level measures based on larger geographic areas (Soobader et al. 2001).
Previous research suggests that results may vary depending on the length of the observation period. For example, the Moving to Opportunity for Fair Housing (MTO) Program, a five-city experimental study during the 1990s in the US, assessed changes in a number of outcomes for families who moved out of public housing in high poverty neighbourhoods to housing in the private market in low poverty neighbourhoods (Sanbonmatsu et al. 2011). While the findings from the short-term evaluation (approximately 2 years) were generally positive, the interim results (4 to 7 years) “yielded a more complicated pattern” (p. 17), and the educational and economic self-sufficiency outcomes that were expected to take longer did not manifest by the final evaluation (10 to 15 years) (Sanbonmatsu et al. 2011).
As another example, some of Wood et al.’s (2016) results changed when they increased the length of the pre-move-in period from 1 to 3 years. Specifically, overnight hospital stays and mental health service use increased over time (i.e., use was higher after moving into public housing) based on a 3-year pre-move-in period but decreased based on a 1-year pre-move-in period. A future study could assess how the duration of the observation period(s) affects the findings. Wood et al.’s (2016) findings suggest that healthcare use may increase before people move into public housing. For many people, public housing may represent their last housing option, which they may only consider after a significant life event occurs, such as a change in health status.
Summary
Residents of public housing tend to be higher users of healthcare services over a 2-year period proximal to their move-in date (i.e., year before and year after) compared to individuals who have similar socio-economic characteristics but are not public housing residents. Although changes in healthcare use over time were generally similar for the two cohorts, which suggests that a move to public housing has little influence on healthcare use in the short term, public housing does serve the public good by providing a safety net to individuals who have a high burden of disease and who may not be able to afford to rent in the private market.
Electronic supplementary material
Acknowledgements
Thanks are owed to Heather Prior for data extraction from the Manitoba Centre for Health Policy Population Health Research Data Repository. Thanks are also owed to Kristine Kroeker for providing statistical consulting advice.
Funding information
This work was supported by funding to AMH from Research Manitoba (PhD Dissertation Award). LML was supported by a Manitoba Research Chair from Research Manitoba.
Compliance with ethical standards
Disclaimer
The authors acknowledge the Manitoba Centre for Health Policy for use of data contained in the Population Research Data Repository under project #2015-002 (HIPC#2014/2015.29). The results and conclusions are those of the authors and no official endorsement by the Manitoba Centre for Health Policy, Manitoba Health, or other data providers is intended or should be inferred. Data used in this study are from the Population Research Data Repository housed at the Manitoba Centre for Health Policy, University of Manitoba and were derived from data provided by Manitoba Health, the Winnipeg Regional Health Authority, and the Department of Families.
Ethics approval
Ethics approval was obtained from the University of Manitoba Health Research Ethics Board. Data access was approved by the Manitoba Health Information Privacy Committee.
Conflict of interest
The authors declare that they have no conflicts of interest.
Footnotes
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- Andersen RM. Revisiting the behavioral model and access to medical care: does it matter? Journal of Health and Social Behavior. 1995;36(1):1–10. doi: 10.2307/2137284. [DOI] [PubMed] [Google Scholar]
- Apparicio P, Seguin A, Séguin AM. Measuring the accessibility of services and facilities for residents of public housing in Montréal. Urban Studies. 2006;43(1):187–211. doi: 10.1080/00420980500409334. [DOI] [Google Scholar]
- Aratani Y. Public housing revisited: racial differences, housing assistance, and socioeconomic attainment among low-income families. Social Science Research. 2010;39(6):1108–1125. doi: 10.1016/j.ssresearch.2010.04.007. [DOI] [Google Scholar]
- Austin PC, Van Walraven C, Wodchis WP, Newman A, Anderson GM. Using the Johns Hopkins Aggregated Diagnosis Groups (ADGs) to predict mortality in a general adult population cohort in Ontario, Canada. Medical Care. 2011;49(10):932–939. doi: 10.1097/MLR.0b013e318215d5e2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Baker RS, Bazargan M, Bazargan-Hejazi S, Calderón JL. Access to vision care in an urban low-income multiethnic population. Ophthalmic Epidemiology. 2005;12(1):1–12. doi: 10.1080/09286580590921330. [DOI] [PubMed] [Google Scholar]
- Battaglia, T. A., Murrell, S. S., Bhosrekar, S. G., Caron, S. E., Bowen, D. J., Smith, E., … Rorie, J.-A. (2012). Connecting Boston’s public housing developments to community health centers: who’s ready for change? Progress in Community Health Partnerships: Research, Education, and Action, 6(3), 239–248. 10.1353/cpr.2012.0035. [DOI] [PubMed]
- Bazargan M, Calderon JL, Heslin KC, Mentes C, Shaheen MA, Ahdout J, Baker RS. A profile of chronic mental and physical conditions among African-American and Latino children in urban public housing. Ethnicity & Disease. 2005;15(Supplement):S5-3-S5-9. [PubMed] [Google Scholar]
- Bazargan M, Norris K, Bazargan-Hejazi S, Akhanjee L, Calderon JL, Safvati S, Baker RS. Alternative healthcare use in the underserved population. Ethnicity & Disease. 2005;15:531–539. [PubMed] [Google Scholar]
- Black BS, Rabins PV, German P, McGuire M, Roca R. Need and unmet need for mental health care among elderly public housing residents. The Gerontologist. 1997;37(6):717–728. doi: 10.1093/geront/37.6.717. [DOI] [PubMed] [Google Scholar]
- Black BSS, Rabins PVV, German P, Roca R, McGuire M, Brant LJJ. Use of formal and informal sources of mental health care among older African-American public-housing residents. Psychological Medicine. 1998;28(3):519–530. doi: 10.1017/S0033291798006631. [DOI] [PubMed] [Google Scholar]
- Carder P, Luhr G, Kohon J. Differential health and social needs of older adults waitlisted for public housing or housing choice vouchers. Journal of Aging and Social Policy. 2016;28(4):246–260. doi: 10.1080/08959420.2016.1156507. [DOI] [PubMed] [Google Scholar]
- Cooper, H. L., Wodarski, S., Cummings, J., Hunter-Jones, J., Karnes, C., Ross, Z., … Bonney, L. E. (2012). Public housing relocations in Atlanta, Georgia, and declines in spatial access to safety net primary care. Health & Place, 18(6), 1255–1260. 10.1016/j.healthplace.2012.08.007. [DOI] [PMC free article] [PubMed]
- Culhane-Pera K, Ellmore D, Wessel LA. Primary care in public housing: voices of clinicians. Journal Of Health Care For The Poor And Underserved. 2007;18:735–743. doi: 10.1353/hpu.2007.0105. [DOI] [PubMed] [Google Scholar]
- Cummings JR, Allen L, Ko M, Bonney L, Hunter-Jones J, Cooper H. Changes in healthcare access and utilization among participants in a public housing relocation program in Atlanta, Georgia. Health and Place. 2016;42:63–68. doi: 10.1016/j.healthplace.2016.07.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- de Boer AGEM, Wijker W, de Haes HCJM. Predictors of health care utilization in the chronically ill: a review of the literature. Health Policy. 1997;42:101–115. doi: 10.1016/S0168-8510(97)00062-6. [DOI] [PubMed] [Google Scholar]
- Diez Roux, A. V., Kiefe, C. I., Jacobs, D. R., Haan, M., Jackson, S. A., Nieto, F. J., … Schulz, R. (2001). Area characteristics and individual-level socioeconomic position indicators in three population-based epidemiologic studies. Annals of Epidemiology, 11(6), 395–405. 10.1016/S1047-2797(01)00221-6. [DOI] [PubMed]
- Digenis-Bury EC, Brooks DR, Chen L, Ostrem M, Horsburgh CR. Use of a population-based survey to describe the health of Boston public housing residents. American Journal of Public Health. 2008;98(1):85–91. doi: 10.2105/AJPH.2006.094912. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Distasio, J. & McCullough, S. (2014). Holding on!: supporting successful tenancies for the hard to house. Retrieved from http://homelesshub.ca/sites/default/files/2014 Successful Tenancies - FINAL.pdf. Accessed 29 September 2018
- Dockery, A. M., Ong, R., Whelan, S., & Wood, G. (2008). The relationship between public housing wait lists, public housing tenure and labour market outcomes. Melbourne.
- Duchon LM, Weitzman B, Shinn M. The relationship of residential instability to medical care utilization among poor mothers in New York City. Medical Care. 1999;37(12):1282–1293. doi: 10.1097/00005650-199912000-00011. [DOI] [PubMed] [Google Scholar]
- Fertig A, Reingold D. Public housing, health, and health behaviors: is there a connection? Journal of Policy Analysis and Management. 2007;26(4):831–859. doi: 10.1002/pam. [DOI] [PubMed] [Google Scholar]
- Finlayson, G., Smith, M., Burchill, C., Towns, D., Peeler, W., Soodeen, R.-A. A., … Guenette, W. (2013). Social housing in Manitoba part I: Manitoba social housing data. Winnipeg, MB.
- Gelberg L, Andersen RM, Leake BD. The behavioral model for vulnerable populations: application to medical care use and outcomes for homeless people. Health Services Research. 2000;34(6):1273–1302. [PMC free article] [PubMed] [Google Scholar]
- Government Assisted Housing. (n.d.). Retrieved from http://www.cmhc-schl.gc.ca/en/co/buho/seca/seca_001.cfm. Accessed 11 January 2014
- Hanley GE, Morgan S. On the validity of area-based income measures to proxy household income. BMC Health Services Research. 2008;8:1–7. doi: 10.1186/1472-6963-8-79. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Heaman, M., Kingston, D., Helewa, M. E., Brownell, M., Derksen, S., Bogdanovic, B., … Bailly, A. (2012). Perinatal services and outcomes in Manitoba. Winnipeg, MB.
- Hinds AM, Bechtel B, Distasio J, Roos LL, Lix LM. Health and social predictors of applications to public housing: a population-based analysis. Journal of Epidemiology and Community Health. 2016;70(12):1229–1235. doi: 10.1136/jech-2015-206845. [DOI] [PubMed] [Google Scholar]
- Hinds AM, Bechtel B, Distasio J, Roos LL, Lix LM. Changes in healthcare use among individuals who move into public housing: a population-based investigation. BMC Health Services Research. 2018;18:411. doi: 10.1186/s12913-018-3109-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Iron K, Goel V. Sex differences in the factors related to hospital utilization: results from the 1990 Ontario Health Survey. Journal of Women’s Health. 1998;7(3):359–369. doi: 10.1089/jwh.1998.7.359. [DOI] [PubMed] [Google Scholar]
- Kushel MB, Perry S, Bangsberg D, Clark R, Moss AR. Emergency department use among the homeless and marginally housed: results from a community-based study. American Journal of Public Health. 2002;92(5):778–784. doi: 10.2105/AJPH.92.5.778. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Latimer, E. A., Rabouin, D., Cao, Z., Ly, A., Powell, G., Aubry, T., … Goering, P. N. (2017). Costs of services for homeless people with mental illness in 5 Canadian cities: a large prospective follow-up study. CMAJ Open, 5(3), E576–E585. 10.9778/cmajo.20170018. [DOI] [PMC free article] [PubMed]
- Lix, L. M., Yao, X., Kephart, G., Quan, H., Smith, M., Kuwornu, J. P., … Sikdar, K. (2015). A prediction model to estimate completeness of electronic physician claims databases. BMJ Open, 5(8), e006858. 10.1136/bmjopen-2014-006858. [DOI] [PMC free article] [PubMed]
- Lix LM, Kuwornu JP, Kroeker K, Kephart G, Sikdar KC, Smith M, Quan H. Estimating the completeness of physician billing claims for diabetes case ascertainment using population-based prescription drug data. Health Promotion and Chronic Disease Prevention in Canada: Research, Policy and Practice. 2016;36(3):54–60. doi: 10.24095/hpcdp.36.3.02. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mackenzie CS, Gekoski WL, Knox VJ. Age, gender, and the underutilization of mental health services: the influence of help-seeking attitudes. Aging and Mental Health. 2006;10(6):574–582. doi: 10.1080/13607860600641200. [DOI] [PubMed] [Google Scholar]
- Marra CA, Lynd LD, Harvard SS, Grubisic M. Agreement between aggregate and individual-level measures of income and education: a comparison across three patient groups. BMC Health Services Research. 2011;11:1–7. doi: 10.1186/1472-6963-11-69. [DOI] [PMC free article] [PubMed] [Google Scholar]
- O’Toole TP, Gibbon JL, Hanusa BH, Fine MJ. Utilization of health care services among subgroups of urban homeless and housed poor. Journal of Health Politics, Policy and Law. 1999;24(1):91–114. doi: 10.1215/03616878-24-1-91. [DOI] [PubMed] [Google Scholar]
- Roos LL, Walld R, Witt J. Adolescent outcomes and opportunities in a Canadian province: looking at siblings and neighbors. BMC Public Health. 2014;14(1):1–16. doi: 10.1186/1471-2458-14-506. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sanbonmatsu, L., Ludwig, J., Katz, L. F., Gennetian, L., Duncan, G., Kessler, R., … Tessler Lindau, S. (2011). Moving to opportunity for fair housing demonstration program: final impacts evaluation. Retrieved from https://www.huduser.gov/publications/pdf/mtofhd_fullreport_v2.pdf. Accessed 15 August 2018
- SAS Institute Inc . SAS/STAT 9.2 User’s Guide. Cary, NC: SAS Institute Inc; 2013. The GENMOD procedure; pp. 2871–3076. [Google Scholar]
- Sibley LM, Moineddin R, Agha MM, Glazier RH. Risk adjustment using administrative data-based and survey-derived methods for explaining physician utilization. Medical Care. 2010;48(2):175–182. doi: 10.1097/MLR.0b013e3181c16102. [DOI] [PubMed] [Google Scholar]
- Simning A, Van Wijngaarden E, Conwell Y. Anxiety, mood, and substance use disorders in United States African-American public housing residents. Social Psychiatry and Psychiatric Epidemiology. 2011;46(10):983–992. doi: 10.1007/s00127-010-0267-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Smith SJ, Alexander A, Easterlow D. Rehousing as a health intervention: miracle or mirage? Health and Place. 1997;3(4):203–216. doi: 10.1016/S1353-8292(97)00015-4. [DOI] [Google Scholar]
- Smith, M., Finlayson, G., Martens, P., Dunn, J., Prior, H., Taylor, C., … Hinds, A. (2013). Social housing in Manitoba part II: social housing and health in Manitoba: a first look. Winnipeg, MB. Retrieved from http://mchp-appserv.cpe.umanitoba.ca/reference/housing_web_version_final.pdf. Accessed 13 January 2014
- Soobader MJ, LeClere FB, Hadden W, Maury B. Using aggregate geographic data to proxy individual socioeconomic status: does size matter? American Journal of Public Health. 2001;91(4):632–636. doi: 10.2105/AJPH.91.4.632. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Srebnik D, Connor T, Sylla L. A pilot study of the impact of housing first-supported housing for intensive users of medical hospitalization and sobering services. American Journal of Public Health. 2013;103(2):316–321. doi: 10.2105/AJPH.2012.300867. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Tsemberis S, Eisenberg RF. Pathways to housing: supported housing for street-dwelling homeless individuals with psychiatric disabilities. Psychiatric Services. 2000;51(4):487–493. doi: 10.1176/appi.ps.51.4.487. [DOI] [PubMed] [Google Scholar]
- Tsemberis S, Gulcur L, Nakae M. Housing first, consumer choice and harm reduction for homeless individuals. American Journal of Public Health. 2004;94(4):651–656. doi: 10.2105/AJPH.94.4.651. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Watson DE, Katz A, Reid RJ, Bogdanovic B, Roos N, Heppner P. Family physician workloads and access to care in Winnipeg: 1991 to 2001. Canadian Medical Association Journal. 2004;171(4):339–342. doi: 10.1503/cmaj.1031047. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wiggers J, Radvan D, Clover K, Hazell T, Alexander J, Considine R. Public housing, public health: health needs of public housing tenants. Australian and New Zealand Journal of Public Health. 2001;25(2):111–114. doi: 10.1111/j.1753-6405.2001.tb01830.x. [DOI] [PubMed] [Google Scholar]
- Wood, L., Flatau, P., Zaretzky, K., Foster, S., Vallesi, S., & Miscenko, D. (2016). What are the health, social and economic benefits of providing public housing and support to formerly homeless people? Melbourne. Retrieved from http://www.ahuri.edu.au/research/final-reports/265. Accessed 28 December 2017
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.