Skip to main content
BMJ Open logoLink to BMJ Open
. 2022 Dec 14;12(12):e065688. doi: 10.1136/bmjopen-2022-065688

Navigator programme for hospitalised adults experiencing homelessness: protocol for a pragmatic randomised controlled trial

Michael Liu 1,2, Katherine Francombe Pridham 2, Jesse Jenkinson 2, Rosane Nisenbaum 2,3, Lucie Richard 2, Cheryl Pedersen 2, Rebecca Brown 2, Sareeha Virani 2, Fred Ellerington 4, Alyssa Ranieri 4, Oluwagbenga Dada 2, Matthew To 5, Gabriel Fabreau 6,7, Kerry McBrien 6,8, Vicky Stergiopoulos 9,10, Anita Palepu 11,12, Stephen Hwang 2,4,
PMCID: PMC9756200  PMID: 36517099

Abstract

Introduction

People experiencing homelessness suffer from poor outcomes after hospitalisation due to systemic barriers to care, suboptimal transitions of care, and intersecting health and social burdens. Case management programmes have been shown to improve housing stability, but their effects on broad posthospital outcomes in this population have not been rigorously evaluated. The Navigator Programme is a Critical Time Intervention case management programme that was developed to help homeless patients with their postdischarge needs and to link them with community-based health and social services. This randomised controlled trial examines the impact of the Navigator Programme on posthospital outcomes among adults experiencing homelessness.

Methods and analysis

This is a pragmatic randomised controlled trial testing the effectiveness of the Navigator Programme at an urban academic teaching hospital and an urban community teaching hospital in Toronto, Canada. Six hundred and forty adults experiencing homelessness who are admitted to the hospital will be randomised to receive support from a Homeless Outreach Counsellor for 90 days after hospital discharge or to usual care. The primary outcome is follow-up with a primary care provider (physician or nurse practitioner) within 14 days of hospital discharge. Secondary outcomes include postdischarge mortality or readmission, number of days in hospital, number of emergency department visits, self-reported care transition quality, and difficulties meeting subsistence needs. Quantitative outcomes are being collected over a 180-day period through linked patient-reported and administrative health data. A parallel mixed-methods process evaluation will be conducted to explore intervention context, implementation and mechanisms of impact.

Ethics and dissemination

Ethics approval was obtained from the Unity Health Toronto Research Ethics Board. Participants will be required to provide written informed consent. Results of the main trial and process evaluation will be reported in peer-reviewed journals and shared with hospital leadership, community partners and policy makers.

Trial registration number

NCT04961762.

Keywords: GENERAL MEDICINE (see Internal Medicine), Organisation of health services, PUBLIC HEALTH


STRENGTHS AND LIMITATIONS OF THIS STUDY.

  • The Navigator Programme is a patient-centred case management intervention informed by a prior prospective cohort study and designed in tandem with community partners, healthcare teams and people with lived experience of homelessness.

  • Linkage of patient-reported data with administrative health data allows for rigorous assessment of a much wider range of posthospital outcomes relative to previous case management studies for people experiencing homelessness.

  • This randomised controlled trial is accompanied by a parallel mixed-methods process evaluation that will investigate intervention implementation, causal mechanisms, study context, participant experiences and outcomes.

  • Blinding of participants, Homeless Outreach Counsellors and healthcare teams is not possible given the active and collaborative nature of the intervention.

  • This study takes place at an urban academic teaching hospital and an urban community teaching hospital in Toronto, Canada, and findings may not be generalisable to individuals experiencing homelessness in other contexts and settings.

Introduction

Background and rationale

More than 235 000 Canadians experience homelessness annually, of which 27% are women, 19% are youth, and a growing number are identifying as racial, ethnic, sexual and gender minorities.1 2 This population experiences disproportionate intersecting physical, mental and social burdens that greatly increase morbidity and mortality relative to the general population.3–5 For example, rates of acute and chronic physical health problems, trauma, mental illness and substance use are much higher among homeless adults.6–8

Homeless individuals often experience substantial barriers to obtaining healthcare and frequently suffer from unmet health needs.9–11 Many have other immediate competing priorities such as securing food and shelter that preclude consistent engagement with healthcare services.12 13 Homeless individuals are also much less likely to have a primary care provider (PCP) or usual source of care compared with the general population.14 15 Lack of primary care likely contributes to poor outcomes among the homeless population, given that access to primary care is associated with lower mortality and reductions in unnecessary emergency department (ED) visits and hospital admissions.16 17 Indeed, there is abundant evidence suggesting that homeless adults rely heavily on acute care services, and rates of ED visits and hospitalisations are much higher among homeless versus non-homeless adults.18–21 A related problem is the high rate of hospital readmissions among homeless adults.18 19 22 23 Many of these readmissions are thought to be potentially preventable with more complete treatment and better coordination of health and social services following hospital discharge.24 25 In the general population, timely access to primary care follow-up after hospitalisation has been consistently associated with lower rates of readmissions.26–28 Qualitative studies have also revealed that homeless individuals face unique challenges following discharge from hospital, such as difficulties storing medication, inability to find shelter and not being provided appropriate discharge instructions.29–31 Altogether, systemic barriers to primary care, competing priorities and poor care transitions all contribute towards poor posthospital outcomes and reliance on acute care settings among homeless individuals.

Case management is a core component of care for homeless individuals, serving to help navigate and coordinate health and social services.32 Such programmes have been adapted and implemented for several subgroups of homeless individuals, including frequent users of acute healthcare services and those with complex needs and mental illness.33–35 Systematic reviews have found that case management is effective in improving housing stability, reducing substance use and removing barriers to securing employment in this population.32 36 However, few studies have rigorously evaluated the effect of case management on broad posthospital outcomes among homeless patients overall.37 One randomised controlled trial (RCT) of a combined transitional housing, long-term housing and case management intervention reported reductions in hospitalisations, number of hospital days and ED visits among homeless adults with chronic illnesses.35 Another more recent RCT of a combined permanent supportive housing and case management intervention found significant reductions in psychiatric ED visits and increases in use of outpatient mental health services.38 However, these studies could not evaluate the independent effects of multiple intervention components and did not assess other posthospital or patient-reported outcomes. One RCT of a physician-led model of multidisciplinary care coordination, advocacy and hospital discharge planning found a significant increase in quality of life and reduction in street homelessness, but the intervention had no effect on length of hospital stay or postdischarge acute care utilisation.39 However, the study was underpowered with low recruitment and follow-up rates. A smaller RCT of a Critical Time Intervention (CTI) case management programme focused specifically on homeless patients with severe mental illness was found to improve continuity of care, prevent homelessness and reduce psychiatric readmissions following hospital discharge.40–42

Accordingly, this current RCT seeks to investigate the effectiveness of an adapted CTI case management programme—the Navigator Programme—in improving posthospital outcomes among adults experiencing homelessness at an urban academic teaching hospital and an urban community teaching hospital in Toronto, Canada. CTI is a time-limited case management programme which delivers focused case management at critical times or situations in the lives of clients, such as transitioning from hospital care to community care.36 The Navigator Programme features Homeless Outreach Counsellors (HOCs)—whose roles are to create strong links between community services and patients through regular contact, supporting patients in following their postdischarge care plans, and helping patients in meeting their health-related and social-related competing priorities. This intervention was informed by a recent prospective cohort study conducted at the urban academic teaching hospital, which found that having an active case manager, sending discharge summaries to PCPs, and informal support were associated with reduced readmissions among homeless adults.22 The first HOC position was created in February 2019 and has since been expanded to two positions and adapted through conversations with community partners and hospital staff. Ultimately, the goal of the Navigator Programme is to help homeless patients who are discharged from the hospital overcome systemic barriers and discontinuities in care that often result in poor health and high acute care utilisation.

Objectives

This RCT seeks to evaluate the effectiveness of the Navigator Programme in improving posthospital outcomes among adults experiencing homelessness. It will specifically evaluate outcomes related to PCP follow-up, acute care utilisation, difficulties meeting subsistence needs, care transition and overall health following hospital discharge.

A detailed mixed-methods process evaluation will be conducted alongside the RCT. This evaluation primarily aims to provide a deeper understanding of intervention implementation, mechanisms of change within the intervention and the way in which the Navigator Programme interacts with the internal and external contexts to influence both implementation and RCT outcomes in expected or unexpected ways.43 44 This evaluation will also aim to understand the in-hospital and postdischarge experiences of participants in the intervention and control arms, exploring differences and similarities qualitatively. It is important to investigate how RCT outcomes are shaped by intervention implementation or by the intervention itself, and to identify which parts of the Navigator Programme did or did not work to achieve the intended goals and why.

Methods and analysis

Design and setting

This study is a pragmatic RCT that is being conducted at an urban academic teaching hospital (St. Michael’s Hospital) and an urban community teaching hospital (St. Joseph’s Health Centre) in Toronto, Canada. Recruitment began in October 2021 and total recruitment is estimated to be completed in 3 years.

Eligibility criteria

To be eligible for the study, patients must meet the following criteria: (1) be 18 years of age or older, (2) be admitted to any medical or surgical service (excluding psychiatry and obstetrics) and (3) be identified as experiencing homelessness (as per the Canadian definition of homelessness) at the time of admission or anytime during the hospital admission.45 Patients will be excluded from the study if they meet any of the following criteria: (1) are unable to provide informed consent or (2) were connected with a HOC prior to the initiation of the RCT and have received services from the HOC within 90 days preceding their current admission. Each individual patient can be randomised only once during the study period. Patients admitted to psychiatric and obstetric services are excluded from this study because optimal immediate postdischarge follow-up for these patients should be with specialists rather than with PCPs.46 47 Furthermore, recommended follow-up time frames for these patients are often longer than 14 days, thus rendering the primary outcome inapplicable.

Recruitment and data collection

Clinical or research staff will identify potential participants on weekdays. Once identified, the patient will be asked by a member of their circle of care for permission to be introduced to the research team. The research team will then confirm patient eligibility and explain the purpose, process, risks and benefits of the study to potential participants. Participants may choose to enrol in the study by providing written informed consent (online supplemental file 1).

Supplementary data

bmjopen-2022-065688supp001.pdf (278.5KB, pdf)

A baseline interview will be conducted with participants prior to randomisation and as a soon as possible after admission to the hospital on confirmation of eligibility. Sociodemographic information will be collected, including age, gender, race, indigenous identity, education level, housing status and social service utilisation. Participants who complete the baseline interview will receive a $C20 gift card to compensate them for their time. Another 30-day interview will take place at least 30 days (but no longer than 50 days) after the discharge date to assess patient-reported posthospital outcomes. At this time, the research team will contact PCPs to ascertain any follow-up visits. Baseline and 30-day interviews will be conducted in person or remotely. Data from interviews will be collected with tablets using electronic surveys hosted by Snap Professional Software.

The research team will also undertake a chart review of hospital records after discharge to ascertain characteristics of the admission, information about discharge, participant health information and history of alcohol and substance use.

Plans to promote continued participation

Several strategies will be implemented to minimise attrition. At the baseline interview, study participants will be asked to provide detailed contact information, as well as the names and contact information of family, friends and other service providers who may be contacted if the research team cannot reach the participant directly. In addition to active outreach from the research team, participants will be asked to contact the research team after discharge to schedule a 30-day interview. Participants who contact the research team 2–3 weeks following their discharge to confirm their contact information and schedule the 30-day interview will receive an additional $C10 honorarium on completion of the interview. Participants who complete the 30-day interview will also receive a $C40 honorarium and reimbursement for any travel-related expenses, when applicable, for the interview.

Randomisation

Participants will be randomised by a third-party internet randomisation service (randomize.net). The programme will assign participants to either the intervention or the usual care arm using permuted-block randomisation, with a 1:1 allocation ratio and random permuted block sizes of 6 or 8. This process will maintain balanced group sizes between the intervention and usual care arms at intermediate points in the recruitment process and minimise the possibility of the research team predicting study allocation.48

Intervention

Participants randomised to the intervention arm will be assigned to work with an HOC. The HOC will connect with participants as early as possible during the admission and will provide support for 90 days after hospital discharge. The period of support may occasionally be extended beyond 90 days for certain patients, if the HOC deems this to be necessary and appropriate. The main role of the HOC is to support continuity and comprehensiveness of care by helping participants follow their postdischarge plans and facilitating strong links with community-based health and social services. Day-to-day HOC activities fall into five main categories: (1) making connections and referrals to community-based providers, (2) supporting and advocating for patients during the hospital stay and discharge process, (3) supporting patients with health-related matters during the postdischarge period, (4) supporting patients with social-related matters during the postdischarge period and (5) transferring patient-related information to other healthcare and community-based providers (table 1). The intensity and types of support from HOCs will be tailored to the specific needs of the individual.

Table 1.

Examples of main activities of Homeless Outreach Counsellors

Category Examples
1. Connection to community-based providers
  • Referral to case managers, housing workers, harm reduction services, and shelters.

2. Patient advocacy during hospital stay and discharge process
  • Link to addiction and harm reduction services within hospital and surrounding area.

  • Help patients apply for housing, social benefits, and identification.

  • Connect to social activities and provide other materials for in-hospital entertainment.

  • Participate in creation of discharge plan and support patient/team in the actual discharge process.

3. Health-related support after discharge
  • Remind patients about their medication regimes.

  • Arrange medication storage at postdischarge setting.

  • Help patients fill prescriptions (direct patients or accompany them to local pharmacies).

  • Help patients with accessing opioid agonist therapy and safer supply.

  • Help patients procure medical aides and devices.

  • Remind patients about upcoming medical appointments.

  • Attend medical appointments with patients.

  • Help patients find and connect with primary care providers.

  • Help patients arrange for home care, wound care, eye care and dental care.

  • Connect patients to disease-specific programmes.

  • Purchase medical-related items to help follow through with postdischarge plans.

4. Social-related support after discharge
  • Help patients apply for housing, social benefits and identification.

  • Arrange transportation to postdischarge setting.

  • Help patients find alternative shelter based on unique needs.

5. Information transfer
  • Follow-up with shelters and case managers to ensure that they have the patient discharge plan and are supporting it.

  • Ensure that outpatient services are also aware of patient discharge plan and following through with it.

  • Ensure that this hospital and other hospitals are aware of the hospitalisation and discharge plan.

Usual care

Participants in the usual care arm will be discharged without support from the HOCs. However, all participants will still receive support as usual from Care Transition Facilitators (CTFs) and/or social workers. CTFs and social workers help patients during their hospital stay to arrange discharge plans and make follow-up arrangements. However, unlike HOCs, CTFs and social workers do not typically work with patients after hospital discharge.

The typical discharge process involves counselling from the discharging physician and healthcare team, who make recommendations or appointments for follow-up care as needed. Patients will also be provided with a written discharge summary and prescription(s) as needed. If the patient has an identified PCP, a copy of the discharge summary is emailed to the PCP.

Data linkage

Participant data will be linked to ICES data. ICES is an independent, non-profit research institute funded by an annual grant from the Ontario Ministry of Health and the Ministry of Long-Term Care. As a prescribed entity under Ontario’s privacy legislation, ICES is authorised to collect and use health-related data for the purposes of health system analysis, evaluation and decision support. Secure access to these data is governed by policies and procedures that are approved by the Information and Privacy Commissioner of Ontario.

Data from participants will be linked to ICES administrative health data from 3 years prior to the admission to 1 year following discharge.49 PCP visits, outpatient visits, ED visits, inpatient hospitalisations and mortality will be ascertained from the Ontario Health Insurance Plan (OHIP) Claims Database, Community Health Center Database, Discharge Abstract Database, Same Day Surgery Database, National Ambulatory Care Reporting System, Ontario Mental Health Reporting System databases, and the Registered Persons Database.

Outcomes

The primary outcome is follow-up with a PCP (physician or nurse practitioner) within 14 days of hospital discharge. This outcome was chosen given the unique and substantial barriers to primary care access faced by the homeless population and the fact that timely access to primary care after hospitalisation is linked to better outcomes.15 50 In-person visits, virtual encounters and phone calls will all be considered as follow-up with a PCP. The primary outcome will be ascertained through participant self-report at the 30-day interview, phone calls to PCP offices, and verification in the OHIP and Community Health Center Databases at ICES. In the event of discrepancies between these sources of information, we will use prespecified rules to adjudicate the primary outcome (online supplemental file 2).

Supplementary data

bmjopen-2022-065688supp002.pdf (71KB, pdf)

Several other outcomes will be assessed given the multiple potential effects that are expected from this complex intervention.44 Secondary outcomes include a composite measure of all-cause mortality or readmission, total number of days spent in hospital postdischarge, and number of ED visits within 30 days, 90 days and 180 days postdischarge. Acute care utilisation (readmissions and days in hospital) outcomes will not include labour and delivery visits and planned readmissions. If a patient is transferred between services within the hospital, the entire hospital stay will be treated as a single admission. Other secondary outcomes include self-reported quality of care transition (three-item Care Transitions Measure) after hospital discharge and self-reported change in difficulties meeting subsistence needs (RAND Course of Homelessness Scale) at the time of the 30-day interview relative to baseline.12 51 Exploratory outcomes include change in health status (EQ-5D-3L; European Quality of Life - Five Dimension - Three Levels) at the time of the 30-day interview relative to baseline, change in quality of life (EQ-5D Visual Analogue Scale) at the time of the 30-day interview relative to baseline, leaving against medical advice at discharge, medication adherence (eight-item Morisky Medication Adherence Scale) at the time of the 30-day interview, connection to a case manager in the community at the time of the 30-day interview, attendance of any non-PCP healthcare appointment within 180 days postdischarge, and time to all-cause mortality or readmission after discharge.52 53 Only non-PCP appointments made by the time of discharge and documented in the discharge summary will be assessed for attendance. Only participants that did not previously report contact with a case manager in the 30 days prior to the baseline interview will be eligible for the connection to a case manager outcome. A summary of outcome domains and study instruments is provided in table 2 and detailed descriptions of study instruments are provided in online supplemental file 3.

Table 2.

Outcome domains, variables and instruments

Domain Variables Instruments
Patient-reported outcomes
  • Health status*†

  • Quality of life*†

  • Medication adherence†

  • Care transition experience†

  • Difficulties meeting subsistence needs*†

  • EQ-5D-3L

  • EQ-5D Visual Analogue Scale

  • Morisky Medication Adherence Scale 8-item

  • Care Transitions Measure 3-item

  • RAND Course of Homelessness Scale

Healthcare utilisation
  • Follow-up with primary care provider†‡

  • Hospital readmissions within 30 days, 90 days and 180 days postdischarge†‡

  • Emergency department visits within 30 days, 90 days and 180 days postdischarge†‡

  • No of days spent in hospital within 30 days, 90 days and 180 days postdischarge†‡

  • Attendance of any non-PCP healthcare appointment within 180 days postdischarge†‡

  • Leaving against medical advice§

Social service utilisation
  • Connection to case manager†

Mortality

*Self-reported from baseline interview.

†Self-reported from 30-day interview.

‡ Ascertained from administrative health data.

§Collected from discharge chart review.

EQ-5D-3L, European Quality of Life - Five Dimension - Three Levels; PCP, primary care provider.

Supplementary data

bmjopen-2022-065688supp003.pdf (103.9KB, pdf)

Sample size

No previous data are available to ascertain 14-day PCP follow-up rates after hospitalisation among people experiencing homelessness under usual care. However, a previous study reported that 14-day PCP follow-up rates after hospitalisation among low socioeconomic status (SES) patients was ~48%.54 An assumption was made that 14-day PCP follow-up rates after hospitalisation among people experiencing homelessness under usual care is around 2/3 that of low-SES patients (32%). This study is powered to detect an effect size of 12%, equivalent to a 37.5% increase in relative rate of follow-up with a PCP within 14 days of discharge. With an α of 0.05, 256 participants per study arm will result in an 80% power to test the study hypothesis. Given an estimated 20% attrition rate based on past studies in this population,55 a total of 640 participants will be recruited for this study.

Blinding

It will not be possible to blind participants, HOCs, or healthcare teams given the active and collaborative nature of the intervention. However, data collectors and data analysts will be blinded to the allocation of participants. The research team member who performs study allocation for a participant will not be involved in the 30-day follow-up interview for that participant.

Statistical analyses

All analyses will follow the intention-to-treat principle. Sample characteristics will be summarised by descriptive statistics (mean, SD, median, IQR and proportion). We will also construct graphs to explore relationships and estimate correlations between participant characteristics and outcomes. Descriptive comparisons between group baseline characteristics and outcomes will be performed with χ2 or Fisher’s exact tests for categorical variables and with t-tests or Wilcoxon rank sum tests for continuous or count variables.

The primary analysis will be performed using the χ2 test to compare two independent proportions of 14-day PCP follow-up. The difference in proportions (risk difference, RD) and 95% CI will be estimated using the Wald method.56 Two secondary analyses will be conducted. The first is a log-binomial regression model including the intervention arm indicator as the covariate. The risk ratio and 95% CI will be estimated from the model. The second is a logistic regression model including the intervention arm indicator as the covariate. The OR and 95% CI will be estimated from the model. To explore potential subgroup effects, multivariable logistic regression models for the primary outcome will be constructed including each of the following prespecified covariates, one-at-a-time and with corresponding interaction terms with the intervention arm: age, sex, current illicit drug use, current risky alcohol use, Charlson Comorbidity Index score and prior acute care utilisation for a mental health reason.

For secondary and exploratory binary outcomes, logistic regression models will be used to estimate ORs and 95% CIs. For count outcomes, Poisson or negative binomial regression models (if overdispersion is suggested by the data) will be used to estimate rate ratios and 95% CIs. For cross-sectional continuous outcomes, linear regression models will be constructed. For longitudinal continuous outcomes, we will consider linear mixed models or generalised estimating equations, depending on the outcome distribution. Models will include the intervention arm indicator, time (baseline vs 30-day interview), and the interaction of intervention arm by time. A significant interaction will indicate that the change from baseline is different between the study groups. This difference and 95% CI will be estimated. For time to all-cause mortality or readmission after discharge, a survival analysis will be performed. Cumulative event rates will be calculated with the Kaplan-Meier method, with event or censoring times calculated from the date of discharge. Differences in Kaplan-Meier survival curves between the study arms will be assessed using the log-rank test.

Any missing data will be considered, and multiple imputation will be performed if indicated either for the main analyses or as sensitivity analyses.57 All analyses will be conducted using R version 4.1.2, STATA version 16 and SAS version 9.4. All statistical tests will be two-sided and a p value of 0.05 or less will indicate statistical significance. Adjustments will not be conducted for multiple comparisons. This protocol follows guidance from the Standard Protocol Items: Recommendations for Interventional Trials.58

Process evaluation methods and analysis

In keeping with recommendations from the Medical Research Council on Process Evaluations of Complex Interventions,43 we have designed a pragmatic mixed-methods process evaluation that will gather quantitative measures on programme activities and qualitative data on how participants experience the intervention and how staff experience its implementation and operationalisation. Three domains will be explored in this evaluation: implementation, mechanisms of impact and context. Multiple data collection methods will be employed to better understand intervention implementation, mechanisms of change and important contextual influences on the Navigator Programme. These methods and their corresponding process evaluation domains, research questions and data sources are outlined in table 3, and include chart review, non-participant observation (NPO), semistructured interviews and field notes. Chart review will include data on the number and nature of interactions between HOCs and participants, community service providers and healthcare team, collected from a database developed specifically for the Navigator Programme. NPO is a process of observing participants and the programme setting without actively participating, and can be helpful for assessing the finer details and spirit of implementation, mechanisms of change and programme activities and contexts.59 In this study, NPO will entail accompanying the HOCs as they do their day-to-day work at the hospital and in the community. Semistructured interviews will be conducted with the HOCs (n=2), the implementation team (n=4), hospital physicians and staff (n=25–50), community service providers (n=10–20) that interact with the Navigator Programme, and individuals experiencing homelessness enrolled in the study in both the intervention (n=15–25) and control arms (n=15–25).

Table 3.

Process evaluation domains, questions and data collection

Process evaluation domains Research questions Core information Data type Data sources Records kept
Across domains: What are some unanticipated consequences of the Navigator Programme?
Domain 1: Implementation To what extent was the Navigator Programme (the intervention) implemented and delivered as intended? Fidelity: quality of the intervention delivery, capturing the nature of what was delivered and not just the specific activities Interviews
Non-participant observation
Research and implementation team meetings
Documentation from planning phase
Interviews with HOCs, hospital physicians and staff, and community service providers, who interact with the Navigator Programme, and with the implementation team
Bi-weekly research team meetings; meetings with HOC and implementation team
Team records
Audio recordings and transcripts, field notes and memos, meeting notes, and memos about team records
What was delivered in practice? Dose: amount of and type of activity Participant interactions with intervention: no and nature of interactions between the HOCs and participants, community service providers, and healthcare team for each participant HOC patient chart for all study participants in the intervention arm
Meetings with HOCs and implementation team to discuss participant discharge from programme
Chart review and meeting notes
Domain 2: Mechanisms
2 a. Mechanisms of Impact and Change
What were the key ingredients and elements of the intervention?
Which elements of the intervention supported meeting intervention goals?
Which elements of the intervention challenged meeting intervention goals?
Mechanisms of impact and change (eg, trust and rapport, relationship-building, communication) will be explored qualitatively Interviews
Non-participant observation
Research and implementation team meetings
Interviews with HOCs, hospital physicians and staff, and community service providers, who interact with the Navigator Programme, and with study participants in both the intervention and control arms
Shadowing HOCs during their day-to-day workflow in the hospital and in the community
Bi-weekly research team meetings and meetings with HOCs and implementation team
Audio recordings and transcripts, field notes and memos, and meeting notes
2b. Mechanisms of Implementation What were the barriers and facilitators to implementing the intervention? Mechanisms of implementation (eg, acceptability of the intervention in the implementation setting) will be explored qualitatively Interviews
Non-participant observation
Research and implementation team meetings
Interviews with HOCs, implementation team, hospital physicians and staff, and community service providers, who interact with the Navigator programme
Shadowing HOCs during their day-to-day workflow in the hospital and in the community
Bi-weekly research team meetings and meetings with HOC and implementation team
Audio recordings and transcripts, field notes and memos, and meeting notes
Domain 3: Context What features of context influenced the intervention implementation and reaching intervention goals? Characteristics of implementation setting (eg, hospital services) Interviews
Research and implementation team meetings
Documentation from planning phase
Interviews with HOCs, implementation team, hospital physicians and staff, and community service providers, who interact with the Navigator Programme
Biweekly research team meetings and meetings with HOCs and implementation team
Team records
Audio recordings and transcripts, field notes and memos, meeting notes, and memos about team records
How do features of context influence intervention implementation and the activities and services delivered?
How do features of the intervention shape the implementation context?
The dynamic influence between multiple domains of the internal and external domains of context (eg, organisational setting, socioeconomic context, and community resources) and implementation and programme activities delivered Interviews
non-participant observation
Research and implementation team meetings
Documentation from planning phase
Interviews with HOCs, hospital physicians and staff, and community service providers, who interact with the Navigator Programme, and with study participants in both the intervention and control arms
Shadowing HOCs during their day-to-day workflow in the hospital and in the community
Biweekly research team meetings and meetings with HOCs and implementation team
Team records
Audio recordings and transcripts, field notes and memos, meeting notes, and memos about team records

HOCs, Homeless Outreach Counsellors.

Quantitative data from chart review will be analysed descriptively to understand intervention fidelity and dose. All qualitative data will be analysed as data are collected. Interviews, field notes and NPO will be analysed separately and then integrated to inform each other on an ongoing basis. Analyses will be conducted by multiple members of the research team and guided by a thematic analysis approach. These qualitative data will be transcribed, notable excerpts coded and similar codes grouped into themes.60 Field notes will be used as initial points of analysis and to contextualise interview data.

Finally, mixed-methods analyses will employ ‘following a thread’ and ‘triangulation’ approaches to bringing quantitative and qualitative data sets together.61 Following separate but concurrent initial analyses of quantitative and qualitative components, key themes and interesting data points arising in one data set will be followed across and explored in other data sets. Ultimately, the data sets will be integrated for interpretation and facilitating the identification of ‘meta-themes’ that cut across the data sets. Data source triangulation and researcher triangulation will enhance reliability of findings and provide a more complete picture.62 Analyses throughout will also pay particular attention to the ways in which intersecting factors such as sex, gender identity, race, ethnicity, sexual orientation, culture, religion, geography, education, disability and income shape the experiences of participants during the intervention.

Patient and public involvement

The Navigator Programme was reviewed by the Community Expert Group (CEG) at the MAP Centre for Urban Health Solutions, Unity Health Toronto. This group is composed of diverse individuals with lived experience of homelessness. The CEG will continue to provide guidance and input on study findings and knowledge translation and exchange.

Ethics and dissemination

Ethics approval

This study has been approved by the Unity Health Toronto Research Ethics Board (REB). All changes to the study protocol are communicated to and receive approval from the REB before implementation.

Participant safety

Study participants who are assigned to the intervention arm will receive the Navigator Programme and may directly benefit from HOC services. Study participants in the usual care arm will not receive any direct benefits.

Involvement in this research poses minimal risks to participants. The baseline and 30-day interviews do not involve questions that are anticipated to cause emotional distress among participants. There is still a possibility that some participants may find certain interview questions to be challenging or uncomfortable. However, participants may decline to answer specific questions and participants may withdraw from the study at any point in time. Should an individual choose to withdraw from the study entirely, they will keep any honorariums, will still have access to usual care, and may request that their information collected up to that point be destroyed.

Dissemination

Study findings will be rapidly communicated to hospital leadership, healthcare systems, community partners and the City of Toronto Shelter, Support and Housing Administration Division. Other key outputs include academic publications, community reports, conference presentations and a Knowledge Sharing Event that will convene people with lived experience of homelessness, hospital staff, community experts, policy makers, shelter managers and staff, researchers, and public health partners to discuss results and implications.

Data protection and retention

The research team will make every effort to keep personal health information private and confidential in accordance with all applicable privacy legislation, including the Personal Health Information Protection Act of Ontario. All participant data that is recorded for study purposes will be deidentified with a random unique study identifier number instead of any personally identifying information. A Master Linking Log with participant identifiers will be stored on a secure computer server in a password-protected file. This file will only be made available to designated members of the research team. Research assistants conducting follow-up interviews will only have access to the name of participants and their unique study identifier.

All study data will be kept on a secure hospital server that cannot be accessed by anyone outside of the research team. Only authorised members of the research team will have access to study data. All study data will be kept for a period of 7 years from the end of the study and then destroyed. The research team will protect study data and keep all information confidential to the greatest extent possible by law.

Supplementary Material

Reviewer comments
Author's manuscript

Footnotes

Twitter: @LiuMichaelON, @JesseJenkinson, @gabefabreau, @StephenHwang

Contributors: SWH conceived of the study. ML, KFP and SWH led the study design and protocol development. MJT, GF, KM, VS and AP assisted with study design. ML, KFP, CP, RB and SWH designed the survey instruments. CP, RB and SV are leading recruitment, data collection and data processing. FE and AR are leading the delivery of the intervention. ML, RN and LR wrote statistical analysis plans and will be conducting statistical analyses. JIRJ led the conceptualisation and development of the mixed-methods process evaluation, with contributions from ML, KFP, OD and SWH. All authors contributed to refinement of the study protocol. ML drafted the manuscript, and all authors reviewed and approved the final manuscript.

Funding: This work was supported by the St. Michael’s Foundation and the Canadian Institutes of Health Research (grant FDN-167263).

Disclaimer: The funders had no role in the analysis and interpretation of the data or the preparation, review, and approval of the manuscript. The views expressed in this publication are the views of the authors and do not necessarily reflect those of the funders.

Competing interests: None declared.

Patient and public involvement: Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.

Provenance and peer review: Not commissioned; externally peer reviewed.

Supplemental material: This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Ethics statements

Patient consent for publication

Not applicable.

References

  • 1.Gaetz S, Dej E, Richter T. The State of Homelessness in Canada 2016, 2016. Available: https://homelesshub.ca/sites/default/files/SOHC16_final_20Oct2016.pdf [Accessed 18 Apr 2022].
  • 2.Liu M, Hwang SW. Health care for homeless people. Nat Rev Dis Primers 2021;7:5. 10.1038/s41572-020-00241-2 [DOI] [PubMed] [Google Scholar]
  • 3.Hwang SW, Wilkins R, Tjepkema M, et al. Mortality among residents of shelters, rooming houses, and hotels in Canada: 11 year follow-up study. BMJ 2009;339:b4036. 10.1136/bmj.b4036 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Aldridge RW, Story A, Hwang SW, et al. Morbidity and mortality in homeless individuals, prisoners, sex workers, and individuals with substance use disorders in high-income countries: a systematic review and meta-analysis. Lancet 2018;391:241–50. 10.1016/S0140-6736(17)31869-X [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Hwang SW. Mortality among men using homeless shelters in Toronto, Ontario. JAMA 2000;283:2152–7. 10.1001/jama.283.16.2152 [DOI] [PubMed] [Google Scholar]
  • 6.Fazel S, Geddes JR, Kushel M. The health of homeless people in high-income countries: descriptive epidemiology, health consequences, and clinical and policy recommendations. Lancet 2014;384:1529–40. 10.1016/S0140-6736(14)61132-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Gutwinski S, Schreiter S, Deutscher K, et al. The prevalence of mental disorders among homeless people in high-income countries: an updated systematic review and meta-regression analysis. PLoS Med 2021;18:e1003750. 10.1371/journal.pmed.1003750 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Liu M, Luong L, Lachaud J, et al. Adverse childhood experiences and related outcomes among adults experiencing homelessness: a systematic review and meta-analysis. Lancet Public Health 2021;6:e836–47. 10.1016/S2468-2667(21)00189-4 [DOI] [PubMed] [Google Scholar]
  • 9.Kushel MB, Gupta R, Gee L, et al. Housing instability and food insecurity as barriers to health care among low-income Americans. J Gen Intern Med 2006;21:71–7. 10.1111/j.1525-1497.2005.00278.x [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Kushel MB, Vittinghoff E, Haas JS. Factors associated with the health care utilization of homeless persons. JAMA 2001;285:200–6. 10.1001/jama.285.2.200 [DOI] [PubMed] [Google Scholar]
  • 11.Hwang SW. Homelessness and health. CMAJ 2001;164:229-33. [PMC free article] [PubMed] [Google Scholar]
  • 12.Gelberg L, Gallagher TC, Andersen RM, et al. Competing priorities as a barrier to medical care among homeless adults in Los Angeles. Am J Public Health 1997;87:217–20. 10.2105/ajph.87.2.217 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Campbell DJT, O'Neill BG, Gibson K, et al. Primary healthcare needs and barriers to care among Calgary's homeless populations. BMC Fam Pract 2015;16:139. 10.1186/s12875-015-0361-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Khandor E, Mason K, Chambers C. Access to primary health care among homeless adults in Toronto, Canada: results from the street health survey. Open Med 2011;5:94–103. [PMC free article] [PubMed] [Google Scholar]
  • 15.White BM, Newman SD. Access to primary care services among the homeless: a synthesis of the literature using the equity of access to medical care framework. J Prim Care Community Health 2015;6:77–87. 10.1177/2150131914556122 [DOI] [PubMed] [Google Scholar]
  • 16.O'Toole TP, Buckel L, Bourgault C, et al. Applying the chronic care model to homeless veterans: effect of a population approach to primary care on utilization and clinical outcomes. Am J Public Health 2010;100:2493–9. 10.2105/AJPH.2009.179416 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Basu S, Berkowitz SA, Phillips RL, et al. Association of primary care physician supply with population mortality in the United States, 2005-2015. JAMA Intern Med 2019;179:506–14. 10.1001/jamainternmed.2018.7624 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Miyawaki A, Hasegawa K, Figueroa JF, et al. Hospital readmission and emergency department Revisits of homeless patients treated at Homeless-Serving hospitals in the USA: observational study. J Gen Intern Med 2020;35:2560–8. 10.1007/s11606-020-06029-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Khatana SAM, Wadhera RK, Choi E, et al. Association of homelessness with Hospital Readmissions-an analysis of three large states. J Gen Intern Med 2020;35:2576–83. 10.1007/s11606-020-05946-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Liu M, Richard L, Campitelli MA. Hospitalizations during the COVID-19 pandemic among recently homeless individuals: a retrospective population-based matched cohort study. J Gen Intern Med 2022:1–10. 10.1007/s11606-022-07506-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Raven MC, Tieu L, Lee CT, et al. Emergency department use in a cohort of older homeless adults: results from the hope home study. Acad Emerg Med 2017;24:63–74. 10.1111/acem.13070 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Wang A, Pridham KF, Nisenbaum R, et al. Factors associated with readmission among general internal medicine patients experiencing homelessness. J Gen Intern Med 2021;36:1944–50. 10.1007/s11606-020-06483-w [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Doran KM, Ragins KT, Iacomacci AL, et al. The revolving Hospital door: Hospital readmissions among patients who are homeless. Med Care 2013;51:767–73. 10.1097/MLR.0b013e31829fafbb [DOI] [PubMed] [Google Scholar]
  • 24.Kripalani S, LeFevre F, Phillips CO, et al. Deficits in communication and information transfer between hospital-based and primary care physicians: implications for patient safety and continuity of care. JAMA 2007;297:831–41. 10.1001/jama.297.8.831 [DOI] [PubMed] [Google Scholar]
  • 25.Auerbach AD, Kripalani S, Vasilevskis EE, et al. Preventability and causes of readmissions in a national cohort of general medicine patients. JAMA Intern Med 2016;176:484–93. 10.1001/jamainternmed.2015.7863 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Misky GJ, Wald HL, Coleman EA. Post-hospitalization transitions: examining the effects of timing of primary care provider follow-up. J Hosp Med 2010;5:392–7. 10.1002/jhm.666 [DOI] [PubMed] [Google Scholar]
  • 27.Brooke BS, Stone DH, Cronenwett JL, et al. Early primary care provider follow-up and readmission after high-risk surgery. JAMA Surg 2014;149:821–8. 10.1001/jamasurg.2014.157 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Hernandez AF, Greiner MA, Fonarow GC, et al. Relationship between early physician follow-up and 30-day readmission among Medicare beneficiaries hospitalized for heart failure. JAMA 2010;303:1716–22. 10.1001/jama.2010.533 [DOI] [PubMed] [Google Scholar]
  • 29.Greysen SR, Allen R, Lucas GI, et al. Understanding transitions in care from hospital to homeless shelter: a mixed-methods, community-based participatory approach. J Gen Intern Med 2012;27:1484–91. 10.1007/s11606-012-2117-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Gunner E, Chandan SK, Marwick S, et al. Provision and accessibility of primary healthcare services for people who are homeless: a qualitative study of patient perspectives in the UK. Br J Gen Pract 2019;69:e526–36. 10.3399/bjgp19X704633 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Jenkinson JIR, Hwang SW, Strike C, et al. “We don't have a good system for people who don't have a home and don't need a hospital”: Contextualizing the hospital discharge process for people experiencing homelessness in Toronto, Canada. SSM - Qualitative Research in Health 2022;2:100056. 10.1016/j.ssmqr.2022.100056 [DOI] [Google Scholar]
  • 32.Pottie K, Kendall CE, Aubry T, et al. Clinical guideline for homeless and vulnerably housed people, and people with lived homelessness experience. CMAJ 2020;192:E240–54. 10.1503/cmaj.190777 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.McCormack RP, Hoffman LF, Wall SP, et al. Resource-Limited, collaborative pilot intervention for chronically homeless, alcohol-dependent frequent emergency department users. Am J Public Health 2013;103 Suppl 2:S221–4. 10.2105/AJPH.2013.301373 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Nelson G, Aubry T, Lafrance A. A review of the literature on the effectiveness of housing and support, assertive community treatment, and intensive case management interventions for persons with mental illness who have been homeless. Am J Orthopsychiatry 2007;77:350–61. 10.1037/0002-9432.77.3.350 [DOI] [PubMed] [Google Scholar]
  • 35.Sadowski LS, Kee RA, VanderWeele TJ, et al. Effect of a housing and case management program on emergency department visits and hospitalizations among chronically ill homeless adults: a randomized trial. JAMA 2009;301:1771–8. 10.1001/jama.2009.561 [DOI] [PubMed] [Google Scholar]
  • 36.de Vet R, van Luijtelaar MJA, Brilleslijper-Kater SN, et al. Effectiveness of case management for homeless persons: a systematic review. Am J Public Health 2013;103:e13–26. 10.2105/AJPH.2013.301491 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Luchenski SA, Dawes J, Aldridge RW, et al. Hospital-Based preventative interventions for people experiencing homelessness in high-income countries: a systematic review. EClinicalMedicine 2022;54:101657. 10.1016/j.eclinm.2022.101657 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Raven MC, Niedzwiecki MJ, Kushel M. A randomized trial of permanent supportive housing for chronically homeless persons with high use of publicly funded services. Health Serv Res 2020;55 Suppl 2:797–806. 10.1111/1475-6773.13553 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Hewett N, Buchman P, Musariri J, et al. Randomised controlled trial of GP-led in-hospital management of homeless people (‘Pathway’). Clin Med 2016;16:223–9. 10.7861/clinmedicine.16-3-223 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Tomita A, Herman DB. The role of a critical time intervention on the experience of continuity of care among persons with severe mental illness after hospital discharge. J Nerv Ment Dis 2015;203:65–70. 10.1097/NMD.0000000000000224 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Tomita A, Herman DB. The impact of critical time intervention in reducing psychiatric rehospitalization after hospital discharge. Psychiatr Serv 2012;63:935–7. 10.1176/appi.ps.201100468 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Herman DB, Conover S, Gorroochurn P, et al. Randomized trial of critical time intervention to prevent homelessness after hospital discharge. Psychiatr Serv 2011;62:713–9. 10.1176/ps.62.7.pss6207_0713 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Moore GF, Audrey S, Barker M, et al. Process evaluation of complex interventions: medical Research Council guidance. BMJ 2015;350:h1258. 10.1136/bmj.h1258 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Skivington K, Matthews L, Simpson SA, et al. A new framework for developing and evaluating complex interventions: update of medical Research Council guidance. BMJ 2021;374:n2061. 10.1136/bmj.n2061 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Canadian Observatory on Homelessness . Canadian definition of homelessness Toronto, 2012. [Google Scholar]
  • 46.Stuebe A, Auguste T, Gulati M. ACOG Committee opinion no. 736: optimizing postpartum care. Obstet Gynecol 2018;131:E140–50. 10.1097/AOG.0000000000002633 [DOI] [PubMed] [Google Scholar]
  • 47.Benjenk I, Chen J. Variation of follow-up rate after psychiatric hospitalization of Medicare beneficiaries by hospital characteristics and social determinants of health. Am J Geriatr Psychiatry 2019;27:138–48. 10.1016/j.jagp.2018.08.006 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Suresh K. An overview of randomization techniques: an unbiased assessment of outcome in clinical research. J Hum Reprod Sci 2011;4:8–11. 10.4103/0974-1208.82352 [DOI] [PMC free article] [PubMed] [Google Scholar] [Retracted]
  • 49.Schull MJ, Azimaee M, Marra M. ICES: data, discovery, better health. Int J Popul Data Sci 2019;4. 10.23889/ijpds.v4i2.1135 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.Purkey E, MacKenzie M. Experience of healthcare among the homeless and vulnerably housed a qualitative study: opportunities for equity-oriented health care. Int J Equity Health 2019;18:1–7. 10.1186/s12939-019-1004-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.Parry C, Mahoney E, Chalmers SA, et al. Assessing the quality of transitional care: further applications of the care transitions measure. Med Care 2008;46:317–22. 10.1097/MLR.0b013e3181589bdc [DOI] [PubMed] [Google Scholar]
  • 52.Moon SJ, Lee W-Y, Hwang JS, et al. Accuracy of a screening tool for medication adherence: a systematic review and meta-analysis of the Morisky medication adherence Scale-8. PLoS One 2017;12:e0187139. 10.1371/journal.pone.0187139 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.Janssen MF, Pickard AS, Golicki D, et al. Measurement properties of the EQ-5D-5L compared to the EQ-5D-3L across eight patient groups: a multi-country study. Qual Life Res 2013;22:1717–27. 10.1007/s11136-012-0322-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54.Kangovi S, Mitra N, Grande D, et al. Patient-Centered community health worker intervention to improve posthospital outcomes: a randomized clinical trial. JAMA Intern Med 2014;174:535–43. 10.1001/jamainternmed.2013.14327 [DOI] [PubMed] [Google Scholar]
  • 55.Stergiopoulos V, Hwang SW, Gozdzik A, et al. Effect of scattered-site housing using rent supplements and intensive case management on housing stability among homeless adults with mental illness: a randomized trial. JAMA 2015;313:905–15. 10.1001/jama.2015.1163 [DOI] [PubMed] [Google Scholar]
  • 56.Fagerland MW, Lydersen S, Laake P. Recommended confidence intervals for two independent binomial proportions. Stat Methods Med Res 2015;24:224-54. 10.1177/0962280211415469 [DOI] [PubMed] [Google Scholar]
  • 57.Jakobsen JC, Gluud C, Wetterslev J, et al. When and how should multiple imputation be used for handling missing data in randomised clinical trials - a practical guide with flowcharts. BMC Med Res Methodol 2017;17:162. 10.1186/s12874-017-0442-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 58.Chan A-W, Tetzlaff JM, Gøtzsche PC, et al. Spirit 2013 explanation and elaboration: guidance for protocols of clinical trials. BMJ 2013;346:e7586. 10.1136/bmj.e7586 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 59.Moore G, Audrey S, Barker M, et al. Process evaluation in complex public health intervention studies: the need for guidance. J Epidemiol Community Health 2014;68:101–2. 10.1136/jech-2013-202869 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 60.Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006;3:77–101. 10.1191/1478088706qp063oa [DOI] [Google Scholar]
  • 61.O'Cathain A, Murphy E, Nicholl J. Three techniques for integrating data in mixed methods studies. BMJ 2010;341:c4587–50. 10.1136/bmj.c4587 [DOI] [PubMed] [Google Scholar]
  • 62.Creswell JW. Qualitative inquiry and research design: choosing among five traditions. 9, 1998. [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary data

bmjopen-2022-065688supp001.pdf (278.5KB, pdf)

Supplementary data

bmjopen-2022-065688supp002.pdf (71KB, pdf)

Supplementary data

bmjopen-2022-065688supp003.pdf (103.9KB, pdf)

Reviewer comments
Author's manuscript

Articles from BMJ Open are provided here courtesy of BMJ Publishing Group

RESOURCES