Skip to main content
NCBI home page
PLOS One logoLink to PLOS One
. 2023 Dec 5;18(12):e0295178. doi: 10.1371/journal.pone.0295178

Impact of San Francisco’s New Street crisis response Team on Service use among people experiencing homelessness with mental and substance use disorders: A mixed methods study protocol

Matthew L Goldman 1,2,*, Megan McDaniel 1, Deepa Manjanatha 1,3, Monica L Rose 1, Glenn-Milo Santos 1,4,5, Starley B Shade 4,6, Ann A Lazar 4,7, Janet J Myers 8,9, Margaret A Handley 4,8,9, Phillip O Coffin 1,8
Editor: De-Chih Lee10
PMCID: PMC10697604  PMID: 38051726

Abstract

Mobile crisis services for people experiencing distress related to mental health or substance use are expanding rapidly across the US, yet there is little evidence to support these specific models of care. These new programs present a unique opportunity to expand the literature by utilizing implementation science methods to inform the future design of crisis systems. This mixed methods study will examine the effectiveness and acceptability of the Street Crisis Response Team (SCRT), a new 911-dispatched multidisciplinary mobile crisis intervention piloted in San Francisco, California. First, using quantitative data from electronic health records, we will conduct an interrupted time series analysis to quantitatively examine the impacts of the SCRT on people experiencing homelessness who utilized public behavioral health crisis services in San Francisco between November 2019 and August 2022, across four main outcomes within 30 days of the crisis episode: routine care utilization, crisis care reutilization, assessment for housing services, and jail entry. Second, to understand its impact on health equity, we will analyze racial and ethnic disparities in these outcomes prior to and after implementation of the SCRT. For the qualitative component, we will conduct semi-structured interviews with recipients of the SCRT’s services to understand their experiences of the intervention and to identify how the SCRT influenced their health-related trajectories after the crisis encounter. Once complete, the quantitative and qualitative findings will be further analyzed in tandem to assist with more nuanced understanding of the effectiveness of the SCRT program. This evaluation of a novel mobile crisis response program will advance the field, while also providing a model for how real-world program implementation can be achieved in crisis service settings.

Introduction

Mobile crisis services for people experiencing distress related to mental health or substance use are expanding rapidly across the US [1]. Mobile crisis has a unique ability to respond rapidly in a less restrictive environment [2] and to coordinate with community partners such as law enforcement and emergency departments to divert people from those settings [3]. Recent federal legislation incentivized Medicaid coverage for mobile crisis services, and state and local governments have begun to invest significantly in expanding these programs [4].

With suicide rates and overdose deaths continuing to climb [5, 6], scarce resources and strained workforce must be positioned to be as high impact as possible. Although clinical trials in crisis services are often unfeasible given the high acuity of clinical scenarios and limitations to ethical randomization to experimental conditions, the creation of a range of crisis programs in real-world settings presents an opportunity to use implementation science methods to characterize which programs and models are meeting their stated objectives and informing future best practices [7].

Prior single-site quasi-experimental studies of mobile crisis programs have found impacts on service utilization and costs [8]. However, there are many remaining questions about how effective mobile crisis teams are at linking people to routine care and social services or at preventing adverse outcomes such as jail entry or reutilization of acute care services. Furthermore, while mobile crisis programs are often justified by reducing criminalization of people with mental illness, few studies have focused on programs that target high-risk populations such as people experiencing homelessness (PEH) [9, 10].

This paper describes the research protocol for an evaluation of a Street Crisis Response Team (SCRT) in San Francisco, California, a model that tailors its services to PEH. Mental illness and substance use disorders are highly prevalent among adult PEH in San Francisco, yet access to appropriately tailored services is limited. Especially troubling is the inequity of the burden of these diagnoses within this population: a third of PEH in San Francisco identify as Black/African American, compared to 5% of the overall population. Across the US, people with serious mental illness comprise approximately one quarter of all PEH, and up to one third has a substance use disorder [2], with people of color dramatically over-represented in this population [11]. Despite these trends, engaging PEH in mental health and substance use care as well as social services is impeded by marginalization, dehumanization, and structural violence, which interfere with trust and engagement in health care and social services [12].

We will employ an implementation science approach to study a novel mobile crisis program by drawing on empirical data from the health care system as well as perspectives from service recipients, which will allow for a deeper understanding of the utility—and potential limitations—of measuring traditional service outcomes in this setting [13]. We anticipate that this study will yield findings that inform both the implementation of existing, and the planning and evaluation of future mobile crisis programs.

Methods

To evaluate the impact of the SCRT, we will use a QUANT-QUAL mixed methods implementation science approach [14, 15]. First, we will use quantitative methods to examine if there are changes in utilization of mental health, substance use, and housing services as well as jail entry following implementation of the SCRT among PEH who present in behavioral health crisis to acute care settings within San Francisco’s public health system. Second, we will evaluate the ability of the SCRT to enhance equity by stratifying our analysis by ethnoracial groups to examine pre-implementation disparities as well as post-implementation worsening, perpetuation, or resolution of baseline disparities. We will use our quantitative analysis to set the sampling frame for qualitative semi-structured interviews, which will be conducted with the SCRT service recipients to understand the facilitators and barriers to achieving their goals. Finally, we will consider the quantitative and qualitative results in combination to help interpret both sets of findings.

Study procedures were approved by the University of California, San Francisco, Committee on Human Subjects Research (Protocol #20–32693). The Committee waived consent for review of healthcare, housing, and jail records, and approved verbal informed consent procedures for telephonic and in-person interview participants.

Program

The Behavioral Health Services division of the San Francisco Department of Public Health (SFDPH) has an extensive infrastructure for mental health and substance use disorder services, yet one important gap has remained: real-time response for people in behavioral health crisis in the streets. In 2019, San Francisco’s 9-1-1 call data indicates that approximately 50,000 behavioral health related calls were received, most of which were responded to by a law enforcement unit. Because most calls were not related to criminal events, and instead to mental health and social needs experienced by PEH, San Francisco behavioral health and governmental leaders worked with community stakeholders to create the SCRT to respond as an alternative able to be more responsive to behavioral health crises that happen on the street [16].

The SCRT was designed based on previous co-responder models [17, 18]. To meet the goal of diverting calls that would typically go to the San Francisco Police Department, this trauma-informed specialty behavioral health team is dispatched solely by 9-1-1 operators. The SCRT utilizes a co-responder model comprised of a behavioral health clinician, a paramedic from the San Francisco Fire Department and a peer specialist. Each team member plays a role in providing care including immediate stabilization of urgent medical need (paramedic), de-escalation of the crisis (behavioral health clinician) and person-centered peer support (peer specialist). The team triages clients to the appropriate level of care, be it through resolution of the crisis in the field, linkage to outpatient mental health and substance use services, or transport to an acute treatment setting. The SCRT was first piloted in San Francisco’s highest demand neighborhoods in December 2020 and then was incrementally expanded to be citywide by June 2021 (Fig 1).

Fig 1. Rollout of the street crisis response team across San Francisco by region, hours of operation, and launch date of each phase of expansion.

Fig 1

Open in a new tab

Landsat-7 image courtesy of the U.S. Geological Survey.

The SCRT also includes follow-up services provided by the Office of Coordinated Care (OCC), which is charged with offering support after the SCRT encounter with the goal of linking clients to outpatient mental health and housing services, thus reducing reutilization of acute services.

Quantitative methods

To assess the implementation of the SCRT, we will use an interrupted time series (ITS) design, which is a quasi-experimental method that allows for non-randomized evaluation of an intervention [19]. In this ITS study, we will measure the effect of the intervention by generating models to assess changes over time in each outcome before and after implementation of the SCRT. This will allow us to model the secular trends in data not due to the intervention itself.

Study population

The SCRT aims to provide specialty mental health response and enhanced resources to adults in San Francisco who experience crises related to mental health and/or substance use disorders and are experiencing homelessness. Therefore, the study population will be defined based on age greater than 18 years, meeting criteria for homelessness in the 12 months prior to or 3 months following the crisis episode, and receipt of crisis care from any of San Francisco’s “front door” programs for people in an acute behavioral health crisis. These settings include two mobile crisis programs (Comprehensive Crisis Services and the SCRT), a crisis stabilization unit (DORE Urgent Care Clinic), and emergency psychiatric services (Zuckerberg San Francisco General Hospital’s Psychiatric Emergency Services [PES] and Emergency Department visits with a primary behavioral health diagnosis) (Fig 2). The population of housed adults utilizing acute behavioral health services will be utilized as a control group in the ITS sensitivity analyses.

Fig 2. System map of crisis system entry points in San Francisco used for defining index crisis episodes.

Fig 2

Open in a new tab

MC–mobile crisis; CF–crisis facility; ED–emergency department; SCRT–Street Crisis Response Team; CCS–Comprehensive Crisis Services (adult/child mobile crisis team in San Francisco); DUCC–DORE Urgent Care Center; ZSFG ED–Zuckerberg San Francisco General Hospital Emergency Department; PES–Psychiatric Emergency Services. ‡—ZSFG ED limited to medical episodes with primary ICD-10 diagnosis of mental health, substance use, or suicide Z-codes.

Data sources and matching procedures

The primary data sources for our quantitative analysis will include electronic health record (EHR) data from the network of clinics funded by the city’s health plan, public housing assessment data, and jail entry data. We will integrate data from SFDPH’s two main EHR vendors, Avatar (NetSmart) and EPIC, which are used by SFDPH mental health and substance use treatment providers, medical clinics, mobile crisis teams, crisis stabilization units, and Zuckerberg San Francisco General Hospital’s medical ED, psychiatric emergency services (PES) and inpatient psychiatry. Homelessness and housing assessment data originate from the Homeless Management Information System (HMIS), which is used by all entry points into the housing service system in San Francisco and automatically links data into the EPIC EHR. Jail entry data originates from the Jail Information Management System (JIMS), which the Department of Public Health’s Jail Health Services clinicians use to assess every person who enters the San Francisco County Jail, as well as Epic following their transition to this EHR in October 2021.

We will use a two-step process for linking records from the Avatar, Epic (including HMIS) and JIMS EHRs by first matching on demographic data fields such as first and last name, date of birth, legal sex, and at least one additional element (e.g., Social Security Number (SSN), full street address, phone number, or email address), and then an additional round of name matching using a Jaro-Winkler based process with matching parameter = 1 (i.e., exact match) [20, 21]. Unique individuals and episodes will be assigned anonymized identifiers to create a limited dataset that includes information about demographics, clinical attributes, dates of service, and zip codes. Authors did not have access to personal health information or other personal data that could identify individuals during or after data collection for the ITS.

Index crisis episode

Conducting an ITS that includes a time-dependent outcome—in this case a routine care episode, acute care episode, housing assessment, or jail entry within 30 days following a crisis episode—requires defining an index crisis episode to start the clock for the outcome time interval. Furthermore, given that there may be multiple index episodes per unique individual, the index crisis episode needs to be defined per ITS interval, which, based on our preliminary power analysis, will be divided as one calendar month per interval. The index crisis episode will therefore be defined as the first instance in a given month that an individual has a crisis episode, with the 30-day post-crisis outcome period trailing the end date of the index episode.

We also need to account for the fact that a single crisis episode may result in multiple contacts with different settings in the acute behavioral health care system. To account for this variability, we will use our clinical knowledge of common care pathways as the basis for a crisis system map (Fig 2) that defines different sequences of care as either a step-up in service intensity (e.g., mobile crisis followed by crisis stabilization or ED), a step-down in service intensity (e.g., PES followed by crisis stabilization), or a distinct crisis episode that mostly likely signifies reutilization (e.g., PES followed by mobile crisis, or two consecutive mobile crisis episodes). We will then combine sequences with a step up or down in service intensity that occur within two or fewer days between the end of the first and start of the second clinical setting, so as not to miscount these care transitions as reutilization. This approach also allows us to create a variable describing these crisis system trajectories, thus allowing us to identify potential differences between index crisis episodes based on crisis system entry point and end point. All crisis services utilized three or more days after the index crisis episode end date will be considered separate from the index crisis episode.

Outcome variables

We will build ITS models using the following four repeated measures outcomes: 1) post-crisis episode routine care utilization within 30 days; 2) post-crisis episode crisis service reutilization within 30 days; 3) post-crisis episode housing assessment within 30 days; and 4) post-crisis episode jail entry within 30 days. Routine care services will include receipt of outpatient services within 30-days following the index crisis episode in programs related to mental health, substance use, primary care, and integrated behavioral health in primary care. We will exclude residential programs and other services that are not intended to serve as routine care nor as a front-door crisis service. Acute care reutilization will include when an individual has a crisis care episode within 30-days following discharge from the index crisis episode. Housing assessment will be determined among those who are identified as PEH as having an HMIS record of receiving a housing assessment within 30-days following the index crisis episode. Finally, we will measure jail entry within 30-days following the index crisis episode based on clinical records from JIMS or Epic that are documented for every person who enters the San Francisco County Jail.

Covariates

Demographic variables for age at time of service, gender identity, sexual orientation, race/ethnicity, housing status and insurance status will be developed in accordance with existing SFDPH reporting guidelines. Gender identity will be determined by information on each person’s sex at birth as well as self-reported gender identity to create categories for cisgender male or female, transgender male or female, and genderqueer or nonbinary. Sexual orientation is based on self-report. Race and ethnicity are recorded separately in both EHR systems and will be cross-referenced and then combined into a single variable by replacing race with ethnicity for those who identify as Hispanic/Latinx [22]. SFDPH defines someone as a person experiencing homelessness if they utilize a service that indicates housing instability (e.g., emergency shelter) or self-report homelessness while accessing health care services. Insurance status at the time of an encounter will be based on EHR billing records and grouped into descriptive categories (e.g., private versus public insurance). A location variable using zip code will be based on last location documented prior to the crisis episode. Multiple imputation will be used to account for missing data.

Diagnoses associated with service encounters will be categorized using the primary ICD-10 diagnostic code based on the Health Care Utilization Project’s Clinical Classifications Software Refined (CCSR) [23]. Given the unreliability of diagnosis data at the time of a crisis encounter [24, 25], we will use diagnoses made in routine service settings in the 90 days prior to the index crisis episode and, only if routine care is unavailable, will use acute-care settings diagnoses.

Additional clinical variables will include whether the index crisis episode resulted in an involuntary psychiatric hold; suicidality as part of the presentation (based on ICD-10 Z codes as well as clinical documentation or indication for involuntary holds as “danger to self”); and violence risk (based on clinical documentation or indication for involuntary holds as “danger to others”) [26]. We will also control for continuous variables describing the number of crisis services, routine care, jail, or housing assessment encounters in the 12-months prior to the index crisis episode. We will also assess the number of crisis services, routine care, jail, or housing assessment encounters in the 12-months prior to the index crisis episode.

Interrupted time series analysis

The ITS analysis will examine three periods: 1) Pre-SCRT baseline (November 2019 to November 2020), 2) SCRT partial implementation (December 2020 to July 2021), and 3) SCRT full implementation (August 2021 to August 2022). Each of the outcome measures will be computed as a monthly proportion, with the numerator equaling the number of individuals meeting criteria for each outcome and the denominator equaling the total target population in a given month. Using month-long time intervals would yield between 8 and 12 data points per time period, though the final interval length may change depending on the trade-offs between length of observation and statistical power.

Generalized estimating equations (GEE) will be used, with robust standard errors to account for within-person correlation [27], to analyze the trends in outcomes pre-implementation, during partial implementation, and after the implementation of SCRT. Models will include a variable for time (month) after the beginning of our observation period, a variable for time each time period subsequent to baseline, and the interaction between these variables to assess change in the trajectory of each outcome. Models will be adjusted by the covariates described above and interaction terms constructed to understand the relative effects of covariates and the ITS variables.

This analysis will use several design and analysis strategies to account for potential threats to internal validity. First, the study design using two nonequivalent groups with staggered implementation (crisis episodes with zip codes corresponding to neighborhoods that did or did not have the SCRT active during the partial implementation phase prior to citywide expansion) will allow for a between-site comparison of the pilot catchment area relative to the non-pilot areas (Fig 1). Additional sub-analyses will examine within-site differences for the pilot neighborhoods across the three time periods, and, separately, within-site differences for the non-pilot neighborhoods. Second, a non-equivalent non-treatment control group (non-homeless adults accessing crisis services) will be compared to the target population (homeless adults accessing crisis services) using a difference of differences approach to account for secular variations in mental health and substance use service utilization. Third, a series of non-equivalent dependent outcome variables that are not expected to be impacted by the implementation of the SCRT (e.g., non-crisis initiation of outpatient mental health services) will be evaluated across the same time periods with a similar goal of accounting for secular variations in mental health service utilization. Additional issues such as autocorrelation of repeated measures on individuals will be corrected for in the final analyses.

Equity analysis

We will conduct additional ITS analyses of each model by stratifying the population by our covariate on race and ethnicity (defined by patient self-report in EHR demographic records). By stratifying the total population into sub-categories of interest, including ethnoracial groups, and comparing the outcomes of the ITS analyses, we will be able to describe whether the potential impact of the SCRT intervention was equitably distributed across racial groups [28–31]. Furthermore, we will be able to identify if potential baseline disparities are perpetuated or reduced by implementation of the SCRT, as has been described in the RE-AIM model [32, 33].

Qualitative design

To identify individual, community and societal-level factors associated with optimal and suboptimal implementation of the SCRT from a client perspective, semi-structured interviews will be conducted with recipients of the SCRT and post-crisis outreach services. Qualitative data will be analyzed using a thematic analysis approach and further support interpretation of the quantitative findings.

Interview participant recruitment and consent

Our study team will partner with the post-SCRT outreach clinicians to engage with individuals who received SCRT services to help recruit a sample purposively selected based on receipt of services and the quantitative post-crisis routine care and reutilization outcomes. Potential participants will be eligible if they received SCRT services 7 to 90 days prior to contact with the study team, to provide time for the resolution of their recent crisis while limiting potential effects of recall bias. During post-crisis follow-up, OCC clinicians will describe the study and, for those interested, document consent for our team to contact them and to review their medical records. Those who consent to be contacted by the research team will have the option to provide their contact information (i.e., phone number, email address) for the research team to identify and contact them for recruitment purposes. Potential participants will also be recruited directly using various methods, such as circulating flyers in areas that frequently serve PEH (e.g., shelters, social service organizations, and on SCRT units themselves). We will then contact prospective participants and arrange to meet participants in person or speak by telephone. Consent will be obtained immediately prior to the interview. Verbal consent procedures were approved by the UCSF Committee on Human Subjects Research to allow for telephonic interviews; written consent was also obtained for in-person interviews. A $60 gift card incentive will be offered to all participants upon completion of the interview. Community stakeholders will be engaged to provide additional suggestions about recruitment strategies.

Semi-Structured Interviews

Questions for the semi-structured interviews will be developed a priori based on examination of the literature, our team’s clinical experience with crisis services, and feedback from the SCRT team and community stakeholders. The 30- to 60-minute interview will include specific questions about baseline engagement in health care and housing systems, SCRT’s accessibility, SCRT intervention and assessment, post-crisis linkage to care, and overall client experience. Open-ended questions and follow-up prompts will aim to elicit the participant’s perception of the encounter and SCRT’s role in their broader experiences of homelessness, mental illness and substance use. All interviews will be conducted by teams of two researchers, audio recorded and transcribed for qualitative analysis using Atlas.ti Version 9 software.

Qualitative coding and analysis

The interview findings will be coded for salient themes using a grounded theory approach [34]. An initial codebook will be developed based on the interview guide, prior literature and overall study goals, and then during the analysis will allow for new codes and themes to emerge organically from the text [35]. Our research team will meet weekly about emerging themes and to discuss iterative changes to the codebook until group consensus determines that saturation has been reached [36]. Finally, we will examine codes and discuss possible models that help organize the themes, such as Bronfebrenner’s Socio-ecological Model [37].

Mixed methods analysis

Once both the ITS and qualitative analyses have been completed, the results of each will be examined in conjunction with the other to assist with interpretation of the overall study findings. For example, measuring quantitative service utilization outcomes among interview participants can help inform how we interpret their reports of how SCRT impacted their lives. If the equity analyses produce concerning signals for disparities in care, we could incorporate this information into how we analyze qualitative findings about discrimination. Another example would be if we find a lack of significant change in the ITS analysis of post-crisis routine care utilization after implementation of SCRT, in which case our interpretation may be informed by qualitative descriptions of barriers to accessing routine care. These kinds of mixed methods approaches allow us to leverage the nuanced details of qualitative research and the more representative findings in population-level quantitative research to arrive at a stronger set of interpretations and conclusions.

Discussion

Many states and counties throughout the U.S. have turned to mobile crisis services as a potential cost-effective solution to constraints in behavioral health service capacity. A 2020 report issued by the Substance Abuse and Mental Health Services Administration, titled “National Guidelines for Crisis Care–A Best Practice Toolkit,” lays out essential services for a crisis continuum of care: call centers, mobile teams, and stabilization centers [38]. Programs such as SCRT are key to health systems seeking solutions to divert 9-1-1 calls away from law enforcement and instead to specialized behavioral health clinicians who can triage and link clients to an appropriate level of care. Further implementation science research is essential to grow the evidence base for effective mobile crisis models to help address the behavioral health needs of people experiencing a crisis, especially vulnerable populations such as people experiencing homelessness and health disparities.

The study described in this protocol exemplifies how implementation science methods can increase our understanding of the effectiveness and acceptability of mobile crisis response programs. By describing factors and mechanisms that facilitate or impede the effectiveness of SCRT in diverting clients from unnecessary additional crisis services or jail entry while improving linkage to routine care and housing services, this study will inform future strategies for implementing mobile crisis interventions into other settings. Additionally, the qualitative approaches will provide a nuanced understanding of how an intervention such as SCRT impacts the lives of adults experiencing homelessness in San Francisco and allow for more refined interpretation of the quantitative findings.

Limitations

There are several limitations to the research methods described above. While ITS designs can measure the impact of a non-randomized intervention, this quasi-experimental approach may not yield definitive results and may be specific to San Francisco and therefore less generalizable to other regions. The electronic records used to describe health, housing and jail service utilization does not capture all services provided in settings outside of the San Francisco Department of Public Health. Though data from 9-1-1 dispatch might help identify which cases are being diverted from SFPD to the SCRT, these data will not be available for this evaluation. It is not possible to fully account for the potential impacts of COVID on data collection and service utilization. Incomplete data on the location of PEH may impede any location-specific analysis; in this case, only the pre-pilot and full implementation segments would be included in the ITS. Limitations in the semi-structured interviews may include sampling bias that results from recruitment through the clinical program, the difficulty of locating potential participants, and the need to exclude potential participants who do not have capacity to consent due to psychiatric or other symptoms; recall bias related to traumatic experiences while in crisis; and desirability bias due to difficulty describing negative experiences of the health care system.

Conclusion

Studying the implementation of this novel mobile crisis service model will take a step toward increasing our understanding and uncovering mechanisms of impact associated with these widely utilized yet under-evaluated programs. As mobile crisis programs are introduced across the U.S., we aim to provide a model for how mixed-methods evaluations can be achieved in real-world program crisis service settings as a way to advance the literature in this important health services area.

Supporting information

S1 Checklist

(DOCX)

Click here for additional data file. (48.2KB, docx)

Data Availability

This manuscript is for a study protocol in which we don't have data or results to potentially share with a third party.

Funding Statement

This work was supported by the Robert Wood Johnson Foundation’s Health Systems Transformation Research Coordinating Center Call for Proposals: Research to Advance Models of Care for Medicaid-Eligible Populations (Grant #78236). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

References

  • 1.Odes R, Manjanatha D, Looper P, McDaniel M, Goldman ML. How to Reach a Mobile Crisis Team: Results From a National Survey. PS. 2023. Mar 20;appi.ps. doi: 10.1097/NRL.0b013e3181c29f12 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Deschietere G. Mobility in Psychiatry, an Alternative to Forced Hospitalization? Psychiatr Danub. 2018. Nov;30(Suppl 7):495–7. [PubMed] [Google Scholar]
  • 3.Dewa CS, Loong D, Trujillo A, Bonato S. Evidence for the effectiveness of police-based pre-booking diversion programs in decriminalizing mental illness: A systematic literature review. PLOS ONE. 2018. Jun 19;13(6):e0199368. doi: 10.1371/journal.pone.0199368 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.sho21008.pdf [Internet]. [cited 2023 Feb 22]. Available from: https://www.medicaid.gov/federal-policy-guidance/downloads/sho21008.pdf
  • 5.Stone DM, Mack KA, Qualters J. Recent Changes in Suicide Rates, by Race and Ethnicity and Age Group—United States, 202. 2023;72(6). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Kariisa M, Davis NL, Kumar S, Seth P, Mattson CL, Chowdhury F, et al. Vital Signs: Drug Overdose Deaths, by Selected Sociodemographic and Social Determinants of Health Characteristics—25 States and the District of Columbia, 2019–2020. MMWR Morb Mortal Wkly Rep. 2022. Jul 22;71(29):940–7. doi: 10.15585/mmwr.mm7129e2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Tabak RG, Khoong EC, Chambers DA, Brownson RC. Bridging Research and Practice. American Journal of Preventive Medicine. 2012. Sep;43(3):337–50. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Guo S, Biegel DE, Johnsen JA, Dyches H. Assessing the Impact of Community-Based Mobile Crisis Services on Preventing Hospitalization. PS. 2001. Feb;52(2):223–8. doi: 10.1176/appi.ps.52.2.223 [DOI] [PubMed] [Google Scholar]
  • 9.Fazel S, Geddes JR, Kushel M. The health of homeless people in high-income countries: descriptive epidemiology, health consequences, and clinical and policy recommendations. The Lancet. 2014. Oct;384(9953):1529–40. doi: 10.1016/S0140-6736(14)61132-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Townley G, Sand K, Kindschuh T, Brott H, Leickly E. Engaging unhoused community members in the design of an alternative first responder program aimed at reducing the criminalization of homelessness. Journal of Community Psychology. 2022;50(4):2013–30. doi: 10.1002/jcop.22601 [DOI] [PubMed] [Google Scholar]
  • 11.Olivet Jeffrey, Dones Marc, Richard Molly, Wilkey Catriona, Yampolskaya Svetlana, Maya Beit-Arie, et al. Supporting Partnerships for Anti-Racist Communities: Phase One Study Findings [Internet]. 2018. Mar [cited 2020 Aug 22]. Available from: https://bit.ly/2uS8odK [Google Scholar]
  • 12.Magwood O, Leki VY, Kpade V, Saad A, Alkhateeb Q, Gebremeskel A, et al. Common trust and personal safety issues: A systematic review on the acceptability of health and social interventions for persons with lived experience of homelessness. Federici S, editor. PLoS ONE. 2019. Dec 30;14(12):e0226306. doi: 10.1371/journal.pone.0226306 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Handley MA, Lyles CR, McCulloch C, Cattamanchi A. Selecting and Improving Quasi-Experimental Designs in Effectiveness and Implementation Research. Annual Review of Public Health. 2018;39(1):5–25. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Teddlie C, Yu F. Mixed Methods Sampling: A Typology With Examples. Journal of Mixed Methods Research. 2007. Jan;1(1):77–100. [Google Scholar]
  • 15.Palinkas LA, Aarons GA, Horwitz S, Chamberlain P, Hurlburt M, Landsverk J. Mixed Method Designs in Implementation Research. Adm Policy Ment Health. 2011. Jan;38(1):44–53. doi: 10.1007/s10488-010-0314-z [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.SCRT Final Report_FINAL- 1 year.pdf [Internet]. [cited 2023 Apr 5]. Available from: https://sf.gov/sites/default/files/2022-06/SCRT%20Final%20Report_FINAL-%201%20year.pdf
  • 17.Acknowledgements [Internet]. Vera Institute of Justice. [cited 2023 Feb 22]. Available from: https://www.vera.org/behavioral-health-crisis-alternatives
  • 18.Balfour ME, Hahn Stephenson A, Delany-Brumsey A, Winsky J, Goldman ML. Cops, Clinicians, or Both? Collaborative Approaches to Responding to Behavioral Health Emergencies. PS. 2022. Jun 1;73(6):658–69. doi: 10.1176/appi.ps.202000721 [DOI] [PubMed] [Google Scholar]
  • 19.Wagner AK, Soumerai SB, Zhang F, Ross-Degnan D. Segmented regression analysis of interrupted time series studies in medication use research. J Clin Pharm Ther. 2002. Aug;27(4):299–309. doi: 10.1046/j.1365-2710.2002.00430.x [DOI] [PubMed] [Google Scholar]
  • 20.Winkler WE. MATCHING AND RECORD LINKAGE.: 38.
  • 21.HIPAA Privacy Rule and Its Impacts on Research [Internet]. [cited 2023 Feb 22]. Available from: https://privacyruleandresearch.nih.gov/pr_08.asp
  • 22.Lewis-Fernández R, Raggio GA, Gorritz M, Duan N, Marcus S, Cabassa LJ, et al. GAP-REACH. J Nerv Ment Dis. 2013. Oct;201(10):860–71. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.DXCCSR-User-Guide-v2023-1.pdf [Internet]. [cited 2023 Feb 22]. Available from: https://www.hcup-us.ahrq.gov/toolssoftware/ccsr/DXCCSR-User-Guide-v2023-1.pdf
  • 24.Bledsoe B, Wasden C, Johnson L. Electronic Prehospital Records are Often Unavailable for Emergency Department Medical Decision Making. WestJEM. 2013. Sep 17;14(5):482–8. doi: 10.5811/westjem.2013.1.12665 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.van Walraven C, Bennett C, Forster AJ. Administrative database research infrequently used validated diagnostic or procedural codes. Journal of Clinical Epidemiology. 2011. Oct;64(10):1054–9. doi: 10.1016/j.jclinepi.2011.01.001 [DOI] [PubMed] [Google Scholar]
  • 26.Randall JR, Roos LL, Lix LM, Katz LY, Bolton JM. Emergency department and inpatient coding for self‐harm and suicide attempts: Validation using clinician assessment data. Int J Methods Psychiatr Res. 2017. Feb 24;26(3):e1559. doi: 10.1002/mpr.1559 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Hubbard AE, Ahern J, Fleischer NL, Van der Laan M, Lippman SA, Jewell N, et al. To GEE or not to GEE: comparing population average and mixed models for estimating the associations between neighborhood risk factors and health. Epidemiology. 2010. Jul;21(4):467–74. doi: 10.1097/EDE.0b013e3181caeb90 [DOI] [PubMed] [Google Scholar]
  • 28.Zhou B, Joudeh A, Desai MJ, Kwan B, Nalawade V, Whitcomb BW, et al. Trends in Infertility Care Among Commercially Insured US Women During the COVID-19 Pandemic. JAMA Network Open. 2021. Oct 6;4(10):e2128520. doi: 10.1001/jamanetworkopen.2021.28520 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Mi T, Hung P, Li X, McGregor A, He J, Zhou J. Racial and Ethnic Disparities in Postpartum Care in the Greater Boston Area During the COVID-19 Pandemic. JAMA Network Open. 2022. Jun 23;5(6):e2216355. doi: 10.1001/jamanetworkopen.2022.16355 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Mooney AC, Giannella E, Glymour MM, Neilands TB, Morris MD, Tulsky J, et al. Racial/Ethnic Disparities in Arrests for Drug Possession After California Proposition 47, 2011–2016. Am J Public Health. 2018. Aug;108(8):987–93. doi: 10.2105/AJPH.2018.304445 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Goldman ML, Vinson SY. Centering equity in mental health crisis services. World Psychiatry. 2022;21(2):243–4. doi: 10.1002/wps.20968 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Carrillo G, Roh T, Baek J, Chong-Menard B, Ory M. Evaluation of Healthy South Texas Asthma Program on improving health outcomes and reducing health disparities among the underserved Hispanic population: using the RE-AIM model. BMC Pediatrics. 2021. Nov 16;21(1):510. doi: 10.1186/s12887-021-02991-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Hughes DL, Flight I, Chapman J, Wilson C. Can we address cancer disparities in immigrants by improving cancer literacy through English as a second language instruction? Translational Behavioral Medicine. 2019. Mar 1;9(2):357–67. doi: 10.1093/tbm/iby030 [DOI] [PubMed] [Google Scholar]
  • 34.Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Research in Psychology. 2006. Jan;3(2):77–101. [Google Scholar]
  • 35.Strauss AL, Corbin JM. Basics of qualitative research: grounded theory procedures and techniques. Newbury Park, Calif.: Sage Publications; 1990. [Google Scholar]
  • 36.Neale J. Iterative categorization (IC): a systematic technique for analysing qualitative data. Addiction. 2016. Jun;111(6):1096–106. doi: 10.1111/add.13314 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Bronfenbrenner U. The Ecology of Human Development: Experiments by Nature and Design. Harvard University Press; 1979. 352 p. [Google Scholar]
  • 38.National Guidelines for Behavioral Health Crisis Care–A Best Practice Toolkit [Internet]. Substance Abuse and Mental Health Services Administration; 2020. Available from: https://www.samhsa.gov/sites/default/files/national-guidelines-for-behavioral-health-crisis-care-02242020.pdf [Google Scholar]

Decision Letter 0

De-Chih Lee

17 Jul 2023

PONE-D-23-12980Impact of San Francisco’s new Street Crisis Response Team on service use among people experiencing homelessness with mental and substance use disorders: A mixed methods study protocolPLOS ONE

Dear Dr. Goldman,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Aug 27 2023 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

De-Chih Lee, Ph.D.

Academic Editor

PLOS ONE

Journal Requirements

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. You indicated that you had ethical approval for your study. In your Methods section, please ensure you have also stated whether you obtained consent from parents or guardians of the minors included in the study or whether the research ethics committee or IRB specifically waived the need for their consent.

3. Thank you for stating the following financial disclosure:

“This work was supported by the Robert Wood Johnson Foundation’s Health Systems Transformation Research Coordinating Center Call for Proposals: Research to

Advance Models of Care for Medicaid-Eligible Populations (Grant #78236).”

Please state what role the funders took in the study.  If the funders had no role, please state: "The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript."

If this statement is not correct you must amend it as needed.

Please include this amended Role of Funder statement in your cover letter; we will change the online submission form on your behalf.

4. Thank you for stating the following in the Competing Interests section:

“Dr. Goldman is a paid research consultant for Vibrant Emotional Health, the National Council for Mental Wellbeing, Peg’s Foundation, the University of California, Davis, and the Research Foundation for Mental Hygiene, Inc.”

Please confirm that this does not alter your adherence to all PLOS ONE policies on sharing data and materials, by including the following statement: "This does not alter our adherence to  PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests).  If there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared.

Please include your updated Competing Interests statement in your cover letter; we will change the online submission form on your behalf.

5. PLOS requires an ORCID iD for the corresponding author in Editorial Manager on papers submitted after December 6th, 2016. Please ensure that you have an ORCID iD and that it is validated in Editorial Manager. To do this, go to ‘Update my Information’ (in the upper left-hand corner of the main menu), and click on the Fetch/Validate link next to the ORCID field. This will take you to the ORCID site and allow you to create a new iD or authenticate a pre-existing iD in Editorial Manager. Please see the following video for instructions on linking an ORCID iD to your Editorial Manager account: https://www.youtube.com/watch?v=_xcclfuvtxQ

6. Please upload a new copy of Figure 1 as the detail is not clear. Please follow the link for more information: https://blogs.plos.org/plos/2019/06/looking-good-tips-for-creating-your-plos-figures-graphics/

7. We note that Figure 1 in your submission contain map images which may be copyrighted. All PLOS content is published under the Creative Commons Attribution License (CC BY 4.0), which means that the manuscript, images, and Supporting Information files will be freely available online, and any third party is permitted to access, download, copy, distribute, and use these materials in any way, even commercially, with proper attribution. For these reasons, we cannot publish previously copyrighted maps or satellite images created using proprietary data, such as Google software (Google Maps, Street View, and Earth). For more information, see our copyright guidelines: http://journals.plos.org/plosone/s/licenses-and-copyright.

We require you to either (1) present written permission from the copyright holder to publish these figures specifically under the CC BY 4.0 license, or (2) remove the figures from your submission:

 a. You may seek permission from the original copyright holder of Figure 1 to publish the content specifically under the CC BY 4.0 license. 

We recommend that you contact the original copyright holder with the Content Permission Form (http://journals.plos.org/plosone/s/file?id=7c09/content-permission-form.pdf) and the following text:

“I request permission for the open-access journal PLOS ONE to publish XXX under the Creative Commons Attribution License (CCAL) CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). Please be aware that this license allows unrestricted use and distribution, even commercially, by third parties. Please reply and provide explicit written permission to publish XXX under a CC BY license and complete the attached form.”

Please upload the completed Content Permission Form or other proof of granted permissions as an "Other" file with your submission.

In the figure caption of the copyrighted figure, please include the following text: “Reprinted from [ref] under a CC BY license, with permission from [name of publisher], original copyright [original copyright year].”

 b. If you are unable to obtain permission from the original copyright holder to publish these figures under the CC BY 4.0 license or if the copyright holder’s requirements are incompatible with the CC BY 4.0 license, please either i) remove the figure or ii) supply a replacement figure that complies with the CC BY 4.0 license. Please check copyright information on all replacement figures and update the figure caption with source information. If applicable, please specify in the figure caption text when a figure is similar but not identical to the original image and is therefore for illustrative purposes only.

The following resources for replacing copyrighted map figures may be helpful:

USGS National Map Viewer (public domain): http://viewer.nationalmap.gov/viewer/

The Gateway to Astronaut Photography of Earth (public domain): http://eol.jsc.nasa.gov/sseop/clickmap/

Maps at the CIA (public domain): https://www.cia.gov/library/publications/the-world-factbook/index.html and https://www.cia.gov/library/publications/cia-maps-publications/index.html

NASA Earth Observatory (public domain): http://earthobservatory.nasa.gov/

Landsat: http://landsat.visibleearth.nasa.gov/

USGS EROS (Earth Resources Observatory and Science (EROS) Center) (public domain): http://eros.usgs.gov/#

Natural Earth (public domain): http://www.naturalearthdata.com/

8. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Additional Editor Comments:

Please have the author reply and make minor revisions according to the three reviewers' comments.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Does the manuscript provide a valid rationale for the proposed study, with clearly identified and justified research questions?

The research question outlined is expected to address a valid academic problem or topic and contribute to the base of knowledge in the field.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

********** 

2. Is the protocol technically sound and planned in a manner that will lead to a meaningful outcome and allow testing the stated hypotheses?

The manuscript should describe the methods in sufficient detail to prevent undisclosed flexibility in the experimental procedure or analysis pipeline, including sufficient outcome-neutral conditions (e.g. necessary controls, absence of floor or ceiling effects) to test the proposed hypotheses and a statistical power analysis where applicable. As there may be aspects of the methodology and analysis which can only be refined once the work is undertaken, authors should outline potential assumptions and explicitly describe what aspects of the proposed analyses, if any, are exploratory.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Partly

********** 

3. Is the methodology feasible and described in sufficient detail to allow the work to be replicable?

Descriptions of methods and materials in the protocol should be reported in sufficient detail for another researcher to reproduce all experiments and analyses. The protocol should describe the appropriate controls, sample size calculations, and replication needed to ensure that the data are robust and reproducible.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: No

********** 

4. Have the authors described where all data underlying the findings will be made available when the study is complete?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception, at the time of publication. The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

Reviewer #3: Yes

********** 

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

********** 

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above and, if applicable, provide comments about issues authors must address before this protocol can be accepted for publication. You may also include additional comments for the author, including concerns about research or publication ethics.

You may also provide optional suggestions and comments to authors that they might find helpful in planning their study.

(Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The protocol described represents an important step in establishing best practices for mobile crisis units and their evaluation.

The authors propose to use mixed methods (quantitative and qualitative approaches) to examine a recently-established mobile crisis unit in San Francisco, California.

Using an interrupted time series analysis, they will measure four main outcomes after an index crisis episode: routine care utilization and housing services assessment (outcomes consistent with stabilization) and crisis care reutilization and jail entry (consistent with de-stabilization). They will compare these outcomes before and after the rollout of the SCRT mobile crisis service in San Francisco, including a pilot period in which the service was rolled out in only some zip codes, providing both time and place-related comparisons. They have appropriately described ways to control for secular patterns in the data.

Using semi-structured interviews, they will obtain qualitative data regarding recipients’ experience with crisis services in the 7 to 90 days following a crisis episode.

The procedures to integrate data from multiple sources are adequately described and seem feasible.

Specific Suggestions:

One potential obstacle to the hoped-for improvements to come from the mobile crisis service and referral plan is a lack of available service providers. For instance, a person could go through the housing assessment but still experience a re-utilization of crisis services because there were not enough housing units, they remained unhoused and a crisis once again ensued on the street. The authors should provide some way of evaluating the availability of such services, perhaps in the qualitative analysis. At the very least, they should comment on it.

Second, in the ITS section (page 12-13), please clarify how the “total target population” will be calculated. Also, how is it estimated that there will be between 8-12 data points per time period?

Third, in the description of the semi-structured interviews, it is unclear whether these have already been performed and remain to be analyzed, or if they will be performed on subjects whose usage of the crisis services does not overlap with the time frame of the qualitative analysis (2020-2022). Please clarify.

Similarly, if the interviews have not yet been performed, will the authors be able to obtain service utilization records for the interview participants as described on page 16, lines 358-359, along with appropriate consent?

Fourth, Figure 1 was very difficult to see, both in the PDF version and when printed.

Finally, please add a statement describing how data will (or will not) be made available after the study is complete.

Reviewer #2: First let me congratulate you on your ambitious and significant study. I enjoyed reading the protocol and think that the methods you are employing could be used in a number of studies to evaluate implementation of real-world interventions that cannot be easily randomized. A few minor comments.

It might be useful to include some hypotheses so that readers can better understand how you expect the outcomes to change. For example, I am assuming that visits to housing services or to primary care would be seen as a positive outcome, while no change or an increase in arrest would be seen negatively.

In describing the equity analysis you write, "Furthermore, we will be able to identify if potential baseline disparities are perpetuated or reduced by implementation of the SCRT, as has been described in the RE-AIM model." Since you have not really described the RE-AIM framework in the paper, it would be good to briefly describe what changes in equity you expect to see (e.g., reach, adoption).

How will eligibility be determined for potential qualitative participants who self-recruit by responding to a flyer. Will you check to see if they have received SCRT services?

When describing methods for matching data you switch unexpectedly to the past tense. Since, I assume that the matching has not yet happened, it would seem better to use the future tense with this as well. In the discussion, "The electronic records used to describe health, housing and jail service utilization does not capture all services provided..." I believe it should be do not capture.

Reviewer #3: This is a significant and worthy study of a critical topic. The QUANT-QUAL implementation methods are appropriate for examining this program launch. Two weaknesses that are correctable:

1. There is a serious imbalance between the detailed quantitative methodology and the sketchy outline provided for the qualitative methodology. In a QUANT-QUAL design, both must be addressed with rigor and (this is missing) a precise plan for staging the merging or integration of the two methods. The protocol falls apart on this dimension. I recommend that the authors consult the NIH guidelines for combining QUANT-QUAL:

https://obssr.od.nih.gov/research-resources/mixed-methods-research

Another helpful resource is:

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7756351/

Both resources describe a clearly designed plan for bringing data from both approaches together and note that a precise plan about when and how should be described. The second resource describes the basic qualifications for rigor in qualitative methods which are missing from the protocol.

2. While the quantitative analytic design is well-described, the equity analysis suffers from inattention to intersectionality and cumulative stress of multiple social identities. The design did not clearly identify how singular versus multiple identities would be accounted for, i.e. the differences between inequities experienced by a white male with serious mental illness symptoms and a black female with similar symptoms is not addressed. The analysis should fully represent these intersecting factors -- it appears that they are treated as discreet factors, not cumulative.

********** 

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Nicole L. Schramm-Sapyta

Reviewer #2: No

Reviewer #3: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2023 Dec 5;18(12):e0295178. doi: 10.1371/journal.pone.0295178.r002

Author response to Decision Letter 0

1 Nov 2023

(10/15/23) Dear Dr. De-Chih Lee,

Thank you for the opportunity to submit a revision of this Study Protocol, titled, " Impact of San Francisco’s new Street Crisis Response Team on service use among people experiencing homelessness with mental and substance use disorders: A mixed methods study protocol." Please see below for our responses to the comments about journal requirements and from the reviewers.

Journal Requirements:

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

>>RESPONSE: Thank you for giving us the opportunity to correct the manuscript to adhere to PLOS ONE’s style requirements. We have revised the manuscript formatting so that it complies with the journal’s style requirements.

2. You indicated that you had ethical approval for your study. In your Methods section, please ensure you have also stated whether you obtained consent from parents or guardians of the minors included in the study or whether the research ethics committee or IRB specifically waived the need for their consent.

>>RESPONSE: Thank you for your attention to ethical research among minors. On Page 8 of our Methods section, under the subsection “Study Population,” we state that “the study population will be defined based on age greater than 18 years.” We therefore did not need to obtain consent from parents or guardians and the IRB did not need to waive consent because we excluded minors from our study population.

3. Thank you for stating the following financial disclosure:

“This work was supported by the Robert Wood Johnson Foundation’s Health Systems Transformation Research Coordinating Center Call for Proposals: Research to

Advance Models of Care for Medicaid-Eligible Populations (Grant #78236).”

Please state what role the funders took in the study. If the funders had no role, please state: "The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript."

If this statement is not correct you must amend it as needed.

Please include this amended Role of Funder statement in your cover letter; we will change the online submission form on your behalf.

>>RESPONSE: Thank you for bringing this to our attention. We have updated the funding statement in our manuscript and cover letter to include the statement that you provided, which accurately describes the funder’s role in our study: ”The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.”

4. Thank you for stating the following in the Competing Interests section:

“Dr. Goldman is a paid research consultant for Vibrant Emotional Health, the National Council for Mental Wellbeing, Peg’s Foundation, the University of California, Davis, and the Research Foundation for Mental Hygiene, Inc.”

Please confirm that this does not alter your adherence to all PLOS ONE policies on sharing data and materials, by including the following statement: "This does not alter our adherence to PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests). If there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared.

Please include your updated Competing Interests statement in your cover letter; we will change the online submission form on your behalf.

>>RESPONSE: We reviewed PLOS ONE policies on sharing data and materials and confirmed that there is no impact. Based on this information, we have added the following statement to the manuscript and cover letter: "This does not alter our adherence to PLOS ONE policies on sharing data and materials.”

5. PLOS requires an ORCID iD for the corresponding author in Editorial Manager on papers submitted after December 6th, 2016. Please ensure that you have an ORCID iD and that itis validated in Editorial Manager. To do this, go to ‘Update my Information’ (in the upper left-hand corner of the main menu), and click on the Fetch/Validate link next to the ORCID field. This will take you to the ORCID site and allow you to create a new iD or authenticate a pre-existing iD in Editorial Manager. Please see the following video for instructions on linking an ORCID iD to your Editorial Manager account: https://www.youtube.com/watch?v=_xcclfuvtxQ

>>RESPONSE: The ORCID ID for the corresponding author, Matthew L. Goldman, is: 0000-0002-2252-9285

6. Please upload a new copy of Figure 1 as the detail is not clear. Please follow the link for more information: https://blogs.plos.org/plos/2019/06/looking-good-tips-for-creating-your-plos-figures-graphics/

>>RESPONSE: We have recreated Figure 1 to improve clarity and to abide by the guidance in comment 7 regarding using an uncopyrighted map image (USGS National Map Viewer).

7. We note that Figure 1 in your submission contain map images which may be copyrighted. All PLOS content is published under the Creative Commons Attribution License (CC BY 4.0), which means that the manuscript, images, and Supporting Information files will be freely available online, and any third party is permitted to access, download, copy, distribute, and use these materials in any way, even commercially, with proper attribution. For these reasons, we cannot publish previously copyrighted maps or satellite images created using proprietary data, such as Google software (Google Maps, StreetView, and Earth). For more information, see our copyright guidelines:http://journals.plos.org/plosone/s/licenses-and-copyright.

We require you to either (1) present written permission from the copyright holder to publish these figures specifically under the CC BY 4.0 license, or (2) remove the figures from your submission:

a. You may seek permission from the original copyright holder of Figure 1 to publish the content specifically under the CC BY 4.0 license.

We recommend that you contact the original copyright holder with the Content Permission Form (http://journals.plos.org/plosone/s/file?id=7c09/content-permission-form.pdf) and the following text:

“I request permission for the open-access journal PLOS ONE to publish XXX under the Creative Commons Attribution License (CCAL) CC BY 4.0(http://creativecommons.org/licenses/by/4.0/). Please be aware that this license allows unrestricted use and distribution, even commercially, by third parties. Please reply and provide explicit written permission to publish XXX under a CC BY license and complete the attached form.”

Please upload the completed Content Permission Form or other proof of granted permissions as an "Other" file with your submission.

In the figure caption of the copyrighted figure, please include the following text: “Reprinted from [ref] under a CC BY license, with permission from [name of publisher], original copyright [original copyright year].”

b. If you are unable to obtain permission from the original copyright holder to publish these figures under the CC BY 4.0 license or if the copyright holder’s requirements are incompatible with the CC BY 4.0 license, please either i) remove the figure or ii) supply a replacement figure that complies with the CC BY 4.0 license. Please check copyright information on all replacement figures and update the figure caption with source information. If applicable, please specify in the figure caption text when a figure is similar but not identical to the original image and is therefore for illustrative purposes only.

The following resources for replacing copyrighted map figures may be helpful:

USGS National Map Viewer (public domain): http://viewer.nationalmap.gov/viewer/

The Gateway to Astronaut Photography of Earth (public domain): http://eol.jsc.nasa.gov/sseop/clickmap/

Maps at the CIA (public domain): https://www.cia.gov/library/publications/the-world-factbook/index.html and https://www.cia.gov/library/publications/cia-maps-publications/index.html

NASA Earth Observatory (public domain): http://earthobservatory.nasa.gov/

Landsat: http://landsat.visibleearth.nasa.gov/

USGS EROS (Earth Resources Observatory and Science (EROS) Center) (public domain):http://eros.usgs.gov/#

Natural Earth (public domain): http://www.naturalearthdata.com/

>>RESPONSE: We have provided a new version of the map image in figure 1 using the USGS National Map Viewer that does not require copyright.

8. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

>>RESPONSE: We have reviewed our reference list and confirmed that it is complete and correct, and no retracted papers were included in the list.

Reviewer #1

9. The protocol described represents an important step in establishing best practices for mobile crisis units and their evaluation. The authors propose to use mixed methods (quantitative and qualitative approaches) to examine a recently-established mobile crisis unit in San Francisco, California. Using an interrupted time series analysis, they will measure four main outcomes after an index crisis episode: routine care utilization and housing services assessment (outcomes consistent with stabilization) and crisis care reutilization and jail entry (consistent with de-stabilization). They will compare these outcomes before and after the rollout of the SCRT mobile crisis service in San Francisco, including a pilot period in which the service was rolled out in only some zip codes, providing both time and place-related comparisons. They have appropriately described ways to control for secular patterns in the data. Using semi-structured interviews, they will obtain qualitative data regarding recipients’ experience with crisis services in the 7 to 90 days following a crisis episode. The procedures to integrate data from multiple sources are adequately described and seem feasible. Specific Suggestions:

>>RESPONSE: Thank you for your review of the manuscript.

10. (1) One potential obstacle to the hoped-for improvements to come from the mobile crisis service and referral plan is a lack of available service providers. For instance, a person could go through the housing assessment but still experience a re-utilization of crisis services because there were not enough housing units, they remained unhoused and a crisis once again ensued on the street. The authors should provide some way of evaluating the availability of such services, perhaps in the qualitative analysis. At the very least, they should comment on it.

>>RESPONSE: Thank you for this comment about the importance of capturing the lack of available service providers. The qualitative interviews will ask about post-crisis service linkage, which may help inform their experience of difficulty accessing services that were lacking—we have added a clause to the manuscript in the subsection on “Semi-Structured Interviews” that now says, “The 30- to 60-minute interview will include specific questions about baseline engagement in health care and housing systems, SCRT’s accessibility, SCRT intervention and assessment, post-crisis linkage to care and difficulties accessing services due to lack of available resources, and overall client experience.” With regard to the quantitative analysis, we would not have data on provider or service availability outside of the service utilization data, so we added the following sentence to the Limitations section of the Discussion: “Additionally, analyses using electronic health records do not capture the availability of services at the time of post-crisis linkage and are unable to determine whether post-crisis outcomes were impacted by service systems capacity.”

11. (2) Second, in the ITS section (page 12-13), please clarify how the “total target population” will be calculated.

>>RESPONSE: The total target population refers to the inclusion criteria described in the “Study Population” section: “Therefore, the study population will be defined based on age greater than 18 years, meeting criteria for homelessness in the 12 months prior to or 3 months following the crisis episode, and receipt of crisis care from any of San Francisco’s “front door” programs for people in an acute behavioral health crisis. These settings include two mobile crisis programs (Comprehensive Crisis Services and the SCRT), a crisis stabilization unit (DORE Urgent Care Clinic), and emergency psychiatric services (Zuckerberg San Francisco General Hospital’s Psychiatric Emergency Services [PES] and Emergency Department visits with a primary behavioral health diagnosis) (Fig 2).” To clarify the description of the ITS methods, we have edited the phrase total target population to instead say, “the denominator equaling the total study population in a given month as described above.”

12. Also, how is it estimated that there will be between 8-12 data points per time period?

>>RESPONSE: Preliminary estimates of the study population, based on historic utilization data from select clinical settings, suggest that there are on the order of a few hundred monthly encounters in the target population. Our statistical analysis suggests that this size population should be powered sufficiently to use month-long time intervals for the interrupted time series, and based on the duration of the three time periods (13 months of Pre-SCRT baseline, 8 months of SCRT partial implementation, and 13 months of SCRT full implementation), this would yield between 8 and 12 data points per time period, which is usually adequate for an ITS analysis. To clarify this in the text, we have added a phrase to the sentence that this is based on preliminary data: “Using month-long time intervals would yield between 8 and 12 data points per time period based on preliminary estimates of the study population size, though the final interval length may change depending on the trade-offs between length of observation and statistical power.”

13. (3) Third, in the description of the semi-structured interviews, it is unclear whether these have already been performed and remain to be analyzed, or if they will be performed on subjects whose usage of the crisis services does not overlap with the time frame of the qualitative analysis (2020-2022). Please clarify. Similarly, if the interviews have not yet been performed, will the authors be able to obtain service utilization records for the interview participants as described on page 16, lines 358-359, along with appropriate consent?

>>RESPONSE: At this point, all interviews have been completed and are in the process of being analyzed, but we would like to keep the voice of this study protocol in the future tense for the sake of consistency and to be referenced in future studies that may draw on these methods. We have therefore opted not to change the language in the methods accordingly but welcome feedback from the journal if we should adjust this approach. With regard to when the interviews were conducted, they were all completed with individuals whose crisis service usage does overlap with the time frame of the quantitative analysis, so the text in the manuscript describing eligibility requirements for the interviews remains accurate: “Potential participants will be eligible if they received SCRT services 7 to 90 days prior to contact with the study team.”

14. (4) Fourth, Figure 1 was very difficult to see, both in the PDF version and when printed.

>>RESPONSE: We have recreated Figure 1 to improve clarity and to abide by the journal’s guidance regarding using an uncopyrighted map image.

15. (5) Finally, please add a statement describing how data will (or will not) be made available after the study is complete.

>>RESPONSE: Data acquired for the proposed study will be available upon request. We added the following statement to the Data Availability in the manuscript: “Datasets used and analyzed for the proposed study will be available after study completion from the corresponding author on reasonable request.”

Reviewer #2

16. First let me congratulate you on your ambitious and significant study. I enjoyed reading the protocol and think that the methods you are employing could be used in a number of studies to evaluate implementation of real-world interventions that cannot be easily randomized. A few minor comments.

>>RESPONSE: Thank you for your review and for the opportunity to respond to your comments.

17. (1) It might be useful to include some hypotheses so that readers can better understand how you expect the outcomes to change. For example, I am assuming that visits to housing services or to primary care would be seen as a positive outcome, while no change or an increase in arrest would be seen negatively.

>>RESPONSE: Thank you for this excellent suggestion. We have added the following description of our main hypotheses to the section titled “Interrupted Time Series Analysis”:

These methods will test the following hypotheses:

1. Monthly rates of post-crisis episode routine care utilization within 30 days of an index crisis episode will increase following SCRT implementation.

2. Monthly rates of post-crisis episode crisis service reutilization within 30 days of an index crisis episode will decrease following SCRT implementation.

3. Monthly rates of post-crisis episode housing assessment within 30 days of an index crisis episode will increase following SCRT implementation.

4. Monthly rates of post-crisis episode jail entry within 30 days of an index crisis episode will decrease following SCRT implementation.

18. (2) In describing the equity analysis you write, "Furthermore, we will be able to identify if potential baseline disparities are perpetuated or reduced by implementation of the SCRT, as has been described in the RE-AIM model." Since you have not really described the RE-AIM framework in the paper, it would be good to briefly describe what changes in equity you expect to see (e.g., reach, adoption).

>>RESPONSE: Thank you for catching this. The RE-AIM framework is not essential to this explanation of our disparities analysis, so we have decided to remove the confusing phrase “as has been described in the RE-AIM model” rather than add potentially confusing or extraneous details about that model.

19. (3) How will eligibility be determined for potential qualitative participants who self-recruit by responding to a flyer. Will you check to see if they have received SCRT services?

>>RESPONSE: Yes, we will confirm that all potential interview participants who self-recruit meet eligibility criteria. We have added the following statement to clarify this: “For those who self-recruit by contacting the study team, we will obtain verbal consent to access electronic health records and the study team will confirm eligibility for the qualitative interviews (i.e., receipt of SCRT services in the previous 7 to 90 days).”

20. (4) When describing methods for matching data you switch unexpectedly to the past tense. Since, I assume that the matching has not yet happened, it would seem better to use the future tense with this as well. In the discussion, "The electronic records used to describe health, housing and jail service utilization does not capture all services provided..." I believe it should be do not capture.

>>RESPONSE: Thank you for bringing this to our attention. We have reviewed and updated these sections to future tense.

Reviewer #3

21. This is a significant and worthy study of a critical topic. The QUANT-QUAL implementation methods are appropriate for examining this program launch. Two weaknesses that are correctable:

>>RESPONSE: Thank you for reviewing our manuscript and for the opportunity to respond to your comments.

22. (1) There is a serious imbalance between the detailed quantitative methodology and the sketchy outline provided for the qualitative methodology. In a QUANT-QUAL design, both must be addressed with rigor and (this is missing) a precise plan for staging the merging or integration of the two methods. The protocol falls apart on this dimension. I recommend that the authors consult the NIH guidelines for combining QUANT-QUAL:

https://obssr.od.nih.gov/research-resources/mixed-methods-research

Another helpful resource is:

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7756351/

Both resources describe a clearly designed plan for bringing data from both approaches together and note that a precise plan about when and how should be described. The second resource describes the basic qualifications for rigor in qualitative methods which are missing from the protocol.

>>RESPONSE: Thank you for this comment requesting further detail about our qualitative methods as well as the QUANT-QUAL approach. We have added additional detail in the “Qualitative Design” section, including:

For the “Interview Participant Recruitment and Consent” subsection:

“For those who self-recruit by contacting the study team, we will obtain verbal consent to access electronic health records and the study team will confirm eligibility for the qualitative interviews (i.e., receipt of SCRT services in the previous 7 to 90 days).”

“Finally, we will obtain demographic and service use information from electronic health records for all respondents.”

For the “Semi-structured Interviews” subsection:

“The 30- to 60-minute interview will include specific questions about baseline engagement in health care and housing systems, SCRT’s accessibility, SCRT intervention and assessment, post-crisis linkage to care and difficulties accessing services due to lack of available resources, and overall client experience.”

“Subsequent iterations of the semi-structured interview guide will be revised periodically during data collection by incorporating emerging themes from early responses. This periodic revision of the interview guide will include sharing the guide with subject matter experts, community stakeholders, and people with lived experience of receiving crisis services to ensure the questions capture important themes for this population. Finally, we will develop an interview team that is diverse in terms of cultural and educational backgrounds to foster reflexivity during data collection.”

For the “Qualitative Coding and Analysis” subsection:

“The codebook will be modified using an inductive process to capture more detailed descriptions of the data. At least two researchers will code each transcript and discuss salient themes based on Braun and Clarke’s “theoretical” thematic analysis.”

“Once the study team codes all transcripts, one of the two original coders will use the final codebook iteration to review their previously coded transcripts to ensure all potentially relevant themes are captured and will resolved any remaining discrepancies.”

Regarding QUANT-QUAL methods, the following statements have been added to the “Mixed Method Analysis” section:

“Guided by Creswell and Plano Clark’s best practices for mixed methods research, we will merge the two types of analyses to mitigate the limitations of the different data types while also highlighting their respective strengths [38]. The ITS analysis is expected to show high-level trends in service usage for SCRT clients but lacks to show any underlying explanation for these trends. On the other hand, the qualitative analysis is expected to show the detailed experiences of an SCRT client, but this information is collected from a subset of SCRT’s clientele and will only generalize to certain circumstances. Any system-level impact of SCRT on service usage among crisis service users will benefit from individual accounts of the program’s implementation and provide a more complete understanding of this crisis service model.

The variety of quantitative and qualitative data that will be collected allows several opportunities to merge the two analyses for better understanding the impact of the SCRT intervention.”

23. (2) While the quantitative analytic design is well-described, the equity analysis suffers from inattention to intersectionality and cumulative stress of multiple social identities. The design did not clearly identify how singular versus multiple identities would be accounted for, i.e. the differences between inequities experienced by a white male with serious mental illness symptoms and a black female with similar symptoms is not addressed. The analysis should fully represent these intersecting factors -- it appears that they are treated as discreet factors, not cumulative.

>>RESPONSE: Thank you for this excellent comment. We agree that it is important to assess for intersectionality in disparities and have added to the following sentence to the Equity Analysis section: “Additional regression analyses using both the stratified and overall samples will use interaction terms to assess for disparities related to intersectionality between other demographic characteristics with race and ethnicity.”

We appreciate your time and consideration.

Sincerely,

Matthew L. Goldman, M.D., M.S., on behalf of the co-authors

Attachment

Submitted filename: Response to Reviewers.docx

Click here for additional data file. (33.8KB, docx)

Decision Letter 1

De-Chih Lee

17 Nov 2023

Impact of San Francisco’s new Street Crisis Response Team on service use among people experiencing homelessness with mental and substance use disorders: A mixed methods study protocol

PONE-D-23-12980R1

Dear Dr. Goldman,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

De-Chih Lee, Ph.D.

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Does the manuscript provide a valid rationale for the proposed study, with clearly identified and justified research questions?

The research question outlined is expected to address a valid academic problem or topic and contribute to the base of knowledge in the field.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

2. Is the protocol technically sound and planned in a manner that will lead to a meaningful outcome and allow testing the stated hypotheses?

The manuscript should describe the methods in sufficient detail to prevent undisclosed flexibility in the experimental procedure or analysis pipeline, including sufficient outcome-neutral conditions (e.g. necessary controls, absence of floor or ceiling effects) to test the proposed hypotheses and a statistical power analysis where applicable. As there may be aspects of the methodology and analysis which can only be refined once the work is undertaken, authors should outline potential assumptions and explicitly describe what aspects of the proposed analyses, if any, are exploratory.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

3. Is the methodology feasible and described in sufficient detail to allow the work to be replicable?

Descriptions of methods and materials in the protocol should be reported in sufficient detail for another researcher to reproduce all experiments and analyses. The protocol should describe the appropriate controls, sample size calculations, and replication needed to ensure that the data are robust and reproducible.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

4. Have the authors described where all data underlying the findings will be made available when the study is complete?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception, at the time of publication. The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above and, if applicable, provide comments about issues authors must address before this protocol can be accepted for publication. You may also include additional comments for the author, including concerns about research or publication ethics.

You may also provide optional suggestions and comments to authors that they might find helpful in planning their study.

(Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: My only comment remaining is about the use of future tense voice. It seems appropriate to use past-tense when referring to recruitment of subjects for the qualitative analysis (subjects were recruited; subjects were interviewed) and future tense when referring to the analysis (transcripts will be analyzed, etc.). However, i defer to the editor on this matter and would not delay the publication of this important plan based on that issue.

This is very important work and I look forward to the results!

Reviewer #2: Your version with untracked changes does not incorporate the changes found in the tracked changes version. I don't know if the original was included with the resubmission, but it caused some confusion. The second version with tracked changes has the changes described in the authors' response.

Reviewer #3: The authors have been responsive to the reviewers' and editors' requests. Refreshing! Thank you! The responses are thoughtful and have received full responses.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Nicole L. Schramm-Sapyta

Reviewer #2: No

Reviewer #3: Yes: Linda S. Beeber

**********

Acceptance letter

De-Chih Lee

24 Nov 2023

PONE-D-23-12980R1

Impact of San Francisco’s New Street Crisis Response Team on Service Use Among People Experiencing Homelessness with Mental and Substance Use Disorders: A Mixed Methods Study Protocol

Dear Dr. Goldman:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at customercare@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. De-Chih Lee

Academic Editor

PLOS ONE

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Supplementary Materials

    S1 Checklist

    (DOCX)

    Click here for additional data file. (48.2KB, docx)
    Attachment

    Submitted filename: Response to Reviewers.docx

    Click here for additional data file. (33.8KB, docx)

    Data Availability Statement

    This manuscript is for a study protocol in which we don't have data or results to potentially share with a third party.

    Articles from PLOS ONE are provided here courtesy of PLOS

    RESOURCES