Transforming first response through non-police, community safety response programmes: a peer-reviewed and grey literature scoping review protocol

Abstract

Introduction

Police are frequently dispatched to a wide range of 911 calls, including mental and behavioural health crises, despite lacking the training, resources and time to respond effectively. In particular, people with serious mental illness are at elevated risk of experiencing excessive use of force, arrest and continued criminal legal involvement following police contact. Following the murder of George Floyd and other highly publicised police killings, Community Safety Response (CSR) programmes, staffed by unarmed peers, mental health professionals and other trained responders, have proliferated to provide non-police responses to mental and behavioural health and other quality-of-life concerns. CSR programmes have expanded rapidly, yet the evidence base remains fragmented and largely outside the peer-reviewed literature.

Methods and analysis

This scoping review will synthesise peer-reviewed and grey literature from 2020 to present on CSR programmes operating in North America. Guided by Joanna Briggs Institute methodology and reported according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews (PRISMA-ScR) standards, we will search multiple databases (Medline, PsycINFO, Embase, SocIndex, Web of Science, Policy Commons) and employ complementary grey literature search strategies, including targeted website searches, reference tracking and review of internal and external reports and evaluations. Inclusion criteria require that programmes provide non-police first response to calls traditionally served by law enforcement and include information on programme operations or outcomes. Two reviewers will independently screen and extract data on process metrics including operational characteristics, dispatch, funding, services provided and outcomes such as populations served, diversion from police, service linkage and use of force.

Ethics and dissemination

No ethical review for this study is required as it will not include human subjects or any identifiable information. Findings will provide the first national synthesis of CSR programme models, operations and outcomes. Results will inform policy-makers, practitioners, researchers and community members. Findings will be disseminated through peer-reviewed publications and public-facing products to support implementation, scale-up and sustainability of CSR programmes.

STRENGTHS AND LIMITATIONS OF THIS STUDY.

• Due to the frequent emergence of Community Safety Response (CSR) programmes and models, as well as a lack of peer-reviewed literature, exploring grey literature will allow us to best understand what is known about CSR programmes across North America since 2020.

• The multistep search strategy and incorporation of an Advisory Board will allow for the most rigorous and exhaustive inclusion of sources and valuable information for the field from data extraction.

• We will follow strongly established scoping review methodology (Joanna Briggs Institute) and gold standard reporting guidelines (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews).

• Given that research on CSR programmes is still emerging, we expect a substantial portion of data to come from grey literature. Consequently, our capacity to systematically evaluate the methodological rigour and quality of these sources may be limited.

Introduction

Every year in the USA alone, about 240 million calls are made to 911 for help with a range of issues, from crime and violence to behavioural health crises and interpersonal disputes.¹ In many communities, police are sent by default. But police officers often lack the training, time and resources to handle these situations effectively. People with mental illness and those experiencing behavioural health crises are among the most vulnerable to the negative effects of using police as default responders.2,4 Mental and behavioural health has been treated as a criminal legal issue, rather than a public health one, resulting in people with serious mental illness (SMI) being overrepresented in police interactions and in incarceration compared to those without mental illness.^{5 6} The high rate of police involvement among people with mental and behavioural health needs has led to a number of disparate outcomes, including inequitable enforcement and arrest, continued police contact, high rates of use of force and even death: people with mental illness are over seven times more likely to be killed by the police than those without.⁴ In recent years, highly publicised instances of police contact ending in physical harm have highlighted the urgent need for change in how our country treats people with mental and behavioural health needs and prompted a larger conversation about limiting the role of police in routine interactions to reduce possible risks for those who may be most vulnerable to negative policing outcomes.

In the wake of the murder of George Floyd, and other police killings of people experiencing mental and behavioural health crises, there was a public outcry to remove police from these responses. Traditionally, methods of responding to mental health crises in the community involve police officers. The primary models that have been relied on thus far for responding to crisis situations aim to improve how police respond. These are Crisis Intervention Team (CIT) training and co-response programmes. In CIT, officers typically elect to take a 40-hour CIT training programme focused on mental health education, de-escalation techniques and role play exercises. However, even in departments with established CIT programmes, often only a fraction of officers receives the full training and those officers may not always be available to respond to crisis calls.^{7 8} Importantly, although CIT has demonstrated effectiveness in improving officer attitudes and knowledge, findings are limited and mixed in terms of its effectiveness in reducing arrest, use of force and increasing referral to mental health services.^{9 10} Co-response models deploy officers (who are often, but not always, CIT trained) to respond alongside mental health professionals. Limited research has explored the effectiveness of co-response models, with mixed findings. Specifically, research has found a relationship between co-response and higher referral rates to mental health services and less time spent on calls but little to no evidence that they avert arrest or police-induced injury.⁹ In a randomised effectiveness trial, co-response was not more effective than police response on important outcomes, including jail bookings, outpatient encounters or emergency department visits.¹¹ Overall, police-involved responses have shown limited effectiveness at best.

Rationale

The emerging field of Community Safety Response (CSR) reframes crisis intervention by asking not only how mental and behavioural health crises are managed, but who is best equipped to respond. CSR programmes dispatch unarmed mental health and medical professionals, and/or peer specialists instead of the police. Since 2020, CSR programmes have expanded rapidly—now exceeding 135 nationwide¹²—responding to not only mental and behavioural health calls for service, but a broad range of other quality-of-life concerns that do not necessitate police involvement.

The national first response landscape is bustling and highly complex, which makes defining the various modalities of these new response programmes a challenging undertaking. Here, when we reference CSR programmes, we are referring to programmes that have been born out of a desire to reduce reliance on police and thus are dispatched to calls which have traditionally been responded to with a law enforcement response. These programmes provide first response services to various call types without police officers, are often housed within a city department or contracted by the city, and may be dispatched by multiple numbers (sometimes within one programme) including 911, 988 and 311 (or a non-emergency line equivalent). Although most calls handled by CSR programmes involve mental and behavioural health issues, depending on the programme, responders may be trained in skills such as mediation, homeless outreach or traffic crash response.

Across the field, practitioners and researchers use a range of terms to describe this emerging service and profession, including community safety, public safety and alternative responders. A commission led by the Council of State Governments recently moved towards developing a shared understanding in this space and used the term ‘community responder programmes’. The commission defined them, stating: “Community responder programs offer an additional option for first response. Composed of multidisciplinary professionals trained to address behavioral health and quality-of-life concerns, community responder programs provide a person-centered response to 911 and other emergency calls for service. Integrating community responder programs into first response systems ensures that all calls for service can receive the most appropriate response.”¹³

Importantly, early evidence and experience from CSR programmes across the country show they have the potential to change the fabric of the nation’s first-response system. For example, one study found that Denver’s CSR Programme (STAR) was associated with a reduction in arrests for less serious offences by 34% in just 6 months.¹⁴ However, peer-reviewed research has yet to explore the potential health-related outcomes of CSR programmes. Despite this, local governments across the country have partnered with external evaluators or produced their own data dashboards, briefs and reports to begin highlighting programme operations and impact. Programmes in Eugene, OR (CAHOOTS), San Francisco, CA (SCRT) and Durham, NC (HEART) have shared data publicly indicating police backup was requested in less than 1%–3% of their CSR calls, highlighting the safety of these calls and their ability to divert community members away from criminal legal involvement.15,17 Still, the accumulation of this data has been ad hoc, and we still know too little about what makes these programmes work, their impact and their potential for broader adoption.

To increase the health and safety of those struggling with behavioural health needs and other communities at risk of escalated police contact, there is an urgent need to establish an evidence base for CSR programmes and widely communicate these findings. The existing data from the CSR field is local and fragmented, and it is unclear how individual programmes are being implemented and what their outcomes are. Rigorous peer-reviewed literature in this area has been limited and is lagging compared to the pace at which programmes are proliferating.

The current paper outlines the protocol for a scoping review of peer-reviewed and grey literature to assess these gaps. This review is essential for synthesising the current evidence on CSR programmes and examining their reported ability to respond safely and effectively to a range of call types. Although CSR has garnered bipartisan support,¹⁸ doubt remains, and there is a need to mine the existing evidence base to inform how programmes can be best implemented and scaled. The findings of this study will have wide-reaching implications for the sustainability of CSR programmes across the country, identify what gaps research needs to address, and set the stage for rigorous quasi-experimental evaluation proposals and public narrative change.

Aims

Research in CSR has yet to synthesise all of the various rich data sources together to (1) identify and characterise components of the CSR interventions and (2) identify initial outcome metrics for CSR programmes. We seek to conduct a large-scale scoping review of available peer-reviewed and grey CSR programme literature to collect and consolidate data from across North America.

Methods and analysis

The research team will conduct a scoping review of all known and identified CSR programmes (approximately 135) in North America, expanding as programmes are identified or created. The majority of these programmes are collected and continuously updated in the Directory of Alternative Crisis Response Programs.¹² A scoping review will allow us to both summarise and disseminate the current state of CSR research to policy-makers and practitioners, and identify research gaps to inform where areas of evaluation are most essential. We will follow the Joanna Briggs Institute (JBI) method, informed by Arksey & O’Malley’s iterative and reflexive five-stage approach, for conducting scoping reviews.^{19 20} Scoping reviews are a common method of evidence synthesis well-suited for broad research questions to map data in a specific area of interest to identify what type of evidence is available, research gaps and clarify concepts and definitions related to the field in an effort to inform policy, practice and decision-making processes.¹⁹ We will also employ the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews (PRISMA-ScR) Checklist to enhance the rigour and transparency of our scoping review method and reporting quality (see online supplemental file).²¹ We intend to review all relevant publicly available information including peer-reviewed literature, public-facing evaluation reports from local governments, external evaluations and other grey literature. We will also engage in knowledge user co-creation by forming an Advisory Board composed of practitioners and other field experts to ensure this review will be most useful to policymakers and practitioners. We will follow the guidance of the JBI Scoping Review Methodology Group for involving knowledge users in scoping reviews and adapt the ACTIVE framework for systematic reviews as appropriate for our project.²² The Advisory Board will be involved in the development, conduct, reporting and dissemination of the scoping review. At this stage, we have solicited interest in participation in the Advisory Board, already received substantial feedback from knowledge users in the field (see acknowledgements), many of whom will be members of the formal Advisory Board, and incorporated a knowledge user in the scoping review process through training and meeting involvement. Consistent with open science practices, this review has been submitted for registration on the Open Science Framework (OSF): https://osf.io/cbau4.

Review questions

1. What are the models, strategies and operational processes and characteristics of CSR programmes across North America?

2. What is known from the existing data about the outcomes and effectiveness of CSR programmes?

Inclusion and exclusion criteria

To be included in the review, sources must report on a CSR programme. CSR programmes include those listed in the Alternative Crisis Response Directory and any additional programmes identified through searches. Programmes must be first response, respond to calls that would normally be responded to by law enforcement, and should not typically respond alongside law enforcement. They should be housed within or contracted by the city and be dispatched through one or more emergency or non-emergency numbers. Only CSR programmes in North America will be included in this review. Because CSR programmes began to proliferate in 2020, we will only include literature published from 2020 to present. Sources must report at least one component from the data extraction table to be included. Priority will be given to primary sources that are peer-reviewed, are independent evaluations or are local government evaluations. Secondary data sources will only be included when primary data sources are not available and are not duplicative.

Search strategy

We will use an iterative approach to the searches. Initial searches of a broad set of databases will be used to identify the most relevant keywords used in CSR-related publications, the disciplines and organisations that most commonly publish on this topic, and the names of existing programmes, which will be incorporated into later iterations of the strategies.

Our initial set of databases to search is Medline, PsycINFO, Embase, SocIndex, Web of Science Core Collection and Policy Commons. The initial searches in these databases will be broad, using known descriptors for CSR and related programmes. An example strategy for use in the Ovid platform is below:

alternative crisis respon* OR alternative emergency respon* OR alternative first respon* OR alternative response model* OR alternative response program* OR alternative response team* OR alternative response unit* OR crisis response model* OR crisis response program OR crisis response team* OR crisis response unit* OR emergency response model* OR first responder deflection OR mobile crisis model OR mobile crisis program* OR mobile crisis team* OR mobile crisis unit* OR (non police adj respon*) OR community based first respon* OR community crisis respon* OR community first respon* OR community safety response OR community response program* OR community response team* OR community responder program* OR community safety response OR public safety response

In addition to database searches, we will use the following complementary strategies to maximise our ability to effectively search and identify grey literature:

1. Identify journals which have either published on CSR programmes or on topics highly relevant to them (eg, ‘Policing and Society’) and search broadly within the full text for any publications missed in the database searches.

2. Follow-up on all protocols, preprints, meeting abstracts and dissertations identified for complete publications.

3. Search the websites of known organisations and those identified during prior steps, which may include: The Council of State Governments Justice Center, The Vera Institute of Justice, The Fourth Branch Institute, Law Enforcement Action Partnership, Georgetown Center for Innovations in Community Safety, Harvard Kennedy School Government Performance Lab, Alternative Mobile Services Association and others.

4. The names of all programmes identified from initial searches or included in the Directory of Alternative Crisis Response Programs¹² will be searched in Google Scholar, in order to capture sources which discuss programmes in the full text (see online supplemental table 1).

5. Employ search engines to identify additional grey literature. These will include filters to focus the search on the .gov domain, specific websites and/or materials published in the PDF format. Due to the limited number of terms allowed in Google and other tools, a web scraping tool may be used to run multiple searches and collect page titles and URLs in a more efficient way. For example, using a ‘.gov’ filter in Google’s Advanced Search will allow us to identify materials from specific local government run programmes.

6. Perform citing/cited reference searches of included publications and relevant reviews identified in prior steps in Scopus and/or manually.

As previously mentioned, there is low consensus with regard to the terminology used to name and describe CSR programmes. Programmes with very similar names may be in or out of scope for this project (eg, programmes with mobile crisis in the title). In addition, searches will need to be customised based on the different types of materials (research articles, press releases, reports, white papers) and search platforms (academic databases, web search engines, specific websites). Overall, we will seek a balance of sensitivity and specificity in each individual search, while using methods like citing/cited searches that do not rely on search terminology. A set of 15 seed articles and reports that we expect to be included in the review has been collected and will be used to validate the sensitivity of the search strategy as we progress.

Study selection

All identified materials will be compiled in Endnote. Records with abstracts will be uploaded to Covidence Systematic Review Software for title/abstract screening by two independent reviewers. Records without abstracts, which will likely comprise most of the grey literature, will be screened by one reviewer for general applicability and appropriate content type. The full texts of all materials found to be potentially relevant in the first round of screening will be uploaded to Covidence and assessed for eligibility by two independent reviewers. Prior to each step, the screening process will be piloted using 25–50 results to train reviewers in the process and refine the inclusion/exclusion criteria. In all steps, conflicts will be resolved by consensus. Reasons for exclusion at the full-text level will be recorded.

Sources ultimately included in this review are expected to include a wide variety of document types and sources, including peer-reviewed articles, government resources, theses and potentially secondary sources when primary data is not available. The AACODS checklist²³ (Authority, Accuracy, Coverage, Objectivity, Date, Significance) will be used to appraise the grey literature sources included. This tool was developed by Jessica Tyndall at the Flinders University Medical Library for the appraisal of a broad set of grey literature sources. Search strategies, results, the study selection process and appraisal will be reported in full, consistent with PRISMA-ScR reporting guidelines.

Data extraction

Two independent reviewers will extract data from sources using a data extraction tool developed by the research team, informed by JBI recommendations. Through consultation with our Advisory Board, we may expand and change our planned data extraction measures; however, we aim to capture data in the following general areas: operational factors, technology, funding, incident level variables and outcomes informed by initial research of programmes, field experts and Harvard’s Government Performance Lab’s list of essential metrics for alternative emergency response programs.²⁴ See table 1 for the current list of data extraction metrics. We will also extract data about each literature source. A database of extracted information will be developed and stored on NYU Box shared with research personnel.

Table 1. Data extraction tool.

Category	Metric	Question/Instructions to reviewers
Source
	Source classification	Primary source vs risk of bias information
	Source type	Peer-reviewed article, government report, external evaluation, news article, etc
	Publication date	Month, day, year
Process
Operational
	Programme structure	Who runs/houses the programme (eg, city run department, within fire department or other government department, private contract, etc)?
	Launch date	When did the programme start?
	Service availability	What are the days and hours of operation?
	Average response time	What is the programme’s average response time from when the call comes in and when the CSR programme arrives at scene?
	Average response duration	What is the programme’s average amount of time spent once they arrive to an incident?
	Call volume	How many calls in a specific time frame (must specify unit: per month, per year, etc)? Do they indicate % of call volume from each call source? If so, include breakdown.
	Geographic region	Does the programme serve an urban, suburban or rural area?
	Service area coverage	What is the geographic area covered by the programme?
	Number of teams	How many teams does the programme have?
	Number of vans	How many vans does the programme have?
	Programme change	Has the programme changed or expanded over time as it relates to data extraction variables? Please note when programme components change, what the change was (eg, team size, hours of operation), and when the change occurred.
Team
	Team composition	Who are the responders (eg, peers, mental health clinicians, Emergency Medical Technicians)?
	Responder race/ethnicity	What is the racial/ethnic composition of the CSR team?
	Responder gender	What is the gender composition of the CSR team?
	Community Safety Responder training	Information about training for Community Safety Responders.
	Police training	Information on any training police receive regarding CSR programme.
	Number of Community Safety Responders	How many Community Safety Responders work for the programme? Include any information on FTE.
Dispatch
	Call origin	What number is used to dispatch the programme (eg, 911, 988, 311)?
	Dispatch location	Where is dispatch housed (within emergency medical services (EMS), police, fire, CSR department)?
	Number of operators/dispatchers	What proportion of operators/dispatchers send out the CSR programme?
	Operator/dispatcher training	Information about training for operators/dispatchers (may include training on coordination, call transfers and cross-training between 911 and other dispatch sources).
	Dispatch protocol	Information about dispatch protocols/decision-making for CSR programme.
	Clinicians in call centre	Are there any clinicians housed within the call centre to help appropriately dispatch calls or de-escalate calls without sending out the on-scene responders?
Technology
	Mobile Data Terminals (MDTs)	Do Community Safety Responders have MDTs in their vans?
	Computer Aided Dispatch (CAD) integration	Is the CSR programme integrated into CAD?
Economic
	Funding source	How is the programme funded?
	Annual budget	What is the annual budget of the programme?
	Medicaid reimbursements	Does the programme file for Medicaid reimbursements for services?
Services
	Services	Any mention of specific services done by CSR programme. Examples may include: De-escalation Referral to services Overdose prevention (eg, carrying naloxone) Mediation/conflict resolution Food/water Providing other tangible resources (clothes, etc) Voluntary transport Involuntary transport
	Follow-up services	Does the CSR programme offer any follow-up services? If yes, please describe, include timeframe for follow-up.
Outcomes		When extracting data on outcomes, if any information is reported by subgroups (eg, race/ethnicity, gender, age, socioeconomic status, etc), please provide this information and as much specificity as possible at the raw data extraction stage.
	Call types	Percentages (and Ns) of call types as described by the source. Indicate whether initial, final or both call types are specified.
	Mental/Behavioural health call descriptors	Descriptions or percentages of type of mental/behavioural health calls
	Call resolutions	How are calls resolved? Provide information on any breakdown on call resolutions, including: Referral to services Transport to services Emergency department visit Emergent detention/involuntary commitment transfers Arrest On-scene resolution
	Referral resources	Where does the CSR programme refer consumers to?
	Transport location	Where do CSR programmes transport consumers to?
	Requests for police backup	How often are police called for backup (% of encounters)?
	Injuries	Information on any injuries reported by consumers or responders.
	Use of force	How often was force used in CSR involved calls? Describe who used force.
	Consumer race/ethnicity	Breakdown of racial/ethnic composition served by CSR programme.
	Consumer gender	Breakdown of gender composition served by CSR programme.
	Complaints	Information on complaints filed about experience with CSR programme.
	Repeated calls	Any information provided on repeat callers, including what percent of calls involve repeat callers.
	Successful connection to service utilisation	Any quantification of consumers successfully linked to services as a result of interaction with CSR programme.
	Police calls diverted	How many calls (% or n) does CSR programme respond to in place of police?
	EMS calls diverted	How many calls do they respond to in place of EMS?
	Emergency department visit reductions	How many calls are they able to avoid sending to the ED that otherwise would have resulted in an ED visit?
	Contact rate	What percent of calls to which the CSR programme is dispatched are they able to make contact with the person of interest?

Data analysis and presentation

Extracted data will describe CSR programmes. Data will be made publicly available and presented in a tabular form at the CSR programme level and hosted on an interactive website to allow policy-makers and researchers to see data across programmes. Charting the data will allow us to identify research gaps and summarise the research evidence on CSR programmes. We will also conduct a descriptive narrative synthesis to characterise programme components and outcomes. We will consult with methodological and other subject matter experts, including our Advisory Board, as necessary. The data collected from this review will be the first step in future evaluative research.

Patient and public involvement

This study is based solely on a review of existing literature and will not involve participants. The only potential members of the public involved in this study would be in the capacity of Advisory Board members. Otherwise, no patients nor members of the public will be involved in this research.

Ethics and dissemination

No ethical review for this study is required as it will not include human subjects or any identifiable information. Extracted data will be made publicly available and results will be submitted for publication in a peer-reviewed journal. Authors will also collaborate on a nationwide storytelling campaign to publicly disseminate findings to demonstrate the broader impact and potential of CSR programmes to the general public, which may result in a mapping visualisation web tool. We will also consult with our Advisory Board on how to best present results.