Achieving gains in state and local child safety systems and workforce development: application of the framework for quality improvement and innovation in child safety

Abstract

Objective

This study investigated the application of the Children’s Safety Network (CSN) Framework for Quality Improvement and Innovation in Child Safety through the Child Safety Learning Collaborative (CSLC).

Methods

The CSN Framework was used by 26 state/jurisdiction teams that participated in cohort 1 of the CSLC, from November 2018 to April 2020. The aim was to strengthen child safety systems and the workforce to spread child safety evidence-based and evidence-informed strategies and programmes for children and adolescents ages <1–19 years.

Procedures

Participating teams’ child safety system development, workforce development, engagement in the CSLC, challenges encountered and overall satisfaction with the CSLC were assessed through ongoing CSLC participation records and an end-of-cohort survey (survey response rate: 73.1%).

Results

Teams showed an average change of 2.4-fold increase in the spread of evidence-based and evidence-informed child safety strategies and programmes, indicating improvement in child safety systems. Knowledge development on CSLC tools and strategies was reported by 77.8% of teams, with 55.5% reporting CSLC tools and strategies contributed to workforce development. Over two-thirds (70.6%) reported being satisfied or very satisfied with the CSLC, but identified some challenges, including staff turnover and the need to strengthen partnerships. All teams demonstrated engagement in the CSLC, based on participation in a virtual meeting, learning session or a monthly report submission.

Conclusions

Despite challenges, teams continued to participate in the CSLC, recognising the importance of collaborative learning. The CSN Framework is helpful for state/jurisdiction teams to improve child safety systems and develop their workforce.

Background

Over 40 000 infants, children and adolescents die each year in the USA, of which one-third are attributable to preventable injuries and violence.¹ Injuries and violence are the leading cause of death for children/adolescents ages 1–19, and the third-leading cause of death for infants age <1 in the USA. Approximately 56% of injury deaths are due to unintentional injuries, 21% are suicides, 20% are homicides and 3% are undetermined deaths. Progress has been made in the field of injury and violence prevention and best practices have been identified,^2–8 yet the need remains to increase the spread of child safety strategies/programmes. The Children’s Safety Network (CSN) is a national resource centre for state/jurisdiction health departments working to spread (ie, dissemination of best practices across a whole system) child/adolescent safety across state/local communities. CSN coordinates the national Children’s Safety Now Alliance of federal, state, local, hospital and academic agencies/organisations and provides training and technical assistance (TTA) in child safety to Title V agencies (ie, state/jurisdiction departments of health).

CSN framework for quality improvement and innovation in child safety

CSN developed the CSN Framework for Quality Improvement and Innovation in Child Safety (The CSN Framework)^{9 10} (figure 1), leveraging Foundation Strategy Group’s (FSG) collective impact approach, Institute for Healthcare Improvement’s (IHI) breakthrough series (BTS) and Associates in Process Improvement’s (API) model for improvement (MFI).^11–14 The CSN Framework aligns with (1) FSG’s model in having a common agenda/aim, shared measurement, mutually reinforcing activities, continuous communication and a backbone organisation,¹³ (2) IHI’s BTS model in using structured activities, including learning sessions and action periods,^{12 14} and (3) API in using the MFI.¹⁴ BTS has been used in initiatives to improve healthcare,^{15 16} mental health,^{17 18} child welfare^{19 20} and early detection and intervention services^{21 22} for children and families. CSN adapted this model to address the complexities of change and collective action at state levels.

Figure 1.

Children’s Safety Network Framework for Quality Improvement and Innovation in Child Safety.

Building on these models, the CSN Framework (1) incorporates a prominent component for leadership and management that provides support and resources for state leaders working in complex environments; (2) expands the expert community to include public health practitioners that inform feasibility of spreading evidence-based child safety strategies; (3) adapts the delivery of learning activities by adding blended learning and outputs for workforce development and systems improvement; and (4) adds distal population-level outcomes of improved child safety. The rationale for these adaptations and supporting evidence is described in previous work.¹⁰ The CSN Framework, used by the Child Safety Learning Collaborative (CSLC), puts forth consistent application of child safety expertise, leadership and management, and systems improvement methods results in strengthening child safety systems and the workforce, ultimately contributing to reductions in fatalities, hospitalisations and emergency department visits. The CSN Framework was first evaluated for feasibility in 2017 and results demonstrated the CSN Framework is a promising approach for learning collaboratives to achieve statewide child safety system improvement.¹⁰

Child Safety Learning Collaborative

Quality improvement learning collaboratives bring teams together to identify, test and spread evidence-based strategies/programmes through a structured process.^{23 24} While there is evidence on positive effects associated with learning collaboratives,²⁴ literature on the strength of the approach and specific factors that contribute to outcomes remains limited.^{23 25} One meta-analysis found intact teams, participation, and continued data collection contributed to successful outcomes, while leadership support, time and resources did not.²³ In another study participants reported collaborative faculty, quality improvement tools and resources (eg, plan-do-study-act (PDSA) cycles, change packages) and learning sessions advanced improvement efforts and knowledge acquisition.²⁶

The CSLC was developed to bring Title V agencies together in their efforts to improve child safety through state/local improvements in child safety systems (ie, the interplay between policies, procedures, infrastructure, resources and human actions)²⁷ and workforce development (ie, knowledge, skills and competencies of a workforce to carry out public health services, including delivery of effective programmes and interventions).²⁸ CSLC cohort 1 took place from November 2018 to April 2020 with eighteen states/jurisdictions that formed a total of 26 teams. Applications were submitted by state/jurisdiction maternal and child health and/or injury and violence prevention directors and were used to assess applicants’ capacity and readiness to spread evidence-based approaches and quality improvement methods. Although all applications were accepted, variations were observed such that 50% of teams reported readiness to spread strategies. CSLC team membership was selected by states/jurisdictions and generally ranged from 3 to 12 members representing state/jurisdiction departments of health and selected partners (eg, other state agencies, hospitals, schools). Each state/jurisdiction chose to address one to three child safety topics: motor vehicle traffic safety, poisoning prevention, suicide and self-harm prevention, sudden unexpected infant death (SUID) prevention and bullying prevention.

CSLC topics were identified by CSN as those with the greatest impact on injury and violence prevention and most likely to be high priority for Title V agencies based on state/jurisdiction selected national performance measures. Table 1 provides prevalence data for bullying victimisation, fatality rates for CSLC topics, and CSLC team descriptive data. After a review of state action plans, CSLC teams self-selected evidence-based or evidence-informed programmes and practices to spread in their states/jurisdictions from a menu of programmes and practices provided by CSN (table 1).²⁹ Participation expectations were to attend CSN coordinated monthly virtual meetings with teams across the CSLC to receive TTA on the CSN Framework, share challenges, successes and lessons learnt in a collaborative peer setting, and submit monthly data reports. Between virtual meetings, teams met within their state/jurisdiction team, setting their own agendas to advance the work.

Table 1.

Child Safety Learning Collaborative (CSLC) cohort 1 child safety topics and population

Child safety topics	Population data			CSLC data
	Population of Interest	Fatality rate per 100 000 Population of prevalence percentage	No of states/jurisdictions working on child safety topics*	Example of evidence based/evidence informed programmes	Example of spread strategies (ie, dissemination practices across a whole system
Motor Vehicle Traffic Safety†	0–19 years	4.3 per 100 000	6	Teens in the Driver Seat36	Spread child passenger safety education to parents/caregivers
					Spread teen driver safety education
Unintentional Poisoning Prevention†	0–19 years	0.9 per 100 000	3	Poison Control Centre Services37	Spread education about poison control centres
					Spread drug misuse prevention programmes for adolescents
Suicide and Self-Harm Prevention†	10–19 years	7.0 per 100 000	7	Sources of Strength38	Spread gatekeeper training
					Spread multi-component suicide prevention programmes
Sudden Unexpected Infant Death Prevention	<1 year	102.4 per 100 000	7	Safe to Sleep Campaign39	Spread safe sleep training to healthcare providers
					Spread training to first responders in infant safe sleep
Bullying‡	High school students	19.50%	3	Second Step Program40	Spread bullying prevention programmes in schools
					Spread bullying prevention programmes to parents

*States/jurisdictions could participate in more than one child safety topic.

†Fatality data are based on the average 2017–2019 data from the National Centre for Health Statistics, Multiple Cause of Death data.

‡Bullying victimisation in not captured in fatality data; prevalence is based on the 2019 National Youth Risk Behaviour Surveillance System.

Methods

Twenty-six teams across 18 states/jurisdictions participated in cohort 1. One team dropped out due to staffing issues, yielding 96.2% CSLC participation rate.

Data sources

Participation records

CSN tracked attendance in monthly virtual meetings and two learning sessions and submission of monthly reports and PDSAs (ie, a structured approach to develop, test, implement and spread strategies through a planned learning process.³⁰ Table 2 describes participation rates across participation record type. Each team designated a point-of-contact who completed self-reported, web-based monthly reports that collected quantitative and qualitative data. PDSA submissions were web based and described teams’ plans to test strategies. Teams decided when and how many PDSAs to submit, with 42% of teams submitting between one and six PDSAs, averaging two PDSAs per team. The quality of PDSAs submitted was inconsistent and CSN provided TTA each time a team advanced through the stages of a PDSA.

Table 2.

Response rates in participation records and end-of-cohort survey

Child safety topic	PDSAs	Monthly reports	Meetings	Learning sessions	End-of-cohort survey
Motor vehicle traffic safety (n=6)	66.7%	100%	100%	100%	66.7%
Unintentional poisoning prevention (n=3)	33.3%	100%	100%	100%	100%
Suicide and self-harm prevention (n=7)	42.9%	100%	100%	100%	85.7%
Sudden unexpected infant death prevention (n=7)	28.6%	100%	100%	100%	71.4%
Bullying prevention (n=3)	0.0%	100%	100%	66.7%	33.3%
Total (N=26)	38.5%	100%	100%	96.2%	73.1%

Note: Response rates were calculated based on submission of at least one PDSA, submission of at least 1 monthly report, attendance at a minimum of one meeting and attendance at a minimum of one learning session during cohort 1.

PDSA, plan-do-study-act.

End-of-cohort 1 survey

Nineteen of the 26 teams completed a self-report, web-based survey at the end of cohort 1 in May/early June 2020 (overall response rate: 73.1%) (table 2). Respondents were similar to non-respondents with respect to group size (respondents: 3–12 members per team vs non-respondents: 4–10 members per team), submission of at least one PDSA (36.8% respondents vs 42.9% non-respondents) and monthly report (100% respondents vs 100% non-respondents) and attendance at a minimum of one meeting (100% respondents vs 100% non-respondents) and one learning session (100% respondents vs 85.7% non-respondents). The survey was sent to the main point-of-contact for each team, who was asked to obtain input from all team members; however, CSN could not determine who contributed to the final responses. The survey contained multiple choice, Likert scale and open-ended questions to assess knowledge development on CSLC tools/strategies, effectiveness of CSLC tools/strategies on workforce development, and other measures, including staff turnover, TTA and overall satisfaction with the CSLC. Missing data were handled through listwise deletion; it ranged from 5.3% to 10.5% (1–2 teams) on workforce development, 10.5% (2 teams) on CSLC satisfaction and 15.8% (3 teams) on open-ended responses to identify challenges.

Narratives from selected teams

Two CSLC teams were invited by CSN to share success stories on child safety system development and workforce development to illustrate progress using the CSN framework.

Measures

Child safety system development was measured by teams’ self-reported progress in spreading strategies/programmes through a web-based monthly report. Teams were provided evidence-based/evidence-informed child safety strategies/programmes and spread measures (table 3).²⁹

Table 3.

Spread of evidence-based and evidence-informed child safety strategies and programmes across teams

Child safety system development measure	Teams reporting data	Start of cohort	End of cohort	Change from start to end of cohort	Average change (fold increase/decrease)
Motor vehicle traffic safety (n=6 teams)
# of schools and organisations providing teen driver safety education to teenagers	3	95	151	56	1.59
# of schools and organisations providing teen driver safety education to caregivers	2	8	11	3	1.38
# of parents informed of parents are the key parent-teen driver agreement	1	31	85	54	2.74
# of inspection and fitting stations established	2	157	174	17	1.11
# of child passenger safety technicians trained and certified	2	494	636	142	1.29
# of organisations distributing child safety and booster seats*	2	68	47	−21	−1.45
# of fitting stations, birthing hospitals or organisations offering child passenger safety education to caregivers	1	12	33	21	2.75
Unintentional poisoning prevention (n=3 teams)
# of individuals receiving education on poison control centre services	2	95	6240	6145	65.68
Suicide and self-harm prevention (n=7 teams)
# of organisations using zero suicide	1	16	17	1	1.06
# of schools and organisations providing gatekeeper training	4	26	219	193	8.42
# of schools and healthcare organisations using valid and reliable screening tools	2	4	20	16	5.0
# of schools and organisations providing multicomponent suicide and self-harm prevention programmes	2	5	20	15	4.0
Sudden unexpected infant death prevention (n=7 teams)
# of hospitals and birthing facilities providing infant safe sleep training to healthcare providers	4	80	89	9	1.11
# of hospitals and birthing facilities providing infant safe sleep education to parents and caregivers	2	60	67	7	1.11
# of organisations distributing free and discounted Pack’n Plays or cribs to parents and caregivers*	4	73	66	-7	−1.11
# of parents or caregivers in historically underserved communities receiving safe sleep education	2	6564	10 687	4123	1.63
# of organisations providing training on infant safe sleep hazards to emergency personnel	1	1	1	0	1.0
# of home visitor programme sites distributing safe sleep educational materials	5	21	34	13	1.62
# of organisations implementing safe sleep campaigns	5	18	37	19	2.06
Bullying prevention (n=3 teams)
# of schools or organisations implementing bullying prevention programmes	1	1	5	4	5
Total (N=26 teams)
# of total measures	17	7829	18 639	10 810	2.38

Note: Teams could select more than one child safety system development measure. Average change is calculated using the fold ratio formula, indicating the number of times the end of cohort unit changed from the beginning of the cohort. For an increase, the end of cohort unit is divided by the beginning of cohort unit. For a decrease, the beginning of cohort unit is divided by the end of cohort unit and the result is noted as a negative.

*Decreases reported by teams were associated with the impact of COVID-19 and reduced staffing on their child safety work.

Workforce development was assessed using end-of-cohort survey data. Workforce development was measured through (1) knowledge development across CSLC child safety strategies/tools and (2) effectiveness of CSLC strategies/tools to use systems improvement tools/approaches, access and implement child safety evidence-based/evidence-informed strategies, and apply leadership and management practices (table 4). The survey also assessed which CSLC strategies/tools across the CSN framework were most helpful to spread strategies/programmes (table 4). Measures were identified based on the CSN Framework and literature review.^{10 23 26}

Table 4.

Items assessing workforce development of Child Safety Learning Collaborative (CSLC) teams

Domain	Items	No or a little development	Some, much, or a great deal of development
Knowledge development across CSLC quality improvement strategies and tools*	Data collection strategies	22.2%	77.8%
	Improvement monitoring processes (eg, PDSA)	27.8%	72.2%
	Systems thinking framework	11.1%	88.9%
	SMART objectives	22.2%	77.8%
	Accessing and implementing evidence-driven practices, programmes, and policies	22.2%	77.8%
	Scaling-up evidence-driven practices, programmes, and policies	27.8%	72.2%
	Knowledge development across all strategies/tools	22.2%	77.8%
		Not at all or a little improvement	Some, much, or a great deal of improvement
Effectiveness of CSLC quality improvement strategies and tools to improve aspects of workforce development†	Child safety system	58.8%	41.2%
	Data collection and analysis process	47.1%	52.9%
	Partnership with other agencies on common goals	41.2%	58.8%
	Organisation/department’s strategic plan	41.2%	58.8%
	Programmatic decisions in programme implementation and spread	44.1%	55.9%
	Programme/project organisation	35.3%	64.7%
	Effectiveness across all workforce development areas	44.5%	55.5%
		Not at all or a little helpfulness	Some, much, or a great deal of helpfulness
Helpfulness of specific CSLC strategies to spread programmes/strategies†	Child safety
	Change packages	33.3%	66.7%
	Topic experts teaching about accessing, implementing, and spreading evidence-driven practices, programmes, and policies	30.6%	69.4%
	Leadership and management
	Interactions with other teams	44.4%	55.6%
	Sharing resources with other teams	47.2%	52.8%
	Systems improvement
	Virtual meetings and learning sessions	16.7%	83.3%
	SMART objectives and PDSA cycles	41.7%	58.3%
	Systems thinking framework	33.3%	66.7%
	Topic experts teaching about systems improvement, data collection and measurement strategies, and improvement techniques	27.8%	72.2%
	Helpfulness across all CSLC strategies to spread programmes/strategies	33.3%	66.7%

*Knowledge development across CSLC child safety strategies and tools was measured on a 5-point Likert scale: 1=Developed no knowledge; 2=Developed a little knowledge; 3=Developed some knowledge; 4=Developed much knowledge; 5=Developed a great deal of knowledge; and dichotomised as No/A little versus Some/Much/A great deal of knowledge developed.

†Effectiveness of CSLC strategies and tools to improve aspects of workforce development and helpfulness of specific CSLC strategies to spread programmes/strategies were measured on a 5-point Likert scale: 1=Not at all; 2=A little; 3=Somewhat; 4=Much; 5=A great deal; and dichotomised as Not at all/A little versus Somewhat/Much/A great deal.

PDSA, plan-do-study-act.

Engagement was measured by participation in virtual meetings, learning sessions, PDSA submission, monthly report submission and/or requesting TTA from CSN. Engagement data were gathered by CSN in participation records, except TTA, which was self-reported by teams in the survey using yes/no responses. TTA corresponded with the CSN Framework: (1) Child safety expertise: Training models/education/resources to increase knowledge/skills across child safety topics; (2) Leadership and management: building capacity to leverage/sustain partnerships/collaborations and develop/implement organisational policy and (3) Systems improvement: Capacity building to develop child safety systems using quality improvement and implementation science, improve data collection/evaluation and improve practice/programme development to ensure spread of child safety strategies/programmes.

Overall satisfaction and challenges were measured using the end-of-cohort survey. Teams rated satisfaction with the CSLC on a 5-point scale, ranging from 1=very dissatisfied to 5=very satisfied. Teams responded to whether they experienced turnover in membership during cohort 1. Response options were ‘no change,’ ‘leadership changed,’ ‘less than 50% of members changed’ and ‘more than 50% of members changed’. Teams responded to an open-ended question on other factors and challenges that contributed to their work on improving child safety.

Analyses

Descriptive statistics were conducted to assess teams’ child safety system development, workforce development, engagement in the CSLC, staff turnover, TTA and overall satisfaction with the CSLC. Open-ended responses to challenges were coded manually using a focused coding or level-2 coding approach to group themes. A second coder independently coded open-ended responses for reliability.³¹

Patient and public involvement

There was no patient or public involvement in study design or analysis.

Results

Child safety system development results

Teams demonstrated positive child safety system development, measured as average change of 2.4-fold increase in the spread of child safety strategies/programmes (table 3). The largest positive average change was in poisoning prevention with two teams achieving a 65.7-fold increase. Following this result was an 8.4-fold increase in average change reported by four teams in the number of schools and organisations providing gatekeeper training for suicide prevention. Of note, 4123 parents/caregivers in historically underserved communities received safe sleep education through the work of two teams. Table 5 provides a narrative on child safety system development from a selected state/jurisdiction.

Table 5.

Examples of child safety development and workforce development from two Child Safety Learning Collaborative teams

Child Safety System Development

State/jurisdiction Team one aimed to increase the number of organisations that implement evidence-based bullying prevention and social and emotional learning programmes in Gay and Straight Alliances (GSAs) across the state/jurisdiction. They developed a strong partnership with an organisation that operates a statewide GSA Network and conducted a series of focus groups with young adults attending college, high school and middle school across the state/jurisdiction, soliciting rich information on students’ preferred methods to receive bullying prevention messaging. They introduced an evidence-based programme through a partner’s summer camp and spread it fivefold to schools working with LGBTQIA adolescents.

Workforce Development

State/jurisdiction Team two built capacity to use systems thinking, develop common SMART objectives with new partners and scale-up evidence-driven practices in poisoning prevention. They identified Woman, Children and Infants (WIC) offices no longer receiving poison control information and established a new partnership with WIC, increasing the number of individuals receiving education through poison control education events/programmes from 94 to 6125 individuals. State/jurisdiction Team two then expanded efforts to reach licensed in-home and centre daycare providers through webinars and continuing education unit credits and added materials from poison control centres to the state/jurisdiction Department of Public Health quarterly newsletter to increase spread across diverse audiences.

Workforce development results

Table 4 shows workforce development reported by teams. Knowledge developed on CSLC strategies/tools ranged from 72.2% on improvement monitoring processes and scaling up evidence-driven practices/programmes/policies to 88.9% on systems thinking framework. Effectiveness of CSLC strategies/tools to support workforce development areas ranged from 41.2% for child safety system to 64.7% for programme/project organisation. Teams reported most helpfulness of virtual meetings and learning sessions (83.3%) and topic experts teaching about systems improvement, data collection and measurement strategies, and improvement techniques (72.2%) to spread programmes/strategies. Table 5 provides a narrative on workforce development from a selected state/jurisdiction.

Engagement results

All 26 teams participated in at least one virtual meeting and submitted at least 1 monthly report, 25 teams participated in at least one learning session (table 2). The percent of teams submitting PDSAs ranged from 28.6% to 66.7% (table 2).

All 19 teams completing the end-of-cohort survey received TTA. Most teams requested TTA on partnership/collaboration (78.6 %), followed by data collection/evaluation (60.0%), practice/programme development/implementation (37.5%), education/resources to increase public/stakeholder awareness (33.3%), systems development/implementation (33.3%); training models (25.0%); and organisational policy development/implementation (14.3%).

Overall satisfaction and challenges results

Most teams (70.6%) reported being satisfied or very satisfied with the CSLC. The majority (68.4%) reported turnover in membership over the course of cohort 1: 21.1% reported less than 50% membership change, 36.8% reported more than 50% membership change, and 10.5% reported more than 50% membership change and leadership change. Major challenges were staff turnover and staff capacity (31.3%); difficulty applying strategies, for example, identifying the population of interest (18.8%); weak partnership/collaboration (12.5%) and lack of funding (12.5%).

Discussion

Historically, prevention efforts across child safety areas are not consistently and widely spread. An innovative approach to spreading child safety strategies/programmes and developing the workforce is through the CSLC, which uses the CSN Framework for Quality Improvement and Innovation in Child Safety.¹⁰ CSLC cohort 1 findings demonstrate the three key areas of the CSN Framework—(1) leadership and management, (2) child safety expertise and (3) systems improvement—contribute to child safety system improvement and workforce development; and are congruent with the literature that subject matter expertise/identification of evidence-based strategies,¹⁴ the capacity to lead³² and implementation science¹⁴ are critical for scaling health interventions to large populations.

An important aspect of the CSLC is that all state/jurisdiction Title V agencies are allowed to join regardless of their knowledge, prior experience, infrastructure or resources to spread selected programmes. Accordingly, teams varied in the spread of evidence-based and evidence-informed child safety programmes/strategies. Teams were encouraged to set and modify spread goals as appropriate, but those goals were difficult to measure as teams worked in complex adaptive systems and the majority of teams (76.9%) did not report goals. In accordance with the literature, a majority of teams (77.8%) indicated participation in the learning collaborative aided in developing knowledge on various strategies/tools.²⁵ Knowledge development was highest for systems thinking framework, an approach to understanding complex systems and a critical component of the CSN framework. Half the teams (55.5%) reported CSLC strategies/tools contributed to workforce development, especially related to programme/project organisation. Approximately two-thirds (66.7%) of teams reported CSLC strategies/tools, particularly virtual meetings and learning sessions on systems improvement, were instrumental in helping teams improve their child safety systems. Virtual meetings and learning sessions offer live TTA to teams and opportunities for teams to participate in peer learning, which are critical components of a learning collaborative model.¹² TTA provides capacity-building support to teams as they learn and master new skills and pursue improvement.¹⁴

Across workforce development areas studied, CSLC strategies/tools were least effective in improving child safety systems. This finding is not surprising given systems are dynamic and transdisciplinary integration of approaches is often non-linear. Identifying and transforming processes to integrate elements in a system includes time delays between taking an action and seeing the effects of that action.¹⁴ In this context, teams work in complex systems on population health aims that require multiple resources, ranging from partnerships to subject matter expertise, expertise in data collection and analysis, and health communication and knowledge and behaviour change.³² A sampling of strategies and tools may fall short of what is required to impact such a system during the timeframe of cohort 1. Teams shared challenges with staff turnover, weak partnerships, lack of buy-in from leadership and funding. Needs for funding and support from networks, collaborations, teams and partnerships are commonly noted challenges to effective systems thinking and modelling in public health.^{25 33} Nonetheless, attrition in CSLC was low, with only one team leaving due to staffing issues.

Limitations

Limitations should be considered when interpreting the findings of this study. Two of three teams focused on bullying prevention did not report data; 33.3% in poisoning prevention; 33.3% in motor vehicle traffic safety; 42.9% in suicide and self-harm prevention; and 14.3% in SUID prevention. Nearly 27% of teams did not respond to the end-of-cohort survey. Staff time and resources were constrained by the COVID-19 pandemic, an example of an unanticipated influence on scaling up evidence-based strategies that requires strong partnership, real time data collection and problem solving to anticipate and mitigate negative impacts.^{25 34} Data were also self-reported and monitoring full fidelity to evidence-based programmes/practices was not assessed. Further, analysis of factors contributing to differential findings by safety topic (eg, bullying prevention, SUID prevention) and their relationship to spread was considered outside the scope of this study. Teams reported output data on the child safety system development measures (eg, number of schools and organisations providing education and number of individuals receiving education around specific evidence-based practice), but not data on the total number of schools, organisations, or individuals in their respective community/state/jurisdiction. Hence, the proportion of organisations or individuals adapting each spread strategy could not be computed. Further, due to the nature of this descriptive study with a lack of a control group or baseline data from teams, the most effective strategies and the effect size of the CSLC programme could not be determined. Future research on learning collaboratives may incorporate experimental design with a control group. In addition, future research should assess how gains in child safety improvement and workforce development are sustained^{23 35} and their relationship to health impact at the population level. Finally, whether teams received simultaneous support and TTA from other sources that could have contributed to workforce development and assisted to spread child safety strategies/programmes was not assessed.

Conclusions

This study contributes to the scant literature on the effectiveness of quality improvement learning collaboratives in population health by providing empirical evidence to a conceptual framework for improving child safety at the state level. Findings suggest the CSN Framework is effective in assisting states to improve child safety systems and develop their workforce, even during times of disruption, by engaging in quality improvement processes with peers and receiving supportive regular TTA from child safety, leadership and management, and quality improvement leaders.

Key messages.

What is already known on this topic?

• Quality improvement learning collaboratives are a widely used approach in healthcare settings, but there is scant literature on the effectiveness of these collaboratives on child safety work at the state level and in community settings.

• CSN’s Framework for Quality Improvement and Innovation in Child Safety (The CSN Framework) is a promising approach for increasing the use of evidence-based practices by states/jurisdictions to promote child safety.

What this study adds

• The CSN Framework is helpful to quality improvement learning collaborative teams in spreading evidence-based and evidence-informed child safety strategies and programmes across state and local communities.

• Even in times of disruption (eg, staff turnover, natural disasters and a pandemic), states/jurisdictions are able to apply the CSN Framework and promote improvements in child safety systems and workforce development.

How this study might affect research, practice and/or policy

• This study could inform how programmes design, implement and assess learning collaboratives for state-level complex systems, highlighting areas that contribute to workforce development and improved child safety systems.

Data availability statement

Data are available on reasonable request. Deidentified data available on reasonable request from the corresponding author.

Ethics statements

Patient consent for publication

Not applicable.

Ethics approval

This study did not meet the definition of human subjects research under 45 CFR 46.102(f) and therefore was not subject to IRB review.