A mixed methods evaluation of parents’ perspectives on the acceptability of the S.A.F.E. Firearm program

INTRODUCTION

Limiting unauthorized youth access to firearms is essential for preventing both unintentional injury and suicide.^1–5 Despite recommendations from the American Academy of Pediatrics⁶ and other organizations⁷ that firearms be stored locked, unloaded, and separately from ammunition, an estimated 4.6 million children live in homes with at least 1 firearm stored unlocked and loaded.⁸ A national survey found that at least 1 firearm is present in 42% of U.S. households,⁹ and firearm sales have surged since the onset of the COVID-19 pandemic.¹⁰ The availability of firearms, coupled with their lethality, amplifies the importance of firearm storage programs directed to parents and caregivers of youth (hereon referred to as “parents”). Pediatric primary care is an optimal setting for implementation of such programs, given the reach of well-child care.¹¹

Suicide and Accident prevention through Family Education (“S.A.F.E Firearm”) is an adaptation of the firearm storage component of the Safety Check violence prevention program.¹² Safety Check was found to be effective in a large clinical trial,¹² but core components have not yet been widely adopted in routine practice.¹³ Aligning with implementation science recommendations to adapt evidence-based interventions for use in new populations and to improve program-setting fit,^14,15 we used the ADAPT-ITT framework¹⁶ to adapt Safety Check to reach a broader age group (i.e., youth up to age 18, expanded from ages 2–11) and to serve as a universal suicide prevention strategy in pediatric primary care.¹⁷ The adapted program retains the core components of the original program: (1) brief counseling provided by pediatric primary care clinicians on secure firearm storage and (2) offering free cable firearm locks. The adapted program changes the entry point of the counseling conversation from an identified parental concern to universal counseling for all parents. Self-disclosure about ownership is not documented in the electronic medical record, and parents are offered additional resources (e.g., a handout describing alternative storage options).¹⁷

Although parent input was obtained for key program elements (i.e., name, logo) during prior stages of adaptation,^17–19 the present study sought to obtain a comprehensive view of parent perspectives on the full S.A.F.E. Firearm program following the last stage of adaptation, in keeping with parents’ role as end users. This reflects principles of user-centered design, which focuses on redesigning and adapting interventions with end users in mind,²⁰ with the goals of maximizing “usability in context”,²¹ patient- (or family-) centered care,²⁰ and acceptability, or the perception that a program is agreeable, palatable, or satisfactory.^20,22

Acceptability is a key determinant of program effectiveness; if it is poor, parent engagement and subsequent behavior change will likely be low.²⁰ Although a majority of parents in 1 study believed that pediatricians should advise on safe storage,²³ the sensitive national discourse and complex social meanings of firearm ownership merit a nuanced approach.^24,25 For example, given the diversity of parent end users of a universal intervention, identifying potential variations in acceptability across primary segments²⁶ (i.e., firearm owning and non-owning parents) and subgroups (e.g., by gender, race and ethnicity, and age of children) is important to determine if further program tailoring is needed. Because sustained partnership²⁷ with clinicians is a central feature of the primary care setting for many families, attending to parents’ perceptions of relationships with their children’s clinicians may also facilitate implementation success.

The current study aimed to develop a comprehensive understanding of the acceptability of S.A.F.E. Firearm from a parent perspective, to ensure usability and optimize effectiveness, with the goal of identifying any outstanding concerns or necessary adjustments prior to the launch of an upcoming hybrid effectiveness-implementation trial.

METHODS

Study Design

This mixed methods study included 2 components: (1) a quantitative cross-sectional survey with a national convenience sample of U.S. parents to measure S.A.F.E. Firearm acceptability and other key variables (e.g., trust in pediatrician) that may impact implementation; and (2) semi-structured interviews with a convenience sample of survey participants, using a qualitative descriptive approach to gain more in-depth understanding of parents’ views. Data integration occurred at the study design, methods, and interpretation levels through a convergent design with concurrent analysis of quantitative and qualitative data, merging of both datasets, and narrative interpretation.²⁸ The University of Pennsylvania Institutional Review Board approved all study procedures; informed consent was obtained prior to initiation of survey and interview components of the study.

Quantitative Methods

Participants.

We fielded a survey on Amazon’s Mechanical Turk (MTurk) platform in July-August 2020. We selected MTurk due to its prior use in firearm-related survey research.²⁹ More information on MTurk and eligibility criteria for the pre-screening survey is available in Appendix A. To be eligible for the full survey, on the pre-screening survey the participant needed to report being a parent or guardian of a child aged 17 or younger who attends well-child visits. We recruited participants who both did and did not report on the pre-screener that firearms were present in their home or on their property.

Procedures.

Participants watched a 6.5-minute video depicting a telehealth visit in which S.A.F.E. Firearm is delivered by a pediatrician to a parent. The video was recorded by a practicing pediatrician, with the study’s principal investigator (RB) acting as the parent recipient of the program. A video transcript is provided in Appendix B. Participants were presented with written instructions asking them to imagine that their pediatric primary care clinician had just had a similar conversation with them at their child’s well-child visit and to complete the survey with this in mind. Participants were then asked to provide demographic information and indicate whether they would like to be contacted for a follow-up interview. In keeping with typical compensation practices on MTurk, they were paid $4.83 for completing the survey.³⁰

Measures.

The survey instrument was developed by the research team and included both original items and items from validated scales when available.

Sociodemographic information.

Participants were asked to report their age, gender, race, ethnicity, state of residence, number of children, age ranges of children, well-child visit attendance in the past year, and the type of pediatric clinician that their children most frequently see for well-child visits.

Acceptability.

We assessed acceptability using the Acceptability of Intervention Measure (AIM), a pragmatic, brief, reliable, and valid 4-item tool.³¹ The tool has a 5-point ordinal response scale ranging from completely disagree (1) to completely agree (5). Higher response options indicate greater acceptability. As a secondary measure of acceptability, we queried whether participants would recommend S.A.F.E. Firearm to a friend using a 5-item scale ranging from completely disagree (1) to completely agree (5).

Firearm ownership.

We asked participants whether they currently have firearms in their home or on their property and provided definitions.^Ɨ Participants were asked to indicate which type of firearm they owned or had on their property. Options included handguns (e.g., pistol, revolver), long guns (e.g., rifle, shotgun), other, or unknown type.

Current firearm storage practices.

For those reporting presence of firearms, we asked how many handguns in their home or on their property were always stored locked, unloaded, and with ammunition locked separately (when not in use) during the last 2 weeks (Cheryl King, PhD, email communication, May 2020). Participants could respond all, some, none, or prefer not to disclose. The same question was used for long gun storage.

Intention to change behavior.

Those with firearms were asked to report their level of agreement with the statement: “I would change the way firearm(s) are stored in my home/on my property after receiving S.A.F.E. Firearm.” Response options ranged from completely disagree (1) to completely agree (5).

Pediatrician relationship.

We assessed participants’ relationship with their pediatrician using the 8-item Patient-Doctor Depth-of-Relationship (PDDR) scale, a conceptually grounded questionnaire with established psychometric properties for use in primary care (Cronbach’s α = .93).³² We modified the scale to replace “doctor” with “pediatrician,” defined as “the health care provider your child(ren) sees most often for primary care check-up.” The 5-point response scale ranges from disagree (0) to totally agree (4).

Trust in pediatrician.

We used a three-item trust measure from the Firearm Safety Among Children and Teens Consortium National Survey, which adapted the validated Wake Forest Physician Trust Scale.^33,34 The measure specifically queried participants about trust in their family’s pediatrician to counsel on what is best for children’s health, vaccines, and firearm safety. The 5-point response scale ranges from strongly disagree (1) to strongly agree (5).

Data analysis.

Individuals who answered “yes” to any part of the questions regarding ownership or presence of firearms on their property were categorized as a “firearm owner” for analysis purposes. State of residence was assigned to Northeast, Midwest, South, or West based on Census region designations. Race and ethnicity categories were collapsed. The child age range variable was dichotomized by (a) any child(ren) ages 0–10 in the household versus (b) only child(ren) ages 11–17, in keeping with the American Academy of Pediatrics’ identification of adolescence as starting at age 11.³⁵ In addition, as detailed in Appendix A, we used several strategies to ensure data quality in preparation for analysis. We calculated descriptive statistics for categorical and continuous measures and used a priori independent sample t-tests to assess for differences in acceptability based on the mean AIM score between firearm owners and non-owners. Post hoc, we examined AIM scores in relation to other key variables of interest, such as gender, race and ethnicity, and trust in pediatrician, to gain an understanding of factors that may inform further adaptations (e.g., culturally tailored program materials) or factors that may be modifiable (e.g., trust), to optimize the implementation of S.A.F.E. Firearm. These associations were examined using independent sample t-tests (parent gender, child age), one-way ANOVA (parent race and ethnicity, region of residence), and Pearson correlations (patient-pediatrician depth of relationship, trust in pediatrician), as appropriate. Analyses were completed in IBM SPSS Statistics version 27.

Qualitative Methods

Participants.

Using a convenience sampling approach,³⁶ qualitative participants were drawn from quantitative survey respondents who had indicated interest in a follow-up telephone interview. Potential interview participants were contacted up to 5 times by phone, email, and/or the MTurk platform, based on participant preference. Qualitative interviews took place from July-September 2020.

Procedures and Measures.

Study staff provided a link to the same S.A.F.E. Firearm video used in the survey and requested that participants re-watch the video prior to the scheduled interview. Interviews followed a semi-structured interview guide developed and piloted by the research team (see Appendix C). In the first part of the interview, participants completed a modified Think Aloud exercise,^37–39 in which they paused the video on their personal electronic device at 3 pre-specified points based on natural shifts in the counseling conversation. Interviewers first asked participants to provide immediate reactions to each section and probed thought processes (e.g., can you talk through why you’re thinking that?) to elicit nuanced feedback on specific program components. Participants were then asked about main takeaways of the program and prior experiences with firearm counseling in a pediatric primary care setting. Firearm-owning participants were asked about current firearm storage practices and preferences for post-visit program follow-up, given equivocal findings in previous work.¹⁷ Participants were paid a $10 bonus in MTurk.

Each telephone interview was conducted by 1 of 4 members of the study staff, who were trained by the team leads and met weekly while interviews were being conducted. Interviews were audio-recorded, with field notes written immediately after. Research assistants transcribed all interviews, and transcripts were loaded into NVivo Qualitative Data Analysis Software, version 12 (QSR International) for data management. Participants did not review transcripts or findings.

Data analysis.

We employed a directed approach to content analysis.⁴⁰ We used key conceptual findings from our research team’s prior studies^17–19 to chart out initial coding categories that we hypothesized would be relevant to acceptability (e.g., privacy, trust). Two authors read each transcript and field notes for immersion⁴¹ and then annotated the same 3 transcripts to refine the initial concepts and identify additional concepts not captured by the preliminary coding categories. They then met to discuss these annotations and develop a codebook.

The manifest content (i.e., visible, obvious components) was analyzed, with interpretation remaining close to the text.⁴² After completing 2 rounds of double coding to ensure reliability and making codebook revisions, the 7 remaining transcripts were divided between coders for independent coding. Using the constant comparative technique,⁴³ codes were inductively and iteratively compared to one another and the entire data set to develop categories.⁴⁴ Each category was then compared and contrasted with other categories within the data set.^44,45 Analytic memos were detailed to enhance rigor.

RESULTS

Quantitative Survey

Participants.

Figure 1 displays details of the participant selection and survey response process. Of the 203 individuals who were eligible for the full survey based on the pre-screener, 97 (48%) completed a survey that passed quality inspection. Participant characteristics are described in Table 1. Over half (57%) of the sample identified as male, and 39% were from racial and ethnic minoritized groups. Most participants’ children (81%) received well-child care from a physician. The majority of participants (77%) reported that all of their children had visited a primary care clinician for a well-child visit in the past year, which included the first 4 months of the COVID-19 pandemic. Fifty-two participants (54%) did not own firearms, and 45 participants (46%) reported having firearms in their home or on their property. In assessing the relation between firearm ownership and demographic variables, the only significant association was between firearm ownership and gender, with males more likely to own firearms (OR = 2.40, 95% CI [1.02, 5.62], p = .042).

Figure 1.

Survey Participant Flow Chart. MTurk indicates Amazon Mechanical Turk. HIT indicates human intelligence task. Additional details are available in the Appendix.

Table 1.

Survey Participant Demographic Characteristics

	Full Sample Mean (SD) or Percent	Firearm Owners Mean (SD) or Percent	Firearm Non-Owners Mean (SD) or Percent
N	97	45	52
Age, yearsa	37.6 (9.0)	36.7 (8.8)	38.3 (9.3)	p = .369
Gender				p = .042*
Female	40%	29%	50%
Male	57%	67%	48%
Prefer not to disclose	3%	4%	2%
Race and ethnicity				p = .142
American Indian or Alaska Native	2%	2%	2%
Asian	5%	2%	8%
Black or African American	10%	7%	14%
Hispanic/Latino/a/x	20%	29%	12%
Multiracial	2%	0%	4%
White	58%	58%	58%
Prefer not to disclose	3%	2%	4%
Region				p = .628
Northeast	18%	18%	17%
Midwest	13%	11%	15%
South	38%	36%	40%
West	29%	36%	23%
Prefer not to disclose	2%	0%	4%
Number of Children < age 18	1.6 (0.7)	1.6 (0.7)	1.5 (0.8)	p = .924
Age Ranges of Children < age 18b				p = .650
Any children 0–10 years	72%	74%	70%
Only children 11–17 years	28%	26%	30%

^an=96; one participant preferred not to disclose age.

^bn=89; 8 respondents were excluded due to inconsistent responses.

^*Significant at p < .05. Some percentage totals do not sum to 100% due to rounding.

Acceptability by firearm ownership.

The mean AIM score for the total sample was 4.35 (SD 0.52), corresponding to high acceptability, with a range of 3–5 (i.e., neutral to high acceptability; Table 2). No significant difference was found in mean AIM scores between firearm owners and non-owners (mean difference = −0.20, 95% CI [−0.41, 0.01], p = .057). Most participants (84%) either agreed or completely agreed that they would recommend that a friend receive S.A.F.E. Firearm from their child’s pediatric primary care clinician, with greater agreement among firearm non-owners compared to owners (mean difference = −0.34, 95% CI [−0.67, −0.01], p = .046).

Table 2.

Survey Participant Scores on the Acceptability of Intervention Measure (AIM), by Parent Firearm Ownership Status and Demographic Characteristics

	Acceptability Scorea			Acceptability Differences Between Groups
	Mean	SD	Range	Test statistic (DF), p-value Standard error difference (SE)
Full sample	4.35	0.52	3–5
Parent firearm ownership status
Firearm owners	4.24	0.45	3–5	t(95) = -1.93, p = .057
Firearm non-owners	4.45	0.57	3.25–5	(SE 0.11)
Parent gender
Male	4.37	0.48	3.25–5	t(92) = 0.08, p = .934
Female	4.36	0.58	3–5	(SE 0.11)
Parent race and ethnicity
American Indian or Alaska Native	4.75	0.35	4–5	F(5, 88) = 0.54, p = .747
Asian	4.35	0.49	4–5
Black or African American	4.3	0.52	3.5–5
Hispanic/Latino/a/x	4.29	0.61	3–5
Multiracial	4.75	0.35	4.5–5
White	4.38	0.51	3.25–5
Child age
Any children 0–10 years	4.45	0.51	3–5	t(87) = 2.00, p = .049*
Only children 11–17 years	4.21	0.48	3.25–5	(SE 0.12)
Region of residence
Northeast	4.40	0.57	3.25–5	F(3, 91) = 1.27, p = .258
Midwest	4.10	0.56	3.25–5
South	4.43	0.52	3–5
West	4.36	0.44	3.5–5

^aScores represent the average of four items with possible values of 1 (completely disagree) to 5 (completely agree). Higher scores indicate greater acceptability.

^*Significant at p < .05

Firearm storage practices.

Of firearm owners, 67% (n = 30) owned a handgun only, 4% (n = 2) owned a long gun only, and 29% (n = 13) owned both. When asked how many of their firearms had been stored locked, unloaded, and with ammunition locked separately over the past 2 weeks, 37% of handgun owners reported storing all their handguns this way, 54% reported storing some of their handguns this way, and 9% reported storing none of their handguns this way. Among participants who owned long guns, these percentages were 47% (all), 40% (some), and 13% (none), respectively.

Intention to change behavior.

Among firearm owners who did not always store all of their firearms locked, unloaded, and with ammunition stored separately (n = 28), the majority (64%) agreed or completely agreed that they would change the way firearm(s) are stored in their home/property after receiving S.A.F.E. Firearm (mean agreement = 3.86, SD = 1.11, range 1–5).

Relationships between acceptability and key variables.

No significant differences in mean AIM scores were found by parent gender, race and ethnicity, or region. Mean AIM scores were significantly higher for parents of younger children as compared to parents who only had adolescents ages 11 or older (mean difference = 0.24, 95% CI [0.001, 0.47], p = .049).

Patient-Doctor Depth-of-Relationship score and AIM score were positively correlated (r = 0.21, p = .038). In contrast, the association between AIM score and trust in the pediatrician to counsel on firearm safety was not statistically significant (r = −0.09, p = .390). Most participants either strongly agreed or agreed that they trusted their pediatrician to counsel them on what is best for their children’s health (95%), vaccines (88%), and firearm safety (75%).

Qualitative Interviews

Participants.

Eleven participants completed an interview; the remaining 23 participants who completed the survey and expressed interest in an interview did not respond to outreach. Mean interview duration was 38 minutes (range 27–45 minutes), and all participants completed a full interview. Over half of the participants identified as female (64%) and half identified as white (64%). Five participants described living in communities in which firearm ownership is common. Three participants owned firearms, and 8 participants did not. Of the 8 non-owners, 4 spontaneously shared during the interviews that they were considering acquiring a firearm or becoming an owner in the future (e.g., “I can relate because I’m tempted to also get a gun for my house,” Participant 08 [P08]).

Content analysis resulted in 5 broad categories explaining factors that shape parents’ views on the acceptability of S.A.F.E. Firearm; these converged on the quantitative results. Categories included pediatric clinicians’ role, quality of delivery, neutral messaging, tangible resources, and opportunities to enhance adaptation. We identified several subcategories within these broader categories (Table 3).

Table 3.

Categories and illustrative participant quotes

Categories and subcategories	Illustrative participant quotes
1. Pediatric clinicians’ role
1.1 Role boundaries	Well, I think it makes a lot of sense for a doctor [to] bring that up. […] my oldest is 14, and I’ve never once had a doctor ask me anything about gun safety. But it really does make sense. (P05) I never would have thought about a doctor doing that, but anything to help save a child or anyone’s life matters. (P02)
1.2 Credibility as messengers	Well, I think it’s great for pediatricians to talk about something like that. I think it’s a good place to do it, because you’re kind of already there for their expert advice. And I think that parents would probably be more open to listening from someone like a pediatrician who, sort of, may be neutral in terms of their feelings about guns. (P06) Well if I had just met the pediatrician, if it was my daughter’s first visit or whatever the case […] I would think it was too intrusive into my life. […] I do think the bond helps. (P02)
2. Quality of delivery
2.1 Positive and skilled delivery	Not all healthcare providers come across that way, but I think her [clinician in video] demeanor is critical in the receptiveness of the patient or patient’s parent. It seems like she’s compassionate and may be empathetic toward the family. (P07)
2.2 Unskilled delivery and unintended consequences	When you start berating people, you make them feel bad, and then it’s almost like you’re saying, “Well you’re not even a good parent cause you’re not locking up your firearm properly.” (P11)
2.3 Role of time	[I]f I have a question – or to really be on my toes about asking it – cause I know our time is sort of precious. (P06)
3. Neutral messaging
3.1 Provider recognition of protection motives	Approaching it from a non-political perspective, framing the problem specifically around child safety and not around trying to shame them or anything for having a gun in the home, specifically stressing empathy for the parent’s desire to keep their family safe, cause that’s an important concern usually. (P04)
3.2 Child safety framing	Keeping my children safe is more important than whether or not I believe we should have guns, or the right to own guns, or how safe guns actually are. [...] It’s not a conversation about your political views on guns. It’s the fact that gun ownership is a thing, your children may be exposed to people who own guns, so therefore, here’s all the information you need to make sure your children are safe. (P09)
4. Tangible resources
4.1 Identifying problems and solutions	I think it’s important that you’re not only preaching it but having the tools there for what you’re preaching and they’re free. (P02)
4.2 Reducing barriers	I mean life just gets in the way of doing things you’re supposed to do. Sometimes work and other responsibilities come up and […] they run out of time, and they say they’ll do it later. (P02)
4.3 Providing information for alternative storage options	Especially when they talk about how the information in the pamphlet regarding all different types of guns and different kinds of gun locks for various things, it’s very inclusive of and anticipatory of possible concerns that gun owners might have about the issue. (P04)
5. Opportunities to enhance adaptation
5.1 Notice prior to appointment	What would make it easier is if we got an email or a text or a letter in the mail prior to the visit, giving us a heads up saying, “We are now discussing firearm safety as a part of the well-child check. Just want to make you aware of that. Should you have any questions or concerns, feel free to ask your healthcare provider during your visit.” It would be nice to know that ahead of time. (P07)
5.2 Resources for talking about firearms	Just knowing that this program exists gives me thoughts about conversations that I maybe need to have […] So maybe this is a good way to help with that conversation just making sure your kids are safe and giving the parents a resource to help them have these conversations. (P09)

Pediatric clinicians’ role.

The pediatric clinicians’ role in promoting firearm safety was described through role boundaries (1.1) and credibility as messengers (1.2) subcategories. Specific to role boundaries, several participants had not previously considered that pediatricians could provide firearm counseling and firearm locks but noted the importance of the topic given the seriousness of firearm injuries. Most participants endorsed that S.A.F.E. Firearm delivery by pediatric clinicians was appropriate.

In the second subcategory, pediatric clinicians’ expertise in child health and treatment of firearm-related injury sequelae, as well as their relationships based on trust with patients and families, enriched their credibility as firearm safety messengers, and thus enhanced program acceptability. Participants identified their pediatric clinician as a trusted source of information, indicating that it “makes sense” (P09) for clinicians to deliver safety counseling given that families attend appointments for their “expert advice” (P06). Several participants noted that the relationship with the clinician would impact S.A.F.E. Firearm acceptability, triangulating the quantitative data. In particular, the longitudinal aspect of clinician-family relationships and subsequent connection mitigated perceived intrusiveness. Given limited time with their clinician during the well-visit, clinicians’ prioritization of firearm safety signaled to parents the importance of the topic.

Quality of delivery.

The clinician’s approach to the counseling component of the program also shaped participants’ views on overall program acceptability; parents’ reactions to the video generated particularly nuanced insights in this category. Key subcategories included positive and skilled delivery (2.1), unskilled delivery and unintended consequences (2.2), and role of time (2.3). In the first subcategory, participants voiced approval of the motivational interviewing techniques demonstrated in the video. Specifically, participants appreciated how the clinician asked for permission and elicited responses from the parent. They appreciated the collaborative approach: “let’s do this together” (P10) and “‘we’ attitude instead of ‘you need to’” (P11). Participants characterized the clinician’s approach as informative, thorough, empathetic, and understanding. They also indicated approval for features of shared decision-making, particularly the emphasis on personal choice (not “make mandatory” or “scare her,” P04), and listening to the parent’s concerns, which amplified acceptability.

While participants broadly voiced their approval for the program, many speculated that unskilled clinician delivery could lead to potential unintended consequences, which could subsequently impact parents’ receptivity to S.A.F.E. Firearm. Participants noted that lack of empathy and caution could generate defensiveness or alienate parents. One participant indicated that an unskilled approach may result in parents taking offense and responding with “that’s my child, you don’t tell me how to raise them” (P02). Similarly, other participants noted that in their approach to firearm counseling, clinicians could inadvertently signal judgments related to good or bad parenting depending on their delivery style. For example, a non-owner explained that parents may feel threatened if clinicians imply “that you’re not a good parent, you’re not a parent that cares about safety” (P07), thereby undermining the intent of program delivery and parents’ receptivity to it.

The reality of time constraints was woven through participants’ perspectives on S.A.F.E. Firearm. Several participants noted the discrepancy between ideal program delivery and the hurried reality of primary care. One participant expressed concern that hasty clinician counseling on this “hot button” issue could lead to “conversational explosions” (P03) among unreceptive parents. Additionally, several participants voiced that they did not want program delivery to interfere with attention to the primary reason for the visit.

Neutral messaging.

Participants identified the program’s neutral messaging as a key feature of acceptability. Subcategories included clinician recognition of protection motives (3.1) and child safety framing (3.2). In reacting to the video, participants appreciated the clinician’s ability to express understanding of the parent’s desire to keep their children safe from both a firearm and a home intruder. Participants acknowledged the salience of external threats and some parents’ perceived obligation to protect their families via firearm ownership. Clinicians’ recognition of parental protection motives – and subsequent desire for quick access – was cited as important, regardless of participants’ firearm ownership status.

Every participant commented on the importance of child safety. Several identified framing firearm storage as a safety issue with health-related outcomes for children as a positive feature of the program. Several preferred bundling program delivery with discussion of other developmentally relevant preventative counseling topics (e.g., seatbelts) or sensitive topics (e.g., substance use). Participants expressed appreciation for straightforward, universal information provision. Child safety served as a neutral, agreeable message that cut through potential political tensions.

Tangible resources.

Participants voiced appreciation for the tangible components of S.A.F.E. Firearm, with subcategories including identifying problems and solutions (4.1), reducing barriers (4.2), and providing information for alternative storage options (4.3). Overall, participants expressed notable approval for the offer of cable locks and liked that both a pamphlet and explanation accompanied the lock. Linking counseling with a tool (i.e., lock) was considered meaningful; two participants specifically indicated their appreciation that the program extended beyond conversation and offered “a solution” (P03, P11). Moreover, providing locks without requiring disclosure of ownership status was perceived as increasing parents’ comfort by maximizing privacy.

Participants highlighted how S.A.F.E. Firearm reduced barriers to secure storage, including financial barriers: “That sounds too good to be true. ‘Cause a lot of things, you don’t get things for free” (P07). Despite the relatively low cost of cable locks, participants still noted that they present an expense. Free lock provision conveyed the message that safe storage is important for everyone, regardless of one’s ability to afford or access storage tools. Participants also noted how S.A.F.E. Firearm reduced barriers of delaying or forgetting. Firearm-owning participants also expressed openness to post-visit contact in the form of reminders, as a potentially helpful way to increase follow-through of installing the lock. Preferences for the frequency of contact ranged from once only to no more than once a month. Overall, they were open to email, text, or telephone reminders.

Participants voiced overall positive reactions on the resource pamphlet and appreciated having extra information to bring home, especially for parents who may not feel ready to make a change or preferred not to discuss firearm storage with their clinician. Participants liked that the pamphlet contained information on different types of locking mechanisms for different types of firearms. The emphasis on options highlighted support for less prescriptive approaches that allow firearm owners opportunities to choose the storage option that works best for their family.

Opportunities to enhance adaptation.

Participants identified 2 key opportunities to further enhance S.A.F.E. Firearm: notice prior to appointment (5.1) and resources for talking about firearms (5.2). Three participants (all firearm non-owners) suggested that parents receive advance notice of firearm counseling, to increase feelings of preparation and decrease feeling “caught off guard” or pressured during the clinical encounter. Several participants indicated interest in additional information for talking to children about firearm safety, as well as talking to friends and family members about firearm storage, as the program had prompted them to think about initiating key safety conversations.

DISCUSSION

The objective of this mixed methods study was to develop a comprehensive understanding of parents’ views on the acceptability of an adapted firearm safety program, S.A.F.E. Firearm, to ensure usability, optimize effectiveness as a universal suicide prevention strategy, and minimize unintended consequences. The study also offered a final check of our systematic adaptation process prior to testing in a hybrid effectiveness-implementation trial. Convergent parent (i.e., end user) feedback across both firearm owner and non-owner participants confirmed that S.A.F.E. Firearm is highly acceptable and ready for implementation in pediatric primary care. Participants’ identification of pediatric clinicians as credible messengers, endorsement of a collaborative approach to decision-making, approval of neutral messaging, and appreciation of tangible resources (e.g., free cable locks) were key factors that contributed to the high acceptability scores. Most firearm-owning survey participants agreed or completely agreed that they would change the way their firearms are stored after receiving S.A.F.E. Firearm, suggesting its potential effectiveness for changing behavior. Moreover, the absence of differences in acceptability scores across parent firearm ownership status, race, ethnicity, gender, and region of residence point to acceptability across a diverse group of stakeholders, confirming that stakeholder feedback has been adequately integrated into S.A.F.E. Firearm. Finally, half of the interview participants who did not currently own firearms volunteered that they were considering a firearm purchase, underscoring the importance of a universal prevention approach.

Findings of high trust in pediatric clinicians suggest that clinicians’ roles as caring experts in child safety is meaningful. This finding echoes previous stakeholder recommendations that clinicians approach firearm counseling from the position of developmental expertise rather than firearm expertise.^23,46 In line with this, both quantitative and qualitative data highlighted a positive association between the quality of the parent-clinician relationship and S.A.F.E. Firearm acceptability; longitudinal family-clinician relationships may enhance S.A.F.E. Firearm acceptability. Parent-clinician mutuality and rapport cultivated over the course of the relationship⁴⁷ may attenuate the potential for program content to feel intrusive.

In the context of overall high acceptability scores, mean scores among parents who had at least one child under age 11 were modestly higher than those of parents with only adolescents in the household. This may have limited clinical significance or could reflect a tendency for parents with younger children in the household to anchor on unintentional injury, whereas discomfort around suicide-related stigma⁴⁸ could be present in parents of adolescents without young children. Examination of unique family considerations is an important avenue for ongoing inquiry specific to firearm storage messaging,⁴⁹ and this is an area where individualized discussion will be especially important.

Our findings suggest that using an established adaptation model (ADAPT-ITT) may enhance family-centered care through explicit integration of parents’ needs and preferences into program refinements. Family-centered care emphasizes a parents-as-partners paradigm for health care decision-making and is considered a core standard of care in pediatrics.^50,51 S.A.F.E. Firearm refinements build on Safety Check’s foundational elements of information sharing, respecting and honoring differences, partnership and collaboration, and negotiation.⁵¹ Additionally, S.A.F.E. Firearm’s program-level characteristics (e.g., emphasizing the shared goal of keeping children safe and offering additional resources from firearm experts) align with needs and preferences of both firearm-owning and non-owning parents.

Our findings map onto major themes generated from engagement with stakeholder groups throughout the adaptation process.^17–19 First, triangulation of prior findings with these parent perspectives adds additional depth to our understanding. Second, we learned that most parents are open to limited post-visit firearm storage reminders via email and text message. Third, we gained insights into other opportunities to enhance adaptation, like providing pre-visit notice about the program and sharing resources for talking with other parents about firearms. Parents’ emphasis on quality of delivery highlights the importance of this key element of fidelity and will inform clinician training prior to implementation.⁵² Finally, time – a ubiquitous implementation determinant in clinical settings⁵³ – continues to be noted as a potential barrier, reinforcing that baseline S.A.F.E. Firearm delivery must be brief, with longer duration determined by parent priorities.

Our study has several limitations. First, our MTurk survey utilized a convenience sample and may not be fully representative of the general population of parents attending pediatric well-child visits. These concerns are mitigated by the fact that participants’ firearm storage practices mirrored those of a nationally representative sample in which only 3 in 10 households with children indicated storing all firearms in the safest manner (i.e., unloaded and locked),⁸ and by the fact that our survey sample was majority male, which is unusual for parent-focused studies. Further, the sample included racial, ethnic, and regional diversity. A larger sample size may have made the difference in acceptability between firearm owners and non-owners significant, but the small absolute difference in overall high acceptability score is unlikely to be a meaningful barrier to implementation. Second, our interview sample may have been impacted by a selection effect, as parents with a greater interest in firearm safety may have pursued interview participation. In addition, despite the goal of a balanced sample of firearm owners and non-owners, our interview sample was skewed toward non-owners. However, some non-owners’ statements that they were considering becoming an owner in the future supports transferability of the findings. Despite the relatively small sample size, we reached saturation (i.e., informational redundancy).⁵⁴ Third, we assessed acceptability based on a Think Aloud exercise with a simulated video discussion; acceptability following actual receipt of the program may differ.

This work also has key policy implications. Multiple federal agencies including the Departments of Defense, Health and Human Services, and Veterans Affairs are creating plans for addressing lethal means safety awareness, education, and training.⁵⁵ Congress is considering legislation to support evidence-based training on youth suicide prevention for health care professionals, in addition to creating a centralized hub with best practices in lethal means safety and suicide prevention.⁵⁶ S.A.F.E. Firearm aligns with these efforts and is an important component of the youth suicide prevention toolkit; successful implementation is key for impact.

CONCLUSION

This study focusing on parent perspectives served as the final check in our iterative process of adapting an evidence-based firearm storage program to reach parents of both young children and adolescents in pediatric primary care. The adapted program now serves as a universal firearm suicide and unintentional firearm injury prevention strategy. The findings not only confirm that S.A.F.E. Firearm is acceptable to parents, but they also enhance our understanding of parents’ nuanced views and provide insights for pre-trial preparation.

An upcoming trial will test the most effective way to implement S.A.F.E. Firearm in two health systems in Michigan and Colorado.⁵⁷ Half of the clinics will be randomized to receive electronic health record reminders (Nudge); the other half will receive Nudge plus one year of practice facilitation to target clinician and organizational implementation barriers (Nudge+). In addition to testing implementation strategies, both the acceptability and effectiveness of S.A.F.E. Firearm will be evaluated. Pilot work for the trial has led to further enhancement of training materials, and additional work is planned for adaptations for different subpopulations. We anticipate that the trial will improve implementation of firearm safety interventions and identify the best approach to national implementation.

Appendix A

MTurk Methods and Quality Assurance Strategies

We collected survey data through the Amazon Mechanical Turk (MTurk) online platform, which matches people (referred to as “Workers”) to tasks called Human Intelligence Tasks (HITs). After a Worker completes a task, the person who posted the task (Requestor) can reject or accept the Worker’s submission based on whether it was completed to the Requestor’s satisfaction. Workers accumulate a HIT acceptance rate, which is the number of HITs accepted divided by the total number of HITs completed. Requiring a high HIT acceptance rate as a study eligibility criterion lowers the risk of survey completion by Internet bots.

To maximize the likelihood of high-quality responses, we required participants to have an MTurk HIT approval rate > 95% and to have previously completed >100 MTurk tasks, as in our prior survey work.¹ To obtain responses reflecting our target population, we required participants be located in the U.S. or U.S. minor outlying islands. To obtain new feedback, we required participants to have had no prior participation in our research team’s other firearm-related surveys, which were collected as part of an earlier step in S.A.F.E. Firearm adaptation. Individuals meeting these criteria were able to view a pre-screening survey in MTurk to further assess eligibility.

For additional quality control, our pre-screening survey contained questions that were and were not relevant to the research study, with the goal of encouraging truthful responses because participants did not know which questions and responses would make them eligible for the full survey. We integrated skip logic in our survey design to assess for any conflicts in reporting of firearm ownership status between the eligibility screen survey and the full survey. When a conflict in answers was detected, participants were provided options to explain (e.g., answer has changed, made a mistake, was not comfortable reporting on pre-screener) or given the option to end the survey. We also included an attention check (i.e., a question specifically designed to test whether participants were reading questions carefully).

We reviewed completed surveys for internal consistency. We identified domains within the survey instrument in which participants might report conflicting information (e.g., child ages, firearm storage practices), and then counted the number of discrepancies per participant (range: 1–4). Firearm owners had a longer survey to complete and presented with more inconsistencies than non-owners. We ran analyses on our primary measure (i.e., Acceptability of Intervention Measure) with (1) a sample that excluded participants with any discrepant responses and (2) a sample that excluded participants with ≥2 discrepant responses. The results were equivalent, and thus we elected only to remove participants with ≥2 discrepant responses from the final analytic sample so as not to increase the difference in sample size between firearm owners and non-owners, and thus biasing the sample away from firearm owners.

There is notable variation in payment amounts for HITs posted on MTurk, but payments tend to be quite low (approximately $2 per hour on average).² Researchers have recommended that those conducting research on MTurk should pay workers at least the federal minimum wage of $7.25/hour.³ Therefore, for the pre-screening survey (estimated to take 30 seconds), participants were paid $0.06 for participating, regardless of their eligibility for the full survey. Participants were paid $4.83 for the full survey, which was estimated to take up to 40 minutes to complete. Because the qualitative interview was more intensive than the typical task posted on MTurk (which usually involve online tasks), we offered higher compensation. The interview was expected to take up to 1 hour to complete and participants were compensated $10 in the form of a bonus on MTurk.

Appendix B

S.A.F.E. Firearm Video Transcript

Pediatrician: That’s excellent, I’m so glad that you guys are using seatbelts and encouraging the kids to always wear their bike helmets. You know, another safety issue that I’ve been discussing with all of my families recently is around the safe storage of firearms. Because we see so many kids who are impacted by firearm injuries in our practice, it’s become such an important issue. And a lot of our patients do have guns in their homes – but even the ones who don’t will often come across guns in other people’s houses, like friends or relatives. So, as a practice, we’ve begun to discuss gun safety with all of our families, whether or not they actually own a firearm. We’ve learned so much from firearm experts for how to keep kids safe – I’d love to share some of that with you. Are you OK having that conversation?

Parent: Okay (looks a little uneasy).

Pediatrician: You know, a lot of my patients don’t want to be asked whether they own a firearm or not, so it’s completely up to you whether we have a general conversation about firearm safety, or we could have a conversation that’s only more specific to your circumstances. And I don’t need to document any of this in your child’s medical record.

Parent: Okay.

Pediatrician: Great. So, we know that in order to prevent injuries and accidents from firearms, it’s incredibly important to store every firearm in the home safely. What do you think – what are your thoughts about keeping guns locked in places where kids spend time, whether it’s your home or the home of a friend or a family member?

Parent: (Pause.) Well, we do have a handgun in our home, but I keep in a place that the kids don’t know about.

Pediatrician: Thank you so much for sharing that. I think it can really help inform the conversation to know whether you have a firearm and where you’re keeping it. So it’s clear that you’ve thought a lot about keeping this gun safe and keeping it away from the kids, which is excellent. Is it OK if I ask, how specifically do you store the gun?

Parent: Yeah, I keep it in a spot where the kids don’t know about, safely, in my bedroom.

Pediatrician: I’m glad they don’t seem to know where it is. You know, one thing we worry about is how curious kids can be – they often go searching to see what they can find in every corner of the house. I know for sure this happens in my own home. So, is it okay if we talk a little about options for safe firearm storage? I’ve been sharing this with a lot of my families recently, and people have found it incredibly helpful.

Parent: Okay, but, you know, ours is definitely stored in a place that only I know about, so I think it’s safe, and it’s really important to me that I have easy access to it in order to protect myself and my family in case there’s an intruder.

Pediatrician: Yeah, I think that is true for a lot of people. You know, if the main reason you own a firearm is for the protection of yourself and your family, then it makes sense – you need to be able to get to it quickly in case of an intruder. You certainly wouldn’t want safe storage to get in the way of being able to access the firearm easily.

Parent: Yeah.

Pediatrician: Yeah, so it really does sound like you’re thinking about both: how to keep your family safe from an intruder, and how to keep the kids safe from the gun. So, one of the reasons I’m a little bit worried is that so many children I take care of have found guns in their homes, even when the parents were absolutely positive that they would never be able to find it.

Parent: Mm, Okay...

Pediatrician: And you know, the reason I bring all this up is because it’s absolutely clear that you want to keep your kids safe, and we know that especially with kids who are really curious, locking up the firearms goes a long way towards reducing the risk of suicide, and also of accidental shootings in children. So why don’t we see if we can troubleshoot this together. I think we could figure out some ways to both keep the kids safe from the firearm while still protecting everyone from an intruder. A lot of my patients do find that with the right storage method, it can actually be pretty quick and easy to unlock a firearm in case of an emergency. What a lot of people don’t realize is, that safe storage does not actually have to get in the way of accessing the firearm in case you should happen to need it.

Parent: I hadn’t thought about it that way before. I guess having the gun and keeping it locked up safely are both really important for keeping my kids safe.

Pediatrician: Right, absolutely. And it’s definitely possible to have access to the gun when you need it, and to keep it locked away when you’re not using it. So what are your thoughts so far? I mean, do you think that locking the firearm could make sense?

Parent: (Hesitating.) I think it’s probably a good idea.

Pediatrician: Clearly it’s your choice. Is there anything in particular that’s making you hesitate?

Parent: No, just the logistics. I’ll need to figure out how best to do it.

Pediatrician: Yeah, like what are the specific things that you think that could get in your way?

Parent: Just, you know, I’m really busy, like when will I find the time to go get one, and the right one, and I don’t know where to buy one and how to use it. That sort of thing.

Pediatrician: Yeah, those are all such excellent questions. So, the good news is that we actually have gun locks available here in the clinic, completely free of charge. And you’re welcome to take as many as you need.

Parent: That’s great – thank you for that.

Pediatrician: If you prefer, all police departments also offer cable locks for free. So we can teach you how to use the lock here. We have a pamphlet that walks you through exactly how to use it with several different types of firearms. The pamphlet also includes links to videos that shows how to use cable locks properly, and the pamphlet gives information about other types of locking options too. It was created by Project ChildSafe, which is an organization that’s part of the National Shooting Sports Foundation, and they have a mission of promoting firearm safety. So would you like a cable lock and one of those pamphlets?

Parent: Yeah, that would be awesome. Thank you.

Pediatrician: Awesome. I know that was really a lot of information. How is all this sounding to you?

Parent: Sounds pretty good, I think!

Pediatrician: That’s great. How confident are you feeling about actually making the change?

Parent: I think I can do it.

Pediatrician: Excellent, I know it’s a really big change, and I do want to be sure that you’re completely comfortable that it will work for you and your family. I would definitely encourage you to use the resources in the pamphlet. There’s a lot of information there, which can help make the whole transition easier. And remember, we’re also here in the office. So if you get home and you realize you have a question, or you want to talk some more, you can always give us a call. Just ask to speak to the nurse or leave a message for me, and I’ll call you back. Another thing, also, that we’re offering is a reminder phone call in about a week or so. People get so busy, and we all tend to forget or put things off. But you know, I really worry that every day a firearm remains unlocked is a day that something could happen. So would you like me to arrange one of those reminder calls?

Parent: I don’t think so – I’m okay.

Pediatrician: Okay, no problem – just let us know if you change your mind.

Parent: Okay, thank you.

Pediatrician: Excellent, so let’s get to the rest of the visit!

Appendix C

Interview Guide

Video Think Aloud

First, we will play the S.A.F.E. Firearm video again. Instead of watching it all the way through, though, this time I will ask you to pause the video at a few points and ask you to react to what was just said in the video. Are you ready to begin?

Okay great. You can press play on the video now. A couple of minutes into the video, you will see a screen appear that says, “Please pause the video now.” When you reach this screen, please pause the video and let me know.

Video Think Aloud

[Keep track of how much time has passed and confirm the video is paused at the appropriate pause point if needed – all pause points are labeled with a #]: 1. What are your reactions to what you just heard in this chunk of the video?  a. Probe: What did you like or not like?  b. Probe: What felt acceptable or unacceptable?  c. Probe: Can you tell me more, elaborate on that, talk through why you’re thinking that?  d. Probe [if having trouble answering]: How do you think your friends or neighbors might respond or feel about this? After Pause Point #1 and #2: Okay, you can resume the video now. Please let me know when you reach the next pause screen. After Pause Point #3: Okay, you can resume the video now. Please let me know when you reach the end of the video.

Note to interviewer: ask these questions at each designated pause in the video. You will ask these questions three times: after Pause Point #1 (1:03), Pause Point #2 (1:55), Pause Point #3 (4:29), and after the video is over (6:43).

Post-Video Interview Questions
1. What is your general reaction to this program as a whole?
2. What is your main takeaway from this program as a whole?  a. Probe [if confusion]: If you were leaving this conversation, what would be the key points in your mind that you’re remembering or taking away from the conversation? What were the salient points to you?
3. Did you feel like any information was missing from the conversation that you would have liked to be included?  a. Probe: What you would like included?
4. If you had a similar conversation with your pediatric clinician, would you have any questions for them? a.  Probe: What would you ask?
[If participant is a firearm owner, proceed below.]
These next few questions pertain to your specific situation. Before we dive in, may I ask how many firearms you have and how they are stored today? If participant asks why: It can help us have a more specific conversation about how this conversation might apply to you and your situation. But it’s okay if you want to have a more general conversation instead! If participant expresses concern/hesitation, can say: It’s okay if you don’t want to say, we can just talk about these questions in a more general way.
6. Imagine that you chose to take a lock from the clinic. How would you feel about receiving a reminder to use the lock?  a. Probe: For example, something to bring home or follow-up contact?  b. Probe: Can you think of any sort of reminder that would be helpful?  c. Probe: Would you want to receive any follow-up contact from the clinic after the firearm safety conversation to remind you to use the lock?   a. Probe [if positive]: What format would you prefer (e.g., phone, email, text)?   b. Probe [if positive]: How often and from whom?   Probe [if negative]: Why? How would you feel about a phone call reminder? How about something to bring home?
7. Is there information that you would like left out of the discussion?  a. Probe [if needed]: For example, some people find statistics or personal stories helpful while others don’t.
[Ask everyone these questions.]
8. Has your pediatric clinician ever discussed firearm safety with you?  a. Probe [if yes]: How did that conversation go?
9. If your pediatric clinician had a conversation with you that was similar to the one demonstrated in this video, how would you feel?  a. Probe: How would you react?  b. Probe: What would you say?  c. Probe [if yes to #8]: What felt similar or different about the program presented in this video compared to the discussion with your pediatric clinician?  d. What would make it easier to talk to your pediatric clinician about firearm safety?  e. What would make it harder to talk to your pediatric clinician about firearm safety?
[Ask everyone these questions.]
10. When in the well-child visit would you prefer to have this firearm safety conversation?
11. We know you just saw this conversation taking place with a doctor, but we recognize that there are many different types of providers in primary care offices. Within the primary care office that your child(ren) goes to most often, who would you prefer to discuss firearm safety with?  a. Probe (if any confusion): For example, would you prefer to talk with the doctor, nurse, or someone else who works at the clinic?  b. Probe: Why this person?
12. What are your thoughts on this firearm safety conversation happening virtually during a telehealth session as opposed to in-person?
13. How often (if at all) do you think pediatric clinicians should have firearm safety conversations with each family?

Is there anything else you’d like to share with us?

Okay great, thank you for your time. We will credit your MTurk account with a $10 bonus within the next 24 hours.