The development and reliability of a national survey of police officers regarding the enforceability of cell phone use while driving laws

Abstract

Even though numerous types of cellphone use while driving (CPWD) laws have been passed in the United States, it does not appear that these laws are heavily enforced by police. Two studies conducted among officers from West Virginia and Washington states found that numerous barriers to CPWD enforcement exist. However, no studies have investigated whether barriers to enforcement exist for officers nationally. The purpose of this study was to design a national survey of police regarding the enforcement of CPWD laws and to assess the survey's reliability. The survey was designed based on the existing literature, cognitive tested, and assessed for face validity among experts. Officers were recruited via convenience sampling to assess the survey for test-retest correlation, test-retest agreement, and internal consistency reliability using standardized Cronbach's α and weighted Kappa statistics. Among participants, the test-retest correlations for most sections of the survey showed acceptable correlation (Cronbach α's > 0.7) (N = 6). For test-retest agreement, the median Kappa equaled 0.77 (N = 6). As for internal consistency reliability, responses thought to be associated showed acceptable correlation with Cronbach α ranging from 0.41 to 0.94 (N = 353). Despite the limited number of officers that participated in test-retest correlation and agreement, the survey appeared quite reliable.

1. Introduction

Cellphone use while driving (CPWD) is a dangerous and prevalent behavior in the United States (AAA Foundation for Traffic Safety, 2019). Consequently, most states and some local governments have enacted laws to curtail cellphone use among drivers (Insurance Institute for Highway Safety, 2020). While there is variation, most states passed one or more of the following types of laws: universal texting bans (UTB), universal hand-held cellphone bans (UHB), or young driver cellphone bans (YDB) (Insurance Institute for Highway Safety, 2020). UTB typically prohibit any licensed driver from sending or reading text-based messages on a hand-held cellphone. UHB usually prohibit any licensed driver from making or receiving cellphone calls on a hand-held device. YDB often prohibit individuals of certain ages (i.e. <18 or < 21 years of age) or licensure status (i.e. learners' permits or intermediate licenses) from virtually any interaction with a hand-held cellphone while driving. As of February 2020, 24 states passed UHB, 48 enacted UTB, and 38 ratified YDB (Insurance Institute for Highway Safety, 2020).

Despite the multitude of laws enacted, research suggests that these laws may not be heavily enforced by police (McCartt and Geary, 2004; McCartt and Hellinga, 2007; Maher and Ott, 2013; Rudisill and Zhu, 2016), which can negatively impact the laws' effectiveness (Stanojevic et al., 2013). While research on CPWD law enforcement is limited, two studies conducted among police officers in Washington and West Virginia, which both have enacted all three CPWD laws, revealed that numerous barriers to enforcement exist (Nevin et al., 2017; Rudisill et al., 2019). Common challenges experienced by officers in both states included drivers' perception of risk (i.e. drivers think they can use a cellphone and drive safely), officers' personal beliefs (i.e. if they find a behavior dangerous or important they may enforce it), social norms (i.e. many people use their cellphone constantly), problems with how the laws are written (i.e. laws are too specific and there are ‘loop holes’), and the nature of police work (i.e. officers have other responsibilities besides traffic enforcement) (Nevin et al., 2017; Rudisill et al., 2019). However, it appeared that the largest obstacle for officers in these states was being able to see the driver on their phone, discerning what the driver was doing (i.e. calling, texting, using applications, etc.) and being able to prove that in court if the charge was contested (Nevin et al., 2017; Rudisill et al., 2019).

However, it is unclear whether the challenges encountered by officers in Washington and West Virginia apply to other states as they may have different combinations of laws in effect; it is possible that officers in other states experience different challenges enforcing CPWD laws. Thus, the purpose of this study was 1) to develop a national survey of police officers regarding the enforcement of CPWD laws and 2) to assess the reliability of the survey. By discerning potential barriers to CPWD enforcement among all officers, future and/or current laws could potentially be amended to make enforcement easier for police.

2. Materials and methods

2.1. Survey development

The survey content was informed by previous research regarding the enforcement of CPWD among police in West Virginia and Washington and thru a literature review (Mastrofski et al., 1987; Engel and Calnon, 2004; Schafer et al., 2004; Alpert et al., 2005; Brown and Frank, 2005; Johnson, 2005; Schafer and Mastrofski, 2005; Schafer et al., 2006; Ingram, 2007; Greenleaf et al., 2008; Allen, 2014; Nevin et al., 2017; Rudisill et al., 2019). Several literature databases (i.e. Academic Search Complete, Scopus, Google Scholar, Transport Research International Documentation) were searched for peer-reviewed studies that investigated traffic law enforcement. Based on these findings, potential survey questions were developed and sorted into overarching constructs which included officers' perception of risk, the prevalence of cellphone use among drivers, frequency of citation issuance for traffic offenses, potential factors which influence their decision to pull someone over, current laws in their jurisdiction/state, the adjudication process (i.e. how easy is it in their opinion to prosecute a driver for CPWD), specific challenges of enforcing each of the CPWD laws if they are in effect, ways to improve enforcement, their opinion of Highway Safety Grants (i.e. these are grants issued by the federal government that provide extra funding to agencies to enforce certain traffic violations such as cellphone use), methods to prevent CPWD, and demographics. The design of the questions was modeled after the Traffic Safety Culture Index Survey (AAA Foundation for Traffic Safety, 2019), which is a national survey administered annually by the American Automobile Association Foundation for Traffic Safety, and the Fairfax County Police Department Officer Survey (Virginia), which was developed by the Center for Evidence-Based Crime Policy at George Mason University (Lum and Stoltz, 2018). Demographic questions were modeled after the Behavioral Risk Factor Surveillance System (BRFSS), which is administered annually by the Centers for Disease Control and Prevention (Centers for Disease Control and Prevention, 2019); BRFSS has been found valid and reliable in previous studies (Pierannunzi et al., 2013). Response options mainly consisted of a 5-point Likert scale. The survey was designed to be anonymous, administered in an electronic format, and be completed in ≤20 min.

After the survey was drafted, it was sent electronically to 5 researchers with expertise in distracted driving, enforcement, transportation safety, and injury epidemiology (response rate = 100%). These researchers assessed the survey for face validity and provided commentary on content and design. The survey was revised accordingly with minor adjustments made to questions and/or response options.

Cognitive interviews were then conducted with police officers (N = 7) in three states (WV, OH, PA) (Beatty and Willis, 2007). The officers were recruited via convenience sampling thru professional contacts of the author. The interviews were conducted by the author at the officers' convenience (i.e. time, location) and preferred format (i.e. in-person, via computer technology, or via phone) due to their schedules. The interviews lasted approximately 40 min in duration. These interviews ensured that the questions and responses were clear and logical to members of the target population and that the survey questions were eliciting the intended information. The officers provided comments on the content, survey flow, and question wording. The survey was revised based on participant feedback and consisted of 33 questions (including branching logic).

2.2. Ethics approval

Approval to conduct this study was garnered from West Virginia University's Institutional Review Board (Protocol #1906609479).

2.3. Assessment of test-retest correlation and agreement via pilot testing

In addition to face validity, the survey was evaluated for both test-retest correlation and agreement. This was accomplished via pilot testing with another small group of officers. Individual officers and four police agencies from four states (WI, OH, PA, WV) were recruited via convenience sampling and sent an email soliciting participation in the pilot test. These individuals and agencies were professional contacts of the author and represented a diverse area of enforcement including rural, urban, and suburban localities. Specific locations are not revealed to protect the officers' identities. The inclusion criteria stated that they had to be ≥18 years of age and have >1 year of experience in law enforcement. Officers who were willing to participate were sent an electronic link to the survey (N = 10). Once the officer completed the initial survey, they were re-sent an electronic link 14 days later to retake the survey. Those who did not respond to the second survey were sent two reminders at ~21 and ~28 days after initial survey completion via email. Six officers completed the survey twice (response rate = 60%).

2.4. Internal consistency reliability

Once test-retest correlation and agreement were completed, the survey was electronically released starting in mid-November 2019 thru April 2020. Police agencies throughout the United States were randomly selected and contacted via email, phone, or their social media pages. They were briefly informed about the survey and provided an electronic link to participate; they were instructed to pass the survey information on to other officers as well. Information about the survey was also posted to police-affiliated social media pages and organizations. The National Fraternal Order of Police also released information about the survey to their members. Anyone who was currently employed as a police officer and was ≥18 years of age was eligible to participate. Surveys (N = 353) were received from officers in 31 states.

2.5. Statistical analyses

Since the survey responses were ordinal, to assess test-retest correlation, the responses from officers who participated in the pilot test and responded to both surveys (N = 6) were compared using standardized Cronbach's α; Cronbach's α measures how related two items are (Cronbach and Meehl, 1955; Tavakol and Dennick, 2011). Responses to questions under each of the 12 sections of the survey (excluding demographics) were put on the same scale and summed for each officer. Because most questions were on a 5-point Likert scale, a score of 5 was assigned to a response “Strongly Agree”, 4 to “Agree”, 3 to “Neutral”, 2 to “Disagree”, and a score of 1 to “Strongly Disagree”. Thus, if there were three questions under a section and an officer answered “Strongly Agree” to each question, the score for that section would be 15 (Bergmark et al., 2016). Test-retest correlations were assessed by Cronbach's α and presented for each survey section. To assess test-retest agreement, individual officers' responses to both surveys were compared overall via a weighted Kappa statistic (Sim and Wright, 2005). Kappa statistics measure how well two measurements agree with one another. Weighted Kappa is used for ordinal variables and places more weight on responses that are closer together on the scale and less weight on responses that are farther apart.

Internal consistency reliability was also assessed using standardized Cronbach's α (Cronbach and Meehl, 1955; Tavakol and Dennick, 2011) using responses from the multi-state survey (N = 353). There were several items within the survey that were believed to be correlated based on the limited extant literature regarding CPWD enforcement. It was hypothesized that the officers' responses to the following questions would be correlated because the behaviors were similar: 1) how much they believed that manually texting or emailing while driving was dangerous and 2) how much they believed checking or updating social media while driving was dangerous. There were several items within the section of the survey that inquired about ways to improve enforcement efforts that were thought to be correlated. It was assumed that officers' responses to having a broadly written law and having one general distracted driving law would be comparable; this is because a general distracted driving law would be written broadly. It was also believed that the responses to making laws applicable to all licensed drivers and responses to eliminating age and licensure requirements from CPWD laws would be correlated. In the section regarding prevention, it was assumed that officers' responses to whether technological advances made by cellphone manufacturers could prevent CPWD and whether technological advances made by car manufacturers could prevent CPWD would be correlated. It was also thought that officers' responses to questions asking whether they believed education on the dangers of cellphone use, educating the public on hands-free technology, and educating the public on current traffic laws would be correlated because each had an educational component. All data management and analyses were conducted in SAS version 9.4.

3. Results

The demographics of participants are shown in Table 1. The population mainly consisted of white, middle-aged males. The test-retest correlations for most sections of the survey showed acceptable agreement as the Cronbach αs' were generally >0.7 (Table 2) (Tavakol and Dennick, 2011). The only exception to this was the section pertaining to the frequency of citation issuance for CPWD infractions. In terms of test-retest agreement, the median Kappa equaled 0.77 (Table 2) (Landis and Koch, 1977). As for internal consistency reliability, responses thought to be associated showed acceptable agreement (Table 3). The only exception was the relationship between having a broadly written law vs. having one general law that prohibits hand-held cellphone use; this correlation was slightly lower than originally anticipated (Table 3).

Table 1.

Demographic characteristics of respondents (N = 353).

Characteristic	N	%
Age (years)
18–44	115	39.7
≥45	175	60.3
Missing	63
Sex
Male	254	87.4
Female	35	12.1
Missing	64
Race
White	255	88.5
Other	33	11.5
Missing	65

Table 2.

Test-retest correlations and agreement of survey questions (N = 6)^a.

Construct	Standardized Cronbach α
Perceived danger	0.94
Commonality of cellphone use	0.92
Frequency of citation issuance	0.40
Factors influencing pullover frequency	0.98
Current state laws	0.88
Ease of adjudication process	0.90
Texting ban challenges	0.71
Hand-held ban challenges	0.98
Young driver ban challenges	0.87
Ways to improve enforcement	0.71
Opinion of highway safety grants	1.00
Ways to increase prevention efforts	0.81

^aThe Kappa statistic, which assessed the test-retest agreement between the officers' two survey responses, ranged from 0.65 to 0.84, with the median = 0.77 and average = 0.75.

Table 3.

Internal consistency reliability of survey questions (N = 353).

Responses to question	Standardized Cronbach α
Danger of texting or emailing while driving vs. danger checking or updating social while driving	0.92
Writing laws more broadly helps make them easier to enforce vs. having one general law that prohibits hand-held cellphone use of any kind would make them easier to enforce	0.41
Making laws applicable to all licensed driver makes them easier to enforce vs. eliminating age or license requirements make these laws easier to enforce	0.74
Technological advances made by car manufacturers would reduce cellphone use among drivers vs. technological advances made cellphone manufacturers would reduce cellphone use among drivers	0.94
Educating the public on the dangers of CPWD vs. educating the public on how to use hands-free technology vs. educating the public on what current traffic laws permit may reduce cellphone use among drivers	0.85

4. Discussion

This study sought to design and test the reliability of a national survey of police officers regarding the enforcement of CPWD laws. Overall, the test-retest correlations, test-retest agreement, and internal consistency reliability of the survey questions were adequate. These findings indicate that survey responses were likely stable over time and that the survey was likely measuring what it intended. The only section that showed lower reliability in the test-retest was the one which gauged how often officers write citations for CPWD compared to other traffic violations. This could reflect the fact that many factors can influence an officer's decision to pull a driver over for an infraction and their choice to write a citation versus a warning; previous studies investigating traffic code enforcement found that aspects such as the officer's personal beliefs, demographics and experience, driver's demographics, the environment in which the stop occurred, and even organizational facets of the department in which the officer belongs can influence their decisions (Mastrofski et al., 1987; Engel and Calnon, 2004; Schafer et al., 2004; Alpert et al., 2005; Brown and Frank, 2005; Johnson, 2005; Schafer and Mastrofski, 2005; Schafer et al., 2006; Ingram, 2007; Greenleaf et al., 2008; Allen, 2014). Thus, this set of questions may be more difficult for officers to answer as the frequency of citation issuance is highly situationally dependent.

The weaker correlation observed in the internal consistency reliability between having a broadly written law vs. having one general law that prohibits hand-held cellphone use of any kind can also be explained. It was assumed that a general CPWD law would have to be written broadly to help with enforcement. Due to the multitude of capabilities of cellphones, perhaps officers felt that the three existing types of CPWD laws could each be written more broadly. If that was the case, then the correlation with having one general distracted driving law would likely be lower than expected.

4.1. Limitations

While the findings of this study are important as the literature pertaining to the enforceability of CPWD laws is limited, this study has several limitations. First, the sample size for the test-retest correlation and agreement was very small. Previous studies found that surveys involving police can result in low response rates or samples sizes given the nature of their job (Nix et al., 2019). Secondly, the officers who participated in the pilot test were from four different states and were recruited via convenience sampling. While each of these states had different combinations of CPWD laws in effect, which was desirable, the officers who participated may not be representative of officers within those states nor from other states who were not represented. Also, due to convenience sampling it is also possible that the officers who participated in this study may be fundamentally different from those who chose not to participate, which could create a selection bias. Lastly, only one type of validity measure was assessed in this study.

5. Conclusion

Despite the limited number of officers who participated in the test-retest correlation and agreement, a survey regarding the enforcement of CPWD laws, was developed and appeared reliable. Discerning potential barriers to CPWD enforcement is crucial as it can inform future and existing transportation laws. Research shows that traffic laws with less barriers to enforcement are more likely to be enforced (Lacey et al., 2000) and active enforcement can alter drivers' behavior (Arhin et al., 2016). This is exceptionally important as CPWD is not only dangerous, but extremely prevalent among drivers in the United States (AAA Foundation for Traffic Safety, 2019).

Abbreviations:

CPWD

cellphone use while driving

UHB

universal hand-held cellphone ban

UTB

universal texting while driving ban

YDB

young driver all cellphone ban