Creating a representative sample of small manufacturing businesses for an integrated workplace safety and smoking cessation intervention study

Abstract

Objectives

We aimed to recruit a representative sample of small manufacturing businesses (20-150 employees) for a group-randomized trial of an integrated workplace safety and smoking cessation program.

Methods

An initial sample was drawn from commercial databases, screened for duplicates or ineligibility and contacted. Participating and non-participating businesses were compared on size, location and type. Employee demographics of participating businesses were compared to a U.S. Census Bureau database of similar businesses.

Results

From an initial sample of 2716 businesses, 328 were eligible and 47 (9%) agreed to participate. Participating companies tended to be larger. Employees were similar to employees in the Census Bureau dataset.

Conclusions

Considerable resources were required to identify eligible businesses; commercial databases are the best resource but may not be comprehensive or current. The sample appeared to be representative of small manufacturing businesses in the study region.

Introduction

The workplace has long been recognized as an important point at which to influence personal health behaviors. However, businesses that seek to influence employees’ personal health behaviors such as smoking, exercise and nutrition may be more successful if they first make an effort to minimize workplaces hazards that cause job-related injuries or illnesses.¹ Thus a comprehensive approach to employee health is needed.²

Some large companies and organizations have made progress toward implementing integrated worker health promotion and protection programs.³ Smaller businesses, on the other hand, experience many barriers to implementing such programs, including lack of resources, personnel and expertise.^4-8 Employees of smaller manufacturing businesses, in particular, could benefit from integrated worker health programs due to a high prevalence of both workplace safety risks and poor personal health behaviors.^9,10 This high-risk group constitutes a significant proportion of the workforce. Small businesses with 20-149 workers employ 20% (22,866,725) of the country’s private sector workers; 11% (2,401,908) of the workforce in businesses of this size are engaged in manufacturing occupations.¹¹

The Wellness Works Project is a group-randomized trial to test an integrated workplace safety and smoking cessation program in small manufacturing businesses. It focused on the manufacturing sector because it has high smoking rates for working adults (24%) in comparison to other economic sectors.¹² Production employees were targeted, in particular, because they have the highest smoking rates (29%) and experience the highest rates of occupational injuries and illnesses of all manufacturing sector occupations.^9,12

Randomized, controlled trials are the most appropriate approach for testing comprehensive workplace interventions, but the extent to which their results are applicable to other workplaces relies heavily on investigators’ efforts to identify and recruit a representative sample of businesses.¹³ An intervention shown to be effective in a study with low external validity might be of interest and disseminable to only a small subset of businesses.¹⁴ In order to provide information about external validity, researchers should describe their recruitment procedure; provide the adoption rate (i.e., the proportion of worksites that chose to participate in the study); and compare participating businesses to those that did not to participate.¹³

Some workplace health promotion studies have reported in detail on the process of recruiting sites for the intervention.^5,15-23 These studies found that recruiting worksites is often difficult but effective strategies include screening eligibility over the telephone; recruiting additional sites to allow for early withdrawal; using personal referrals when possible; and offering some minimal intervention activities to control sites.

A very small number of studies have assessed the representativeness of participants by comparing participants with non-participants; none have employed an external database to validate these comparisons. Using information collected from company contacts or available in the original database for a health promotion study involving large (> 200 employees) manufacturing or warehouse/distribution companies, Beiner et al. (1994) found that 27 participating businesses were more likely to be smaller, faster in paying their bills and displaying greater fiscal strength than 64 non-participants.

For a study most like the one described in this publication, Barbeau, et al (2004) found no statistically significant differences in workforce characteristics, presence of safety or wellness programs and attitudes toward health promotion between 26 participating and 105 non-participating small (50-150 workers) manufacturing companies eligible for an integrated workplace safety and wellness intervention study.⁵

This paper describes the resources, practices and protocols used to create a complete database of eligible businesses and recruit a representative set of businesses from that database. To examine whether the recruited businesses were representative of small manufacturing companies in the region, we compared the (1) size, location and product manufactured with those of other small manufacturing companies in the study geographic area; and (2) the demographic characteristics of employees at participating companies with those of all similar-sized manufacturing businesses using Census Bureau data for the same geographic areas and time periods. Finally, we discuss some lessons learned for future studies aimed at identifying and recruiting small manufacturing businesses to intervention trials.

Methods

Study Design

Wellness Works was a group-randomized trial that tested an integrated work safety – smoking cessation program. The 12-month program used a low intensity approach that relied on existing infrastructure and required minimal resources from the participating businesses. The goal was to encourage businesses to address workplace safety problems identified by employees and to motivate employees who use tobacco to quit with the help of resources for smoking cessation available to them through their health insurance or free to all state residents (e.g. telephone quit line resources).

Identifying Eligible Businesses

The study population consisted of small manufacturing businesses in five Minnesota counties - four located in the Minneapolis/St. Paul metropolitan area (Scott, Carver, Ramsey and Hennepin) and one approximately 60 miles south (Steele). Manufacturing businesses were eligible if they employed between 20 and 150 employees, were listed in a business database as manufacturers (Standard Industry Classification codes {SIC} 20-39 or North American Industrial Classification System {NAICS} codes 31-33), and were able to make independent decisions about workplace safety and employee wellness initiatives.

We used two electronic databases available through public and university libraries (D&B Million Dollar Database (D&B) and ReferenceUSA U.S. Businesses Database) to identify potentially eligible businesses. ^24,25 Search criteria included the five targeted counties, number of employees (20-250 in ReferenceUSA and 20-200 in D&B) and industry group (“manufacturing” in ReferenceUSA and “manufacturing indicator” in D&B).

In initial screening, we removed businesses not meeting eligibility criteria (size, location, industry group) or comprising duplicate listings found by sorting by company name and address. Secondary screening involved telephone contacts with remaining businesses to verify size, address, type of business, and names of the human resources (HR) representative and owner or president of the company. Screening was staggered by region over a 27-month period (October 2010 to December 2012). The larger counties (Hennepin and Ramsey) were divided into smaller blocks by zip code to facilitate screening.

We added a small number of businesses to the database throughout the recruitment period through the identification of companies not on our list by county health department personnel or participating businesses.

Recruiting Businesses

Once screening was completed by county or zip code block within larger counties, we sent personalized letters to the HR manager and the president or owner of each company. All businesses received a letter inviting study participation accompanied by a brochure and link to a project web site (www.wellnessworksproject.org). We also invited businesses in four of the five target counties to an event held at a nearby public location. In one county (Steele), a business manager from a participating company helped recruit nearby businesses.

We attempted to reach by telephone each company that had passed initial and secondary screening to explain the study, determine interest and schedule a recruitment visit. We contacted each company at least twice and left messages that included a description of the study and the address of the study website if the person sought was not available. A recruitment visit, the last step in the recruitment process for interested companies, included a description of the study timeline, survey measures and intervention activities. To enroll, the business contact was asked to sign an informal written agreement that detailed both what we required from the company (e.g., work time to perform the surveys) and the program components that we would deliver.

Recruitment continued until the targeted number of businesses or the time period allocated for recruitment was reached. Thus, recruitment stopped before all businesses in all zip code blocks were contacted.

Surveying Employees

We collected baseline survey data at participating businesses between December 2010 and November 2012. Company representatives provided lists of current employees and each employee was assigned a unique ID number. In most cases, research staff administered the survey during work time. For a few companies that did not allow release time from work, we asked employees to place a completed survey in a locked box at the workplace or to mail it to project staff in a pre-paid addressed envelope. We sent a second survey if we did not receive the initial survey within 10 business days.

Surveys included questions about employee socio-demographic and job characteristics; employee perceptions of workplace (organizational) characteristics; and individual employee characteristics and health behaviors. Socio-demographic characteristics included age; sex; ethnicity (Hispanic/Latino or not) and race (White, Black or African American, Asian, Native Hawaiian or Other Pacific Islander, American Indian/Alaska Native, Other); marital status (Married/Live-in partner, Divorced, Separated, Widowed, Never married); and education level completed (Grade School, Junior High, High School, Vocational Training [beyond High School], Some College, College/University Degree, Graduate or Professional Education).

Judging the Representativeness of Participating Businesses

We compared the set of participating businesses to those that declined to participate in order to examine their representativeness on the basis of size (number of employees), location (county) and type of business (NAICS 2- and 3-digit codes). We categorized company size as smaller (20-49 employees) and larger (50-150 employees). Chi-square analyses were used to examine differences between participating businesses and non-participants.

Using data from baseline surveys returned by employees, we compared workforce demographics (race and ethnicity, gender, age and education) of the participating businesses to the U.S. Census Bureau Quarterly Workforce Indicators dataset. The latter was queried using the 2013 beta release of the Local Employment Dynamics (LED) extraction tool using the following search criteria: State = Minnesota; Counties = Scott, Carver, Steele, Ramsey and Hennepin; Time Period = Q4 of 2010 to Q4 of 2012; Firm Size = 20-49 and 50-249 employees; Industry = manufacturing sector (NAICS codes 31, 32 and 33), Firm Age = all ages; Firm Ownership = private; Indicators = EMP (estimated total number of jobs on the first day of the quarter). Separate queries generated datasets for sex (male and female), age (14-18, 19-21, 22-24, 25-34, 35-44, 45-54, 55-64, 65+), education (< high school, high school or equivalent, some college, bachelor’s degree), race (White, Black or African American, American Indian or Alaska Native, Asian, Native Hawaiian or other Pacific Islander, two or more race groups) and ethnicity (Hispanic or Non-Hispanic).²⁶

To compare demographic data with those collected from businesses enrolled in the study, we recoded race and ethnicity information as White, Black, Hispanic, Asian, Pacific Islander, Native American, Mixed or Other and education level as high school or less, vocational training or some college, and college or graduate degree. The similarity of the characteristics of the employees of the participating companies with those of all employees of small manufacturing companies in the study area was tested using chi-square goodness of fit tests.

Results

Eligible Businesses

The initial database contained 2845 businesses. The recruitment period ended before 129 companies could be contacted; thus, the final database contained 2716 companies (Figure 1). The majority (2669, 98%) were obtained from the two electronic databases; a small number of companies not on the databases (47) were identified by county health department contacts or by participating businesses. Initial screening removed 1562 businesses because they were exact duplicates or did not meet eligibility requirements. Secondary telephone screening of the remaining 1154 businesses resulted in the removal of 852 companies; 195 (23%) of these could not be reached after at least three telephone calls and 631 were found to not meet eligibility criteria. The final dataset eligible for recruitment included 328 businesses (Figure 1).

Figure 1.

Flow Chart of the Company Screening and Recruitment Process

Business Recruitment

Of the 328 eligible companies, 26 had not made a decision by the end of the recruitment period (Figure 1). Of the remaining 302 businesses, 255 (84%) declined participation – 222 (87%) of these declined after telephone contact and 33 (13%) after the recruitment visit by study staff. Nearly half of those declining (113, 44%) did not give a specific reason. Of the 142 business that gave a reason, 61 (43%) reported they were too busy or lacked staff to implement the study, 49 (35%) had no interest in smoking cessation and 22 (15%) thought the intervention duplicated existing programs.

Of the 302 eligible and deciding businesses, 76 were sufficiently interested to agree to a recruitment visit, 43 of these enrolled in the study after the visit and 33 declined. Four businesses enrolled in the study after telephone contact only, for a total of 47 participants. Using a conservative standard provided by the American Association for Public Opinion Research (AAPOR) that counts respondents whose eligibility has not been ascertained as non-participants,²⁷ the response rate among eligible and potentially eligible companies was 9%.

Nearly all the eligible businesses were recruited through the process described above, but some were recruited through alternate efforts: two heard of the study through business contacts and contacted study personnel. Many of the eligible businesses (176, 54%) were also invited to a formal recruitment event in their county that included a free meal and worksite wellness presentations; representatives from 20 companies attended events held in October 2010, March 2011 and January 2012. Of these, 18 received a recruitment visit and five enrolled in the study. These events proved to be time-consuming and expensive and yielded few participants. Personal contacts at these events provided information about business concerns and interests but equally useful information was gained at less expense by attending existing community and professional meetings.

There was considerable variation in the time between first contact with a company and formal enrollment in the study. The mean length of time was greater than one month (43 days) but a quarter of the companies (26%) took 100 days or more to sign an agreement and to begin the study. There was no significant difference by size of company in the length of time between first contact and formal enrollment.

The type of company representative who served as our contact during the recruitment process varied significantly with the size of the business. In the 20 smaller participating companies (employee size < 50), the CEO, CFO or Comptroller was our first contact and remained the main contact in nearly half the cases (45%). In the 27 larger companies (employee size > = 50), a Human Resources (HR) manager was our first contact and remained the main contact in 81% of the cases.

Representativeness of Participating Businesses

Comparison of Participating and Non-participating Businesses

Slightly more than half of all eligible businesses (n=328) were in the smaller category (i.e., 20-49 employees) (182, 55%) (Table 1). Larger businesses (i.e., 50-150 employees) were somewhat more likely to agree to participate than were smaller ones, but this difference did not reach significance (p = 0.054).

Table 1.

Comparison of the characteristics of all eligible businesses with those of participating businesses

	Distribution of All Businesses Eligible to Participate (n=328) n, %	Distribution of all participating businesses (n=47) n, %	p*
Company Size (number of employees)
Smaller (20-49 employees)	182 (55%)	20 (43%)
Larger (50-150 employees)	146 (45%)	27 (57%)	0.054
County (Minnesota)
Carver	33 (10%)	2 (4%)
Hennepin	173 (53%)	24 (51%)
Ramsey	86 (26%)	14 (30%)
Scott	25 (8%)	0 (0%)
Steele	11 (3%)	7 (15%)	<0.001
North American Industry Classification System (NAICS) 2-digit code
NAICS 31: Food, beverage, tobacco,  textiles, apparel and leather  manufacturing	13 (4%)	1 (2%)
NAICS 32: Wood products, paper, printing,  petroleum and coal products, chemicals,  pharmaceuticals, paint and other coatings,  plastics and rubber products, glass,  cement, nonmetal mineral products  manufacturing	93 (28%)	18 (38%)
NAICS 33: Metal products, machinery,  computer and electronic products,  electrical equipment, transportation  equipment, furniture and related products  manufacturing	222 (68%)	28 (60%)	0.233

^*Differences between companies that participated and eligible non-participants.

Most (79%) of the eligible companies were located in Hennepin and Ramsey Counties, the two largest counties in the Twin Cities metro area, and the distributions of participating companies in these counties were quite similar to the distributions of eligible companies in those counties (Table 1). However, there was a significant difference in participation by county (p<0.001) due to high and low participation rates in the outlying counties. The highest participation rate (7 participating companies / 11 eligible: 64%) was in Steele County, the only county located outside the Twin Cities metropolitan area. The two counties in the outlying Twin Cities metro (Carver and Scott) had low participation rates (4 and 0%, respectively).

About two-thirds of eligible companies (222, 68%) were in NAICS code 33, which includes manufacturers of primary metal and metal products (e.g. machinery, computer and electronics, electrical equipment, appliances, metal furniture and miscellaneous metal products). Most of the remaining eligible businesses (93, 28%) manufactured non-metal products (NAICS 32) (e.g., paper manufacturing, printing, or manufacturing of petroleum and coal products, chemicals, plastics and rubber products or nonmetallic mineral products). There was no significant difference between participants and decliners in type of manufacturing as classified by 2-digit NAICS codes (p-value = 0.233) (Table 1).

Examination of 3-digit NAICS codes of participating companies (n=47) indicated that businesses in NAICS code 33 were primarily involved in the fabrication of metal products (NAICS 332, 17%), machinery (NAICS 333, 13%) and miscellaneous metal products (NAICS 339, 15%). These proportions were roughly similar to those of non-participating businesses in these classifications (23, 16 and 11%, respectively). Participating companies in NAICS code 32 were primarily printers (NAICS 323, 13%) and manufacturers of chemicals (NAICS 325, 11%) or plastics and rubber products (NAICS 326, 11%). The proportions of non-participating businesses in these classifications were roughly similar (11, 5 and 4%, respectively).

Employee Demographics

In the 47 participating businesses with a total of 3072 employees, 2652 (87%) completed the baseline survey. Return rates for individual businesses ranged between 50 and 100%; the median return rate was 90%. The majority of employees were 35-64 years, male, white and non-Hispanic (Table 2). The age and Hispanic ethnicity distributions of employees in participating companies were not statistically different from those of employees of all small manufacturing companies in the study counties. Age and race distributions of employees of participating companies were significantly different from all employee (p<.01 and p<.05, respectively), but an inspection of the percentages indicates that the differences were very small in magnitude. The difference between these groups in education was more pronounced: in participating companies a greater fraction of employees (40 vs. 33%) had some college education and a smaller fraction (2 vs. 9%) had less than high school education, when compared to the Census Bureau data.

Table 2.

Demographic characteristics of employees for all small private manufacturing firms (50-249 employees) and for 47 participating businesses in five Minnesota counties (Carver, Hennepin, Ramsey, Scott and Steele) during the study period (Q4 2010 to Q4 2012)

Characteristic	All Private Manufacturing Firms*		Participating Businesses (n=47)
	Number of Employees	Percent	Number of Employees	Percent	p **
Age
14-21	9,764	3	49	2
22-24	12,263	4	97	4
25-34	62,139	19	571	22
35-44	71,652	22	564	22
45-54	98,952	30	762	29
55-64	58,957	18	457	18
65+	12,678	4	72	3
Total	326,405		2,572		<.01
Sex
Male	238,829	73	1,917	72
Female	87576	27	735	28
Total	326,405		2,652		.41
Education ***
Less than High School	30,698	9	55	2
High school or equivalent, no college	93,073	28	706	28
Some College	104,583	33	1,037A	40
Bachelor’s Degree	76,047	23	614B	24
Education Not Available (aged<25 years)	22,127	7	146	6
Total	326,528		2,558		<0.01
Race
White	276,441	85	2,098	85
Black or African American	12,652	4	105	4
American Indian or Alaska Native	1,979	1	8	0
Asian	30,811	9	196	8
Native Hawaiian or Other Pacific Islander	438	0	5	0
Two or More Race Groups	3,924	1	28	1
Other race	--	--	61	2
Total	326,245		2,501		<.05
Ethnicity
Hispanic	18,196	6	140	6
Non-Hispanic	308,049	94	2,391	94
Total	326,245		2,531	.21

^*Data from Census Bureau LED Extraction Tool: 5 Counties, Manufacturing, Private Firms, Q4 2010 – Q4 2012, n = Estimate of the total number of jobs on the first day of the reference quarter.

^**Probability level for chisquare goodness of fit tests of the characteristics of employees in participating businesses compared with those in all small private manufacturing businesses in the study area.

^***Educational attainment not available in the Census data for workers aged 24 or younger; data presented for ages 25+ only.

^AIncludes vocational training and some college

^BIncludes bachelor degree and graduate or professional education

Discussion

This paper describes the methods used to recruit small manufacturing companies to a group-randomized trial to test an integrated workplace safety and smoking cessation program. Eligibility criteria eliminated about 44% (1029 of 2346) of companies identified primarily through the Dun & Bradstreet and ReferenceUSA commercial databases. Among sites that were eligible or whose eligibility was not established, 9% chose to participate. This rate is considerably lower than rates quoted in other studies. For instance, Barbeau et al (2004) reported that 20% of eligible small manufacturing companies decided to participate in an integrated health promotion and occupational health protection program.⁵ This difference in response rates was due in part to our inclusion in the denominator of the participation rate the 26 companies whose eligibility had not been established by the end of the Wellness Works recruitment period. If Barbeau et al. had included the 27 companies they could not reach in their estimates, their participation rate would have been 16%.

To our knowledge, this is the first worksite study in which an external database was used to examine the external validity of participating companies. Our comparison of participating company employee demographics with those of all companies of similar size and industrial sector in a Bureau of Census dataset collected during the study time period and geographic area supports our contention that participating companies were representative, despite a relatively low participation rate among businesses asked to participate.

Participating businesses were somewhat more likely to be larger (50-150 employees), suggesting that resources may play a role in a company’s ability to implement employee health programs. For those companies that declined to participate, lack of resources was the most common reason given. This finding is consistent with the results of the 2004 National Worksite Health Promotion Survey, which found that small businesses were much less likely to provide health promotion activities than were large businesses.⁷ Only 8.8% of workplaces with 50-99 employees had offered smoking cessation programs in the previous 12 months compared to 68.1% of businesses with 750 or more employees. Small businesses are generally less able to afford the costs of health promotion programs and often lack personnel to devote to employee health.⁶

Participating businesses were similar to non-participants in type of manufacturing (e.g., 60% made metal products compared to 68% in all eligible companies). The demographic profile of employees in participating companies was also similar to that of all manufacturing employees in small private firms in the five target counties. Thus, although participating companies were somewhat larger than non-participants, they appear to be representative of all companies in important features that contribute to both workplace safety and personal health behavior risks. It is important to note, however, that characteristics of both businesses and employees tested were limited to those that were available in national datasets. They did not include variables such as organizational climate, job strain and social cohesion that are likely to be associated with both safety practices and the personal health behavior risks of employees. These organizational variables are also likely to be related to whether a business agrees to participate in a health promotion / work safety intervention, the thoroughness with which the business implements the intervention, and employee participation rates. ^13,28

There is no perfect source of information on the universe of manufacturing businesses in a particular geographic area. Chamber of Commerce (COC) listings were initially explored, but manufacturing companies were found to be unlikely to join COCs. In the largest county (Hennepin) no single COC could provide a list of all manufacturers located in that county. The commercial business databases, supplemented by short lists of contacts from local health departments, provided the most complete source of manufacturing businesses.

These databases have some shortcomings including missing companies as well as incorrect or out-of-date company addresses, telephone numbers, and contact information. After initial screening for eligibility and duplication across the two databases, 18% were not reachable by telephone and 50% did not meet eligibility criteria.

Information from each of the commercial database organizations indicates that multiple sources of information are used to identify businesses and demographic information and regular efforts are made to verify information. Companies in the D&B database must have either $9 million sales or more than 180 employees at headquarter locations or 900 employees at branch locations (personal communication), which may explain why one participating business with less than $5 million in sales was found in neither database.

While these and similar commercial databases are frequently employed by researchers, there have been few efforts to examine their validity. Cook et al. (2012) tested the validity of ReferenceUSA’s New Business database by calling randomly-selected businesses that were one month, six months, and one year old. Telephone numbers were found to be incorrect for 39%, 35% and 38% of businesses, respectively. The investigators were able to reach a person (owner or employee) at 40%, 46% and 31% of businesses or an answering machine at 21%, 20% and 32% of businesses, respectively. As well, they found that the number of employees reported by companies in business for one month was significantly different from the number indicated in the database. The investigators concluded that this particular database had not been adequately vetted. ²⁹

We could have reduced the amount of time spent in preliminary screening by limiting the number of ineligible companies through the use of more selective criteria. For example, both databases allow selection of companies that are independent or single sites. There was some concern, however, that using these criteria would eliminate potentially eligible companies. We cannot determine if this was true, however, because we did not compare the companies removed in preliminary screening with those that would have been removed if different selection criteria had been employed.

More recruitment contact attempts are also recommended. We limited initial contacts to determine eligibility to two letters and three phone calls. For a sizable number of companies (221), this was not sufficient to establish contact and confirm eligibility. A protocol requiring more contact attempts would be necessary to make a complete determination for all companies. More intensive follow-up of the 26 companies that were determined to be eligible, but which did not make a decision, would also have increased recruitment rates.

Business conditions (e.g. contact turnover, CEO illness, economic up- or downturns) also may have extended or prevented decisions to participate. Some businesses were interested in the project but were not immediately ready to participate. Some businesses were unable to commit to participation despite lengthy extensions in recruitment time.

Recruitment for this study took place from 2010 to 2012 during a period immediately following a downturn in the U.S. economy. Many of the small manufacturing businesses eligible for this study were still dealing with uncertain financial situations and may have judged they had insufficient resources to commit to this project. On the other hand, some businesses did view the study as a low cost resource for safety and smoking cessation promotion.

Our results indicate that it is possible to identify and recruit a representative sample of small manufacturing businesses for public health intervention studies using commercial databases as the primary information source. The efforts required to create a preliminary set of eligible business can be time consuming, however, due to inaccuracies in the commercial databases and failure of businesses to respond to letters and telephone calls. Recruitment may require multiple contacts with a single company before identifying the correct decision maker and gaining their agreement. Investigators should expect to allocate considerable resources to identifying and recruiting businesses if they wish to ensure a representative sample.