Development of the Environmental Assessment Tool (EAT) to Measure Organizational Physical and Social Support for Worksite Obesity Prevention Programs

Abstract

Objective

To describe the development, reliability, and validity of the Environmental Assessment Tool (EAT) for assessing worksite physical and social environmental support for obesity prevention.

Methods

The EAT was developed using a multi-step process. Inter-rater reliability was estimated via Kappa and other measures. Concurrent and predictive validity were estimated using site-level correlations and person-level multiple regression analyses comparing EAT scores and employee absenteeism and healthcare expenditures.

Results

Results show high inter-rater reliability and concurrent validity for many measures and predictive validity for absenteeism expenditures.

Conclusions

The primary use of the EAT is as a physical and social environment assessment tool for worksite obesity prevention efforts. It can be used as a reliable and valid means to estimate relationships between environmental interventions and absenteeism and medical expenditures, provided those expenditures are for the same year that the EAT is administered.

INTRODUCTION

Obesity is a major public health concern, with recent surveys showing about two thirds of adults in the U.S. classified as overweight or obese¹ and consequently at risk for health care problems such as type-2 diabetes, cardiovascular disease, stroke, cancer, osteoarthritis, depression, gallbladder disease, and respiratory disorders.^{2, 3} The high incidence of obesity and its relationship to other major medical disorders makes it a costly condition, accounting for an estimated 5.5% – 7.0% of U.S. health expenditures between 1986 and 1995.⁴ In addition, obese adults have approximately 36% higher medical expenditures than their normal weight counterparts⁵ and higher rates of absenteeism and presenteeism.^6-8

To help control costs and improve the health of their workers, employers are introducing a variety of health promotion and risk reduction programs including those that address the growing problem of overweight and obese workers. Worksite health promotion programs aim to improve the health status of workers by offering individual risk reduction interventions coupled with efforts to address environmental, social and ecological forces that contribute to unhealthy behaviors. Specifically, employers are taking steps to address the “obesogenic” environment at the workplace that promotes overeating and lack of exercise.⁹ Changing the work environment to induce positive health improvements is supported by social-ecological theory, which emphasizes the multi-level interaction of individuals and their physical and social environment, and the effect of this interaction on individual health behaviors.^{10, 11}

An obesogenic environment encourages excess intake of calories and discourages physical activity leading to weight gain.⁹ Recent social and environmental secular trends, including providing greater access to and increased marketing of high calorie and high fat food products in combination with more sedentary leisure activities, contribute to the obesogenic environment.¹² A large source of added sugars comes from soft drinks that are abundantly available in worksite and school vending machines.¹² Due to time pressures and convenience, individuals and families are eating out at restaurants more than ever, and foods away from home tend to be more energy and fat dense.¹² Additionally, psychologically, consumers aim to get more for less, thus “supersizing” has become a common marketing strategy for fast food restaurants.¹³ To add to this, physical activity levels among adults have declined.¹² With the convenience of the automobile, fewer people walk or bike¹² and they spend more of their leisure time sitting in front of televisions and computers.¹² Finally, heavy manual labor occupations have declined over the past few decades being replaced by more sedentary jobs.¹²

The U.S. Department of Health and Human Services (DHHS) Healthy People 2010 goals emphasize the worksite is an important setting for introducing environmental and ecological changes aimed at reducing the incidence of obesity among workers.¹⁴ Adults spend a significant portion of their waking hours at work and opportunities abound for changing the work environment so that it promotes healthier lifestyles. Introducing low cost environmental and ecological interventions at the worksite can support individual health improvement efforts by workers aimed at reducing overweight and obesity.

Current Tools to Evaluate Workplace Environments Supporting Health Improvement Efforts

To measure the extent to which physical and social elements of an environment support health improvement efforts, a number of assessment tools have been developed. Specifically, two kinds of environmental assessment instruments have been described in the literature: those relying on perceptions and self-report and those relying on first-hand observation. The two types of instruments are intended for different kinds of uses. Perception/self-report tools would most often be administered to groups of people through telephone or in-person interviews. Direct observation tools, on the other hand, are primarily intended to be completed while the person is actually walking through a specific environment. Brownson et al. describe three perception/self-report tools, the San Diego instrument, South Carolina instrument, and St. Louis instrument, that are designed to measure community and neighborhood environmental influences on physical activity.¹⁵

The 98-question San Diego instrument was developed by Saelens et al. to measure the perception of neighborhood design features supporting physical activity.¹⁶ The questions assess types of residences, proximity of stores and facilities in the neighborhood, perceived access to these places, street characteristics, facilities for walking and cycling, neighborhood aesthetics, and safety regarding traffic and crime. The South Carolina instrument is a 61-question instrument developed by Ainsworth et al. to assess both the physical and social environment.¹⁷ That assessment measures perceptions of the community environment, safety, access to recreation and shopping destinations, conditions of the neighborhood and facilities, employment activity, moderate and vigorous physical fitness activities, and walking behaviors. The St. Louis instrument is a 104-question survey developed by Brownson et al. to measure physical activity and environmental influences on physical activity.¹⁸ The questionnaire includes a detailed assessment of walking behavior, places to walk, barriers to being physically active, neighborhood infrastructure for walking and cycling, perceptions about places for walking, social assets, social support for physical activity, community assets, policy attitudes, and sedentary behaviors.

A somewhat broader perception/self-report, Heart Check, was developed as part of New York State’s Healthy Heart Program to measure organizational factors that support employer cardiovascular disease (CVD) risk reduction efforts.¹¹ Heart Check is a 226-item inventory that measures the following worksite features: organizational foundations, administrative supports, tobacco control, nutrition support, physical activity support, stress management, screening services, and company demographics.¹¹ The 250-item Working Well tool is based on the Heart Check but is more comprehensive, measuring cancer and diabetes risk in addition to CVD risk and organizational structure supporting risk reduction.¹⁹

More recently, a direct observation tool, the 112-item Checklist of Health Promotion Environments at Worksites (CHEW), was developed to evaluate a worksite’s physical and “information distribution” environments within the context of physical environment in the immediately surrounding community as they relate to physical activity, eating habits, alcohol consumption, and smoking.²⁰ However, the CHEW does not include measures specific to social-organizational/administrative supports for health improvement efforts among workers.

Purpose

The purpose of this paper is to describe the development, reliability, and validity of the Environmental Assessment Tool (EAT), which assesses the physical and social environment of a worksite in terms of its support of obesity prevention efforts. The EAT was developed as a part of a National Heart, Lung, and Blood Institute (NHLBI) research initiative to study the impacts of innovative workplace interventions that emphasize environmental approaches or a combination of environmental and individual approaches to prevent obesity among workers.^{21, 22} The study, currently underway at The Dow Chemical Company (Dow), is testing two levels of environmental interventions: 1) a moderate-level intervention that introduces an array of inexpensive environmental changes, primarily to the physical environment, and 2) an intensive-level intervention that reflects a higher level of management commitment throughout the organization aimed at achieving an impact on the social-organizational environment.

Our review of previous environmental assessment instruments found that they primarily addressed physical activity-related issues at the workplace and only touched upon environmental supports for obesity prevention and healthy eating. In addition, the instruments reviewed supported data collection through observation of the physical environment and non-evaluative description of findings. Since our studies called for the scoring and valuing of environmental supports for obesity prevention efforts, and an assessment of progress over time, we needed an instrument that would score our observations using more objective and quantifiable methods. Thus, the EAT was developed and based on previous knowledge and experience gained through the administration of the CHEW and Heart Check tools.

METHODS

Twelve Dow sites began implementing individual and environmental interventions to address overweight and obesity in 2006. These interventions were preceded by a year of formative research and intervention design.²² The EAT was developed during that formative research period in three stages: 1) contextual analysis and literature review, 2) prototype development, and 3) pilot testing. It was used to document aspects of physical and social environments that may influence healthy eating and physical activity and was first administered before interventions began (in the spring and summer of 2005), and annually thereafter.

Contextual Analysis and Literature Review

The contextual analysis involved working cooperatively with Dow corporate staff to become familiar with the specific work and operational environments and the broader site and location characteristics of the facilities participating in the project. Site characteristics were evaluated in terms of the number, types, sizes, and arrangement of buildings; parking facilities; roadways; green space areas; and safety and security requirements. Location characteristics were assessed in terms of whether the facilities were located in urban, suburban, or rural areas; access and availability of stores, restaurants, and recreational facilities; typical commuting and transportation options; and climate and weather conditions.

Construction of the EAT was based on standards of best practices regarding worksite health promotion interventions reported in the literature.^{12, 23-50} As noted previously, the EAT developers adapted several concepts found in the CHEW and Heart Check instruments, as well as best practices reported in the literature related to environmental and social-ecological worksite interventions. Using the CHEW and Heart Check instruments as a basis, EAT developers integrated the physical characteristics of the worksite, features of the information environment, and characteristics of the immediate neighborhood around the workplace from the CHEW and the characteristics of employer and administrative support systems from the Heart Check. Questions for the EAT were based on these concepts as they applied to environmental and social supports for physical activity and obesity prevention. EAT items addressed the job factors, physical and social-organizational work environment, and socio-cultural and economic/legal environment variables found in DeJoy and Southern’s social-ecological model for workplace environmental interventions, upon which the Dow environmental interventions are based.⁵¹

Prototype Development

The information sources described above were used to develop a series of prototype instruments that were reviewed and critiqued by the project team through conference calls and face-to-face meetings. The project team consisted of specialists in worksite health promotion, nutrition and dietetics, exercise science, communications, occupational safety and health, applied psychology, and statistics and research methods. Professional staff members from the partnering organization were also actively involved in this review and revision process. As the full instrument took form, project team members were asked to independently review the instrument for completeness and to rate the various items in terms of their relative importance to supporting healthy eating, weight management, and physical activity. This rating exercise was a precursor to developing a scoring system for the EAT.

The final EAT prototype consisted of 105 items broken into two sections, Section I was completed by site staff and Section II was completed by independent observers who toured the site and recorded their observations. Section I consisted of questions that could best be answered by those closely affiliated with the physical plant, and included such topics as work rules and requirements, current health promotion programs and services, and formal policies that support or facilitate healthy eating and/or physical activity participation.

The items in the EAT, including those from Sections I and II, can be broken down into three subscales pertaining to 1) Physical Activity, 2) Nutrition/Weight Management, and 3) Organizational Characteristics and Support. The Physical Activity scale assesses access to and availability of parking for motor vehicles and facilities for securing bicycles; stairs and elevators; showers and changing facilities; signage and bulletin boards containing messages pertinent to physical activity; and physical activity and fitness facilities. The Nutrition/Weight Management scale focuses on vending, cafeteria, and other food service options and facilities, and signage and bulletin boards pertinent to diet and weight management. The Organizational Characteristics and Support scale assesses general site characteristics, work rules, written policies, and existing health promotion programming and services.

The EAT tool is comprised primarily of dichotomous (yes and no) items. The number of yes answers in each section constitutes the score for that section on the EAT. If it is unclear as to whether an item should be scored at all, researchers label the item as N/AP, indicating that it is not applicable to the site, or as N/AV, indicating the feature is “not available” at the site. During initial use of the EAT, paper forms were completed by site staff (Section I) and the independent observers (Section II). In subsequent applications, Section I was distributed electronically to site staff, and Section II was completed using computer tablets by on-site independent project team observers. Using database software, a variety of checklists, windows, boxes, and drop-down menus were developed to facilitate data collection and analysis. Data collection and scoring were accomplished at different times and by different individuals. The procedures for scoring the EAT are described below.

Pilot Test and Tool Refinement

Section of early EAT prototypes were field tested for completeness and practicality in campus buildings and facilities at the University of Georgia. Once a final prototype was developed, the complete instrument was pilot tested at one of the Dow control sites participating in the study. As described above, Section I was completed by company staff and Section II was completed by project team members during a scheduled site visit. While a single project team member was responsible for completing the Section II form during the pilot test, other team members and company representatives were present to observe the process and to take notes. At the end of the site tour, the project team and company staff met to review the use and performance of both parts of the EAT.

As a result of the pilot testing, several modifications were made to the assessment tool. First, the section describing food preparation facilities was expanded to reflect the reality that many Dow employees had access to full kitchen facilities while at work not just refrigerators and microwaves. Second, to facilitate quicker and more accurate assessment of vending machine options, every snack and beverage item contained in each vending machine was recorded instead of attempting to classify each item as healthy or unhealthy while collecting data in the field. Third, some of the questions originally placed in Section II were moved to Section I because they could be more accurately answered by site staff. Fourth, the job categories included in Section I were modified to better reflect the specific terminology used by site-level personnel. Finally, a small number of questions were deleted because they were duplicative of similar information collected elsewhere in the instrument.

Scoring System

A 100-point scoring system was developed to allow for quantitative comparisons of environmental supports across control and treatment sites, and to monitor changes over time. Table 1 shows the major components of each of the three subscales, and the point values assigned to each. A weighting exercise was performed to assess the relative importance of each component in terms of supporting Nutrition/Weight Management and Physical Activity programs in the workplace. This weighting was performed by three specialists in workplace health promotion who made independent judgments about the importance of each item on the EAT based on their experience in worksite health promotion and knowledge of the relevant intervention effectiveness literatures. Equal total importance was assigned to the Physical Activity and Nutrition/Weight Management subscales (i.e., 32 points each). Items with N/AP and N/AV responses are scored the same as a no response because the environmental component being measured is not in place; therefore, not available to facilitate behavior change. However, the N/AP or N/AV responses are useful in interpreting the results (e.g., the site did not have a healthy cafeteria policy because it does not have a cafeteria onsite). The rating process resulted in a somewhat higher total point value for the Organizational Characteristics and Support subscale based largely on the enabling potential afforded by having supportive policies and programs in place at the work site.

Table 1.

Summary of EAT components and scoring system

Organizational characteristics and support	36 points
Site characteristics	4 points
Work rules	6 points
Written policies	6 points
Health promotion programs
Physical activity	7 points
Diet/nutrition	7 points
Weight management	6 points
Physical activity	32 points
Parking/bike assessment	4 points
Stairs/elevator assessment	4 points
Shower/changing facilities	6 points
Physical activity signs	4 points
Physical activity/fitness facilities	14 points
Nutrition and weight management	32 points
Nutrition/weight management signs	4 points
Vending	12 points
Cafeteria/food service	16 points

Data Collection and Analysis

The EAT was used to collect baseline data related to physical and social support for obesity prevention at each of the 12 sites participating in the study. Site staff provided self-report responses to the items in Section I of the EAT prior to the site visits, and responses to Section I items were reviewed for completeness by the observers during the site visit. Researchers visited the sites to complete Section II of the EAT survey, which took about 4 hours per building. They completed Section II independently before comparing their results to achieve consensus on their responses. Due to safety and security concerns by the company, Dow employees always escorted the observers when EAT data were collected. However, the Dow employee escorts were only consulted when guidance was needed in navigating through the sites, and they had no impact on data collection activities. Two research analysts who were not present during the site visits scored the EAT (Sections I and II). They used a scoring rubric to aggregate the EAT responses into overall and subscale scores, so that a higher scores reflected greater environmental support for healthy eating, access to physical activity, or weight management. The two research analysts independently scored the EAT and then their scores were compared for inter-rater reliability.

Since many of the sites were too large for observers to inspect every building or area (sites ranged in size from 50 to 5000 acres and 12 to 300 inhabited buildings), with the assistance of local Dow staff, approximately six occupied buildings or areas that were representative of the site and its employees were selected for assessment. Scores for the areas observed were then aggregated and an average rating was computed for each site with a higher scores reflecting greater environmental and support for healthy eating, access to physical activity, and weight management. The scoring of the EAT occurred after the actual site visit had been completed, and it was performed by different members of the project team. For this initial application of the EAT, two independent scorers were used and their rating compared.

The analyses reported here are for data collected using the EAT during the 2005 baseline period (see Table 2).

Table 2.

EAT scores by site.

Site Name	Number of Buildings	Number of Employees	Total Score (100 pts)
Control
C-A	7	1,112	25.14
C-B	6	168	43.16
C-C	6	1,056	38.43
Moderate Intervention
M-A	7	208	30.62
M-B	6	659	34.17
M-C	6	445	38.10
M-D	3	100	27.30
Intense Intervention
I-A	7	4,202	37.57
I-B	3	323	39.67
I-C	3	146	18.40
I-D	1	566	56.00
I-E	7	1,600	47.04

Assessing the Validity and Reliability of the EAT

To test the validity and reliability of the EAT, we demonstrate criterion-related validity and inter-rater reliability. Following Carmines and Zeller’s⁵² approach, we disaggregated criterion-related validity into predictive validity and concurrent validity, to determine whether the relationships between EAT scores and other variables of interest were predictive or concurrent in nature.

Inter-Rater Reliability

Instrument reliability was assessed by comparing the ratings of the two observers who were trained by the instrument’s developers. Percent agreement between the observers was calculated by combining the proportion of items in agreement divided by the total number of items. Kappa statistics were calculated for individual EAT items, and Pearson correlations coefficients were derived for the EAT subscales and total score.

Concurrent Validity

Concurrent validity was addressed by first testing the hypothesis that higher EAT scores would be associated with lower payments for health care services and lower rates of absenteeism in the same year that EAT scores and payments were measured. The expectation was that more supportive environments would be associated with healthier employees and lower absenteeism from work during that year, so that payments for healthcare services and absenteeism would be lower for sites with higher EAT scores. Such associations would support the concurrent validity of the EAT.

Concurrent validity analyses were conducted at the site- and person- levels. Site-level EAT scale scores were correlated with the following site-level Dow payment figures, derived from Dow’s insurance claims and administrative absenteeism data, for claims incurred in 2005: average total medical payments; average inpatient, outpatient, emergency room, and pharmacy payments; and average absence payments. (Average absence payments were calculated by multiplying days lost from work due to illness by a $30 average hourly wage figure.¹)

While the site-level analyses were informative, there were only 12 sites included in the analysis, so there were only 12 observations available to estimate each correlation coefficient. To avoid problems that may be related to low statistical power, we also conducted person-level analyses that used data from several thousands of employees who worked at the 12 sites. One-part or two-part exponential regression models were used for these person-level analyses. Two-part models were used for analyses of inpatient and emergency room payments, because substantial percentages of employees had zero-dollar expenditures for these payment metrics. Detailed explanations of two-part regression models can be found in Mullahy.⁵³

The person-level regression models controlled for demographics (age, gender), location (residence in a rural vs. urban area), and comorbidities (using the Charlson Comorbidity Index (CCI) and the number of psychiatric problems found in the medical claims data). The predictive power of the regression models was then estimated, by comparing mean predicted expenditures with actual average expenditures for claims incurred in 2005. Regression models were then adjusted (usually by removing one or both comorbidity variables) to arrive at final models that maximized their predictive power. The exponential regression models were estimated within a general estimating equations (GEE) framework, to account for the fact that employees were nested within sites.

It should be noted that the EAT scores that were used for the person-level regression analyses were measured at the site-level (e.g., each employee at site A was linked with that site’s overall EAT score and subscale scores reflecting organizational support for physical activity, healthy food, and overall management support). The regression analysis allowed for the reporting of increases or decreases in average health care or absenteeism payments resulting from one-unit increases in average site EAT scores, at the person-level.

To further test concurrent validity, correlation statistics were calculated between the EAT Organizational Characteristics and Support scale with another instrument, the Leading by Example (LBE) questionnaire. The LBE has been found to be a valid tool for diagnosing management issues and challenges at Dow, and tracking management support for obesity prevention over time.⁵⁴ It was adapted from the Partnership for Prevention’s version of the LBE.⁵⁵ The LBE was independently administered to site staff and leaders, including the leadership team at the site, cross-disciplinary team members (i.e., individuals who work as production leaders, production engineers, operators, etc.) and health services staff. It asks respondents to agree or disagree with certain statements regarding site leadership and its commitment to health promotion as an important investment in human capital and whether the site provides support for employees to stay healthy, reduce their high-risk behaviors, and/or practice healthy life styles. A full description of the LBE and its psychometric properties is described in Della, DeJoy, Goetzel, Ozminkowski, and Wilson, now in press. The expectation was that similar levels of environmental support would be captured in both the LBE and the EAT Organizational Characteristics and Support scale. If observed, these associations would support the concurrent validity of the EAT.

Predictive validity

Predictive validity was examined in the same way as concurrent validity, with one exception. For predictive validity, EAT scores for 2005 were correlated with medical and absenteeism payments for claims incurred in 2006. The hypothesis tested was that better work environments would lead to savings in future health care or absence-related expenditures. If this notion were true, and if the EAT had a high degree of predictive validity, one would expect to observe that that higher EAT scores in 2005 would be associated with lower payments in 2006.

RESULTS

Inter-rater reliability

There were generally high levels of inter-rater reliability observed for the EAT. The percentage agreement scores between observers ranged from a low of 83.5% to a high of 97.0%, with the majority of the instrument measures being above 90%. Specifically, the observers were most consistent in the stair/elevator and fitness facility assessments (both at 97.0 %). Table 3 presents the percentage agreement for each component of the EAT.

Table 3.

Inter-rater reliability of the EAT administered in 2005.

	Number Agree	Total Opportunities	% Agreement
Stairs/elevator Assessment
Stair/Elevator/Entrance Count	152	162	93.8%
Stairwell #1	199	200	99.5%
Stairwell #2	116	120	96.7%
Stairwell #3	46	48	95.8%
Stairwell #4	32	32	100.0%
Stairwell #5	1	1	100.0%
Stairwell #6	1	1	100.0%
Total	547	564	97.0%
Physical Activity & Nutrition/Weight Management Signs
PA messages	250	306	81.7%
DN messages	261	306	85.3%
Total	511	612	83.5%
Shower/Changing & Fitness Facilities
Changing facilities	108	112	96.4%
Fitness facility/hours	54	57	94.7%
Fitness equipment	151	160	94.4%
Fitness classes/activities	258	260	99.2%
Sports opportunities	69	70	98.6%
Path intervention	46	49	93.9%
Total	686	708	96.9%
Written Policies
Worksite policies	60	60	100.0%
Corporate support for fitness	44	50	88.0%
Total	104	110	94.6%

Key: “PA” represents physical activity; “DN” represents diet and nutrition

Kappa statistics, which indicate the level of non-random agreement between observers, were calculated for 66 EAT items and were unable to be calculated for an additional 56 EAT items because the observers had 100% agreement. Of the 66 EAT items with Kappa statistics, 63 items demonstrated substantial or almost perfect agreement among raters, with Kappa statistics ranging from a low of 0.615 (p = 0.035) to a high of 1.000 (p = 0.000). Only three EAT items did not have statistically significant Kappa statistics: fitness center cost subsidy (k = 0.412, p = 0.107), no stair safety warnings in stairwell 3 (k = 0.667, p = 0.083), and stairwell door unlocked in stairwell 3 (k = 0.667, p = 0.083). Kappa statistics could not be calculated for nine items because one of the observers did not record an observation for those items that the other observer included; consequently, a symmetric table could not be created.

Kappa statistics could not be calculated for summarized scale scores because they are continuous and not dichotomous (yes/no) variables, so Pearson correlation coefficient analyses were conducted to determine the relationship between the two observers’ subscale and total scores. Pearson correlation coefficients were calculated for the EAT subscales, components of these subscales, and the EAT total score. Pearson correlation coefficients were not calculated for the third EAT subscale, Nutrition/Weight Management, because two of the three components in this subscale, vending and cafeteria/food services, were scored by a single observer. All but four of the relationships were strong, demonstrating Pearson correlation coefficients ranging from a low of 0.757 (p=0.004) to a high of 1.000 (p=0.000). Three of the relationships among subscale components were not significantly correlated: site characteristics (r = 0.541, p = 0.069), work rules (r = 0.184, p = 0.567), and overweight/obese (r = 0.071, p = 0.826). Organizational Characteristics and Support was the on ly subscale score that was not significant (r = 0.158, p = 0.624). All three of the non-significant subscale component relationships were components of the single insignificant subscale relationships, Organizational Characteristics and Support.

Concurrent validity

At the site-level, the relationship between higher EAT scores and lower payments for claims incurred in 2005 was as hypothesized for absenteeism, but not consistently for medical care (see Table 4). There were statistically significant negative correlations between the EAT Nutrition and Weight Management subscale scores and absenteeism payments (p = 0.0305) and the EAT Organizational Characteristics and Support subscale scores and emergency room payments (p = 0.0387). There was also a negative correlation between the EAT Physical Activity subscale scores and absenteeism payments, but this correlation fell short of statistical significance (p = 0.0752).

Table 4.

Concurrent validity: Correlations between 2005 EAT scores and 2005 site-level expenditures.

	Correlation Matrix (N=12)
Parameter	Correlation Coefficient	p-value
Nutrition and Weight Management Score
Absenteeism	-0.62288	0.0305
Total medical	0.00623	0.9847
Inpatient	-0.20878	0.5149
Emergency room	-0.15259	0.6359
Outpatient	-0.13752	0.6700
Pharmacy	0.09643	0.7656
Organizational Support Score
Absenteeism	-0.22097	0.4901
Total medical	-0.35662	0.2552
Inpatient	-0.14762	0.6471
Emergency room	0.60109	0.0387
Outpatient	0.37503	0.2297
Pharmacy	-0.42819	0.1649
Physical Activities Score
Absenteeism	-0.53166	0.0752
Total medical	0.27632	0.3846
Inpatient	-0.12074	0.7086
Emergency room	-0.07333	0.8208
Outpatient	0.46116	0.1313
Pharmacy	0.10193	0.7526

Generally speaking, the person-level analyses produced similar and slightly stronger support for the hypothesized relationship between higher EAT scores and lower payments for claims incurred in 2005 for absenteeism, but not for medical care (see Table 5). A one-unit increase in the EAT Nutrition and Weight Management subscale scores, resulted in a $35.60 per employee per year reduction in absenteeism payments (p = 0.0000) and a $21.55 per employee per year reduction in outpatient medical payments (p = 0.0380). A one unit increase in the EAT Organizational Characteristics and Support subscale scores, resulted in a $25.06 per employee per year reduction in absenteeism payments (p = 0.0000). Finally, a one unit increase in the EAT Physical Activity subscale scores resulted in an $87.91 per employee per year reduction in total medical payments (p = 0.0000), and a $47.25 per employee per year reduction in outpatient payments (p = 0.0003). The associations between increases in EAT scores and changes in payments were generally in the expected direction for non-significant relationships.

Table 5.

Concurrent validity: Marginal effects from the 2005 EAT scores and 2005 person-level expenditures regression analyses.

	Section II: ECM model, among users (N=5452)
Parameter	Parameter Estimate	Odds Ratio	p-value	Marginal Effect*
Nutrition and Weight Management Score
Absenteeism	-0.0355	0.9651	0.0000	-$35.60
Total medical	-0.0074	0.9926	0.1734	-$26.21
Inpatient	-0.0038	0.9962	0.8598	-$14.78
Emergency room	-0.0180	0.9821	0.0667	-$1.05
Outpatient	-0.0118	0.9882	0.0380	-$21.55
Pharmacy	0.0103	1.0103	0.0681	$7.12
Organizational Support Score
Absenteeism	-0.0321	0.9685	0.0000	-$25.06
Total medical	0.0054	1.0054	0.4921	$15.70
Inpatient	0.0450	1.0460	0.0937	-$9.63
Emergency room	0.0198	1.0200	0.1552	-$2.24
Outpatient	0.0115	1.0116	0.1646	$15.05
Pharmacy	0.0016	1.0016	0.8420	-$1.02
Physical Activities Score
Absenteeism	0.0070	1.0070	0.2244	-$25.45
Total medical	-0.0261	0.9742	0.0000	-$87.91
Inpatient	-0.0090	0.9910	0.6750	-$24.29
Emergency room	-0.0032	0.9968	0.7411	-$0.14
Outpatient	-0.0213	0.9789	0.0003	-$47.25
Pharmacy	-0.0009	0.9991	0.8797	-$0.37

Control variables: age, gender, location, and comorbidities

^*Marginal effects show the estimated change in expenditures per 1-unit increase in EAT scores

The EAT Organizational Characteristics and Support subscale was significantly related to several LBE items including those that ask whether employees are educated regarding the true cost of health care (p = 0.0286), whether the site offered incentives to stay healthy (p = 0.0016), and whether the site promoted a culture of health and well being (p = 0.0051). The correlation between the EAT Organizational Characteristics and Support subscale and the overall LBE score fell just short of significance (p = 0.0503). The EAT Organizational Characteristics and Support subscale was also strongly related to LBE items addressing whether health programs were aligned with business goals (p = 0.0638) and whether health benefit programs support prevention (p = 0.0640).

Predictive validity

Generally speaking, the predictive validity of the EAT, as measured in terms of whether 2005 higher EAT scores were associated with 2006 financial measures, was supported for absenteeism, but not for medical payments (see Table 6). A one unit increase in the EAT Nutrition and Weight Management subscale scores was associated with a $4.93 per employee per year reduction in absenteeism payments (p = 0.0296), but all other relationships were either not significant or resulted in increased costs. Similarly, a one unit increase in the EAT Physical Activity subscale scores, resulted in a $36.44 per employee per year reduction in absenteeism payments (p = 0.0151), but all other relationships were not significant.

Table 6.

Predictive validity: Marginal effects from the 2005 EAT scores and 2006 person-level expenditures regression analyses.

	Section II: ECM model, among users (N=5452)
Parameter	Parameter Estimate	Odds Ratio	p-value	Marginal Effect*
Nutrition and Weight Management Score
Absenteeism	0.0160	1.0161	0.0296	-$4.93
Total medical	0.0168	1.0170	0.0015	$65.34
Inpatient	0.0093	1.0093	0.6056	$3.70
Emergency room	0.0218	1.0220	0.0303	$1.06
Outpatient	0.0162	1.0163	0.0032	$33.47
Pharmacy	0.0185	1.0186	0.0008	$16.60
Organizational Support Score
Absenteeism	-0.0007	0.9993	0.9238	-$2.40
Total medical	-0.0087	0.9913	0.3568	-$2.73
Inpatient	0.0450	1.0461	0.1383	$25.72
Emergency room	0.0119	1.0120	0.4077	-$0.41
Outpatient	-0.0091	0.9909	0.2611	-$21.18
Pharmacy	-0.0015	0.9985	0.8484	-$1.93
Physical Activities Score
Absenteeism	-0.0170	0.9832	0.0151	-$36.44
Total medical	-0.0047	0.9953	0.3896	-$12.37
Inpatient	-0.0036	0.9964	0.8523	-$5.83
Emergency room	-0.0081	0.9920	0.4451	-$0.66
Outpatient	-0.0067	0.9933	0.2352	-$14.66
Pharmacy	-0.0007	0.9993	0.9019	-$2.02

Control variables: age, gender, location, and comorbidities

^*Marginal effects show the estimated change in expenditures per 1-unit increase in EAT scores

DISCUSSION

Summary of findings

To assess organizational, environmental and social support for health promotion programs in the workplace, and in particular those directed at overweight and obesity, the EAT was developed and tested at 12 Dow worksites in 2005 as part of a baseline assessment for a larger multi-year study. To evaluate the psychometric properties of the EAT, inter-rater reliability was assessed as well as the instrument’s ability to predict current and future spending on healthcare and employee absenteeism.

Our preliminary analyses indicate that the EAT may be used reliably to measure the physical and social environments at the worksite, specifically as they relate to physical activity, food choices and weight management, and general organizational characteristics that support health promotion for workers. The high levels of inter-rater reliability buttress this finding.

Concurrent validity was demonstrated by associating EAT scores with company payments for incurred health care services and employee absenteeism. Generally speaking, employees who worked in environments that promote healthier eating, more physical activity, and exhibit organizational characteristics and support for healthy behaviors have lower costs, especially regarding absenteeism payments. EAT scores were not as strongly associated with medical payment metrics, however. Concurrent validity was also demonstrated by the high, positive correlation between the results obtained from EAT Organizational Characteristics and Support subscale and the LBE questionnaire.

Limitations and future directions

Using the EAT to measure environmental support for healthy behavior has the following limitations:

First, the EAT is designed to assess environmental and organizational supports pertinent to healthy eating and weight management. It is not intended to be used to assess overall or general support for health promotion. The EAT builds upon previous efforts to develop observational assessments specific to physical activity, and adds content relevant to healthy eating and weight management. Fundamentally, the EAT adheres to an energy intake-energy expenditure model of weight management. The EAT also seeks to assess the extent to which the social-organizational environment supports healthy eating, weight management, and physical activity. A potential limitation of the EAT scoring rubric is that sites are penalized if certain environmental supports that may not be feasible (e.g., cafeteria) are not present. An alternate scoring rubric could calculate EAT scores on the basis of a dominator that does not add to 100%, and re-scale as if it did. This would avoid penalizing sites that may not have the resources to afford housing a cafeteria, workout facility, or other health promotion assets.

The EAT is specific to a given workplace environments. Although there was a concerted effort to construct an instrument that would be applicable to a variety of different types of work environments, our initial experience using the EAT suggests some pilot or preliminary work should be done to fine tune the Tool prior to deploying it in a given work setting or industry.

Part of the customization process involves selecting buildings at the worksite to be evaluated by the EAT. Some may argue that all buildings, or if resources are limited a random sample of buildings, should be evaluated. Others may argue that selection should be based on the number of employees in a building or its traffic patterns in terms of use by workers. The site selection process may induce some bias into the evaluation if the selected buildings are not representative of the site, and that should be noted by the evaluators. For the most part, the EAT is most applicable to traditional work situations in which employees spend all or most of their work day within the physical boundaries of the workplace.

Second, those charged with using the EAT should receive specialized training on its use beforehand. Several factors contribute to this recommendation. Perhaps most importantly, environmental features should be assessed from the general viewpoint of the employees working within the building or facility. For example, many work settings have multiple entrances and exits, but employees often use certain entrances and exits that are separate and distinct from those used by the public and/or those that are most prominent from the outside of the building. Although stairways and elevators were assessed using mostly “yes-no” categories, the right stairways and elevators needed to be assessed. Also, given the size and complexity of many work environments, it is usually not practical to assess all building or facilities. In such cases, care should be taken in establishing the sampling parameters and in documenting the specific locations that will be assessed. Documentation is crucial if repeated observations are planned. The level of detail involved in adequately assessing vending and food service facilities is another complicating factor. Our experience suggests that advance information from employers in the form of vending machine orders, cafeteria menus, and price lists can speed up and increase the accuracy of data collection considerably. A final consideration is that although the EAT is basically an observational tool rather than an interview tool, situations do arise in which raters will need to seek clarification from site personnel and/or make additional observations about specific characteristics. Training sessions should review the EAT thoroughly and identify areas where additional information may be needed. As a case in point, it is relatively easy to document fitness equipment and facilities, but a tour of the site may not provide clear information about hours of operation or the access enjoyed by various categories of employees. Both of these factors are important considerations in evaluating this feature of the environment.

Third, validity assessment will always be challenging. It is not possible to directly answer the question of whether the EAT measures well what it is intended to measure. One will always have to infer an answer to that question by relating EAT scores to other metrics, using some logical framework. That is what we intended to do here, but others may wish to use other metrics for validity analysis, or may be able to craft other logical frameworks to be used for validity testing. As a result, inferences about validity will always be tentative, but the analyses conducted here suggests that the EAT may be a valuable tool for measuring environmental factors that are related to absenteeism in a current or future year, or for measuring environmental factors that affect health care expenditures in the current year. We believe the EAT to be a valuable tool for the researcher’s or practitioner’s arsenal.

The methods used in this analysis of the EAT have the following limitations:

First, since the program being evaluated was immature (only one year old), further research is needed to examine a fully developed program and the predictive (e.g., year 1 EAT with year 3 claims) and concurrent (e.g., year 3 EAT with year 3 claims) validity of the EAT.

Second, analysis of the EAT did not account for significant associations that are likely to occur randomly. With 54 analyses, one would expect to find two or three statistically significant associations purely by chance. We found 13 statistically significant associations, which suggest that chance alone does not explain the results.

Third, this analysis did not examine the ability of the EAT to predict future or show concurrent relationships between environmental factors and health behaviors, health risks (e.g. BMI), or disease conditions related to obesity, lack of physical activity, and poor nutrition. Further research is needed to examine these relationships.

Implications for Research and Practice

As shown in this paper, the reliability of the EAT was found to be high, which suggests that other researchers and practitioners can be confident that the EAT is clear in its data collection procedures relating to assessing the physical and social environments of a worksite and that it can be used reliably in these settings. Users are advised to estimate reliability and validity of the EAT independently, because neither reliability nor validity are inherent properties of any instrument. Rather, reliability and validity describe how instruments are used, and implementation may vary from site to site.

We also found evidence of concurrent and predictive validity, but primarily for relationships between environmental characteristics and absenteeism. Thus, researchers and practitioners may confidently use the EAT to assess the relationships between these factors. Concurrent validity tests also showed relationships between EAT scores and health care payments for claims incurred in the same year that the EAT was measured. However, we did not find significant relationships between EAT scores and future health care payments.

We conclude that the EAT is a useful instrument for auditing or assessing the characteristics of work environments that have the potential to facilitate or thwart healthy eating and exercise behaviors among workers that, in turn, may influence their absenteeism and medical expenditures.

APPENDIX

WORKSITE HEALTH PROMOTION ENVIRONMENTAL ASSESSMENT TOOL