Learning from a Natural Experiment

When we (Erin Kelly and Phyllis Moen) applied for funding to establish the Flexible Work and Well-Being Center at the University of Minnesota (as part of a larger National Institutes of Health, Centers for Disease Control [NIH-CDC] initiative to create an interdisciplinary, collaborative network on work, families, and health), we saw an opportunity to engage in a real-world investigation of an actual private-sector policy change aimed at lessening work-family conflicts and strains. This would be far different from simply studying possible associations between an existing policy and various work-family outcomes. We were interested in both (1) the process of organizational change and policy implementation and (2) the impacts of policy shifts aimed at reducing work-family conflicts and enhancing employee well-being at different points over the life course. We were especially drawn to this NIH initiative because it focused on policies changing work, not on changing employees (by promoting their coping strategies or teaching them stress-reduction techniques, for example).

Although neither of us had previously initiated a true policy intervention, we have always been drawn to engaged scholarship bridging the divide between theory and practice (Van de Ven 2007). The request for applications from NIH and the National Institute for Occupational Safety and Health suggested just such a possibility, one that would permit scholars to address a real-world problem by partnering with practitioners and stakeholders in the private sector.

Our goal in crafting the proposal was to theorize a workplace intervention that could potentially have high impact on employees and their families. To do so, we drew on our combined knowledge from (1) our previous research on flexibility policies and practices in organizations and the adaptive strategies of working families across the life course (e.g., Kelly 2003; Kelly and Kalev 2006; Moen 2003; Moen and Roehling 2005), as well as (2) the broad literature on workplace policies, job characteristics, and employee and family well-being, and (3) ongoing discussions with human resources professionals in corporate settings. How did we go from this knowledge base to crafting our actual proposal?

Research Problem and Theory Formulation

First, we recognized that most working families and, indeed, most employees face escalating time pressures, suggesting their need for greater temporal flexibility on the job. But a reduced-hours intervention didn’t make sense given the secondary status of part-time employees and the reality that most employees need a full-time income. It was equally clear that an organizational intervention based on existing flexibility policies would not be sufficient, since such policies often result in minimal options for employees (Kelly and Moen 2007; Still and Strang 2003). We did not want to invest our time and taxpayers’ money in studying yet one more policy officially “on the books, ” but in fact on the margins. We had previously observed that flexibility policies are actually available to a relatively small number of employees, often those who are especially valued by their employers. Existing flexibility policies may sound promising, but are not really integrated and legitimated within the culture and structure of most employing organizations.

Second, we were impressed by evidence from occupational health psychology, especially Karasek and TheoreH’s (1990) findings on the importance of job control for health. Karasek and Theorell have spawned a large body of theory and research (Bosma, Siegrist, and Marmot 1998; Bourbonnais et al. 1996; Butler, Gasser, and Smart 2004; Butler et al. 2005; Cheng et al. 2000; de Jonge et al. 2000; Dwyer and Ganster 1991; Fox, Dwyer, and Ganster 1993; Hemingway and Marmot 1999; Kristensen 1995, 1996; Landsbergis et al. 1992; Schnall, Landsbergis, and Baker 1994). Such evidence is also congruent with theories of the importance for health of self-direction and control more generally (e.g., Bandura1982; Heckhausen and Schulz 1995; Rodin 1986). Still other studies report findings that do not support Karasek and Theorell’s job strain model in certain populations or with certain health outcomes (Evans and Steptoe 2002; Marshall, Sayer, and Barnett 1997). Our own research (e.g., Kim, Moen, and Min 2003; Moen, Waismel-Manor, and Sweet 2003; Roehling, Moen, and Batt 2003) also pointed to the importance of employees having flexibility and control over the time and timing of their work. Taken together, this diffuse set of studies and theoretical developments, along with mounting evidence of the increasing time pressures on employees at work and at home, suggested that an ideal policy intervention in the corporate sector would be flexibility plus, with the “plus” being employees’ greater self-direction and control over their working time.

Third, we knew that some employees—especially those higher up occupational status ladders—already have considerable control over where and when they work. Scholars have previously shown (in mostly cross-sectional research) that control over the time and timing of work matters in predicting some outcomes (e.g., work-family conflict, work-life balance, schedule control; see Baltes and Heydens-Gahir 2003; Day and Chamberlain 2006; Linzer et al. 2002; Madsen 2003; Tausig and Fenwick 2001; Valcour and Batt 2003), but is it because those who have control over their time are also advantaged in other ways?

After many discussions with colleagues and graduate students, Moen and Kelly keyed in on the time and timing of work as a critical issue for employees, especially those with family responsibilities. We theorized the importance of employees’ control over the time and timing of their work as one—if not the most—important mechanism for reducing employees’ work-family conflicts, strains, and overloads.

In Karasek’s (1979) job strain model, job control refers to employees’ skill discretion and decisionmaking authority, that is, their control over how the work is done. It does not attend to employees’ control over when and where the work is done. We believe that control over work time is yet another dimension of control in the world of work, providing an important complement to the concept of job control. Control over work time (also called work-time control) is defined as the flexibility and discretion that employees have regarding the number of hours they work, the schedules (or timing) of their work, the predictability of their work hours if the employees are not choosing their schedules, and sometimes control over the location where they work, which affects their commuting time (Kelly and Moen 2007; Kim et al. 2003; Moen and Spencer 2008). This concept reaches beyond flextime to a broader understanding of employees’ control over the temporal conditions of their work.

Moreover, while job control is traditionally theorized in the job strain model as especially important for employees facing high job demands (Karasek and Theorell 1990), work-time control may be particularly important for employees with high family or job demands or both, since it offers employees greater ability to organize their work hours and/or work location in response to family and personal needs as well as work demands. Based on the existing job control research and previous research on schedule control and flexibility (Barnett and Brennan 1995; Carayon and Zijlstra 1999; Kim et al. 2003; Kossek, Lautsch, and Eaton 2005; Roehling et al. 2003; Thomas and Ganster 1995), we proposed a policy intervention designed to enhance employees’ control over the time and timing of their work, theorizing that greater work-time control should reduce the chronic stressors of work-family strains and conflicts, as well as time pressures on and off the job.

We proposed a multilevel intervention study grounded in an ecology of the life-course theoretical framing (Kelly and Moen 2007; Moen and Chesley 2008; Moen, Elder, and Luescher 1995; Moen, Kelly, and Magennis 2008), as well as stress process theory (Pearlin 1989, 1999; Pearlin et al. 1981). Our policy intervention would be framed in conjunction with stakeholders from a partnering firm and would be aimed at enhancing employees’ work-time control, what we saw as a theoretically motivated yet pragmatic intervention goal.

We hypothesized that increasing employees’ control over the time and timing of their work would be associated with less work-family conflict, greater time adequacy, and better health-related behavior and well-being outcomes, over and above other conditions characterizing employees’ job and family ecologies. Moreover, we expected that employees in different job and family ecologies (including their team configurations, occupations, and supervisory statuses, as well as their family statuses, ages, and life stages) might well experience both greater need for and greater benefits from increased control over their work hours and schedules. This is because of the particular confluence of demands and resources at home and on the job for employees in different occupations and job levels, as well as at different points in their life courses. Specifically, we were (and are) interested in effects of greater work-time control on employees at different levels in the organizational hierarchy, as well as employees who differ by gender, family, age, and life stage: young single employees with no children, single parents, married mothers and fathers actively raising young children, and married employees with no children at home, as well as those caring for an aging, infirm parent or a child with special needs and older employees thinking about retirement. We are also interested in similarities and differences, within and across occupational levels and teams, in the implementation of such a policy offering employees greater control over their work time.

We theorized an intervention that challenges the existing temporal organization of jobs by encouraging working practices designed to increase all employees’ degree of control over the time, timing, and scheduling of their work. This intervention would not, however, be a “one size fits all” policy or a single “treatment.” Rather, it reflects a process of replacing institutionalized clockworks with an emphasis on the quality of the job done. We envisioned this process of rethinking the temporal organization of work as taking place at the work-group level. This process is, of necessity, clearly tailored to the requirements of the type of work being accomplished. Our “dream” policy initiative would move away from the metric of time (of equating being “at work” with working), rewarding employees for their productivity and accomplishments, not simply their presence. We hypothesized that such a policy shift would serve to legitimate a more family-friendly and flexible corporate environment that, in turn, could enhance the well-being and effectiveness of employees at home and at work, as well as life quality of their partners and children.

Research Design

Given that the goal of the first phase of the NIH-CDC collaborative network was to conduct pilot work, our stated aims were to pilot test the implementation and impacts of a work-time control intervention by conducting a field experiment involving longitudinal (before and after intervention) research, using a range of both qualitative and quantitative methods. We proposed to investigate the implementation process as well as the process by which scholars can develop and sustain a long-term research partnership with key stakeholders within a corporation.

We were (and are) aware of no previous studies of private-sector policy interventions that have the enhancement of employees’ control over the time and timing of their work as their primary goal. Accordingly, we discussed several possible alternatives among ourselves and our graduate students and broached these ideas with several of our corporate human resource contacts in the Twin Cities area. As we prepared the proposal to be submitted to NIH, we faced two related challenges: developing a work-time control policy intervention that would be powerful enough to actually matter for employees and their families while simultaneously finding a corporate partner willing to launch such a broad-scale organizational design change and let us study the effects on employees. We aimed for a major reorganization and redesign of managing, moving away from clocking employees’ time. Such a redesign could be legitimated as “the way we work here” at an organization willing to partner with us. Note that we recognized that such a private-sector business policy offering employees greater control over when (and sometimes where) they work comes with two important caveats: employers might reasonably cede some control over work time to employees, provided the specific changes were tailored to fit various types of jobs and employees remained productive, meeting or exceeding the requirements of their jobs.

Despite our underlying concerns about access, we were sufficiently networked into the corporate community in the Minneapolis-St. Paul region to feel that it might just be possible to develop, roll out, and study an innovative work-time policy that would actually make a difference to the quality of life of employees and their families.

Multiple Challenges

To summarize, our objective was to develop and undertake successful research in a corporate setting, focused on investigating an intervention that would increase employees’ discretion over (or at least predictability about) when they work, and sometimes where they work and to do so within an ecology of the life-course framework. Only after receiving support for this project from the National Institute of Child Health and Human Development did we fully recognize its Janus-faced nature: the need for focusing on good science by maintaining the highest scholarly research standards, while simultaneously cooperating with our partners within a corporate environment.

In the following sections we draw on observational and interview materials (our own, as well as systematic field notes from graduate students on the research team, including Samantha Ammons and Kelly Chermack) to capture the challenges of this double focus. We describe some of the difficulties and accommodations but also some of the rewards and lessons learned from research in a corporate environment. These include (1) gaining access to a corporation willing to consider and pilot test an initiative offering employees greater work-time control, (2) dealing with multiple, often conflicting timetables, and (3) implementing a rigorous research design that can offer the best scientific evidence. We conclude with (4) a brief overview of some early findings and (5) the usefulness for good science of engaged research in corporate settings.

Gaining Access: Developing a Corporate Partnership

Serendipity

Robert Merton wrote about the importance of serendipity in research (see Merton and Barber 2004), along with the fact that scholars rarely explicitly acknowledge accidental discoveries or circumstances as key ingredients of the research process. Our experience gives credence to his serendipity thesis: being at the right place and the right time for the “accident” to occur. Early in 2005 (see timeline, figure 5.1) we (Moen and Kelly) were doing our “dog and pony” presentation to interest corporations in this project (should it be funded). Members of an informal “think tank” of work-life practitioners and researchers that we participate in were invited to come to a presentation by two members of this group, Cali Ressler and Jody Thompson, who were employed at the headquarters of Best Buy, a large retail corporation headquartered in the Twin Cities. We sat stunned as they laid out ingredients of what sounded very much like our ideal work-time control intervention. We dared not even look at one another across the room because we were at once startled, pleased, and excited about the fit between the Best Buy innovation called ROWE (results-only work environment) and our own thinking about the importance of enhancing employees’ control over the time and timing of their work. Both of us were thinking, “What a lucky coincidence! Could ROWE be our intervention? Could the stakeholders of Best Buy become our partners?”

Figure 5.1.

Timelines for Results-Only Work Environment Study

ROWE was developed as an internal organizational intervention at Best Buy. It was designed to move employees and supervisors from existing, implicit contracts about the expected amounts of time at work toward a more explicit contract based on what is required by the job and what are appropriate measures of employees’ effectiveness in the job. Teams transitioning from conventional time-based work practices to a ROWE arrangement aim to foster an environment where employees are free to complete their work whenever and wherever, provided they are productive and doing what works best to accomplish the tasks at hand, as well as each team’s longer-term goals. This type of working environment shifts the spotlight away from time-oriented measures of work success (how many hours a worker put in last week; how much time she spent on a given task) to a completely results-based appraisal of productivity and accomplishment. With the ROWE innovation, Best Buy aims to change the temporal organization of work, shifting to an environment where employees have the tools (ways to roll over phone calls, etc.) they need to accomplish their assigned objectives while simultaneously giving them the freedom to accomplish their tasks with whatever type of work schedule is best for them. The participatory process in which teams create a new work environment—with new expectations, assumptions, and interaction practices—is called the ROWE “migration.” Table 5.1 describes the differences between the ROWE innovation and existing ways most employees work. (See also Kelly and Moen 2007 for a discussion of how ROWE contrasts with common flexible work arrangements.)

Table 5.1. What is ROWE?

Results-only work environment. A transformation spotlighting and rewarding productivity and job requirements, not time at work or scheduling, and customized at the work-group level.

From	To
A focus on work hours (just being there or “face time”)	A focus on job requirements (doing work well and on time)
Supervisor sets hours, schedules	Individual and team set hours, work times, schedules
Meetings are a regular part of work routine	Meeting held only as need
Reliance on face-to-face interaction	Varied methods of virtual and transparent communication
A “reactive” orientation, dealing with crises as they occur	Proactive, early planning to avoid crises where possible
Flexibility arrangements negotiated between individual and supervisor	Flexibility is the norm. Team members cross-train to cover for one another and set schedules
If work needs are met, presence is still required	Customized work time and schedules aimed at achieving goals
Essential ingredient: tracking employees’ time spent working	Essential ingredient: defining specific nature of job and expectations
Problematic: absenteeism, tardiness, presenteeism	Problematic: not meeting job deadlines, expectations

Note that nowhere does the ROWE initiative mention work-family issues. This is deliberate on the part of its creators, as a way of not pigeon-holing ROWE as yet another “mother” friendly or even “family” friendly initiative. Rather, the rationale for the ROWE innovation emphasizes better work results for the firm by moving away from working time as a gauge of effectiveness. It is an innovation meant to be applicable to all employees at all job levels, regardless of their family or personal circumstances, and to be understood as a strategy for recruiting and retaining a skilled workforce.

We immediately recognized the potential value of RQWE as a transformative policy challenging the existing temporal design of paid work and fostering an environment that might reduce employees’ work-family conflicts and offer the possibility of greater work-family (or work-life) integration. As work-family, policy, and life-course scholars, we find ROWE particularly promising because, as Ellen Ernst Kossek and Brian Distelberg (this volume) write, it incorporates the “three-legged stool” of reducing work-family conflict through human resource policy, informal organizational culture, and rethinking the structure of work (see also Kossek 2006).

Timing

ROWE met all of the criteria that we had laid out for a potentially powerful intervention targeting employees’ control over the time and timing of their work. We quickly met with the innovators at Best Buy (Jody Thompson and Cali Ressler, now of CultureRx) to discuss the feasibility of a possible research partnership. Our argument went like this: The research team from the University of Minnesota {UMN) could promise Best Buy an objective, outside assessment of ROWE, while Best Buy could serve as our corporate partner in our NTH-funded research. Because this partner had already developed what amounted to a work-time control policy innovation, this was truly an “experiment of nature” (Bronfenbrenner 1979), taking place whether or not we studied it. The frustration was that’ Jody Thompson and Cali Ressler were ready to roll out ROWE before we could know if our proposal would be funded. We decided not to wait, hiring graduate students with funds from the Alfred P. Sloan Foundation (after seeking the approval of program director Kathy Christensen for this reallocation of existing support) in order to begin observing the ROWE migration right away.

But first we had to obtain the cooperation and buy-in of Best Buy management, someone at the senior level who could and would endorse the UMN/Best Buy partnership, including giving members of the research team access to the organization. We sought approval not only to interview and survey a sample of employees, but also to be allowed into the organization, permitted to attend ROWE sessions, and observe work groups. Our goal was not only to study the impact of ROWE on particular outcomes, but to capture the process of organizational transformation as ROWE was introduced, rolled out, interpreted, and implemented by work teams, individual employees, and supervisors.

We met with a vice president at Best Buy who signed off on the study and our access to the corporate headquarters. We were pleasantly surprised at the ease of our first entry, which was clearly facilitated by Ressler, Thompson, and their staff. As we discuss later on, there were some challenges once other officials from the two institutions (i.e., attorneys) got involved, but the initial entry and partnership was quite simple to negotiate.

Ongoing observation was key, given that ROWE involves a process of migration, not a one-shot treatment, and is implemented at the team level. We observed ROWE facilitators holding periodic meetings over a period of weeks: first with the leaders of a work group (team), then with all the members of the group, including the leaders. We were in the room when group members worked through what it would mean for their team, given its mission, to focus on results, not time at work. In these sessions we watched as employees and supervisors brainstormed about barriers and possibilities, as well as strategies to be more effective by getting rid of “low-value” work (such as regular meetings that have no real purpose) and focusing on the results expected of the team, as well as of each employee. Throughout the period of migration, employees and supervisors experimented and reported back in the ROWE sessions as they attempted to change the way they worked over a period of weeks. We regularly sat in on these sessions, following teams through the migration process and observing the transformation as it was taking place.

Throughout the research process we walked a fine line between creating an atmosphere of openness and trust while simultaneously keeping our distance. We remained somewhat friendly, but not friends, with people at Best Buy throughout the years of data collection. We knew and conveyed to our partners that, unlike some consultants they might hire, we were independent academic researchers, funded by the federal government, and would report findings based on the best scientific analysis, which might well include evidence they might not want to hear. That said, the trust that developed on both sides of the partnership was invaluable.

Even though we kept a degree of social distance, graduate student field researchers “adopted” teams to study in depth, developing much closer ties to team members. The development of relationships with employees they were observing and shadowing lead to enhanced communication, acceptance, and becoming “one of the team.” There were many aspects of these relationships that made our graduate student researchers truly feel like participants as much as observers. They were invited to team lunches, team-development outings, off-campus meetings, birthday celebrations, and even a baby shower. This led to ongoing discussions among our research team about the process, ethics, and implications of participant-observation research.

The graduate student researchers also became closer to the leadership team rolling out ROWE. Two examples from field notes during observations of ROWE sessions show this:

She [a facilitator, who worked with Thompson and Ressler] said hello to me and I said hi back. We did the usual, haven’t seen you in a while deal and asked how things had been. She … commented to me that [two of the facilitators] are in Mexico and that [another facilitator] is really sick, so she’d be doing this session on her own and that she was pretty nervous. I told her not to worry, that she’d do fine and took a seat in the back of the room near the door.

One thing that I found particularly interesting about this group was that [the facilitator] began to use the word “we” a lot. She would say things like, “we’re going to show you a short video…” And, “look over at the charts we put up…” Since I’ve gotten in the habit of helping her figure out A/V issues and help put up and take down her giant stickies before the sessions, she has said to me a number of times that, “we make a good team… and she doesn’t know what she would do without my help… ”

This hands-on participant observation led to a fuller understanding of the social ecology of employees in teams transitioning to ROWE, as well as those in comparison-group teams who were not yet migrating. Employees were able to share a tremendous amount of information because the graduate student interviewers already had insider knowledge and understanding of the dynamics of the team and each employee’s unique time pressures. When the graduate students conducted semi-structured, in-depth interviews, several employees commented that it wasn’t so much that they were being “interviewed” as it was having a long “chat” with a familiar individual.

Lesson 1: The Importance of Relationships

Our experience underscores the importance for conducting workplace research of seeking out and maintaining ongoing ties with professionals employed within corporations. We knew Thompson and Ressler only slightly prior to becoming Best Buy’s research partner, but our participation in the local work-family network (of human resource professionals and academics) was key to even learning about their initiative. We had already envisioned this group as a potential source of partnerships, and this turned out to be the case.

Fostering trust and deeper connections with Ressler, Thompson, and other stakeholders at Best Buy was essential to gaining and maintaining access. We were up front about research progress and our inability to offer tentative findings early. Best Buy folks were equally open about the ROWE implementation. Relationships that we developed with other leaders in the organization, as well as with the employees who participated in the field research, provided valuable information and insight.

Managing Multiple Timetables

Even though we believed in and were studying ways people could loosen time constraints, our research team itself operated under considerable time pressures, exacerbated by multiple, overlapping, and sometimes contradictory timetables. Study design requirements necessitated collecting ethnographic evidence at Best Buy’s corporate campus right away, because we wanted to capture the entire ROWE implementation process. Since our corporate partner was rolling out ROWE on their timetable, not ours, we had to fashion our research effort around their timetable (see figure 5.1).

Then there are the timetables related to good science. Before we could begin, we needed permission from the institutional review board (IRB) at the University of Minnesota, which required evidence that the proposed research would not harm human subjects. Did we need the IRB approval before we could even sit in on Best Buy meetings? We weren’t sure, but thought we should proceed as if it were necessary, delaying our entry into the field. And then there was the research imperative of launching a pretest survey with our target sample before they were to undergo the ROWE migration, necessitating cobbling together a survey, which could not be launched prior to approval by the IRB. The Janus-faced aspects of this project in the form of conflicting timetables were becoming evident.

There were also the UMN’s own timetables, requiring teaching and committee meetings for Kelly and Moen and classes for graduate students. We laughed at the contradictions of doing research on an innovation designed to give employees more work-time control, while the actual research process was creating layers of deadlines and time pressures for all the members of the research team.

The different cultures of research and business became even more evident when dealing with attorneys at UMN and at Best Buy who aimed to create a formal agreement for the research partnership. We had waited for IRB approval before beginning the study, but decided to let both legal representatives fashion the officially authorized nature of the research partnership even as we simultaneously moved into the field. We were wise to allow the attorneys to work this out, since their cautious pace would have seriously slowed the research.

There were also timetables related to employees’ expectations. For example, people at Best Buy wanted to know “what we were finding” almost as soon as we started our observations. After each survey, several respondents asked what we “had found.” We recognized that even as we knew little about organizational processes, employees knew little about research processes—the time it takes to clean and code survey data, much less to transcribe and code in-depth interviews and field notes. Employees in this corporation are used to brief internal surveys and equally brief descriptive summaries of findings that are available right away. They were amazed at how slow we were.

Lesson 2: Research Takes Place on a Moving Platform of Obligations, Expectations, and Change

As our timeline indicates (see figure 5.1), the researchers and the stake-holders {management, the innovation team, employee respondents) faced numerous pressures throughout the study period. Sometimes we found ourselves and our colleagues at Best Buy facing similar time pressures. But whether concurrent or staggered, these pressures affected the research process as we moved back and forth between business and academic timetables.

Moreover, the “targets” we were studying did not stand still. Best Buy was moving into new markets (such as China), dealing with external challenges, and designing new stores and delivery systems. There was one bout of extensive layoffs. The lives of employees were similarly in flux, with 45 members of our sample voluntarily leaving the corporation. Those who stayed experienced a plethora of life changes in the six months between our two surveys (see figure 5.2).

Figure 5.2.

Life Changes of Employees in the Six Months between Surveys (percent)

Implementing a Rigorous Research Design

The proposal we submitted to NIH called for a multi-method field experiment, including two waves of surveys completed by employees and managers in both ROWE and comparison teams. The objectives of the surveys are to assess the potential impacts of a work-time control intervention on a range of possible outcomes, including changes in employees’ perceptions of work-time control; their actual behavior regarding when and where they work; employees’ work-family conflicts and other aspects of the work-family interface; employees’ health and health-related behaviors; and employees’ job satisfaction, commitment, turnover expectations, involvement, and perceptions of the organization. These two computer-based surveys would constitute the main data source for outcome measures.

We also requested access to administrative data collected by the corporation, which would permit us to capture any “harder” outcomes, such as changes in actual health care usage and costs and changes in productivity measures or turnover. We were eventually able to obtain some institutional health records and turnover statistics from Best Buy, but the organization was unwilling to share other data related to productivity.

Our research design also called for analysis of the organizational process of implementation. We did this by observing the implementation of ROWE through daily team observations including shadowing and by doing in-depth interviews with team members. Since understanding the actual process of ROWE implementation required ethnographic observations (as well as in-depth interviews) over time, four study-team members (the authors and two graduate students, Kelly Chermack and Samantha Ammons) sought and obtained regular access to the headquarters campus. Our 16 months of fieldwork and interviewing at Best Buy was a unique experience. As Susan Lambert (this volume) discusses, such workplace field research allows for important insights into the everyday work of employees, insights crucial to our understanding of the work environment and its challenges. Our multi-method approach provided us with a vast knowledge of workers’ daily tasks, behaviors, attitudes, and experiences that we would not have otherwise been able to comprehend, further enabling our understanding of the impacts of the work-time control innovation.

Confidentiality was absolutely crucial to the research process, both for ethical reasons and to maximize the validity of our evidence. All respondents (surveyed, interviewed, or shadowed) received written and oral assurances of full confidentiality. We guaranteed that any information we collected would be kept strictly confidential. Not only would we use pseudonyms for employees in publications, but we assured respondents that no identifying information would make it back to their supervisors or colleagues. We feel that our assurances of confidentiality, together with the climate of trust we actively fostered, allowed many of our study sample, particularly those who completed in-depth interviews, to really open up and share their perspectives without the worry of potential job penalties. The research also benefited from the company’s identity as an organization that values transparency and open communication. While employees did not always experience the cooperation of their managers as open and transparent, they had been repeatedly exposed to the idea that it is part of Best Buy’s culture to be open and transparent.

Our original research design mapped nicely onto the ROWE innovation, with the important exception that we were not able to randomize employees into an experimental and a control group. Rather than launching a true field experiment (e.g., Wilier and Walker 2007), we were privileged to observe what Bronfenbrenner (1979; see also Bronfenbrenner and Crouter 1983) calls an “experiment of nature,” or what Van de Ven (2007) terms a “quasi-experiment,” occurring ready-made at Best Buy without any involvement on our part.

The process of choosing teams to observe, survey, interview, and shadow required both patience and persistence. The basic order of events began with Thompson and Ressler identifying teams that might soon undergo the ROWE migration, as well as teams whose ROWE migration was, not imminent, so these teams could serve as comparison groups. Usually after the researchers had tentatively selected a team, Thompson or Ressler would approach the team manager and the relevant vice president to see whether they might be willing to have their unit participate in the research. If the manager and the vice president were willing to cooperate, a meeting between them, Thompson and Ressler, and our UMN team was then arranged. During this meeting, the UMN team was able to meet with these parties to ascertain whether there was a fit and an agreement from all parties. We then set up another meeting with the manager and employees in that team and our graduate student researchers so they could all get to know one another, and the researchers could introduce the informed consent and participation requirements in more depth. For the most part, everyone—vice presidents, managers, and line employees— was cooperative and very willing to participate. We had a few vice presidents who expressed some concern over just how much of their employees’ time we would consume. Aside from that, we experienced very little hesitation or resistance on the part of employees, supervisors, or corporate leaders.

Fitting In

In April of 2005, we were able to begin our observations. Although the timelines for starting the study sometimes felt slow and frustrating, this was only four months after we first learned about ROWE. When we received our electronic badges we were allowed to come and go at the Best Buy headquarters campus as we pleased, using the employee entrance and not being considered visitors (who had to be escorted) anymore. It was exciting to be somewhat “official.” Our badges were green, indicating that we were independent contractors, not regular employees. But they were legitimate, giving us a sense of legitimacy as well. Still, getting in did not mean fitting in. Because we knew that we were interlopers, we felt that everyone else must know it too. Our field notes underscore this discomfort.

Attending a leadership panel. We arrived a little early and entered into the ramp. After chatting a while in the car, we headed into the north entrance. My badge worked fine and Phyliis’s set off the alarm but a crowd of 4 or 5 people deliberately set off the alarm as a joke, pointing at a woman in their group with big smiles when the guards looked up. So Phyliis’s alarm wasn’t too obtrusive. She got her badge easily, and we chatted about the thrill of being semi-official and free in the corporate setting. We waited a few minutes, and I felt awkward because we weren’t going anywhere or working in the public spaces. So even though there is plenty of room to “hang out,” hanging out and looking around makes me feel unproductive and awkward.

Observing in the common areas. It feels weird and kind of thrilling to be walking a round in the building without an escort. I am excited to have my green badge but still feel that it must be obvious to everyone that I don’t really belong there. As we sat there, I felt awkward because we were not talking enough to be “normal” in that space. Sam (graduate student) was looking around with more intention than I was. I was aware of the security guard who could see us and seemed, to me at least, to be watching us and our note-taking. I realized I had a half smile plastered on my face ….

Another challenge that we faced as academics being thrust into a large corporate organization was getting a grasp of its organizational culture. From our first visit to Best Buy, we faced the task of learning about the organizational culture through our own observations and experiences. We had to grow accustomed to their rhythm of work. For example, there is the “seven-minute rule.” We would arrive at a scheduled session on time, only to watch employees file in between six and seven minutes late.

We also had to learn an entirely new vocabulary. During ROWE training and subsequent team adoption sessions, we took notes while simultaneously trying to understand the process and terminology. “Drivebys” (coworkers randomly stopping by an employee’s cube) and “firedrills” (last-minute deadlines that require employees to drop what they are currently working on and “put out the fire”) are examples of aspects of this culture that we could only learn by watching and listening. But learning their language and culture was key to conducting optimal in-depth interviews because respondents assumed correctly that we “knew” their daily work lives and routines and what they were talking about.

Ambiguous Identities

Early in the process, collecting data sometimes seemed awkward because we did not have any official role at meetings. Even though we planned to introduce the study at each session and the facilitators agreed to give us two minutes to do so, we were not always introduced, and even when we were it sometimes felt awkward. Whether it was being ignored or being put on the spot, we often felt the anxiety of being different from everyone else. Examples abound from notes about attending various sessions.

I was looking around the room and caught Tyler looking at me once and then quickly glancing away when he noticed I saw him. I took fewer notes than I usually did and tried to fade away into the wall as much as possible. I felt REALLY uncomfortable.

We’re already way into this meeting and Jody has not asked me to introduce myself. Obviously the group knows I don’t belong there, but no one asks me who I am. Half of the time Jody and Cali forget to have me introduce myself, but with a group this small, it’s a little awkward. I’m never sure what I should do though, so I continue to sit there quietly.

I feel like everyone expects us to take notes during the sessions, we’re there to observe those sessions. But, I feel weird about writing notes before and after sessions or during the casual conversation that Sam and I witnessed with Jody and . the HR staff… I’m not sure which is better, to witness the conversation and then try and get down as many notes as possible after, or risk writing during the conversation, knowing that people may be more likely to say less and censor their comments with us in the room writing.

Sometimes we felt very visible, as when our security badges expired because of a computer error and no longer granted us access to the organization.

I’m entering the parking ramp at 8:30 and I have a ton of cars behind me, and my badge won’t let me in. I thought about pushing the red button and asking to be let in, but I wasn’t sure what to say or if they’d even let me in… I wasn’t sure what to do, and I was contemplating just going home. But, this was the day I was supposed to shadow Dick and I had already rescheduled once and didn’t want to bail on this day. I got out my cell phone and called 411 and asked for the Best Buy corporate office. When the operator at Best Buy answered, I asked for Corinne (she’s always here this early) and told the operator that I couldn’t remember her extension. She gave me the number and then offered to connect me… Corinne answered, thank god, and I told her the situation. She said to just come to the visitor’s entrance and then she’d come down and get me, which is what I was going to ask her to do. I apologized to her and she said that “you’re funny” and that it would be no problem to come and get me. I parked in the visitor’s lot and headed to that entrance.

Events like this reminded us that our access was a “gift” and could be rescinded at any time.

In an effort to reduce the amount of employees’ time we consumed, we tried to be accommodating to their schedules, keeping the interviews as short as possible. However, it took about an hour for the interviews for those in the comparison groups, and an hour to an hour and a half to interview members of the treatment groups. We interviewed everyone twice, before and after the ROWE migration. Paying special attention to employees’ needs and time constraints sometimes meant splitting up the interview into two parts or ending the interview and resuming it later if need be. We made every effort to be sensitive and responsive to time pressures.

Dick, a 27-year-old white male buyer, had told me beforehand that, although we’d reserved an hour for this interview, it was really difficult for him to give that amount of time right now. I thought about rescheduling, but it had been difficult to schedule with him in the first place and I didn’t want to risk it. So I decided that we could go ahead and do this interview now, even if it ends up being short. I did feel a little rushed to read a question, wait for a brief answer, and then move to the next question.

We also tried to be sensitive about having graduate students shadowing employees: spending an entire day with an individual, attending his or her meetings, observing desk work, having lunch with him or her, and so on. Our goal was to garner as much information as possible during our limited time at Best Buy while interfering with employees’ daily work as little as possible. As an example from shadowing notes shows, we made every effort to observe employees in their work environments in an unobtrusive way.

Seriously, when do these people use the restroom? Maybe it’s just because I drink a lot all day, but they never seem to go. Daphne, a 35-year-old white female buyer, told me once before that sometimes she just doesn’t have the time to go; I can’t even imagine that!!!

As we, the researchers, and our subjects, the employees, became more comfortable, being there became even easier. After a while, the graduate student researchers were just another part of the usual scenery and were able to ask questions, participate in water-cooler gossip, and attend meetings and lunches just as if they belonged there. Once it seemed normal to have us there, our own stress at feeling intrusive slowly disappeared.

Shadowing employees proved to be simpler than we had thought. Most employees were happy to have us follow them around. In fact, a number of them found it comforting to have an unbiased individual to confidentially “vent” to throughout the day. Most of the time, employees would share their own commentaries and thoughts about how a meeting went or how much they liked or disliked a coworker or supervisor. This gets back to the importance of trust. Some of the employees tried to skirt the shadowing, but eventually everyone we needed agreed to participate. There were also employees who, at the mention of the possibility of being shadowed, could not wait for it to happen. One of our researchers spent the day with Grant, a 27-year-old man who was a demand planner in marketing, for instance, when another employee actually asked if she could also be shadowed.

When we were done with copies, we swung by Reese’s desk so that she could take a look at the finished handouts. Reese actually remembers my name and exactly what I’m doing here. I’d only met her a couple of times, and I was impressed. Grant told her about how I was shadowing him today. “I want you to follow me!” she said.

In relation to our own sensitivities about their time, we began to notice when employees were under a great deal of time pressure and stress. The marketing teams especially had weekly deadlines and updates for which they were responsible. This created stress and used up a large amount of their time, and the process was repeated week after week. The teams in marketing faced unique weekly time pressures, aside from the cyclical stress of holiday preparations and quarterly and annual reports. On a weekly basis they spent most of the day on Monday preparing reports on sales from the previous week and adjusting their forecasts for the upcoming week. They also encountered increased stress on Friday when they had stores calling for last-minute increases in shipments for the weekend. We attempted to be sensitive to the ebb and flow of their time pressures and worked with their schedules as best we could. This meant extending our stay with some teams to be sure we had gathered all the data we needed. Rescheduling interviews and shadowing, as well as rescheduling our own meetings, tended to draw out this process longer than we had expected.

This is part of the balance required with any study partnership. There are obvious goals for and challenges to both parties. And we were acutely aware of the tenuousness of our situation. At any moment, the organization, the teams, or the people we were observing could withdraw from the research process, or even kick us out of the organization entirely. This awareness made us even more sensitive about our presence on campus and our level of intrusiveness. There was, however, never an instance of our legitimacy being questioned. We were never told that we could not do something. The cooperation that we experienced was incredible.

Lesson 3: Aim at Minimal Intrusiveness

It is often said that the business of business is business. Just being on the Best Buy campus reinforced this fact; people were there to do their jobs, not to participate in our study. While research was of course our priority, it was not theirs. Accordingly, we tried to reduce our visibility and “footprint” by being minimally intrusive and accommodating our research needs to employees’ availability.

Does ROWE Make a Difference? Early Survey Findings

Recall that the aim of the UMN study was to assess whether working in a ROWE environment changes the nature and quality of employees’ work experiences, reduces conflict between employees’ work and their family or personal lives, affects employees’ health and health-related behaviors, and alters their commitment to and perceptions of the organization. We briefly summarize early findings from two surveys of 658 employees conducted six months apart. Characteristics of this sample are shown in table 5.2.

Table 5.2.

Sample Characteristics of Those Participating in Two Survey Waves (percent}

Characteristic	Total N=658	Men n=339 (51.5%)	Women n = 319 (48.5%)
Family situation
Not married, no children	30.2	30.1	30.4
Married, no children {or none at home)	34.8	34.2	35.4
Children at home (mostly married)	35.0	35.7	34.2
Age group
20–29	45.3	42.4	48.5
30–39	39.2	40.9	37.3
40–60	15.5	16.7	14.2
Job level
Individual contributor	66.3	63.4*	69.3
Manager	19.8	19.8	19.7
Senior manager	14.0	16.8	11.0
Exempt status
Exempt	95.0	97.0**	92.8
Nonexempt	5.0	3.0	7.2
Education
High school or less	1.6	0.9	2.2
Some college	12.8	14.6	10.8
Bachelor's degree	73.2	70.4	76.1
Graduate or professional degree	12.5	14.0	10.8
Tenure at Best Buy
Less than 1 year	18.8	17.3	20.5
1–5 years	46.7	45.2	48.3
Longer than 5 years	34.5	37.5	31.2

Source: Authors' data and calculations.

^*p< .10;

^**p< .05;

The sample is divided roughly equally between employees whose teams began the ROWE migration just after the first survey and a comparison group of employees in teams who were not yet slated for ROWE at the time the surveys were launched. We compare differences between the ROWE and comparison-group respondents in terms of any changes employees experienced in the six months spanning the period before and after the ROWE migration.

We find no statistically significant differences between ROWE and comparison-group respondents in terms of changes (in the six months between pre- and post-study surveys) in the hours they put in on the job, their income adequacy, or any positive spillover from work to family or from family to work. Neither do we see any meaningful difference in the shifts between waves in the level of negative spillover from family to work. Analysis to date has also not detected significant ROWE effects on changes in employees’ satisfaction with their managers or coworkers, in their assessments of their own individual and their teams’ performance, or in their decision authority on the job. Not significant, as well, are ROWE versus comparison-group changes in employees’ overall assessment of their health, their psychological distress or well-being, their emotional exhaustion, their sense of mastery, or their number of physical symptoms.

We do, however, find evidence that ROWE has statistically significant impact on changes in employees’ sense of control over their work time, their decisions about where and when they work, their sense of work-family conflict, some aspects of their health and wellness, and their work pressures and commitment. The evidence is particularly convincing because we examine changes within people over the six-month period, finding different patterns of change for ROWE and comparison employees. Summarizing statistically significant changes in work-time control and working patterns, we find the following.

• Fewer ROWE employees than comparison employees have had their commute times to and from work increase.

• More ROWE employees than comparison employees have greater control over where and when they work, choose to work at home or off campus more frequently, and have greater variability in their work hours and schedules each week.

We also find changes in the work-family interface.

• More ROWE employees than comparison employees experience a decline in work-family conflict; have less negative spillover from work to family; report more time adequacy in terms of doing their work, family, and personal observations; and describe the Best Buy culture as more family friendly.

There were also significant changes in some health and health-related behavior outcomes, specifically, by the second survey.

• Fewer ROWE employees than comparison employees come to work on campus when sick.

• More ROWE employees than comparison employees sleep more than seven hours a night, see improvements in the quality of their sleep, now go to the doctor when sick, exercise more frequently (three or more times a week), and report gains in energy.

Finally, we document some changes in work conditions and effectiveness between the two survey waves.

• Fewer ROWE employees than comparison employees do low-value (unnecessary) work, have high turnover intentions, experience inter ruptions at work, and feel pressure to work overtime.

• More ROWE employees than comparison employees view the work culture as being family friendly, have greater organizational commitment, and report more job satisfaction.

Our findings suggest that ROWE maybe a realistic work-time innovation with potential for broad adoption and impact. It was developed in a company rather than by academics, providing it with the legitimacy that may aid in its diffusion to other organizations. At the same time, it also makes sense in light of theory and research findings on work-family time pressures and the importance of a sense of control for health, well-being, and effectiveness.

But our evidence comes with certain caveats. First, this initiative occurred in the headquarters of a major corporation with primarily white-collar and professional employees. ROWE in a retail setting, where employees’ jobs require their physical presence, would take different forms and require different types of strategies to increase employees’ work-time control. Second, six months between surveys is a short period in which to capture changes in work-family and health arenas, and yet funding constraints made it impossible to field a third survey later on.

Our next steps will be to assess the impact of this workplace intervention within particular family and job ecologies. We also want to develop an understanding of the implementation process through comparisons within and across work units.

Summing Up: Getting to the Science

ROWE is what Bronfenbrenner (1979) called an ecological transition— a fundamental change in the setting in which a role (that of employee) is played out. It is truly an experiment of nature as Bronfenbrenner and Crouter defined changes that occur naturally (i.e., without the intercession of researchers). As such, it is ideal for the study of the effects of environmental change “with a built-in, before/after design in which each subject can serve as his own control” (Bronfenbrenner and Crouter 1983, 381).

Our field observations and investigations of this quasi-experiment in a corporate setting have taught us more about work, family, and organizations—and organizational policy change—than we could ever learn from the analysis of a random sample of adults working for various employers. We have offered our lessons learned from this study: about i he importance of relationships with key stakeholders, the need to recognize and respond to multiple timetables (of employees, leadership, and researchers), and the value of being minimally intrusive in work settings. We conclude with a fourth lesson.

Lesson 4: Recognize and Accommodate Gaps between Ideal Research Design and Corporate Realities

An “experiment of nature” has many built-in strategic advantages. For instance, the design of and the case for the policy innovation have been made by those in the corporate setting and do not need to be developed or argued by the research team.

Still, there are undeniable drawbacks. We previously raised the issue of the impossibility of random assignment in this natural experiment setting, a factor making this a quasi- rather than a true experiment. Other modifications and accommodations were also necessary. For example, in our proposal we noted that our ecology of the life-course approach would include analysis of older workers and their work-time needs and wants as they move toward retirement, but Best Buy’s workforce is relatively young, providing insufficient numbers of older workers to undertake such analysis. There is also the issue of “generalizability”— including how ROWE would operate in other industries, with other workforces. That question can only be addressed through replication. For us, these modifications and accommodations were clearly worth it as part of our effort to get rich and detailed data about the process of organizational change and the impact of work-time control on employees.

Two other challenges to traditional research design—a lack of consistency in the treatment or intervention and the problem of contamination across work groups—puzzled us more and became the subject of many conversations. We have not reached firm conclusions yet, but we share our thinking here.

No control over timing of intervention and no way to keep intervention “the same.” We all know that ideally an intervention delivery should be consistent, but even as we entered the corporate environment in order to learn about organizational change, so too did the organization continue to learn from itself. This meant that the change process itself adapted and innovated over the course of our study. This raises a number of potential challenges because the specific content of ROWE shifted over time. We responded by carefully tracking and monitoring these changes, by attending well over 100 ROWE sessions and taking detailed field notes in each. For example, the following excerpts from our field notes track changes in the session that prepares department leaders for ROWE:

Jody and Call started the slide show, and [their phrase] “results-oriented work environment” has now become “results-only work environment.” Jody said that oriented was “too wimpy” and that “results-only is stronger.” She explained that they changed it because the “new teams are getting savvy.”

The changes I noticed since the last leadership session I’d seen were the glossy slides and also the guideposts arc shown and then there is a section with each guide-post and “What it is” and “What it isn’t.”

She (Jody) mentioned that they were doing sludge [a way of having employees recognize how traditional time-related work expectations creep into conversations] a little differently now… a script change that they were trying out.

Moreover, ROWE is less a one-shot intervention than a process of participatory change. Not all employees receive the same exposure, since the intervention specifically encourages teams to make it their own. The result? Different “pockets” of ROWE where individual teams developed their own version of how ROWE would look and “rules” to accompany their version. We believe that ROWE has powerful effects in part because of this participatory process, allowing employees and teams to experiment with new ways of working that meet their personal needs and the needs of the business. However, this change strategy requires that researchers be vigilant in capturing and analyzing those experiments across both individuals and teams.

No way to prevent contamination

Good science requires that the comparison (control) group should be totally separate and independent from the “treatment” group, something nearly impossible among employees working in the same building. Best Buy is known for its cross-functionality and teamwork, and prides itself on developing relationships. In fact, one internal measure of success is the high number of employees who report having “a best friend at work.” With cross-functional work networks and highly developed social networks, we found that at least some information about ROWE was known by employees in different parts of the organization. Whether it was factual information or just a rumor, employees were talking. This meant teams that were beginning ROWE or teams that were considering adopting ROWE often had preconceived and sometimes misguided perceptions early in the process. We also found that as teams learned that they would begin ROWE in the near future, they sought out information and best practices from other teams that were already in the ROWE environment.

These information flows were not ideal for the research and were also viewed as problematic by Thompson and Ressler. They found it more difficult to implement ROWE in teams that had already planned how they might change or assumed they had prior knowledge regarding what to expect and how to incorporate ROWE within their teams. Yet, considering the information that they were able to access through regular organizational communication channels or sought out themselves, every team that we studied had at least some prior expectations going into the ROWE migration.

Why Engaged Research Is an Optimal Design for “Basic” Science

Urie Bronfenbrenner (1979) always said that if you want to understand something, try and change it. Bronfenbrenner also said that nothing was more useful to furthering basic, theory-driven scholarship than research on a particular policy. What have we learned?

The three years we have been engaged in this research project at Best Buy have demonstrated in concrete ways the absence of life-course “fit” between occupational and family clocks (Moen and Kelly forthcoming; Moen and Roehling 2005) and the way this lack of fit plays out in the day-to-day lives of employees. The costs to employees, their families, and business of this chronic, systemic misfit between the two most fundamental institutions for human development—work and family— are evident to the observer but not always tangible in the ways that other costs are calculated.

Studying the process of implementation of a temporal policy shift at Best Buy provided ample evidence that trying to alter some aspect of the engrained, taken-for-granted clockworks of work is extremely difficult. We had not expected the degree of difficulty employees experience in trying to think of work as outcomes, not time put in at a desk or elsewhere. Neither had we anticipated the degree of pushback from others made uncomfortable with this new definition that changes basic assumptions about the “right” way to work. We had hypothesized that employees experiencing the ROWE innovation would report less time pressure and work-family conflicts, and that was the case. But we were surprised that a corporate policy innovation such as ROWE could make a difference on such a range of outcomes.

We have not yet had a chance to think about and process unanticipated consequences. For example, Best Buy believes that having a “best friend” at work is good for employee productivity, and some leaders wondered if working at different times and places might lessen such social network ties. We also want to look at the outcomes for midlevel managers, who may feel they are losing control over their work because they are no longer in charge of scheduling their employees’ time and have to monitor results rather than attendance.

This hands-on research experience has led us to question the value of the cross-sectional research that we and other scholars typically engage in. Finding so many changes in outcomes and other descriptive variables over even a short six months challenges the scientific value of evidence showing an association between two or more variables at a single point in time. People live, work, and raise their families in a dynamic, ever-changing environment in which nothing is static—except perhaps the outmoded policies limiting their options.

Urie Bronfenbrenner was right: evaluating a real-world (as opposed to hypothetical) policy change as it is implemented is difficult, even messy, but remains essential for good science and theory development. Despite its departures from the ideal experimental design, we are grateful for the opportunity to investigate ROWE, since the payoffs of studying such an experiment of nature, a ready-made policy change in an actual corporate setting, far outweigh any costs or disadvantages. It is also fundamental for developing real-world solutions to the work, family, and health challenges confronting most of today’s workforce, solutions that are both feasible and effective.