Making or buying evidence: Using transaction cost economics to understand decision-making in public school districts

Abstract

We propose transaction cost economics theory as a tool for exploring when school administrators rely on information from two types of sources: internal sources like their own colleagues, and external sources like researchers and government agencies. The theory’s application is illustrated in a comparative case study of two public school districts in Michigan. Consistent with the theory’s predictions, the smaller, homogeneous, high-performing district used more external sources of information, while the larger, diverse, low-performing district used internal sources of information. We conclude by identifying some strengths and limitations of the theory, which can serve as starting points for debate.

There is significant pressure to use evidence-informed decision making (EIDM) and adopt evidence-based programming (EBP) in public schools in the United States and elsewhere (Sharples & Sheard, 2015), but many accounts suggest that schools continue to use programs with questionable effectiveness (Greenberg et al., 2003; Hallfors & Godette, 2002; Ringwalt et al., 2009). At the same time, questions remain about what counts as evidence, as well as where evidence does and should come from. Research in this area has identified a wide range of potential sources of research evidence, including colleagues within the school building, academic journals, websites, government agencies, and non-governmental advocacy organizations (e.g., Everton, Galton, & Pell, 2000; Honig & Coburn, 2008; Neal, Neal, Kornbluh, Mills, & Lawlor, 2015a; Williams & Coles, 2007). Education policymakers including administrators in public school districts are likely to rely on some sources more than others. Therefore understanding the factors that contribute to school district administrators’ selection of sources of evidence is critical to promoting the use of research evidence in decision-making.

In this paper, we explore the usefulness of transaction cost economics for understanding the factors that influence school administrators’ decision to seek information and evidence from two broad types of sources: external and internal. External sources include the range of possibilities noted above, and are typically the focus of research on evidence use. Internal sources include the administrator’s own colleagues within the school building and district, as for example when a school district collects its own data on students and subsequently develops its own program. Transaction cost economics (TCE) was developed to understand a related issue confronted by manufacturing businesses: when should a firm buy the parts it needs from others, and when should it make the parts it needs (Williamson, 1981). By viewing information and research evidence as a key “part” in the making decisions in schools, we explore the possibility of translating TCE into the school context, thereby proposing a new framework for making sense of evidence use in program and policy decisions.

We begin by briefly reviewing the literature on the use of research evidence in public schools, focusing on past attempts at identifying both the determinants of use and sources of evidence. We then introduce transaction cost economics in its original manufacturing context, trace its use in non-business contexts, and translate its basic hypotheses into the context of schools and research evidence. Using 19 semi-structured interviews from a comparative case study of two public school districts, we illustrate what an application of TCE might look like in the school context. Consistent with expectations derived from TCE, internal sources were more often used in the larger, more diverse, less stable district, while external sources were most often used in the smaller, less diverse, more stable district. We conclude by identifying some strengths and limitations of the theory for understanding schools, which can serve as starting points for debate.

Acquisition of Research Evidence in Public Schools

The focus of much recent work has been on schools’ and school administrators’ use of research evidence, but it is also important to consider a logical and temporal precursor to use: acquisition. That is, before an administrator can use research evidence, he or she must find it first. Indeed, Neal et al. (2015b) identify acquisition as a key link in closing the research-to-practice gap. This is no small task, in large part because administrators are busy and locating the right kind of information is time consuming. Moreover, there are many sources administrators might turn to for this information, including trusted colleagues, professional organizations, and websites (Honig & Coburn, 2008; Neal et al., 2015a, Williams & Cole, 2007). Despite the number of possible sources of information from which administrators might acquire information and research evidence, one useful classification distinguishes them as either internal or external sources.

Internal sources include individuals, subgroups, objects, and events within the administrator’s own district or building. Typical examples of internal sources of information might include instances of formal hierarchy, as when a superintendent (the final decision maker) acquires information from an associate superintendent of instruction charged with locating new programs. It can also include instances of more diffuse or non-directed advice giving or diffusion of information, as when a teacher acquires information from a packet produced by the district’s curriculum council, or by attending a district-sponsored professional development workshop. Reliance on internal sources offers a number of advantages. First, internal sources of information are already familiar with the subtle contextual factors that characterize specific school districts (e.g. unique student populations, budgetary or regulatory constraints), and can effectively filter out irrelevant information. For example, an associate superintendent will likely not pass along to a superintendent research evidence about a program that state laws would prohibit adopting. Second, precisely because they are internal, internal sources are more readily available and relationships between seeker and source are more likely to be characterized by trust. For example, a teacher is more likely to see and trust a fellow teacher’s advice than the advice of a stranger. However, reliance on internal sources can also come with some disadvantages. As Neal et al. (2015b) suggest, turning for information to others within one’s own context “may lead to echo chambers or closed loops that hinder” the acquisition of new information (p. 182). That is, internal sources are less likely to provide access to the most cutting-edge research evidence on new programs and practices.

External sources are a much broader category because they include the full range of possible sources of information outside the administrator’s own district or building. A significant body of research has recently emerged that attempts to identify and classify these sources, which are often referred to as “intermediaries” or “brokers” (Daly, Finnigan, Moolenar, & Che, 2014; Knight & Lyall, 2013; Neal et al., 2015a; Scott & Jabbar, 2014). In some cases, external sources might provide direct access to high-quality research evidence, for example when a school administrator turns to a research aggregator like the What Works Clearinghouse, or to a researcher through a research-practice partnership (Coburn, Penuel, & Geil, 2013; McEwen et al., 2008). In other cases, external sources like advocacy organizations might provide more partisan perspectives (e.g., Scott & Jabbar, 2014), and sources like friends and relatives might offer anecdotes that are not backed by research evidence. Thus, reliance on external sources may come with both advantages and disadvantages. On the one hand, they can reduce the echo chamber effect and increase the range and novelty of information and research evidence acquired by administrators, while on the other hand, they can sometimes provide information of dubious quality.

In this paper, we do not ask whether internal or external sources are better, but instead ask: why do some schools look for research evidence internally, while others look for it externally? Understanding why different schools adopt different information acquisition strategies can inform the development of targeted interventions designed to facilitate and promote the use of research evidence in decision-making.

What is Transaction Cost Economics?

To answer this question, we turn to an unlikely source: industrial economics. The theory of transaction cost economics (TCE) was initially proposed by Williamson (1979; 1981) to understand how organizations manage costs through boundary setting. The theory views organizations not primarily as systems for producing things, but rather as systems for coordinating interactions, or transactions. Thus, the basic unit of analysis for understanding how organizations work is the transaction, which broadly includes the transfer of any valuable resource used by the organization from one entity to another. The theory is often illustrated, for the sake of concreteness, in the context of a manufacturing firm, where the transactions are the exchanges of component parts (e.g. tires) during the assembly of some finished product (e.g. a car).

For organizations, transactions are inherently costly because they require negotiation and represent opportunities for things to go wrong, which in turn can impede the organization’s goals. For example, if the tires are not available when they are needed (i.e. the “tire transaction” breaks down), the organization’s goal (i.e. making a car) is hindered. Williamson (1981) argued that organizations seek to manage these transaction costs by drawing their boundaries strategically so that some transactions take place inside the organization, while other transactions take place outside the organization. In the car tire example, the issue of where to draw organizational boundaries involves asking: Should we make our own tires (internalize the transaction), or should we buy them from an outside supplier (externalize the transaction)?

The theory of transaction cost economics aims to identify the factors that influence this make-or-buy decision. Although the theory has a number of nuances, three factors are held to be particularly important: frequency, asset specificity, and uncertainty. Frequency refers to how often (or how much of) the resource is required by the organization. If the organization needs the resource only infrequently (e.g. a purple leather car seat), then it would be too costly to make it internally and is more sensible to buy it from an external supplier as needed. In contrast, if the organization needs the resource frequently (e.g. standard rubber tires), then it is too costly to constantly re-negotiate with external suppliers and more sensible to make it internally. In fact, this is why, in the 1920s, the Ford Motor Company operated its own rubber plantations in Brazil (Russell, 1942). Thus, according to TCE, frequently needed resources will be made, while infrequently needed resources will be bought.

Asset specificity refers to the uniqueness of the resource required by the organization. If the organization needs a resource that is also used by many other organizations for many different purposes (e.g. light bulbs), it is simply unnecessarily costly to make it internally, because the large market for the resource ensures it will be readily available from external suppliers. In contrast, if the resource is highly specific to the organization (e.g. a fuel cap that only fits the 2016 Porsche 716 Boxster), the organization will likely make the part itself because it is not available to buy from external suppliers, due to the lack of a market for the part. Thus, according to TCE, highly specific resources will be made, while non-specific generic resources will be bought.

Finally, uncertainty refers to the degree to which the organization’s environment is changing and unpredictable. If the organization is seeking to acquire key resources in an unstable or poorly regulated market (e.g. where potential suppliers have a track record of not delivering parts on time), it is too risky to rely on external suppliers who might fail to come through or who might engage in wrongdoing, and can shield itself from risk and unexpected costs by making the resources internally. In contrast, if the organization is seeking to acquire resources in a stable and predictable market (e.g. where potential suppliers have a long track record for reliability), it is able to carefully evaluate external suppliers without placing itself at any undue risk. Thus, according to TCE, resources will be made when the environment is uncertain, while they will be bought when the environment is certain.

Two final points about TCE are important, and as we explain below, are particularly relevant when translating the theory into a school context: perception and scope. First, in some cases, the dimensions identified by TCE can be measured objectively (e.g. whether a particular fuel cap really does fit). However, what really matters for reaching make-or-buy decisions is the decision maker’s perception of the dimension (e.g. whether the decision maker believes the cap will fit). Second, the theory is not intended to be a comprehensive theory of organizations, but rather has a much narrower scope: it is designed to help understand how an organization draws its boundaries, and more specifically, how it decides when to make or buy the key resources it requires. There are other important aspects of organizations that TCE does not attempt to explain. For example, while TCE may help explain why some car manufacturers make their own tires while others buy tires from an outside supplier, it is silent on whether the tires used by the manufacturer are of high or low quality, as well as on whether the manufacturer will actually use the tires once they have been made or bought.

Translating TCE to the School Context

The contexts within which TCE was initially developed might seem quite far afield from evidence use and decision-making in public schools. Indeed, Rowan and Miskal (1999) “did not find much organizational analysis of schooling using…transaction cost economics,” but they also argue that “this is unfortunate because [such theories] potentially can improve the efficiency and productivity of educational transactions” (p. 379). Since their review, Williamson (1999) sketched an extension of the theory designed to include public sector organizations, and the theory has been applied in a range of social science contexts (Macher & Richman, 2008), including some applications in the education context, primarily in a series of recent doctoral dissertations.

House (1996) used TCE as a framework for understanding when proposed school reforms, such as the adoption of a new curriculum, are likely to be successful. He illustrated the risk inherent in engaging with external influences by asking “Why should teachers risk knowledge assets built up over many years by switching to new teaching materials or techniques of unproved quality (p. 8)?” In cases where the future of teachers’ jobs are uncertain, he contended that because “Teachers know from experience that [outside] reformers cannot guarantee the promised outcomes of their reforms,” risk-averse teachers avoid externally-proposed reforms (p. 8). Eisenberg (2014) also focused on the uncertainty dimension, exploring how “a school district [decides] to outsource students with emotional disabilities to private therapeutic day schools” versus to provide such programming internally (p. 1). He found that “Uncertainty influenced district decisions for students with mental health needs…represented in the desire to develop internally controlled programs, and the preference to utilize limited, trusted partnerships with increased transactions” (p. 117). That is, as with House’s (1996) reform example, in the presence of uncertainty, schools avoid risk by internalizing activities.

While both House (1996) and Eisenberg (2014) examined the role of uncertainty in choosing between internal and external reforms and programs, other studies adopting the TCE lens do so in a more piecemeal fashion. For example, Maine’s (2005) comparison of scheduling in small and large schools, Kuncewicki’s (2014) case study of a school’s outside vendor selection, and Ivins’ (2014) case study of a contract between a charter school and special education provider, each refer to TCE, but do not examine make-or-buy decisions between internal and external options. In contrast, Panagos (2014) directly examines a school’s decision to “run an internally developed, rather than outsourced, fully online course” (p. 79). However, he ultimately explains the decision in terms of the course’s actual direct cost (i.e. how much would an outside vendor charge?), rather than in terms of transaction costs (i.e. what are the risks of dealing with an outside vendor?).

While the use of transaction cost economics as a theoretical framework for understanding educational contexts is still under development, these early studies highlight its potential. Our aim in this paper is to further translate TCE into the school context to help explain how schools and school administrators decide when to “make” their evidence by relying on internal sources for information, and when to “buy” their evidence by relying on external sources for information. To avoid “the piecemeal…use of TCE reasoning” typical of adaptations of the theory (Macher & Richman, 2008, p. 1), in our theoretical translation, we try to adhere closely to the original theory by conceptualizing the school (or district) as akin to a manufacturing firm with the goal of “producing” well-socialized and well-educated young people. To achieve this goal, schools are systems that coordinate the interactions of, or transactions between, many different entities.

In the broadest sense, TCE specifies how organizations draw their boundaries, deciding which activities are “inside” the organization and which activities are “outside.” Figure 1 illustrates how such decisions might define different boundaries for two hypothetical schools. Each rectangle represents an activity that a school might engage in, while the two bolded rectangles identify the activities in which nearly all schools must engage. The organizational boundaries of school A and school B both include “Deliver instruction” and “Provide healthy food,” but otherwise the two schools’ organization boundaries differ. Characteristic of the type of community or full-service schools described by Dryfoos (1995), in addition to these basic activities, school A also provides programs for community members and grows its own vegetables in a community garden. School A could leave such activities up to the local YMCA and grocery store, but instead has chosen to internalize them. In contrast, School B does not provide community programming or grow food, but it does conduct its own research and develop its own programming. School B could leave such activities up to traditional academic researchers and others, but instead has chosen to internalize them. While TCE could, in principle, be helpful for understanding why schools internalize or externalize many different kinds of activities, our focus is narrower: why do some schools “make” their own research internally and draw their organizational boundaries like School B, while other schools “buy” their research evidence from external sources and draw their organization boundaries like school A?

Figure 1.

Boundaries of two hypothetical school organizations

Using TCE as a framework for answering this question requires identifying an analogue for each of its three dimensions – frequency, asset specificity, and uncertainty – in the context of decision-making about programs and practices in schools (see Table 1). In traditional formulations of TCE, frequency refers to how often a key resource is needed by an organization. In the context of school decision-making, this is analogous to how often the school must make evidence-based decisions about adopting new programs. Because the key resource for making evidence-based decisions is evidence, the more often schools make such decisions, the more frequently they require this resource. Viewed in this way, TCE’s hypothesis concerning the role of frequency in make-or-buy decisions can be translated directly: Schools that must explore new program options often (e.g. large schools, schools with high student mobility) are expected to rely on internal sources, which reduces the cost of searching for this information and the risk of not finding it when needed. Conversely, schools that only consider adopting new programs occasionally (e.g. smaller schools, schools with low mobility) are expected to rely on external sources, because the cost of searching and risk of a failed search are minimal compared to the costs of conducting research internally.

Table 1.

Dimensions of transaction cost economics in schools

Dimension	In the school context	Hypothesis
Frequency	How often the school must make evidence-based decisions about adopting new programs.	Schools that must explore new program options often (e.g. large schools, schools with high student mobility) are expected to rely on internal sources
Asset Specificity	The (perceived) uniqueness of the school’s context.	Schools with contexts that are perceived to be unique (e.g. non-white, non-middle class student populations) will rely on internal sources
Uncertainty	The lack of clarity about the school’s future.	Schools facing threats of change (e.g. low performing schools) will rely on internal sources

Turning to asset specificity, this typically refers to the uniqueness of the resource required by the organization. When school-based programs are developed, they are developed for and tested on specific student populations. But, when schools make decisions about whether to adopt a new program, they require evidence not simply that it will work there, but that it will work here (Cartwright, 2013). If school officials perceive “here” to look about the same as “there,” they are able to trust in the generalizability of past research and to inform their decisions using non-specific evidence. However, if school officials perceive “here” to be quite unique, a decision about program adoption will require more context specific evidence. Thus, in the school decision-making context, asset specificity refers to the degree to which the school’s context is unique, and thus evidence must be context specific to be useful for informing decisions. Again, TCE’s hypothesis concerning the role of asset specificity in make-or-buy decisions can be translated directly: Schools with contexts that are perceived to be unique (e.g. non-white, non-middle class student populations) will rely on internal sources, which can be more closely tailored to a school’s own context. Conversely, schools with contexts that are perceived to be similar to those where programs are often developed and tested (e.g., white, middle class student populations) will rely on external sources, because this outside evidence generalizes to and is relevant for the school’s context.

Finally, uncertainty refers to the degree to which the organization’s environment is changing and unpredictable. This dimension of TCE has the most direct analogue because schools confront uncertainty in many different ways, which frequently manifest in a lack of clarity about the school’s future. Some key sources of uncertainty for schools include the threat of closure due to low performance or enrollment, budget reductions, staff turnover, and building reorganization. Once again, TCE’s hypothesis concerning the role of uncertainty in make-or-buy decisions can be translated directly: Schools facing threats of change (e.g. low performing schools) will rely on internal sources, which shield itself from an external environment perceived to be hostile. Conversely, schools that do not anticipate disruptive changes or that are able to anticipate future change (e.g. high performing schools) will rely on external sources, because they have the flexibility to tolerate the potential risk of engaging with the external environment.

As Table 1 summarizes, the dimensions of frequency, asset specificity, and uncertainty each shape considerations about whether a school should “make” its own evidence and place these activities inside its organizational boundary, or should “buy” evidence from others like academic researchers and leave these activities outside its organizational boundary. Translating these dimensions into the school decision-making context, but relying on TCE’s core hypotheses about their role, the theory suggests that increases in any of these dimensions increase the likelihood that a school will rely in internal sources of information when exploring and making decisions about new programs. However, because the scope of TCE is restricted to explaining organizations’ make-or-buy decisions, it does not make any predictions about whether the school’s source of information will be of high or low quality, nor about whether the school will use the information obtained.

A Tale of Two Districts

Because this paper aims to introduce and explore the potential utility of TCE for understanding decision making in schools, it would be premature to attempt a formal test of the theory’s predictions. However, in this section we offer a comparative case study of two school districts as an illustration of what TCE might look like in practice.

Sample

The two Michigan school districts – City and Village¹ – were selected using what Seawright and Gerring (2008) call the “most different” technique, which aims to select two cases that maximize variation on the dimensions of interest and minimize variation on other dimensions. Table 2 reports each district’s level on several indicators of the three TCE dimensions, along with the state mean. First, total enrollment and student mobility are indicators of frequency because larger districts and districts with high student turnover must re-evaluate the program decisions more often. As the table illustrates, Village district is smaller and has less turnover than the state mean, placing it lower on the frequency dimension, while City district is larger and has more turnover, placing it higher on the frequency dimension. Second, the percent of the district’s population that is non-white and that is economically disadvantaged are indicators of asset specificity because relatively few school-based programs are specifically developed for or tested on these underrepresented populations. Village district serves fewer students from these groups than the state mean, placing it lower on the asset specificity dimension, while City district serves more students from these groups, placing it higher on the asset specificity dimension. Finally, the percent of students proficient in math and reading are indicators of uncertainty because these values often drive decisions to close school buildings and replace staff. More students in Village district are proficient in math and reading than the state mean, placing it lower on the uncertainty dimension, while fewer students are math or reading proficient in City district, placing it higher on the uncertainty dimension. Taken together, Village district is low on all three TCE dimensions and City district is high on all three dimensions, compared to each other and compared to the state mean.

Table 2.

TCE Dimensions & Indicators in Illustrative Case Sites

TCE Dimensions/Indicators	State Mean	Village	City
Frequency		Low	High
Enrollment	2509	Less than 2000	More than 10,000
Student Mobility	5.46%	1%	7%
Asset Specificity		Low	High
Percent Non-white	19.38%	10%	73%
Percent Low Income	49.01%	19%	60%
Uncertainty		Low	High
Percent Math Proficient	33.61%	55%	13%
Percent Reading Proficient	46.69%	63%	22%
Similarities –		Same State & County Governance
		Both governed by elected board
		Same per-pupil funding
		Both serve Pre-K-12 within a geographic enrollment zone w/vouchers

Note: Values for Village and City districts have been rounded to the nearest percentage point to preserve confidentiality.

In addition to maximizing variation on the three dimensions of interest, focusing on these two districts also minimizes variation on (i.e. holds constant) other factors that could otherwise confound our exploration. Both districts are subject to the same educational governance structures because they are located in the same state and county, and are overseen by elected school boards. By state law, both districts receive the same per-pupil budget allocation. Finally, both provide the full range of public educational services from pre-kindergarten through 12^th grade within a geographically defined enrollment zone, but also allow students from outside the zone to enroll through the Michigan “School of Choice” program.

Data and coding

In Spring 2013 we interviewed 19 school staff in these two districts about their roles in identifying and selecting school-based programs. The sample was developed using a snowball referral method, and included each district’s superintendent, as well as additional central office staff, principals, and teachers. Each of the 19 semi-structured interviews was conducted in person, and lasted about an hour. The interview began with a “grand tour” question – Can you tell me about a program you or your school district recently considered? – and proceeded through a series of follow-up probes concerning the respondent’s role in and reflections on the process (Bernard 2011; Patton 2002). To avoid bias, the interviewer used generic words like “information” and avoided words like “research” and “evidence” until they were used by the interviewee. Each interview was recorded and transcribed for subsequent qualitative coding.

To analyze the interview transcripts, we used directed content analysis, which is designed for research that seeks to apply an existing theory from which codes can be defined a priori (Hsieh & Shannon, 2005). In the present analysis, we sought to determine whether districts tend to rely on external or internal sources of information for making decisions about programs. Directed content analysis is well suited to this goal because the relevant codes – information seeking, internal source, and external source – are pre-defined by our research question. Using this approach, the first and second authors independently coded all interview transcripts, identifying instances where respondents described seeking information to help inform the selection or adoption of a school-based program. For each identified instance, the two coders also coded whether the respondent’s source of information was internal (i.e. a person/group in the same district) or external (i.e. a person/group outside the district). The two coders met to discuss and reach consensus on any coding discrepancies. This coding process yielded 210 distinct instances of information seeking across the two districts.

Illustrative findings

Respondents in Village district mentioned searching for information from an external source 79 (62.6%) times, and from an internal source only 47 (37.3%) times. In contrast, respondents in City district mentioned searching for information from an external source 42 (50%) times and from an internal source 42 (50%) times. These findings suggest that both districts use a combination of internal and external sources of information to inform their decision-making about program adoption. However, consistent with the expectations from TCE outlined above and in Table 1, Village district relied more heavily on external sources than City district. That is, in the language of TCE, because Village district must make program adoption decisions infrequently, to serve a typical student population, in a fairly certain environment, it is able to “buy” information and research evidence from external sources. In contrast, because City district must make program adoption decisions frequently, to serve a specific student population, in a highly uncertain environment, it chooses to “make” its own information internally more often than Village district.

What does frequency look like?

In larger districts like City that are faced with the constant need to review programming, multi-level bureaucracies often develop to manage the process. As a central office administrator in City explained:

“I don’t make those decisions. I’m part of the decision-making process, but it certainly doesn’t start and/or end with me. It has to go through the instructional council, has to go then back to the superintendent, has to go to the executive team, back to the instructional council. The superintendent has the final yay or nay.”

As her description of City’s decision-making process illustrates, the bureaucratic structure means that the information used to make decisions is constrained to the internal sources that lie along the chain of command. That is, the frequency of decision-making and need for information in City require a heavier reliance on internal sources, for the sake of expediency and bureaucratic control. In contrast, smaller districts like Village, which are not faced with the need to coordinate frequent decision-making, can manage with smaller bureaucracies that afford greater flexibility. The Village superintendent noted this flexibility in the district’s decision to outsource financial services and technology support via a contract with another local district, but even research evidence used to inform programming is heavily outsourced. For example, one Village elementary principal explained that

“[the county] spent several years researching [Reading Street], and then pushed that down to all the districts in the county. I mean, we’re kinda like a medium sized district, but we just don’t have the administration support to do a lot of that research. The county’s really taken on the role of…they really set the tone in a lot of cases and provide the support and the expertise when were looking at things like curriculum, and research, and data.”

Here, the county-level intermediate school district, which maintains a large staff of consultants and facilitators, serves as an important source of information guiding decision-making in Village, but it is a source of information that lies entirely outside Village district itself. These two quotes suggest that in school contexts, TCE’s frequency dimension may be closely linked to school size, bureaucratic structure, and internal capacity.

What does asset specificity look like?

When a school perceives its student population to be different from the populations for which most school programs are developed and tested, there is an increased need to tailor programming solutions, which requires a reliance on internal sources of information and evidence. Administrators in City district often described their student population as unique, for example, when the director of high school education explained that “you can’t compare City district to [other local district] who have a two family well-over $100,000 each parent makes.” This perception of uniqueness had led them to undertake a large-scale internal data collection effort, using a tool called Snapshot. While this project was costly, a City central office administrator offered the following rationale: “Ya know, this is you. Not your neighbor, not somebody from California where you have to adapt it. This is you, what are we going to do. This is your building.” The Snapshot data was compelling because it was narrowly tailored to City’s own student population, and presumably would allow the district to develop its own programming to meet the specific needs of its students. Indeed the perceived value of internal data collection in City district was so compelling that one City central office administrator noted “[there is] not a lot of research behind [using Snapshot data], but it was one of those, oh my god, we have to do this. Why didn’t we think of this before kind of moment.”

Interestingly, although Village district serves a much more homogeneous, predominantly white middle-class student population, administrators there also perceived their district to be unique. A member of the school staff at the Village middle school observed that

“We don’t have hardly any racial diversity, it’s pretty much a white town. When you look at a program that would foster diversity…you know it would be easily dismissed because people sometimes just say, ‘well, we don’t really have that kind of group here’.”

Even in a district where the student population lacks diversity, administrators still perceive that off-the-shelf programs developed elsewhere cannot easily be adopted in their own districts because they are not sufficiently tailored to the district’s specific student population. If all administrators perceive their districts to be unique, this raises questions about whether asset specificity actually plays a role in choosing internal or external sources of information in the way that TCE hypothesizes.

What does uncertainty look like?

All public schools and school districts experience some uncertainty; it is simply the nature of the field. However, how schools confront and respond to uncertainty can vary. At the time of our interviews, both City and Village were exploring the adoption of an evidence-based reading curriculum called Read 180, but they differed in how they approached the decision. A central office administrator in City noted that

“Read 180 is a tremendous computer reading program, but it’s about $150,000 a year. Ya know, with the cuts we have, and out budgeting issues, I don’t care what you give me unless it’s free.”

City district had recently experienced some severe and continuing budget cuts due to declining enrollments and underperforming schools, casting a great deal of uncertainty on its future. Although program cost is always a barrier to implementation, this administrator’s comment viewed in light of the district’s situation, suggests the concern goes beyond simply the direct cost of the Read 180 program itself. Not only would adoption of Read 180 expose the district to potential budget shortfalls, but as outside trainers are brought in during program implementation, it would also expose the already struggling and uncertain district to greater external scrutiny. As the high school principal explained, the consideration of Read 180 in Village looked different:

“We also had a change in teachers and she was open, the new teacher was open to that change. Read 180 requires a little bit of training through the company…you don’t just sit down, it’s a cultural change.”

Village had also experienced some uncertainty, in this case, driven by staff turnover. However, despite the risks that accompany uncertainty, staff in the district were much more open to change, including large-scale cultural change facilitated by an outside company. Because Village already had a strong track record of student performance, there was little risk inviting outsiders like Read 180 trainers into the district. Thus, although all schools are likely to encounter some level of uncertainty in their environments, the magnitude of the uncertainty (e.g. long-term budget issues vs. a recent front-line personnel change) may shape the perceived risk in engaging the external environment.

Limitations of this illustration

As a small scale comparative case study designed to illustrate but not test the theory, a number of limitations are important to note. First, the case selection aimed to maximize variation along all three dimensions of TCE simultaneously, which makes it impossible to determine the independent effect of each dimension. A more complete test of the theory would require a larger sample in which districts varied on all dimensions. Second, our interview questions asked narrowly about identifying and selecting programs in the areas of instruction, health, and social skills. This is useful because each respondent was describing their strategies for seeking information around similar kinds of topics, but it does not allow us to explore whether TCE helps understand decisions to make or buy evidence to inform smaller (e.g. what to serve for lunch) or larger (e.g. should we open a new school) decisions. Finally, as an illustration informed by two cases, these data do not allow us to draw conclusions about the theory’s ability to make accurate projections about the future (i.e. its predictive validity) or its ability to distinguish districts that use internal sources from those that use external sources (i.e. its discriminant validity).

The Utility of TCE: An Invitation to Debate

In this paper, we have aimed to explore the potential utility of transaction cost economics, a theory initially developed to understand manufacturing and other production-oriented firms, for understanding decision-making in public schools. We conclude by identifying some strengths and limitations of TCE for understanding schools, which can serve as starting points for debate.

When school administrators make decisions, they have a choice to inform their decisions using internal or external sources of information (or both). The primary strength of the theory of transaction cost economics is that it offers a way to unpack this formerly black-box process by identifying some of the factors that influence turning to internal versus external sources. It is, perhaps, not the only way, but the factors of frequency, asset specificity, and uncertainty at least give us a starting point. This strength is reinforced by the fact that the three factors identified by TCE have relatively clear analogues in the school context (see Tables 1 & 2). Although our comparative case study does not test or prove the validity of TCE, the fact that it yielded findings consistent with expectations based on TCE at least suggest that the theory is worth considering further. A final strength is that the theory, if it continues to hold up under further scrutiny, points to some practical implications. By understanding where schools look for information, and why they turn to the sources they do, interventions and policies designed to facilitate or promote the use of research evidence can be more targeted. For example, promoting the use of research evidence in districts like City that are high on the frequency, asset specificity, and/or uncertainty dimensions, may be quite challenging because administrators are likely exposed to information that simply recirculates among colleagues within the school, in a kind of echo chamber (Neal et al., 2015b). In such cases, efforts would need to be targeted on finding ways for useful and novel information to penetrate the district’s boundaries. This might include reducing the (transaction) costs of searching for external information by making this type of information more readily available, easier to use, and more compatible with the district’s needs.

In the face of these potential strengths, TCE also has a number of limitations. We have already noted above the limitations associated with our small-scale comparative case illustration, and here focus on the more conceptual limitations of the theory for understanding schools. First, as we have already observed from talking to administrators in two districts, it is common for districts to rely on both internal and external sources of information. Although TCE does not view the make-or-buy decision as a strict dichotomy, it is unclear how different combinations of the three dimensions might translate into different mixes of internal and external information. For example, how might the information used to make decisions in a district high on two dimensions look different from the information used to make decisions in a district high on only one dimension? Additionally, a single district focused on a single decision may shift from using internal to external information over time. For example, a decision about adopting a new program may first be informed by internal information about teachers’ qualitative experiences using the current program, but later be informed by external information about the demonstrated effectiveness of the new program. To be useful in a school context, we believe TCE must have ways to incorporate and explain the simultaneous use of internal and external information, as well as over-time shifts from one source to another.

Second, our illustration focused on how district-level decisions about program selection are informed by different sources of information, and thus provides insight about TCE when applied at one hierarchical level (i.e. the district) and one decisional scale (i.e. program selection). However, in schools, decisions get made at different levels, ranging from decisions made by teachers about their classrooms, to principals about their buildings, to superintendents about their districts. Likewise, decision can vary in their scale, ranging from minor day-to-day decisions like what to serve for lunch, to major strategic decisions like whether to open a new school building. Of course, the level where the decision is made and the scale of the decision often coincide, but it remains unclear how well TCE helps us understand decisions at different levels and scales. For example, is the theory equally useful for understanding district-level strategic decisions and school-level operational decisions? To the extent that decisions at different levels and scales also vary on the dimensions identified by TCE (e.g. routine operational decisions are made more frequently and at lower levels than strategic decisions), we expect that the theory can be useful at multiple scales and is worth further investigation.

Finally, like all theories, TCE is an abstraction that seeks a parsimonious explanation, but as a result is necessarily incomplete. We see two important ways that it is incomplete and might be expanded or refined. It is incomplete because it identifies only three factors that influence whether schools will rely on internal or external sources of information, while in reality many other factors likely also play a role (c.f. Zardo, Collie, & Livingstone, in press). In our illustration, we focused on cases with similar economic and political resources, but these can vary significantly between districts and may also shape make-or-buy decisions. For example, a district like Village that is expected to rely on external sources of information may still rely on internal sources if administrators know there is no budget for, or no political support for, programs developed outside the district. Thus, in addition to finding ways to incorporate these factors alongside the three identified by TCE, future applications of the theory to the school context will also need to consider the relative role that each factor plays. For example, do political factors outweigh frequency or asset specificity when it comes to selecting information sources?

The theory is also incomplete because it focuses on just two sources of information or other resources: a school can “make” and rely on internal sources, it can “buy” and rely on external sources, or it can use a combination of the two. But, there is a third kind of source omitted by TCE but that we see in schools: schools can “collaborate” and work together with outsiders to jointly produce information to inform decisions. This practice might appear in the form of research-practice partnerships (RPPs; Dagenais et al., 2012; Coburn, Penuel, & Geil, 2013; Mady, 2013; McEwen et al., 2008), or in what Galvin and Fauske (2000) describe as “intermediary hierarchies.” Adaptations of TCE for the school context will need to find ways to incorporate these types of collaborative information-producing arrangements. Thankfully, Powell (1990) has already sketched one possibility for such an extension. He noted that when organizations “make” their key resources internally, they are organized as hierarchies, while when organizations “buy” their key resources externally, they are organized around markets. But, when organizations form long-standing relationships to collaboratively produce key resources with an outside party, they are organized around networks. Thus, future applications of TCE to schools may benefit from exploring the expanded make-buy-or-collaborate choice in structures of hierarchy, markets, and networks.

The theory of transaction cost economics offers a new lens for understanding the factors that shape decisions in schools. The theory is supported by an extensive literature in other fields, and our comparative case study illustration in two districts suggests that it can be adapted to the school context. However, there is also much room for debate and refinement, and the limitations and illustration we present here offer a few places to get started.