Contracting and Procurement for Evidence-Based Interventions in Public-Sector Human Services: A Case Study

Abstract

Sustainment of evidence-based interventions (EBIs) in human services depends on the inner context of community-based organizations (CBOs) that provide services and the outer context of their broader environment. Increasingly, public officials are experimenting with contracting models from for-profit industries to procure human services. In this case study, we conducted qualitative interviews with key government and CBO stakeholders to examine implementation of the Best Value-Performance Information Procurement System to contract for EBIs in a child welfare system. Findings suggest that stakeholder relationships may be compromised when procurement disregards local knowledge, communication, collaboration, and other factors supporting EBIs and public health initiatives.

Sustainment of evidence-based interventions (EBIs) in public-sector human service systems depends on several outer-context (e.g., policies, funding, contracting) and inner-context (e.g., organizational characteristics, workforce, fiscal viability) factors (Aarons, Hurlburt, & Horwitz, 2011). In particular, government policies and contracting procedures can pivotally shape local implementation contexts and support or jeopardize the institutionalization of EBIs within these systems (Willging, Green, Gunderson, Chaffin, & Aarons, 2015). Throughout the nation, public administrators have sought to streamline processes for procurement by introducing performance-based contracting to decrease micromanagement, support devolution of “risk” from government entities to community-based organizations (CBOs), reduce costs for purchasers, and usher in service innovations based on local needs (Collins-Camargo, McBeath, & Ensign, 2011; Lee, Allen, & Metz, 2006). Performance-based contracts describe contracts in which provider agencies are only paid when they meet certain performance targets and are also limited to a specific amount of compensation regardless of the quantity of resources that they must expend to deliver agreed upon services (Smith, 2010). Evolving models for performance-based contracting rooted in for-profit industries are also gaining traction in human service systems (Kashiwagi, 2011; Kashiwagi, 2013a), such as child welfare.

Child welfare systems operate under mandates to facilitate delivery of services that enable parents to maintain safe home environments, prevent child maltreatment, and reduce expensive out-of-home placements. Yet, researchers note that child welfare systems commonly support implementation of parent-focused interventions that lack an evidence base, despite the existence of efficacious programs backed by solid empirical support that aim to change the family environment and improve the lives of children (Horwitz, Chamberlain, Landsverk, & Mullican, 2010). Given lags in the adoption of EBIs within these systems, little is known about factors that might also affect their sustainment (Aarons et al., 2014). In this qualitative case study, we draw from semi-structured interviews with key stakeholders from government agencies and CBOs to examine one model of performance-based contracting for sustainment of a home visitation EBI already instantiated within a large child welfare system. This case study elucidates both the possibilities and the perils of introducing contracting strategies created in for-profit industries within public human service systems, focusing specifically on the potential effects on collaborative relationships between government administrators and private vendors that have previously facilitated EBI sustainment.

Contracting with Nonprofit Organizations in Child Welfare Systems

Long before state legislation established public service systems responsible for child welfare, private non-profit organizations provided direct services to reduce and prevent child maltreatment and neglect (Embry, Buddenhagen, & Bolles, 2000; Smith, 2010). Since the 1960s, government contracting was the main source of financing for CBOs delivering human services (Van Slyke, 2003). Longstanding relationships between government and CBOs that were both informal and predictable undergirded contracting relationships (Johnston & Romzek, 2008; Smith, 2010). Since the 1980s, however, contracting has been shaped by the “marketization” of procurement processes, emphasizing competition and growth of for-profit options, movement away from cost-reimbursement models, the assumption of financial risk (e.g., responsibility for cost overruns) by providers, and performance outcomes (Smith, 2010; Smith, 1996). Reflecting current pressures to increase efficiency and accountability in all government processes (Eggers & O’Leary, 1995; Hood, 2002), these efforts to “modernize” the public sector are often intended to overturn relationship-based models of administering government contracts.

For three decades, states and counties nationwide have adopted market-oriented strategies, including privatization via performance-based contracting, to manage government-funded child welfare services (Collins-Camargo et al., 2011; Martin, 2003; Rosenthal, 2000). Privatization in the child welfare arena has paralleled the corporatization of services in the fields of health, mental health, job training, and education (Gronbjerg, 2001). Through performance-based contracting, buyers focus proposals for services on key objectives rather than on detailed scopes of work. Ideally, contractors are afforded greater freedom in how objectives are met but with payment provided only for services that meet prescribed performance levels (Office of Management and Budget, 1998). In contrast, traditional cost reimbursement contracts that previously dominated the child welfare sector generally reimburse nonprofit agencies for allowable service expenditures, with all risks and responsibilities for meeting child welfare goals retained by government agencies (McCullough & Lee, 2007). Such contracts center on service type and quality. Performance-based contracts, however, are much less concerned with the nuts and bolts of service provision, instead placing far greater emphasis on achieving pre-defined results and shifting fiscal risk to private providers (Gronbjerg, 2001; Wulczyn, Barth, Yuan, Harden, & Landsverk, 2005). The assumption of risk by private agencies, coupled with a focus on outcome objectives rather than process objectives, requires public agencies to relinquish a certain level of control with respect to decision making and implementation activities under performance-based contracts entered into with private providers (Collins-Camargo et al., 2011).

Performance-based contracting is intended to spur competition among private suppliers of public services to lower costs, foster innovation, and improve quality (Smith & Smyth, 1996). Smith and Smyth (1996, pp. 279–280) astutely observe that “the success of contracting as a policy strategy rests largely on the ability of government to select the right suppliers and sustain a well-functioning market in social and health services.” However, reliance on competition to procure quality services at reduced costs tends to be problematic in markets where the pool of potential suppliers (e.g., providers) may be limited. This is especially the case for rural areas that often lack any sizeable provider with required service capacity. Although proponents of performance-based contracting contend that competition may lead to innovation and higher quality services within the marketplace, it is difficult for government administrators to create stable provider networks locally able to deliver EBIs without first expending substantial public resources (Johnston & Romzek, 2008). Government administrators operating within such circumstances often find themselves continuing to rely on trust and prior working relationships with private CBOs to guide their contracting decisions (Johnston & Romzek, 2008; Smith & Smyth, 1996; Van Slyke, 2007). In many places, competition is only possible by including large non-profit and for-profit providers from outside the area at the expense of local CBOs (Smith & Smyth, 1996), potentially imperiling trust building between government administrators and CBOs and creating instability within the extant marketplace (Johnston & Romzek, 2008).

Best Value-Performance Information Procurement System

As social welfare reform efforts align with market-oriented practices and rhetoric, business strategies created in for-profit industries are emerging as appealing options. We utilize a case study approach to examine the implementation and short-term consequences associated with the use of one such strategy, the Best Value-Performance Information Procurement System or BV-PIPS (Kashiwagi, 2011). We focus on the application of the BV-PIPS to contract for home-based services within a child welfare system with a long history of relying on three local CBOs to deliver a specific home visitation EBI to reduce child maltreatment and neglect.

The BV-PIPS is a performance-based contracting system originally developed to streamline and standardize contract bidding and execution in the construction and engineering industries. Its developers define it as a universally applicable procurement paradigm (Kashiwagi, 2011; Kashiwagi, 2013b; Kashiwagi, Kashiwagi, Smithwick, & Kashiwagi, 2012). Since 1993, the BV-PIPS has reportedly been used to contract for over 1600 projects and services both nationally and internationally, totaling $5.7 billion in funding. These goods and services range from information technology to furniture to storeroom management to table top water systems to hazardous waste removal and, recently, to human services (Kashiwagi 2011, 2013a).

The public administration literature has emphasized the importance of collaborative relationships in contracting for human services (Dawes & Eglene, 2004; Horwath & Morrison, 2007; Johnston & Romzek, 2008). The developers of the BV-PIPS explicitly contrast their approach to procurement to “relationship-based” contracting. They argue that “trust” and relationships in contracting create non-transparency and function as “tools of the blind” (Kashiwagi, 2013b, p. 84). The BV-PIPS was designed to replace these collaborative relationships with “transparency, dominant metrics, and ‘seeing into the future’” (Kashiwagi, 2013b, p. 84). As a contracting system, the BV-PIPS is also characterized by a number of key principles that commonly characterize performance-based contracting approaches, including the transfer of risk, responsibility, and expertise away from the buyer or client (e.g., government funder) and onto the vendor or contractor (e.g., CBOs), and the minimization of buyer oversight. Procurement follows a blind bidding and review process whereby vendors use non-technical language in proposals and rely on “performance metrics” (e.g., number and cost of projects, deviations from cost or schedule, and customer satisfaction) to demonstrate superior value. Crucially, the developers of the BV-PIPS contend that buyers or clients should not be experts in the services to be procured and that superior vendors should similarly be distinguished by their project and risk management skills, rather than technical expertise. Nor should buyers “manage, control, or direct” vendors with a detailed or restrictive contract; instead, “best value” vendors define and direct their own scopes of work (Kashiwagi et al., 2012, p. 1077).

The BV-PIPS reportedly establishes the “best value” vendor (defined as the “best value at lowest cost;” Kashiwagi et al., 2012, p. 1078). The rationality of this process depends on what its developers call the “dominance” of best value, which is described in terms of being simple to understand and transparent, negating the need for subjective standards or decision making (Kashiwagi, 2013b), and allowing “everyone to predict the future outcome” (Kashiwagi, 2011, p. 34). Although the dominance of best value may be appropriate for procurement of basic items (e.g., light bulbs), the implementation and sustainment of evidence-based health and allied health interventions so that they can be delivered with fidelity entails considerable complexity (Aarons et al., 2011). Complexity arises from the type of intervention to be delivered, intensity of labor required to deliver the intervention (which can undermine productivity gains and cost reduction), and undercapitalization and insufficient infrastructure among possible vendors (Smith, 2010). It is unclear whether the BV-PIPS can sufficiently address this complexity.

The BV-PIPS is already used in several states, including Alaska, Arizona, Georgia, Idaho, Oregon, and Oklahoma. Yet, to our knowledge, BV-PIPS outcomes derive from evaluation designs that are best described as opaque, and the system itself has yet to be subjected to external assessment. Of note, the BV-PIPS developers appear to eschew independent evaluation, asserting that “the peer review for validity of [procurement] research comes from dominant test results (no vendor caused deviations and client satisfaction) and continuing industry demand … instead of subjective peer review of other academic researchers” (Kashiwagi, 2011, p. 5). In their own published evaluations of the BV-PIPS, the systematic presentation of methodology and data on outcomes is lacking. The developers do not explain how they arrive at metrics (or outcomes) that they identify as crucial to assessing success of the BV-PIPS. These include client satisfaction, vendor-caused deviations, waste, and performance. Despite acknowledging “special considerations” for human services, including the complexity and political sensitivity of the context, the BV-PIPS developers provide no evidence of strengths to support its implementation within such circumstances (Kashiwagi, 2014)

The application of the BV-PIPS within a child welfare system provides a unique opportunity to examine the implications of procurement ideologies and strategies that have evolved within for-profit industries and are now being applied to performance-based contracting within the realm of human services for EBIs. Contracting processes epitomize interactions between outer- and inner-contextual factors that influence uptake of EBIs in public systems in order to improve their performance. Although a growing literature on EBI implementation exists (Aarons, Fettes, Sommerfeld, & Palinkas, 2012; Aarons et al., 2011; Bruns et al., 2008), the study of EBI sustainment within these systems is still in its nascent stages (Aarons et al., 2014). This case study is relevant to the study of sustainment precisely because the EBI considered here was already fully instantiated within the service system and had been for more than ten years.

Study Context

This case study derives from longitudinal research on factors affecting sustainment of an established home visitation EBI to reduce child maltreatment across 11 child welfare systems in two states. This manualized EBI is used in multiple child welfare service systems to reduce child maltreatment through home-based behavioral skills training and education for parents.

In 2013, one of the 11 service systems (henceforth referred to as the “study system”) experienced a major departure from the traditional contract procurement process on which the majority of the sites relied. In this site, the EBI had been the preferred model of providing in-home services to families at-risk for child maltreatment and neglect since 2002. Every five years until 2013, the child welfare agency overseeing this system released an elaborate Request for Proposals (RFP) specifying requirements for applying the EBI and maintaining the infrastructure and professional expertise necessary to do so. Local CBOs typically responded to this RFP. This approach was consistent with theories of EBI implementation and sustainment that support the congruence and collaboration of the service system with community stakeholders, including CBOs and academic partners (Aarons 2011, 2014; Damschroder et al., 2009; Greenhalgh et al., 2004).

Traditional procurement in the study system mirrored the dominant processes utilized in government contracting with private nonprofit CBOs nationwide since the 1960s (Smith & Lipsky, 1993; Smith, 2010). The RFP favored local nonprofit CBOs with adequate workforce and capabilities required to deliver the EBI in both urban and rural areas on a large scale. Due to the small number of adequately staffed and equipped CBOs locally, the CBOs that were awarded contracts could usually count on retaining their contracts during periodic rebidding as long as no serious problems occurred. As elsewhere in the nation, the CBOs had come to rely on government contracts for most or all of their funding. Having worked together closely for over ten years to implement the EBI, the local government and contracted CBOs had come to know and trust one another. Contracting relationships before the transition to the BV-PIPS were thus predictable, “relational,” and “long term,” necessitating “cooperation” between government administrators and the CBOs (Johnston & Romzek, 2008; Smith, 2010, p. 149).

A change in top executive leadership within the study system led to a change in the traditional procurement process. The BV-PIPS was first introduced to bid for foster care contracts, and then to purchase home-based services. These changes were also accompanied by a shift in contract type from the traditional cost reimbursement structure, in which CBOs were paid for expenses incurred, to a daily rate payment model of a set amount per family per day. The new contracts also introduced rigorous monthly reporting requirements. Under previous contracts, reimbursement depended on the satisfactory completion of required paperwork, i.e., documentation of services rendered. However, contractors had been required to take part in university-led efforts to evaluate implementation of the EBI within the study system, so the previous contracts were not without a monitoring component. The new leadership generally lacked familiarity with the extensive history of the home visitation EBI and prior efforts to build and sustain its implementation infrastructure within the study system.

Inner and Outer Contexts

The analytic constructs of inner and outer contexts illuminate the range of factors that arise from the shift to performance-based contracting using the BV-PIPS and potentially affect sustainment of the home visitation EBI in the study system. Inner-context factors are associated with organizations, such as CBOs and their staff, whereas outer-context factors refer to the system level of the broader environment in which these organizations operate (Aarons et al., 2011; Damschroder et al., 2009). Inner-context factors related to EBI implementation and sustainment include organizational-level variables, such as leadership, organizational culture and climate, and workforce retention, as well as provider-level factors, such as attitudes toward EBI, emotional exhaustion, and job autonomy (Novins, Green, Legha, & Aarons, 2013). Key outer-context variables that can impact service provision at the organizational level are leadership within government systems, legislation, policies, regulations, inter-organizational networks, contracts, and funding processes (Aarons et al., 2011; Damschroder et al., 2009). Such variables often emerge as crucial to implementation and sustainment of EBIs, with government administrators structuring contracts to reflect priorities of public agencies and to influence organizational and provider behaviors regarding EBI provision (Bruns et al., 2008).

Methods

We used a case study design to assess the transition to the BV-PIPS in a human services delivery context. The design allows for a comprehensive, systematic, and in-depth investigation of how the BV-PIPS was implemented from the perspectives of stakeholders who experienced this transition, in addition to the short-term outcomes of this transition (Feagin, Orum, & Sjoberg, 1991; Yin, 2013). A case study approach is useful for illuminating the ways in which a purportedly generalizable model like the BV-PIPS responds to the variable and context-specific factors of human services. Although its developers cite several internally-conducted evaluations as providing success stories for the BV-PIPS (Kashiwagi et al., 2012), to our knowledge this case study represents the first independent assessment of the model in any human service system.

Sample

In recent fieldwork within the study system (2012–2013), we completed extensive interviews and focus groups with key stakeholders involved in implementing the EBI, including home visitors, supervisors, EBI coaches, CBO administrators, and government officials. During these interviews, CBO administrators and government officials in the study system spoke with trepidation as well as hope about the pending implementation of the BV-PIPS to procure service contracts and its possible impact on EBI sustainment. Although well-versed in the nuances of EBI delivery, home visitors, supervisors, and coaches possessed little knowledge about contract procurement and lacked information about the BV-PIPS. Thus, they were not included in the sample for this case study. Given the iterative nature of qualitative research in which findings from one round of fieldwork may inform a subsequent round (Patton, 2015), we returned in 2014 to assess how individuals with the most knowledge of contract procurement within the context of the EBI understood and experienced the overall transition to the BV-PIPS. We interviewed the same CBO administrators and government officials who participated in the previous round of research, and broadened the sample to include other system stakeholders involved in the procurement process.

The purposive sample for the in-person, semi-structured interviews analyzed for this case study consisted of 20 individuals, including eight high-level and five mid-level administrative staff of three CBOs, six government administrators (including three procurement and contract specialists), and an academic collaborator who participated in the latter phases of the procurement process. All persons had one or more direct roles to play in developing or responding to the solicitation notice for home-based services, reviewing proposals, and awarding contracts under the BV-PIPS. We invited participants via both phone and email, resulting in a 100% response rate. The sample included 14 women and 6 men. The Human Research Protections Program of the University of California, San Diego, approved this study. Informed consent was obtained from all participants.

Data Collection and Analysis

An anthropologist and a psychologist (1^st and 2^nd authors) undertook one-hour semi-structured interviews to elucidate participant knowledge and involvement in the use of the BV-PIPS approach and its implications for delivery and sustainment of the home visitation EBI. The interviews occurred between six to eight months after the award of contracts for home-based services. This allowed time for the process to have been instantiated and for reflection about perceived outcomes. Participants were asked to describe the status of the home visitation EBI; the impact of the BV-PIPS on sustainment of this EBI; how the BV-PIPS worked; their specific role in, and experiences with this process; other key players involved in the process; the effects of the process on CBOs, including employees involved in management and/or delivery of the EBI; changes to interagency relationships across CBOs; and overall positive and negative impacts of the BV-PIPS on the inner and outer contexts of the study system.

All interviews were digitally recorded, professionally transcribed, and checked for accuracy by at least one author. We employed an iterative process to review the textual data from interviews and utilized NVivo 10 qualitative data analysis software to facilitate this work (QSR International, 2012). Data analysis proceeded by engaging in both focused and open coding to locate the themes and issues that emerged from the interview transcripts (Corbin & Strauss, 2008). The transcripts were independently coded by two research assistants to condense the data into analyzable units. Segments of text ranging from a phrase to several paragraphs were assigned codes based a priori on the particular topic areas and questions that made up the interview guides (Patton, 2015). These codes thus centered on “expectations for new contracting process,” “BV-PIPS implementation,” “BV-PIPS outcomes,” and “status of the home visitation EBI program,” and enabled us to examine both the salience and meaning of these constructs for participants through the provision of descriptive data derived from their actual words. The resulting narratives directly reflected their perceptions and experiences related to the BV-PIPS and sustainment issues related to the home visitation EBI. We employed these constructs to help make sense of the data, but not “to the point of straining or forcing the analysis” (Patton, 2015, p. 545). During our review of the transcripts, new codes were subsequently identified and defined to capture information on emergent themes (e.g., “blind review,” “outsourcing,” and “de-professionalization”). We then used subsequent rounds of focused coding to determine which of these themes surfaced frequently and which represented unusual or particular concerns to the research participants.

In the staged approach to analysis, the research assistants coded sets of transcripts, created detailed reports that described and linked codes to each theme and issue, and shared their work with the authors for review. Through the process of constantly comparing and contrasting codes with one another (Corbin & Strauss, 2008; Glaser & Strauss, 1967), we grouped together those with similar content or meaning into broad themes linked to segments of text.

Results

We organized the themes under two overarching categories: perceptions of implementation of the BV-PIPS for home-based services (Table 1); and perceptions of resulting performance-based contracts (Table 2). Relative to the prior procurement process, the BV-PIPS generated concerns about outsourcing and the application review process, the status of local expertise and government-CBO collaboration, the survival of the EBI, and impacts on CBO staff. Reactions to the BV-PIPS illustrate the importance of interagency CBO relationships within the study system, the desire among administrators for greater control over decision-making and financing with respect to EBI delivery, and overall CBO support for transitioning to new measurement and reporting standards. Below we discuss the themes comprising our main categories and provide quotations exemplifying the views and experiences of participants. Some quotations were edited to enhance readability, i.e., we eliminated expressions such as “um” and “you know” and redundant text. Since there was only one academic collaborator, we minimized the quotations attributed to this person to protect her/his identity but will note that her/his views often dovetailed with those of the child welfare government personnel and CBO administrators. We use pseudonyms in place of the names of government agencies, CBOs, and people.

Table 1.

Major Themes Regarding Perceptions of Implementation of the BV-PIPS for Home-Based Services

Outsourcing concerns	The BV-PIPS opened the bidding process to out-of-state companies, which many participants viewed as threatening to local economies and the provision of the home visitation EBI. Consideration of ill- prepared out-of-state bidders was described as a waste of time.
Disregard for “subjective” elements: Local expertise and government-CBO collaboration	The BV-PIPS blind bidding process prevented consideration of relevant variables considered “subjective,” such as CBO reputation, community relationships, and preexisting government-CBO relationships. CBO administrators generally felt that their expertise was not fairly evaluated and the complexity of the human services context was minimized.
Threat to home visitation EBI and other evidence-based programs	The BV-PIPS blind bidding process prohibited specification of the home visitation EBI in bids. The majority of participants feared the loss of a successful and well-established EBI in the study system.
Review team (in) expertise	The BV-PIPS reviewers were criticized for lack of knowledge about child welfare services in general and EBIs in particular.
Job insecurity and stress within CBOs	The CBO administrators indicated that a lack of communication and delays during bidding processes adversely affected staff and potentially contributed to staff turnover.

Table 2.

Major Themes Regarding Perceptions of Resulting Performance-Based Contracts

Interagency relationships	Despite contract changes, the CBOs maintained preexisting collaborative relationships that buffered an unsuccessful CBO from financial hardship through subcontracts.
Greater CBO control over decision making within the inner context	The CBO administrators appreciated the newly instituted daily rate payment model, which allowed greater discretion in expenditures and did not require CBOs to subsidize services. However, some government administrators feared negative long-term financial effects and possible de-professionalization of service provider staff.
Transition to new measurement and reporting standards	The CBO administrators reported satisfaction with more stringent, outcomes-focused reporting requirements.

Perceptions of Implementation of the BV-PIPS for Home-Based Services

Since 2008, the Department of Purchasing within the local government that oversees the study system utilized the BV-PIPS model to contract for a variety of durable goods and services, especially in the areas of construction and properties. Having utilized the BV-PIPS for 30 major procurements, the Department of Purchasing recommended the BV-PIPS to the executive leadership of the child welfare system, which was preparing to procure new contracts for foster care and home-based services. This leadership was expressly interested in moving from “the prescriptive sort of RFPing and contracting” to “more of a specific performance-based approach to service delivery,” and diversifying the range of potential bidders within the marketplace. Government administrators specializing in purchasing and procurement, rather than child welfare stakeholders, expressed enthusiasm for the model and high satisfaction with the outcomes of the previous BV-PIPS projects in terms of quality improvement, cost reduction, and a massive decrease in paperwork associated with the review and management of awards. In keeping with the dominant BV-PIPS literature, these outer-context, i.e. system-level, actors explained that the model worked to minimize risk, responsibility, and decision making for the “client,” i.e., the local government. They also claimed that the BV-PIPS would decrease the amount of oversight exercised by the client regarding the vendor’s use of funds once a contract was granted, whereas ongoing performance measurement would minimize deviation from the terms of the contract.

We asked all participants about the steps of the BV-PIPS enacted to procure home-based services to assist families in preventing child maltreatment. Following the issuing of the solicitation, potential vendors completed an application divided into three parts: (1) Project Capability Plan; (2) Risk Assessment Plan (described as “Risk Mitigation” by multiple participants); and (3) Value Added Plan. Each section was allotted two pages of text total. The application solicited a minimal amount of information from vendors for specific items, including “documented performance,” “risk description and impact,” “value added claims,” and “cost impact.” To prevent bias during the bidding process, names of applicants and their proposed service models were not included in the proposals. As described by the government administrators, the rationale for the blind review was the belief that potential vendors will more accurately portray their capability to the non-expert client if they are prevented from leveraging their reputation, or from using technical terminology, to impress or mislead reviewers during the application process. As we clarify below, other participants were of the opinion that the blind review requirement reduced the burden on vendors to substantiate claims about their expertise.

The next three phases of the BV-PIPS included: (1) Selection (procurement); (2) Clarification (or pre-award); and (3) Management by Risk Minimization (project management). To enter into the third phase, companies applying for a contract were to outbid their competitors by demonstrating superior project management capability coupled with both feasibility and cost efficiency associated with delivering specific products or outcomes. Government employees with experience in traditional contracting processes for child welfare services were excluded from taking part in the Selection phase due to concerns that they would “pick apart” the proposals and that their unique perspectives, biases, and emotions would cloud their judgment. The initially selected bidders—one out-of-state company and one of the three local CBOs—were invited back for interviews that took place during the Clarification phase. The out-of-state company was later eliminated because its proposed intervention lacked an appropriate evidence base. A second local CBO was then brought back to the table. Eventually, two local CBOs received contracts.

Outsourcing concerns

With the exception of the government administrators specializing in purchasing and procurement, all but one of the participants described the bidding phase of the BV-PIPS as disconcerting. Participants worried that the BV-PIPS opened the door to unqualified out-of-state companies and potentially compromised both the entire child welfare system and sustainment of the home visitation EBI.

Participants suggested that the prospect of funding out-of-state agencies threatened local economies and prior government investments to instantiate a proven EBI in the service system. One CBO administrator explained, “This should be in-state providers providing these services. If they are qualified, effective providers [locally], then I think that’s where our [government] dollars should be spent on local businesses versus out-of-state, for-profit companies.” This CBO administrator complained that the inclusion of out-of-state companies siphoned revenue out of the service region, and also “[flew] in the face” of past public-private efforts to support a home visitation EBI that addressed specific needs within the service system. Two government administrators articulated this same concern, with one commenting on the lack of preparedness of representatives from the out-of-state company invited to the Clarification phase interview:

[The out-of-state applicants] couldn’t afford to even do the travel [to deliver a home-based service] for the rate they were proposing. When I inquired, it was, ‘Well, we’ll find a location somewhere convenient and the families can come to the parenting classes.’ I said, ‘This is a home-based service, if you didn’t pick up on that in all the bid materials.’

The second government administrator speculated that the company had been selected for an interview because its proposed costs were so low: “I looked at the cost and knew that they didn’t understand the project or really realize what it would require in [this child welfare system] because their cost per family per day was about half of what [a local CBO’s] was.”

Participant concerns about the outsourcing of child welfare services also stemmed from convictions about the importance of local knowledge. A CBO administrator commented, “I don’t believe anybody can come in here and do a better job. We know the nooks and crannies of the counties, we know the personalities, we know the cultures, and I can’t believe this is going to turn over to somebody who doesn’t know this stuff.”

In the end, contracts were not awarded to the out-of-state bidders, a development that the procurement specialists attributed to the BV-PIPS process. Yet, most other participants shared the belief that, by allowing seemingly unqualified companies from outside the system to take part in the bidding process because of lower-cost projections, the BV-PIPS drained both time and resources, provoking both “fright” and “anxiety” within local CBOs and compromising prior collective efforts to embed an EBI within the child welfare system.

Disregard for “subjective” elements: Local expertise and government-CBO collaboration

Participant outsourcing concerns reflected widespread apprehension about how passivity embedded within the BV-PIPS could abruptly bring about the dismantlement of local government (outer context) and CBO (inner context) expertise in the child welfare system. Participants noted that implementation of the BV-PIPS necessitated indifference to significant elements of CBO expertise within communities, EBI infrastructures, and preexisting trust and relationships between government administrators and the CBOs. The source of these concerns was the blind review requirement of the BV-PIPS, which prohibited applicants from disclosing identifying information in their proposals. This prevented consideration of relevant information considered “subjective” in the BV-PIPS, such as CBO reputation, geographic feasibility, type of intervention, and relationships among the CBOs and communities comprising the study system.

The majority of participants criticized restrictions placed on information that the applicants could include in their six-page proposals, as it prevented CBOs from showcasing both their expertise and capacity to be accountable to communities. The CBO administrators expressed dismay that the track records and accomplishments of their organizations did not factor into the selection process, causing reviewers to overlook bidders with a history of performing well in favor of those with lower costs. According to one CBO administrator,

[I assumed that] when we submitted our bid and showed that we’ve been doing a model that has the specific proven outcomes that they were looking for that we would have been at the top of the list. We were the only ones doing this work that they’re asking to be done and we’re using the model that has demonstrated effectiveness in achieving this.

A second CBO administrator explained, “We’ve earned a reputation, we’ve earned the community respect, and it puts us in this position sort of guaranteeing to the community [that] we are a known entity and there’s some equity that comes with that as a promise of good service back to the community.” Relatedly, a government administrator who worked closely with the CBOs clarified, “There’s nothing like knowing an agency, knowing their ethics, knowing their practices, knowing the quality of their documentation and the miracles they work with families.”

In addition to the disregard for the reputations of the CBOs and their relationships to communities, some participants lamented the breakdown of “trust” between the child welfare agency and the CBOs under the BV-PIPS. One CBO administrator claimed that the BV-PIPS created a “paradox:” “You [the child welfare agency and CBOs] are acting like you’re not partners when you go through this process but then you know you’re partners as soon as you leave the process and get the award because you can’t function without being partners.” For most participants, delivery of home-based services to reduce child maltreatment and neglect necessitated close collaboration between the local government administrators and contractors, a belief that made the competitive bidding process troublesome to both sets of stakeholders. On this topic, a second CBO administrator commented, “No matter what you do, the world of child abuse doesn’t work like building a house. It’s the building of this relationship and trust and problem solving….” A government administrator further elaborated,

I saw this discrepancy because [our agency] has the mandate to fulfill child protection. That made us, in my mind, the experts about what we needed from contractors knowing our families, knowing the issues that they’re dealing with. I have lots of faith in [CBO A] and [CBO B]. They know our population and they’ve done good work with them.

As this participant suggested, the operating assumption of client need and vendor expertise under the BV-PIPS minimized the complexity of the human services context, which required pragmatism based on familiarity with local dynamics and trust of agencies providing services.

Threat to home visitation EBI and other evidence-based programs

Participants felt that the blind review process undermined the status of EBIs in the service system. As indicated earlier, adherence to the BV-PIPS prohibited the child welfare agency from specifying any particular EBI in the solicitation notice to encourage bidders to present innovative practices. This requirement weakened the prioritized status of the home visitation EBI, which most participants praised for producing measurable outcomes (including family and service system improvements). Although participants supported moving into new directions, the identity and research-base of interventions proposed under the BV-PIPS were unknown to the initial reviewers, making it impossible to judge whether they met criteria for what other child welfare officials knew worked locally. In fact, three applicants recommended program initiatives that failed to meet the basic EBI criteria. One government administrator stated,

In the [solicitation], it said it had to be an evidence-based practice that they had experience with and could show their success with. We had people from both within and without the [study system] that brought practices to the table that were not evidence-based at the level that we requested.

One such vendor was invited to an interview in the Clarification phase, leading this administrator to opine that “time was wasted by a lot of people from out of state” who “should have never got through the first phase because they didn’t have an evidence-based practice.”

In general, the government administrators directly involved in the procurement process agreed that sustaining the home visitation EBI was “not a core goal” under the BV-PIPS. Interestingly, nearly all of the administrators who cited themselves as having the greatest amount of involvement in the initial procurement and selection processes admitted to having very little knowledge of the home visitation EBI. This suggests a disconnection between the stated need for evidence-based services locally, and the knowledge of those involved in procurement processes.

In contrast, the CBO administrators were extremely concerned about the potential loss of a successful and established EBI. One CBO administrator remarked,

We came really close to tossing out [an EBI] that we knew had been effective and that we had proof of its effectiveness, for something that wasn’t truly evidence based or targeted for these specific families. If I was a decision maker at the [local government] level, that would be really scary to me that this new process got us really close to tossing out something that works and spending a lot of money on something that was unproven.

A second CBO administrator acknowledged the importance of innovation while resisting the implication of bias towards the home visitation EBI:

Now not to say if something else comes along that that might not be adopted too at some point in time but I don’t feel like we’re the boiled frog where we’ve been doing something for so long that we’re no longer conscious and able to make a decision. I feel like it really is something that we believe in and support.

Others agreed that previous efforts of government administrators to systemically support the home visitation EBI as a premiere program were at risk because of the BV-PIPS.

Review team (in)expertise

Along with concerns about disregarding local expertise, preexisting government-CBO relationships, and the importance of identifying proposed EBIs during the blind review, the CBO administrators also universally criticized insufficient knowledge about child welfare services, in general, and EBIs, in particular, among the BV-PIPS reviewers who vetted the proposals during the Selection phase. One CBO administrator recounted,

Someone had shared with me that [a reviewer] had called [Dr. XX] from [the college] and they were trying to understand what evidence-based practices were, what that meant, and [s/he] was shocked by the lack of understanding that the decision makers had of even what they were considering and the ramifications of the choices they were making.

A second CBO administrator stated,

To me it was confusing. I thought if you’re asking people to come and show you’re the expert and you’re going to evaluate their level of expertise—I mean if a neurosurgeon came in and told me what practices they use and I can’t even [evaluate that]—I’d just say ‘Oh that sounds great, but I don’t really know anything.’ I think that seems a little silly.

Yet a third CBO administrator, who described the process as “traumatizing,” said the reviewers were “unable” to grasp the meaning of concepts of import to child welfare workers, and s/he found her/himself trying to “educate” them during the Clarification phase. S/he summarized the application of the BV-PIPS to the home-based services as “de-professionalizing,” because it lessened reliance on the expertise of persons accomplished in the child welfare field.

In the end, the chosen contractors were staffed by home visitors certified to implement the EBI. Yet, this outcome was not considered inevitable. One CBO administrator said,

We were literally on the edge and it could have tilted away from us. In fact I think it was probably already heading in that direction. Luckily someone was able to exert enough influence or get the message across that ‘No, you guys are throwing out an evidence-based model that actually works, wake up.’ I don’t know who that was.

Other CBO administrators similarly speculated that the eventual involvement of an individual who was especially knowledgeable about child welfare EBIs during the Clarification phase helped to steer the reviewers back toward the home visitation EBI, because it had already delivered clear positive family and workforce outcomes in the service system.

Job insecurity and stress within CBOs

Participants reported that the initial phases of the BV-PIPS were destabilizing for CBOs and the service system workforce. For example, the majority of CBO administrators reported a weakening of staff morale, which, in some instances, resulted in significant turnover. Participants repeatedly discussed a profound sense of insecurity pervading their organizations, which they attributed to insufficient communications from local government reviewers regarding the status of each CBO’s contract with the child welfare agency. As one CBO administrator described,

The [local government] wasn’t giving us any information. Every month, we didn’t know what was going to happen. We didn’t know if on, you know, September 2nd they were going to say, ‘Oh no, sorry, we’re going to start the new contract,’ and we have all these staff members that we are going to have to let go or pay their time out.

Another said that their staff members were, “Super, super nervous because approximately half of the work we do as an organization is related to [this] contract. Staff [was] becoming increasingly concerned, ‘Are our jobs okay?’ There was a lot of anxiety at all levels.” Yet another CBO administrator estimated that up to ten staff members resigned and sought work elsewhere due to fear that loss of the contract would leave them without work. Similarly, a second CBO administrator described how delaying the BV-PIPS contract deadlines impacted staff: “It was a negative experience. It definitely dampened morale around here. People were going and finding other jobs.” A colleague at the same organization added, “They didn’t know for sure if they were going to have a job, and I mean, we reassured them to the best of our ability that, we hoped that even if we didn’t get the contract, that we had a back-up plan.” An overwhelming number of participants agreed that future bid review periods would benefit from having improved communication requirements between applicants and reviewers at the local government level.

Perceptions of Resulting Performance-Based Contracts

Government administrators with the greatest prior knowledge of the home visitation EBI were the least optimistic about the BV-PIPS or its possible impacts on the program. The “non-expert” government administrators shared little insight in this regard. They were unaware of the negative experiences of CBOs under the BV-PIPS. These administrators also suggested that the CBOs had found the BV-PIPS review process to be “refreshing,” “all positive,” or exactly the same as the previous contracting process (“a contract is a contract,” one said).

Despite an implementation process often characterized in interviews as a “nightmare,” participants described positive changes and adjustments in financial and reporting processes that were attributed to the performance-based contracts procured through the BV-PIPS. Changes and adjustments on the financial side included more accurate budgeting for service delivery costs and CBO control over expenditures. On the monitoring side, changes and adjustments centered on enhanced tracking of EBI provision, referrals, and family progress and outcomes. Although these changes and adjustments could have been instituted independent of the BV-PIPS, the CBO administrators were enthusiastic about the increase in flexibility they experienced in relation to CBO operations and the constructive effects of having stronger reporting requirements.

Interagency relationships

The three local CBOs had a history of productive working relationships to deliver the home visitation EBI as well as a broader array of child welfare services, including exchanging information about recruiting qualified staff, salary levels, and maintaining an organizational infrastructure for EBI delivery under their contracts with the local government. Despite being pitted against one another during the bidding process, the CBOs took steps to preserve their collaborative relationships under the new contract to address pending challenges. For example, the CBOs had proposed to expand their services to additional sites in the study system when they submitted their bids and two CBOs did so upon receipt of the award, while one lost sites it had previously served. These changes presented challenges for all the CBOs, including the need to quickly hire staff to cover new regions and the threat of financial losses. These challenges were managed through subcontracts between the one CBO that had not gotten a contract and the two that had, an uncommon practice within the study system, though not unprecedented. This arrangement ensured both continuity of service to the community and, for the short term, buffered one CBO from major financial hardship. Several CBO administrators felt that this kind of interagency support was essential in a market where only a small number of local CBOs possessed capacity to deliver EBIs on a large scale, and was necessary to mitigate the threat of out-of-state agencies being awarded contracts. One CBO administrator explained,

We had the conversation about out-of-state agencies being very competitive. We said, ‘I don’t know if they [BV-PIPS reviewers] like us and you [another CBO] don’t know if they like you. Why don’t [we] bid on both regions and if we win [them], we will give you your existing areas and you guys do the same? We were basically hedging our bets.

Although the administrators of the subcontracting CBO voiced some unhappiness with their loss of control, most CBO administrators felt that strong interagency relationships were preserved.

Greater CBO control over decision making within the inner context

Contract awardees appreciated that the performance-based contracts reduced the micromanagement of services by the local government. The CBO administrators spoke favorably about the system increasing the control of CBOs over service provision by instituting a daily rate payment model and disallowing contract changes by the child welfare agency. Although some CBO administrators expressed doubt that the child welfare agency would completely relinquish control over service provision, they suggested that the new contract put them in a “structurally” stronger position to make decisions and negotiate with the child welfare agency.

Even in the larger CBOs that provided a wider spectrum of services beyond the home visitation EBI, the shift toward more risk-based payment mechanisms engendered by the BV-PIPS represented a relatively recent development. As one CBO administrator with four decades of experience in the non-profit sector explained,

Everything is moving into these unusual different kinds of payer methods where the provider assumes greater risk…. We got that experience through BV-PIPS, so we now turned this into a ‘We’re more prepared because we have the BV-PIPS badge of honor.’ We have gone through it. We now know you have to cost things out very, very carefully or you’ll lose your shirt on [the contract].

Thus, the BV-PIPS was viewed as providing the opportunity for CBOs to better prepare themselves to adapt to changes in the economic landscape of public-sector contracting.

Until the BV-PIPs had been implemented, the dominant form of payment for services funded by the child welfare system was based on the cost-reimbursement model versus a fee-for-service model. The CBO administrators were pleased with the daily rate payment model incorporated into the new performance-based contracts. Although the traditional contracting process could have led to this same outcome, participants believed that it was due to the BV-PIPS. This new model allowed CBOs to exercise greater discretion in making expenditures without first having to take the time to obtain permission from multiple government bureaucrats. One CBO administrator clarified, “[The pay rate is] $28 per family, per day. We get to spend the money however we want to. They give us the money and they expect us to do the job that we said we’d do and they have very little control or say in however we budget those individual dollars.” A second administrator echoed this sense of greater flexibility under the new contract:

When we want to pay for someone’s electric or whatever, we have that freedom to use our own judgment better. We don’t have to go through all the channels and get the permission and get the caseworker on board and all of that. We [now also] have the ability to just go ahead and hire, so we don’t have to worry so much.

The CBO administrators were also pleased that their agencies no longer had to subsidize services until they were reimbursed and could build in remuneration for staffing and agency overhead costs. One CBO administrator stated, “We were losing hundreds of thousands of dollars per year if you looked at our actual expenses compared to what our cost reimbursement contracts allowed us. So far this year it looks like we’re going to come much closer to breaking even.” Another CBO administrator commented, “In this proposal, we were able to actually try to cover our costs and try to break even with these contracts. We’re pretty excited about that as an agency.” This administrator noted, however, that her/his organization did assume the risk of putting forth a budget that, although more accurately reflecting the costs of the home visitation EBI, was almost passed over in favor of lower bids submitted by less qualified competitors.

Although the new payment model was received positively by most participants, we interviewed government administrators who believed that the financial terms associated with the use of the BV-PIPS would negatively impact the child welfare agency in the long run. According to one government administrator,

It’s not helpful financially because we had to pay roughly 10% to them [the Department of Purchasing] to manage our contract and they dragged it out for so [long]. We’ve worked with them forever as the [Department of Purchasing], and now they’ve learned this contracting method from [the BV-PIPS developers] and they have to pay for [technical assistance] from them, so they’re charging the agencies that are using BV-PIPS roughly a 10% fee. But the contracts don’t have awards, so we [the local government] lost about $1.4 million out of the contract account.

This government administrator was unsure about how the loss would impact contracting for the home visitation EBI or other types of human services in the study system down the road.

Additionally, government administrators expressed apprehension about the possible loss of quality service delivery over time, explaining that the change to a daily rate payment model and elimination of the traditional contract meant that administrators could no longer stipulate the educational, credentialing, and salary levels for direct service providers. One government administrator suggested that the CBOs might seek to bring in less educated, lower-paid staff to implement programs, such as the home visitation EBI, which would contribute to the potential de-professionalization of service provision within the inner context of the study system. These participants once again struggled with the BV-PIPS operating assumptions that the “client” had no expertise and should provide minimal oversight of contracts. They also suggested that the provision of child welfare services presented complexities that the more simplistic perspective embedded in the BV-PIPS contracting formula was unable to accommodate.

Transition to new measurement and reporting standards

Although the use of the home visitation EBI remained constant after the performance-based contracts procured by the BV-PIPS had been implemented, the CBOs experienced changes in both reporting requirements and in billing procedures. With few exceptions, participants across all stakeholder groups generally described the administrative changes as manageable and, in some instances, beneficial adjustments.

Reporting requirements for the CBOs now occurred monthly. Based on the outcomes projected during the contract bidding process, CBOs were responsible for tracking each family’s progress through the home visitation EBI. Although the “metrics relative to project performance” differed slightly for each CBO, they generally centered on provision of EBI modules and fidelity, staff training and supervision, and family outcomes (e.g., improvements in parenting skills and problem solving within families, referrals to mental health, substance use, and domestic violence services for families in need of such assistance, and families reentering the child welfare system). “Metrics relative to risk mitigation” focused on monitoring of workforce (staff turnover, vacancies, and injuries), referrals from the state government, wait lists, child deaths due to abuse or neglect by a parent/caregiver or person in the home, and delayed payment to the CBOs. The reporting requirements seemingly represented a substantial new workload for CBOs that would likely have repercussions on both their staffing and labor costs, yet most CBO administrators responded to them positively. One CBO administrator explained, “It does make you more aware of the things that we’re doing and the things that are being measured… It seems like they’re beneficial. It kind of gives me an overview of where we’re at with what we’re doing.”

The CBO administrators were optimistic about the overall impact of the new performance-based contracts, emphasizing that their agencies were becoming more outcome-oriented. One CBO administrator said, “[The new contract is] really focused on the outcomes [of the home visitation EBI]. It helps the agency and the program measure those outcomes in a more accurate form.” A second said that, under the new contract, her/his CBO was…

…much more focused on the outcomes. We can be very conscientious of, ‘Are we doing a good job?’ ‘Are the families better when we closed the case than they were when we came in?’ ‘Are we paying attention to that?’ And that causes us to look at each individual worker. ‘Are some of our workers having more success than others and teaching families the [home visitation EBI model]?’ If yes, then we know where we need to provide more training or which workers could use additional support. That’s a positive outcome.

A third said, “[We’re] having more conversations about [the EBI] with the case managers, the supervisors, [and] with everyone…” Her/his colleague added, “We learned the importance and the value of our [EBI] and how important it is as we move forward and make sure we’re hiring strong staff and training them, and that we’re continuing to retain staff.” Having more robust measurement and reporting mechanisms thus appeared to exert beneficial impacts on CBO staff improvement efforts. Yet, as one government administrator noted, this emphasis on measurement and reporting was not unique to performance-based contracts procured under the BV-PIPS.

Discussion

Across the nation, states and local governments are moving toward the adoption of market-oriented, performance-based contracting mechanisms to procure goods and services in public service systems, including those focused on child welfare. The BV-PIPS model represents one such approach. Ideally, such approaches to contracting decrease costs for and prevent micromanagement of these goods and services by purchasers, reduce the size of government, devolve “risk” to private and/or nonprofit companies, and facilitate competition and innovation in the public sector by disrupting traditional contracting approaches that discourage vendors from pursuing fresh ideas and approaches (Lee et al., 2006; McBeath & Meezan, 2010; Smith, 2010).

The proponents of one such approach, the BV-PIPS, desire to transform public-sector procurement into an objective process by rationalizing bureaucracy and eliminating the human element in decision making related to purchasing (Kashiwagi, 2011; Kashiwagi et al., 2012). As part of this process, its architects aim to reduce reliance on “trust” and “relationships” to remove personal bias from contracting processes. Yet, in the context of human services, this approach does not accommodate the view that trust and relationships are not merely “subjective” factors that unfairly advantage the favored few, but instead are central to the effective delivery of services (Johnston & Romzek, 2008; Milward & Provan, 2003; Van Slyke, 2007), particularly EBIs (Aarons et al., 2014; Hurlburt et al., 2014). Contemporary approaches to developing and implementing health and social welfare interventions generally emphasize the importance of ongoing partnership among funders, service practitioners, academic researchers, and other key stakeholders within inner and outer contexts (Dawes & Eglene, 2004; Horwath & Morrison, 2007). These partnerships can facilitate integration of the voices of relevant stakeholders in service design and implementation processes (Butterfoss, 2007; Hurlburt et al., 2014).

Several types of partnership models exist in public health (Butterfoss, 2007). They include Community-Based Participatory Research, or CBPR, in which trust, cooperation, and reciprocal working relationships are to be cultivated, celebrated, and enacted across a broad range of stakeholders over the long term (Minkler & Wallerstein, 2008; Selby, Beal, & Frank, 2012). Partnership models are also embodied in national initiatives to advance health and wellbeing within the general populace, most recently in the efforts of the Patient Centered Outcomes Research Institute to improve health and health care delivery (Selby et al., 2012). Moreover, in focus groups with stakeholders from 12 states, public- and private-sector participants identified trust and communication, as well as engagement of all key stakeholders, as two main facets of successful child welfare system contracting arrangements (Flaherty, Collins-Camargo, & Lee, 2008). Also relevant is the nature of collaborative relationships between CBOs given the possibility of “co-opetition,” referring to complicated interagency relationships that shift between, or have overlapping elements of, competition and collaboration within contexts for EBI procurement and delivery in public-sector systems (Bunger et al., 2014).

Researchers of public administration suggest that successful contracting for human services typically requires ongoing collaboration, in addition to “an extraordinary amount of advance planning and negotiation” (Romzek & Johnston, 2002, p. 448), particularly when market conditions are considered imperfect, i.e., when there is a deficit of local CBOs with capacity to perform needed services (Milward & Provan, 2003; Smith, 2010). Relational contracting is not without perils, however, as it can “degenerate into collusion between the buyer and the seller at the expense of the taxpayer if not closely watched” (Milward & Provan, 2003, p. 10). Yet, longer-term partnerships allow the client and vendor to develop familiarity with one another and their respective organizational cultures, goal agreement, communication quality, and cooperation in contract implementation (Amirkhanyan, Kim, & Lambright, 2012), the combination of which can portend favorably for both the implementation and sustainment of EBIs in child welfare systems (Aarons et al., 2014; Hurlburt et al., 2014). In addition, contractor stability may also contribute to provider effectiveness and the success of contract implementation; such stability tends to be predicated on networks of contractors and government managers that provide a cushion against the “unforeseen impacts that inevitably occur in the delivery of complex social services” (Romzek & Johnston, 2002, p. 449).

Contract specification detailing contractor roles, responsibilities, and procedures can contribute positively to collaborative relationships (Johnston & Romzek, 2008), and may have contributed to the strong partnership that reportedly existed between government administrators and CBOs in the study system prior to the adoption of the BV-PIPS. Contracts procured under the BV-PIPS do not require this level of specification. It is instead the vendor’s responsibility to stipulate their scopes of work without interference from the client. In this case study, the fact that government administrators and CBOs had forged collaborative relationships and engaged systemically in developing and maintaining a robust infrastructure to support a carefully chosen and proven home visitation EBI was of little consequence when the BV-PIPS was introduced. Rather, by failing to recognize and value and even discouraging relationship-building between clients (public stewards) and vendors (CBOs), the BV-PIPS, by design, casts suspicion upon established relationships, engendering communication vacuums during the bidding process that can contribute to organizational and system stress. We argue that such stressors can be detrimental in system change initiatives. In this way, as illustrated in this case study, procurement process models and contracting systems derived from for-profit and non-human services procurement, such as the BV-PIPS, may fundamentally alter roles and relationships between local governments and CBO stakeholders that have, in the past, collaborated successfully to support EBI implementation and sustainment and, in partnership, to work towards effective solutions to critical human service and public health problems.

Over a decade ago, Milward and Provan (2003) warned that competition in human services may discourage potentially useful collaboration between government administrators and CBOs and thus hinder the overall performance of a human service system. Of relevance to EBI sustainment, this case study has raised the prospect that government-led efforts to engender competition within an imperfect provider market could result in the abrupt discontinuation of service delivery models, such as the EBI described in this study. There was general consensus among participants that the EBI would no longer be practiced had the contracts been awarded to the bidders who did not propose this particular program in their BV-PIPS application. Services for families, moreover, may have been adversely impacted, as it would likely take time for organizations new to the provider marketplace to establish service delivery capability within the study system. These organizations would likely require local staff, which could lead to staff turnover for the in-state CBOs and strain provider network capacity (Johnston & Romzek, 2008).

In addition to threatening a viable and locally proven EBI, most participants argued that opening the bid to out-of-state for-profit companies might negatively affect local economies and diminish the capacity of government to mobilize local expertise for EBI delivery. Although it is possible for CBOs with strong local presences within particular service systems to be units of larger organizations headquartered in other states, the BV-PIPS opened the door to vendors with no previous experience in the community and thus with no viable infrastructure locally to support the timely delivery of EBIs within expansive service areas. The threat of outsourcing did not materialize; however, participants worried about its possible future manifestation on an even larger scale. In the end, the majority of participants indicated that the new contracting process was divorced from the need for EBIs for critical service populations, and that the BV-PIPS was out of touch with national initiatives for improving services for families at risk of child maltreatment through EBIs (Chaffin & Friedrich, 2004; Paulsell, Del, & Supplee, 2014).

In addition to dismissing preexisting relationships in systems, the BV-PIPS blind review process suppressed consideration of other supposedly “subjective” factors, such as reputation and geographic feasibility, in favor of “objective” measures of vendor expertise (or “documented performance”), “risk description and impact,” “value added claims,” and “cost impact.” However, despite the emphasis on vendor expertise, participants largely felt that the six-page application made it impossible to accurately document and assess bidder capability. The BV-PIPS not only ignored crucial factors of the service environment, i.e., CBO knowledge of local delivery contexts and relationships with client bases and communities, it left decision making up to reviewers with little knowledge of child maltreatment and EBIs in human service settings.

For a system to fully benefit from introducing competition, government buyers must be knowledgeable and savvy connoisseurs of the goods and services they purchase within the private sector (Kettl, 1993; Van Slyke, 2003). This entails examining reliable data and working with private-sector partners to understand complexities of the service environment, needs of clients, and realities of service delivery capability and cost (Flaherty et al., 2008; Van Slyke, 2003). In contrast to the principles of the BV-PIPS, which aims to dismantle buyer expertise while advancing a “hands-off” approach to contract management, successful contracting depends on knowledge, preparation, and negotiation on the part of public administrators, especially at the beginning of the contracting process, regardless of whether the contracting relationship is based more on incentives and monitoring (e.g., principal-agent model) or on trust and relationships (e.g., stewardship model; Van Slyke, 2007). This requires increased management capacity for public agencies in establishing RFP and contract specifications, pursuing bids, monitoring outcomes, and a greater investment in solidifying trust, exchanging information, and engaging in joint decision-making with contractors (Romzek & Johnston, 2002; Van Slyke, 2007).

Despite the implementation issues associated with the BV-PIPS, the CBO administrators were largely positive about the finalized performance-based contracts. The CBO administrators believed they were better prepared to compete for future contracts because they had been prompted to shore up both the budgeting and financial capabilities of their respective agencies. The leadership of the contracted CBOs is content with the amount of financial flexibility and decision-making ability they now have. Beyond the bidding period, staff engagement and relations have purportedly improved within the CBOs. Outcomes measurement and reporting are also said to have improved—changes that the CBO administrators feel are beneficially impacting management of the organizations and use of the home visitation EBI. While these results could have been achieved by other performance-based contracting processes, they also seemingly contradict the BV-PIPS developers’ claim that the assumption of risk and expertise is difficult and disruptive for vendors who are used to “avoiding accountability” (Kashiwagi, 2011, p. 33). In fact, the CBO administrators were enthusiastic about their increased accountability.

Limitations

This case study pertains to a single child welfare system and focuses on one specific but widely-used performance-based contracting system, the BV-PIPS. Its generalizability is thus constrained. Additional studies of implementation and outcomes borne from models derived from for-profit industries and applied to publicly-funded EBIs are needed. In addition, we drew from a purposive sample of local government officials and CBO personnel who were knowledgeable about and involved in the bidding process under the BV-PIPs. We did not interview representatives from the out-of-state companies, the developers of the BV-PIPS, or service recipients of the child welfare system. However, other data (available upon request) suggest that service recipients were highly satisfied with the EBI under the previous contracting regime. Satisfaction is considered to be a proxy measure for quality of services (Amirkhanyan et al., 2012). However, the BV-PIPS excluded current or potential service recipients among the stakeholder groups involved in reviewing bids and selecting contractors. Hence, we did not sample service recipients for this study. At this time, we cannot comment on the extent to which the BV-PIPS is influencing quality of services. We intend to track these issues over time through future data collection. Finally, we were unable to observe the implementation process first hand, although we did collect richly descriptive interview data to shed light on common perceptions and experiences soon after it had been implemented. Our dataset is only able to share information on reported short-term effects. Future research must focus on the longer-term intended and unintended effects of the BV-PIPS at both organizational and system levels.

Conclusion

This study reports on lessons learned from implementation of the BV-PIPS that may be of value to public administrators interested in the procurement and contracting of human services that will promote desired performance. These lessons stem from the rigid structure of the proposal that disallowed inclusion of traditionally key information (e.g., identity of the bidder), the blind review process, and inadequate expertise of the initial review team. Although performance-based contracting has positively expanded the “softer” criteria on which contracting decisions were previously based (e.g., “reputation”) to include carefully-defined performance standards and quality indicators (Johnston & Romzek, 2008; Smith, 2010), the BV-PIPS limits buyer capacity to assess for past performance in the service system. Furthermore, although the lack of adequate expertise among reviewers and key decision makers within the study system is not a result related to performance-based contracting per se, it is the result of the BV-PIPS. In the BV-PIPS literature reviewed for this case study, the system’s proponents repeatedly emphasize that the vendors (human service agencies) be the experts, not the buyers (government).

Given the complexities of human service delivery described earlier, we believe it is imperative for buyers to be knowledgeable about the product they are purchasing and its producer throughout all phases of the contract procurement process. This is not only crucial for ensuring EBI sustainment but would also make it possible for government agencies to spearhead more contextually-relevant strategies to deliberately discontinue and descale ineffective practices within child welfare systems, rather than relinquish all their control to vendors to select against these practices via competition within imperfect and complicated human service marketplaces. We should also note that as a general approach to procurement, performance-based contracting need not prevent government administrators invested in improving service system performance from specifying preferred EBIs within RFPs and other materials released to bidders.

Contracting for human services is distinct from other kinds of contracting (Smith, 2010) and collaboration between the client and vendor may be the key to scaling up and sustaining EBIs (Aarons et al., 2014; Hurlburt et al., 2014). Although the BV-PIPS did not shift the system and the evidence-based model, most participants believed that it jeopardized a system of collaborative relationships, community engagement, and an established EBI with proven results. The BV-PIPS approach is antithetical to an engaged participatory process to facilitate the practice of EBIs in the way that it trumps the voices of vendors and, for that matter, the intended service recipients. It also discourages local capacity building, a hallmark of intervention strategies associated with CBPR and other participatory methodologies (Minkler & Wallerstein, 2008). The very collaborative relationships that the BV-PIPS seeks to undo are widely acknowledged as essential to successful EBI implementation, and are enthusiastically promoted as part of successful public health initiatives both nationally and world-wide (Collins-Camargo, Armstrong, McBeath, & Chuang, 2013; Roussos & Fawcett, 2000; Weiss, Anderson, & Lasker, 2002). This case study thus provides a cautionary example against using the BV-PIPS to contract for human services without closely monitoring its implementation for unintended consequences that may negatively affect CBOs and service recipients that would likely benefit from an EBI.

This case study raises several questions about use of the BV-PIPS to contract for human services. For example, to what degree will the BV-PIPS alter the financial status of nonprofits and affect service provision within the inner context (either positively or negatively)? Will the BV-PIPS create new incentives and efficiencies for the human services CBOs, as it is said to do for industry and manufacturing, or will it destabilize their financial viability and capacity for service provision? Does the goal of competition under the BV-PIPS enhance local service delivery capacity for EBIs or will it facilitate the outsourcing of public services to private and for-profit entities lacking a foundation in the communities to be served? What are the long-term implications of pressuring CBOs into risk-bearing contractual arrangements and the effects of these arrangements on sustainment of EBIs? To what degree are public agencies prepared to relinquish control over contract-related decisions to market-based forces and how will this affect their own public-management capacity (Kettl, 1993; Van Slyke, 2003)? Finally, research must examine the short- and long-term consequences of the BV-PIPS for service recipients.