Child Welfare Workers’ Adoption of Decision Support Technology

Abstract

Child welfare workers must process complex information in deciding to refer clients to appropriate mental health services. Decision support systems (DSS) have been demonstrated in other fields to be an important tool, yet little research has been done in child welfare. This study focused on the adoption of a specific DSS into child welfare practice. Quantitative analysis was used to demonstrate the diffusion of innovation process among a sample of state child welfare workers, while qualitative analysis was used to explain the facilitators and barriers to DSS adoption. Results indicate that for DSSs to be widely adopted in child welfare practice, they should be integrated into the referral system and include workers’ knowledge and experiences with referral resources. For successful adoption, DSSs need to respect the natural logic and flow of worker interaction, as well as organizational constraints.

Child welfare workers must process complex information in deciding whether to provide services or to refer clients to appropriate mental health services. Many frontline workers are ill-trained to make adequate mental health assessments and therefore they either fail to identify mental health and substance abuse concerns or they refer clients to services that are inappropriate for their particular needs (McDonald & Marks, 1991). Yet such workers often serve as the “gateway” to mental health services (Stiffman, et al., 2000; Stiffman, Pescosolido, & Cabassa, 2004), as do physicians, counselors, teachers, etc. Decision support systems have been shown to assist physicians in processing varied and complex bits of information (Chen et al., 2004; Kawamoto, Houlihan, Balas, & Lobach, 2005; Pluye & Grad, 2004). These systems might also help child welfare workers as they navigate practice and referral decisions requiring the integration of complex information.

In essence, child welfare workers are important providers of mental health services. Decision support technology has been demonstrated in other fields to be an important tool for evidence-based practice, and the few extant studies about social workers’ use of these systems indicate they have the potential to change the way services are delivered (Giffords, 2003; Monnickendam, 2000; Monnickendam, Savaya, & Waysman, 2004, 2005; Nurius & Hudson, 1989; Savaya, Monnickendam, & Waysman, 2000; Schuerman & Vogel, 1986; Schwab, Bruce, & McRoy, 1986; Stiffman, Foster, Hamburg, & Doré, 2004). Developing technology that assists child welfare workers to better meet client needs is insufficient without a thorough understanding of the factors impacting the integration of technology into practice. It is therefore imperative that we gain a better understanding of the adoption process of DSS in child welfare practice. This paper examines the process of and reaction by child welfare workers’ to the adoption of a new decision support system specifically designed to enhance their assessment and referral capacity.

Background

Decision Support Systems in Child Welfare

Processing large amounts of assessment and referral information when working with clients can be overwhelming for professionals who work with children and adolescents (Giffords, 2003). Decision support systems (DSS) are designed to mitigate the complexities associated with the assessment and referral process and to overcome the limitations of human cognition. Computers can organize and store large amounts of data exceeding human knowledge capacities (Pluye & Grad, 2004), rapidly integrate and analyze that information, and assist in systematic recall.

Much of the research to date on decision support systems for clinical settings has been done in medicine. These studies have shown DSSs improve practice, reduce errors, and improve adherence to evidence-based practice methods (Chen et al., 2004; Kawamoto et al., 2005). Other studies suggest that DSSs enhance practice by giving clinical information at the point of care (Fischer, Steward, Mehta, Wax, & Lapinsky, 2003). Recent qualitative studies in social work suggest that DSSs are more useful in atypical cases where workers have less formal knowledge of the presenting issues (Monnickendam et al., 2005).

The introduction of decision support systems into a workplace represents, especially for clinicians, a seismic shift away from traditional decision making methods (Cantillon, Irish, & Sales, 2004). Child welfare workers usually rely on firsthand experience and consultation with colleagues for referrals (Schwab et al., 1986). Therefore, especially for them, DSSs represent new ways of performing daily tasks.

Diffusion of Innovation

Examining the facilitators and barriers to DSS adoption among child welfare workers is essentially examining the diffusion of the technology. Diffusion is the process whereby a particular population adopts an innovation over time (Rogers, 1995; Sterman, 2000). This process begins when the innovation is introduced and continues until the stock of potential adopters is depleted through saturation and contact with other adopters (Sterman, 2000). The Diffusion of Innovation (DOI) model assumes that adopters will continue using the innovation and not become discontinued adopters. However it is possible that use will discontinue, so theoretical models should be proposed that account for movement between the stocks of potential adopters, adopters, and discontinued adopters for an adequate conception of the dynamic problem. As such, the SIR (Susceptible – Infected – Recovered) model from epidemiology can be used to modify the DOI model (Sterman, 2000) to account for attrition among adopters (Potential Adopters – Adopters – Discontinued Adopters). That is, while the DOI model assumes only movement between the potential to adopt and actual adoption of the innovation, the model proposed here considers that adopters may forever discontinue use and therefore never move back to the stock of potential adopters. The PAD model recognizes that once a user adopts the innovation, that user gains certain knowledge and formulates assumptions about the innovation that may cause her to discontinue use and makes it unlikely that she will adopt it in the future. In this regard, such users never again become “potential” users; the PAD theoretical model better accounts for this disaggregation of nonusers.

Facilitators and Barriers

Studies have noted particular barriers to the adoption of innovations generally and technology specifically. In their systematic review of 495 sources on diffusion of innovations among UK service organizations, Greenhalgh et al. (2004) found that innovations were less likely to be adopted if potential users saw no relative advantage to their use, and if the innovation carried a high degree of outcome uncertainty. Other studies identified barriers such as: an absence of added value (Damodaran & Olphert 2000); time limitations; the effort necessary to learn a new technology (Chen et al., 2004; Mukoyama, 2004); and low comfort with technology (McAlearney, Schweikhart, & Medow, 2004; Mukoyama, 2004). Research into knowledge management systems, which are similar to the DSS this article examines, has shown that barriers include lack of user friendliness (the corollary to Greenhalgh et al.’s and Mukoyama’s findings); slow system response time; inadequate user support and training; and cultural, policy and practice issues (Damodaran & Olphert, 2000). In addition, barriers to the adoption of decision support systems also include difficulty finding information; encountering outdated information (Chen et al., 2004); and a preference for paper over information on computer screens (McAlearney et al., 2004). Screen size and the awkwardness of carrying additional equipment are barriers specific to handheld computers (Beasley, 2002; Fischer et al., 2003; McAlearney et al., 2004).

As Rogers (1995) posits, individual and organization aspects of the innovation facilitate the adoption process. Generally, innovations in service organizations are successfully adopted when they are clearly effective; relevant to the user’s work; improve the performance of the intended task; can be adapted or modified to better meet users’ needs; and are easy to use (Damodaran & Olphert, 2000; Fagan, Neill, & Wooldridge, 2008; Greenhalgh et al., 2004). On an individual level, facilitators include: innovation’s congruence with the intended adopter’s values; perceived needs (Greenhalgh et al., 2004); higher skill level; and education level (Mukoyama, 2004). Organizationally, compatibility with professional norms and ways of working facilitate adoption of innovations (Greenhalgh et al., 2004). With respect to DSS specifically, facilitators include: technology interface; perceived impact; worker’s attitude toward the application (Chakraborty, Hu, & Cui, 2008; Fagan et al., 2008; Savaya et al., 2000); and user preparedness (Monnickendam et al., 2004; Mukoyama, 2004). Primarily utilizing qualitative methods, this paper seeks to enhance our understanding of the facilitators and barriers to child welfare workers’ adoption of decision support systems.

Methods

The IMPROVE (Intervention for Multisector Provider Enhancement) system, funded through the National Institute on Mental Health, is a technology-based decision support intervention designed to increase child welfare workers’ identification of mental health and substance abuse problems, to assist in making appropriate referrals, and to improve the provision of mental health care through appropriate referral resources. The intervention is housed in both handheld (PDA) and personal computer (PC) formats. IMPROVE provided names and details about referral sources via an assessment, keyword search, or alphanumeric search of the referral database (see Appendix). The system development team worked closely with the United Way of Greater St. Louis, the Missouri Children’s Division, and local foundations to develop a comprehensive database of over 2,100 agencies and programs (mental health clinics; drug treatment; social welfare programs for housing, clothing and food; and alternative programs in sports, the arts, and skill building). Referral information was vetted, but the development team did not directly evaluate the quality of service as a result of using IMPROVE. Users could provide resource updates directly to the development team via the application’s comment feature.

IMPROVE’s assessment checklists for depression, conduct disorder, and attention deficit hyperactivity disorder were based on the highest loading Child Behavior Checklist (CBCL) items indicative of meeting Diagnostic and Statistical Manual of Mental Disorders (DSM) criteria for each of these disorders (Achenbach, 1991; Achenbach, Dumenci, & Rescorla, 2001). The assessment checklist of substance abuse problems was based on items found to be associated with risk of substance abuse problems in the absence of direct knowledge of the abuse (Oetting & Beauvais, 1983a, 1983b; Swaim, Oetting, Edwards, & Beauvais, 1989; U.S. Department of Health and Human Services, 1999). The selection of three or more checklist items in an assessment triggered a referral list directly related to observed behaviors and/or problems.

IMPROVE development occurred over four years and included earlier beta and pilot testing that revealed necessary changes to the user interface and suggestions for tools and functions that would enhance the usability and incorporation into usual work practices (e.g., storing a personalized list of “favorites”).

Design

The IMPROVE study, utilizing the revised application, employed a mixed quantitative-qualitative method in a delayed panel design. Three panels (see Bernard, 2006) entered the study over six weeks to accommodate worker schedules and to reduce the trainer to participant ratio. Participants were randomized into three groups: one having the application on a handheld computer (e.g., Palm™ Pilot) and their office desktop computer (n = 10), one group having it only on their desktop computer (n = 10), and one group conducting assessments and referrals under usual conditions (n = 8). Usual conditions consisted of utilizing personally developed lists and loose-leaf binders, and consulting coworkers.

Training on IMPROVE included both seminar and in vivo sessions. Handheld and PC groups were initially trained as their panel entered the study and were given identical training sessions in a university instructional computer lab. Training was didactic and hands on, lasting one hour for the computer-based version and an additional hour for the handheld version. In vivo training occurred in conjunction with shadowing, i.e., providing users assistance in actual field situations.

The quantitative aspect of the study included three data collection points (pre, 6-, and 12-month) to gather self-reported data about worker practices and their actions with clients, and automatically generated data from IMPROVE use. The qualitative components of this study consisted of shadowing each worker (participant observation) in the handheld and PC groups as part of the in vivo training to assess technology adoption, and technology assessment focus groups at 6 and 12 months. Focus groups were voice recorded and transcribed for analysis; participant observations relied on observer field notes.

Sample

All participants were child welfare workers in the St. Louis City or County Children’s Division, Missouri Department of Social Services. Inclusion criteria were: (1) behavioral and/or mental health assessments and action as part of their regular job duties; (2) an active caseload of 15 adolescents; and (3) similar job descriptions to other workers in the study (to control for other factors which might differentially impact application usage across job types). All workers who met the criteria were invited to participate and self-select into the study. Participants (N = 28) were randomly assigned to the study groups. The development team obtained written consent from each participant and the university’s institutional review board approved all study protocols. Eight participants withdrew or were removed from the study for non-compliance (i.e., not completing surveys or responding to inquiries).

Of the 28 participants who began the study, four were male and 24 were female; 85 percent held a bachelor level degree (BA, BS, or BSW), nearly 8 percent held a master of social work degree, and nearly 8 percent held another type of master degree. The mean number of years worked at the division was 3.19 years (SD = 2.14) while the mode was 2.0 years.

A majority (92%) of the participants owned a personal computer and 96 percent reported using a computer daily. Eighty-three percent of the participants reported having internet access at home and using it daily at either work or home. Four participants owned a personal digital assistant (PDA) similar to the device housing the intervention, but only one individual was randomized into the intervention group using IMPROVE on the PDA.

Instruments

This paper concentrates on the results from the qualitative data obtained concerning facilitators and barriers to use. Focus groups utilized a guided questioning route and the participant observation utilized a semi-structured interview guide (Bernard, 2006). Key topics included the participant’s use of IMPROVE and the handheld device, IMPROVE problems, participant reaction to IMPROVE, reactions to overall usage patterns, organizational supports and constraints, and suggestions for changes.

Data Collection

Social work students (three doctoral and two master’s level) trained in qualitative data collection conducted the participant observations either in the workers’ offices or while shadowing workers in clients’ homes. Separate focus groups for handheld users, PC users, and the usual care group were conducted simultaneously. Two development team members moderated each group.

Nineteen participant observations and the data from the six- and 12-month focus groups are included in the analysis for this article. Nine of the participant observations are from the handheld group and 10 are from the PC group. Participants in the six-month focus groups included seven from the handheld group, six from the PC group, and six from the usual care group. The distribution for the 12-month focus groups was: five handheld; six PC; and five usual care.

Two development team members independently coded the observation notes and transcripts using NVivo 7.0 (QSR, 2006). The development team derived a codebook of themes from the literature to guide the qualitative coding and the coders also developed nodes in vivo using a grounded theory approach (Strauss & Corbin, 1998). Ambiguities or coding discrepancies were resolved by reviewing the original transcripts and group consensus.

Results

Key Findings: Before the Intervention

Prior to the introduction of the decision support technology, workers in this study employed several strategies to match clients with services. The primary strategy relied on personally developed lists of providers assembled in resource binders. “You got to have a resource book. You pray for that book.” Providers in such resource binders are those with whom the workers have established relationships and referred to in the past. Workers cite tending “to stick with the ones I know” and going “back to the ones I’ve definitely used before.”

Workers also cited close collaboration with coworkers as an important strategy for connecting clients to services. “You have to learn from your coworkers. You really have to rely on your coworkers to show you how to do things.” For workers in this study, “word of mouth does seem to be the way everything ends up happening, most of the time.”

Other referral strategies included internet searches and consulting specialists within the organization. Workers cited using the referral information provided through the United Way of Greater St. Louis website and another through the University of Missouri. They indicated using the internet to search particular words and “seeing what you come up with.” The Division also designates certain workers to be specialists in particular service areas. “If we need something we’re not familiar with, we can go to [the specialist] and they’ll look into different resources for us.”

Key Findings: After the Intervention

Quantitative Results

The automatically generated usage data reveal that users would start and stop IMPROVE without performing any additional actions; such uses are considered false starts and not included in the analysis below. Uses counted here are those where the user started IMPROVE, performed at least one action (search, assessment, view favorites, etc.), and stopped IMPROVE. This process is counted as one use regardless of the number of actions taken between starting and stopping the program.

The handheld group had access to both the handheld and desktop versions; usage was counted separately for each version and is reported separately here for the entire 13- month study period. The range of application uses on the handheld device was 0 to 35 while the mean was 14 uses. The range of use on the desktop version of the application for the handheld group was 0 to 87 while the mean was 16 uses. Participants in the handheld group used the desktop version slightly more than the handheld version.

The PC group only had access to the desktop version of IMPROVE. Use among the PC group was similar to the desktop use among the handheld group. Individual use ranged from 2 to 42 and the mean number of uses was 16.

The pattern of IMPROVE use over the 13-month study period demonstrates the movement from potential adopters to adopters to discontinued adopters. Here the mean usage by intervention group and month of the study period are presented (Figure 1). Data for the handheld group are disaggregated by application type. Use of the handheld spiked in month two with 14.6 mean uses and dropped precipitously to three mean uses by month four and one by month five. Much of the initial spike can be attributed to installing the application in workers’ offices and workers exploring the application. Mean application use on the desktop by the handheld group shows a gradual increase until month seven (1.0 – 5.5 uses), a spike in use at month nine to 10 uses, then usage remains relatively stable until the end of the study period with a range of mean use between 3.0 and 4.5.

Figure 1.

Mean application usage by month and application type. Months 1 and 2 reflect the startup period. Thirteen months are included to capture 12 months of usage across all three panels.

Mean application use in the PC group peaked in month three with 5.3 mean uses and again in month 11 with 4.5 mean uses. Mean use in this group dropped to 1.5 uses by the end of the study. Use for all groups declined after month 10, which is likely due to knowledge the study was coming to an end and IMPROVE would no longer be available. Having clients who became connected to services during the study period might also account for the drop in IMPROVE usage, though this is speculative.

Qualitative Results

As Figure 2 indicates, workers did use the decision support technology as a referral strategy. We asked workers about situations where they used the technology, when it was most beneficial, and about the pros and cons of utilizing it.

Figure 2.

Referral Pathways Child Welfare Workers Use to Move Clients from Assessment to Referral

When was it used?

Child welfare workers in this study used the application primarily for accessing referral information through agency and/or keyword searches. When asked when and why they would use it, workers cited instances when a client’s presenting needs could not be met through their usual lists of providers or consulting coworkers. In these unfamiliar cases, workers used IMPROVE to access referral information. “If it was something obscure or something that I didn’t deal with on a regular basis, then I would consult [IMPROVE].” Another worker noted how IMPROVE was helpful with a special needs child: “I mainly used it when I had the child that was deaf…I tried to use it a lot when I had her.” Such unfamiliar cases also included referring to non-clinical resources such as food pantries, utility assistance, clothing resources, and housing.

Though only one new worker was among the participants in this study, her usage suggests that new workers might be more likely to adopt decision support technology. We could not track her use of IMPROVE over the full 13 months because a position change excluded her from further participation; however, her usage during the 4 weeks of the study outpaced most other users in the handheld and PC groups (total of 35 uses on the handheld and 4 on the desktop). At the initial shadowing she was using the assessment feature as the primary mechanism to make referral matches. This worker did not have a “book” of known providers from which to draw; therefore, she relied on the technology to provide referral resources.

When was it not used?

The decision support technology was not used when workers could quickly and easily rely on referral strategies developed prior to the intervention. Reliance on these non-technology strategies was particularly strong when a worker faced significant time limitations and when colleagues could provide the needed information.

Workers in this study reported a preference for relying on prior experience with providers when making referral decisions. “Usually, if I need a therapist there’re four or five that I try first – a bunch of people I used a lot and I have almost all their phone numbers memorized.” Another noted that, “All the resources that the clients were asking for were things that I already knew, or I had my own personal resources.” Instructive are the comments, “I really didn’t use [IMPROVE] to be honest with you. I tend to use the same [providers] every time because in the past they’ve worked.”

Consulting with coworkers for referral suggestions is common among the child welfare workers in this study and emerged as a significant theme for nonuse of IMPROVE. The following exchange demonstrates the extent to which workers rely on each other. (R indicates study participant and M indicates the focus group moderator. Respondent number allows a single participant’s comments to be followed through the discussion.)

R3: If we were to have a foster kid that was placed up in North County, then we may email the North County office saying do you know of a therapist that would work well with this type of child. I don’t work in North County so I don’t know what resources they use in North County.

M2: So you email somebody?

R3: Because I’m down South. So they would know who a therapist was up there because I wouldn’t.

R5: And within the city, we do get a lot of messages like that. I have a family that I am looking for …Do you have any resources? And then if you know something then you can shoot an email back. [IMPROVE] could be used a little more but within the building it’s just easier to go walk across the aisle.

R6: And you pretty much just stand up and ask a question and everybody will answer you.

R1: Everybody will answer.

R5: Yeah. I was just going to say sometimes you’ll be on the phone and you’re not really asking anybody else but they’ll hear you. Kind of in your conversation, say, Oh but you should call… So it’s pretty informal within offices of how that information gets shared.

Workers prefer to rely on each other’s knowledge because they can trust the information and know what to expect from the provider. “I usually use the same agencies because I can depend on them to follow up with me, I know the quality of service they provide, and they complete reports on time.”

How was it used?

It is important to place the use of decision support technology into the larger context of child welfare workers’ referral strategies. Our data show that when IMPROVE was introduced, it became one of three main referral strategies workers use to match their clients with needed resources: 1) use known providers; 2) consult coworkers; and 3) use technology (Figure 2). Workers discussed choosing a referral strategy based on the familiarity with the client’s presenting resource need. If the need is familiar, workers rely on previously developed resource lists. To address unfamiliar resource needs – either outside the workers’ usual geographic service area or beyond the capacity of personal resource lists – workers either consult coworkers or utilize the decision support technology.

Why was it used?

Using technology was a third level strategy employed after other methods failed. Data reveal that IMPROVE was beneficial when client needs extended beyond a worker’s resource or knowledge base. Workers in the handheld group report using the application when they needed “on the spot” information, particularly when in court or in the field and faced with an imminent client need (e.g., eviction).

Why was it not used?

Workers identified a number of reasons for not using IMPROVE: clients are already “hooked up” with services, personally developed referral lists and consulting coworkers are reliable methods to connect clients with resources, time constraints prevented them from gaining increased comfort with the keywords, and lengthy results lists. Workers also describe their use/nonuse of IMPROVE as attributable to “being set in my ways.” A worker said, “You know some people who have been doing this job for 20 years and have a system that works for them, they’d be like yeah, but no, I’m not learning something new.” As such, these established referral strategies governed the adoption of this decision support technology.

Reactions to Specific Aspects of the Technology

Facilitators and barriers to technology adoption specific to the technology also arose. While some are a result of the workers’ practice and environment, others were related solely to the technology.

Ease of Use

When workers used the application, they found that its structure was quick and easy to use. In this regard, the architecture of the application acted as a facilitator to technology adoption. “It’s pretty simple. There’s not a whole lot of complicated directions, you know, learning curve or anything. It is not too many buttons to hit.”

Workers also noted how the system’s architecture could streamline their work and connect clients to resources more quickly. “I didn’t have to look up a whole bunch of stuff. If the person needed something for food, I didn’t have to look at a sheet on shelter. I could just go straight to the food and get what I needed for that particular client.” Another worker noted, “I think it’s helpful that there’re multiple ways to do searches. If you don’t know the name of an agency you can just do by subject or by service that you need…[or] you could do an alphabetical [search] and look it up really quickly too.”

Assessment

“I really didn’t use the assessment tool that often.” was a common response from many workers in the study. As one worker commented, “I thought that was just a little too cookie cutter. All the kids would have the same diagnosis. I didn’t want to label the kids.”

Keywords

Workers said the keywords were confusing, thereby acting as a barrier to IMPROVE adoption. One worker who tried to use IMPROVE but found it easier to fall back on her own resource lists noted, “What boxes you check give you different answers and I think that may have been a lot of my reluctance to keep on trying. I’m like, ‘Okay, did I search it right by this set of words or…Oh, forget it. Just go get your own packet. You know where that is.’” Another worked said, “My problem was you have to have a tailored specific search when you go in and try to narrow your searches, which is why I say it’s more complex. It was difficult to zero in on those specific entities that [provided the services I needed].”

Referral Lists

The lengthy referral lists acted as a barrier to technology adoption, so much so that some workers cited this as a reason to stop using IMPROVE. Workers found it difficult to craft keyword combinations that resulted in a referral list they considered targeted and manageable.

R3: Did anyone ever have the problem where you put in some key words and you get three hundred responses?

R1: Yeah.

R4: I did that in the beginning. That’s kind of why I quit.

R3: Yeah. You really had to get used to what the most restrictive words were to narrow it down, because if you just used two or three pretty general words, everybody who’s ever thought about providing a service came up.

One worker commented that even upon refining her search, the system returned 266 referral options.

Portability

Workers in the handheld group cited the portability of the PDA as an important facilitator of their use. “Whenever the client asked for information and you can access that information and be able to give them all these different places that they can go for resources.” Discussing the factors that encouraged one worker to use the application, she said, “I thought that was good carrying it around in my purse, the fact that it’s mobile. You can carry it from place to place.”

Discussion

The implications of the research presented from this current study focus on two areas: (1) facilitators and barriers to IMPROVE adoption among child welfare workers; and (2) the uptake of decision support technology generally in child welfare practice.

Facilitators and Barriers to Adoption

Child welfare workers experience similar barriers to technology adoption as do physicians and other social service workers. However, facilitators to the adoption of decision support technology found in prior studies did not hold in this study of child welfare workers. Prior studies found that workers’ favorable attitude toward technology has been shown to facilitate adoption (Chakraborty, Hu, & Cui, 2008; Fagan et al., 2008; Savaya et al., 2000). However, workers in this study reported favorably toward IMPROVE but did not widely adopt it. Many other organizational factors impact the assessment-referral process that when not accounted for in the design and implementation of DSSs inhibit the overall uptake into practice.

Similarly, user preparedness (i.e., training) was not necessarily a facilitator to technology adoption as other studies have found (Damodaran & Olphert, 2000; Monnickendam et al., 2004; Mukoyama, 2004). Once assessment-referral patterns are established, it becomes difficult to change them despite the relative acceptance of the idea that DSSs are an important tool. Training alone was not sufficient to encourage adoption of this technology.

User involvement in the development of decision support systems has been shown to be important for user satisfaction and successful implementation (Fitch, 2007; Monnickendam, 1999). Given the variance between our findings and those of prior studies, one might assume a mismatch between user needs, technology interface, and the functionality of this DSS. The IMPROVE development team engaged child welfare workers in each phase of the design of the interface, the features, and keyword lists, thus reducing concerns about a mismatch. This is not to discount the problems workers experienced with the technology; however, it is to argue that IMPROVE was congruent with user needs and expectations.

For workers in this study, comfort with technology was not related to the adoption of IMPROVE. Where others have argued that low comfort with technology is a barrier (McAlearney et al., 2004; Mukoyama, 2004), high levels of daily computer usage among these workers (96%) suggests relatively high comfort with technology. Exogenous factors (e.g., state contracts, court orders, familiarity with resource need) in the assessment-referral process nullify any effect that prior comfort with technology might have on the overall adoption of the DSS by child welfare workers.

As the Diffusion of Innovation model posits, workers adopted IMPROVE upon receiving training (Figure 1). However, over time, many workers moved from being users to being potential or discontinued users despite the perceived benefit of IMPROVE to child welfare practice. Those workers who indicated IMPROVE would be useful for new cases fell back into the stock of potential users, while those who indicated it was easier to rely on their personally developed lists or to consult coworkers fell into the stock of non-users (Rogers, 1995; Sterman, 2000). As noted earlier, some workers are reluctant to change established work patterns that have proven most efficient and effective in the past.

Decision Support Technology and Child Welfare Practice

The system in which workers make referral decisions is complex. Making a referral is not a simple matter of assessing need and finding an appropriate provider; it is a balancing act between competing demands, only one of which is client need. Child welfare workers may not adopt decision support technology into their practice because the transactions costs associated with such new practices might outweigh the perceived benefits (Tyagi, 2004; Williamson, 1981). Judges, supervisors, and court appointed advocates have preferred providers. State contracts mandate usage of certain providers when public funds are used to provide care. Workers have established relationships with providers who respond promptly and submit timely reports. Certain costs (e.g., time, reprimand for using a non-court approved provider) may result from using providers not already familiar to and within the system. These transaction costs act as a barrier to technology adoption.

Workers suggested that incorporating their comments, knowledge and experiences into the agency descriptions and making that knowledge available to other users would facilitate IMPROVE’s use (see Fitch, 2006). Having the knowledge of coworkers embedded in the application would, in their words, “be real helpful” and make it more useful as a first order intervention tool. A successful system will not only consider the myriad of variables Monnickendam (1999) found in his systematic review, but also have the capacity to incorporate worker knowledge into the system (see Schoech, Fitch, MacFadden, & Schkade, 2001). With DSSs that “get smarter” with use, workers can learn from one another, have knowledge available as part of the referral process, and knowledge will not be lost in the high turnover environment of child welfare.

Limitations

These findings should be approached with some caution. Our data are limited by the small number of participants who come from a particular child welfare setting. We experienced 29% (n = 8) attrition over the study period and not all workers participated in all focus groups. It is therefore difficult to generalize these findings broadly to child welfare workers, to DSSs with other goals, and the adoption of other technology in child welfare settings.

Significant changes occurred in the Missouri Department of Social Services between the pilot and implementation studies. The Division added a resource person at the family courts whose sole responsibility is to provide immediate referrals to families after adjudication and the state began contracting out much of the child welfare services to private organizations. This left workers with caseloads of youth already receiving services while many new cases were contracted out.

The usage data also presents a slightly distorted picture of overall usage patterns. We experienced technical problems obtaining usage data from some users which skew usage numbers downward. The application and/or handheld device crashed for at least two users (workers did not always inform the development team of technology problems) which resulted in a loss of all usage data not yet synchronized with the university’s computer.

The quantitative surveys in the IMPROVE study did not examine technology satisfaction but focused on the referral activity of the child welfare workers. This is a significant study limitation with respect to assessing the technology itself. Nevertheless, the data presented here provide new insights into the technology adoption process among child welfare workers and provide directions for future research.

This study also raises areas ripe for inquiry. Future studies should examine the differences in adoption and use between new hires and employees of longer tenure, incorporate other’s knowledge into the application to examine its effects on technology use, and examine quantitatively the technology itself.

Conclusion & Implications

Child welfare workers develop specific assessment and referral patterns that become embedded and normative across workers. These established work patterns are difficult to change, particularly when workers are asked to reconceptualize a central job function –providing referrals. This study found that workers were reluctant to use IMPROVE as a first order tool and instead primarily turned to it when other methods failed. Worker comments suggest that for decision support technology to be adopted widely into child welfare practice, it should be fully integrated into the system-wide referral process, get smarter with use, and connect clients more easily with approved service providers. Technology like IMPROVE should consider the complex referral environment and account for the limitations imposed within that environment so that workers may use the DSS with full confidence that the courts and supervisors will approve the provider, that the state will pay for the provider’s service, that the provider has a positive track record with other workers, and that the provider is a good match for the client’s needs. Our study found that individual, organizational, and technological characteristics act together to facilitate or limit the adoption of technology. The mere introduction of decision support technology into child welfare agencies will not change the ways workers connect clients to needed resources.

Appendix

The main screen used to navigate to all features.

Assessments use checklist items and can only be conducted as part of a youth profile; age and gender are used to narrow the results.

The assessment summary indicates the assessment sections triggered and the accompanying keywords. Referral resources will be searched via the “Problem Keywords Indicated.”

Keywords are selected from lists via specific categories. All referral resources are searched via keywords. Each resource has one or more affiliated keyword.

Referral resources are listed with zip code and fee information; the list may be sorted via zip code and fee. Users highlight a resource select “agency detail” to view programmatic details.

Agency details include: program name; available services; requirements; areas served; ages served; and contact information.