Commentary: Social Experimentation at Its Best: The Cash and Counseling Demonstration and Its Implications

The Cash and Counseling demonstration was the most significant long-term care policy experiment undertaken in more than a decade. It is an archetypal example of the ability of research to play an important role in public policy (Kemper 2003). It has altered thinking about the effects of consumer-directed care and led a number of states to incorporate consumer-directed care in their home care programs (Mahoney et al. 2007). The funders who conceived and paid for it, the national program office that ran it, the states that were willing to be laboratories to test it, and the research team that evaluated it all should be commended for their achievement. This paper assesses the demonstration's strengths and weaknesses and explores its implications for social experimentation and policy.

CASH AND COUNSELING

Having a long-term disability is a risk that all people face and is largely outside their control. Current policy treats long-term care like acute care: Medicaid insures specific services provided by agencies to assist in living with disability. Thus, current policy recognizes that long-term living with disability requires support. However, current policy fails to recognize the many ways of living with disability and the importance of personal preference in choosing among them.

An alternative is to model long-term care programs after disability insurance rather than acute care insurance. Social Security Disability Insurance provides income to people who cannot work due to disability. This model could be extended to long-term care by paying people with disability a regular cash benefit that depends on the level of disability. Such a policy would permit consumers to choose to live with disability in different ways according to their individual preferences.

Interest in cash disability programs has grown in recent decades as it has been adopted abroad. It became a matter of national policy discussion when it was included in President Clinton's health reform proposal (Wiener et al. 2001). However, policy makers have hesitated to adopt cash disability programs out of concern that consumers would use the cash for things other than needed assistance, that paying family members would substitute for care that they would provide without pay, and that the attraction of cash benefit would increase program participation and Medicaid expenditures.

Despite its name, Cash and Counseling was not a cash disability program. It was a consumer-directed care program providing a cash-like benefit within the constraints of Medicaid's service delivery model. It was conceived as a “paradigm shift in the delivery of long term care …” that was “primarily intended to give Medicaid beneficiaries … the same degree of choice and control over how to best meet their needs … as private payers …” (Doty, Mahoney, and Simon-Rusinowitz 2007). Federal officials also were interested in its potential “to provide a less costly approach to delivering Medicaid waiver and personal care services” and “help expand the labor pool (e.g., by allowing consumers to pay family and friends …)” according to Knickman and Stone (2007).

A MODEL SOCIAL EXPERIMENT

Several features of the demonstration and its evaluation contributed to its success.

Significant Policy Issue Calling for Evidence

Social experiments are costly and take a long time to yield results. Justifying an investment in them requires an important debate that depends on unanswered questions about the effects of a policy change. Cash and Counseling was a response to such a need for evidence.

Rigorous Experimental Design

Random assignment of participants to either Cash and Counseling or traditional agency services ensured the internal validity of the results. More generally, in designing the evaluation Brown and Dale (2007) considered potential threats to validity, addressing those they could and qualifying findings when they could not.

Focus on the Full Range of Effects

The evaluation was designed to estimate a wide range of effects on consumers, family caregivers, workers, service use, and expenditures. As a result, the study was able to observe unintended as well as intended effects.

Emphasis on Implementation

Intentional use of qualitative and quantitative information to understand the program's implementation and identify ways of improving the program was a great strength of the evaluation. The papers by Phillips and Schneider (2007), Schore, Foster, and Phillips (2007), and San Antonio et al. (2007) show the value of qualitative research in doing so. The evaluation also used quantitative data effectively to identify lessons for program implementation. For example, Dale and Brown's (2007) analysis of care plan cost estimates (which determined allowances) identified increasingly generous allowances the longer consumers' were enrolled. This led them to recommend operational changes to control care plan costs. The evaluation's emphasis on implementation led to many operational lessons for states adopting consumer-directed care.

Early Dissemination of Results

Given the time experiments take, policy makers sometimes act without waiting for results or lose interest in the policy before they are available. Publications early in the demonstration described early implementation experience (e.g., Mahoney, Simone, and Simon-Rusinowitz 2000) and reported early impact results even though not all the evidence was in (e.g., Foster et al. 2003). Not only did this provide useful information about consumer-directed care, but it also maintained interest in the demonstration and the policy it tested.

DEMONSTRATION EFFECTS AND THEIR DISTRIBUTION

Cash and Counseling was intended to shift to more flexible service delivery. It was not intended to increase access to services (except in Arkansas). Although consumers could hire family members, it was not intended to pay them for care they were already providing. As it turned out, the demonstration did all three, at least in part. It shifted the mode of service delivery, increased access to paid services, and paid family members for some care they would have provided without pay.

Assessing Cash and Counseling's implications for policy and politics requires understanding its effects and their distribution among stakeholders. Table 1 summarizes the effects and how they affect consumers, family caregivers, other directly hired workers, agency workers, and taxpayers. Asterisks indicate effects for which the evidence is strong; those without an asterisk are supported by some but not all the evidence.

Table 1.

Overview of Cash and Counseling's Effects and Their Distribution among Stakeholders

	Consumers	Family Members	Nonfamily Workers	Agency Workers	Medicaid Taxpayers
Personal assistance
Paid hours	+*	+*	+	−
Unpaid hours	−*	−*
Total hours	−	−	+	−
Consumer benefits
Met needs	+*
Satisfaction with care	+*
Quality of care†	+
Satisfaction with life	+*
Family caregiver benefits
Satisfaction with care		+*
Confidence in care		+*
Satisfaction with life		+*
Health		+*
Expenditures‡
Personal assistance		+*	+*	−*	+*
Nonpersonal assistance					−
Total					+*

Source: Author's interpretation of findings in Carlson et al. (2007), Dale and Brown (2007), and Foster, Dale, and Brown (2007).

^*The evidence overall indicates that the demonstration increased (+) or decreased (−) this outcome; absence of an * indicates that some, but not all, evidence suggests that the demonstration has this effect.

^†Quality as indicated by absence of adverse outcomes.

^‡Medicaid expenditures per enrollee during the first 2 years after enrollment.

Improved Access from Expanding the Workforce

Cash and Counseling was in part a workforce intervention. It increased the supply of workers by paying family members—53–78 percent of workers hired directly by consumers were relatives (Foster, Dale, and Brown 2007)—and other directly hired workers. The additional supply appears to have alleviated a shortage of agency workers and hence access to paid care, particularly in Arkansas. Consequently, more consumers received paid services, and those receiving them received more paid services per month (Dale and Brown 2007). Except for a modest increase, the hours of paid care that consumers received under Cash and Counseling replaced hours that would have been provided by agency workers (Carlson et al. 2007). Consequently, agency workers' job opportunities were reduced. The impact on the market for agency workers was small, however, because Cash and Counseling participants were only a small share of all participants in Medicaid home care programs.

Improved Access from Increasing Participation

The demonstration's cash-like benefit attracted some consumers who would not have participated in the traditional agency program. As evidence of this, Dale and Brown (2007) point to the high proportion of new enrollees in Arkansas who had not been in the traditional agency program and were assigned to the control group who then did not receive agency services. The magnitude of program-induced participation (sometimes called a “woodwork effect”) is difficult to judge. Whatever its magnitude, however, it occurred despite the federal Office of Management and Budget's (OMB's) limitations on participation imposed with the intent of preventing program-induced participation. They included requiring previous enrollment in agency services (except in Arkansas) and limiting Cash and Counseling's share of Medicaid home care enrollees (Doty, Mahoney, and Simon-Rusinowitz 2007).

Improved Consumer Outcomes

By expanding access to paid care and increasing flexibility in care arrangements, Cash and Counseling clearly benefited consumers. They reported greater satisfaction with their care and their lives, and quality of care (as indicated by fewer adverse outcomes) was at least as good as agency care and may have been better. The extent to which these benefits were due to increased access versus greater flexibility cannot be determined. At least some of the benefits appear to be due to greater flexibility: more needs were met, not by providing more total hours of care but by changing the way assistance was provided during those hours and permitting spending on assistive equipment, home modifications, and other needs (Carlson et al. 2007).

Increased Income of Family Caregivers

Under Cash and Counseling, total hours of care, paid plus unpaid, remained the same or decreased slightly (Carlson et al. 2007). This implies that some of the paid hours of care provided under Cash and Counseling were hours of care that family members would have provided without pay. Thus, as a group, family caregivers benefited from additional income.

Improved Family Caregiver Outcomes

Conceived primarily as a service delivery paradigm shift for consumers, Cash and Counseling turned out to be a program for family caregivers as well. Not only were many family caregivers paid for caregiving, but primary informal caregivers (most of whom were family members) benefited in other ways. They were more satisfied with and confident in the care received, were more satisfied with their own lives, and reported better health than primary caregivers of consumers receiving agency care (Foster, Dale, and Brown 2007).

Increased Medicaid Expenditures

By increasing access to paid care, the demonstration increased home care expenditures. Partially offsetting this increase, Cash and Counseling reduced somewhat Medicaid expenditures on other services, particularly nursing home and home health care. However, total Medicaid expenditures were greater than they would have been under agency care: the median increase over the first 2 years after enrollment in Cash and Counseling was 8 percent, with a range of 4–14 percent across states and target groups (computed from Dale and Brown 2007). Even a modest increase like this will concern some taxpayers and others who oppose increased government spending.

A PERMANENT PROGRAM'S EFFECTS MIGHT DIFFER

The demonstration's policy implications depend partly on judgments about how a permanent consumer-directed care program's effects would differ from the time-limited demonstration's. Behavior of workers, consumers, or families may differ in the face of a permanent program, or states may alter the program design as a matter of policy.

Access: Expanded Workforce

A permanent program is likely to expand the size of the workforce slightly and increase access to care. The magnitude of the effect would be reduced, however, if public concern about quality or pressure from agency workers or their unions led to increased training or other requirements for directly hired workers.

Access: Increased Participation

The program-induced increase in participation in Medicaid home care programs is likely to be greater under a permanent program than under the demonstration. As more eligible consumers and their families learn about the program and its attractive benefits, more can be expected to enroll. Requirements that applicants first receive agency services can be expected to have less effect as people learn to circumvent the intent of the requirement by enrolling in agency care solely to become eligible for consumer-directed care. States have the option of limiting the number of consumer-directed care slots to limit increased participation, but doing so would reduce access to consumer-directed care.

Consumer and Family Outcomes

A permanent program would almost certainly shift service delivery, pay family members, and increase access, benefiting both consumers and their families.

Medicaid Expenditures

The effect of a permanent consumer-directed care program on Medicaid expenditures is uncertain. On the one hand, greater program-induced participation under a permanent program would increase expenditures more than under the demonstration. On the other hand, states can adopt program changes that have the potential to limit expenditures, including those recommended by Dale and Brown (2007). Importantly states have a powerful tool for controlling costs. They control the size of allowances by setting the discount from the estimated care plan cost. Thus, the effect of a permanent program on Medicaid expenditures depends partly on policy decisions.

Program Implementation and Operation

Running a permanent program is more difficult than running a demonstration. For example, staff enthusiasm wanes; many more people must be trained; program procedures must be routinized; and legal challenges limit flexibility. The counseling provided to consumers about hiring and supervising their own workers and the development of care plan cost estimates are particularly important program components that are likely to differ in a permanent program.

IMPLICATIONS FOR SOCIAL EXPERIMENTATION

Foundations and policy agencies should emulate the role that The Robert Wood Johnson Foundation and the Office of the Assistant Secretary for Planning and Evaluation played in Cash and Counseling. Key officials in both organizations understood the importance of the policy and were committed to obtaining rigorous evidence about its effects. They made a substantial investment in the experiment and had the patience to wait many years for the results. They chose expert researchers with integrity to conduct the evaluation and talented managers to run the demonstration. They did not waver in their commitment to randomization. And, they were willing to have unfavorable as well as favorable findings reported, maintaining the independence of the evaluators.

Playing such a role is a major challenge. Congressionally mandated studies are often underfunded; large investments in research can be difficult to sell within foundations and government agencies; and political pressure to report only results that are favorable to administration policy has increased in recent decades. It takes vision and fiber among foundation and government officials to meet these challenges.

What can be learned from a social experiment depends heavily on the design of the program being tested. In retrospect, a weakness of Cash and Counseling's design was its limited ability to provide information on the extent of program-induced participation in consumer-directed care. Addressing this issue would have required a program design without the restrictions on enrollment imposed by OMB. Ironically, its insistence on these restrictions limited what could be learned about the very effect on participation that OMB was concerned about. The implication for social experimentation is to take care in defining the program being tested to maximize what can be learned about the central issues in the policy debate.

IMPLICATIONS FOR POLICY

Many states are adopting some form of consumer-directed care. The implementation research reported in this issue demonstrates the value of monitoring the program as it is implemented. States should setup the data systems needed to monitor important cost and participation rates over time to identify problems in the program design and then change policy to address them.

Other states are still considering consumer-directed care options. The evaluation findings do not permit an unambiguous policy recommendation for these states. Instead, a policy judgment about consumer-directed care depends on weighing the clear benefits to consumers and their families against reduced job opportunities for agency workers and increased Medicaid expenditures. Weights will depend on the perspective taken. Consumers and their families are very likely to view consumer-directed care favorably; agency workers and those concerned about the Medicaid budget and taxes are likely to view it unfavorably. Many of those who take the broader perspective of society as a whole will conclude that the benefits of consumer-directed care justify its cost—that is, that the evident benefits to consumers and their families seem worth the modest increase in Medicaid expenditures. Moreover, as discussed, states have options for reducing expenditures if necessary through changes in program design.

Brown and his colleagues' evaluation of Cash and Counseling provides sound evidence on its effects. If there is disagreement about the policy, it will be because of differences in how its effects are weighted, not because of the absence of sound evidence on its effects. This experiment has made a very important contribution to the long-term care policy debate and taken us a step closer to a cash disability policy.