Why Truth Matters: Research versus Propaganda in the Policy Debate

I begin my remarks with a confession: I wrote the title of my talk—“Why Truth Matters”—before I wrote the talk. I know such behavior is neither unique nor egregious. But it has caused me some anxiety. First came the challenge from my son, who is studying philosophy. To paraphrase his reaction: “Mom—What are you thinking? Surely you know there's no such thing as truth.” Second came my fear that some in the audience, recalling my political experience, might leap to the microphone saying: “This woman claims to know what truth is?” And third—worst of all—came my conversation with myself, when (in the middle of the night, of course), I said: “So, hot shot, why does truth matter in the policy debate?”

I share these anxieties to let you know not only that I've struggled with the question I posed for these remarks, but also that I've resolved my struggle. I do think truth matters and I'm going to explain why. I'll start with a distinction between truth and propaganda, compliments of my son; then, I'll describe how truth operates in the policy process. I'll conclude with why truth matters.

Participants in this year's annual meeting of the Academy have discussed and debated a wide range of health research and policy issues—examining how institutions, individuals, and incentives interact in the delivery and financing of health care. Figuring out how things work is the business of the “academy” writ large and of AcademyHealth. It's this pursuit of knowledge that I'll call “truth.”

Fundamental to pursuing truth is to start with a question, rely on intellectually sound techniques and methods to answer it, and report the answer, whatever it may be. The question and even the kinds of evidence brought to bear may be shaped by values. After all, there is no such thing as value-free social science. But in “truthful” research, the answers are what they are, regardless of the researcher's point of view. By contrast, “propaganda” starts with an answer and relies on research not to find out how things work but to prove a predetermined conclusion.1

With the difference between truth and propaganda defined, the next question is whether that difference makes any difference in the policy process. My sense is that it's frustratingly hard to see any difference in impact or influence. I'm confident that every researcher in this audience has watched in horror debates on Capitol Hill—each wondering about priorities, about legislation that seems to make no sense, about partisan distinctions that seem to make no difference, about, indeed, ignoring research. It's easy for a researcher to look at politics and conclude, “Why bother?”—that is, that truth, or research, doesn't matter at all. But that conclusion would be wrong. Let me tell you why.

In an analysis that provides a foundation for the study of public policy, John Kingdon identifies three parts of the process of getting items on the political agenda: identification of the policy problem, development of solutions, and debate and decisions about taking policy action.2 Research—or truth—plays a role in all three. Let's first focus on defining the policy problem. Research may not be definitive here, but it clearly plays a role. Whether the issue is quality (as presented in Don Berwick's keynote address for this meeting), prescription drugs, racial disparities, immunization rates, deaths from AIDS, or a host of other human concerns, research has made an enormous difference in putting problems for people on the policy and political agenda. Research on trends—on health care coverage, health care costs, or other matters of concern—may not be the most exciting of research topics, but it's amazing what data can do to attract political attention. It is politics that determines whether any of these issues will grab the political limelight at any given moment. But it is data and research that determine whether there's something to grab.

What about developing policy solutions? Here's where much of the action is in the research-to-policy connection—measuring how a policy intervention will affect a myriad of behaviors in the health care world. It's a tough task, let's face it, in part because, as researchers, we recognize the enormous uncertainties surrounding any policy intervention. On the issue I concentrate on, for example—health care coverage—questions exist about whether a policy intervention will increase coverage, whether an increase in coverage will improve access to care, and whether improved access to care will lead to better health. There's not always a great scientific base on which to make a case for a specific action. But that's why they pay us the big bucks, and all our truth-telling skills are called into play. We draw on all our evaluation, assessment, and projection methodologies to estimate an intervention's impacts, incentives, and outcomes. Specifically,

• Who's affected and how?

• Whose behavior will change and how?

• What perverse incentives and unintended consequences (always the researcher's favorite) will emerge?

• How likely is it that the intervention will make matters worse rather than better?

Probably the most frequent impact of answers to these questions is to take some policy proposals off the policy agenda. Although not as satisfying as getting an initiative on the agenda, this impact nevertheless reflects the power of our ability to distinguish what will work from what won't in the policy arena.

And that takes us to the third part of the policy process—the arena of policy action. It's in this arena that we come to the heart of the matter regarding the role of “truth”: in the political world, who really cares about what will work and what won't? An answer to this question has to start from the premise that politics is not about truth; it's about values, and interests, and votes, and money. As a card-carrying political scientist, I can tell you, truthfully, that's just how it is.

But politics creates “windows” of opportunity,3 and in those windows, research provides the tools or the weapons to carry the day. Research is the source for claims regarding the deaths prevented, dollars saved, jobs created, that are the essence of policy advocacy.

So that's how research, or truth, is used in the various parts of the policy process and past leaders of the Academy—Jack Hadley, Diane Rowland, Arnie Epstein—as well as our keynote speaker Don Berwick have given ample evidence of our membership's impact on the policy front.

But I've begged my own question. I didn't ask whether truth plays a role in the policy process. I asked whether truth is more powerful than propaganda in that process—in identifying the problem, developing the solution, or supporting political action.

As an honest woman, I think the answer is no. I can't make the case that truth matters with the argument that truth will triumph over propaganda. As we all know, when the politics are propitious, there may be no need for data to get political action. In defining a policy problem, tragedy will trump research any day as a motivator. Witness the impact of September 11 on public health investment or of nursing home deaths on enforcement of nursing home regulations. And the policy initiative that results may or may not be what research tells us will work.

In identifying solutions, uncertainty is intrinsic to the research enterprise. The fact is that researchers disagree—in my issue area, health insurance, for example, on “take-up rates,”“employer dropping,” price, and behavior in the health insurance market and a host of other factors critical to estimating the impact of a policy proposal. Even though we all agree on the importance of subsidies to expanding health insurance coverage, we disagree about the evidence and the values embodied in different strategies to provide those subsidies.

In the political debate, sadly, a bad study can be as powerful as a good one. The press and politicians tout numbers and outcomes regardless of their intellectual credibility. Even a bad study provides cover—a rationale for a particular vote—and cover may be all a politician is actually seeking.

I don't believe this is a terribly sophisticated argument. And, if I'd looked a little harder, maybe I'd see that where researchers do agree, or over time, that truth really does overcome propaganda. But given the nature of politics, I remain a skeptic.

In my view, the case for truth rests somewhere other than in its victory—specifically, in the integrity and purpose of the researchers who pursue it. As members of academe, we are, after all, not primarily in the business of politics. We're in the business of truth. We're in it because we like the challenge of figuring out how things work and because we care not just about winning a political battle but about generating and using truth to make policy that will promote whatever values we hold dear. Indeed, the issue is not so much whether anybody else cares about what policy will work; it's that we care. And we feel an obligation and commitment to make sure that—even if it doesn't triumph—truth is at the table in the political debate.

So what is the take-away message from this conclusion? What does it say about what each of us does? It depends upon how researchers, as individuals, view the political process. Some in our community don't really want to be part of that process. For this group, I have two thoughts. First, though you don't need my permission, I think that's fine. Not every researcher has to commit to policy work. A division of labor makes sense, and some of us like participating in the political process more than others. Those of us who do will be happy to use your research. Indeed, Academy staff are primed to share your research with the policy community.

But my second thought to that group is not to discount or ignore politics just because you don't want to participate. What your research shows matters to people—to their health and financial well-being, to the nature of the society in which they live, to the distribution of resources in that society, and to the effectiveness of the private and public institutions that shape many of its activities. Powerful stakeholders are well aware of these implications. Even if you don't use your research in the political arena, it will be used by others, and used in ways and for ends that you may not support. It behooves you always to consider how your research might be used and to write in a way that minimizes its misuse.

For those who want to participate in policy politics, I have a different set of suggestions.

• Pick your research questions not only based on what is but on what ought to be on the political agenda. Your best work comes out of your own enthusiasm, and you never can tell when that policy window will open.

• Use your research to tell a story—the valuable advice I got while writing my dissertation. If you don't know what your story is—if you can't answer the “so what?” question—you simply haven't done your job.

• Beware of traps. The line between research and propaganda gets fuzzy when you want to play in the political process. The push for certainty in an uncertain environment—for truth about a future that is, truthfully, unknowable—is powerful. Speaking truth to power, as Aaron Wildavsky defined the role of a policy analyst, is not always pleasurable. But commitment to truth under pressure is a measure of your own integrity.

The bottom line? As members of academe and AcademyHealth, pursuit of truth is our hallmark. We have reason to be proud of what we do. Just as politics is not about the pursuit of truth, the pursuit of truth is not about politics. We pursue it because it's the right thing to do.