A Celebration of Failure

“Every failure is a step to success. Every detection of what is false directs us towards what is true: every trial exhausts some tempting form of error. Not only so; but scarcely any attempt is entirely a failure; scarcely any theory, the result of steady thought, is altogether false; no tempting form of Error is without some latent charm derived from Truth.”

--W. Whewell¹

Contemporary society cherishes success. Success defines the person, the organization, the culture. It is a clear goal for every initiative that has an outcome. It is a gauge by which one measures impact, influence, and consequence. Success is defined as much by tangible achievement of a predefined goal as it is by its polar opposite, failure. The commonly held view is that failure is to be avoided because success is to be achieved, and both cannot coexist. In this editorial, I will dwell on failure and make the case that failure has at least as important a role in our experience, education, and professional development as success—if we would only learn from it.

We are first exposed to the concept of failure in elementary school, quickly realizing how it can affect our educational progress. My generation of students lived in fear of failing tests, subjects, and, ultimately, grades, thereby being left behind to repeat the school year. This early, first experience with failure obviously colors our perception of the concept with great negativity. Defined in this way, failure is simply the opposite of success, a notion that sets the stage for the role of failure and its interpretation throughout one’s life. This is particularly true for physicians and scientists, who, as consummate overachievers, strive to succeed and implicitly strive to avoid failure with each challenge. Nothing but perfection will suffice, as failure renders our professional efforts, view of accomplishment, and sense of ourselves imperfect.

In the early 19^th Century, the term failure was commonly used to describe a “breaking in business,” or going broke or bankrupt (2). Over time, this purely commercial definition evolved to pertain to personal deficiency, as well as tangible accomplishment or moral behavior. How did this change occur? How did one’s financial inadequacy morph into personal inadequacy? Sandler, in his book Born Losers: A History of Failure in America, argues that, in part, this evolution reflects the interpretation of the American dream in 19^th Century America. He points out that the failures among us “embody the American fear that our fondest hopes and our worst nightmares may be one and the same….the [American] dream that equates freedom with success…could neither exist nor endure without failure.” We need failure, “the word and the person…to sort out our own defeats and dreams” (3). Put in contemporary Darwinian contrast, “It is not enough to succeed. Others must fail” (4).

Contemporary American education has taken this interpretation to an extreme, doing whatever it can to eliminate failure. Driven by the view that discouraging criticism (in the best case) and objective failure (in the worst case) will impair effective learning, the educational establishment has evolved to minimize the likelihood that students can fail—a course, or a grade, or a program. Failure in this educational ideal is considered a reflection of institutional inadequacy rather than a true learning limitation of the individual. Maintaining the student’s self-esteem at all costs has been the mantra of American public education now for some time, and while it has had its benefits, especially for students who would be severely defeated by even modest failure, it has had its disadvantages, as well, creating a self-affirming culture of narcissism among many students.

Over the past decade, this change in educational philosophy has ‘trickled up’ from elementary school to professional school and postgraduate medical education. Rather than assume the old ‘sink or swim’ philosophy of past generations, we as medical educators must now be ever-sensitive to the needs of students, residents, and fellows, providing critical feedback at just the right time in just the right way. Sensitivity to the needs of trainees has also influenced the process of identifying candidates for residency and fellowship, even at the most elite institutions: residents and fellows are now recruited rather than selected as they were in the past. In this way, we dampen the slight of rejection, viewing it now as a failure of the program to recruit the applicant rather than a failure of the applicant to be chosen by the program.

While encouragement is clearly important in early education and constructive, positive criticism essential for optimizing the learning experience, there comes a time in each person’s development at which clear criticism and the risk of true failure need to be conveyed. The complexity of life, biologically and experientially, is rife with uncertainty, and in the course of its execution, rich with the possibility of failure—to meet an aspiration or achieve a goal. As parents, educators, and role models, we are not meeting our obligations to trainees unless we instruct them in the importance of failure, how to react to it, and, most importantly, how to learn from it. Failure is imperative for developing the tenacity and self-control necessary to interact effectively with our complex environment, and as such, is the true “secret to success,” as Paul Tough pointed out in his New York Times piece in 2011 (5).

A moment’s reflection will lead one to appreciate how important and pervasive failure is in the normal course of one’s personal and professional life. In fact, among some highly specialized goal-oriented professions, failure is a dominant and expected outcome. In the pharmaceutical industry, the clinical failure rate for drugs entering phase II testing was reported to be 81% for 50 illustrative compounds that entered clinical testing in 1993-2004 (6). Major league batters fail to hit the ball ~75% of the time (the overall major league baseball batting average for 2013 was 0.253 ) (7). Meteorologists have an overall error rate for predicting precipitation over three days of 15%, with precipitation predicted but not observed 43% of the time, and precipitation observed but not predicted 10% of the time (for San Francisco, the last three months of 2011) (8). Health economists’ prediction algorithms are notoriously challenging, with absolute prediction errors of the actual means ranging from 98% (9) to 79% (10).

Failure is, of course, part of the scientific method. All well designed experiments are framed in terms of the null hypothesis, which more often turns out to hold rather than its alternate. Exploring the bases of biological phenomena or diseases requires the same approach and has the same low yield of positive results. Every failed experiment changes the researcher’s perspective, helps reframe the experimental design, and leads to an increasingly refined approach to the problem narrowing iteratively over time the possibilities for fruitful study. Engineering disciplines have taken this process to an extreme, having developed the field of failure analysis using forensic engineering approaches (11). No matter how insightful an investigator is believed to have been in retrospect, the scientific approach is one of informed trial and error in the best of circumstances, and, therefore, invariably subject to the play of chance. Given the importance of failure in scientific research and problem solving, it should come as no surprise that the current educational strategy of ego preservation by limiting challenging course work to minimize harm to self-esteem runs counter to optimal scientific education. Research and innovation in science require failure, which must be taught (12), nurtured, understood, and incorporated in one’s scientific paradigm. The scientific community can only achieve this level of understanding if it is aware of both positive results (i.e., those in which the null hypothesis is rejected) as well as negative results (i.e., those in which the null hypothesis is not rejected). Hence, publishing negative trial outcomes is essential for the scientific enterprise, a fact that had not been widely accepted by major journals until relatively recently.

Preconceived bias driven by the desire to achieve an hypothetical outcome can lead to significant type I errors, misleading future experimentation in that scientific community. With ever more riding on the success of an experiment and the scientist ever less equipped to handle a negative result (which equates with a failed experiment), is it any wonder that published experimental results are increasingly recognized as incorrect or false (13)? Add to this the great importance placed on publishing papers in journals of the greatest impact, which invariably require proving an hypothesis in multiple systems using a variety of internally consistent methodologies, and any reasonable experimentalist can easily understand how this growth in false outcomes has occurred. As anyone who has performed scientific experiments in different systems (cellular, animal, or human) can attest, it is invariably the case that complete consistency is extraordinarily uncommon owing to experimental imprecision, biological variability, and the play of chance. Thus, one can either conclude from reading these elegant studies in which everything works that some of the data must be overinterpreted or, worse, manipulated, or that inconsistent experiment results were excluded from the published study. This point again illustrates how fear of failure in an ever-competitive scientific landscape with increasingly limited resources misguides the scientific enterprise.

Scientific failure has more challenging dimensions that affect all of us, including the most accomplished investigators. Risk taking and innovation are particularly dependent on failure, but in a fiscally conservative environment such as that which currently exists in the US economy, true innovation is rarely rewarded. NIH grant proposals are more likely to be funded if the proposed research is viewed as likely to meet the specific aims than are proposals that address high-risk hypotheses with limited supportive data. Even in industry where, it can be argued, high-risk research can more readily be performed than in academics, the fiscal environment has taken a toll, with the research and innovation units of many major pharmaceutical companies having been downsized or eliminated.

Biomedical investigators—even the most academically successful—are painfully aware of common failures of other sorts, as well. Rejections of manuscripts for publication, unsuccessful or delayed promotions, and the lack of success of trainees to secure desirable positions all are part and parcel of our academic lives. Ideally, each of these failures pushes us to improve, serving as a barometer against which to compare progress through the profession.

Failure in clinical medicine is particularly troublesome to trainees and established practitioners alike. As medical students, residents, and fellows, we continue to aim for perfection in our educational experience, viewing failure as a personal shortcoming. Bias enters into our scientific mindset in this realm, as well, by clinically framing a case within the context of diagnoses with which we are familiar. We are forever attempting to force square pegs in round holes, rather than focusing on the features of a clinical case that do not fit the mould. We do so to gain comfort in an uncertain clinical landscape, avoiding failure and perceived inadequacy in the process. Yet, as clinicians, we worry that we have not made the best decision for our patients, concerned that we may have missed a diagnosis, misread a laboratory result, or chosen a therapeutic strategy that is suboptimal or, worse, incorrect. None of us wishes to be perceived as the physician who failed his patient.

Failure is, however, a great teacher, especially in clinical medicine. I can clearly remember many of the clinical mistakes I have made, learning from each of them; clinical successes are far less memorable and far less instructive. Osler, in A Way of Life, wrote of keeping a journal of one’s mistakes, stating that one should “[b]egin early to make a threefold category—clear cases, doubtful cases, mistakes ….[as] it is only by getting your cases grouped in this way that you can make any real progress….only in this way can you gain wisdom with experience” (14).

From an even broader, more philosophical perspective, failure has great importance for a number of reasons. Bradaton highlights some of them with keen insight (15). Failure gives us the opportunity to see our existence close-up. It is a lens through which we begin to see the flaws in our otherwise perfect and perfectly predictable being. When failure gives us this insight, it emphasizes for us the existential threat that constantly stalks our lives, giving us pause to consider how extraordinary life is. For this reason, failure can be therapeutic, forcing us to realize that the world does not revolve around us, guarding against unbridled arrogance, offering the comfort of humility in its stead.

I believe we should celebrate failure. In fact, we should exhort our students, trainees, and offspring to be alert to failure and its potential as a learning experience, rather than guard against it at all costs. We cannot avoid failure, and if we view it through the constructive lens of self-improvement, we need not cushion its blow to self-esteem. “The only real mistake” one can make “is the one from which we learn nothing” (16).