Feedback as a Strategy to Change Behavior: The Devil Is in the Details

Abstract

Background

Performance feedback is one of a number of strategies used to improve clinical practice among students and clinicians.

Objectives

The aims of this paper were to examine conceptual underpinnings and essential components for audit and feedback strategies, assess the extent to which recently published audit and feedback interventions include these components, and recommend future directions for feedback to improve its educational and behavioral impact.

Methods

Based on the actionable feedback model, we examined the presence of four theoretical constructs-- timeliness, individualization, lack of punitiveness, and customizability--in studies published during 2009–10 which included a feedback intervention.

Results

There was wide variation in the definition, implementation and outcomes of ‘feedback’ interventions, making it difficult to compare across studies. None of the studies we reviewed included all of the components of the actionable feedback model.

Conclusions

Feedback interventions reported to date, even when results are positive, often fail to include concepts of behavior change. This may partially explain the large variation in approaches and in results of such interventions, and presents major challenges for replicating any given intervention. If feedback processes are to be successfully used and disseminated and implemented widely, some standardization and certainly more clarity is needed in the specific action steps taken to apply behavioral theory to practice.

A number of strategies are used among students and practicing clinicians to improve clinical performance and adherence to standards of practice and guidelines. These include traditional educational programs and marketing techniques (e.g. lectures and seminars, reminders or other motivational techniques) as well as monitoring performance and providing individual or group feedback on performance. Two literature reviews of interventions to improve clinician prescribing practices published in 2009 both concluded that no single approach (academic detailing, feedback, formal educational programs, reminders) was clearly superior and that the effect sizes were generally small (1, 2). Auditing and feedback mechanisms are predicated on several assumptions. First and foremost is the assumption that recipients are able and willing to modify their behavior, and agree on the goals of change with those delivering the audit and feedback intervention. Further assumptions include, for example, that the feedback provided is relevant and meaningful and delivered in a way that is readily accessible to the learner, and that the learner knows how to interpret and act on the results once they are received. The extent to which these assumptions are considered when planning how to provide feedback is not clear and there are wide variations in approaches and components of what are generally referred to as ‘audit and feedback’ interventions.

Unfortunately, studies of the impact of audit and feedback interventions have rarely articulated whether a theoretical approach underpins their choice of intervention or how it is delivered (3). The aims of this paper are to review conceptual underpinnings and essential components for audit and feedback interventions, assess the extent to which such recently published interventions include these components, and recommend future directions for feedback to improve its educational and behavioral impact.

Conceptual Underpinnings and Essential Components for Feedback Interventions

While variations of performance feedback have been published in the literature for many decades, one of the most extensive discussions of the history of feedback was published by Kluger and DeNisi in 1996 (4). They also conducted a meta-analysis of research in which performance feedback was compared with another intervention as well as a comparison group which received no intervention. A total of 131 studies met their criteria, from which 607 effect sizes were identified. They found that while the preponderance of evidence suggested that feedback had a positive effect on behavior, more than one-third of the interventions were actually associated with worse performance. The variability in approaches and outcomes is remarkable. Since it is likely that many additional reports with null results remain unpublished, it is clear that strategies referred to as ‘performance feedback’ are widely divergent either in their content or in the process used to implement them.

Based on their work, Kluger and Denisi developed the ‘feedback intervention theory’ (FIT), which is based on five assumptions: (1) behavior is regulated by comparing practice with a goal or standard, (2) these goals or standards are rank ordered by importance, (3) only those gaps between feedback and goals which receive explicit attention will have an impact on behavior, (4) attention by the learner is normally directed to a moderate level of control hierarchy (from task learning, task motivation, and meta-tasks), and (5) feedback affects behavior by changing the locus of control so that the learner feels more ‘in charge’. Hence, the central explanatory theme to FIT is not how feedback affects one’s learning or motivation to perform a task but rather how the feedback focuses one’s attention (4).

Components of the FIT were actually borrowed from control theory (5, 6), which hypothesizes that people change their practices when there is a gap between their current behavior and a behavioral goal which they wish to attain. Gardner and colleagues (3) pointed out, however, that behavior change theories rarely specify exactly how they should/could be applied, i.e. describing the actions to be taken to apply the theoretical constructs. With regard to feedback interventions, they suggest that for control theory to be effectively applied, the feedback must be accompanied by a clear articulation of behavioral performance goals and an action plan for how to meet those goals. If, for example, the discrepancy between current practice and the goal is too large and there is no plan for how to bridge the gap, the recipient of the feedback may give up, i.e. the feedback could actually have a negative effect.

A taxonomy of 26 behavior change techniques, including performance feedback, was produced by Abraham and Michie in 2008 (7). The change techniques described emanated from five different theoretical frameworks: information-motivation-behavioral skills model, theory of reasoned action, theory of planned behavior, social-cognitive theory, control theory and operant conditioning. This taxonomy was then used in three systematic reviews to code the presence or absence of these change techniques in behavioral interventions. They too found wide variation in results of interventions, but demonstrated that it is possible to accurately categorize behavior change interventions and establish a common language. They suggested that published reports specify and clearly describe techniques and theoretical frameworks used in behavioral interventions (7).

One systematic review of 118 studies using feedback interventions reported variations in effects from 16% decrease to 70% increase in adherence to guidelines(8). The fact that the underlying conceptual underpinnings of audit and feedback interventions are often unclear, not specified or perhaps not even considered may help to explain the large variations in approaches used and the equally wide variation in results of such interventions. A theoretical framework or model provides a context in which interventions can be assessed and coherent explanations for why particular interventions do or do not work (3). In fact, some authors have commented on the absence of the application of behavioral theories in reports and reviews of educational strategies such as audit and feedback (3, 4).

Using a multivariate meta-regression of 85 interventions, 61 (72%) of which were feedback only, Gardner reported that only three interventions (4%) actually incorporated feedback, performance goals and an action plan (3). Similarly, Foy, et. al. (9) concluded from a review of 85 randomized trials assessing feedback interventions that variations in findings differed substantially, probably as a result of varying methods of delivery of the feedback or other contextual differences. They also noted that there was little detail available about how to actually implement feedback. It seems clear, then, that the problem is not a paucity of theoretical models but rather a lack of understanding or effort to determine specific steps to move the theory into practice. Based on these findings, despite the fact that it is recognized that a theoretical approach should be used to design feedback (10), such an approach is often not used and/or not described in publications.

Hysong and colleagues (11) examined the characteristics of successful feedback mechanisms in six Veterans Affairs medical centers and described four components of feedback interventions that were common in high performing settings: timeliness, individualization, lack of punitiveness, and customizability (i.e., the ability to change components of the feedback over time and across settings, and actively engage the recipient of the information in the process). The authors note that these characteristics have shared features with the FIT theory of Kluger and Denisi (4) and other underlying principles of behavior change. As depicted in Figure 1, the authors developed a model which they termed, ‘actionable feedback’ which may serve as a useful theoretical guide to the planning and evaluation of feedback interventions used for educational purposes.

Figure 1.

Model of Actionable Feedback. From (11) available Open Access under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0)

In 2009, Hysong re-examined and updated a Cochrane Systematic Review (8) to determine whether published research applied the components of feedback interventions proposed by Kluger and Denisi in the FIT model (4). Among 519 studies identified, she found only 19 that included characteristics of the feedback (content, format, and frequency) theorized to be important. Nevertheless, based on a meta-analysis she concluded that overall, feedback interventions appeared to have a statistically significant but moderate positive impact on outcomes (12).

Current State-of-the Art in Performance Feedback Interventions

In recently published reports of feedback interventions, we examined whether the four components of this actionable feedback model were evident. We searched PubMed^™ using the keywords and title/abstract words of ‘performance feedback’ and ‘audit and feedback’ for research published in English between January 2009 and October 2010. In nine studies, multiple interventions including traditional educational programs, decision support tools, simulations and various forms of reminders or alerts were used in addition to feedback, and therefore were not assessed, resulting in 13 eligible studies.

Four studies described feedback interventions used for college students; three were efforts to reduce heavy alcohol consumption and the fourth to reduce marijuana use. No effect was reported for a single web-based personalized feedback intervention to reduce marijuana use (13). A modest effect on alcohol use was reported for a targeted intervention delivered electronically on a single occasion to 263 college athletes when compared with a non-targeted intervention or education only group (14). Similarly, when single versus biannual feedback was compared among 818 first year college heavy drinkers, gender-specific feedback as compared with controls was associated with significantly fewer drinking problems and there was a modest effect of biannual versus annual feedback for women but not for men (15). Butler and Correia compared a single face-to-face or computerized feedback intervention in 84 heavy drinking students; both interventions were more successful than control(16).

We found six studies in which feedback interventions were used to influence clinician behaviors, five of which reported positive results. Three studies used feedback to improve antibiotic prescribing. Naughton, et. al. provided 50 general practitioners on a single occasion with written feedback on their individual prescribing practices for the previous 12 months and a second group (n=48) received the same feedback accompanied by a 15–30 minutes meeting with a member of the research team (academic detailing). The feedback was associated with improved prescribing and the academic detailing added no further improvement (17). In a cross-over interventional study over two 6-month time periods on two medical units, the clinical records of patients receiving antibiotics were reviewed by an infectious disease specialist who provided specific feedback to the charge physician (n=204 and 226 antibiotic courses in the intervention and control periods, respectively). Their reports included an assessment of appropriateness as well as recommendations for modification. While the intervention was not associated with overall consumption of antibiotics, the duration of antibiotic therapy was significantly shorter during the intervention period (18). Nair and colleagues tested several different feedback mechanisms: an email regarding missed documentation, monthly report summaries, and real-time electronic decision support alerts (which is not a feedback intervention). Only the decision support alerts were associated with improved antibiotic prescribing (19).

Three additional studies assessed the impact of feedback in areas other than antibiotic prescribing. In the first study, physicians in four Israeli primary care clinics received group feedback regarding their performance compared to the other clinics. Then two of the four clinics were randomized to have diabetic patients also receive a letter and phone call asking them to remind their physician at the next clinic visit to address relevant topics regarding quality diabetic care. Providing feedback to physicians directly and through patients resulted in significant improvements in quality outcomes for diabetic care when compared with feedback to physicians alone (20).

The purpose of the second study was to assess the impact of performance feedback on prescribing antipsychotic medication to outpatients with schizophrenia (n=2,284 prescriptions). Over a period of 4.5 years, verbal and written aggregated feedback was provided only once to one service and at four time intervals to two other services. Prescribing practices showed significantly more improvement in the groups receiving additional feedback (21). In the third study, a group of 60 young adults assessed to be maladaptive perfectionists were randomized to receive a single-session feedback intervention in which their assessment results were individually discussed and their input and active participation were solicited. Overall, there was a statistically significant effect of the treatment condition on the relationship between perfectionism and emotional reactivity(22).

As summarized in Table 1, none of the recently reported studies employing feedback interventions included all of the components of the actionable feedback model. Several provided feedback only a single time or at infrequent intervals (e.g. yearly), some were individualized while others were presented as aggregated reports and none were designed to be customizable. Because the exact wording of the interventions was generally not published, it was not possible to confirm that all feedback was non-punitive, but based on the general descriptions provided, that appeared to be the case.

Table 1.

Components of ‘actionable feedback’(11) included in examples of recent (2009–2010) studies employing performance feedback

Citation	Topic	Description of Intervention	Population	Outcome Direction	Components of Actionable Feedback
					Timely*	Individualized	Non-Punitive	Customized
(13)	Marijuana use	Electronic personalized feedback (single time)	College students	No effect	No	Yes	Yes	No
(14)	Alcohol consumption	Electronic personalized feedback (single time)	College athletes	Modest positive	No	Yes	Yes	No
(15)	Alcohol consumption	Gender-specific normative feedback (single time)	College students	Modest positive	No	Yes	Yes	No
(16)	Alcohol consumption	Personalized, face-to-face vs computerized feedback	College heavy drinkers	Positive	No	Yes	Yes	No
(17)	Antibiotic prescribing	Written feedback (single time) compared with written and verbal feedback	General practitioners	Positive for written, no added value of verbal	No	No (group)	Yes	No
(18)	Antibiotic prescribing	Written feedback from infectious disease specialist	Internal medicine physicians	Positive	Yes	Yes	Yes	No
(19)	Prophylactic antibiotic prescribing	Several forms of feedback	Surgeons	Positive for real-time guidance and reminders	Yes	Yes	Yes	No
(20)	Diabetes care	Written group feedback	Primary care physicians	Modest positive	No	No (group)	Yes	No
(21)	Antipsychotic prescribing in schizophrenia	Aggregated verbal and written reports to groups	Outpatient mental health services facilities	Positive	No	No (group)	Yes	No
(22)	Emotional reactivity and psychological symptoms	Verbal feedback	Young adults diagnosed with maladaptive perfectionism	Positive	No	Yes	Yes	No

^*Defined as monthly or more and/or in ‘real time’

Another characteristic of described feedback interventions in general is that often the recipient of the feedback had minimal input into its content or delivery modality, i.e. the recipient was passive in the process. Consistent with control theory and with concepts of adult learning, it is likely that the relevance and importance of feedback provided will be enhanced by more active involvement of recipients in the development of the feedback intervention before it is even initiated. In addition to assuring that the information provided in feedback is meaningful to the recipient, it would likely also enhance the credibility of the information provided and assure that the communication medium (e.g. electronically, by mail, or verbally) is convenient, easy to understand, and readily accessible to recipients.

Conclusions and Recommendations

Feedback interventions reported to date, even when results are positive, continue to be relatively atheoretical. This, at least in part, may explain the large variation in approaches and in results of such interventions, and presents major challenges for educators or researchers who attempt to replicate any given intervention(12). There is emerging but still quite weak evidence that theory-driven interventions in this field are likely to be more effective than atheoretical interventions, but the main value of theory is to inform generalizability and, hopefully, a better understanding of why interventions might work in one context but not another.

If feedback processes are to be successfully used as an educational strategy and disseminated and implemented widely, some standardization and certainly more clarity is needed in the specific action steps taken to apply behavioral theory to practice. That is, the salient features of an intervention should be fully described in any research repot (23). We recommend that educators and researchers seeking to develop and test audit and feedback interventions to improve clinical practice and effect behavior change incorporate components of the actionable feedback model (Figure 1). In addition, we suggest that the targeted recipients of the feedback be actively involved from the beginning in developing, testing, and vetting the content and process of the performance feedback.