Key Features of Academic Detailing: Development of an Expert Consensus Using the Delphi Method

Abstract

Background

Academic detailing is an outreach education technique that combines the direct social marketing traditionally used by pharmaceutical representatives with unbiased content summarizing the best evidence for a given clinical issue. Academic detailing is conducted with clinicians to encourage evidence-based practice in order to improve the quality of care and patient outcomes. The adoption of academic detailing has increased substantially since the original studies in the 1980s. However, the lack of standard agreement on its implementation makes the evaluation of academic detailing outcomes challenging.

Objective

To identify consensus on the key elements of academic detailing among a group of experts with varying experiences in academic detailing.

Methods

This study is based on an online survey of 20 experts with experience in academic detailing. We used the Delphi process, an iterative and systematic method of developing consensus within a group. We conducted 3 rounds of online surveys, which addressed 72 individual items derived from a previous literature review of 5 features of academic detailing, including (1) content, (2) communication process, (3) clinicians targeted, (4) change agents delivering intervention, and (5) context for intervention. Nonrespondents were removed from later rounds of the surveys. For most questions, a 4-point ordinal scale was used for responses. We defined consensus agreement as 70% of respondents for a single rating category or 80% for dichotomized ratings.

Results

The overall survey response rate was 95% (54 of 57 surveys) and nearly 92% consensus agreement on the survey items (66 of 72 items) by the end of the Delphi exercise. The experts' responses suggested that (1) focused clinician education offering support for clinical decision-making is a key component of academic detailing, (2) detailing messages need to be tailored and provide feasible strategies and solutions to challenging cases, and (3) academic detailers need to develop specific skill sets required to overcome barriers to changing clinician behavior.

Conclusion

Consensus derived from this Delphi exercise can serve as a useful template of general principles in academic detailing initiatives and evaluation. The study findings are limited by the lack of standard definitions of certain terms used in the Delphi process.

The term “detailing” has conventionally referred to face-to-face promotional activities to physicians conducted by pharmaceutical sales representatives.1,2 Although detailing began in the nineteenth century, the rise of modern pharmaceutical detailing began in the 1940s and resulted from major expansions within the pharmaceutical industry.3 Major pharmaceutical companies currently devote substantial resources to detailing activities for their major products,4 with estimated annual spending of $7 billion to $20 billion.5 This type of detailing can be problematic because the focus on increasing sales can distort physicians' perception of the safety and efficacy of the promoted drug and affect prescribing choices. Ultimately, this can affect patient outcomes and increase the costs of care.6–8 Conversely, academic detailing, also known as educational outreach, uses the one-on-one outreach approach of pharmaceutical detailing to provide clinicians with unbiased, evidence-based information that can be used to improve clinical practice.9–14

Originally developed in the 1980s in the context of prescribing decisions, the methods of academic detailing have since been adapted to other areas of healthcare, such as smoking cessation, cancer screening, and reducing hospital readmissions, to disseminate information and change behavior.15–18

A systematic review of the effect of academic detailing in 69 studies, representing more than 15,000 health professionals, showed that this intervention could be effective in changing physician prescribing behavior and in improving clinical practice.9 That review also showed that the effect of academic detailing in reducing inappropriate prescribing was modest but significant, with a median (adjusted) risk difference of 4.8% and an interquartile range of 3% to 6.5%.

In terms of behavior not related to prescribing, the effect range was greater (median adjusted risk difference, 6%; interquartile range, 3.6%-16%). Although studies generally have shown that academic detailing can be effective as a stand-alone intervention or as part of a multipronged approach, the context of the intervention and the environment in which it is delivered may influence its effectiveness.19

Many approaches to implementation of academic detailing have been described in the literature, reflecting a lack of agreement on what this intervention should entail among those who implement and evaluate it. Academic detailing programs exist in various settings. Government-supported programs of academic detailing have been implemented in many countries, including a nationwide program in Australia and multiple provincial programs in Canada.20,21

In the United States, some academic detailing programs are supported by local or state government (eg, New York State Medicaid Prescriber Education Program), some exist as part of the organizational structure of integrated health systems (eg, Kaiser Permanente system in California), and others are supported by insurance companies or nonprofit entities.9,22

The US Veteran Affairs health system has implemented several academic detailing programs to improve mental healthcare and prescribing practices for antihypertensive medications.23,24

Although the diversity of implementation settings and the lack of uniformity in the literature can reflect the adaptability of this educational approach, such variation poses a challenge for those who seek templates to guide the implementation of new academic detailing programs and for those who wish to evaluate the effectiveness of academic detailing.9,25–27

KEY POINTS

• ▸The goal of academic detailing is to improve clinician decision-making through unbiased information to enhance evidence-based care and improve patient outcomes.
• ▸The implementation of academic detailing is not uniform, which can make the evaluation of its effectiveness difficult to measure.
• ▸We used expert-based consensus (via the Delphi process) to determine what academic detailing should entail.
• ▸The academic detailing message has to be relevant and focused on the needs and interests of the clinician.
• ▸Evaluating the effectiveness of academic detailing should include measuring changes in clinicians' decision-making, in patient-centered outcomes, and in utilization of resources.

This article is the second of a 2-part series on academic detailing. Part 1 was descriptive findings from a systematic review about important characteristics of academic detailing, published in American Health & Drug Benefits in 2015.28 In this second article, we used the Delphi method to collect information from an international panel of experts to develop a consensus regarding the standards for academic detailing.

The Delphi method is a widely accepted technique of structured and systematic information-gathering from a group of experts (ie, the Delphi Panel) on a specific topic using a series of questionnaires.29–32 The Delphi method allows a panel of experts to provide insight and opinions, even when not located in the same geographic area. The questionnaires are administered, in iterative rounds, to all experts who agree to participate. The responses from each previous round are compiled and provided to the panel before the subsequent round begins. This structure allows each participant to reevaluate his or her initial answer after considering the responses of the other panel members.

Rounds are conducted until consensus is reached or until there is stability in the questionnaire answers; the process typically requires 3 or 4 rounds. A key advantage of the Delphi method is that it allows each participant to have a voice, in an anonymous manner. Participants do not interact or speak with each other during the process; they are merely given feedback about their own answer in the context of the group's responses.

Methods

Recruitment of the Delphi Expert Panel

We used purposeful sampling of potential experts. We invited 26 experts to participate in our Delphi exercise, 20 of whom accepted the invitation. The experts were known to one of the study investigators (M.A.F.), who directs the National Resource Center for Academic Detailing (NaRCAD), an initiative supported by the Agency for Healthcare Research and Quality (AHRQ) to encourage and support academic detailing programs. Experts were selected based on experience in areas of clinical content, research, or administration related to academic detailing. All experts indicated their willingness to participate in the Delphi process and did not receive any monetary compensation for participation.

Survey Design

We previously identified 5 critical characteristics of academic detailing, including (1) content, (2) communication process, (3) clinicians targeted, (4) change agents delivering intervention, and (5) context for intervention.4 Based on these 5 critical characteristics, we developed 72 survey items for the Delphi exercise, covering 9 topics (see Appendix at www.AHDBonline.com). The experts were asked to rank each item on a scale of 1 to 4, from least to greatest importance (1 = not at all important, 2 = somewhat important, 3 = very important, 4 = essential).

The 9 general survey topics were:

1. What information do you believe should be offered through academic detailing visits?

2. What outcomes should studies using educational outreach be seeking to change and to measure?

3. Should information provided in academic detailing visits be tailored?

4. In what ways should academic detail interventions select providers for invitation to participate?

5. Which facets of visits are most important to academic detailing?

6. Which groups are critical to academic detailing visits?

7. Which qualifications are critical for academic detailing outreach workers to have?

8. What areas of training should academic detailing workers receive?

9. Should the academic detailing outreach worker have the same or a different employer than the providers being visited?

Survey Administration

The experts participated in 3 rounds of the Delphi consensus exercise during a 2-month period, from January to February 2014. Each expert was e-mailed a link to the confidential online survey for each of the 3 rounds. The experts were given 2 weeks to complete each survey round, and 2 reminder e-mails were sent (1 week before and 1 day before the conclusion of each round). All 20 experts were sent round 1 of the online survey. Nonresponders to a given survey round were excluded from subsequent rounds.

Consensus Determination

We divided the consensus into “critical” and “not critical” features of academic detailing. “Critical” consensus comprised the ratings “essential” and “very important,” whereas “not critical” consensus comprised the ratings “somewhat important” and “not at all important.”

Consensus was reached via 1 of 2 ways. First, if an item received at least 70% of the votes in a single rating category (ie, 1 of the 4 options), then we assigned consensus based on the respective rating category. For items that did not achieve a single rating of 70% or more, consensus was reached if the sum of the rating percentages within one of the dichotomized consensus categories represented 80% or more of the votes.

For example, the survey item pertaining to whether “best practices” information should be delivered during academic detailing intervention achieved ratings of “essential” (28%) and “very important” (67%). Although this item did not receive at least 70% of responses in 1 rating category, “critical” consensus was achieved because the sum of “essential” and “very important” ratings was at least 80%.

Delphi Exercise

In round 1, the panel was asked about the 9 elements described above. Before round 2, the survey was modified to reflect the findings from round 1. At the start of round 2, each panel member was reminded of his or her original answers and was shown the aggregated group findings from round 1 to situate his or her position relative to that of the group. Each expert was then asked to consider his or her original responses in the context of the group findings using the same ordinal scale presented in round 1.

The survey for round 3 was modified to reflect the results from round 2 in 2 ways. First, the panel was shown the results of the consensus items and was given the opportunity to make open-ended comments. Second, the panel was presented with the list of nonconsensus items and their relative rating. Each expert was then given the final opportunity to modify his or her answers.

Results

Table 1 shows the demographics of the panel participants. The participants included physicians (N = 5), pharmacists (N = 10), and those with advanced training in statistics/epidemiology or a related healthcare field (N = 5). A total of 70% (N = 14) were based in the United States, 45% (N = 9) were primarily affiliated with an academic medical center, and 70% (N = 14) had authored published peer-reviewed research or review articles related to academic detailing.

Table 1.

Demographics of the Delphi Expert Panel

Characteristic	Participants, N (%) (N = 20)
Sex
Female	12 (60)
Institution location
United States	14 (70)
Canada	4 (20)
Australia	2 (10)
Type of institution
Academic medical center	9 (45)
Nonprofit private institutiona	8 (40)
Public governmental institution	3 (15)
Academic degrees
MD or its equivalent	5 (25)
Pharmacist (RPh or PharmD)	10 (50)
Advanced degrees (master's or doctorate) in epidemiology/statistics or related healthcare field	5 (25)
Percentage with peer-reviewed published work in academic detailing	14 (70)

^aIncludes foundations, programs, or health delivery institutions that train and/or implement academic detailing.

In round 1, 19 of the 20 invited participants completed the online survey. In round 2, 18 of the 19 remaining participants completed the survey. In round 3, 17 of the 18 remaining participants completed the survey. Overall, we achieved a response rate of at least 94% in each of the 3 rounds of the Delphi exercise, and a cumulative overall response rate of 95% (54 of 57 surveys). We did not detect any particular features that would differentiate nonresponders from those who completed the survey.

At the end of round 1 of the Delphi exercise, 36% (N = 26) of the survey items achieved consensus using the predetermined metric. After receiving feedback from the original survey, including group responses, the panel achieved consensus on 58% (N = 42) of the survey items by the end of round 2. At the end of round 3, 75% (N = 24) of the 32 items that previously had not met consensus achieved it. The remaining 8 survey items attained response stability (Table 2).

Table 2.

The 8 Survey Items That Reached Response Stability

Survey item	Not at all important, %	Somewhat important, %	Very important, %	Essential, %
1. The importance of clinician reminders in academic detailing intervention	0	65	29	6
2. The importance of evaluating academic detailing outcomes based on changes in healthcare cost	0	47	47	6
3. The importance of the number of academic detailing visits a detailer makes to the same provider	0	47	53	0
4. The importance of the duration of the academic intervention (eg, time between first and last visit)	6	35	59	0
5. The importance of the length of individual visits in academic detailing	0	53	47	0
6. The importance of a team-based approach in academic detailing interventions	0	65	23	12
7. The importance of training academic detailers on networking skills	18	29	41	12
8. The importance of training academic detailers on the use of social marketing techniques	18	29	47	6

Overall, 92% (N = 66) of the 72 survey items achieved consensus by completion of round 3 of the Delphi exercise. The list of consensus items was then presented to the panel for the final time, for informational purposes, in the form of a brief report (Table 3). The panel experts were given the opportunity to make open-ended responses to the list of questions.

Table 3.

Survey Items That Achieved Consensus After 3 Rounds of the Delphi Exercise, and Their Delphi Rating

Item	Consensus	Delphi Rating
The importance of the following groups targeted in academic detailing
1. Clinicians	Critical	Essential 83%; Very important 17%
2. Patients	Not critical	Somewhat important 67%; Not at all important 17%
The importance of the following factors in selecting providers for academic detailing
1. Geographic	Critical	Essential 11%; Very important 83%
2. Specialty or specific setting	Critical	Essential 22%; Very important 72%
3. Patient population	Critical	Essential 12%; Very important 71%
4. Pattern of care	Critical	Essential 6%; Very important 82%
The importance of the following components in academic detailing interventions
1. Clinician education	Critical	Essential 79%; Very important 16%
2. Feedback about clinical performance	Critical	Essential 11%; Very important 78%
3. Recommendations about practice change	Critical	Essential 61%; Very important 39%
4. Opinion leaders	Critical	Essential 6%; Very important 78%
5. Commitment to change	Critical	Essential 28%; Very important 67%
6. Best practices	Critical	Essential 28%; Very important 67%
7. Decision support	Critical	Essential 82%; Very important 0%
8. Patient education	Not critical	Somewhat important 72%; Not at all important 6%
9. Patient reminders	Not critical	Somewhat important 67%; Not at all important 22%
10. Patient care equipment	Not critical	Somewhat important 78%; Not at all important 11%
11. Financial and other incentives	Not critical	Somewhat important 72% Not at all important 11%
12. Practice facilitation	Not critical	Somewhat important 71%; Not at all important 0%
13. Technical and other assistance	Not critical	Somewhat important 82%; Not at all important 0%
14. Patient resources	Not critical	Somewhat important 71%; Not at all important 0%
15. Community resources	Not critical	Somewhat important 88%; Not at all important 0%
16. Documentation tools for clinicians	Not critical	Somewhat important 82%; Not at all important 0%
17. Care coordination	Not critical	Somewhat important 59%; Not at all important 26%
18. Continuing education guidance	Not critical	Somewhat important 77%; Not at all important 12%
19. Social marketing	Not critical	Somewhat important 65%; Not at all important 18%
The importance of the following outcomes in evaluating academic detailing interventions
1. Clinician behavior or performance	Critical	Essential 83%; Very important 17%
2. Patient outcomes	Critical	Essential 6%; Very important 72%
3. Clinician knowledge or awareness	Critical	Essential 28%; Very important 56%
4. Resource utilization	Critical	Essential 6%; Very important 84%
5. Clinician attitude	Critical	Essential 12%; Very important 71%
6. Clinician skill	Not critical	Somewhat important 72%; Not at all important 0%
7. Care intensity	Not critical	Somewhat important 78%; Not at all important 6%
8. Changes in educational activities	Not critical	Somewhat important 73%; Not at all important 11%
9. Changes in workflow	Not critical	Somewhat important 89%; Not at all important 6%
10. Family satisfaction	Not critical	Somewhat important 61%; Not at all important 28%
11. Prevalence of condition or disease	Not critical	Somewhat important 77%; Not at all important 0%
The importance of the following qualifications for academic detailers
1. Physician	Critical	Essential 12%; Very important 71%
2. Pharmacist	Critical	Essential 6%; Very important 77%
3. Clinical or health educator	Not critical	Somewhat important 88%; Not at all important 0%
4. Quality improvement expert	Not critical	Somewhat important 53%; Not at all important 35%
5. Public health practitioner	Not critical	Somewhat important 67%; Not at all important 22%
6. Pharmaceutical industry representative	Not critical	Somewhat important 11%; Not at all important 89%
7. Nurse	Not critical	Somewhat important 72%; Not at all important 6%
The importance of the following elements of information in academic detailing interventions
1. Discussion: barriers and solutions	Critical	Essential 83%; Very important 17%
2. Discussion: challenging cases	Critical	Essential 11%; Very important 78%
3. Discussion: feasibility	Critical	Essential 33%; Very important 61%
4. Discussion: context	Critical	Essential 29%; Very important 65%
5. Discussion: quality improvement approach, strategies, interventions	Critical	Essential 53%; Very important 35%
6. Ad hoc visits, resources, and support upon request	Critical	Essential 18%; Very important 71%
7. Discussion: general progress	Critical	Essential 65%; Very important 18%
8. Model of discussion or teaching technique	Not critical	Somewhat important 83%; Not at all important 6%
9. Role playing	Not critical	Somewhat important 67%; Not at all important 17%
10. Meetings with peers and specialists facilitated	Not critical	Somewhat important 88%; Not at all important 0%
11. Ongoing needs assessment	Not critical	Somewhat important 71%; Not at all important 0%
The importance of training in the following areas for academic detailers
1. Academic detailing process	Critical	Essential 94%; Very important 0%
2. Anticipated barriers	Critical	Essential 89%; Very important 11%
3. Behavior change and persuasion	Critical	Essential 83%; Very important 11%
4. Communication and interpersonal skills	Critical	Essential 94%; Very important 6%
5. Content of study	Critical	Essential 83%; Very important 11%
6. Debriefing on visits	Critical	Essential 17%; Very important 72%
7. Education	Critical	Essential 17%; Very important 83%
8. Supervising observations	Critical	Essential 0%; Very important 77%
9. Rehearsals/role playing	Critical	Essential 72%; Very important 17%
10. Resources and interventions being offered	Critical	Essential 56%; Very important 39%

Context and evaluation of academic detailing. Several themes emerged from the Delphi panel's consensus responses. All experts clearly agreed that academic detailing should be targeted toward clinicians and not to patients (consensus rating, 100%), with targeting of clinicians focused on practice location, practice specialties, and/or patient characteristics (consensus ratings, 83%-94%). In contrast to practice facilitation, care coordination, or technical assistance, panel members defined the core of academic detailing as providing education to clinicians (consensus rating, 95%). They noted that the goal of this education is to improve clinical performance (consensus rating, 89%), recommend practice changes (consensus rating, 100%), and offer decision support (consensus rating, 82%).

The panel agreed that evaluation of academic detailing should focus on changes in clinician behavior or performance (consensus rating, 100%), patient outcomes (consensus, 78%), resource utilization (consensus rating, 90%), and clinician knowledge (consensus rating, 84%), rather than on changes in practice workflow or efficiency. The impact of academic detailing programs on changes in the prevalence of diseases was considered less appropriate as a target outcome. Panel members did not reach consensus on whether healthcare cost should be an outcome in the evaluation of academic detailing.

Delivering and tailoring academic detailing interventions. The panel identified physicians or pharmacists as the individuals best qualified to perform the frontline work of academic detailing (consensus rating, 83%); rankings were lower for nurses, those involved in quality-improvement work, and public health workers. There was strong agreement that academic detailing should be tailored based on the clinical context of the practice or provider (consensus rating, 94%) and on specific barriers faced by the practice or provider (consensus rating, 100%)

Characteristics of academic detailing visits. Although the panel assigned high importance to the frequency of academic detailing visits, they disagreed about the optimal length of the intervention period (eg, time between first and last intervention), the duration of each visit, and the overall number of visits. In terms of training for academic detailing programs, the experts would like detailers to receive instructions on communication and interpersonal skills (consensus rating, 100%) to overcome anticipated clinician barriers to behavior change (consensus ratings, 94%-100%). The experts also believe that detailers should be provided with specific strategies and solutions to give to the targeted practice or provider (consensus rating, 95%).

Discussion

The findings of this Delphi panel process further expand the principles identified in an early report defining academic detailing.11 Academic detailing provides clinicians with useful and relevant information to change behavior in ways that improve patient outcomes. To achieve behavior change through academic detailing, the interventions need to maintain credibility with frontline clinicians.

Our findings suggest that individuals with relevant clinical expertise, such as physicians and pharmacists,33 can be very effective in promoting change by targeted providers. To promote active learner involvement, academic detailing messages should relate to the needs and interests of the provider and should identify barriers to evidence-based clinical decisions.

Furthermore, detailing programs need to provide concise decision support material to further encourage evidence-based clinical decisions, and detailers should be equipped with the skills needed to identify providers' baseline knowledge and motivation for prescribing patterns.

The inclusion of experts with varying institutional and operational experiences in different facets of academic detailing represents a particular strength of this process. We were able to achieve a high level of response throughout the entire Delphi process. Although we cannot find a published definition of what constitutes “consensus,” we chose relatively high thresholds for consensus: 70% (single Likert option) or 80% (combined Likert options).34

Despite this, our panel achieved agreement on nearly 90% of the survey items; the remaining 10% of items failed to achieve consensus after 3 rounds of the Delphi exercise. This is consistent with the literature indicating that most changes in survey responses occur within the first 2 rounds, and that little is gained in further iterations.35

Limitations

The limitations of our study should be kept in mind when considering its findings. Although our Delphi panel included experts from various countries and regions, many individuals involved in academic detailing have communicated or worked together at some point, and thus the experiences of the panel are unlikely to be completely independent of one another.

Furthermore, we only included participants from English-speaking countries, but there are academic detailing programs in Japan and Brazil and perhaps in other non–English-speaking countries.

In addition, because standard definitions are lacking for some terms used in our survey (eg, resource utilization, practice change, decision support), ambiguity may have existed about how to interpret these terms, and this may have influenced the choices made.

Conclusion

Our Delphi consensus exercise was undertaken to develop an accepted standard of what academic detailing should entail. Insights from our study, along with evidence from existing academic detailing literature and the experiences of those involved in academic detailing, should serve as the basis in formulating this framework. The results of the Delphi exercise can inform individuals (leaders and researchers) and organizations (institutions and agencies) involved in the development, implementation, research, or evaluation of academic detailing and provide a set of general principles for academic detailing initiatives.

Our Delphi panel of international experts reached consensus on several main themes related to academic detailing. The messages of academic detailing need to be tailored to provide strategies and solutions for clinicians to overcome barriers to evidence-based best practices. Evaluation of academic detailing programs should focus on changes in clinician knowledge and performance, improvements in patient outcomes, and utilization of resources.