Organizational Characteristics of High- and Low-Performing Anticoagulation Clinics in the Veterans Health Administration

Abstract

Objective

Anticoagulation clinics (ACCs) can improve anticoagulation control and prevent adverse events. However, ACCs vary widely in their performance on anticoagulation control. Our objective was to compare the organization and management of top-performing with that of bottom-performing ACCs.

Data Sources/Study Setting

Three high outlier and three low outlier ACCs in the Veterans Health Administration (VA).

Study Design

Site visits with qualitative data collection and analysis.

Data Collection/Extraction Methods

We conducted semi-structured interviews with ACC staff regarding work flow, staffing, organization, and quality assurance efforts. We also observed ACC operations and collected documents, such as the clinic protocol. We used grounded thematic analysis to examine site-level factors associated with high and low outlier status.

Principal Findings

High outlier sites were characterized by (1) adequate (pharmacist) staffing and effective use of (nonpharmacist) support personnel; (2) innovation to standardize clinical practice around evidence-based guidelines; (3) the presence of a quality champion for the ACC; (4) higher staff qualifications; (5) a climate of ongoing group learning; and (6) internal efforts to measure performance. Although high outliers had all of these features, no low outlier had more than two of them.

Conclusions

The top-performing ACCs in the VA system shared six relatively recognizable characteristics. Efforts to improve performance should focus on these domains.

Warfarin (Coumadin) is an oral anticoagulant given to millions of patients to treat or prevent blood clots. Warfarin requires routine blood tests to ensure that patients are sufficiently anticoagulated to prevent blood clots, but not so thoroughly anticoagulated as to place them at excessive risk of bleeding. Warfarin can be difficult to manage, and clinicians must carefully monitor and adjust doses to keep the patients in the target range. Management in a dedicated anticoagulation clinic (ACC) can improve anticoagulation control (van Walraven et al. 2006). Veterans Health Administration (VA) medical centers are required to manage all patients receiving warfarin in an ACC, which is generally run by clinical pharmacists (Veterans Health Administration 2010).

Nevertheless, even among patients receiving treatment in an ACC, there is substantial variation in anticoagulation control (Rose et al. 2011a). Many patients have poor control (van Walraven et al. 2006; Rose et al. 2010), which increases rates of stroke, venous thromboembolism, major hemorrhage, and death (Veeger et al. 2005; White et al. 2007; Wallentin et al. 2010). Anticoagulation control can be measured by the percentage of time in therapeutic range (TTR) (Rosendaal et al. 1993). We have proposed using TTR as a quality measure (Rose et al. 2009). In our recent study, mean site TTR varied among 100 VA ACCs by more than 20 percent, a large and clinically important difference (Rose et al. 2011b).

We sought to understand this variation in performance using the positive deviance approach (Bradley et al. 2009). This approach emphasizes in-depth qualitative study of organizations with exceptionally high performance to understand the factors that contribute to their excellence (Peters and Waterman 1982; Spear and Bowen 1999; Gawande 2007). Qualitative methods are ideal for studying organizational culture, which can be difficult to characterize using quantitative methods (Sofaer 1999; Patton 2002). The Anticoagulation Forum has proposed nine key domains (Table 1) that are essential to establishing and maintaining a high-quality ACC (Garcia et al. 2008). We sought to explore these nine domains, while remaining open to others that would emerge through this qualitative study.

Table 1.

Nine Recommendations for Optimal Anticoagulation Care as per the Anticoagulation Forum Consensus Statement (Garcia et al. 2008)

Domain	Explanation
1. Qualifications of personnel	ACC staff should be licensed health care professionals possessing core competency related to anticoagulation therapy.
2. Supervision	The ACC should have a well-defined relationship with responsible/referring health care providers (e.g., the primary care physician).
3. Care management and coordination	The ACC should have written policies and procedures that are approved by the clinic's medical director. An efficient system for scheduling and tracking patients should be utilized.
4. Documentation	The ACC should have an accurate system of documentation to ensure that clinically relevant data are available to staff at all times.
5. Patient education	The delivery of anticoagulation care should address the educational needs of patients and their caregivers.
6. Patient selection and assessment	Optimized anticoagulant therapy should be instituted only after careful consideration of the risk and benefit for an individual patient. The appropriateness of anticoagulation therapy should be periodically reviewed for each patient.
7. Laboratory monitoring	Optimized anticoagulation therapy should incorporate regular laboratory monitoring of anticoagulant effect, using either a high-quality laboratory or a well-maintained point of care device.
8. Initiation of therapy	The initiation of optimized anticoagulation therapy should use a systematic, evidence-based approach.
9. Maintenance of therapy	The delivery of optimized anticoagulation should use a systematic process for longitudinal patient assessment, adjustment of anticoagulant drug doses, scheduling of follow-up visits, and interruptions of therapy for elective procedures.

Methods

Selection of Study Sites

Based on our rankings of risk-adjusted anticoagulation control at 100 VA sites (Rose et al. 2010, 2011a), we selected 3 of the top 10 and 3 of the bottom 10 sites, and obtained IRB approval from all study sites. Sites were not informed of their high or low outlier status. Although we selected sites based on their performance from October 2006 to September 2008 (the “measurement period”), site visits occurred between May 2010 and March 2011. We took care to identify personnel and organizational factors that had changed since the measurement period. To a large extent, these factors and personnel had not changed, but when they had, we viewed this as an additional opportunity to examine not only static models of high and low performance but also changes in performance. When conditions as we found them differed in important ways from what was present during the measurement period, we have noted this in the text.

Data Collection

Interviews

One author (A. J. R.) visited each study site for two working days. At each site, we sought to interview all staff members who work in the ACC or whose work impacts the ACC. All employees agreed except one, and all participants gave informed consent. Each participant was interviewed in private for 20–60 minutes using a semi-structured interview technique. Interview topics were based largely upon the Anticoagulation Forum recommendations (Garcia et al. 2008) and included ACC work flow, staffing, organization, and quality assurance efforts. We also asked about the clinic's patient population, the challenges of managing anticoagulation, and the participant's opinion of the clinic's performance. All interviews were audio-recorded and transcribed verbatim.

Observation

At each site, we observed approximately 4 hours of direct clinical care, which occurred face-to-face or by telephone. We observed delivery of care, work environment, work pace and flow, patient-provider interactions, and interactions among pharmacists and pharmacy technicians working in the ACC.

Documents

We obtained ACC-related documents, including ACC protocols, note and consult templates from the electronic medical record, training manuals for ACC staff, education materials for non-ACC staff, patient education materials, patient-provider treatment agreements, form letters sent to patients, quality assurance forms and reports, and screen shots of clinical management software used by the ACC.

Data Analysis

Using grounded thematic analysis, we coded interview transcripts, identifying broad “domains” of ACC organization and subtopics within each domain (Patton 2002). Having coded the interview transcripts, we added information gleaned from the field notes and the ACC documents. We synthesized the data from each site and produced a profile of each site's organization and management. As successive study sites were added, we used constant comparison to contrast our findings from various sites, noting similarities and differences between high- and low-performing sites. We made a conscious effort to note counterexamples, particularly positive aspects of bottom-performing sites and negative aspects of top-performing sites. We identified key features present or absent at each site, noting relationships between these features and site performance. Finally, we performed internal member checking among our co-authors, many of whom have direct experience managing oral anticoagulation, to ensure that our findings resonate with anticoagulation practitioners.

Results

The six sites we visited represented six different states and all geographic regions of the United States. Details about site performance are located in Table 2. Table 3 lists the job descriptions of the 55 interviewees.

Table 2.

Three High Outlier and Three Low Outlier Sites, as Measured by Performance on Percent Time in Range (TTR), a Measure of Anticoagulation Control

Site Number	Outlier Status	TTR, 2007–2008 (%)
1	High	67
2	High	67
3	High	70
4	Low	52
5	Low	54
6	Low	54

Notes. Although Sites Were Selected for Inclusion Based on Risk-Adjusted TTR (Rose et al. 2011b), Unadjusted TTR Is Presented Here for Ease of Understanding. TTR was computed using the Rosendaal method (Rosendaal et al. 1993). TTR above 60 percent is generally considered adequate, whereas TTR above 70 percent is generally considered excellent.

Table 3.

Type of Staff Interviewed at Study Sites

	Interviews, n
Direct-care ACC staff
Pharmacist/ACC coordinator	5
Pharmacist, not ACC coordinator	19
Pharmacy resident	1
ACC support staff
Pharmacy technician	3
Clerk/secretary	2
Nurse	1
Pharmacy administration
Chief of pharmacy	5
Associate chief of pharmacy	4
Middle manager, pharmacy	2
Physicians (ACC medical director)
Staff physician	1
Chief of primary care	3
Chief of medicine	1
Chief of cardiology	1
Other staff
Laboratory supervisor	4
Medical director of laboratory	1
Phlebotomist	1
Nurse practitioner, cardiology	1
Supervisor of clerks	1
Total	55

We identified 11 domains related to site organization; six were related to site performance and five were unrelated (Table 4). Although high outliers had all six of the features related to performance, no low outlier had more than two of them. The important domains were (1) adequate (pharmacist) staffing levels and effective use of (nonpharmacist) support personnel; (2) innovations to standardize clinical practice around current evidence and guidelines; (3) the presence of a quality champion; (4) higher staff qualifications; (5) a climate of ongoing group learning; and (6) internal performance measurement.

Table 4.

Summary of Six Anticoagulation Clinics on Six Domains Related to Site Performance and Five Domains Not Related to Site Performance

	High Outliers			Low Outliers
	Site 1	Site 2	Site 3	Site 4	Site 5	Site 6
Domains related to performance
Staffing	✓	✓	✓
Innovation	✓	✓	✓		✓
Champion	✓	✓	✓
Qualifications	✓	✓	✓			✓
Group learning	✓	✓	✓			✓
Measurement	✓	✓	✓
Domains not related to performance
Telephone clinic		✓	✓			✓
Commitment	✓	✓	✓	✓	✓	✓
Integration	✓	✓	✓	✓	✓	✓
Patient selection
Tenure		✓	✓	✓	✓	✓

Key to table:

✓ = present during measurement period of 2007–2008.

No symbol = absent during measurement period of 2007–2008.

Key to domains:

Staffing: Sufficient staffing to handle workload without rushing.

Innovation: Innovations to encourage evidence-based practice.

Champion: Presence of one or more quality champions for the ACC.

Qualifications: Hiring residency-trained pharmacists.

Group learning: Creating a climate of ongoing group learning.

Measurement: Internal performance measurement.

Telephone clinic: Majority of patients managed via telephone rather than face-to-face.

Commitment: Demonstrated commitment to serving veterans.

Integration: Effectively leverages the advantages of membership in an integrated health system.

Patient selection: Selectively refusing to treat difficult or troublesome patients (gaming).

Tenure: Clinic established more than 20 years ago.

Domain #1: Staffing

We observed, and were told by interviewees, that ACC staff at the low outlier sites had a rushed work pace and a chaotic work environment, whereas staff at the high outliers worked at a comfortable pace in a well-organized environment. Based on expert opinion, consensus guidelines for organizing an ACC recommend a ratio of no more than 400 patients per pharmacist full-time equivalent (FTE) (Ansell et al. 2008). Details about staffing at our six study sites are in Table 5. In general, the adequacy of staffing was not fully characterized by the patient/FTE ratio but also depended on the effective integration of support staff into the ACC, the degree to which staff could focus on their ACC duties without other responsibilities, and avoiding an inappropriate reliance upon trainees (pharmacy residents) to provide manpower, as will be discussed below.

Table 5.

Details for Clinic Staffing in Six VA ACCs

Panel Size		FTE	Patient/FTE Ratio	Support Staff	Details
High outlier sites
Site 1	550	1.5	375	Specially trained clerks	ACC providers have considerable time for administrative tasks, much of which is also devoted to the ACC
Site 2	1700	4	425	2 pharmacy technicians	Workflow is highly streamlined and is aided by clinical management software
Site 3	250	1.5	167	None
Low outlier sites
Site 4	600	2	300	None	Pharmacists not dedicated to ACC; distraction may reduce effectiveness
Site 5	1000	1	1000	None	Approximately 1 additional FTE of resident labor in ACC
Site 6	800	1.5	533	1 pharmacy technician

Notes. Panel Sizes Were Determined from VA Automated Data (Rose et al. 2011b), whereas Pharmacist FTEs Assigned to the ACC during the Measurement Period (10/06-9/08) Were Determined through Interviews and Direct Observation. A Patient to FTE Ratio of No More Than 400 Is Generally Recommended by Consensus Guidelines (Ansell et al. 2008).

ACC, anticoagulation clinic; FTE, full-time equivalents (pharmacists, not counting support staff); VA, Veterans Health Administration.

Pharmacists

Anticoagulation clinic staff at low outlier sites consistently reported being rushed due to inadequate staff allocations; we also observed this rushed work pace. In contrast, staff at high outliers only rarely reported mild issues with staffing related to vacations and sick leave. Understaffing at low outliers was manifested through frequently overbooked appointments, chaotic work days, and sagging morale. An interview with one pharmacist revealed the stress prevalent at the low outlier sites:

Q: Are you pretty happy with how your job here is going?

A: I'm overwhelmed…over the years, our numbers have grown … and no one has really looked at our workload and I think we're really stretched. (Pharmacist, Low Outlier)

A pharmacist at another low outlier suggested that being short-staffed may prevent them from following some patients as frequently as needed, a potentially serious issue with quality of care:

Sometimes we need closer follow-up, but we don't do closer follow-up ‘cause we don't have enough help. So I think we're doing the best we can with what we've got. (Pharmacist, Low Outlier)

By contrast, ACC providers at the top sites had fewer patients per FTE, which was reflected in their work pace. One participant noted:

I don't think we're capacity most days. I don't think we have 40 face-to-face visits most days, but that's good, because that allows us to see patients more frequently if necessary. There are still appointments left to see those patients without overbooking. (Associate Chief of Pharmacy, High Outlier)

Support Staff

Adequate staffing also relates to the role of support staff, individuals who work in the ACC but do not have the authority to prescribe warfarin. When present, pharmacy technicians were universally lauded as contributing to the smooth functioning of the ACC. For example, at one high outlier, four FTE of pharmacists were supplemented by two FTE of pharmacy technicians, who handle incoming calls, distribute workload, and mail letters, thereby freeing the pharmacists to concentrate on higher level tasks, such as managing patients who are out of range. However, personnel other than pharmacy technicians can also provide effective support. At another high outlier, only the most experienced clerks are given special training and permission to schedule appointments for the ACC. Low outliers were generally characterized by ineffective approaches to support staffing: either untrained clerks or, at two sites, no support staff at all.

Dedicated Time for ACC Duties

At most study sites, ACC pharmacists had dedicated days when they staffed the ACC without other obligations. In contrast, at one low outlier, ACC pharmacists were simultaneously responsible for anticoagulation and other tasks related to primary care pharmacy. Pharmacists at this site were frequently interrupted from ACC duties to attend to non-ACC matters.

For example, we observed a pharmacist from this low outlier site treating a patient who had suffered a blood clot in the lung. The patient was hard of hearing, possibly suffered from dementia, lived in a trailer, and did not fully understand or accept the need for anticoagulation. The visit was interrupted six times by people walking into the room (often without knocking) to ask the pharmacist questions in her role as a primary care team member. Each interruption caused a loss of focus and momentum, and the overall effect was to prolong the visit and to transform an already difficult encounter into an almost insurmountable challenge.

Avoiding Inappropriate Use of Pharmacy Residents

At most of the sites we visited, pharmacy residents were in the ACC primarily to learn and were not expected to contribute greatly to staffing due to their inexperience and need to be supervised. However, at one low outlier site, residents were expected to be about as productive as the senior pharmacists. They handled about half of the workload of the ACC and we were explicitly told that they constituted an important part of the staffing plan. A pharmacy resident from this site, who had recently rotated through the ACC for a month, remarked:

I feel like the pharmacy residents do a lot here … just because there are so many patients … I'm interviewing patients in 10 or 15 minute increments, very fast. (Pharmacy Resident, Low Outlier)

The fact that half the care at this site was delivered by pharmacy residents may have contributed to uneven quality of clinical management. A pharmacist at this site admitted that the quality of the pharmacy residents can vary from day to day and from month to month: “It's hit and miss with them.” This does not suggest that these residents or their training are lacking, merely that they need to complete residency before so much can be expected of them.

Domain #2: Innovations to Encourage Uniform, Evidence-Based Practice

Top sites developed innovative ways for encouraging uniformity of practice. In general, the organizing principles for standardization were drawn from existing clinical guidelines about anticoagulation care, especially the American College of Chest Physicians (ACCP) guidelines for managing warfarin (Ansell et al. 2008). This domain, with its emphasis on the systematic pursuit of uniform, evidence-based practice, is also embodied in the Anticoagulation Forum Consensus Guidelines for organizing an ACC (Garcia et al. 2008). We noticed at least three examples of how sites pursued uniform, evidence-based practice: developing note templates as an aid to clinical reasoning and documentation, adopting new software packages to enhance workflow, and developing systems to reduce loss to follow-up.

Evidence-Based Note Templates

One way to reinforce evidence-based practice is to design note templates around it. Although all of the sites had note templates, some were especially effective at encouraging adherence to clinical guidelines. For example, one site's ACC Coordinator created a note to guide clinicians through the decision-making process of how to manage a temporary interruption of warfarin therapy for a procedure. Although this was a low outlier site, this note was introduced after the measurement period as part of an effort to improve the ACC.

Previously … if we heard about a patient … stopping [warfarin] for a procedure we would kinda wait and see if somebody had the idea of using bridge. I standardized that process … [now] if we hear a patient's stopping Coumadin he's gonna do it through a standardized review process that's based strictly on the [ACCP] guidelines. I created a note template that quotes what the guidelines say … and then we walk ‘em through the peri-procedure risk assessment note which gets documented in the medical record and is sent to the doctor. (Pharmacist, Low Outlier)

Adopting New Software to Aid Clinical Practice

One high outlier site used a software package to help streamline anticoagulation management. Participants commented, and we observed, that the software contributed to this site's ability to handle a large patient load without compromising quality. The software improved work flow, helped make sure patients were not lost to follow-up, and allowed for internal performance measurement with a minimum of effort. A low outlier site also started using this software package in 2010 (after the measurement period) with similar positive results.

Reducing Loss to Follow-Up

It is important not to lose ACC patients to follow-up, because without proper monitoring, patients may be at risk for serious adverse events. All six sites had various techniques to avoid losing patients—the most common was to limit warfarin prescriptions to a short period of time (often 30 days), to ensure that the patient must follow-up. However, some sites went farther. At one high outlier, the ACC Coordinator “runs the list” monthly to manually search for patients who have not been seen for over 30 days (a laborious process). The two sites that had adopted the new software package easily performed this task, because the software monitors this automatically. One low outlier had no system to prevent loss to follow-up:

Actually we don't have a system in place for that. Now, if we could run our clinics quarterly or have some kind of computer program where if a person hasn't been to see us in say, two months, then we could follow it, but right now we don't have a way of tracking ‘em down. (Pharmacist, Low Outlier)

Note that this pharmacist recognizes the potential for manually running the list or a “computer program,” elements that are present at the high outlier sites.

Domain #3: Quality Champion

The best sites had one or more strong quality champions for the ACC, although this champion could occupy many different formal positions. At one high outlier, this champion was the Associate Chief of Pharmacy, also the ACC's founder. She wrote all the procedures for the ACC, trained her replacements, and continued to take a special interest in the ACC's performance. At another high outlier, this champion was the ACC Coordinator, while at a third, the main quality champion for the ACC was the Chief of Pharmacy. In the following quote, a quality champion explains her role with regard to the ACC:

I would say my main thing is to make sure that I really kinda do the quality aspect…make sure that the…care that we're providing is appropriate. That if we're gonna offer a program, that it's doing what it needs to do, and if it isn't then what do I need to do to fix it. (Chief of Pharmacy, High Outlier)

None of the low outliers had a quality champion. For example, one low outlier had a middle manager in the pharmacy department whose job was to oversee credentialing, peer reviews, academic pursuits, and quality assurance for pharmacists. However, he had almost no role with regard to the ACC pharmacists, because they were located in primary care rather than pharmacy. Instead, these pharmacists were supervised by the director of primary care, a physician. Although she has made her primary care clinic into a model of performance, she saw the pharmacists primarily in their role as supporting cast for the primary care clinic and was relatively unaware of the pharmacists as the central movers in managing anticoagulation. Thus, there were two people qualified to serve as the quality champion for the ACC (the pharmacy middle manager and the ACC medical director), but one was not given the authority to do so and the other did not see it as part of her job.

Domain #4: Staff Qualifications

Many pharmacists complete a 1-year residency after obtaining their PharmD degree. The purpose of residency is to gain a set of skills in a mentored setting, especially the ability to interact with patients to directly manage chronic conditions. All of the ACC pharmacists at the top outlier sites had completed residencies in pharmacy. In contrast, none of the pharmacists at two of the low outlier sites were residency trained.

We observed important differences between pharmacists with and without residency training. The residency-trained pharmacists had greater facility communicating with patients. The pharmacists that lacked residency training struggled to handle difficult situations, often despite years of experience. At one site, we did observe a nonresidency trained pharmacist who was instead completing a protracted apprenticeship in the ACC. This suggests that other kinds of training can also help pharmacists to achieve a similar degree of facility with the skills needed for direct patient management.

Domain #5: Climate of Group Learning

Clinicians frequently discuss difficult cases to solicit the opinions and insights of their colleagues. At the high outlier sites, such interactions were explicitly encouraged, and they occurred frequently:

So it's good to have other people next to you in the room you can bounce ideas off of. (Pharmacist, High Outlier)

These interactions were largely absent from the low outlier sites; providers at these sites tended to practice without the benefit of their colleagues' opinions. When asked about how often she seeks advice from colleagues, this pharmacist said:

Not very often. I would probably have to say less than once every 3 months. I would say we don't tend to collaborate a lot. (Pharmacist, Low Outlier)

We directly observed a marked contrast between low and high outliers with regard to the frequency and quality of discussions of difficult cases. The availability of colleagues for discussion at the high outlier sites seemed to contribute not only to the management of the case at hand but also to an atmosphere of learning from each other.

Domain #6: Internal Performance Measurement

The top-performing sites emphasized internal performance measurement. One high outlier manually collected data at every patient encounter (a laborious task) to be able to calculate and track percentage of lab values in range and rates of adverse events in their ACC. The Associate Chief of Pharmacy, when asked about the strengths of his site's ACC, demonstrated detailed awareness of its performance:

About two-thirds of our patients on average are within the therapeutic range. Our no-show and cancelation rate typically combined is less than 12 to 15 percent … thromboembolism is usually less than two percent, and major bleeding is usually less than 5 percent. (Associate Chief of Pharmacy, High Outlier)

At other sites, anticoagulation software or regionally produced performance reports made performance measurement particularly easy to accomplish. One high outlier site began using such software well before our measurement period. A low outlier site recently adopted the same software. The ACC coordinator at this site commented:

When we started using software as of January 2010, we kinda changed our QA process. We were now able to calculate our TTR or actually get reports from that from the software … we look at … thrombotic events, bleeding events … patients lost to follow-up … When the software took over we started doing that more routinely … it's something that has become a part of our practice. (Pharmacist, Low Outlier)

Low outlier sites did not measure performance in 2007–2008, although two of them have begun doing so since then. At the third low outlier, the ACC medical director admitted that they do not yet measure ACC performance in a rigorous, proactive way. When asked about ACC performance, she appeared to be unprepared for the question and thus uncomfortable with the topic. She focused on one-time episodes such as complaints or adverse events as a way to measure quality:

You mean in terms of bad outcomes with the patients or in terms of? [Clears throat] Yes we definite—that, I would know [Clears throat]. We do have adverse outcomes. It's something that we look at. So if … a patient comes in with [a critical degree of overanticoagulation] and nobody did anything about it, I would know about this … if there's an adverse occurrence, then I'm told about this. (ACC Medical Director, Low Outlier)

In the absence of quality measurement, ACC pharmacists at this site believed that their clinic was doing very well—but in fact, their TTR was among the lowest of any VA site. One pharmacist guessed that her patients spend 70 percent of time in range, whereas another guessed 75 percent. In fact, TTR at this site was closer to 55 percent, a large and clinically important difference.

Five Domains Not Related to ACC Performance

There were at least five domains that were not associated with ACC performance, although we had expected that they might be (Table 4).

1. We had expected to find that quality might vary due to the configuration of the ACC, that is, telephone versus face-to-face management of patients and the use of point-of-care (fingerstick) testing versus reliance upon venous blood samples. We observed highly variable configurations of care among our six sites. For example, three sites had predominantly telephone clinics and three saw their patients face-to-face, but neither model was associated with high or low outlier status.

2. All sites, whether low or high outliers, exhibited a strong commitment to serving veteran patients. Staff went to great lengths to help patients, contacting them after hours when necessary and even assisting them with matters unrelated to anticoagulation.

3. The VA's integrated system of care, and particularly the electronic medical record, are thought to contribute to the VA's generally high performance (Longman 2007). However, we did not notice any differences among sites regarding the extent to which they utilized these advantages.

4. We had suspected that high outlier sites might achieve better performance in part by being more selective in the patients they agree to manage (gaming). However, all sites went to extraordinary lengths to manage even the most challenging patients.

5. We had expected that ACCs established long ago might perform better than those established recently; in fact, only one of our ACCs was established less than 10 years before our site visit, and it was a high outlier.

In addition to these five domains, it should be noted that the nine domains identified as important by the Anticoagulation Forum (Garcia et al. 2008) generally were not sufficient to separate high- and low-performing sites (Table 1). Exceptions included Item 3, where high outlier sites generally had better systems for tracking patients and preventing loss to follow-up, and aspects of some of the other items, in that high outliers achieved greater uniformity of practice. In most respects, however, high and low outliers fulfilled the Anticoagulation Forum recommendations equally.

Discussion

In this study, we visited three of the best-performing and three of the worst-performing ACCs in the VA system. We sought to understand how elements of ACC organization and management related to these differences in site-level performance. We found that six critical domains separated high and low outlier sites. Although we had anticipated some of these domains, our investigation revealed details that we had not anticipated. For example, while we had suspected that staff credentials might be important, we had assumed that all ACC pharmacists would be residency trained, and that those at the top sites would be distinguished by additional credentials. We discovered the importance of residency training through its absence among staff at some low outlier sites and its apparent impact on their fluency as clinicians.

We also found that the recommendations of the Anticoagulation Forum (Garcia et al. 2008) were not sufficient to separate high outliers from our three low outliers. To a large extent, all six study sites had met all of these criteria. It is apparent, therefore, that the pursuit of excellence in oral anticoagulation care will require a comprehensive program of quality measurement and quality improvement (Rose et al. 2009).

In a previous study, we examined many of the same domains, including staffing, presence of quality improvement programs, and training of ACC staff but were unable to link them to site-level performance (Rose et al. 2011b). The previous study relied upon a site-level questionnaire to gather data, but complex ideas like adequate staffing may not easily be encapsulated in a single number. In the present study, we examined not only the FTE ratio but also the context and configuration of care delivery, the workflow, and the importance of support staff in allowing pharmacists to focus on higher level tasks. The present study suggests that adequate staffing is indeed important but that a simple numerical ratio of pharmacists to patients may not fully capture all the data needed to understand this issue.

Anticoagulation clinics improve anticoagulation control and usually reduce adverse events compared with “usual care” (van Walraven et al. 2006; Ansell et al. 2008), but less is known about how ACC management improves these outcomes. Our study suggests that an ACC should be characterized by a core group of well-qualified and well-trained staff, dedicated time to focus on anticoagulation (without other responsibilities), development of innovations to improve care, opportunities to discuss difficult cases and learn from each other, internal performance measurement, and a quality champion, whose role it is to facilitate all of these other things. To the extent that our low outlier sites lacked these features, they did not fully embody the ideal of an ACC. It should not be surprising, therefore, to find that anticoagulation control at these low outliers was similar to what has been reported for non-ACC settings (van Walraven et al. 2006).

Our study also suggests that building time into schedules for the purpose of problem solving and knowledge sharing can increase the value created per person. Discussion of difficult cases allows for group learning and may even encourage group solutions to frequently encountered problems. In contrast, while providers at low outlier sites may learn from challenging cases as individuals, such learning is unlikely to spur practice innovations. It may not be necessary to add additional people to set aside time for these activities; rather, by emphasizing these activities, an ACC can foster a climate of efficiency in which people simply accomplish more and do it better.

Insights gained from this study are not limited to ACCs alone. In fact, Curry et al. found remarkably similar themes that differentiated between hospitals ranking in the top and bottom 5 percent on risk-adjusted mortality after acute myocardial infarction (Curry et al. 2011). Thus, there may be considerable overlap between the ingredients of success in a large program (such as treatment of myocardial infarction) and a small program (such as management of anticoagulation).

This study has important strengths. Our in-depth observation allowed us to uncover previously unsuspected findings, as discussed above. However, some limitations should also be noted. First, we were unable to gain access to some of the worst-performing ACCs in the VA system, primarily because those sites did not have an IRB of record, and therefore could not participate in research. We might have gained further insight had we been able to visit low outlier sites with worse performance. Second, a considerable amount of time elapsed between our measurement period and our site visits. This was unavoidable because of the time needed to identify outlier sites and obtain IRB approvals. Third, due to time and resource constraints, we were only able to visit six sites. We might have learned more if we had visited additional sites. Fourth, we realize that local circumstances may contribute to some of the suboptimal features we observed, such as a heavy reliance on pharmacy residents, high patient-FTE ratios, or a lack of residency-trained pharmacists. Strategies for addressing local resource limitations are beyond the scope of our study. Finally, the site visitor was not blinded to outlier status, and the analysis of data was also not blinded. However, we consciously tried to note what was suboptimal about high outlier sites and what was good about low outlier sites.

In summary, we found that six domains of organization and management are related to ACC performance. Clinics have the potential to transform themselves, and their performance, by focusing on changing these six domains for the better. This study reminds us that excellence is within our reach if we focus on the most important determinants of performance.

SUPPORTING INFORMATION

Additional supporting information may be found in the online version of this article:

Appendix SA1: Author Matrix.

Please note: Wiley-Blackwell is not responsible for the content or functionality of any supporting materials supplied by the authors. Any queries (other than missing material) should be directed to the corresponding author for the article.