Development of explicit criteria to measure adherence to hypertension guidelines

Abstract

Measures of adherence to hypertension guidelines have historically been based on prescription data or physician survey data regarding treatment practices. These methods have limitations that decrease their accuracy. As part of a randomized controlled study testing the effects of pharmacist/physician collaboration on adherence to hypertension guidelines, the investigators and an expert panel developed a JNC 7 measurement tool. The final guideline adherence measurement tool includes 22 explicit criteria in four domains of care. An exploratory factor analysis, conducted to assess the structure of the tool, suggests three underlying treatment dimensions in hypertension care. The adherence measurement tool will allow researchers to link specific elements of care to improved blood pressure control. In addition, use of the tool will provide clinicians with a taxonomy for evaluating practice and describing the effect of improved patient care on patient outcomes.

Introduction

Measuring the degree to which processes of care follow evidence-based practice recommendations is essential to quality assessment.^1-4 Evaluation of guideline adherence can provide insight regarding processes of care that are weak, and thus provide a reference for quality improvement. Historically, measures of quality in hypertension management have shown overall adherence to blood pressure guidelines to be low. However, several methodological limitations have hindered earlier assessments of adherence.⁵ Many studies have focused on pharmacologic treatment and neglected to address the complete range of processes that comprise hypertensive care, including diagnosis, non-pharmacologic interventions and follow-up.^6-10 Furthermore, studies have not consistently utilized quantifiable measures of adherence or accounted for instances where non-adherence to guidelines might have been justifiable.^11-14

A comprehensive study conducted by Asch et al.¹¹ avoided many of the weaknesses of other studies. Specifically, this study measured quality of care provided to hypertensive women using explicit indicators of care quality and related these indicators to blood pressure control. The Asch indicators, which addressed a range of hypertensive care processes, were supported by evidence from the JNC VI guidelines and from clinical trials. However, the Asch study focused on quality of care rather than on guideline adherence. In addition, the seventh report of the Joint National Committee (JNC 7) on the Prevention, Detection, Evaluation and Treatment of High Blood Pressure¹⁵ has provided new recommendations for hypertension management. It is important to develop criteria to measure adherence to these standards.

The primary goal of this study was to develop an explicit tool precisely for measuring guideline adherence. The objectives of the current project were to (1) define a comprehensive set of reliable and valid process of care criteria reflecting the JNC 7 hypertension practice recommendations and (2) derive a scoring method for these criteria that would allow for a comprehensive and quantitative characterization of physician adherence (Table 1).

Table 1.

What is known about this topic Overall, studies have demonstrated that physician adherence to guidelines is poor5 Previous evaluations of guideline adherence have focused on pharmacologic treatment A complete range of processes that comprise hypertensive care including diagnosis, non-pharmacologic interventions and follow-up, have not historically been included in measures of adherence

What this study adds The adherence tool includes a comprehensive set of explicit process criteria that can be used to evaluate guideline adherence The adherence tool provides a mechanism for quantifying adherence to guidelines, enabling researchers and clinicians to systematically evaluate practice and describe the effect of improved patient care on patient outcomes

Materials and methods

The explicit criteria described herein were developed as part of an ongoing prospective, multicentre, randomized, controlled study (NIH: HL070740) designed to measure the effects of pharmacist/physician collaboration on adherence to hypertension guidelines within family medicine offices. A process modified from Ashton et al.¹ was used to develop and score explicit criteria for a guideline adherence tool. The following steps were used to help ensure the selection of reliable, valid and explicit process-based criteria: (1) formulating the criteria, (2) refining the criteria, (3) weighting the criteria, (4) selecting a scoring method and (5) data collection.

Formulating the criteria

An initial list of 30 hypertension care processes was compiled by the study investigators from evidence-based practice recommendations in the JNC 7 guidelines. The goal was to develop a set of criteria that would allow for an overall evaluation of adherence to JNC 7. Therefore, processes of care related to the diagnosis, treatment, monitoring and follow-up of hypertension were included. In addition, the criteria reflected selected attributes used by Asch et al.^11,16 and the Institute of Medicine (Table 2) to evaluate explicit criteria.

Table 2.

Attributes used to evaluate explicit criteria

Degree to which the selected process is based on published evidence Relationship to the outcome of improved blood pressure control Importance to practice Applicability to multiple clinical settings Appropriateness of provider accountability Ability to justify instances of nonadherence

Refining the criteria

Three physicians, Dr Henry Black, Dr George Bakris and Dr Daniel Jones, recognized experts in hypertension, co-chairs of the JNC 7 writing team, and members of the National High Blood Pressure Education Program (NHBPEP) Coordinating Committee, served as an expert panel for development of the tool. The expert panel received the initial draft of 30 explicit criteria via email, giving panel members an opportunity to form individual opinions regarding items included in the tool. The panel subsequently met face-to-face on three different occasions with the study investigators in order to evaluate individual criteria for face and content validity, recommend modifications to the criteria list and review the performance of the explicit criteria as a tool for measuring adherence.

At the initial meeting, the panel discussed each criterion using a modified nominal group method, in which a structured discussion provides a procedure for gathering qualitative information.¹⁷ The expert panel was asked to evaluate each individual criterion and to reach a consensus recommendation regarding the item's status on the tool. Some items were retained without change, others were retained with modification and some were eliminated. Members of the expert panel also suggested additional criteria for inclusion. Detailed notes outlining the discussion, recommendations and conclusions were recorded, and the tool was subsequently modified by the investigators according to the expert consensus. A refined list of criteria was then redistributed to the expert panel via email to again give each expert the opportunity to independently review the revised criteria.

During the second meeting, the panel was given the opportunity to discuss, approve, modify or eliminate each individual criterion in the revised tool. The list was again edited to reflect the expert panel's opinion. At the third meeting, the expert panel judged that four of the criteria that were initially included in the list were not clearly recommended in the JNC 7 guidelines. Consequently, these criteria were eliminated. The remaining 22 criteria were grouped into four general domains of hypertension care based on their clinical commonalities: diagnosis, drug therapy, follow-up intensity and laboratory monitoring.

Weighting the criteria

Weighted process criteria have been used in previous evaluations of quality of care.^18-20 Weighting criteria give certain processes priority over others based on their impact on outcomes. Weighting allows for a scoring adjustment based on the quality of evidence and reinforcement of certain aspects of clinical process.^20,21 Members of the expert panel judged some criteria as essential elements of care that were deemed critical to an overall measure of adherence and other criteria as important but less critical indicators of adherence. This judgment led to the decision to weight criteria for the purpose of scoring. During their second meeting, the expert panel assigned each criterion a weight of either 1 or 2 (2 being essential) based on its importance to an evaluation of adherence.

Scoring the adherence tool

Once the list of weighted, explicit criteria were finalized, a detailed algorithm was developed for scoring practice data, with the goal of eliminating subjectivity. Scoring entails two basic steps. First, the algorithm assigns a score to each individual criterion. Some of the 22 criteria are applicable to all persons being treated for hypertension. The algorithm scores these criteria as either ‘met’ (1) or ‘unmet’ (0). Other criteria are applicable only to particular subpopulations (e.g., persons with specific co-morbid conditions) or at specific points in time during hypertensive care (e.g., when blood pressure is not controlled to goal). For patients for whom one of these criteria applies, the algorithm assigns a score of ‘met’ (1) or ‘unmet’ (0). For patients for whom the criterion does not apply, the algorithm assigns a score of ‘non-applicable’ (9). Criteria in the diagnosis, drug therapy and general lab domains are scored only once across a specified time period of data abstraction to ensure that a sufficient stretch of time has elapsed for work related to these criteria to be completed and recorded. In contrast, criteria in the follow-up domain are scored for multiple visits in a specified time period. All visits where hypertension was addressed in the physician progress note are assigned a separate score for each of the follow-up criteria. This scoring procedure allows us to evaluate the overall intensity of hypertension care delivered over multiple visits, providing a better measure of adherence than would evaluation of adherence at a single visit. Visits where hypertension was not addressed in the physician progress note, such as an acute care visit, are not scored for the follow-up domain criteria. This method of scoring prevents physicians from being penalized undeservedly, as scoring is not negatively impacted by visits where hypertension was not a major concern.

The second step in scoring is calculation of an overall adherence score. The overall score is compiled by dividing the total number of explicit criteria successfully met (marked as 1) during care for an individual patient by the total number of criteria applicable to that patient (marked either 1 or 0). Non-applicable criteria are not included in the overall adherence score calculation. Domain scores, in addition to an overall adherence score, can also be calculated by dividing the total number of explicit criteria in a domain that are successfully met by the total number of criteria in the domain that are applicable to that patient.

Study subjects

The tool was piloted on a group of 311 patients with uncontrolled hypertension participating in our ongoing study. Patients with uncontrolled blood pressure were eligible for the study if they were over 21 years of age and currently taking one to three blood pressure medications with no change in the regimen or dose within the 4 weeks before enrolment. Patients were excluded from the study if they had blood pressure $\ge 180∕110\mathrm{mm}$ Hg, myocardial infarction or stroke within 6 months before enrolment, New York Heart Association Class III or IV heart failure, unstable angina, serious liver disease (defined as liver enzymes $\ge$three times the upper limit of normal), serious kidney disease (defined as $\ge 1\mathrm{g}$ of proteinuria daily), pregnancy, life expectancy estimated less than 3 years or dementia or cognitive impairment. Study patients received care at one of six community-based outpatient family medicine clinics in the state of Iowa. Each clinic is owned and operated by the community hospital and their respective medical education foundation. These clinics are all affiliated with the family medicine training programme and are staffed by faculty with clinical appointments with the University and residents. Each clinic maintains separate medical records.

Data collection

A detailed data collection instrument was created for recording data needed to evaluate adherence to the explicit criteria. The entire medical record was reviewed for information regarding past medical history, social history, family history, physician's blood pressure goal for the patient, laboratory or diagnostic test results, adverse drug reactions, current drug therapy and clinic blood pressure values. When multiple blood pressure measurements were taken at each visit, the lowest documented measurement was abstracted. At each visit with the physician where hypertension was addressed (either defined as the chief complaint, or mentioned in the progress note assessment or plan), information regarding drug therapy changes, lifestyle modification recommendations and physician acknowledgement of uncontrolled blood pressure was collected.

The investigators developed a detailed manual of instructions for abstracting data and completing the data collection instrument. In our ongoing study, research nurses were trained in the study protocol and data abstraction process during six 90 min interactive workshops broadcast over the Iowa Communications Network (ICN), a statewide fibre-optic telecommunications network that allows for two-way video and audio conferencing. In addition, study personnel visited with each research nurse on site to address questions regarding the abstraction process and to perform accuracy audits of the collected data. All abstracted data were double entered into a secure computer. Finally, the study's project manager reviewed all abstract forms and compared data with information in physician progress notes to ensure the accuracy of abstracted data.

Data analysis

The study's expert panel judged that each of the 22 explicit criteria was important to a comprehensive measurement of guideline adherence. However, investigators wanted to know whether the 22 criteria might be indicators of a much smaller number of clinically important treatment dimensions in hypertension health care. It was hypothesized that certain criterion values were likely to cooccur based upon broader influences such as patient factors, the availability of resources or the philosophy or knowledge of the treating physician and that these influences might act in a systematic way on a smaller number of underlying treatment dimensions. Therefore, an exploratory factor analysis was undertaken to examine the contribution of each criterion to the measurement of an adherence construct.²² A scree plot analysis was conducted to determine the number of such factors contained within the adherence tool.²³ Individual criteria loading onto extracted factors were evaluated to ensure their logical interpretability. Factor analytic techniques were conducted using SAS^® software (SAS Institute Inc., Cary, NC, USA)

The majority of criteria in the tool were examined individually for the purpose of factor analysis; however, it was necessary to condense the criteria in the drug therapy and the follow-up intensity domains. The contributions of the six drug therapy criteria for specific compelling indications were combined into a single criterion (STDCRT) for two reasons. First, the degree of overlap between guideline appropriate drug therapies for specific compelling indications is high. An angiotensin converting enzyme (ACE) inhibitor, for example, would be appropriate for patients with heart failure, post-myocardial infarction and diabetes.¹⁵ A patient with any one or more of these compelling indications who is receiving an ACE inhibitor would receive a score of (1) for the STDCRT criterion. Furthermore, as the number of patients with some compelling indications was low, combining the drug therapy criteria increased the likelihood that a factor contribution would emerge if one existed. A total of 16 criteria that were used for the purposes of factor analysis.

As the number of follow-up visits evaluated for each patient was variable, the multiple scores assigned to each patient for each criterion in the follow-up intensity domain were condensed into one criterion score in order to maintain consistency across the patient population. This was accomplished by setting a cutoff point of 60% for each follow-up criterion over the continuum of visits evaluated. Specifically, each criterion in the follow-up intensity domain was assigned a single rating of (1) if that criterion was met at 60% or more of the applicable visits and a single rating of (0) if it was met at less than 60% of all applicable visits. For example, if a patient had a total of four follow-up visits during the evaluation period and the physician fulfilled the criterion at three out of four of those visits, the criterion received a final score of (1) ‘met’ for the purpose of factor analysis. Conversely, if the given criterion was satisfied at only one of the four follow-up visits, the criterion was scored as unmet (0) for the continuum of visits. The 60% cutoff point for the follow-up intensity criteria was selected to reflect that each criterion was met for a majority of the applicable follow-up visits (those in which hypertension was addressed).

Results

The final tool is comprised of a list of 22 explicit criteria and an algorithm for scoring adherence to individual criteria and calculating an overall adherence score and domain scores. The 22 explicit criteria and their weights are found in Table 3.

Table 3.

Explicit criteria for a JNC 7 adherence tool

Criteria	Weight
Diagnosis criteria
Major cardiovascular risk factors have been documented by the physician Diabetes Elevated LDL or total cholesterol or low HDL cholesterol Family history of cardiovascular disease Elevated urine albumin Obesity Tobacco use	[1]
A recommended goal blood pressure is documented	[1]
The subject was provided with his/her blood pressure goal verbally or in writing	[1]
Drug therapy criteria
The subject is being treated with a thiazde diuretic or with a loop diuretic when appropriate	[2]
Appropriate pharmacological treatment for subjects with heart failure	[2]
Appropriate pharmacological treatment for subjects with a history of a myocardial infarction	[2]
Appropriate pharmacological treatment for subjects with coronary disease or angina	[2]
Appropriate pharmacological treatment for subjects with diabetes	[2]
Appropriate pharmacological treatment of subjects with a history of stroke or TIA	[2]
Appropriate pharmacological treatment for subjects with chronic kidney disease	[2]
Follow-up intensity criteria
For subject's whose hypertension was uncontrolled/above goal at the last blood pressure visit, this visit occurs within 4 weeks of the last visit	[1]
The physician mentions the absence of blood pressure control in the progress note	[2]
Subjects whose hypertension is uncontrolled/above goal have either an increase in medication or a change in medication	[2]
The progress note documents that lifestyle recommendations were discussed	[1]
Laboratory monitoring criteria
A fasting lipid profile (LDL, HDL and TG) was measured within the past 12 months	[2]
Subjects with diabetes or chronic kidney disease are screened for urine albumin	[2]
A serum creatinine has been measured within the past 12 months	[2]
A blood glucose has been measured within the past 12 months	[2]
A haematocrit has been measured within the past 12 months	[1]
A potassium level has been measured within the past 12 months	[1]
A calcium level has been measured within the past 12 months	[1]
An EKG has ever been measured	[2]

Abbreviations: HDL, high-density lipoprotein; LDL, low-density lipoprotein; TG, triglycerides.

Factor analysis

A principal component analysis indicated three potentially meaningful factors or constructs based on the results of a scree plot of the obtained roots. The size of the first three roots declined precipitously and then tended to level off across the remaining roots. The validity of this technique relates to Cattell's early demonstration that it was possible to determine the number of true factors using a scree analysis on data of a known factor structure.²⁴ Generalizing from data of known dimensionality, it was established that the number of roots required to reach the scree levelling point also equaled the number of true factors within the data (Figure 1). A subsequent varimax rotation provided the factor pattern (Table 4). Individual criteria were then assessed for whether their grouping under a particular factor was logical and meaningful. As a result of this evaluation, the three factors retained and their loading are displayed in Table 5. A total of 25% of the variance is described by these three factors.

Figure 1.

Scree plot.

Table 4.

Factor scores of adherence criteria

	Factor 1	Factor 2	Factor 3
Documentation of risk factors	0.14109	0.07905	0.29027
Documentation of goal BP	0.01165	0.22562	0.04389
Treatment with diuretic	0.02927	0.23599	0.05093
Appropriate drug therapy for compelling indications	0.00010	0.12029	0.27329
Screening for fasting lipid profile	0.33255	0.07867	0.06082
Screening for urine albumin (for patients with diabetes)	0.09162	0.05815	0.34209
Screening for serum creatinine	0.82369	0.16042	0.04951
Screening for blood glucose	0.71743	0.29176	0.08630
Screening for haematocrit	0.32078	0.02656	0.08125
Screening for potassium	0.86901	0.18726	0.05677
Screening for calcium	0.78741	0.06444	0.07907
EKG ever measured	0.15658	0.19595	0.13608
4-week follow-up	0.08414	0.19954	0.03360
Documentation of uncontrolled BP	0.02586	0.48697	0.05731
Intensification of drug therapy for uncontrolled BP	0.08826	0.45424	0.14134
Documentation of discussion of lifestyle modifications	0.05677	0.01528	0.22754

Abbreviations: BP, blood pressure; EKG, electrocardiogram.

Criteria factor loadings are indicated in bold text.

Table 5.

Factor loading of adherence criteria

Criteria	Factor score
Factor 1: Laboratory Factor
Screening for fasting lipid profile	0.33255
Screening for serum creatinine	0.82369
Screening for blood glucose	0.71743
Screening for haematocrit	0.32078
Screening for potassium	0.86901
Screening for calcium	0.78741
Factor 2: Follow-Up Intensity Factor
Documentation of goal BP	0.22562
Treatment with diuretic	0.23599
EKG ever measured	0.19595
4-week follow-up	0.19954
Documentation of uncontrolled BP	0.48697
Intensification of drug therapy for uncontrolled BP	0.45424
Factor 3: Co-Morbidity Factor
Documentation of risk factors	0.29027
Appropriate drug therapy for compelling indications	0.27329
Screening for urine albumin (for patients with diabetes)	0.34209
Documentation of discussion of lifestyle modifications	0.22754

Abbreviations: BP, blood pressure; EKG, electrocardiogram.

Overall, criteria loading onto each of these factors paralleled their initial domain designation on the adherence tool. Criteria found to load onto Factor 1 included those that evaluate patient laboratories. Criteria loading onto Factor 2 were those that address the intensity of follow-up, including documentation of goal blood pressure and lack of blood pressure control, length of time between follow-up visits, changing drug therapy when blood pressure was uncontrolled and treatment with a diuretic. Finally, criteria loading onto Factor 3 included those that address elements of care for patients with comorbidities, namely the use of appropriate drug therapy for patients with co-morbidities, screening patients with diabetes for serum albumin and documentation of patient risk factors. Only two criteria, measurement of Electrocardiogram (EKG) and documentation of lifestyle modifications, did not load as would be expected. Conducting an EKG was found to load under the Factor 2 (Follow-up Intensity) instead of under Factor 1 (Laboratory), as anticipated. As the discussion of lifestyle modifications should occur at patient visits which address hypertension, it was expected that this criteria would load under Factor 2 (Follow-up Intensity). Instead, results of the factor analysis grouped this criterion under Factor 3 (Co-morbidities).

Discussion

Our approach to evaluating guideline adherence identified key care practices recommended in the JNC 7 guidelines and formalized them into a set of explicit adherence criteria. The explicit criteria include measures for a variety of hypertensive care practices as well as measures related to subpopulations of patients with compelling indications for specific antihypertensive drug therapy. A complex scoring algorithm determines whether each individual explicit criterion is met, enabling clinicians or health systems to describe the degree of physician adherence to recommended practices. The algorithm also quantifies an overall adherence score, which can be used to compare differences in adherence among groups and over time. The explicit criteria and the scoring algorithm formally comprise an ‘adherence tool.’

It appears that the adherence tool contains three unique factors, which we have interpreted as the Laboratory Factor (Factor 1), the Follow-Up Intensity Factor (Factor 2) and the Co-Morbidity Factor (Factor 3). The results of the preliminary factor analysis revealed that the explicit criteria comprising each factor were relatively consistent with their original domain designation. This outcome is logical, as process criteria addressing like elements of care resulted in groupings under an extracted factor that were similar to their initial characterization in the adherence tool domains. Only two factors did not load as anticipated before the factor analysis. The loading of EKG measurement under the Follow-Up Intensity Factor might indicate that physicians who follow their patients with greater intensity also tend to order an EKG routinely. The finding that discussion of lifestyle modifications loaded under the Co-Morbidity Factor might indicate that physicians have knowledge of the importance of lifestyle modifications as well as the appropriate pharmacologic management of specific co-morbid conditions, and use both approaches in tandem.

The factor analysis results were not highly robust. The majority of individual criterion factor scores were low, and the variance described by the three extracted factors represented only a small portion of the total variance across the criteria. Two explanations for these findings seem possible. The first explanation is based on the study population. Specifically, the factor analysis was conducted only for patients whose blood pressures were not controlled. Physician adherence might have been somewhat more random for these patients, contributing both to the lack of blood pressure control and to the small portion of variance explained by the extracted factors. If blood pressure control is positively associated with physician adherence, applying factor analysis to adherence data for a set of patients with both controlled and uncontrolled blood pressures might result in a greater portion of variance being explained.

The second explanation assumes that the model of adherence represented by the tool contains a moderate level of random error. Although the 22 criteria in the tool have substantial face and content validity as explicit measures of guideline adherence, physician adherence might be confounded by multiple outside influences, including physician knowledge and attitudes, patient factors such as age, co-morbid conditions and medication compliance, and health system barriers such as lack of insurance for clinic visits or medications. When evaluating adherence, it will therefore be necessary to utilize statistical modelling that takes into account the effects of such independent variables.

The method for evaluating adherence to guidelines described herein represents an explicit review process. Adherence is evaluated based on the provider's documentation in a patient's medical record. Limitations associated with an explicit review process, including potential problems with completeness and accuracy of medical record data, have been well described.^25-27 The use of an explicit review process can make it difficult to differentiate true deficiencies in clinical performance from poor record keeping. However, studies have demonstrated that good medical recording is related to better quality of practice.^28,29 In addition, an explicit review process does not permit evaluation of extenuating circumstances that might affect physician guideline adherence. Consequently, a comprehensive evaluation of guideline adherence will require implementation of an implicit review process in conjunction with the explicit review. The implicit review process can describe the existence of extraneous circumstances or barriers such as patient noncompliance, financial limitations or complex psychosocial factors that might affect a physician's ability to follow guidelines. Combining the explicit and the implicit processes helps ensure the most accurate characterization of overall adherence for each individual patient case.

Implications for clinical practice

The tool's scoring process provides a quantitative, numerical evaluation of the care provided to an individual patient. A composite adherence score can be calculated as an overall measure of quality delivered to a group of patients. A composite score could be used to give individual physicians' feedback on the degree to which they adhere to recommended guidelines across numerous hypertensive patients in their practice. A composite score could also be used to evaluate the degree to which a group of physicians adheres to recommended practices and serve as a key instrument for bench-marking practice. In addition, the tool could be used to track potential changes in practice over time that result from a change in clinical protocol or from a research intervention. Finally, clinicians could choose to evaluate only criteria in a specific domain where practice was thought to be weak (e.g., only criteria in the follow-up intensity domain).

Conclusion

The results of our exploratory factor analysis suggested three underlying treatment dimensions in hypertension care: laboratory, follow-up intensity and co-morbidities. The factor loadings for each criterion were relatively consistent with their original domain designation. However, many of the criterion factor scores were low. Future studies should test the tool in patients with uncontrolled and controlled blood pressure to determine if more robust findings emerge with a varied population. Future studies should also evaluate the effects of confounding factors on physician adherence. Finally, process criteria ideally should be positively correlated with patient outcomes in order to be meaningful. Therefore, the relationship of each individual adherence criterion and overall adherence to blood pressure control should also be examined in future studies.