Lack of chart reminder effectiveness on family medicine resident JNC-VI and NCEP III guideline knowledge and attitudes

Abstract

Background

The literature demonstrates that medical residents and practicing physicians have an attitudinal-behavioral discordance concerning their positive attitudes towards clinical practice guidelines (CPG), and the implementation of these guidelines into clinical practice patterns.

Methods

A pilot study was performed to determine if change in a previously identified CPG compliance factor (accessibility) would produce a significant increase in family medicine resident knowledge and attitude toward the guidelines. The primary study intervention involved placing a summary of the Sixth Report of the Joint National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure (JNC VI) and the National Cholesterol Education Program Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol in Adults (NCEP III) CPGs in all patient (>18 yr.) charts for a period of three months. The JNC VI and NCEP III CPGs were also distributed to each Wayne State family medicine resident, and a copy of each CPG was placed in the preceptor's area of the involved clinics. Identical pre- and post- intervention questionnaires were administered to all residents concerning CPG knowledge and attitude.

Results

Post-intervention analysis failed to demonstrate a significant difference in CPG knowledge. A stastically significant post-intervention difference was found in only on attitude question. The barriers to CPG compliance were identified as 1) lack of CPG instruction; 2) lack of critical appraisal ability; 3) insufficient time; 4) lack of CPG accessibility; and 5) lack of faculty modeling.

Conclusion

This study demonstrated no significant post intervention changes in CPG knowledge, and only one question that reflected attitude change. Wider resident access to dedicated clinic time, increased faculty modeling, and the implementation of an electronic record/reminder system that uses a team-based approach are compliance factors that should be considered for further investigation. The interpretation of CPG non-compliance will benefit from a causal matrix focused on physician knowledge, attitudes, and behavior. Recent findings in resident knowledge-behavior discordance may direct the future investigation of physician CPG non-compliance away from generalized barrier research, and toward the development of information that maximizes the sense of individual practitioner urgency and certainty.

Background

In the past decade physician and governmental organizations have developed a proliferation of clinical practice guidelines (CPG) to facilitate high quality, appropriate, and cost-effective medical care, while decreasing regional and systemic variations in practice [1,2]. Resident and practicing physicians have reported positive attitudes toward these guidelines, however evidence demonstrates a relative failure of these guidelines to actually change clinical practice patterns [1-9]. A recent study concerning asthma practice guidelines reported that only 44% of family physicians surveyed used any practice guidelines routinely, and only 55% had seen the asthma guidelines [6]. There has been a lack of recommendations concerning successful implementation strategies, despite extensive efforts toward guideline development and distribution.

Cabana et. al. did an extensive literature review of clinician CPG non-compliance barriers [10]. These researchers found 293 potential barriers that included, awareness (n = 46), familiarity (n = 31), agreement (n = 33), self-efficacy (n = 19), outcome expectancy (n = 8), ability to overcome inertia from previous practice patterns (n = 14), and the absence of external barriers to perform recommendations (n = 34). Fifty-eight percent of the studies reviewed examined only one barrier [10]. An independent review of the current literature (Medline 1966–2003) revealed that increasing the distribution and frequency of continuing medical education lectures have demonstrated very little success [11,12]. Moderate success had been derived using a multi-intervention approach including peer review chart reminders, chart audit and feedback [13,14]. The purpose of this investigation is to determine if altering one CPG compliance barrier (increased CPG access), would have a significant effect on the resident's knowledge or attitude. The JNC VI and NCEP III guidelines [15,16] were chosen, as they are internationally recognized and represent two of the most commonly treated diseases in Family Medicine [3].

Methods

Participants

Thirty-four (PGY 1–3) Wayne State University (WSU) family medicine residents (FMR) at two clinical sites (suburban and urban) were voluntarily requested to participate in this pilot research project. Of the original 34 participating FMRs, 30 individuals completed both the pre-and post-intervention tests. This study was reviewed and approved by the WSU Human Investigations Committee.

Intervention

The pilot study intervention consisted of three parts: 1) a summary of the two CPGs (JNC VI, NCEP III) were placed in each patient chart for all individuals greater than 18 years of age. The CPG summaries were presented in each chart on bright colored paper opposing the encounter sheet; 2) each resident was given a personal copy of both CPGs; and 3) the guidelines were placed in the preceptor's area of both clinics. This study was completed before the publication of JNC VII.

Measures

A single group voluntary pre-intervention (baseline) and post-intervention test instrument was developed de novo to measure the effects of increased CPG exposure on the FMRs knowledge and attitudes [Appendix A: see Additional file 1]. The test instrument was not validated for this pilot study. Future large-scale follow up studies on this topic would include a fully validated test instrument. A study information sheet was attached to each questionnaire, outlining study purpose, procedure, voluntary participation, confidentiality, risks/benefits, and contact numbers for questions [Appendix B: see Additional file 2]. Each test instrument contained 4 multiple-choice questions from each guideline, and 8 attitude-based Likert-scaled questions on CPGs. The participant's year of training was collected to control for a knowledge bias. Each participating resident completed a three-month intervention period before the post-test was written.

Analysis

Post-intervention test differences in knowledge and attitude were calculated and compared using both paired T-tests, and a Wilcoxon signed ranks test. The Wilcoxon signed ranks test uses the hypothesis that the medians (versus the means) are equal in two paired samples. When observations are not normally distributed, the Wilcoxon signed ranks test is more powerful than the paired T-test, especially in small sample sizes. The alpha level used in this study was 0.05. Pre-and post-attitude questions were compared utilizing a 5-point Likert scale: 1 (strongly disagree) – 5 (strongly agree).

One question concerning self-knowledge evaluation (#10) utilized a Likert scale: 1=no knowledge/no utilization- 5 excellent knowledge/constant utilization.

This pilot study tests the feasibility of a full experimental trial concerning the effects of increased FMR CPG exposure, on their attitudes and knowledge. This study does not provide hypothesis testing due to being underpowered and susceptible to type 1 error.

Results

Eighty-eight percent (30 of 34) of the participating FMRs completed both the pre- and post-intervention tests. No inter-test differences were found between the Wilcoxon Signed Ranks Test and the Paired T- Test analysis. One question (2d) was disregarded due to an error in transcription.

There were no significant differences found between pre- and post-intervention CPG knowledge of the FMRs either by year or with all of the residents combined [Table 1]. Statistically significant post-intervention differences were found on the attitude question 10A for year 2 (p = 0.025) and year 3 (p = 0.05) [Figure 1].

Table 1.

Pre- and post intervention % CPG knowledge test scores. Paired T-Test and Wilcoxon Signed Ranks Test analysis for significance.

Year of residency	Pretest Score	Post Test Score	Paired T-Test (p = 0.05)	Wilcoxon SR Test (p value)
Year 1(n = 11)	32%	33%	0.779	0.763
Year 2 (n = 10)	36%	36%	1.0	1.0
Year 3 (n = 9)	33%	40%	0.347	0.380
Year 1–3 (n = 30)	34%	36%	0.423	0.423

The table demonstrates the lack of statistical significance of the post-intervention CPG FMR knowledge scores.

Figure 1.

Changes in family medicine resident self perception of JNC VI knowledge. This figure represents a significant post-intervention finding of significance in CPG attitude. This may represent an overestimation of actual knowledge, as there was not a corresponding demonstration of knowledge gain. Question 10A: How well do you think that you know the JNC VI Guideline? Likert Scale (1–5): 1 = no knowledge or utilization of JNC VI guideline – 5 = excellent knowledge and utilization of JNC IV guideline.

Year 1 and year 3 FMR reported that the most common pre-intervention barrier was lack of CPG instruction. Year 2 FMR reported that CPG inaccessibility was the most important pre-intervention barrier. The year 1 and 2 FMRs both reported that most important post-intervention compliance barrier was insufficient time. Year 3 FMRs reported that the most important post-intervention barriers were insufficient CPG instruction and lack of faculty modeling.

Discussion

This study did not replicate the results of previous studies concerning the importance of CPG access and exposure barriers. Increased CPG access across all levels of training did not significantly influence resident physician knowledge, and had very little effect on their attitudes. The reported barriers to CPG utilization identified in this study were the perceived time constraints, faculty modeling and knowledge, access to the CPG, knowledge and of the CPG, and critical appraisal ability of CPGs. The strengths of this study were the high response rate, the use of an "at service" intervention level, the focus on a single knowledge and attitude barrier (access) intervention, and the use of two influential guidelines for common chronic conditions.

Previous efforts to increase CPG distribution and the associated continuing medical education have failed to produce significant changes in physician compliance [11,12]. CPG non-compliance may not be a result of a single barrier (accessibility), but may be multi-factorial. Moderate increases in CPG compliance has been derived from a combination of chart reminders and chart audit interventions [13,14].

This study also found that the post-intervention compliance barriers reported by the residents differed according to their year of training. Years 1 and 2 residents reported that a perceived lack of time was the most important barrier, as had been reported in previous studies [1,11]. The perceived time barrier in early resident development (years 1–2), may be related to a focus on hospital based medicine, with less time spent in the out patient clinic. The more experienced third year residents reported that a lack of specific guideline instruction and staff role modeling were the most important CPG compliance barriers [2,17]. Positive pre-intervention attitudes toward the guidelines were demonstrated in this study [Figure 2] and in previous studies [2,11]. This positive pre-intervention CPG bias may have prevented the detection of significant post-intervention differences in this study.

Figure 2.

Percentage of positive family medicine resident responses to pre- and post-intervention CPG attitude-based questions. This figure represents the lack of CPG attitude change between pre- and post-intervention scores. The baseline positive attitudes toward the CPGs may have prevented the detection of post-intervention changes. Likert Scale (1–5): 1 = strongly disagree – 5 = strongly agree.

A significant barrier to CPG adoption may be the complicated format of the guideline itself. Although most national guidelines have an executive summary, the information distributed in the CPG may be presented in a more simplified and acceptable format [3,18]. A single intervention guideline (i.e. the use of calcium channel blockers to treat hypertension) is a suggested solution. Individual guidelines would qualify their use within a particular clinical situation, where the relative or absolute contraindications are considered [19].

Future research directions could include the investigation of standard electronic records, as a reminder and feedback system. This type of system may prevent the resident physician from overestimating personal guideline compliance or knowledge (11, 20). The significant JNC VI attitude result found in this study (Fig. 1, Q. 10A) for Year 2 (0.025) and Year 3 (0.05) may have been an example of this knowledge overestimation phenomenon. The knowledge results did not reflect a corresponding significant post-intervention change. Reinforcement through electronic feedback would be given at regular intervals from either staff or peer review. Early studies of physician feedback using a computer CPG reminder system seem to effect only modest change. Multi-site computer-based reminder and feedback studies still need to be done to demonstrate efficacy [3,21].

The implementation of standard electronic charting can also be used to promote more non-physician involvement with CPG protocol and appropriate testing. Direct involvement by ancillary staff would not only raise the profile of the guideline recommendations, but may add to office efficiency and the team motivational concept.

A study of the influence and importance of faculty CPG attitudes and knowledge upon the resident must not be overlooked in future research investigations, as a primary source of influence during this important developmental phase in the resident's training.

The literature fails to provide a clear and comprehensive analysis of successful CPG implementation. A complete understanding and definition of the causes of CPG non-compliance and their interactions is needed before interventions can be appropriately implemented. Human behavioral theories (e.g. social learning/social cognitive, self-efficacy) have previously been studied with patient intervention [22], but very little investigation has been done to investigate their influence upon physician CPG non-compliance factors such as self-perceived effectiveness [23]. Integration of these behavior theories with previously identified barriers and interventions in a matrix analysis of CPG non-compliance is worth further investigation.

Tu et. al. found that current CPG studies lack the appropriate methodologies needed to define the causal factors of non-compliance [3]. Schnelle et al. recently examined the clinical outcomes, and described the staffing requirements of a geriatric incontinence and exercise intervention. These researchers found that although the appropriate guidelines produced positive results, individual nursing home environments were not equipped to institute them [24]. Appropriate intervention may only be possible through a matrix analytical approach, such as those used in other epidemiological studies of traumatic injury. This type of study would consider the complex factorial interactions that are unique to each situation [25].

A recently completed qualitative study of FMR used a novel methodology of clinical action analysis following a CPG educational intervention [26]. This study objectively demonstrated the gap between resident knowledge and behavior. The authors found that the rationalizations used for clinical actions based on CPG knowledge by the residents were similar to previously reported barriers by Cabanna et. al. (1999). Residents were found to use identical rationalizations to justify different courses of action based on the same CPG knowledge. Kennedy et. al. state that the individuality of the reported rationalizations sheds significant doubt on the practical applications of barrier research. These authors identified that the conditions of urgency a level of certainty were directly related to individual behavior. These two conditions were suggested as future avenues of knowledge-behavior research [26].

Study Limitations

This study was limited several factors that included a small population size and a non-validated test instrument, which may have decreased the power to detect differences in pre- and post-test responses. Other limiting factors included the short duration of the study, the lack of geographical diversity, a control for other sources of CPG access, and the lack of parallel faculty CPG knowledge, and attitudes investigations.

Conclusions

Valid CPGs such as those used in this study provide the resident physician with an important framework (latest evidence, expert opinions, and recommendations) upon which present and future decisions concerning individual medical treatment can be based. This pilot study demonstrated non-significant post-intervention (increased CPG accessibility) changes in family medicine CPG knowledge and attitude. The discordance between CPG knowledge, attitude and behavior demands further methodological investigation. There is a need for investigations that use different methodologies that include large-scale multi-factorial analytic studies to properly define CPG non-compliance factors, their interactions, and interventions. Recent findings of resident knowledge-behavior discordance may direct the future investigation of physician CPG non-compliance away from generalized barrier research, and toward the development of information that maximizes the sense individual practitioner urgency and certainty.

Competing interests

None declared.

Authors' contributions

PE was primary investigator and author; RU participated in project design, statistical analysis, and manuscript revision; TM participated in project design, and manuscript revision.

Note

This study was previously presented at the WSU Family Medicine Resident Research Day 04/09/2003.

Pre-publication history

The pre-publication history for this paper can be accessed here:

http://www.biomedcentral.com/1471-2296/5/14/prepub

Supplementary Material

Additional File 1

Original test instrument: "Clinical Practice Guidelines (CPG) Project Questionnaire" This is a copy of the original pre- and post-intervention test instrument used in this study.

Additional File 2

Appendix B. Original participant study information description. "Study information Sheet" This is a copy of the study information sheet that was attached to each test instrument. It was used to ensure that each volunteer participant was fully informed about the study.