Abstract
This study evaluated physician adherence to the Seventh Report of the Joint National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure (JNC 7) hypertension guidelines in 6 community‐based clinics. Explicit review of retrospective medical record data for patients with uncontrolled hypertension measured guideline adherence using 22 criteria. Mean overall guideline adherence was 53.5% and did not improve significantly over time. Random‐effects models demonstrated significant associations between guideline adherence and various demographic and medical predictors, including age, minority status, comorbid conditions, and number of medications. A subsequent implicit review evaluated the degree to which nonadherence was justifiable and identified factors that might have affected adherence. Nonadherence was rated as justifiable for only 6.6% of the failed explicit criteria. In general, adherence to the JNC 7 guidelines was modest even when barriers that might have affected adherence were taken into consideration.
An estimated 59 million persons in the United States have hypertension. 1 Despite the publication by the National Heart, Lung, and Blood Institute of 7 sets of guidelines for the management of hypertension, blood pressure (BP) is controlled to recommended levels for only about 34% of persons with hypertension. 2 One possible explanation for low control rates is poor physician adherence to guidelines. Several studies have examined physician adherence and arrived at differing conclusions. 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 These studies, however, failed to evaluate multiple aspects of care, describe adherence of patients with specific comorbid conditions, use both explicit and implicit reviews, or consider factors that might explain instances of nonadherence. 17
The goals of this study were to: (1) describe physician adherence to the guidelines of the Seventh Report of the Joint National Committee on the Detection, Evaluation, and Treatment of High Blood Pressure (JNC 7) 2 for patients with uncontrolled hypertension using a set of explicit criteria; (2) evaluate adherence in subpopulations with specific comorbid conditions; (3) detect any trend in adherence following release of the guidelines; (4) evaluate the degree to which nonadherence is justified; and (5) identify barriers that might contribute to poor adherence. Unique features of this evaluation include a comprehensive review of medical records and the inclusion of community‐based primary care practices.
MATERIALS AND METHODS
The study was conducted at 6 family medicine community‐based residency training offices in Iowa and approved by each site's institutional review board and by the University of Iowa institutional review board. All subjects gave informed consent, and all procedures were followed in accordance with institutional guidelines.
No formalized, structured training was provided on the management of hypertension or the JNC 7 report before this investigation, but all 6 sites had clinical pharmacists who were members of the training faculty and who had frequent contact with the physicians about specific patients. All physicians also received educational updates on hypertension from the clinical pharmacy faculty members before this study.
Medical Record Abstraction. Six clinic study coordinators, each a nursing staff member, enrolled patients and abstracted medical record data. Participants were identified through prescriptions for antihypertensive medications, diagnostic coding for hypertension, and physician referral. Patient eligibility criteria appear in Table I. Enrollment took place from February 2004 to October 2004.
Table I.
Inclusion |
Age at least 21 years |
Average blood pressure 135–179 mm Hg systolic or 85–109 mm Hg diastolic for patients with diabetes, otherwise 145–179 mm Hg systolic or 95–109 mm Hg diastolic |
Taking 0–3 antihypertensive medications |
No change in medication regimen in the past 4 weeks |
Exclusion |
Clinic blood pressure ≥180 mm Hg systolic or ≥110 mm Hg diastolic |
Evidence of hypertensive emergency |
Myocardial infarction in the past 6 months |
New York Heart Association class III or IV heart failure |
Unstable angina |
Serious renal or hepatic disease |
Pregnancy |
Life expectancy shorter than 3 years |
Dementia or other cognitive impairment |
Study coordinators were trained in medical record abstraction during six 90‐minute joint meetings over the Iowa communications network and given comprehensive written instructions. An index visit was identified for each patient, and a standardized form was used to abstract the following data elements for the 18‐month period before and including the index visit: demographics, medical and family history, comorbid conditions, clinic visit dates, BPs, weights, medications, lifestyle recommendations, laboratory values, and consultations. Because the study was retrospective, there were no controls on measuring clinic BPs. Study coordinators photocopied patient summary sheets, medication sheets, physician notes that mentioned hypertension, reports of electrocardiograms (ECGs), hospital admission and discharge information, and referral and consultation reports. All patient, physician, hospital, worksite, and geographic identifiers were redacted, yielding blinded abstracts.
An investigator (GA) audited the first 33 abstracts for accuracy and found an error rate of 1.1% for 5027 unique data elements. Each error was corrected, and study coordinators were instructed on how to avoid future errors. The investigator subsequently compared every submitted abstract form with the photocopied records to verify accuracy and make corrections when needed. Data were double‐entered into a computer database.
Evaluation of Physician Adherence. Physician adherence was evaluated using both explicit and implicit methods. The explicit review used 22 criteria created by 3 hypertension experts to measure guideline adherence; its reliability has been described previously. 18 A computerized algorithm assigned a score for each of the explicit criteria. Due to the extensive time required to review each case using implicit methods, a random sample of 73 cases (21.2%) stratified by site were evaluated by implicit review. Two blinded investigators (BLC, GRB) independently conducted the implicit review, reading every progress note, laboratory report, and discharge summary that addressed hypertension. Investigators identified patient, physician, and system factors that might have influenced adherence in each case and rated physician failure to adhere to each explicit criterion as justifiable or not justifiable. Nonadherence was judged justifiable when a patient, health system, or other factor beyond physician control was identified that might have influenced adherence to a criterion. Investigators also rated the overall quality of care using a 7‐point Likert‐type scale (1=very poor, 7=excellent).
Data Analyses. Physician adherence scores were calculated for each patient during the 6‐month period before and including the index visit. A mean overall adherence score described the percent of physician adherence to all applicable criteria. Since the study design involved potential hierarchal effects of physician and clinic, random‐effects models were used to adjust for these effects, as necessary, when testing for significant predictors of the adherence scores. Predictors of interest primarily consisted of demographic and medical factors. We also compared adherence scores for patients enrolled during 3 distinct 3‐month time periods between February 2004 and October 2004 to assess whether adherence improved following the release of the guidelines in 2003. Each risk factor was considered by itself in univariate analyses, after which a manual stepwise procedure was implemented to find a multivariate regression model.
Analyses of implicit review included: (1) mean number of extenuating factors identified per case; (2) mean percent of failed criteria that were judged to be justifiable; and (3) mean score for overall quality of care. A Spearman correlation coefficient test evaluated the inter‐rater reliability of the 2 reviewers' ratings for overall quality of care.
RESULTS
Subjects. The study included 345 patients who were mostly white (71.4%) women (61.3%), with a mean age of 60.0±14.7 years (range, 22–99 years). The most frequent payment sources were Medicare (38.2%) and private insurance (37.3%); 27.4% had documented prescription coverage. Chronic comorbid conditions included dyslipidemia (63.2%), diabetes (40.9%), chronic kidney disease (18.3%), and coronary artery disease (CAD) (13.9%).
Explicit Adherence Scores. Table II reports scores for the 22 individual explicit criteria. Documentation of all cardiovascular risk factors occurred for 62.2% of patients, but only 4.4% had a goal BP documented. Most (69.9%) were treated with a thiazide diuretic or a loop diuretic when appropriate. Most patients with specific comorbid conditions received at least 1 appropriate drug therapy recommended by JNC 7. Adherence to specific laboratory monitoring criteria varied widely (range, 29.1%–79.7%). ECG documentation existed for 41.5% of patients.
Table II.
Criteria Evaluated Once During the Entire Audit Period | Patients, % (No.*) |
---|---|
All major cardiovascular risk factors are documented | 62.2 (199/320) |
A recommended goal blood pressure (BP) is documented | 4.4 (15/345) |
Patient provided with goal BP | 62.5 (5/8) |
Patient treated with a thiazide or a loop diuretic when appropriate | 69.9 (241/345) |
Appropriate pharmacologic treatment for patients with: | |
Heart failure | 93.3 (14/15) |
History of myocardial infarction | 95.0 (19/20) |
Coronary artery disease | 100 (40/40) |
Diabetes | 100 (149/149) |
History of stroke or transient ischemic attack | 94.4 (17/18) |
Chronic kidney disease | 77.8 (49/63) |
Fasting lipid profile measured within the past 12 months | 51.9 (179/345) |
Patients with diabetes or chronic kidney disease screened for urine albumin | 29.1 (52/179) |
Serum creatinine measured within the past 12 months | 79.7 (275/345) |
Blood glucose measured within the past 12 months | 58.3 (201/345) |
Hematocrit measured within the past 12 months | 31.9 (110/345) |
Potassium level measured within the past 12 months | 78.8 (272/345) |
Calcium level measured within the past 12 months | 62.6 (216/345) |
An electrocardiogram has ever been performed | 41.5 (143/345) |
Criteria Evaluated During Multiple Visits | Visits, % (No.†) |
When BP is not controlled, a return visit occurs within 1 month (35 days) | 39.8 (270/678) |
Absence of BP control referenced in progress note | 62.2 (483/776) |
Medication increased or changed when BP uncontrolled | 55.9 (434/776) |
Discussion of lifestyle recommendations documented | 21.5 (183/853) |
*Number of patients for whom criterion was met divided by number of patients for whom criterion applied. †Number of visits at which criterion was met divided by number of visits at which criterion applied. |
Physicians provided moderately intensive care when BP was elevated, mentioning lack of BP control at 62.2% of these visits, adjusting medications at 55.9% of such visits, and seeing patients within 1 month (35 days) following only 39.8% of such visits. Physicians documented discussion of lifestyle modifications at 21.5% of all visits.
Mean overall physician adherence to guidelines across all cases was 53.5%±16.2% (range, 0.0%–88.2%). Based on an analysis of random‐effects models, the within‐physician correlation was estimated to be small (7.2%), while the within‐clinic correlation was even smaller (2.4%). These findings suggest little variance among physicians and clinics. We found several factors to be significantly associated with adherence scores (Table III). The estimates that are shown represent the amount of change in the adherence scores that were observed. For example, the mean score for minority (non‐white) patients was 59.4%, while the mean score for white patients was 52.7% (ie, a difference of 6.4%–6.7% was found between adherence for different races).
Table III.
Predictor | Univariate | Multivariate |
---|---|---|
Age (per 10‐year increase) | −1.1* | −1.7‡ |
Age >65 years | −3.1* | – |
Sex (female vs male) | 0.22 | – |
Minority | 6.4† | 5.6† |
Number of comorbidities | 2.6‡ | 1.9† |
>2 comorbidities | 8.4§ | – |
Ever smoked | 2.9* | – |
Current smoker | 2.0 | – |
Coronary artery disease | 10.5§ | 7.6‡ |
History of myocardial infarction | 8.9† | – |
Stroke/transient ischemic attack | 7.8† | – |
Diabetes | 3.5† | – |
Chronic kidney disease | 2.9 | – |
Number of medications | 3.5§ | 3.1‡ |
Private insurance | .96 | – |
Insurance covered | 1.2 | – |
Quarterly comparison|| | ||
2 vs 1 | 1.6 | – |
3 vs 1 | 3.3 | – |
*P<.100. †P<.050. ‡P<.010. §P<.001. ||Comparing adherence over three 3‐month quarters in 2004 to evaluate effect of time following publication of guidelines in 2003. |
In addition to a difference due to minority status, we also found significant univariate associations with total comorbidities, CAD, history of myocardial infarction, history of stroke or transient ischemic attack, diabetes, and the number of medications. We found a slight negative association with increasing patient age. The older the patient, the less physician adherence to guidelines we found.
Scores were higher, but not significantly so, for patients who had ever smoked (P=.0999). Adherence showed a nonsignificant improvement over the 3 successive enrollment quarters (P=.31). With the multivariate model, we found simultaneous associations for age, minority status, comorbidities, CAD, and number of medications; a white 75‐year‐old with no comorbidities and no medications would receive care consistent with a mean adherence score of 40%, while a minority 55‐year‐old with CAD, 4 comorbidities, and 4 medications would receive care consistent with a mean adherence score of 77%.
Implicit Review. Two investigators reviewed a random sample of 73 cases (21.2%), identifying extenuating factors that might have influenced physician adherence. The most frequently identified factors were the presence of comorbidities and patient noncompliance (Table IV). The mean number of possible influencing factors identified was 0.9±0.9 per patient. For 14 cases (19.2%), neither reviewer identified any extenuating factor that might have explained adherence failures.
Table IV.
Extenuating Factor | No. of Times Identified* |
---|---|
Complex comorbidities requiring ongoing attention | 51 |
Patient noncompliance | 31 |
Intolerance to the recommended regimen | 18 |
Infrequent clinic visits | 13 |
Limited resources | 9 |
Pain or trauma | 6 |
Patient declines recommended regimen | 5 |
Blood pressures better controlled outside of the clinic | 4 |
Focus on lifestyle changes to control blood pressure | 1 |
Alternate blood pressure goal | 1 |
Patient seen by multiple physicians | 1 |
Language barrier | 1 |
Patient sensory deficits | 1 |
Patient contraindication to the recommended regimen | 1 |
Substance abuse | 1 |
*Out of 146 possible. |
A mean of 15.3±1.5 criteria applied in the 73 cases. The mean failure rate for applicable criteria was 7.8±2.7 per case. The 2 reviewers agreed for 503 out of 561 criterion failures (89.7%), with both rating 493 failures (87.9%) as not justifiable and 10 (1.8%) as justifiable. Reviewers rated a mean of 6.6%±12.8% failed criteria per patient as justifiable (range, 0%–100%).
The mean rating for overall quality of care was 3.5±1.6 (1=very poor, 7=excellent). The 2 reviewer ratings were identical for 14 cases (19.2%), differed by 1 point for 34 cases (46.6%), and differed by an average of 1.4±1.1 points per case. The Spearman correlation coefficient comparing the 2 reviewers' scores was 0.45732, indicating modest agreement for overall quality‐of‐care ratings.
DISCUSSION
We found modest physician adherence to JNC 7 guidelines. The comparison of scores for patients enrolled in successive calendar quarters demonstrated only a nonsignificant increase in adherence following release of the new guidelines. Most patients, however, received appropriate diuretic therapy; this suggested increased prescriptions for diuretics since publication of previous evaluations. 3 , 5 , 12 , 14 In addition, physicians generally adhered to the pharmacologic recommendations for patients with specific comorbid conditions. Adherence was low, however, for criteria that reflect intensity of care. Some delays in follow‐up might have resulted from patient noncompliance with the recommended return schedule, but this was not likely a consistent cause, because the implicit reviewers identified only 13 instances (of 146 possible) where infrequent patient visits might have affected adherence. Failure to adjust medication at 44% of visits where BP was uncontrolled suggests that BP control was not pursued aggressively. The study results suggest continued clinical inertia, that is, the failure to promptly initiate/adjust therapy and follow‐up with patients who had abnormal clinical BPs. Clinical inertia has been described by other researchers 19 and is the major reason BP remains poorly controlled. 19 , 20 , 21 , 22
Physicians infrequently documented discussion of lifestyle recommendations, even though weight loss, sodium restriction, and increased activity are known to lower BP. 23 , 24 , 25 This finding might reflect poor documentation. Alternatively, physicians might not prioritize this aspect of care, believing that many patients will be unresponsive to lifestyle recommendations. Infrequent documentation of lifestyle recommendations could, however, reflect another type of clinical inertia—namely, missed opportunities to promote patient self‐management.
Many of the laboratory criteria indicated poor adherence, which might reflect cost‐saving efforts. Annual monitoring of glucose and lipids, however, does not constitute extravagant care, and at least 1 ECG is warranted and recommended for all persons with hypertension. 2 ECGs performed in a different setting might inadvertently be missing, but missing data nevertheless result in absence of information pertinent to care.
Infrequent documentation of goal BP might reflect documentation omission rather than true error in practice. Nevertheless, setting a clear goal and treating to goal is a critical component of care. Physicians tended to communicate goal BPs that were documented to patients. Specifying a goal BP might help physicians focus care on goal attainment.
The presence of comorbidities and patient noncompliance were identified most frequently as extenuating factors that might have led to poor adherence to guidelines. Nonadherence, however, cannot be attributed primarily to these factors, since only 6.6% of criterion failures were judged to be justifiable. In addition, the overall quality of care, rated as 3.5 on a 1‐to‐7 scale, indicates care that was less than acceptable for many cases. The implicit review, therefore, gives added support to the finding that overall physician adherence to hypertension guidelines was modest.
The higher scores noted for nonwhites might indicate more aggressive care for high‐risk African Americans, and higher scores for patients with increased medication use or more comorbid conditions suggest an increased focus on hypertension when patients have complex medical conditions.
Possible Limitations of the Study. Reliance on medical records as the primary data source is a study limitation. The potential for incompleteness and inaccuracy has been well described 26 , 27 , 28 and can make it difficult to differentiate true deficiencies in performance from poor record‐keeping. Nonetheless, since good documentation has been related to better quality of care, 29 , 30 medical records may serve as a reasonable data source when investigating quality of adherence.
High adherence by physicians to pharmacologic recommendations for patients with specific comorbid conditions might, in part, result from the guideline's inclusion of multiple recommended drug classes for a given comorbid condition. JNC 7 does not specify drugs of choice or suggest combinations for comorbid conditions. Guideline recommendations for diabetics, for example, include any diuretic, angiotensin‐converting enzyme inhibitor, angiotensin receptor blocker, β‐blocker, or calcium channel blocker. Recommendations for other comorbid conditions are equally liberal, and recommended classes of agents are not ranked, precluding discrimination among classes. These liberal inclusion requirements made the likelihood of meeting these criteria inherently high. Nonetheless, these criteria and the elements required to implement them accurately reflect guideline recommendations. 18
This study measured guideline adherence only for patients with uncontrolled BPs. This might have led to a selection bias that included cases more likely to have lower adherence scores. The study provided evidence, however, that most of these cases should have received more intensive care. Consequently, it remains reasonable to conclude that physician adherence to BP guidelines is modest at best.
Acknowledgments and disclosure: We are grateful to the study coordinators: Maureen Blankenship, RN; Sandi Veberg, RN; Alice Ciavarelli, CMA, CCRC; Angie Bixby‐Ellerman; Pam Trenkamp, RN; and Kristine Cox, CMA. We also acknowledge the clinic physicians who served as local principal investigators for the study: Steven Joyce, MD; Dean Bunting, MD; Andrea Silvers, MD; Jerrold White, MD; John Sutherland, MD; and Douglas Workman, MD. Finally, we thank Henry R. Black, MD; George L. Bakris,MD; and Daniel W. Jones, MD, for developing the explicit criteria. This study was supported by the National Heart, Lung, and Blood Institute, R01 HL070740. Dr Carter and Ms Franciscus are also supported by the Center for Research in the Implementation of Innovative Strategies in Practice (CRIISP), Department of Veterans Affairs, Veterans Health Administration, Health Services Research and Development Service (HFP 04‐149). The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs.
References
- 1. Fields LE, Burt VL, Cutler JA, et al. The burden of adult hypertension in the United States 1999 to 2000: a rising tide. Hypertension. 2004;44:398–404. [DOI] [PubMed] [Google Scholar]
- 2. Seventh report of the Joint National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure. Hypertension. 2003;42:1206–1252. [DOI] [PubMed] [Google Scholar]
- 3. Monane M, Glynn RJ, Gurwitz JH, et al. Trends in medication choices for hypertension in the elderly. The decline of the thiazides. Hypertension. 1995;25:1045–1051. [DOI] [PubMed] [Google Scholar]
- 4. Siegel D, Lopez J. Trends in antihypertensive drug use in the United States: do the JNC V recommendations affect prescribing? Fifth Joint National Commission on the Detection, Evaluation, and Treatment of High Blood Pressure. JAMA. 1997;278:1745–1748. [DOI] [PubMed] [Google Scholar]
- 5. Siegel D, Lopez J, Meier J, et al. Changes in the pharmacologic treatment of hypertension in the Department of Veterans Affairs 1997–1999: decreased use of calcium antagonists and increased use of beta‐blockers and thiazide diuretics. Am J Hypertens. 2001;14:957–962. [DOI] [PubMed] [Google Scholar]
- 6. Xu KT, Moloney M, Phillips S. Economics of suboptimal drug use: cost‐savings of using JNC‐recommended medications for management of uncomplicated essential hypertension. Am J Manag Care. 2003;9:529–536. [PubMed] [Google Scholar]
- 7. Henderson SO, Bretsky P, DeQuattro V, et al. Treatment of hypertension in African Americans and Latinos: the effect of JNC VI on urban prescribing practices. J Clin Hypertens (Greenwich). 2003;5:107–112. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Cuspidi C, Michev I, Lonati L, et al. Compliance to hypertension guidelines in clinical practice: a multicentre pilot study in Italy. J Hum Hypertens. 2002;16:699–703. [DOI] [PubMed] [Google Scholar]
- 9. Asch SM, Kerr EA, Lapuerta P, et al. A new approach for measuring quality of care for women with hypertension. Arch Intern Med. 2001;161:1329–1335. [DOI] [PubMed] [Google Scholar]
- 10. Hyman DJ, Pavlik VN. Self‐reported hypertension treatment practices among primary care physicians: blood pressure thresholds, drug choices, and the role of guidelines and evidence‐based medicine. Arch Intern Med. 2000;160:2281–2286. [DOI] [PubMed] [Google Scholar]
- 11. Mehta SS, Wilcox CS, Schulman KA. Treatment of hypertension in patients with comorbidities: results from the study of hypertensive prescribing practices (SHyPP). Am J Hypertens. 1999;12:333–340. [PubMed] [Google Scholar]
- 12. Nelson CR, Knapp DA. Trends in antihypertensive drug therapy of ambulatory patients by US office‐based physicians. Hypertension. 2000;36:600–603. [DOI] [PubMed] [Google Scholar]
- 13. Cloher TP, Whelton PK. Physician approach to the recognition and initial management of hypertension. Results of a statewide survey of Maryland physicians. Arch Intern Med. 1986;146:529–533. [PubMed] [Google Scholar]
- 14. Manolio TA, Cutler JA, Furberg CD, et al. Trends in pharmacologic management of hypertension in the United States. Arch Intern Med. 1995;155:829–837. [PubMed] [Google Scholar]
- 15. Naiman DJ, Barker LR. Adherence to the JNC VI guidelines for the treatment of hypertension in the resident clinic. Md Med J. 1999;48:307–311. [PubMed] [Google Scholar]
- 16. Campbell NR, Tu K, Brant R, et al. The impact of the Canadian Hypertension Education Program on antihypertensive prescribing trends. Hypertension. 2006;47:22–28. [DOI] [PubMed] [Google Scholar]
- 17. Milchak JL, Carter BL, James PA, et al. Measuring adherence to practice guidelines for the management of hypertension: an evaluation of the literature. Hypertension. 2004;44:602–608. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Milchak JL, Carter BL, Ardery G, et al. Development of explicit criteria to measure adherence to hypertension guidelines. J Hum Hypertens. 2006;20:426–433. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Okonofua EC, Simpson KN, Jesri A, et al. Therapeutic inertia is a impediment to achieving the Healthy People 2010 blood pressure control goals. Hypertension. 2006;47:345–351. [DOI] [PubMed] [Google Scholar]
- 20. Berlowitz DR, Ash AS, Hickey EC, et al. Inadequate management of blood pressure in a hypertensive population. N Engl J Med. 1998;339:1957–1963. [DOI] [PubMed] [Google Scholar]
- 21. Oliveria SA, Lapuerta P, McCarthy BD, et al. Physician‐related barriers to the effective management of uncontrolled hypertension. Arch Intern Med. 2002;162:413–420. [DOI] [PubMed] [Google Scholar]
- 22. Andrade SE, Gurwitz JH, Field TS, et al. Hypertension management: the care gap between clinical guidelines and clinical practice. Am J Manag Care. 2004;10:481–486. [PubMed] [Google Scholar]
- 23. Minami J, Ishimitsu T, Matsuoka H. Effects of smoking cessation on blood pressure and heart rate variability in habitual smokers. Hypertension. 1999;33:586–590. [DOI] [PubMed] [Google Scholar]
- 24. Baster T, Baster‐Brooks C. Exercise and hypertension. Aust Fam Physician. 2005;34:419–424. [PubMed] [Google Scholar]
- 25. Sacks FM, Svetkey LP, Vollmer WM, et al. Effects on blood pressure of reduced dietary sodium and the Dietary Approaches to Stop Hypertension (DASH) Diet. N Engl J Med. 2001;344:3–10. [DOI] [PubMed] [Google Scholar]
- 26. Bentsen BG. The accuracy of recording patient problems in family practice. J Med Educ. 1976;51:311–316. [DOI] [PubMed] [Google Scholar]
- 27. Romm FJ, Putnam SM. The validity of the medical record. Med Care. 1981;19:310–315. [DOI] [PubMed] [Google Scholar]
- 28. Tufo HM, Speidel JJ. Problems with medical records. Med Care. 1971;9:509–517. [DOI] [PubMed] [Google Scholar]
- 29. Zuckerman ZE, Starfield B, Hochreiter C, et al. Validating the content of pediatric outpatient medical records by means of tape‐recording doctor‐patient encounters. Pediatrics. 1975;56:407–411. [PubMed] [Google Scholar]
- 30. Lyons TF, Payne BC. The relationship of physicians' medical recording performance to their medical care performance. Med Care. 1974;12:714–720. [DOI] [PubMed] [Google Scholar]