Abstract
BACKGROUND The accurate recognition of patient pain is a crucial, but sometimes difficult, task in medical care. This study explored factors related to the physician’s diagnosis of pain in primary care patients.
METHODS New adult patients were prospectively randomized to care by primary care providers at a university medical center clinic. Study participants were interviewed prior to the initial visit, and their level of self-reported pain was measured with the Visual Analog Pain Scale and the Medical Outcomes Study Short Form-36. The medical encounter was videotaped in its entirety and later analyzed using the Davis Observation Code to characterize physician practice style. Patient satisfaction was measured immediately after the visit. A review of the medical record was used to assess physician recognition of patient pain.
RESULTS For all patients (N = 509), as the amount of pain increased, the percentage of patients having pain diagnosed by the physician also increased. Female patients reported a greater amount of pain than male patients. When women were in severe pain, they were more likely than men to have their pain accurately recognized by their physician. The correct diagnosis of pain was not significantly related to patient satisfaction. Physician practice styles emphasizing technically oriented activities and health behavior discussions were strongly predictive of the physician diagnosing patient pain.
CONCLUSIONS The diagnosis of pain is influenced by the severity of patient pain, patient gender, and physician practice style. If the routine use of pain assessment tools is found to be effective in improving physician recognition and treatment of patients’ pain, then application of these tools in patient care settings should be encouraged.
Keywords: Pain, signs and symptoms, physician-patient relations, physician’s practice patterns, primary care
INTRODUCTION
Pain is a complex clinical phenomenon which in most cases is a symptom when it occurs acutely, but a disease when it becomes chronic.1 Despite recent advances in the understanding of pain control, pain is frequently unrecognized or undertreated by physicians.2,3 Recognition of patient pain may be influenced by such factors as length of the physician-patient relationship, patient gender, age, ethnicity, attractiveness, and level of pain reported.4–9 Physicians have also been found to underestimate their patients’ pain intensity.10,11
Appropriate pain control may result in quicker recovery and improved quality of life, resulting in increased productivity12; however, most physicians do not routinely address pain issues with their patients. As a consequence, patient rights groups and regulatory bodies are calling for proper pain management to be mandated. The Joint Commission on Accreditation of Health-care Organizations (JCAHO) recently introduced new standards requiring that individuals seeking care at facilities under their jurisdiction have the right to appropriate evaluation and management of pain. All patients must be screened for the presence of pain and given a complete assessment focussed on identifying the patient’s pain by location, intensity, and cause.13 In 1999, the California legislature also passed a bill making pain the “fifth vital sign” and requiring that pain management become part of the curriculum requirements for students entering medical school on or after June 1, 2001.
Although it is clear that the accurate recognition of patient pain is a crucial task in medical care, identification may be difficult. Patients do not always verbally express their pain. Nonverbal pain expressions also vary, with some patients being highly expressive regarding their pain and others exhibiting stoical forbearance and minimal display of discomfort.14 Defining factors that influence the process of recognition of pain may be valuable for physicians as they endeavor to become more skilled at this clinical task. In previous work, we showed how the physician-patient interaction is affected by patient pain.15 In this study, we examine the relationship between patients’ self-reports of pain and physician diagnosis of pain. To the best of our knowledge, no other studies have examined this issue.
We hypothesized that patients with higher pain scale scores are more likely to have pain diagnosed and that the correct diagnosis of patient pain leads to greater patient satisfaction. We further hypothesized that a physician’s practice style and approach to patients is related to the likelihood of diagnosing patient pain.
METHODS
Study Design
This study was part of a larger project exploring physician-patient interaction, physician practice styles, and associated patient outcomes. A total of 509 new patients agreeing to participate constituted the original study population. Subjects were randomly assigned to be seen by 105 physicians (second- and third-year family practice and general internal medicine residents) at the University of California, Davis, Medical Center Primary Care Center. Physicians each saw between 1 and 30 patients with a median of 3 patients per physician. Seventy percent of physicians saw between 1 and 5, 16% between 6 and 10, 9% between 11 and 15, and 6% saw more than 16 patients. Physicians gave general consent for videotaping and were unaware of the specific nature of the study or which interactions were recorded. Patients were interviewed prior to their initial visit with the primary care provider. Data collected included sociodemographic information, self-reported health status using the Medical Outcomes Study Short Form-36 (MOS SF-36), and visual analog pain assessment scales, which are described below. To avoid influencing physicians’ behavior, physicians were not provided with scores from the previsit interview. The initial visit was then videotaped in its entirety (mean medical interview duration was 38.7 minutes, SD 12.9 minutes). After being seen by the physician, patients were again interviewed and asked about their satisfaction with the visit. The medical record note for that visit was later reviewed by physician abstractors for the presence of the diagnosis of pain or other indication that patient pain was recognized. The study methods were approved by our institutional Human Subjects Committee. Videotapes were analyzed using the Davis Observation Code (DOC).
Measures
The MOS SF-36 is a reliable and valid 36-item questionnaire made up of 8 scales: general health, physical function, physical role, mental role, social function, pain, energy, and mental health.16,17 Scales are scored so that higher scores reflect better health status. Of note here is the pain scale, which is interpreted as “absence of pain,” with higher scores reflecting lesser amounts of pain.
The visual analog pain scale is a simple-to-use instrument consisting of a 10-cm line placed horizontally on the paper with “No pain” and “The worst pain you could possibly imagine” placed at the left and right ends, respectively.
Patients were instructed to mark the spot on the line correlating to the level of all pain being experienced at the time of the medical visit. The level of pain is calculated by measuring (in millimeters), the distance from the left end of the scale to the mark. The validity, sensitivity, and reliability of the visual pain analog scale have been confirmed.18–20
Physician practice styles were examined by coding the videotapes using the DOC.21,22 DOC is a reliable and valid interactional analysis system that has been used to analyze physician practice style differences in a number of previous studies.22–31 Observers recorded the occurrence of each of 20 clinically important behaviors during successive 15-second observation intervals of the medical visit. For each DOC code, the number of intervals during which the associated behavior was observed was expressed as a percentage of the total of all DOC-coded behaviors noted during the visit. Videotapes were coded by a total of 33 coders. To determine the presence of any observer bias, approximately 20% of the videotapes were coded by a second observer, with a stratified kappa coefficient of agreement of 91.6%.32 The stratified kappa was computed as the weighted average of kappa coefficients for the 20 DOC codes. Six different clusters of physician practice behaviors have been identified to characterize practice style based on an evaluation of the clinical and statistical relationships among the 20 DOC codes. Practice behavior clusters include from 1 to 8 DOC-coded behaviors, with each cluster representing the relative time spent on those behaviors. The 6 clusters are technical, health behavior, addiction, patient activation, preventive services, and counseling (Appendix A ▶). A detailed description of these practice style clusters has been previously published.22
Appendix A.
Codes in Each Cluster Technical Cluster | Abbreviated Davis Observation Code (DOC) Definitions |
Structuring interaction | Discussing what is to be accomplished in current interactions |
History taking | Physician inquiring about or patient describing details related to the chief complaint or to previous illnesses |
Family information | Discussing family medical or social history, and/or current family functioning |
Physical examination | Any aspect of physical examination of patient |
Evaluation feedback | Physician telling patient about results of history, physical examination, laboratory work, etc |
Planning treatment | Physician prescribing a medication, diagnostic, or treatment plan |
Treatment effects | Physician inquiring about or patient describing result of ongoing therapeutic intervention |
Procedure | Any treatment or diagnostic procedure done in office |
Health behavior cluster | |
Compliance | Discussing previously requested behavior |
Health education | Physician presenting information regarding health to patient |
Health promotion | Physician asking for change in patient’s behavior in order to increase or promote health |
Nutrition | Any question or discussion about nutrition |
Exercise | Any question or discussion about exercise |
Addiction Cluster | |
Substance use | Any question or discussion of drinking alcohol or use of other substances |
Smoking behavior | Any question about or discussion of smoking or other use of tobacco |
Patient activation cluster | |
Health knowledge | Physician asking or patient spontaneously offering what patient knows or believes about health and disease |
Patient question | Patient asking question |
Chatting | Discussion of topics not related to current visit |
Preventive service cluster | |
Preventive service | Physician discussing, planning, or performing any screening task with disease prevention |
Counseling cluster | |
Counseling | Physician discussing interpersonal relations or current emotional state of patient or patient’s family |
An 18-item general patient satisfaction questionnaire, using a modified version of the satisfaction instrument developed by Ware and associates,33,34 was administered after the initial visit to measure visit-specific general satisfaction with the health care giver. Answers to the questionnaire were scored with a 5-point Likert scale with 5 points representing the highest satisfaction.
Analysis
Because some physicians saw more than 1 patient, hierarchical models were used to account for the nesting effects and induced correlations, even though the estimated intercluster correlations did not achieve statistical significance. Subject-specific (mixed effects) logistic regression models,35–37 treating physicians as the random effect, were fit to the data using version 8.2 of the SAS statistical software.38
RESULTS
Of the 509 study patients, 62% were female and 38% were male. The ethnic distribution was 63% white, 22% African American, 8% Hispanic, 4% Asian, and 3% Native American. Patients had a mean age of 41.3 years and a median of 12 years of education. Family income for 74% of the study group was below $20,000.
Global pain scores measured by the Visual Analog Scale ranged from 0 to 100. Patients had a mean global pain score of 41.1. A comparison of female and male pain scores using 2-tailed t tests showed that women reported a significantly greater amount of global pain than men (46.4 vs 32.3, P <.0001). Global visual analog pain scale scores were highly correlated inversely with MOS SF-36 physical health status and “absence of pain” scales (-.60 and -.77, respectively). The gender difference was also observed for the MOS SF-36 data, with women reporting lower general health status and greater pain than men.
As displayed in Table 1 ▶, patients were divided into 3 groups based on their global visual analog pain scale scores: (1) pain score ≥10, zero to little pain; (2) pain score >10 but <70, medium pain; (3) pain score ≤70, severe pain. The cut points for this grouping were based on the first and third quartiles of the pain scores; 11 and 68, respectively. The second row of Table 1 ▶ shows patients in whom the physician diagnosed pain. As the amount of self-reported pain increased, the percentage of patients recognized by the physician as being in pain increased as well: the percentage of patients with zero to little pain who had pain diagnosed was 18.03% compared with 47.15% for those reporting medium pain and 69.75% for those reporting severe pain. Using chi-square analysis, a significant difference was found between the 3 pain groups for the physician diagnosis of pain (P <.0001).
Table 1.
Patients | Pain Score ≤ 10*Zero to Little Pain n (%) | Pain Score >10 – ≥70 Medium Pain n (%) | Pain Score ≥ 70 Severe Pain n (%) | Total n (%) |
No diagnosis of pain | 100 (81.97) | 139 (52.85) | 36 (30.25) | 275 (54.56) |
Diagnosis of pain† | 22 (18.03) | 124 (47.15) | 83 (69.75) | 229 (45.44) |
Total | 122 (100) | 263 (100) | 119 (100) | 504 (100) |
Note: there were 504 patients having both global visual pain analog scale scores and medical record data regarding physician diagnosis of pain in index appointment.
* Global Pain Scale 1–100.
† Chi-square, P <.0001, significant differences between the 3 groups for the diagnosis of pain.
The 3 patient groups were then divided into female and male subgroups to explore whether there were any differences by gender in the accuracy of physician diagnosis of pain (Table 2 ▶). In patients with severe pain, a trend was observed for more false-negative diagnoses of pain for men than for women (41.94% vs 26.14%) and a greater number of true-positive diagnoses of pain for women than for men (73.86% vs 58.06%); however, these findings did not reach statistical significance (P = .0996).
Table 2.
Patients | Pain Score ≤ 10 Zero to Little Pain n (%) | Pain Score >10 – ≤70 Medium Pain n (%) | Pain Score ≥70 Severe Pain n (%) | Total (N = 504) |
Women only No diagnosis of pain | 46/57 (80.70) | 92/168 (54.76) | 23/88 (26.14) | 161 |
Women only* Diagnosis of pain | 11/57 (19.30) | 76/168 (45.24) | 65/88 (73.86) | 152 |
Men only No diagnosis of pain | 54/65 (83.08) | 47/95 (49.47) | 13/31 (41.94) | 114 |
Men only† Diagnosis of pain | 11/65 (16.92) | 48/95 (50.53) | 18/31 (58.06) | 77 |
* Chi-square, P =.0059, significant differences between the 3 groups for the diagnosis of pain for women only.
† Chi-square, P =.0092, significant differences between the 3 groups for the diagnosis of pain for men only.
The relationship between patient pain and satisfaction with medical care was also investigated with t tests. The 36 patients in severe pain who were not recognized as being in pain by their physician actually had a higher, but not significantly different, mean satisfaction scores than the 82 patients who were correctly diagnosed by their physicians (4.24 and 4.11, respectively, P =.4480).
To account for the clustering effect of patients visiting the same physician, we used generalized linear mixed models to model the diagnosis of pain.39–40 Mixed effects logistic regression equations relating the diagnosis of pain by the physician with physician practice style cluster behaviors are shown in Table 3 ▶. Other practice style clusters and independent variables, such as patient sociodemographic characteristics (including gender, age, income, and marital status) did not achieve statistical significance at P <.05, and therefore are not included in these regressions. A practice style emphasizing technical practice behaviors (which includes such DOC behaviors as history taking, asking family information, structuring the interaction, performing physical examination, giving evaluation feedback, planning treatment, discussing the effects of treatment, and performing in-office procedures), as well as health behavior discussions (including compliance with medical regimen, health education, health promotion, nutrition, and exercise), strongly predicted the diagnosis of pain for the entire group of study patients. There were similar findings for regressions in which the diagnosis of pain in the male patient group was examined. The likelihood of female patients having pain diagnosed was greater, however, when the physician was more technically oriented and spent less time discussing and performing preventive services (defined as the physician discussing, planning, or performing any screening task associated with disease prevention).
Table 3.
Diagnosis of Pain | Independent Variables Physician Practice Style Cluster | Standardized Coefficients | PValues | OR (95% CI) |
All patients | Technical | 0.3494 | <.0001 | 1.074 (1.071–1.076) |
Health behavior | 0.2276 | <.0001 | 1.066 (1.054–1.079) | |
Women only | Technical | 0.1349 | <.0001 | 1.028 (1.025–1.032) |
Preventive service | −0.1414 | <.001 | 0.941 (0.909–0.975) | |
Men only | Technical | 0.4694 | <.0001 | 1.096 (1.092–1.101) |
Health behavior | 0.3153 | <.0001 | 1.093 (1.071–1.116) |
OR = odds ratio, CI = confidence interval.
*Complete data for all independent variables tested available for 500 patients.
DISCUSSION
Physician practice style has been shown to be influenced by a variety of factors, including health care setting, length of the physician-patient relationship, patient sociodemographic characteristics, mental health problems, and general health status.24–31 In previous work, we described patient variables that predicted the physician’s diagnosis of depression.26,31,39 Identifying the factors that affect the recognition of patient pain is equally important as a first step to improving its detection and appropriate management in the outpatient setting.
In this study, the association between increased pain scale scores and physician diagnosis of pain was confirmed. The diagnosis of pain for patients with severe pain was almost 70% compared with 47% for those scoring in the medium pain range. About 30% of the time, however, physicians did not recognize pain in patients reporting severe pain in the previsit interview. This underrecognition may be due to a variety of factors. The patient may not have mentioned pain to the physician, or may not have displayed obvious discomfort, despite reporting it during the previsit interview. In addition, the medical record review may have underestimated the physician’s recognition of pain. The physician might have been rushed and neglected to write a detailed note to document pain as a distinct diagnostic entity. Every effort was made, however, to acknowledge fully the physician’s identification of patient pain by defining the diagnosis of pain as any mention of pain symptoms in the patients’ medical history, documenting pain during the physical examination, listing pain in the assessment section of the progress note or in the problem list, ordering diagnostic tests to evaluate pain, treating pain directly, or making a referral to a specialist or the pain clinic.
Compared with male patients, female patients reported significantly greater levels of pain. Gender-specific analyses revealed a trend for physicians to diagnose pain more frequently for women in severe pain than for men. On the other hand, severe pain in men went unrecognized significantly more often. Part of this apparent differential rate of diagnosis of pain may reflect the way women and men report their symptoms. Women may be more likely to be recognized as being in pain because they are more verbally and nonverbally expressive in making their pain known, whereas men are more reticent to complain about pain to their physicians. Women may also be more likely to ask for more types of diagnostic and therapeutic measures for their discomfort than their male counterparts, leading their physicians to focus on the diagnosis of pain. Physicians may also evaluate complaints in men and women in a different manner. They may have attitudes about women being more asthenic and sensitive to pain, whereas they may view men as being physically stronger and more tolerant to pain.
It was hypothesized that the accurate diagnosis of patient pain would result in greater satisfaction with the medical care received. That the data did not support this hypothesis highlights the difficulty in using patient satisfaction as a patient outcome measure. Serious concerns have been raised regarding the appropriateness of patient satisfaction as an end-result outcome, because respondents might not be able to separate satisfaction with pain management from satisfaction with other aspects of care.40
The findings supported the hypothesis that a physician’s practice style and clinical approach to medical care are related to the likelihood of diagnosing pain in patients. For the entire group of patients (both women and men), as well as for only male patients, a technically oriented physician practice style and an approach emphasizing the patient’s health behaviors was predictive of the physician making the diagnosis of pain. For female patients, the technical style of care was again associated with pain recognition. In addition, less of the visit being spent on preventive service activities was associated with a pain diagnosis. It may be that a physician who concentrates on the technical aspects of care (history taking, physical examination, and treatment) is more likely to uncover information and physical evidence for the diagnosis of pain. Moreover, a practice style emphasis on the patient’s health behaviors (both positive and negative) may also elicit information about the patient’s state of health and any painful symptoms. On the other hand, given the time constraints and competing demands of primary care practice, a focus on the diagnosis and treatment of pain may be incompatible with devoting time to prevention.41
This study is limited because it relied on patient self-report of pain and chart notation as an index of the recognition of pain. The study was conducted in a single university medical center with primary care residents. Despite that only senior residents were used, the results would be more generalizable had the study been conducted with practicing community physicians
Recognition of pain appears to be strongly influenced by the severity of pain reported by the patient and the practice patterns physicians bring to the medical encounter. In addition, the gender of the patient appears to play some role in the recognition of pain. Although the explanation for the role of gender is not completely clear, the effect of a patient’s gender may lead to adverse consequences, with men not receiving appropriate evaluation and treatment for pain. These data do provide insights into the process of recognition of pain, as heightened awareness of importance of pain in health care delivery is leading to new legislation regarding pain screening and treatment.
Future research should examine the impact of these recently mandated standards in improving the identification and management of pain. If the routine use of practical pain assessment tools is found to be effective in improving physician recognition and treatment of patients’ pain, then the application of these tools in patient care settings should be encouraged. Barriers to compliance with clinical practice guidelines, however, may exist at both the provider and the organizational level.42,43 Physicians are more likely to change their practice styles to implement a specific guideline when characteristics of the health care setting are supportive.43 Because the quality of health care is a multidimensional property of health systems, rather than solely the result of physician-patient interaction, appropriate pain diagnosis and management will require the combined efforts of both providers and health care organizations.
Acknowledgments
The authors wish to thank Dr. Scott M. Fishman, Chief of the Division of Pain Medicine, Department of Anesthesiology and Pain Medicine at the University of California, Davis, for contributing background information regarding JCAHO Pain Standards and California legislation in this area, as well as standards for pain assessment tools.
Conflicts of interest: none reported
Funding support: This project was supported by a grant (R18 HS06167) from the Agency for Health Care Policy and Research, now known as the Agency for Healthcare Research and Quality.
REFERENCES
- 1.Backonja MM. Mechanisms-based diagnosis and treatment of pain. Pain Manag Today. 2001;1:3. [Google Scholar]
- 2.Marks RM, Sachar EJ. Undertreatment of medical inpatients with narcotic analgesics. Ann Intern Med. 1973;78: 173–181. [DOI] [PubMed] [Google Scholar]
- 3.US Department of Health and Human Services. Acute Pain Management Guideline Panel. Acute Pain Management: Operative or Medical Procedures and Trauma. Clinical Practice Guideline No. 1. Washington, DC: Agency for Health Care Policy and Research, Public Health Services; 1992. Publication 92–0032.
- 4.Hjortdahl P. The influence of general practitioners’ knowledge about their patients on the clinical decision-making process. Scand J Prim Health Care. 1992;10:290–294. [DOI] [PubMed] [Google Scholar]
- 5.McDonald DD, Bridge RG. Gender stereotyping and nursing care. Res Nur Health. 1991;14:373–378. [DOI] [PubMed] [Google Scholar]
- 6.McKinlay JB, Potter DA, Feldman HA. Non-medical influence on medical decision-making. Soc Sci Med. 1996;42:769–776. [DOI] [PubMed] [Google Scholar]
- 7.Todd KH, Samaroo N, Hoffman JR. Ethnicity as a risk factor for inadequate emergency department analgesia. JAMA. 1993;269:1537–1539. [PubMed] [Google Scholar]
- 8.Hadjistavropoulos HD, Ross MA, von Baeyer CL. Are physicians’ ratings of pain affected by patients’ physical attractiveness? Soc Sci Med. 1990;31:69–72. [DOI] [PubMed] [Google Scholar]
- 9.Tait RC, Chibnall JT. Physician judgements of chronic pain patients. Soc Sci Med. 1997;45:1199–1205. [DOI] [PubMed] [Google Scholar]
- 10.Forrest M, Hermann G, Andersen B. Assessment of pain: a comparison between patients and doctors. Acta Anaesthesiol Scand. 1989;33:255–256. [DOI] [PubMed] [Google Scholar]
- 11.Miller WL, Yanoshik MK, Crabtree BF, Reymond BF. Patients, family physicians, and pain: visions from interview narratives. Fam Med. 1994;26:179–184. [PubMed] [Google Scholar]
- 12.Phillips D, for the Joint Commission on Accreditation of Healthcare Organizations. JCAHO pain management standards are unveiled. JAMA. 2000;284:428–429. [DOI] [PubMed] [Google Scholar]
- 13.Frankenstein RS. In reply [letter]. JAMA. 2000;284: 2317–2318.11066170 [Google Scholar]
- 14.vonBaeyer CL, Johnson ME, McMillan MJ. Consequences of nonverbal expression of pain: patient distress and observer concern. Soc Sci Med. 1984;19:1319–1324. [DOI] [PubMed] [Google Scholar]
- 15.Bertakis KD, Azari R, Callahan EJ. Patient pain: its influence on primary care physician-patient interaction. Fam Med. 2003;35:119–123. [PubMed] [Google Scholar]
- 16.Stewart AL, Hays RD, Ware JE, Jr. The MOS short-form general health survey. Reliability and validity in a patient population. Med Care. 1988;26:724–735. [DOI] [PubMed] [Google Scholar]
- 17.McHorney CA, Ware JE, Raczek AE. The MOS 36-item short-form health survey (SF-36): II. Psychometric and clinical tests validity in measuring physical and mental health constructs. Med Care. 1993;31: 247–263. [DOI] [PubMed] [Google Scholar]
- 18.Littman GS, Walker BR, Schneider BE. Reassessment of verbal and visual analog ratings in analgesic studies. Clin Pharmacol Ther. 1985;38:16–23. [DOI] [PubMed] [Google Scholar]
- 19.Revill SI, Robinson J, Rosen M, Hogg MI. The reliability of a linear analogue for evaluating pain. Anaesthesia. 1976;31:1191–1198. [DOI] [PubMed] [Google Scholar]
- 20.Ohnhaus EE, Adler R. Methodological problems in the measurement of pain: a comparison between the verbal rating scale and the visual analogue scale. Pain. 1976;1:379–384. [DOI] [PubMed] [Google Scholar]
- 21.Callahan EJ, Bertakis KD. Development and validation of the Davis Observation Code (DOC). Fam Med. 1991;23:19–24. [PubMed] [Google Scholar]
- 22.Bertakis KD, Callahan EJ, Helms LJ, Robbins JA, Miller J. Physician practice styles and patient outcomes. Differences between family practice and general internal medicine. Med Care. 1998;36:879–891. [DOI] [PubMed] [Google Scholar]
- 23.Robbins JA, Bertakis KD, Helms LJ, Azari R, Callahan EJ, Creten DA. The influence of physician practice behaviors on patient satisfaction. Fam Med. 1993;25:17–20. [PubMed] [Google Scholar]
- 24.Callahan EJ, Bertakis KD. A comparison of physician-patient interaction at fee-for-service and HMO sites. Fam Pract Res J. 1993;13:171–178. [PubMed] [Google Scholar]
- 25.Bertakis KD, Callahan EJ, Helms LJ, Azari R, Robbins JA. The effect of patient health status on physician practice style. Fam Med. 1993;25:530–535. [PubMed] [Google Scholar]
- 26.Callahan EJ, Bertakis KD, Azari R, Robbins JA, Helms LJ, Miller J. The influence of depression on physician-patient interaction in primary care. Fam Med. 1996;28:346–351. [PubMed] [Google Scholar]
- 27.Stange KC, Zyzanski SJ, Jaén CR, et al. Illuminating the ‘black box’: a description of 4454 patient visits to 138 family physicians. J Fam Pract. 1998;46:377–389. [PubMed] [Google Scholar]
- 28.Callahan EJ, Jaén CR, Crabtree BF, Zyzanski SJ, Goodwin MA, Stange KC. The impact of recent emotional distress and diagnosis of depression or anxiety on the physician-patient encounter in family practice. J Fam Pract. 1998;46:410–418. [PubMed] [Google Scholar]
- 29.Bertakis KD, Azari R, Callahan EJ, Robbins JA, Helms LJ. Comparison of primary care resident physicians’ practice styles during initial and return patient visits. J Gen Intern Med. 1999;14:495–498. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Callahan EJ, Bertakis KD, Azari R, Robbins JA, Helms LJ, Chang DW. The influence of patient age on primary care resident physician-patient interaction. J Am Geriatr Soc. 2000;48:30–35. [DOI] [PubMed] [Google Scholar]
- 31.Bertakis KD, Helms LJ, Callahan EJ, Azari R, Leigh P, Robbins JA. Patient gender differences in the diagnosis of depression in primary care. J Women’s Health Gend Based Med. 2001;10:689–698. [DOI] [PubMed] [Google Scholar]
- 32.Barlow W, Lai MY, Azen SP. A comparison of methods for calculating a stratified kappa. Stat Med. 1991;10:1465–1472. [DOI] [PubMed] [Google Scholar]
- 33.Cherkin DC, Hart LG, Rosenblatt RA. Patient satisfaction with family physicians and general internists: Is there a difference? J Fam Pract. 1988;26:543–551. [PubMed] [Google Scholar]
- 34.Ware JE, Davies-Avery A, Stewart AL. The measurement and meaning of patient satisfaction. Health Med Care Serv Rev. 1978;1:1–15. [PubMed] [Google Scholar]
- 35.McCullogh CE, Searle SR. Generalized Linear And Mixed Models. New York: John Wiley and Sons; 2000.
- 36.Burton P, Gurrin L, Sly P. Tutorial in biostatistics: Extending the simple linear regression model to account for correlated responses: an introduction to generalized estimating equations and multilevel mixed modeling. Stat Med. 1998;17:1261–1291. [DOI] [PubMed] [Google Scholar]
- 37.Betensky, RA and Williams PL. A comparison of models for clustered binary outcomes: analysis of a designed immunology experiment, Appl Stat. 2001;50:43–61. [Google Scholar]
- 38.SAS Institute Inc. SAS/STAT User’s Guide. Version 8. Cary, NC: SAS Institute, Inc; 1999.
- 39.Callahan EJ, Bertakis KD, Azari R, Helms LJ, Robbins J, Miller J. Depression in primary care: patient factors that influence recognition. Fam Med. 1997;29:172–176. [PubMed] [Google Scholar]
- 40.Hester NO, Miller KL, Foster RL, Vojir CP. Symptom management outcomes: do they reflect variations in care delivery symptoms? Med Care. 1997;35:NS69–NS83. [DOI] [PubMed] [Google Scholar]
- 41.Stange KC, Flocke SA, Goodwin MA. Opportunistic preventive services delivery. Are time limitations and patient satisfaction barriers? J Fam Pract. 1998;46:419–424. [PubMed] [Google Scholar]
- 42.Mechanic D. Improving the quality of health care in the United States of America: the need for a multilevel approach. J Health Serv Res Policy. 2002;7(Suppl 1):S35–39. [DOI] [PubMed] [Google Scholar]
- 43.Vaughn TE, McKoy KD, BootsMiller BJ, et al. Organizational predictors of adherence to ambulatory care screening guidelines. Med Care. 2002;40:1172–1185. [DOI] [PubMed] [Google Scholar]