Accuracy of General Practitioners’ Assessment of Chest Pain Patients for Coronary Heart Disease in Primary Care: Cross-sectional Study with Follow-up

Abstract

Aim

To estimate how accurately general practitioners’ (GP) assessed the probability of coronary heart disease in patients presenting with chest pain and analyze the patient management decisions taken as a result.

Methods

During 2005 and 2006, the cross-sectional diagnostic study with a delayed-type reference standard included 74 GPs in the German state of Hesse, who enrolled 1249 consecutive patients presenting with chest pain. GPs recorded symptoms and findings for each patient on a report form. Patients and GPs were contacted 6 weeks and 6 months after the patients’ visit to the GP. Data on chest complaints, investigations, hospitalization, and medication were reviewed by an independent panel, with coronary heart disease being the reference condition. Diagnostic properties (sensitivity, specificity, and predictive values) of the GPs’ diagnoses were calculated.

Results

GPs diagnosed coronary heart disease with the sensitivity of 69% (95% confidence interval [CI], 62-75) and specificity of 89% (95% CI, 87-91), and acute coronary syndrome with the sensitivity of 50% (95% CI, 36-64) and specificity of 98% (95% CI, 97-99). They assumed coronary heart disease in 245 patients, 41 (17%) of whom were referred to the hospital, 77 (31%) to a cardiologist, and 162 (66%) to electrocardiogram testing.

Conclusions

GPs’ evaluation of chest pain patients, based on symptoms and signs alone, was not sufficiently accurate for diagnosing or excluding coronary heart disease or acute coronary syndrome.

When general practitioners (GP) treat patients with chest pain, they have to decide whether there is a serious underlying pathology requiring urgent action or whether a “wait and see” strategy can be applied. Chest pain can be caused by a wide range of different illnesses, among which life-threatening cardiac disease is of the greatest immediate concern (1,2). However, chest pain is caused by coronary heart disease (CHD) in only around 12-15% of primary care patients (3-5). For most of patients with chest pain, the GP remains the main point of entry into the health care system. The effectiveness of GPs’ gatekeeping role, ie, identifying patients with CHD and protecting patients from over-diagnosis and treatment, depends on the accuracy of their provisional diagnosis after taking the patient’s history and performing the basic clinical examination. So far, a limited number of studies have addressed this question (6-10). There is a need for additional data on GPs’ management decisions after assumed CHD diagnosis, derived from a large and consecutively recruited sample of chest pain patients in primary care.

In this study, we aimed to investigate how accurately GPs’ assessed the probability of CHD in patients presenting with chest pain and to analyze management decisions taken as a result.

Methods

The data for this study came from a larger project aimed at investigating the accuracy of signs and symptoms for chest pain patients with CHD, and the methodology and results of the main study have been presented elsewhere (5,11). In the present study, a secondary analysis was conducted to assess the accuracy of GPs’ diagnostic assessment of chest pain patients in primary care facilities in the state of Hesse, Germany. The final diagnosis was established by an expert panel after 6 months of follow-up, a procedure referred to as delayed-type reference standard (12).

GPs and patients

A convenience sample of 209 GPs in the German state of Hesse was chosen; 35% agreed to participate in the study. The GPs consecutively recruited every attending patient with chest pain over a period of 12 weeks. They also recruited patients during home visits and emergency calls. For logistical reasons recruitment was staggered in four waves between October 2005 and July 2006. Inclusion criteria were age above 35 years and pain localized to the anterior chest wall in the area between clavicles, lower costal margins, and the posterior axillary lines. Patients were eligible irrespective of whether their complaints were acute or chronic, and whether they had previously known conditions including CHD or related risk factors. Patients whose chest pain had subsided for more than one month and whose chest pain had already been investigated were excluded.

Data collection

GPs took patients’ standardized history, performed a physical examination, and recorded their preliminary diagnoses, investigations, and management of chest pain on a standardized case report form used for the purposes of this study. They also rated the certainty of their provisional diagnosis and assessed the probability of CHD on a visual analogue scale ranging from 0 to 100%. We used these data to allocate the patients into 4 different groups of CHD probability as assessed by the GP: no (0%); low (1-5%); intermediate (6-80%); high (81-100%).

Our study assistants contacted the patients by phone 6 weeks and 6 months after the consultation at the time of recruitment (index consultation) and asked about the course of the patients’ chest pain and treatments including hospitalizations and drugs. Possible discharge letters were requested by GPs from specialists and hospitals. The assistants also contacted GPs to recover the missing data for patients who were lost to follow-up or who provided incomplete information.

After a completed follow-up at 6 months, a reference panel of one cardiologist, one GP, and one member of the research staff of the Department of General Practice at the University of Marburg reviewed each patient’s data. Neither the cardiologist nor the GP were involved in study design; the researcher was the principal investigator and therefore involved in study planning. They determined whether CHD was present or absent at the index consultation. As patient history is part of the definition of acute and chronic CHD, providing the reference panel clinical data recorded by GPs, including preliminary diagnoses, would have raised the possibility of incorporation bias. Therefore, the panel assessed each patient first without the index test; in other words, they used only the information gathered at follow-up (blinded reference standard). In the second round, the panel reviewed the patients' follow-up data together with history and findings recorded by GPs (unblinded reference standard). Patients’ files were reordered before the records were presented to the panel during the second round of review.

GPs were visited at 4 week intervals to check report forms, recruitment logs, and compliance with study procedures. Random audits were performed by searching routine documentation of participating practices to identify cases of chest pain not included in the study. The study protocol was approved by the Ethics Committee of the Faculty of Medicine, University of Marburg and it complies with the declaration of Helsinki (13).

Statistical analysis

The analysis of the diagnostic accuracy of GPs’ provisional diagnoses and further management was based on the sample of all 1249 patients presenting with chest pain. The χ² test was used to analyze two-by-four tables of selected baseline characteristics to detect significant differences in the GPs’ assessment of CHD probability. In the next step, these characteristics were included as independent variables in multivariable logistic regression analysis. The dependent variable was high CHD probability (81-100%). Variable selection was conducted using the backward stepwise procedure with a significance threshold of P < 0.05. Odds ratios and 95% confidence intervals were calculated for both patient groups.

Two-by-two tables were used to calculate diagnostic properties (sensitivity, specificity, and predictive values) of the GPs’ provisional diagnoses.

The diagnostic accuracy of GPs’ assessment that chest pain was related to CHD was listed as a continuous variable in the case report form filled out by GPs. To measure this diagnostic accuracy, we calculated the area under the curve (AUC) of receiver operating characteristic (ROC) curves, with corresponding confidence intervals. Analyses were performed with SPSS software, version 15.0 (SPSS Inc., Chicago, IL, USA).

Results

GPs’ and patients’ characteristics

The mean age of the participating GPs was 49 years (95% confidence interval [CI], 47.5-50.5), 64% worked in urban areas, and 67% were male. Their demographic characteristics were similar to those of the population of GPs in the state of Hesse (data available upon request). They had approximately 190 000 consultations during the study period. A total of 1355 patients with chest pain were asked to participate; 7 patients did not meet the inclusion criteria because they were aged 35 years or below and 99 refused to participate. Valid case report forms were returned for 1249 patients. The vast majority of patients were known to their GPs from former consultations.

Three cases were early drop-outs and were not included. For 34 cases, follow-up information was lacking, incomplete, or ambiguous so that no final diagnosis could be made. Additional 60 cases were lost to follow-up and 11 died; nevertheless, all 71 of these cases provided enough information to be judged by the reference committee. At 6 months, we thus analyzed 1212 patients (534 men) for the etiology of their chest pain (Figure 1). The mean age in this group was 59 years (95% CI, 57.8-61.0 years). Fifty-three percent of the patients had chest pain at the time of consultation and 30% had acute chest pain, defined as chest pain lasting less than 48 hours. A total of 180 (92 men) were diagnosed as having CHD, corresponding to a prevalence of 15%.

Figure 1.

Flow of patients in the study.

Association between baseline characteristics of chest pain patients and GPs’ diagnosis of CHD probability

GPs assessed CHD probability to be high in 83 (6.6%) patients, intermediate in 759 (61%) patients, and low in 172 (14%) patients. A 0% probability of CHD was established for 235 (19%) patients. These GP assessments were associated with certain baseline characteristics of patients with chest pain. Characteristics, such as patients’ assumptions that their chest pain was cardiac in origin, aspects of pain character (pressing, tightness), worsening of pain with exercise, and risk factors or previous diseases (hyperlipidemia, diabetes mellitus, hypertension, known vascular disease) showed an association with the degree of CHD probability assessed by the GP (Table 1).

Table 1.

Baseline characteristics of chest pain patients (n = 1249) in relation to the general practitioners’ provisional diagnosis of coronary heart disease (CHD) or no CHD after history taking and clinical examination

Baseline characteristic (n, %)	CHD probability assessed by general practitioners*
	no CHD (n = 235)	low (n = 172)	intermediate (n = 759)	high (n = 83)	P
Patient assumes cardiac origin of pain†	56 (27)	72 (47)	484 (71.3)	62 (89.9)	<0.001
Acute pain (lasting <48 h)†	69 (30)	62 (36)	209 (27.9)	24 (30.4)	0.214
Pressing pain†	66 (28)	68 (40)	346 (45.8)	58 (72.5)	<0.001
Pain worsens with exercise	30 (13)	28 (16)	167 (22)	40 (48)	<0.001
Tightness	39 (17)	55 (32)	274 (36)	45 (54)	<0.001
Hyperlipidemia	53 (23)	25 (15)	217 (29)	42 (51)	<0.001
Diabetes mellitus	24 (10)	17 (9.9)	100 (13)	26 (31)	<0.001
Smoking	42 (18)	29 (17)	92 (12)	6 (7.2)	0.023
Hypertension	71 (30)	44 (26)	357 (47)	56 (68)	<0.001
Family history of CHD	14 (6.0)	14 (8.1)	90 (12)	11 (13)	0.038
History of vascular disease‡	21 (8.9)	10 (5.8)	139 (18)	52 (63)	<0.001
Left-sided pain	120 (51.1)	109 (63)	521 (69)	46 (55)	<0.001
Retrosternal pain	114 (48.5)	84 (49)	459 (61)	63 (76)	<0.001
Old age (women ≥65 y or men ≥55 y)	89 (37.9)	55 (32.0)	410 (54)	73 (88)	<0.001

*No – 0%; low – 1-5%; intermediate – 6-80%; high – 81-100%.

†For some index tests, data were missing from the case report form. Therefore, the indicated index tests may not add up to 1249 patients.

‡Vascular disease was defined as CHD, occlusive vascular disease, or cerebrovascular disease.

Table 2 shows the predictors that were significantly related to GPs’ estimation of high CHD probability. The patients' own assumptions and a history of known vascular disease showed the highest correlation. Pain localization (left-sided pain) was negatively associated.

Table 2.

Association between baseline characteristics and general practitioners’ assessment of high coronary heart disease probability (n = 1082)*

Patients’ characteristics	Adjusted odds ratio (95% confidence interval)	P
Patient assumes cardiac origin of pain	7.86 (2.94-20.99)	<0.001
Acute pain (lasting <48 h)	1.82 (0.89-3.71)	0.100
Pressing pain	2.64 (1.40-4.96)	0.003
Pain worsens with exercise	3.24 (1.76-5.98)	<0.001
Hyperlipidemia	1.70 (0.94-3.08)	0.081
History of vascular disease†	7.76 (4.16-14.46)	<0.001
Left-sided pain	0.39 (0.21-0.73)	0.003
Old age (women ≥65 y or men ≥55 y)	2.80 (1.31-6.02)	0.008

*The number of patients included here (1082) is lower than in Table 1 because of missing data that were not included in regression analysis; 63 of the 1082 patients (5.8%) included in Table 2 were classified as having high CHD probability.

†Vascular disease was defined as CHD, occlusive vascular disease, or cerebrovascular disease.

Accuracy of GPs’ diagnosis

When GPs' diagnoses of CHD were compared with the reference panel's determinations using the ROC statistics, the AUC was 0.83 (95% CI, 0.80-0.87; Figure 2). Sex differences were not observed: the AUC was 0.82 (95% CI, 0.76-0.87) for female and 0.85 (95% CI 0.80-0.89) for male patients. Table 3 shows the comparison between the GPs’ diagnoses of CHD and acute coronary syndrome (ACS) and the reference standard. GPs assessed CHD probability with a sensitivity of 69% and a specificity of 89%.

Figure 2.

Accuracy of general practitioners’ assessment that chest pain was related to coronary heart disease (receiver operating characteristic, AUC – area under the curve).

Table 3.

Diagnostic accuracy of general practitioners’ (GP) assessment of coronary heart disease and acute coronary syndrome in comparison with the final reference diagnosis (n = 1212)

Diagnostic accuracy (%)	Coronary heart disease assessed by GP; mean (95% confidence interval)	Acute coronary syndrome assessed by GP; mean (95% confidence interval)
Sensitivity	69 (62-76)	50 (36-64)
Specificity	89 (87-91)	98 (97-99)
Positive predictive value	53 (47-60)	51 (36-67)
Negative predictive value	94 (93-96)	98 (97-99)

Of the 44 patients determined by the reference panel to have ACS, only 18 (41%) were sent immediately to the hospital.

GPs’ management decisions for patients after diagnosing or excluding CHD

Table 4 lists the management decisions that GPs took after assessing whether the patient had CHD. Of the 245 patients diagnosed with CHD, 41 (17%) were referred to the hospital, 77 (31%) to a cardiologist, and 162 (66%) to electrocardiogram (ECG) testing.

Table 4.

General practitioners’ management decisions for patients after diagnosing or excluding coronary heart disease (CHD) (n = 1249)

Management decision (n, %)	Provisional diagnosis of CHD
	no CHD (n = 1004)	CHD (n = 245)*
Hospital admission	15 (1.5)	41 (17)
Referral to cardiologist	67 (6.7)	77 (31)
Electrocardiogram	552 (55)	162 (66)
Stress test	46 (4.6)	23 (9.4)

*113 (46.1%) of these had previously been diagnosed with coronary heart disease.

Discussion

GPs assessed the probability of CHD with a sensitivity of 69% and a specificity of 89%. Significant predictors of GPs’ estimation of a high CHD probability were patient’s assumption that his or her pain was cardiac in origin and a history of vascular disease.

GPs showed moderate diagnostic accuracy for CHD, with a sensitivity of nearly 70% and a specificity of nearly 90% when compared with the reference standard. Nilsson et al found very similar results for sensitivity (72%) and specificity (89%) (7). A study conducted in Belgium that included only patients with a new episode of chest pain showed that GPs made a correct initial diagnosis in 82% of patients. In that study, in 9 of 10 cases with myocardial infarction, and 19 out of 21 cases with angina, the initial working diagnosis matched the final reference diagnosis (6). In a study with standardized patients, 95% of GPs correctly diagnosed angina pectoris (9). The better diagnostic accuracy of their provisional diagnosis might be explained by the fact that our sample included patients with all types of chest pain, while theirs included patients with a history of typical angina pectoris. Martina et al examined whether in patients with abdominal or chest pain, a non-organic diagnosis could be reliably distinguished from an organic one; they found a diagnostic accuracy of 88% for organic origin of chest pain and 100% for non-organic pain (8). A study looking at GPs’ ability to diagnose myocardial infarction found an unexpectedly low diagnostic accuracy, with a sensitivity of 44%, leading them to conclude that myocardial infarction should not be diagnosed on the basis of symptoms and signs alone (10). This corresponds with our findings that show a sensitivity of 50% for the provisional diagnosis of ACS.

In our study, GPs assessed CHD probability to be very low (0%) or low (1%-5%) even if risk factors like smoking, hyperlipidemia, or diabetes mellitus were present. In a study by Nilsson et al, GPs also found low CHD probability in a group where risk factors were present (7). However, comparison might be problematic as Nilsson et al (7) left the estimation of CHD probability (“high” or “low”) to the discretion of the GP.

In general, classical cardiac risk factors that predict the long-term development of CHD mostly have not been helpful in the acute care setting (14). However, Chun et al found age and history of prior myocardial infarction to be relevant predictors of CHD (15). These two factors also positively correlated with GPs’ assessment of high CHD probability in our sample.

There were no differences in GPs’ assessment of the probability that chest pain was caused by CHD between male and female patients. This contrasts with reports that GPs tend to assume a higher CHD likelihood for men (16).

GPs referred 16.7% of patients with assumed CHD to the hospital. The corresponding percentage in a Swiss study group was 19% (4), while in Iceland and Sweden the hospital referral rate was 35% (7,17). These differences cannot be explained by the inclusion criteria in the studies as they did not differ much. An explanation, however, may be a large variation in referral procedures among primary care systems in different countries. Indeed, a Belgian study reported that nearly 40% of all patients with chest pain were referred to the hospital (18).

GPs referred 31% of patients with assumed CHD to a cardiologist and 9.4% to an exercise test. These actions were reasonable given that 46% of these patients had previously been diagnosed with CHD and therefore did not need to be sent again to a cardiologist. The low rate of exercise test referrals may be explained by the fact that only some German GPs perform this procedure in their own practices. When this referral rate is added to the rate of referrals to a cardiologist, the figure is similar to the findings of Nilsson et al, who found an exercise testing referral rate of 42% for CHD patients (7).

In the present study, an ECG was performed in over a half of patients, irrespective of whether GPs had provisionally diagnosed CHD or not. This corresponds with findings of Klinkman et al, who reported obtaining an ECG in 51% of chest pain episodes (2). While certain ECG findings are specific markers of acute CHD, the ECG shows low sensitivity and is of limited value for CHD diagnosis (19). Despite this fact, GPs tend to order an ECG for the majority of patients with chest pain.

Strengths of this study are prospective design, a large and representative consecutive sample, and small drop-out rates. Study procedures, including random audits, reduced the possibility of selection bias, and an interdisciplinary team provided a precise diagnosis as reference standard. There was no interference with the work-up provided by the participating GPs. As a result, only limited clinical data were available to the reference panel for part of the patients, which may have been a limitation to the study.

This was especially the case in elderly patients, some of whom had already been diagnosed with CHD, and whom GPs did not examine beyond taking a resting ECG. Even fewer data were available for the first assessment by the reference panel, ie, the assessment involving only the index tests and blinded to clinical data recorded by GPs. We made a trade-off between reducing incorporation bias by blinding and including comprehensive data, including results of index tests, to allow fully-informed reference decisions. Therefore, the reference standard in our study cannot be regarded as perfect.

The GPs’ evaluation of chest pain patients, based on symptoms and signs alone, was not sufficiently accurate to safely diagnose or exclude CHD. Further tools such as clinical decision rules may be of additional help (20,21). Nevertheless, the analysis of GPs’ management decisions showed that the general practice setting provides the necessary filter function for the management of patients with chest pain. In other words, GPs’ subsequent management decisions compensate to some degree for the lack of accuracy and sensitivity of their initial assessment.