Physician Surveys to Assess Customary Care in Medical Malpractice Cases

Abstract

OBJECTIVE

Physician experts hired and prepared by the litigants provide most information on standard of care for medical malpractice cases. Since this information may not be objective or accurate, we examined the feasibility and potential value of surveying community physicians to assess standard of care.

DESIGN

Seven physician surveys of mutually exclusive groups of randomly selected physicians.

SETTING

Iowa.

PARTICIPANTS

Community and academic primary care physicians and relevant specialists.

INTERVENTIONS

Included in each survey was a case vignette of a primary care malpractice case and key quotes from medical experts on each side of the case. Surveyed physicians were asked whether the patient should have been referred to a specialist for additional evaluation. The 7 case vignettes included 3 closed medical malpractice cases, 3 modifications of these cases, and 1 active case.

MEASUREMENTS AND MAIN RESULTS

Sixty-three percent of 350 community primary care physicians and 51% of 216 community specialists completed the questionnaire. For 3 closed cases, 47%, 78%, and 88% of primary care physician respondents reported that they would have made a different referral decision than the defendant. Referral percentages were minimally affected by modifying patient outcome but substantially changed by modifying patient presentation. Most physicians, even those whose referral decisions were unusual, assumed that other physicians would make similar referral decisions. For each case, at least 65% of the primary care physicians disagreed with the testimony of one of the expert witnesses. In the active case, the response rate was high (71%), and the respondents did not withhold criticism of the defendant doctor.

CONCLUSIONS

Randomly selected peer physicians are willing to participate in surveys of medical malpractice cases. The surveys can be used to construct the distribution of physician self-reported practice relevant to a particular malpractice case. This distribution may provide more information about customary practice or standard of care than the opinion of a single physician expert.

Malpractice law holds physicians liable for injury caused by their negligence. The negligence standard, however, is not one of outstanding skill or care. “Unless they contract to do more, the law exacts of physicians and surgeons only that they possess and exercise that reasonable degree of skill, knowledge, and care ordinarily possessed and exercised by members of their profession under similar circumstances.”¹ In other areas of negligence law, customary practice is not the standard for liability, but “in medical malpractice cases there is no practical alternative to customary norms as the standard of due care.”^2,3 In justifying reliance on custom as the standard of care, “Courts have recognized the fact that laymen [jurors] lack the capacity to adequately evaluate a physician's conduct or to adequately determine what a reasonable and prudent man under the same circumstances with specialized training and knowledge would have done.”⁴ Courts have refined customary care by specifying that the customary standards are determined locally or at least by similar communities. “The most common rule is a statewide standard for general practitioners and a nationwide standard for specialists.”⁵

A qualification to this reliance on custom is the practice of medicine according to a second school of thought or respectable minority formulation. According to this school “a doctor is not negligent merely by differing from some of his peers if he follows a course of conduct that has the support of a ‘school of practice’ or a considerable number of practitioners in good standing.”⁶ Treatments conforming to such a second school do not constitute malpractice.⁷

To determine customary standards at trial, each party's attorney selects and hires a physician expert. Because experts are selected, hired, and prepared by one of the litigants, their testimony is often partial and conflicting. At best, medical expert witnesses are subjected to subtle pressures and influences by the adversarial system that can compromise accuracy. Some argue that the system makes impartial testimony impossible⁸ and may cause lay jurors difficulty in assessing the validity of conflicting testimony.

There have been attempts to correct the inherent flaws of adversarial experts by improving the objectivity and the quality of expert testimony in medical malpractice cases.^9,10 However,

[medical] organizations have not assumed a strong role over physician expert witnesses because of the inherent difficulties, both conceptual and practical, in conducting peer review of expert witness testimony. Several of those difficulties include assembling a true peer review panel, determining all of the relevant facts, defining the standard of acceptable testimony, imposing meaningful sanctions, and—most problematic—subjecting medical and specialty organizations and physicians conducting peer review to legal risk.¹¹

Other efforts to neutralize the excesses of adversarial expert witnesses include court-appointed experts and medical malpractice screening panels. According to one author, “The most important reform we could make to raise scientific standards in the courtroom would be for judges to appoint expert witnesses rather than to rely on witnesses hired by opposing lawyers.”¹² Thus far, however, court-appointed experts have been used infrequently.¹³

Medical malpractice screening panels have also been used to provide impartial input to reduce the number of frivolous lawsuits. The role of these panels has been severely curtailed, because the U.S. Constitution and similar state provisions give any litigant the right to a jury trial.¹⁴

Even the use of objective, highly qualified experts, however, does not eliminate fundamental problems with the current legal system; experts are not asked how medicine should be practiced but rather how it is practiced. Since literature on customary care for most medical problems is lacking, the opinions of experts on customary care is speculative and may be skewed by biases that affect human judgment.¹⁵

This study explored surveys of peer-group physicians as a method for obtaining medical input on customary standards of practice. We evaluated whether these surveys may be feasible and of potential value in malpractice cases.

METHODS

Case Description

A Nebraska law firm supplied depositions of the contending medical experts from 3 medical malpractice cases that are summarized in Table 1. The cases were chosen so that the defendant was a family physician, the issue was whether the physician met customary standards of medical practice, there were depositions from medical experts hired by each of the litigants, and the dispute did not hinge on factual information (e.g., what the physician told or did to the patient). For each case we used information from physician depositions to create a 1-page abstract summarizing the available medical information. After adjudication, the lawyers who argued the cases reviewed each summary. They agreed that, with an exception of 1 error on a patient's weight, the summaries were fair representations of the relevant facts and circumstances of the cases.

Table 1.

Summaries of Case Abstracts

Case	Summary
1. Alleged failure to diagnose coronary artery disease	The patient was a 47-year-old man who complained of chest pain and requested medical clearance for an exercise program. The pain occurred at random times, was unrelated to exertion, and lasted one minute or less. It involved the left side of the chest and the anterior axillary line and did not radiate elsewhere. The pain was precipitated when his arm was held out and relieved when the arm was returned to his side. The physician's record indicated that if the patient subsequently recognized an exertional component to the pain or the pain persisted he should schedule an exercise tolerance test. The patient died three months later from a cardiac arrhythmia that the autopsy concluded was due to coronary artery occlusion.
2. Alleged failure to make a timely diagnosis of appendicitis	The patient was a 29-year-old woman who presented to the emergency room after three days of abdominal cramping. On physical examination she had active bowel sounds, a thick white cervical discharge, cervical motion tenderness, and tenderness in the right adnexa. There was no rebound tenderness, distension, or rigidity. Vomiting was not a prominent symptom. The ultrasound report was normal except for a large collection of fluid and debris in the posterior cul-de-sac. The patient did not respond to antibiotics given for presumptive pelvic inflammatory disease, and on the fifth day of hospitalization she had a laparoscopy that revealed a ruptured appendix.
3. Alleged failure to make a timely diagnosis of breast cancer	The patient had a normal mammogram in June. In September, she had a small axillary node and was told to have another mammogram, which for unexplained reasons, was never performed. Six months later she had a lump in her breast, a positive mammogram, and a biopsy that showed breast cancer. The patient died from breast cancer 21 months after the diagnosis.
4. Modification of case 1	The patient survived myocardial infarction.
5. Modification of case 2	The patient had a difficult six-month postoperative course with several surgeries to lyse adhesions and drain recurrent abscesses. These surgeries finally resulted in a bowel resection and a colostomy.
6. Modification of case 1	The patient complained during a first office visit of a new burning chest pain precipitated by exercise and that resolved with rest, and complained during a second visit that the pains occurred with less exertion than previously. To make the physician's behavior more understandable, the abstract reported that the patient presented the critical information during the second visit on his way out of the office after he had arrived 15 minutes late for his appointment and put the physician behind schedule. In addition, the physician had a vacation scheduled shortly after seeing the patient and could not reschedule the patient for another three weeks. The patient had a nonfatal myocardial infarction one week after the second exam.

In addition to abstracts for 3 actual cases, we created 3 additional abstracts by modifying the outcome or presentation of the patients in abstracts 1 and 2. These modified abstracts are summarized as cases 4 to 6 in Table 1. The modifications allowed us to examine the extent that patient outcome and presentation affected referral decisions and assessment of quality of care.

Survey Procedure

With the approval of the University of Iowa College of Medicine Institutional Review Board, an abstract of each case and an accompanying questionnaire to determine how physicians assessed the case were mailed to family physicians in Iowa with at least 5 years of experience. These physicians served as a peer group to assess customary care. To determine how the opinions of family physicians differed from the opinions of specialists, we also submitted the abstracts and questionnaires to community specialists who managed similar patients in an ambulatory setting: cardiologists for the coronary artery disease case, general surgeons for the appendicitis case, and gynecologists for the breast cancer case. Community physicians in each category were randomly selected from the Iowa physicians in the American Board of Medical Specialties directory¹⁶ and their addresses were verified from the website of the American Medical Association (AMA).¹⁷ Academic physicians in Family Practice were selected from faculty in Iowa family practice residency programs. Academic physicians in other specialties were selected from the faculty of the University of Iowa College of Medicine. Our goal was to obtain responses from 25 to 30 physicians to describe the group of physicians and to compare responses across groups. Estimating a 40% response rate for family physicians and lower response rates for specialists, we mailed the questionnaires for the 3 original cases to 70 community family physicians, 60 to 86 community specialists (depending on the number of available specialists), and to all appropriate academic faculty, (which were available in smaller numbers). Physicians only received a mailing for 1 case with the exception of the 15 family physicians in the University of Iowa College of Medicine who were professionally affiliated with the study investigators. These physicians completed questionnaires for both the coronary artery disease and appendicitis cases.

To evaluate how change in the patient's presentation affected physician evaluation, we sent a modification of the patient's presentation of coronary artery disease to 70 community family physicians. Abstracts with altered patient outcomes were sent to 80 community family physicians: 40 of these abstracts modified the appendicitis case to have worse patient outcome and 40 modified the coronary artery disease to have better patient outcome. A total of 419 abstracts and accompanying questionnaires were mailed to family physicians and 270 to specialists.

The pivotal issue in all 6 cases was whether the defendant physician made a timely referral to a specialist or for more tests. To determine the survey respondents' behavior, we asked them to give the probability they would have referred a similar patient. This probability takes into account physician recognition that nonmedical factors may affect their responses to some cases and measures the responders' strength of conviction about a case. The survey also asked physicians to estimate the percentage of primary care physicians who would have made such a referral, to rate the adequacy of care, and to rate the accuracy of the key statements made by the expert witnesses (see Table 2). Ratings for quality of care were from 1, excellent care, to 7, terrible care. Ratings of expert testimony were from 1, completely agree, to 7, completely disagree.

Table 2.

Comments by Medical Experts for the Actual Cases

Case	Expert Comments
Coronary artery disease	Plaintiff expert:  “I certainly would have referred him to a stress test or to a cardiologist.”
	Defense expert:  “Pain, the one that seemed to occur before he came in, seemed…even wild for atypical angina…I would think it would not be necessary to do a treadmill in every case of a mysterious kind of chest pain.”
Appendicitis	Plaintiff expert:  “36–48 hours of incorporating a good trial of antibiotics, [she] wasn't getting better, white count, temperature remains [I would have scoped].”
	Defense expert:  “Anybody looking at that record [before the 5th day of hospitalization] would have concluded that it was PID.”
Breast cancer	Plaintiff expert:  “It's…my opinion that by, I think, [September '92] you had sufficient soft findings to warrant biopsy…of that axillary lymph node.”
	Defense expert:  “Symptoms that come and go…are just almost always innocent…there are many things that will cause axillary adenopathy…[in September '92], the standard of care did not require subsequent examination.”

Demographic information on physician respondents, including gender, years of experience, and ZIP code for community of practice, was available from either the American Board of Medical Specialties listing or the AMA website database. Community size was obtained from ZIP code by using a database from the U.S. Census Bureau that specified the percentage of the population in each of the following categories: urban, inside urbanized area (1 or more adjacent areas with a minimum of 50,000 people), urban outside urbanized area (2,500 or more people incorporated as cities, villages, or boroughs), rural nonfarm, and rural farm.¹⁸ We classified a physician with a given ZIP code into the category that accounted for the majority of the population of that ZIP code.

Active Case

In addition to surveying physicians about adjudicated cases described above, we also surveyed physicians about an active case. By including an active case, we were able to assess whether physicians were as willing to respond to an active as to a closed case, and how attorneys responded to survey results. The active case involved a patient with symptoms of coronary artery disease. Negligence was claimed for failure to refer the patient for coronary angiography following a negative stress thallium test. A second claim was that the patient should have been hospitalized following an emergency room visit for chest pains on the 15th day after the initial visit. One of the questions to the physicians about this case was as follows: “Please describe how the results of the exercise thalium scan affected your decision on referring the patient for coronary angiography.”

An abstract of the case was prepared by 2 disinterested physicians. The plaintiff's attorney considered the abstract fair and complete and did not suggest changes. Although the abstract for the survey included all relevant details about the case, except outcome, the plaintiff's attorney who provided the case asked that details be omitted from the manuscript.

Because information obtained from this case was only qualitative and descriptive with no statistical analyses performed, we sent the abstract and questionnaire to only 35 community family physicians: 25 randomly selected from responders to a previous questionnaire and 10 randomly selected from nonresponders. Physicians were told that their responses could potentially affect the settlement of the case. The plaintiff's lawyer was interviewed before and after being provided with the results of the survey to determine the effect of the survey on his assessment of the case.

Statistical Analysis

Physicians responded on continuous or ordinal scales to questions about their likelihood of referral, quality of care assessment, or agreement with medical experts. To maximize the power of statistical tests, we used these scales in the analysis. However, to facilitate interpretation of results, we converted the scales to binary. We classified physicians as “likely to refer” if their probability of referring was greater than 50% and “almost certainly refer” if their probability of referring was 90% or more. Physicians were considered to agree with the expert witness if their rating was 5 to 7 on a 7-point scale where 1 = completely disagree and 7 = completely agree. Care was classified as adequate if the adequacy rating was from 1 to 3 on a 7-point scale with anchors of 1 for excellent care and 7 for terrible care. The care was also classified as adequate if the adequacy rating was coded as 4, the physician's probability of referral was 75% or less, and the physician estimated that 75% or fewer of other physicians would have referred the patient. We considered a rating of 4 as possibly adequate because it was the midpoint of the quality scale and 46% of physicians who gave this rating believed that 50% or less of other physicians would refer this patient. The cutoff point chosen influences the percentage of physicians who considered the care adequate but not the assessment of which cases had the best or worst care.

For each of the 3 closed cases, we used the mean of the referral probabilities for all community family physicians as the “best” estimate of the percentage of family physicians who would have referred the patient. Each family physician was also asked to estimate the percentage of other family physicians who would have referred the patient. If this estimated percentage differed from the best estimate by 25% or more, the physician was classified as having poor ability to judge the behavior of other physicians.

Statistical significance of differences in percentages was determined with χ² contingency table tests. When the outcome was probability of referral or inadequacy of care scored from 1 to 7, we used analysis of variance adjusting for case, or multiple linear regression as appropriate. SAS statistical software (Version 8; SAS Institute, Inc., Cary, NC) was used for all analyses.

RESULTS

Responders

The response rates to the closed cases were 63% for 350 community family physicians, 91% for 54 academic family physicians (the 15 respondents from University of Iowa College of Medicine Department of Family Medicine completed questionnaires from 2 cases), 51% for 216 community specialists and 54% for 54 academic specialists (P < .001 for the differences in response rates). Respondents and nonrespondents were very similar with respect to gender and community size for both community family physicians and community specialists. The 219 family physician respondents practiced 1.4 years longer than the 120 nonrespondents (P = .01), and the 110 specialist respondents practiced 4.5 years less than the 103 nonrespondents(P = .002). For the community family physicians, response rates were higher for the coronary artery disease case (67%) than for the appendicitis case (53%) (P = .04). There were no other significant differences in response rates by case either among specialists or among family physicians.

Characteristics of the physician responders are shown in Table 3. There was a higher percentage of women among the academic physicians than among the community physicians (P < .001). With the exception of academic family physicians, the majority of physicians had more than 15 years of experience. Many of the community family physicians (but not physicians in other groups) practiced outside of urbanized areas. Half of the family physicians had been sued for medical malpractice. The academic physicians were much less likely to have been sued than the community family physicians (P < .001), possibly because they had practiced for fewer years and saw fewer patients per day. About 29% of the community family physicians had served as an expert witness for the plaintiff, 39% had served as an expert witness for the defense, and 19% had done both. Fifty-one percent of the community family physicians had never testified as experts.

Table 3.

Characteristics of Physician Responders

	Family Physicians		Specialists
Characteristic	Community	Academic	Community	Academic
Gender, % (n)	N = 219	N = 64	N = 110	N = 29
Female	17 (38)	28 (18)	6 (7)	35 (10)
Experience, % (n)	N = 219	N = 62	N = 110	N = 27
≤15 years	30 (65)	52 (32)	26 (28)	30 (8)
16–29 years	53 (117)	43 (27)	58 (64)	44 (12)
30 + years	17 (37)	5 (3)	16 (18)	26 (7)
Community size, % (n)	N = 206	N = 64	N = 108	N = 29
Rural, non-farm	10 (21)	0	2 (2)	0
Urban, outside urbanized area	54 (111)	14 (9)	28 (30)	0
Urban, inside urbanized area	36 (74)	86 (55)	70 (76)	100 (29)
Malpractice Claim, % (n)	N = 216	N = 64	N = 106	N = 23
	50 (108)	30 (19)	75 (79)	13 (3)
Served as expert for plaintiff, % (n)	N = 218	N = 64	N = 104	N = 28
Never	71 (154)	80 (51)	83 (86)	68 (19)
1–3 times	22 (48)	11 (7)	14 (15)	29 (8)
4 or more times	7 (16)	9 (6)	3 (3)	3 (1)
Served as expert for defense, % (n)	N = 217	N = 64	N = 104	N = 28
Never	61 (133)	77 (49)	47 (49)	57 (16)
1–2 times	23 (49)	14 (9)	27 (28)	14 (4)
3 or more times	16 (35)	9 (6)	26 (27)	29 (8)

Assessment of Customary Care

The community family physicians estimated their own probability and the probability of other physicians for referring similar patients for further evaluation. The correlation between these probabilities was high, .78. There were 81 physicians (37%) whose estimates were outliers, i.e., they differed from the average by 25% or more. Only 36% of these 81 physicians could accurately estimate the behavior of other physicians, i.e., their estimates of the average referral probability of other physicians differed from the actual number by less than 25%. More than half the estimates from academic family physicians were outliers (34/64), and a minority of academic family physicians who gave outlying estimates could accurately estimate the behavior of other physicians, 29% of the 34 estimates.

The referral probabilities for the defendants' peer group of community family physicians are shown in Table 4. The first column is the percentage of physicians who stated they would probably have practiced medicine differently than the defendant, i.e., they would have referred the patient described in the case. The majority of physicians stated they would have made the same management decision as the physician in the coronary artery disease case, but less than 25% of the physicians would have made the same decision as the defendant in the other cases. Strong deviations from usual practice are indicated by the percentage of physicians who almost certainly would have referred. Only 21% of these physicians for the 2 coronary artery disease cases and 43% of these physicians in the cancer case almost certainly would have referred compared to the majority of physicians in the unstable angina and appendicitis cases.

Table 4.

Referral Probabilities for Community Family Physicians

Case Type	Likely to Refer*, % (n)	Almost Certainly Refer†, % (n)
Coronary artery disease, good outcome (N = 25)	44 (11)	20 (5)
Coronary artery disease, poor outcome (N = 45)	47 (21)	22 (10)
Coronary artery disease, either outcome (N = 70)	46 (32)	21 (15)
Coronary artery disease, unstable angina (N = 52)	81 (42)	62 (32)
Appendicitis, good outcome (N = 34)	88 (30)	62 (21)
Appendicitis, poor outcome (N = 21)	86 (18)	76 (16)
Appendicitis, either outcome (N = 55)	87 (48)	67 (37)
Cancer (N = 40)	78 (31)	43 (17)

^*Likely to refer: referral probability is 51% to 100%.

^†Almost certainly refer: referral probability is 90% to 100%.

There were minimal differences between the percentage of physicians who reported they would be likely to refer the patient with the less-severe outcome compared to the percentage referring the patient with the more-severe outcome (44% versus 47% for the coronary artery disease patient and 88% versus 86% for the appendicitis patient). On the other hand, the patient's presentation greatly affected physician reported behavior: only 21% of physicians reported they would almost certainly have referred the case with an atypical presentation of coronary artery disease, but 62% almost certainly would have referred the case with probable unstable angina (P < .0001).

Assessment of Quality of Care

The assessment of the quality of care for each of the cases is shown in Table 5. In contrast to the results in the previous table, which describe the physicians' own probable behavior, the results in Table 5 show the physicians' assessment of the quality of care provided by the defendant. In general, physicians thought the care provided for the coronary artery disease patient was adequate; about half thought the care provided for the cancer patient was adequate; and less than one third thought the care provided for the appendicitis and unstable angina cases was adequate. There was a strong congruence between a physician's self-reported tendency to refer and the physician's criticism of the care: 83% of the physicians with a high likelihood (i.e., greater than 75%) of referring the patient thought the care provided by the defendant, who did not refer, was inadequate.

Table 5.

Percentage of Physicians Who Considered the Care Adequate

	Family Physician, % (n)			Specialist, % (n)
Case*	Community	Academic	Academic + Community	Community	Academic	Academic + Community
Coronary artery disease, good outcome	73 (26)	—	73 (26)	—	—	—
Coronary artery disease, poor outcome	62 (45)	65 (31)	63 (76)	77 (30)	85 (13)	79 (43)
Coronary artery disease, unstable angina	33 (52)	—	33 (52)	—	—	—
Appendicitis, good outcome	37 (35)	24 (33)	31 (68)	17 (48)	0 (5)	15 (53)
Appendicitis, poor outcome	20 (20)	—	20 (20)	—	—	—
Cancer	54 (39)	—	54 (39)	66 (29)	11 (9)	53 (38)

^*Percentages within a column varied significantly at P < .01 for all columns. Percentages across the row for the cancer case varied significantly at P = .02. Variations in percentages across other rows were not significant at P < .10.

n, number of physicians who answered the question; —, type of physician specified in the column was not sent the case specified in the row.

The results in Table 5 can be used to compare quality assessments by physicians with different backgrounds. The percentage of physicians who considered the care to be adequate varied significantly by case for each type of physician. Specialists were less likely than were family physicians to consider the care adequate for the appendicitis case (P = .04). Academic surgeons were more critical of the quality of care than were community surgeons for both the appendicitis case (P = .03 [using a t test of mean scores]) and the cancer case (P < .01 [using a χ² test to compare percentages]).

Patient outcome was not significantly associated with physician referral probabilities or quality-of-care judgments at the P < .10 level using either χ² test or analysis of variance. There was a great difference in the percentage of respondents rating the care as inadequate for the myocardial infarction patient and the similar patient who had unstable angina (P < .0001).

We tested all physician characteristics shown in Table 1 for an association with probability of referral or quality assessment using regression analysis to adjust for case. No factors were associated with physician likelihood of referring at P < .10, and the only factor associated with the quality of care rating was number of years of practice. Physicians with more experience tended to give more negative ratings to quality of care (P < .01).

Agreement with Expert Witnesses

The percentages of physicians who agreed with the expert witnesses are shown in Table 6. Thirty-five percent or less of the community family physicians agreed (i.e., 65% or more of physicians disagreed) with 1 of the expert witnesses for each case. The majority of physicians agreed with the defense expert for the coronary artery disease case and agreed with the plaintiff experts for the other 2 cases. This is consistent with the finding that most physicians judged the quality of care as adequate for the coronary artery disease case but not for the other cases.

Table 6.

Percentage of Physicians Who Agreed with Expert Witnesses.

		Agreed with Expert Witness*
Case	Physician Group	Plaintiff†, % (n)	Defense†, % (n)	Both†, % (n)
Coronary artery disease	Community family practice	35 (71)	67 (70)	13 (70)
	Academic family practice	29 (31)	74 (31)	13 (31)
	Community cardiology	37 (30)	87 (30)	23 (30)
	Academic cardiology	39 (13)	92 (13)	31 (13)
	All physicians	35 (145)	75 (144)	17 (144)
Appendicitis	Community family practice	79 (56)	29 (56)	18 (56)
	Academic family practice	79 (33)	27 (33)	15 (33)
	Community surgery	82 (49)	12 (49)	6 (49)
	Academic surgery	60 (5)	0 (5)	0 (5)
	All physicians	79 (143)	22 (143)	13 (143)
Cancer‡	Community family practice	75 (40)	30 (40)	13 (40)
	Community OB/Gyn	43 (30)	50 (30)	7 (30)
	Academic OB/Gyn	82 (11)	18 (11)	0 (11)
	All physicians	64 (81)	36 (81)	9 (81)

^*Physicians were considered to agree if their rating was 5–7 on a 7-point scale where 1 = completely disagree and 7 = completely agree.

^†P < .001 for the variation by case in the percentages of community family physicians who agreed with the plaintiff expert (also with the defense expert). Variation in agreement rates were also statistically significant for the specialists.

^‡Percentage of physicians who agreed with the expert witness was significantly associated with physician type for the plaintiff's witness in the cancer case (P = .01). Other associations with physician type were not significant.

Active Case

The responses to the active case were analyzed separately. Twenty-five of the 35 physicians who were sent the case responded. This included 21 of the 25 physicians who had responded to a previous case and 4 of 10 physicians who were contacted for a previous case but did not respond (P < .02). The majority of responding physicians would have managed the case differently than the defendant: 59% of physicians said that they would have been very likely to refer the patient for angiography, and 52% would have been very unlikely to send the patient home from the emergency room on day 15. However, care was considered more than minimally acceptable by 88% of responding physicians on day 1 and by 50% on day 15.

The determining factor in the physician decision making was their belief in the reliability of the thallium scan. Only 33% of the12 physicians who regarded the thallium scan skeptically considered the care on day 15 adequate, compared to 80% of the 10 physicians who accepted the results of the scan.

The plaintiff's attorney considered the physicians' responses to be unbiased because they were congruent with the responders' written comments about the case, especially those concerning the role of the thallium scan. The results of the survey tempered the attorney's criticism of the defendant and helped the attorney better understand the role of the thallium scan. However, the attorney also hired a family physician as an expert witness to help his client's case.

DISCUSSION

This study explored surveys of peer group physicians as an alternative or supplement to party-employed medical experts for information about practice standards. The summary of the case was accepted as fair by opposing lawyers for each of 3 actual cases. The high response rate, 63% of the 350 unpaid community family physicians surveyed, suggests the physicians considered the research question important and the burden of the survey low. The physician participation rate was as great for an active case (also not funded) as it was for the inactive cases. These rates might be even higher were medical associations and local physician leadership to promote the importance of the surveys and if surveyed physicians were to receive payment.

Because the most common legal standard for medical malpractice is customary practice, the survey information of greatest potential importance is how physicians would have managed the case. For the 7 cases (3 closed, 3 constructed from closed cases, and 1 open case) there were clear differences in the percentage of family physicians who would have practiced in the same way as the defendant.

Survey results provide more objective and precise information about whether the defendant's care fell within customary practice than does partisan testimony from conflicting experts whose opinions are unsupported by data. They also provide better information than do other alternatives, such as recommendations from consensus conferences or practice guidelines, which are often difficult to apply to specific patients and may be closer to optimal than to customary or reasonable care. In contrast to physician surveys, general recommendations must of necessity ignore the context in which a specific patient was seen, e.g., the physician's previous experience with the patient and the patient's expressed needs or social situation. These factors do and should influence physicians' management strategies.

We also collected information about physician judgment on quality of care. This information may be useful because there has been some recent movement away from customary community practice and toward adopting a reasonableness standard for physician negligence.¹⁹ The assessment of peer-group physicians should influence juries as to whether the care was reasonable.

This study suggests that the testimony of expert witnesses may not provide reliable information about the standard of practice. In every case, the majority of family physicians (at least 65%) disagreed with the experts representing one side. Because for every case there was a variety of physician opinion, and because physicians with opinions outside of the mainstream were generally not aware of how other physicians practiced, it is not surprising that experts could be found who would testify incorrectly, but according to their beliefs, about majority practice. The direct assessment of medical judgment or opinion in the population avoids biased estimates from one expert.

Validity of the Survey Results

The survey's validity may be affected by bias. Hindsight bias may occur because physicians reviewing a medical malpractice case know that the patient had a bad outcome and that physician management decisions might have been a factor. Hindsight bias can be evaluated by examining the effect of patient outcome on physician assessment. One previous study found that an adverse outcome influenced the judgment of physicians who reviewed medical records for quality of care,²⁰ although another study did not find outcome to have such an effect.²¹ Severity of the adverse outcome was not significantly associated with physician reporting in our study. In contrast, making the patient's presentation more typical of a serious condition greatly influenced self-reports of how the physicians would have managed the patient. Therefore, the appropriateness of the care seems to influence physician judgment much more than any hindsight bias.

Surveys may also be biased because physicians may report how they should practice rather than how they do. This type of bias may affect the physician's rating of other physicians less than it affects self-reported behavior. The bias could be eliminated by alternative methodologies for assessing community practice norms. One alternative is the use of structured reports by trained actors who present to many physicians with the same symptoms as the plaintiff.²² Another is to review large numbers of medical records similar to the plaintiff's to determine how these patients are typically managed. Although these methodologies may reduce some types of bias, they would be enormously expensive to design and implement and may not be possible. They may also not be necessary. A previous study found that case vignettes can be used to obtain a valid overall measure of the process of care provided by groups of physicians for a range of common outpatient conditions.²³ The vignettes provided better information on how physicians actually practice than information from the medical record, which depended on physician effort to chart the details of patient interaction.

The plaintiff's attorneys' greatest concern about the survey method is anti-litigation bias or allegiance bias (oral communication; M. Diaz, JD, May 2000; W. Wagner, JD and J. Palmer, JD, June 2000). This latter bias causes physicians to report favorably on their colleagues even if they disagreed with their patient management. However, the high percentage of physicians who criticized their peers and the high correlation (r = .63) between a physician's self-reported probability of referral and assessment of the quality of care provides evidence against strong allegiance bias. In the active case, the plaintiff's lawyer considered the survey results to be unbiased even though they were often not favorable to his client. This bias may be affected by how survey questions are framed. We asked physicians for a probability or quantitative response and then put the response into binary categories. Physicians might have responded differently if the question had forced them to make a binary response.

Sampling validity may also depend on nonresponse bias, i.e., physicians who respond to a survey may not adequately reflect the community of physicians. The only indication of possible nonresponse bias we had for family physicians in this survey was that nonrespondents had practiced medicine 1.4 years less than respondents. We did not assess whether nonrespondents and respondents had similar opinions about standard of care. If high percentages of physicians respond, the potential for nonresponse bias is reduced. In any case, responses by groups of physicians surveyed by an objective professional organization should provide a better indication of physician practice than partisan experts.

To evaluate the judgment of family physicians about the quality of care, we compared them to specialists, who have a different knowledge base and often see patients with more-severe disease. For 1 of the 3 cases, we found specialists substantially more critical of the care than family physicians, although even for this case, less than one third of the family physicians considered the care adequate. Overall, the results suggest that specialists will give responses qualitatively similar to those of primary care physicians.

Survey results may depend on how the peer group of physicians is defined. Of several physician characteristics examined in this study, none were related to physician responses. These results suggest that survey results should be insensitive to demographic characteristics of the physicians sampled. However, we were not able to test for the effect of variation in some physician characteristics that possibly should be used to define peer group, such as state in which the practice was located. If sampling physicians from different states gives different results, then the definition of the appropriate peer group will be important.

Value of the Survey Method

Although surveys may provide accurate population estimates of medical opinion and judgment, surveys of physician opinion about medical facts (e.g., the usual time required to administer antibiotics for bacterial meningitis), may still yield biased results.¹⁵ In addition, surveys will not be helpful if the case hinges primarily on disputed factual information that is not resolved until trial. Interpretation of survey results may also be complicated by legal rulings that accept a second school of thought to justify medical practice, even if the practice is not considered justified by the majority of physicians in a survey. Survey results are well suited for cases in which the allegation is failure to diagnose, including failure to consult other sources, failure to refer to a specialist, or failure to order additional diagnostic tests. These cases rarely involve a second-school claim in which a substantial number of respected physicians prefer a management strategy that differs from the one recommended by the majority of physicians. They account for about 26% of all malpractice cases and 33% of the malpractice costs.²⁴ The survey method could also be used in informed consent cases in those states that apply the “reasonable doctor” standard of disclosure.

Cost would probably not be a major barrier to the use of surveys for failure-to-diagnose cases. Less than 5 physician-hours should be required for two physicians to create an abstract and write a questionnaire. Piloting the questionnaire, randomly selecting physicians, mailing surveys, and mailing follow-up letters can be done rapidly by support personnel once appropriate procedures have been developed. We also recommend a $50 payment to physicians who complete surveys, although we have not studied the impact of this payment on response rates. Malpractice litigation is quite expensive for the parties. The marginal cost of a survey is quite modest in comparison to other litigation expenses and may actually reduce overall costs if survey results stimulate settlements. Moreover, if this technique were to become established, procedures could become more standardized, pushing costs down further.

The greatest potential use of surveys may be for the 90% of medical malpractice cases that are settled out of court.²⁵ Survey results could help one of the parties to perceive the case more realistically; they could help avoid trials for some cases and influence the arbitration, mediation, or settlement proceedings for those cases settled without a trial. Although the results would be most influential when both litigants agreed on the survey, even without agreement one party might want to perform the survey in collaboration with an objective medical society. This party (e.g., an HMO) might use the survey results in deciding whether to settle a case. If an expert witness is employed, this witness could use the survey results in forming or supporting an opinion about customary care. The survey results might be admissible as an explanation of the basis of the expert's opinion.²⁶ Moreover, survey results might be used to impeach an adversarial expert who testifies contrary to the survey's results. Indeed, the mere existence of survey results should temper some of the unfortunate excesses of adversarial experts. Thus, surveys of a representative group of peer physicians performed by an objective organization should provide better information on customary care than information provided by selected experts under the current system.

Survey results will require interpretation, particularly if 20% to 30% of physicians report they would have made the same decision as the defendant. Although this is a substantial percentage of physicians, it may represent the residual percentage who are sympathetic to the difficulties of practice and will excuse any but the most flagrant violations of care. To detect allegiance bias, it may be helpful to include specific questions that would explain variation in physician decision making. An example of such a question is the question about the reliability of thallium scans that we included in the questionnaire about the active case. Further experience with the survey method will help refine the methodology and the interpretation of results.

We recommend that specialty medical societies conduct the surveys with the assistance of an auxiliary litigation service provider that has developed expertise with malpractice surveys. Medical societies will be sensitive to both physician fears of harassment and public fears of incompetent medical care. They will be able to recruit appropriate samples of physicians and convince them that the fair assessment of cases is necessary for maintaining public support for the survey method. Research will be needed to improve the administration and interpretation of malpractice surveys, but once the methodology is fully developed, surveys may offer the best practical approach for assessing standard of care in specific malpractice cases.