Development of a Composite Measure of State-Level Malpractice Environment

Abstract

Objective

To develop a composite measure of state-level malpractice environment.

Data Sources

Public use data from the National Practitioner Data Bank, Medical Liability Monitor, the National Conference of State Legislatures, and the American Bar Association.

Study Design

Principal component analysis of state-level indicators (paid claims rate, malpractice premiums, lawyers per capita, average award size, and malpractice laws), with indirect validation of the composite using receiver-operating characteristic curves to determine how accurately the composite could identify states with high-tort activity and costs.

Principal Findings

A single composite accounted for over 73 percent of total variance in the seven indicators and demonstrated reasonable criterion validity.

Conclusion

An empirical composite measure of state-level malpractice risk may offer advantages over single indicators in measuring overall risk and may facilitate cross-state comparisons of malpractice environments.

The U.S. medical malpractice environment has been the subject of much policy and research interest. Despite the volume of research that has been conducted on these topics, measurement of malpractice environment has not been well studied.

Researchers have used a number of indicators of malpractice environment, including frequency of malpractice claims within a population (paid claims/capita or per physician) and lawyer density (lawyers/capita) to measure probability of litigation (e.g., Danzon 1986; Chandra, Nundy, and Seabury 2005); malpractice payments per claim or per physician (“claim severity”) to measure expected losses (e.g., Baicker, Fisher, and Chandra 2007); and malpractice premiums to measure both probability and expected magnitude of loss (e.g., Dubay, Kaestner, and Waidmann 1999). State malpractice laws have also been used to proxy for malpractice environments (e.g., Kessler and McClellan 1996). Reforms requiring affidavits of merit or pretrial screening that are intended to reduce claims volume have been used to proxy for claims rates. Caps on noneconomic or total damages, contingency fee limits, and collateral source rules affect the magnitude of payouts and have been used to proxy for claim severity.

These indicators have been used as measures of “malpractice environment” (Studdert et al. 2005; Dhankhar, Khan, and Bagga 2007) “malpractice risk” (Dubay, Kaestner, and Waidmann 1999; Kim 2007; Yang et al. 2008) “malpractice pressure” (Kessler and McClellan 1996; Chandra, Nundy, and Seabury 2005), and “malpractice climate” (Matsa 2007), although the literature makes no distinction between these concepts. We define malpractice risk as the expected disutility of a malpractice claim: the probability of experiencing a malpractice loss multiplied by the expected magnitude of loss. Malpractice environment refers to the level of risk inhering in the set of circumstances (e.g., litigiousness), legal structures (e.g., malpractice laws/reforms), and/or market conditions (e.g., availability of underwriters, premiums) in a practice location.

Single indicators are useful if research aims to test hypotheses about specific, measurable aspects of malpractice environments, but interest often lies in a broader construct of “malpractice environment.” In these situations, it may be challenging to synthesize results within and across studies if indicators behave inconsistently, if they are inversely correlated, or if the internal consistency of the indicators has not been previously established. From a modeling standpoint, multiple indicators may be too collinear to use simultaneously in a single equation, but including them in separate models may inflate experiment-wise error. For these reasons, a composite measure of malpractice environment may be desirable. Composites may facilitate interpretation and cross-state comparisons. The goal of this research was to use principal component analysis to demonstrate one approach to building a composite measure of state-level malpractice environment to summarize the level of malpractice risk to health care providers in different states, and to explore the validity of this measure.

METHODS

Data Sources and Measures

We assembled seven measures of malpractice environment that were used in previously published studies using 2010 data from the following sources. Table 1 provides summary statistics, data sources, and sample references illustrating previous use for each measure.

Table 1.

Individual State-Level Malpractice Environment Indicators, Data Sources, and Descriptive Statistics

Measure	Freq. (%)	Mean (SD)	Median	Data Source
Number of paid malpractice claims per 100 active, nonfederal physicians, 2010		1.22 (0.68)	1.01	Numerator. National Practitioner Data Bank Public Use Data File (NPDB), 1990–2012. U.S. Department of Health and Human Services, Health Resources and Services Administration, Bureau of Health Professions, Division of Practitioner Data Banks. Available at http://www.npdb-hipdb.hrsa.gov/. The NPDB was created under the Health Care Quality Improvement Act of 1986 and is a registry of professional sanctions and malpractice payments paid on behalf of physicians and other health professionals Denominator. Area Resource File (ARF) 2011–2012. U.S. Department of Health and Human Services, Health Resources and Services Administration, Bureau of Health Professions, Rockville, MD 2011
Average malpractice award size, 2010		$354,902 (173,229)	327,292	National Practitioner Data Bank Public Use Data File (NPDB), 1990–2012. U.S. Department of Health and Human Services, Health Resources and Services Administration, Bureau of Health Professions, Division of Practitioner Data Banks. Available at http://www.npdb-hipdb.hrsa.gov/
Number of resident, active attorneys per 100,000 capita, 2010		323 (118)	292	Numerator. American Bar Association (ABA) Market Research Department. Table: 2010 Number of Attorneys Resident & Active from National Lawyer Population by State. Available at http://www.americanbar.org/resources_for_lawyers/profession_statistics.html. The ABA obtained these estimates of the total number of resident and active attorneys practicing in each state from state bar associations and licensing agencies
				Denominator. Area Resource File (ARF) 2011–2012 (see above)
Average malpractice premiums, 2010
Internal medicine		$14,513 (11,133)	11,859	Medical Liability Monitor (MLM). Annual Rate Survey Issue (2010). The MLM is an independent survey of the largest physician malpractice insurance underwriters that offer policies with claims limits of $1 to $3 million. The MLM publishes premium rates for internal medicine, general surgery, and obstetrics/gynecology for markets within each of the 50 states. These data have been used extensively by others in public and private research (for example)
General surgery		$46,947 (21,421)	44,498
Obstetrics/gynecology		$68,532 (29,871)	64,072
Unfavorable malpractice laws
No affidavit/certificate of merit	26 (52)			National Conference of State Legislatures Medical Liability/Medical Malpractice Laws. (Last updated August 15, 2011. Incorporates 2011 enactments). Available at http://www.ncsl.org/issues-research/banking/medical-liability-medical-malpractice-laws.aspx
No peer review panels	2 (4)
No pretrial panels	34 (68)
No alternative dispute resolution mechanisms	12 (24)
No physician apology/sympathetic gestures	14 (28)
No expert witness requirements (Source: NCSL5)	20 (40)
No caps on damages	20 (40)
No limits on attorneys' fees	33 (66)
No general compensation fund	38 (76)
No joint and several liability	10 (20)
No periodic payment	14 (28)
Index of unfavorable malpractice laws		4.58 (1.75)	4.00
Number of tort cases filed/10,000 capita		18.17 (7.90)	16.70	National Economic Research Associates Economic Consulting. Creating Conditions for Economic Growth: The Role of the Legal Environment. A study for the U.S. Chamber Institute for Legal Reform. October 26, 2011 (accessed on July 31, 2012). Available at http://www.instituteforlegalreform.com/sites/default/files/Economic_Growth_Working_Paper_Oct2011_0.pdf
Number of large personal injury verdicts/100,000 capita		17.98 (13.30)	16.90
Total costs of tort litigation (millions $US)		$4,961.12 (5,897.19)	3,068

The National Practitioner Data Bank (NPDB) Public Use Data File is a registry of professional sanctions and malpractice payments made on behalf of physicians. Using these data, we constructed state-level measures of payments/physician capita (claim rate) and average payout size.

The Medical Liability Monitor (MLM) Annual Rate Survey is an independent survey of premiums centerged by major malpractice insurers for internal medicine, general surgery, and obstetrics/gynecology in markets within states. We aggregated premiums for claims-made policies across regions within each state to obtain state-level specialty-specific averages.

The number of active, resident lawyers/1,000 capita in each state was obtained from the American Bar Association.

The National Conference of State Legislatures maintains a summary of 11 malpractice reforms in each state: caps on damages, caps on attorneys' fees, periodic payment, physician apologies/sympathetic gestures, alternative dispute resolution mechanisms, pretrial panels, expert witness criteria, certificate of merit, peer review panels, general compensation funds, and joint and several liability. Two of the authors (KB and CM) coded these summaries, constructed reform indicators, and summed these indicators to create a “legal risk” index in which higher values reflected greater malpractice risk (fewer reforms). Malpractice reforms are complex: some reforms are more important than others (e.g., Paik, Black, and Hyman 2013), and the stringency of a given reform can vary across states (Hyman et al. 2009). Our coding is only one approach. Avraham's (2011) Database of State Tort Law Reforms offers an alternative.

Because no gold-standard measure of malpractice risk exists, we indirectly validated our composite against three state-level indicators of general tort activity: the number of tort cases/10,000 capita, the number of large personal injury verdicts/100,000 capita, and total estimated costs of tort litigation (National Economic Research Associates [NERA] (2011). We constructed dichotomous indicators to identify states in the top quartile of each metric.

For further validation, we compared state rankings on our composite against rankings on the only other composite malpractice risk measure that we are aware of—a 2010 Medical Tort Liability Index (MTLI), Pacific Research Institute (2010). The MTLI is an unweighted average of state rankings on eight indicators: malpractice losses/health expenditures, caps on noneconomic damages, caps on punitive damages, attorneys' fees limits, pretrial screening/arbitration, limits on tort claims involving drugs or medical devices, expert witness criteria, and statutes of limitations.

Statistical Methods

Construction of a Composite Measure of Malpractice Environment. We used principal components analysis (PCA) to construct a composite measure of malpractice environment from the seven indicators described earlier. Because PCA is sensitive to scaling and skewness of indicators, we log transformed all indicators except the reform index. All variables were then standardized using z-scores.

The feasibility of PCA was assessed using Pearson pairwise correlations, Bartlett's Test of Sphericity (BTS), and the Kaiser–Meyer–Olkin (KMO) measure of sampling adequacy. BTS tests the null hypothesis that a set of measures are uncorrelated (and thus unsuitable for PCA). The KMO evaluates the appropriateness of performing a PCA on a group of variables and is an index that ranges from zero to one, with values ≥0.60 deemed acceptable for PCA (Kaiser and Rice 1974).

The number of components to be extracted from PCA was determined using Kaiser's criterion (K1) (Dziuban and Shirkey 1974), Velicer's minimum average partial correlation procedure (MINAP) (Soldz 2002), and Horn's parallel analysis (PARAN) (Dinno 2009). K1 recommends retaining all components with eigenvalues greater than 1.0 (i.e., components should account for as much variance as that in a single indicator), but this may lead to overextraction and consequently, unreliable, noisy components, so we relied on two additional tests (Zwick and Velicer 1986; Coste et al. 2005).

For j variables, MINAP performs j-1 PCAs, extracts the largest component, and then computes average residual correlations among the variables. So long as extracted components account for the variance of two or more variables, the average residual correlations after extraction will decrease. When extracted components account for the variance of no more than one variable, average residual correlations will rise. MINAP identifies the point beyond which unique components are extracted, and it recommends extracting only as many components as there are which explain the variance of two or more variables.

A simulation-based approach, PARAN, compares eigenvalues from the observed data matrix to those of a large number of randomly generated data matrices with the same dimensions. If components extracted from actual data are valid, their eigenvalues should be larger than those derived from random data. Factors are retained so long as their “actual” eigenvalue is greater than the average eigenvalue from simulated data.

We examined the overall pattern correlations or “loadings” between observed indicators and extracted components to interpret the meaning of each component. We examined varimax (assumes components are orthogonal and seeks solutions where each factor has a few large loadings and many small [zero] loadings) and promax (allows correlation across components) rotations. A final composite was constructed by summing scores of the PCA components.

Assessing Validity. Although our composite is a measure of malpractice environment, criterion validity may be demonstrated if our measure identifies states with high versus low levels of general tort activity. We used receiver-operating centeracteristic curve (ROC) analysis to assess whether our composite could accurately identify states in the top quartile for each measure. We used the area under this curve (AUC) to measure how well the composite identified states with high volumes of tort activity, tort awards, and tort costs (AUC = 0.50 = no criterion validity; AUC = 1.00 = perfect criterion validity). ROC analysis was also used to compare the predictive ability of our composite versus the MTLI and seven individual indicators measures that went into the PCA in identifying high-tort states. We compared state rankings on our composite to MTLI using Spearman rank-order correlation.

All analyses were performed using Stata Statistical Software (Release 10, StataCorp 2007).

RESULTS

Descriptive Statistics

Table 1 presents descriptive statistics for the untransformed, unstandardized variables in our analyses and frequencies for the individual legal reforms. In 2010, there was an average of 1.22 (SD = 0.68) claims/100 physicians. The average payout was $354,902 (SD = $173,229). In 2010, average premiums were lowest in medicine (mean = $14,513, SD = $11,133) followed by surgery (mean = $46,947, SD = $21,421) and obstetrician/gynecologists (mean = $68,532, SD = $29,871).

Each variable was significantly correlated with at least one other variable (Pearson p < .05, correlation matrix available upon request). BTS rejected the null hypothesis of no correlation. The KMO test statistic of 0.68 further supported the feasibility of PCA.

Table 2 summarizes our PCA results. The K1, PARAN, and MINAP procedures all suggested retaining the first two components, which accounted for 48 and 25 percent of the total variance in the seven indicators, respectively. Rotated and unrotated solutions were similar: premiums loaded most strongly onto the first component, whereas claims rate, average award size, lawyer density, and our legal index loaded more strongly onto the second component. These patterns suggest two dimensions in describing malpractice environments: insurance climate (premiums) and legal climate. Scores from these components were summed to create our composite.

Table 2.

Results of Principal Components Analysis of Malpractice Environment Indicators

	Unrotated		Varimax Rotation		Promax Rotation
	Component 1	Component 2	Component 1	Component 2	Component 1	Component 2
Eigenvalue	3.37	1.75
Proportion of variance explained	0.48	0.25
Horn's adjusted eigenvalue	2.76	1.31
Loadings
Number of claims/MD	0.39	0.03	0.26	−0.46	0.23	−0.45
Average malpractice award size	0.38	0.43	0.05	0.66	0.08	0.66
Lawyers/capita	0.31	0.50	0.37	0.44	0.39	0.46
Avg. premiums: medicine	0.45	−0.14	0.51	−0.02	0.51	0.00
Avg. premiums: surgery	0.40	−0.29	0.48	−0.08	0.47	−0.06
Avg. premiums: Ob/Gyn	0.47	−0.08	0.54	0.05	0.54	0.07
Malpractice law summary	−0.16	0.68	−0.15	0.39	−0.13	0.39

Table 3 summaries the results of ROC analysis demonstrating the ability of our composite to predict general tort activity. Our composite reasonably predicted states in the top quartile of large personal injury jury verdicts (AUC = 0.71, 95 percent CI [0.53, 0.91]) and total tort litigation costs (AUC = 0.84, 95 percent CI [0.71, 0.96]), but not states with the highest tort rates (AUC = 0.57, 95 percent CI [0.34, 0.80]). In comparative ROC analyses (Table 3), our composite significantly outperformed the MTLI, paid claims rate, average payout, and legal index. For example, in identifying states with high total tort costs our composite had an AUC of 0.84, compared to AUC = 0.36 for MTLI; AUC = 0.54 for claim rate; AUC = 0.56 for average payout; and AUC = 0.36 for the legal risk index (all p < .01).

Table 3.

Receiver-Operating Centeracteristic Curve Analysis Comparing PCA Composite against Other Indicators in Predicting States in the Top Quartile of Numbers of Large Personal Injury Jury Verdicts per Capita, Total Tort Litigation Costs, and Tort Claims Rate

	Area under Receiver-Operating Centeracteristic Curve (95% CI)
	Top Quartile of Large Personal Injury Jury Verdicts/Capita	Top Quartile of Total Tort Litigation Costs	Top Quartile of Number of Tort Claims per Capita
PCA composite	0.72 (0.53–0.91)	0.84 (0.71–0.96)	0.57 (0.34–0.80)
PRI MTLI	0.56 (0.34–0.79)	0.36 (0.17–0.55)†	0.57 (0.36–0.78)
Number of claims/MD	0.62 (0.44–0.80)	0.54 (0.36–0.72)†	0.52 (0.30–0.75)
Avg. malpractice award size	0.54 (0.33–0.74)	0.56 (0.34–0.77)†	0.68 (0.50–0.86)
Lawyers/capita	0.68 (0.49–0.87)	0.80 (0.67–0.93)	0.57 (0.38–0.77)
Avg. premiums: medicine	0.75 (0.57–0.93)	0.84 (0.72–0.96)	0.53 (0.30–0.76)
Avg. premiums: surgery	0.69 (0.50–0.87)	0.72 (0.54–0.89)	0.55 (0.32–0.77)
Avg. premiums: Ob/Gyn	0.73 (0.55–0.91)	0.78 (0.63–0.92)	0.56 (0.31–0.82)
Law summary variable	0.59 (0.39–0.78)	0.36 (0.20–0.52)†	0.61 (0.41–0.80)

^†Difference (from PCA composite) significant at p < .01 level.

Table 4 lists the 50 states and their ranking on our malpractice composite, each subscore, and the MTLI. The MTLI was not correlated with our composite (Spearman ρ = −.40, p = .77) or insurance subscore (ρ = −.20, p = .16), but it was modestly correlated with the legal subscore (ρ = .37, p = .01).

Table 4.

State Rankings on Composite Measure and Medical Tort Liability Index (MTLI), in Order of Ascending PCA Composite Rank, from Lowest Malpractice Risk to Highest Malpractice Risk

State	PCA Composite Rank	PCA Component 1: Malpractice Insurance Climate Subscore	PCA Component 2: Malpractice Legal Climate Subscore	MTLI Rank
ND	1 (lowest risk)	8	2	20
NE	2	4	5	18
SD	3	2	23	36
ID	4	7	12	14
AR	5	6	20	22
MN	6	1	44	34
IA	7	9	16	46
MS	8	16	11	1
KS	9	25	3	10
WI	10	3	45	35
SC	11	18	13	37
IN	12	24	6	19
ME	13	14	21	40
TN	14	13	22	26
NC	15	11	26	31
AL	16	5	43	33
VT	17	10	34	50
CA	18	23	19	11
OR	19	12	36	39
UT	20	38	4	28
HI	21	15	35	45
NM	22	35	8	42
WV	23	49	1	15
TX	24	32	15	6
NH	25	21	30	17
VA	26	22	29	27
KY	27	17	39	48
OK	28	28	25	9
AK	29	19	40	12
MT	30	40	10	13
WY	31	41	7	44
WA	32	20	38	32
NV	33	37	17	2
AZ	34	33	24	23
GA	35	27	32	24
DE	36	30	27	25
CO	37	26	33	4
OH	38	31	28	21
MI	39	43	14	3
LA	40	45	9	5
MO	41	36	37	29
RI	42	29	46	49
MD	43	39	42	41
PA	44	46	31	47
NJ	45	44	41	30
FL	46	50	18	7
MA	47	34	48	16
IL	48	42	49	8
CT	49	47	47	38
NY	50 (Highest risk)	48	50	43

SUMMARY

This study demonstrated the use of PCA to construct a composite measure of state-level malpractice environment from seven commonly used indicators. We found that a two-dimensional centeracterization of state malpractice environment, with one “insurance” component and one “legal climate” component, accounted for nearly 75 percent of total variance in all seven indicators. Our composite demonstrated reasonable criterion validity and was superior to the MTLI or individual indicators in predicting states with high personal injury jury verdicts and total tort litigation costs.

LIMITATIONS

This work has important limitations. The NPDB provides information on paid claims only: data on filed claims are not available. A recent study estimated that only 22 percent of malpractice claims resulted in payment to plaintiffs (Jena, Seabury, and Chandra 2011), so our measure likely underestimates litigiousness. The MLM also provides a very limited measure of malpractice premia, but no better data source is available. Our summary index of legal reforms is simplistic, but there is consensus among researchers as to how to construct a better one.

PCA is often discussed as a large-sample procedure, with sample size potentially influencing the stability of factor solutions. Although our sample size exceeds a suggested observation-to-variables ratio of 5:1 (Gorsuch 1974), our study is not based on a sample but the entire population of states in the United States. Diagnostic statistics indicated feasibility of PCA. Although not presented, we replicated these analyses using previous years of data and found factor loadings and components to be stable over time. We also find support for our approach in the many precedents for the use of PCA in constructing state-level composites of, for example, well-being (Pesta, McDaniel, and Bertsch 2010), social capital (Kawachi, Kennedy, and Wilkinson 1999; Putnam 2000; Hawes, Rocha, and Meier 2013), and legal/policy climates (Holbrook and Percy 1992; Sorens, Muedini, and Ruger 2008).

We lacked a direct measure of malpractice risk against which to validate our composite so we relied on measures of general tort activity. Our composite could reasonably identify states with high levels of tort activity. Our measure correlates negatively with the only other known composite, the MTLI, but this may reflect limitations of the MTLI itself. The 2010 MTLI was comprised of one interval-level measure that was treated as ordinal and seven discrete variables to which MTLI developers assigned subjective scores. Each MTLI indicator was equally weighted, although indicators may vary in importance in determining malpractice risk. In contrast, PCA weights each indicator based on its contribution to explaining overall variance. Neither the internal consistency of the MTLI components nor the measure's external validity has been formally assessed, and we found that it did not predict general tort environment.

We have not studied whether our composite is superior to other possible composites. A useful extension of this research would be to explore alternative composite measures of malpractice risk and to assess their comparative predictive value for outcomes of policy interest.

CONCLUSION

This study suggests the possibility of centeracterizing state-level malpractice environments using a composite measure as a complement, not a substitute for individual indicators. Single indicators should be favored for testing specific policies or market mechanisms. However, when interest is in centeracterizing malpractice environment as a construct rather than a specific aspect of risk, then a composite offers several advantages. First, our composite is derived from standard indicators of malpractice risk using regularly collected, publicly available data (Table 1) and provides a simple approach to combining multiple indicators. Second, PCA results themselves can offer insight into shifts in the nature of malpractice environment if one dimension becomes more/less salient over time, or factor loadings change substantially. One can also use our insurance climate and legal climate subscores simultaneously without fear of collinearity because PCA components were constructed to be orthogonal. This would enable the simultaneous study of the effects on providers of pressure from insurance costs and from the “legal reality” claims and payouts.

SUPPORTING INFORMATION

Additional supporting information may be found in the online version of this article:

Appendix SA1

Author Matrix.