Studying Complexity is Complex

This supplement contains 9 articles that arose from a Department of Veterans Affairs sponsored State of the Art (SOTA) conference “Managing Complexity in Chronic Care.” It is appropriate that these articles appear in the Journal of General Internal Medicine. Advocates for general internal medicine as a specialty often mention the care of complex chronically ill patients as a special competency of General Internists.¹ Moreover, the Society of General Internal Medicine and its members have been leaders in recognizing and addressing the research and policy importance of this large patient population.² The editors of JGIM believe that this supplement provides critical early exposure for research topics that will appeal to funders such as the VA, other federal agencies, and foundations like the Robert Wood Johnson Foundation who wish to encourage research regarding the care for this challenging, but growing population.

During the months spent overseeing and contributing to the peer review of the articles included in this supplement, we were struck that some of our usual editorial criteria were difficult to apply. It appeared to us that the science of studying the chronic care of complex patients required significant changes in our concept of “important research results.” Three of the articles, by Luck et al, Sevick et al and Vogeli et al, present reviews of the literature. We were relieved to see that these authors also struggled to identify research that met traditional standards for rigorous methodology. It was interesting that Luck et al, who examine the business case for improving care, were able to find considerable information on patients with single complex chronic diseases, but had much less information regarding persons with multiple conditions, one of the features that distinguishes many complex patients. Young et al, whose literature review examined chronic illness care in general, although with a focus on the complex patient, had an easier time discovering high-quality relevant research.

One likely contributor to the dearth of high-quality studies of the care of complex patients is the lack of agreement on what constitutes a complex patient. Although the explicit conference focus on “chronic care” excluded the obviously complex patient developing multiorgan system failure during an intensive care unit for sepsis, the definition of complexity remained unclear. It seems likely that the authors of these manuscripts would differ in how to rank the importance of the number of chronic conditions versus their long-term morbidity implications versus the complexity of their current management. It seems that almost all complexity literature would identify a patient with depression, diabetes with microalbuminuria, osteoarthritis, and hypertension as complex. However, many VA primary care providers would note that almost all their patients are complex by this criterion! If asked to identify the factors that identify their “more complex” patients, we suspect that many clinicians would answer “it depends”—a hard standard to apply when analyzing large populations, although an experienced provider might be able to pick out such a patient after just one visit.

With this in mind, we were glad to see Safford et al present one theoretical construct of complexity, an important exercise that will need to be repeated by other researchers. Similarly, we were pleased to note that the 3 articles focused on presenting original data took different approaches to defining complexity, providing the reader and future researchers ample food for thought regarding the difficulty of the issue. We hope that the following insights gleaned from our struggle with evaluating this diverse set of manuscripts will provide some guidance to future editors, scientific reviewers, and funding agencies, as well as persons trying to clarify how best to study this complex topic.

First, we were struck by the broad variety of papers we saw. The diversity of methods and topics was daunting and exciting. In the same conference, one set of authors tried to develop a conceptual model needed to define the field, whereas others looked at applying systems engineering approaches to that field. A third paper presented the business case for developing new approaches to treating the target population, something typically left for last in American medicine. Whereas this diversity of topics in part reflects the Department of Veteran’s Affairs dual focus on methodologic rigor and practical application, we also believe that the topic’s breadth and importance attract researchers from many disciplines who bring many perspectives about what is important.

Second, we noted that both generalizability and internal validity are likely to be threatened in studies of complex patients. Generalizability has been an issue in the interpretation of study results for many years: all health care providers, except those practicing in one Massachusetts town, have had to wonder whether the Framingham data could be applied to their own patients. However, it is particularly hard to decide whether the results seen in a researcher’s set of complex patients should be applied to an individual complex patient being seen by a practicing Internist seeking to practice evidence-based medicine, or even to the group of complex patients served by a given health care system.

Internal validity is enhanced when a study is adequately powered to minimize type I and II errors, clearly defines the study intervention(s), and measures outcomes in a clear, reproducible fashion. Unfortunately, complexity makes it hard to do each of these. It is difficult to assemble a population that is large enough to identify anything but the strongest relationships. In this supplement, we see this tension as researchers must decide whether sharing several common chronic conditions is enough to define a meaningful population even when many members of that population have other, more important conditions that were not common enough to warrant inclusion in the definition. Interventions that are of interest in a diverse population of complex patients are likely to be multifaceted interventions, which may need to be delivered one way to some complex patients and in a different way to others. Quality of life and functional outcomes that are increasingly staples of health services research are harder to interpret and utilize in the same way among persons with differing types of complexity.

Third, we thought that the counterintuitive results of Lee et al, might provide particularly important methodologic lessons. These investigators used the number of conditions defined by ICD9-CM diagnosis codes to measure increasing complexity and found that the addition of certain conditions appeared to decrease mortality, although overall, the more conditions that were present, the greater the mortality. Over 3 decades of work with claims data has taught the Health Services Research community that incentives inherent in the process by which claims are created can generate spurious associations.^3,4 Thus, because more “optional” diagnoses may be coded for patients who are less severely ill, osteoarthritis (one of the protective conditions) may simply be a marker for limited severity of illness. However, it is unclear that the effects seen by Lee et al are associations created by the method, or if greater complexity may lead to more encounters with the health care system, proving greater opportunities for appropriate preventive services to be delivered. As the research presented in this supplement was conducted in the VA, the largest health care provider in the United States, and used its robust electronic medical system, it is not surprising that ICD9-based definitions of complexity were used. However, such results clearly call for confirmation in research using alternate methods.

Because of the importance of understanding these approaches, we encouraged these authors to describe their methods in detail so that other researchers could use or modify them as they see fit. It will be as important to clearly state how one defined complexity as it is to explicitly state whether one considers hypertension present above 160/95, 140/90, or 125/75 mmHg. One interesting next step may be to use different definitions of complexity within a project, in a type of sensitivity analysis. It may be that treating mental health issues as a separate complexity axis should be an important part of all studies of complexity. Similarly, one wonders if existing readily available measures of socioeconomic context (ZIP code level variables for example) should be routinely added to counts of comorbid conditions. When considering clinical comorbidities, researchers will have similar tough decisions. For example, they will need to decide whether they need to go beyond tumor (yes/no) and metastatic (yes/no) used by the Charlson comorbidity scale when deciding how much added detail about severity adds to counts of diseases.⁵ We suspect that in most cases the limited clinical detail included in the ICD9-CM coding system will feel very limiting indeed. Moreover, even when added levels of detail are present, it is likely that the accuracy of coding will decrease as the level of detail increases.

Related to this concern is the observation that although Safford et al point out that complexity has many components or axes, the papers that present empiric data tend to focus simply on burden of illness. This is also seen in the classic references showing, for example, that the 23% of Medicare patients with 5 or more chronic conditions account for more than two thirds of program expenditures.⁶ As claims provide more data on medical than social contributors to complexity, the emphasis on the former is in part a response to data availability. However, it will be necessary to develop databases that include measures of these other axes. It will take significant reorientation of funders to provide resources for this new kind of data collection. In turn, researchers will need to take advantage of existing datasets that do combine social and medical data to a greater extent—Medicare’s Current Beneficiary Survey comes to mind.⁷

Finally, we noted that the problems we are seeing with complex patients are not novel. Rather, they are just more complex!! A recent supplement to the American Journal of Medicine identified heterogeneity of treatment effects as an important issue. One article focused on differences in response to hypertension treatment within and between randomized trials.⁸ The authors note that patients with different age and genetic background may respond differently to the same treatment. This problem seems quite tractable when one compares this to determining whether a specific strategy can improve the outcomes of patients afflicted with “complexity.” In an editorial accompanying the supplement, Sheldon Greenfield adds illness severity, concomitant medications and comorbidities, treatment setting, and diet as potential influences on treatment response.⁹ Each of these is likely to differ among complex patients.

In addition, practical issues may make it problematic to fund research that addresses complexity. The lack of a specific disease focus will limit access to the increasingly large pool of money that the National Institutes of Health (NIH) is directing at studying “real life” disease problems. Just as quality improvement researchers found it hard to craft practical projects sufficiently rigorous to be competitive for VA Health Services Research and Development (HSRD) or Agency for Healthcare Research and Quality (AHRQ) funding, complexity researchers may have difficulty convincing these same study sections that they have been able to adequately define populations, interventions, and outcomes in a way that reliably answers a set of clear hypotheses. Funding agencies and research agencies need to find ways to deal with this type of research. Within VA, this may be possible through the Quality Enhancement Research Initiative (QUERI) mechanism as well as the recently convened Chronic Disease Management study section and HSRD solicitation regarding this important area of research. Within AHRQ and the NIH, specific dollars have been set aside to fund novel, high-priority areas like developing practice-based research networks, expanding the reach of health information technology or fostering community-based participatory research. Foundations have often been key funders of the first researchers in novel areas. We hope this supplement raises the profile of complexity research so that these key constituencies recognize its importance.