A Novel Approach to Gathering and Acting on Relevant Clinical Information: SCAMPs

Abstract

The current tools to adequately inform the process of improving health-care delivery consist primarily of retrospective studies, prospective trials, and clinical practice guidelines. We propose a novel and systematic approach that bridges the gap of our current tools to affect change, provides an infrastructure to improve health-care delivery, and identifies unnecessary resource utilization. The objective of this special article is to introduce the rationale and methods for this endeavor entitled “Standardized Clinical Assessment and Management Plans” (SCAMPs). SCAMPs take a relatively heterogeneous patient population and through a process of iterative analysis and modification of standardized assessment and management algorithms, SCAMPs allow the intrinsic biologic variability in a patient population to emerge and be understood. SCAMPs can be used to complement our currently available tools in order to result in incremental and sustained improvement in health-care delivery.

Introduction

The current US health-care debate is fueled by the alarming fact that by 2018, if left unchecked, health-care expenditures will comprise 20% of the gross domestic product.¹ The implications of uncontrolled health-care spending have direct negative consequences on the US economy, impede US competitiveness in a global marketplace, preclude the ability to provide universal coverage, and shift a larger portion of the financial health-care burden to individual taxpayers. Some of these increased expenditures can be explained by the advent and accessibility of expensive technological advancements. Improved technologies, however, are not always associated with improved health-care delivery and/or effectiveness.²

It is inevitable that health-care reform will occur, with the primary agenda to reduce health-care expenditures. Precisely which tools will inform this process while preserving, if not improving, health-care delivery remain unclear. We propose a novel and systematic approach that improves our means of effecting change, provides an infrastructure to improve health-care delivery, and identifies unnecessary resource utilization. The objective of this special article is to introduce the rationale and methods for this endeavor entitled “Standardized Clinical Assessment and Management Plans” (SCAMPs).

Current Theory and Tools

Borrowing and adapting from theories in management research, significant work has been done on descriptive versus prescriptive theory building.³ Lessons learned from the application of these concepts in the business world may have direct and important implications on medical and scientific explorations.

Retrospective Studies and Descriptive Theory Building

Descriptive theory building has three essential steps and is diagramed schematically in Figure 1.

Figure 1.

Descriptive versus prescriptive theory building. A schematic representation of descriptive versus prescriptive theory building, adapted from Carlile et al. The Cycles of Theory Building in Management Research; 2006. In descriptive models of theory building, an observation is made, attributes categorized, and attempts are made at proving or disproving a correlation in a retrospective fashion. The prescriptive approach starts with a hypothesis that states there may be a causal relationship between an attribute and an outcome of interest. This is prospectively investigated to try to prove or disprove the causal relationship.

This is the typical process undertaken with any retrospective study, case-control, or cohort study and to some extent, prospective registries. The purpose of a retrospective study is to prove via inductive reasoning a correlation between the attribute and outcome of interest. The limitation of this approach is that a causal relationship often cannot be proven.

Prospective Studies and Prescriptive Theory Building

Prospective studies, in particular randomized controlled trials, attempt to directly prove the causal relationship between interventions and clinical outcomes. In this prescriptive approach to theory building, research begins with the generation of a hypothesis that states that there may be a causal relationship of interest (Figure 1). The trial is designed in order best explore this pre-defined relationship. Typically, the entry criteria for trial participation select a relatively homogeneous group because reduction in variability among study subjects reduces the sample size necessary for sufficient statistical power. The goal of the trial is to assess whether an intervention improves outcomes, and to guide decisions about the use of the intervention in future patients with the disease of interest. The randomized control trial remains the “gold standard” for attributing a causal relationship, but a highly homogeneous patient population and the narrowly defined outcome variables of interest can limit generalization of results to other patient populations and to other relevant outcomes.

Clinical Practice Guidelines

In 1973, a landmark article⁴ demonstrated the substantial variation that existed in clinical practice among different geographic areas. Since then, there has been a nationwide and concerted effort to reduce practice variation, a phenomenon that leads to higher resource utilization and negatively affects care delivery.⁵ This effort has taken the form of clinical practice guidelines (CPGs). CPGs have been shown to standardize care, diminish local variation, and improve health outcomes.⁶ The fact that CPGs rely on both evidence and opinion also means that they are neither infallible nor absolute, and are subject to influence and bias.^7,8 The success of CPGs relies, in part, on the concept of “best clinical practice.” Unfortunately, for many conditions, the data to support this concept are lacking, especially as they apply to a specific patient with a specific condition.⁹ Most CPGs do not include an infrastructure to capture critical data related to their use, such as compliance, unexpected outcomes, or resource utilization. CPGs are therefore revised periodically based upon data generated from new publications or information resources outside of the CPG itself.

The Role of Anomalies or Deviations

Unexpected findings, anomalies, or practice deviations are often lost in the implementation of CPGs, as the emphasis is primarily on ensuring compliance rather than understanding why deviations occur in the first place. When they do occur, anomalies or deviations require revisiting the categorization or hypothesis stage (Figure 1).³ Unfortunately, critical data elements are often missing, precluding the ability prove the correlation or causal relationship. When data are available and guidelines are redesigned, these types of findings often lead to significant advances in our understanding and can eventually lead to the discovery of the fundamental causal relationship between an attribute and the desired outcome.

The results of even the most robust retrospective study primarily infer a correlation, and generally cannot establish a causal relationship between attribute and outcome. Theorizing on the existence of such causality requires a “cognitive leap”; the jump from a proven correlation to the hypothesis that an attribute (or set of attributes) actually causes the outcome of interest.³ It is a common error in research to assign validity to a hypothesis generated from descriptive research; however, validity can only be provisionally established by testing a hypothesis through prospective or prescriptive research. In clinical research, tools to facilitate this process––making the “cognitive leap” from a descriptive study to a prescriptive one––are in short supply and make moving from observation to proof of causality a long and unpredictable process.

We propose a novel and systematic approach entitled SCAMPs in order to bridge these gaps and supplement the current tools of retrospective studies, prospective trials, and CPGs in an attempt to improve health-care delivery and identify unnecessary resource utilization.

SCAMPs Methodology

SCAMP Assumptions

SCAMPs are an innovative and systematic approach to gathering and acting on relevant clinical data. There are several assumptions that drive the aims, methodology, and design of SCAMPs:

1. There is uncertainty in virtually all medical decision–making, and actual data informs a minority of most decisions made in real world practice.⁹

2. “Best” clinical practice is a moving target and a possibly unattainable goal; however, there are “sound” and reasonable clinical practices.

3. Standardizing the assessment and management of a relatively diverse patient population will allow for differences, based on clinical heterogeneity, to emerge.

4. Deviations from “sound” practices are an important source of information and can potentially lead to significant discoveries.

5. Since “sound” practices are never assumed to be “best” practices, the rules for modifying them should be flexible, not rigid.

SCAMP Aims

The aims of SCAMPs are as follows:

1. Reduce diversity of patient assessment and management through standardization of care.

2. Improve patient care delivery through systematic optimization of assessment and management algorithms.

3. Assess the effectiveness of diagnostic testing and management interventions.

4. Reduce ineffective or unnecessary resource utilization without negatively impacting patient care delivery.

SCAMP Development

Developing a SCAMP is a stepwise and incremental process. The major steps require the creation of: (1) a background position paper; (2) plausible outcomes; (3) consensus regarding entry criteria, assessment, and management algorithms; (4) targeted data collection; and (5) SCAMP data forms (SDFs). Figure 2 is a schematic diagram of the overall SCAMP process.

Figure 2.

A schematic representation of the Standardized Clinical Assessment and Management Plan (SCAMP) process. Starting with a background paper, plausible outcomes are generated that drive the consensus around entry criteria, assessment, and management algorithms. Using targeted data collection, multiple plausible outcomes are explored simultaneously and through iterative data analysis, SCAMP modifications, and exploration of deviations.

To better illustrate the components of a SCAMP, examples from the Hypertrophic Cardiomyopathy (HCM) SCAMP will be used throughout the remainder of this manuscript. Variable and unpredictable outpatient assessment and management for this patient population makes it difficult to draw meaningful conclusions from existing data. There is clearly no “best” assessment and management practice for this cohort.

Background Position Paper

Development of a SCAMP starts with an exhaustive review of the available and current literature on the disease topic of interest. In particular, this review should cover the incidence and natural history of the disease, modern diagnostic tools and methods, data on subclassifications and known complications, current management algorithms, and unanswered questions. This process often requires new clinical retrospective studies to better understand patterns of current resource utilization, potential risk factors, and outcomes specific to a particular institution.^10–12 This substantive effort and understanding of a disease process must occur first before any meaningful progress is possible.

Plausible Outcomes

Plausible outcomes are 8–12 “hypotheses” that can be explored by a SCAMP, but do not require the same rigor in development that would be necessary for a randomized control trial. When possible, they should be specific and relatively precise as they are instrumental in creating the algorithms and important data points to be collected by the SCAMP. SCAMPs are designed to explore multiple questions simultaneously; however, SCAMPs are not always expected to prove or disprove causal relationships. Table 1 lists examples of some of the plausible outcomes from the HCM SCAMP.

Table 1.

Selected examples of plausible outcomes from the HCM SCAMP

In patients older than 2 years with a positive family history, an abnormal left ventricular septal E′ velocity will be more predictive of a positive genotype result than a wall thickness z-score > 2.5.

Left ventricular outflow tract obstruction on stress echocardiogram is predictive of impaired exercise tolerance in patients with HCM.

Myocardial delayed enhancement is a surrogate of hypertrophy and adds no additional value in predicting outcome in HCM in children.

HCM SCAMP, Hypertrophic Cardiomyopathy Standardized Clinical Assessment and Management Plan.

Consensus

Consensus is required among knowledgeable clinicians to standardize entry criteria, assessment, and management algorithms. SCAMPs make the assumption that there are no “best” clinical practices, that the SCAMPs are designed to be changed, and that deviations are allowed. This facilitates the generation of precisely defined agreement for all aspects of the SCAMP. For the management algorithms, known as decision trees, branches or decision points must be explicitly defined, even if they are arbitrary. Figures 3 and 4 detail a clinical assessment encounter and decision tree, respectively, from the HCM SCAMP.

Figure 3.

Clinical assessment encounter for the family history of HCM. An example of a standardized assessment and management encounter for patients with a family history of HCM. SDF, SCAMP data forms; ECG, electrocardiogram; LA, left anterior; LV, left ventricular; ECHO, echocardiogram; HCM, hypertrophic cardiomyopathy; SCAMP, Standardized Clinical Assessment and Management Plan.

© Copyright 2010 Boston Children’s Heart Foundation.

All Rights Reserved. For permissions contact Bridget Stewart [617 355-7567].

Figure 4.

An example of a decision tree from the Hypertrophic Cardiomyopathy Standardized Clinical Assessment and Management Plan describing how to approach genetic testing. CV, cardiovascular.

© Copyright 2010 Boston Children’s Heart Foundation.

All Rights Reserved. For permissions contact Bridget Stewart [617 355-7567].

Targeted Data Collection

Systematic, but targeted data collection is critical to developing a SCAMP. Identifying too many unnecessary data points leads to noncompliance, while collecting too few data elements limits the ability to draw meaningful conclusions. Data collection should be focused on addressing the plausible outcomes. Data elements outside the spectrum of routine clinical care (e.g., similar to what would be collected in a prospective investigative research trial) must be avoided. A color coding scheme is used to prioritize the data collection: (1) green data elements are those routinely done as part of current practice, but will not be collected as part of the SCAMP; (2) yellow data elements represent important data that could prove useful in analysis or future management, and are recorded by the SCAMP; and (3) red data elements represent critical information that is collected by the SCAMP and used as part of the decision management algorithms. Figure 3 illustrates the types of targeted data collection and color scheme used by the HCM SCAMP.

SCAMP Data Forms

SDFs, in either paper or electronic form, must be concise and efficient allowing for the least amount of work-flow disruption for clinicians. Appropriate management algorithms should be made immediately available to clinicians on the form. Deviations are allowed based on best clinical judgment, but the forms must prompt the clinician to explain their rationale for the deviation. Figure 5 shows a page from one of the paper versions of an SDF.

Figure 5.

An example of a paper-based SCAMP data form from the HCM SCAMP. SCAMP, Standardized Clinical Assessment and Management Plan; HCM, hypertrophic cardiomyopathy; ECG, electrocardiogram; ECHO, echocardiogram; MRI, magnetic resonance imaging; NP, nurse practitioner.

© Copyright 2009 Boston Children’s Heart Foundation.

All Rights Reserved. For permissions contact Bridget Stewart [617 355-7567].

SCAMP Implementation

Success of a SCAMP hinges on an implementation strategy that ensures the highest rate of compliance with the least amount of interference to the work-flow for clinicians. Depending on the infrastructure and resources available, an implementation strategy must be tailored to the culture and environment of that particular department, division, or practice. Any implementation strategy must include at a minimum the following components: (1) methods to identify and screen eligible patients; (2) mechanisms to collect and store data; and (3) quality control procedures to ensure data integrity and compliance. Although the above SCAMP components can be implemented using a paper-based strategy, there are significant benefits to using sophisticated information technology tools. This will often require the modification of current clinical information systems (e.g., electronic medical record software) and/or the creation of new, but highly integrated computer applications. We are currently developing an exportable computer software application that will facilitate the development, implementation, and analysis of SCAMPs.

SCAMP Analysis

A key distinguishing feature of the SCAMPS process is iterative analysis and modification. Depending on the disease process being evaluated, these intervals are defined by time (e.g., every 6 months) or by number of patients enrolled (e.g., every 100 patients). Recommendations can include adding or removing history, physical exam, or clinical testing requirements, limiting certain evaluations to patients meeting certain criteria, changing testing or visiting intervals, or otherwise modifying decision algorithms. Recommendations are also made for changing entry or exclusion criteria, for adding or eliminating plausible outcomes, and for the data collection process. Although the SCAMP data are used to inform this process, recommendations do not need to be based on rigid statistical criteria as would be typically used to make inferences in clinical research (e.g., P < 0.05) since SCAMPs are only designed to reflect sound, consensus-based practice.

Decisions about whether or not to modify any particular aspect of a SCAMP will be based not only on the importance of the therapeutic effect or the severity and likelihood of adverse outcomes, but also on the resources consumed. Costly tests to identify rare, low severity clinical conditions will be the most likely to be eliminated. This incorporation of both pre-SCAMP and post-SCAMP probabilities into the clinical algorithms is an informal application of Bayesian decision-making. We are currently in the process of formalizing this Bayesian approach, by prespecifying both estimates and level of confidence for key data parameters, (e.g., the incidence of sudden death in patients with hypertrophic cardiomyopathy). Prespecification of post-SCAMP estimates that would justify SCAMP modification will also be made.

Discussion

SCAMPs are a novel quality improvement initiative to improve patient care delivery through the standardized assessment and management of a heterogeneous population. The purposes of SCAMPs are to reduce clinical practice variation, reduce resource utilization, and optimize patient care. By standardizing care in a somewhat heterogeneous population, SCAMPs will allow the biologic and clinical variability inherent in that patient population to be identified and analyzed and will allow disease-relevant residual sources of practice variation to emerge. Data generated by the SCAMPs can then be used to improve both the efficacy and efficiency of clinical care in an iterative fashion.

SCAMPs may offer solutions where the traditional tools of retrospective studies, prospective studies, and CPGs have had limited success. Figure 6 schematically displays the role of SCAMPs in relationship to these other tools. The lessons learned from the SCAMP process can drive the design of new retrospective and prospective studies. SCAMPs will improve our ability to identify important subcategories for a particular disease entity and highlight potential correlations and causal relationships between attributes and outcomes of interest. Current real-world experience has taught us that medical discovery and understanding is a never-ending and continually evolving process. If and when medical “truth” is discovered, a CPG focusing on physician compliance and elimination of deviations can then be developed and implemented.

Figure 6.

The role of SCAMPs, retrospective studies, and prospective studies. A schematic representation of how SCAMPs fit in the overall paradigm of descriptive (retrospective) and prescriptive (prospective) study design. SCAMPs, as a quality improvement initiative, can be an integral tool in the generation of data than can lead to better retrospective and/or prospective research studies. When “truth” is known from a compilation of data from all three methods, then a clinical practice guideline can be created and implemented. SCAMPs, Standardized Clinical Assessment and Management Plans; CPG, clinical practice guideline.

Intermountain Healthcare

Comparison can be drawn between SCAMPs and care process models (CPMs) developed at Inter-mountain Healthcare (IHC), a nonprofit health system based in Salt Lake City, Utah that has long been an innovator and leader in quality improvement efforts.^13,14 CPMs are evidence-based, presumed “best” practice protocols used to guide patient care and facilitate collection of data on clinical, cost, and satisfaction outcomes. In many ways, IHC’s efforts are similar to CPGs combined with improved data capture and periodic analysis and review. Although CPMs and SCAMPs have similarities in both design and purpose, they differ in at least three major ways. First, CPMs are specifically developed in areas for which significant evidence from clinical research exists, such that best practices can be identified and delineated.¹⁴ SCAMPs, on the other hand, were designed to address diagnoses for which only limited evidence exists and thus only “sound” practices can be claimed. This discrepancy in background evidence leads to the second major difference. CPMs tend to focus on the execution of guidelines (process improvement) rather than on the content of these guidelines. SCAMPs have the capacity to address process improvement and guideline content concurrently. Indeed, the definition of plausible outcomes early in the development process prioritizes content development as an objective of a SCAMP. Lastly, CPMs and SCAMPs differ in their handling of deviations. CPM reviews focus on outcomes data and new research, and while information on deviations from the protocol is considered, it is not solicited from clinicians and must be volunteered.¹⁴ Through purposeful design, SCAMPs actively invites knowledge-based deviations from its protocols and solicits clinician explanations of these deviations. The goal of this solicitation is to more quickly understand and adopt clinician-based innovations, which are perceived by the SCAMP process to be a rich source of improvements to “sound” practices.

Quality Improvement versus Research

The question has been raised as to whether the SCAMP process requires a review by the Institution Review Board (IRB). IRB review is required for human subjects research, which is defined as an activity designed to test a hypothesis, permit conclusions to be drawn and thereby develop or contribute to generalizable knowledge, whereas medical practice refers to interventions that are designed solely to enhance the well-being of patients. The intent of the SCAMP process is similar to the intent of a CPG, that is, to improve patient care. From the perspective of patient care, the two fundamental aspects of the SCAMP initiative are: (1) guidelines concerning clinical care; and (2) enhanced data capture. Consensus regarding the assessment and management algorithms focuses on developing sound clinical practices. No clinical tests or interventions are undertaken for purposes other than patient care. The enhanced data capture is intended to document the basis of clinical care decisions, including those decisions which are not in alignment with the SCAMP clinical care algorithm. The intent of the SCAMP process is therefore to provide clinical care guidelines using a process that includes a feedback loop, whereby the care algorithm can be revised and improved over time, and as such, falls squarely within the scope of quality improvement. The enhanced data capture clearly provides enormous research potential, and this post hoc analysis will require IRB approval, which will be requested on an individual basis as these retrospective analyses are undertaken. Figure 6 diagrammatically outlines the SCAMP development and SCAMP analysis and modification processes as quality improvement initiatives, while any retrospective or prospective studies performed on the collected data are considered research.

The Prior Experience at Children’s Hospital Boston

In 2006, the Department of Cardiology at Children’s Hospital Boston began a system-wide effort to develop numerous CPGs for outpatient practice. After considerable effort it became clear that there was not enough reliable evidence to establish recommendations for “best clinical practice.” A seminar at the Harvard Business School led by Richard Bohmer, along with a review of his publications on health-care management,^15,16 led to the realization that an iterative strategy could result in improved health-care delivery. In retrospect, it became apparent that the progress made in numerous disease entities was due to a similar SCAMPs-like methodology.^17,18

Current Progress

We have currently developed and implemented six SCAMPs in the Department of Cardiology at Children’s Hospital Boston and enrolled nearly 800 patients. These SCAMPs include patients after the arterial switch operation, patients with hypertrophic cardiomyopathy, dilated aorta, valvar aortic stenosis, valvar aortic regurgitation, and patients between the first two palliative operations for single ventricle physiology. Although our first SCAMPs have focused primarily on outpatient assessment and management, the next five SCAMPs in development focus on inpatient diagnoses (e.g., patients with hypoplastic left heart syndrome with mitral stenosis and aortic stenosis), common diagnoses (e.g., chest pain in children), and diagnoses that require integration and coordination with primary care providers (e.g., chest pain in children) and other specialists (e.g., cardiac and neurological evaluation of syncope).

Future Directions

Implementation of SCAMPs using a paper-based approach is feasible and for many organizations, might be the only practical approach. Unfortunately, this approach requires a significant commitment of personnel resources to screen and enroll patients, distribute forms, enter data, and to follow-up on missing or incomplete data. Our experience has also taught us that a sophisticated information technology solution is critical for any large-scale SCAMP deployment. We are currently developing a computer application that facilitates SCAMP development, implementation, and data analysis. This system will encompass data collection, integration with other clinical systems, assessment and management algorithms, and capture of deviations. This information technology system will be exportable to other centers and scalable for multisite deployment with centralized aggregation of patient de-identified data. Our intention is to make this software available at minimal cost.

Conclusion

SCAMPs are a novel quality improvement initiative designed to reduce practice variation, improve patient care, and reduce unnecessary resource utilization. The regular periodic revision of SCAMPs is informed by both experience derived from their application and by newly published literature. Furthermore, by suggesting a management scheme for biologically diverse patients and recording the results and decision path, SCAMPs can facilitate identification of subgroups of patients who benefit from particular testing or management strategies. This systematic approach has the promise to result in incremental and sustained improvement in health-care delivery for many conditions where the traditional tools of clinical research have been ineffective.