Validation of a Perfusion Registry: Methodological Approach and Initial Findings

Abstract:

Although regional and national registries exist to measure and report performance of cardiac surgical programs, few registries exist dedicated to the practice of cardiopulmonary bypass (CPB). We developed and implemented a cardiovascular perfusion registry (Perfusion Measures and outcomes [PERForm] Registry) within the structure of the Michigan Society of Thoracic and Cardiovascular Surgeons (MSTCVS) to improve our understanding of the practice of CPB. The PERForm Registry comprises data elements describing the practice of CPB. Fourteen medical centers within MSTCVS have voluntarily reported these data on procedures in which CPB is used. We validated the case count among procedures performed between January 1, 2011 to December 31, 2011, and validated the values among 20 fields at three medical centers. We queried database managers at all 14 medical centers to identify the infrastructure that contributed to best overall data collection performance. We found that 98% of all records submitted to the PERForm and 95% of those submitted to the Society of Thoracic Surgeons (STS) matched. We found quite favorable agreement in our audit of select fields (95.8%). Those centers with the most favorable performance in this validation study were more likely to use electronic data capture, have a perfusionist as the STS database manager, and have involvement of the STS database manager in the PERForm or STS databases. We successfully and accurately collected data concerning cardiovascular perfusion among 14 institutions in conjunction with the MSTCVS. Future efforts will focus on expanding data collection to all MSTCVS participating institutions as well as more broadly outside of Michigan.

Open heart surgical procedures are some of the most commonly performed procedures. Patients undergoing these procedures are at risk for a number of adverse sequelae including morbidity and mortality. Such events may be attributed to patient and disease characteristics as well as processes of clinical care. A number of cardiac surgical registries, including the Society of Thoracic Surgeons (STS) Adult Cardiac Surgical Database, have emerged as vehicles to help inform clinical practices. In addition, regionally based quality improvement collaboratives have leveraged such data to engage clinical providers in developing and implementing targeted quality improvement interventions.

In the latest update of the STS/Society of Cardiovascular Anesthesiology blood management guidelines, Dr. Ferraris and colleagues noted the association between cardiopulmonary bypass (CPB) practice and risk of red blood cell and platelet transfusions (1). Other investigators have noted the association between CPB practice and the rate of other morbid outcomes such as acute kidney injury, neurologic injury, and low cardiac output syndrome. Although the STS database contains a number of fields reflecting the practice of CPB, there are additional details and data that may improve the assessment and outcomes of patients undergoing cardiac surgery.

We sought to develop and implement a cardiovascular perfusion registry (Perfusion Measures and outcomes [PERForm] Registry) to improve our understanding of the practice of CPB within the setting of a regional cardiovascular surgical collaborative (Michigan Society of Thoracic and Cardiovascular Surgeons [MSTCVS]).

MATERIAL AND METHODS

Data Sources: Surgical Data Elements

The MSTCVS adult cardiac surgery database is comprised of STS data as well as state-specific data fields from participating sites in Michigan. Records include 1) patients whose surgeon and/or hospital is an active participant in the STS National Database; 2) surgical procedures on the heart and/or great vessels with or without cardiopulmonary bypass; and 3) patients who are 18–110 years of age. All 33 medical centers in the state of Michigan that perform cardiac surgery voluntarily report their data to the STS and MSTCVS. Methods for data collection at participating centers and subsequent submission to the STS and MSTCVS have been reported previously (2). Each patient has one record in the MSTCVS database per admission; any subsequent procedures during the admission are considered “postoperative events” or “complications” of the primary procedure in accordance with the policies and procedures dictated through participation in the STS Adult Cardiac Surgical Database. Data elements include patient and disease characteristics, intraoperative practices, and clinical outcomes (both in-hospital and 30-day). Each MSTCVS site is subject to an audit through its participation in the STS adult cardiac surgery database. In addition, there is an internal auditing process through the MSTCVS, whereby sites are audited by MSTCVS staff for consistency and range checks. In addition, there is a very active database managers group that meets several times a year to discuss issues or concerns regarding data elements and definitions.

Data Sources: Perfusion Data Elements

The PERForm Registry comprises data elements describing the conduct and practice of CPB (see Appendix). Fourteen medical centers in the state of Michigan have voluntarily reported these data on procedures in which CPB is used. Data are submitted through a web portal using a Citrix client to a dedicated data warehouse developed by Armus Corporation (San Mateo, CA), a certified STS vendor. We used this perfusion data along with the MSTCVS STS database to identify whether, for each PERForm record, there was a corresponding record in the MSTCVS surgical database.

Variables were selected based on the input of clinicians in- and outside of the state of Michigan. Field selection and definitions drew heavily on the experiences from the Northern New England Cardiovascular Disease Study Group, Perfusion Downunder Collaborative, and prior experiences related to the development of cardiovascular perfusion registries. The final data set comprises 77 fields reflecting the following categories: patient demographics, personnel, extracorporeal circuit characteristics, anemia, fluids, blood products, myocardial protection, temperature, aortic disease, drugs, and duration (time) indices.

Case-Level Validation Algorithm

All STS cases performed between January 1, 2011 to December 31, 2011, with the field CPB Utilization marked as “full” or “combination” in the PERForm Registry were identified and extracted from the MSTCVS STS data warehouse. Using a series of Structured Query Language (SQL) routines, each MSTCVS case was compared with each case entered into the PERForm perfusion registry. A hierarchical SQL automated case match was achieved if one of the conditions in Table 1 were met. A complete match was achieved if the following variables across both data sources matched: hospital National Provider Identifier, CPB and clamp time duration (within a 3-minute window), patient date of birth, admission date, and surgery date. Medium and high matches shared similar criteria to the complete match but required agreement between the two data sources on either one or two date variables (date of birth, admission date, and surgery date), respectively. A low match shared similar criteria to the complete match except made no restrictions on date of birth or admission date and allowed for the surgery date to vary within a 2-day period.

Table 1.

Matching criteria between the Society of Thoracic Surgeons and PERForm registries.

Match Type	Criteria
	Hospital NPI	CPB Time	Clamp Time	Date of Birth	Admission Date	Surgery Date
Match confidence level
Complete	Yes	±3 mins	±3 mins	Yes	Yes	Yes
High	Yes	±3 mins	±3 mins	Yes	Yes	No
				Yes	No	Yes
				No	Yes	Yes
Medium	Yes	±3 mins	±3 mins	Yes	No	No
				No	No	Yes
				No	Yes	No
Low	Yes	±3 mins	±3 mins			±2 days

NPI, National Provider Identifier; CPB, cardiopulmonary bypass.

Records that were not matched using this SQL automated process were reviewed by the registry manager (TP). Often simple data entry errors such as transposing CPB and cross-clamp times could be identified by simultaneous review of records from both data sources. These matches were left to the discretion of the reviewer and identified in the match table as a manual match.

Field-Level Audit

Trained data abstractors (PT, GB) conducted a random audit at three medical centers on a select set of 20 data elements within the records of 55 patients undergoing cardiac surgery. The MSTCVS audits all 33 member sites once a year. In anticipation of a large STS upgrade release in the second half of 2011, the MSTCVS audit was completed in the first half of the year. PERForm sites that have been audited to date are included in this article.

Abstractors compared the values reported through the PERForm Registry relative to the center’s electronic or paper medical record including the operative report, perfusion and anesthesia records, flow sheets, and medication logs. The following fields were audited: medical record number, date of birth, date of surgery, date of admission, surgeon, perfusionist, procedure type, CPB duration, cross-clamp duration, circulatory arrest duration, constituents included within the prime, packed red blood cells while on CPB, lowest hematocrit during CPB, and inotrope count during CPBwean, at 4 hours, and at 48 hours beyond surgery.

Characteristics of Data Collection Infrastructure

Perfusion and surgery database managers at each participating medical center were surveyed regarding the infrastructure in place to facilitate data collection for the PERForm and STS databases.

Statistical Analysis

All analyses were performed with Stata 12.0 (College Station, TX).

Human Subjects

This study was approved by the Institutional Review Board (IRB) of the University of Michigan Hospitals (IRB HUM00053934, Notice of Determination of “Not Regulated” Status).

RESULTS

Table 2 shows the characteristics of the patients in this matched cohort. Just over 36% of the cohort was 70 years and older, 33.4% were female, 12.3% had a body mass index over 37 kg/m², 33.9% were diabetic, 12.7% had peripheral vascular disease, 7.2% had severe chronic obstructive pulmonary disease, 5.6% were on dialysis or had creatinine 2 mg/dL or greater, 18.9% were classified as New York Heart Association III–IV, 2.8% had a prior myocardial infarction less than 1 day, and 4.9% had an emergent or salvage procedure. Of those undergoing coronary artery bypass grafting (CABG) surgery, 30.2%had left main stenosis over 50% and 71% had three-vessel disease.

Table 2.

Preoperative characteristics.

Variable	Value
Number of procedures	3719
Demographics
Patient age (years, %)
<60	32.4
60–69	31.3
70+	36.4
Female gender (%)	33.4
Body mass index (kg/m2)
<31	63.9
31–36	23.9
37+	12.3
Comorbid disease
Diabetes, % yes	33.9
Peripheral vascular disease, % yes	12.7
COPD, % yes
No	69.4
Mild	17.1
Moderate	6.4
Severe	7.2
Dialysis or creatinine ≥2, % yes	5.6
Cardiac history
New York Heart Association III–IV (%)	18.9
Prior myocardial infarction (%)
No	65.4
≤6 hours	1.0
>6 hours but <24 hours	1.8
1–7 days	11.5
8–21 days	3.7
>21 days	16.7
Cardiac anatomy and function (among patients having a CABG procedure
Left main stenosis >50% (% yes)	30.2
Diseased vessels (no.)
0	1.1
1	7.5
2	20.4
3	71.0
Priority (%)
Elective	52.11
Urgent	43.0
Emergent/salvage	4.9

Within this cohort, isolated CABG procedures were the most frequently performed (47.7%) (Table 3). Aortic assessment by palpation alone occurred in less than half of patients (42.9%) with an additional 27.6% using transesophageal echocardiography. The median bypass time was 102 minutes and cross-clamp time 78 minutes. The most common cardioplegia strategies included: solely antegrade for induction (54.8%) and maintenance (47.5%) and 4:1 blood cardioplegia ratio (44.7%). Nearly half of patients had nadir hematocrit on bypass over 26% (48.1%). In terms of intraoperative transfusions, 21.9% received red blood cells, 4.6% fresh-frozen plasma, and 6.0% platelets.

Table 3.

Intraoperative characteristics.

Variable	Value
Number of procedures	3719
Type of procedures
Isolated CABG	47.7
Isolated valve	25.7
CABG + valve	11.1
Other	15.5
Aortic assessment (%)
None	18.2
Palpation only	42.9
Epi-aortic only	0.8
TEE only	27.6
Mixture	2.2
Missing	8.2
Cardiopulmonary bypass duration (min, median)	102.0
Cross-clamp duration (min, median)	78.0
Cardioplegia
Induction route, % yes
Antegrade only	54.8
Retrograde only	12.1
Both	33.1
Maintenance route, % yes
Antegrade only	47.5
Retrograde only	24.4
Both	28.2
Type, % yes
None	2.3
Crystalloid	1.8
1:1	0.2
2:1	0.1
4:1	44.7
8:1	36.3
Variable ratio	14.6
Intraoperative blood management
Nadir hematocrit (%)
<21	17.7
21–23	17.3
24–25	16.9
26+	48.1
Transfusions, % yes
Red blood cell transfusions	21.9
Fresh-frozen plasma	4.6
Platelets	6.0

CABG, coronary artery bypass graft; Epi-aortic, epi-aortic echocardiography; TEE, transesophageal echocardiography.

In terms of postoperative transfusions, 28.7% received red blood cells, 15.4% fresh-frozen plasma, and 18.5% platelets (Table 4). Strokes developed among 2.0%, return to the operating room for bleeding occurred among 2.8%, renal failure requiring dialysis among 1.8%, and mortality within 30 days among 3.1%.

Table 4.

Postoperative characteristics.

Variable	Value
Number of procedures	3719
Transfusions, % yes
Red blood cell transfusions	28.7
Fresh-frozen plasma	15.4
Platelets	18.5
Complications
Stroke, % yes	2.0
Return to the operating room for bleeding, % yes	2.8
Renal failure requiring dialysis, % yes	1.8
Mortality within 30 days, % yes	3.1

There were a total of 3811 records entered into the PERForm database and 3903 in the STS database from MSTCVS during the reporting period. Of these, 3722 matched, including 95% (n = 3416) of the submitted STS records and 98% of those submitted to PERForm (n = 3811) (Figure 1). Of the 3722, 92% (n = 3416) were a complete match from the PERForm, 1% a medium match (n = 21), 3% a low match (n = 94), and 5% a manual match (n = 191) (Figure 2).

Figure 1.

The percent of Perfusion Measures and outcomes (PERForm) and Society of Thoracic Surgeons (STS) data forms that were matched across all participating institutions. Data from PERForm are noted in gray and data from the STS are noted in black.

Figure 2.

The comparison of the quality of matches across participating institutions. The quality of matches range from (best to more than >worst): complete/high, medium, low, manual.

Overall, 95.8% (1035 of 1150) of the fields audited at three of the medical centers were in agreement. The fields least likely to be in agreement were inotrope counts at 4 hours (60% in agreement) and 48 hours (66.7% in agreement). Agreement ranged from 94.4% to 97.8% across the three medical centers.

Centers having more favorable performance regarding percent matches were more likely to have the following characteristics: 1) electronic data capture (meaning perfusion data acquisition systems and electronic data capture for submission to the STS registry) for either the PERForm or STS databases; 2) have involvement of the STS database manager in the collection of perfusion fields; and 3) have a perfusionist be the STS database manager (Table 5).

Table 5. Characteristics of the data infrastructure.

DISCUSSION

We found that 98% of all records submitted to the PERForm and 95% of those submitted to the STS matched within 14 participating medical centers in the state of Michigan. We found quite favorable agreement in our audit of select fields (95.8%). Although performance varied across medical centers, most matches were complete matches. Those centers with the most favorable performance in this validation study were more likely to use electronic data capture, have a perfusionist as the STS database manager, and have involvement of the STS database manager in the PERForm or STS databases. Based on these data, we are developing quality assurance reports that will be sent back to the surgeons and perfusionists within the state of Michigan. Both groups convene several times a year to go over the data. It is our expectation that such a venue, which now includes aspects of cardiovascular perfusion practices (and their relationship to clinical outcomes), will provide new opportunities for ensuring quality and driving targeted improvements.

Since the Health Care Financing Administration first reported surgeon mortality rates, a number of cardiac surgical groups have taken proactive steps to collect and report their performance (3,4). These early efforts mostly focused on reporting mortality secondary to CABG surgery as well as developing risk prediction models using information known before the operation. Such efforts as well as improvements in technology have led to substantial reductions in mortality (5). Although the collection of data at a medical center level concerning cardiac surgery used to be the noteworthy exception back in the late 1980s, nearly 100% of adult cardiac programs in the United States are now submitting to the STS National Adult Cardiac Surgery Database.

These efforts and successes should be placed in the greater context and climate of health care. Payers, accrediting organizations, patients, and families are expecting and demanding the collection and reporting of process and outcomes data. Of note and importance, these groups are increasingly holding providers and institutions financially accountable to quality “standards.” These standards, unfortunately, are often developed not by the clinicians themselves, but by other external parties (i.e., payers and accrediting organizations). Cardiovascular perfusionists, although to date immune to these pressures, should not assume that similar pressures will not be applied to their practice in the future.

Certainly a number of others groups have emerged within our perfusion profession with the focus on collecting and using data to drive quality. The earliest work stems from northern New England, where eight centers have collected perfusion data and merged such data with data collected by their surgical and anesthesia colleagues since 1995 (6–10). These data have been used to draw insight regarding the relationship between nadir hematocrit and risk of mortality (10), to track practice as it relates to guideline recommendations (6), and to monitor and reduce the risk of hyperthermia, to name a few (11). In addition, the Perfusion Downunder Collaborative has recently shown the benefit of such a registry to drive quality assurance and improvement targets in Australia and New Zealand (12–14). It is our expectation to use similar methodologies to drive quality assurance and improvement, namely through the use of timely and accurate data reports, while seated within the context of a quality collaborative.

It is commonly recognized that variation exists in the equipment and methods used for performing CPB. Although used, and undoubtedly improved, since the first successful open heart operation by Gibbon in 1953 (15), the concept of defining “best practices” for CPB has only recently begun to emerge (16). This effort has been complicated by the lack of strong evidence supporting many of the equipment and technique choices used today. The development and use of clinical registries will certainly assist in further defining and implementing best practices for our profession.

We sought to develop and test the feasibility for collecting data concerning cardiovascular perfusion. With this in mind, we reflected on the experiences of other existing regional perfusion registries (8,9,13). We also recognized that other groups had attempted and reported on their experiences, including a report by Riley and colleagues (17) reflecting on the experience of nine institutions back in the late 1990s. These sets of experiences as well as the data fields collected through these registries served as the foundation of our current version of the PERForm Registry. Importantly, we adopted similar definitions and fields across these registries to enable future anticipated benchmarking opportunities.

One of the hallmarks of a clinical registry is ensuring its integrity. Two facets that were most important and serve as the focus of this initial methodological report concern case count and field-level validation. With regard to case counts, we sought to ensure that centers accurately captured all eligible cases. With this in mind, we collected unique procedural and patient identifiers and subsequently linked the perfusion records to the records submitted to the STS. Such information will also enable linkage to other data sets, including the National Death Index, for ascertaining longterm survival. Although we found variation across the centers, overall performance was excellent. We subsequently queried each of the centers to understand which aspects of their data collection infrastructure might be associated with better overall performance. Two overall factors appeared to distinguish center performance: 1) use of electronic data collection; and 2) involvement/leadership in data collection for both PERForm and STS data collection. Such information may be helpful in appropriately resourcing data collection for both the STS and PERForm registries.

We developed several strategies to ensure the integrity of the data collected through the PERForm Registry. First, akin to other registries, our vendor (Armus Corporation, Burlingame, CA) embedded field-level validation within our data collection form. Second, we developed a center “thumbprint,” whereby 17 common data elements reflecting aspects of a particular circuit would be preset by the perfusionist. A particular medical center could preselect up to three unique circuits. Participants will be asked to review the content of their thumbprint on a regular basis to ensure the accuracy of the submitted information. Our colleagues have found that the thumbprints provide accurate data concerning their practice while speeding up the data collection process. Third, with limited exception, we purposely did not ask for perfusionists to collect information that was already collected through other data collection streams, including most patient and disease characteristics, surgical practices, and clinical outcomes. The use of the unique patient and procedural identifiers enables us to pull such information through our linkage with each patient’s STS data collection form.

In conclusion, we report the findings from our initial experience through the PERForm Registry. We found that we were able to successfully and accurately collect data concerning cardiovascular perfusion among 14 institutions in conjunction with the MSTCVS. We additionally found that 98%of the PERForm records matched to STS records, and 95% of all STS records matched those submitted to the PERForm Registry. Furthermore, we found that 95.8% of all queried fields submitted to the PERForm Registry matched the gold standard medical record. Future efforts will focus on expanding data collection to all MSTCVS participating institutions as well as more broadly outside of Michigan. Ultimately, the PERForm Registry will provide perfusionists and the cardiac surgical team with valuable information to improve the practice of CPB in adults. Importantly, such a registry will reinforce the important role of perfusion in contributing to clinical outcomes and patient safety within the broader cardiac surgical team.

APPENDIX LEGEND

This Appendix represents the version of the PERForm Registry used during the pilot phase of the project.