Abstract
Objectives
We sought to develop quality indicators (QIs) for outpatient management of adult congenital heart disease (ACHD) patients.
Background
There are no published QIs to promote quality measurement and improvement for ACHD patients.
Methods
Working groups of ACHD experts reviewed published literature and US, Canadian and European guidelines to identify candidate QIs. For each QI we specified a numerator, denominator, period of assessment and data source. We submitted the QIs to a 9-member panel of international ACHD experts. The panel rated the QIs for validity and feasibility in 2 rounds, on a scale of 1–9, using the University of California Los Angeles (UCLA)/RAND modified-Delphi method and final QI selection was based on median scores.
Results
Sixty-two QIs were identified regarding appropriateness and timing of clinical management, testing and test interpretation. Each QI was ascertainable from health records. After the first round of rating, 29 QIs were accepted, none were rejected and 33 were equivocal; on the second round, 55 QIs were accepted. Final QIs included: 8 for atrial septal defects; 9 for aortic coarctation; 12 for Eisenmenger; 9 for Fontan; 9 for D-transposition of the great arteries; and 8 for tetralogy of Fallot.
Conclusions
This project resulted in development of the first set of QIs for ACHD care based on literature, guidelines and a modified Delphi process. These QIs provide a quality of care assessment tool for six ACHD conditions. This rigorously designed set of QIs should facilitate measuring and improving quality of care for this growing group of patients.
Keywords: Congenital heart disease, quality of care, quality improvement
Introduction
The purpose of improving quality of care for adult congenital heart disease (ACHD) patients is to provide health services that increase the likelihood of desired health outcomes and are consistent with current professional knowledge.(1) The goal in providing quality care is to decrease variation in practice patterns and improve health outcomes for patients with these lifelong conditions and provide a common language for dialogue among patients, healthcare providers, administrators and policy makers.
Measurement is the first step to quality improvement. In the 1960s, Avedis Donabedian described a framework for quality assessment that formed the foundation of most quality of care research methods used today.(2) This Donabedian model divides aspects of healthcare quality into structure, process, and outcome to describe the effects of healthcare on outcomes. We constructed a conceptual framework based on the Donabedian structure-process-outcome model to illustrate this for ACHD patients.(2) This model shown in Figure 1 illustrates the integrated effects of structural, process and outcome aspects of healthcare for ACHD patients and includes moderating patient factors.
Figure 1. ACHD Quality of Care Conceptual Model.
Conceptual framework based on the Donabedian model of quality of care illustrating the integrated effects of structural, process and outcome aspects of healthcare including moderating patient factors(2)
Developing and implementing clinical care guidelines and assessing adherence to them using structure and process quality indicators can provide a baseline for quality improvement. However, guidelines differ from quality indicators in important ways. Guidelines are recommendations for care meant to be applied prospectively to individual patients while quality indicators are measures applied retrospectively to a group of patients to assess if a care process was delivered or not. (3)
For adults with congenital heart disease, comprehensive guidelines for care have been available since 1998 when Canada published the first set.(4) Subsequently Europe and the United States also published their own sets of guidelines and Canada and Europe reported their most recent revisions in 2010.(5–10) All sets of guidelines were developed in similar manners using the opinions of national and international experts in ACHD and related specialties as well as available literature and evidence. The recommendations are meant to be comprehensive for caring for ACHD patients with any lesion. Much of the evidence is level 1C (expert consensus opinion)(11) because that was the best available when the guidelines were written.
The purpose of this project was to develop quality of care assessment tools for ACHD patients based on published guidelines in order determine the extent to which the guidelines are being applied and to arrive at a set of measures that can be evaluated, revised and updated as new data become available. Given the breadth of congenital heart defects and the wide scope of inpatient, outpatient and procedural care, we chose to focus our attention on outpatient care of common adult congenital heart lesions for this first set of quality indicators (QI). Our objectives were to develop valid and feasible quality indicators for outpatient management of 6 ACHD conditions: tetralogy of Fallot (TOF), secundum atrial septal defect (ASD), coarctation of the aorta (COA), D-transposition of the great arteries (TGA) with atrial switch operation, single ventricle with Fontan operation (FONTAN) and Eisenmenger syndrome. ASD commonly has survival into adulthood, and another 3 lesions (TOF, COA, TGA) are highly represented in adult CHD clinics. This is not surprising, since palliative and reparative surgeries for TOF, COA and TGA were among the earliest CHD surgeries performed in the 1940s–1960s. We included the remaining two conditions, Eisenmenger syndrome and single ventricle physiology with Fontan surgery, due to the complexity of underlying disease and cardiovascular physiology, requiring highly specialized medical management.
Methods
Definitions of Anatomic Subgroups
Our goal in choosing the lesions and the anatomic subgroups was to attempt to capture a large but consistent proportion of patients currently followed in ACHD clinics. Patients with ASD are those with an isolated secundum ASD repaired with or without a residual shunt or unrepaired. Patients with COA are all those with repaired or unrepaired COA distal to the left subclavian artery with or without bicuspid aortic valve. Patients with ES are all those with cyanosis and pulmonary vascular disease in association with a prior right to left shunt. The Fontan subgroup includes all forms of single ventricle physiology with any type of Fontan repair. Patients with TGA are only those with D-TGA who have undergone an atrial baffle repair of the Mustard or Senning type. The TOF subgroup includes patients with uncomplicated TOF and complete intracardiac repair; the subgroup is not intended to include complex TOF with pulmonary atresia and major aorto-pulmonary collaterals or combination lesions like TOF and atrioventricular canal.
Working Groups
Published guidelines from the US, Canada and Europe and relevant medical literature were reviewed by working groups of ACHD experts to obtain candidate QIs.(5–10) In order to develop the preliminary quality indicators, working groups composed of 3–5 ACHD clinicians were organized for each condition. The working groups developed up to 10 QIs per condition. The QIs were designed to be applied to patients being followed in any outpatient cardiology practice, not specifically ACHD practices. After the working groups completed their literature reviews and developed draft QIs for their assigned conditions, all guidelines and QIs for the different conditions were systematically reviewed and revised for consistency by Drs. Marelli and Gurvitz. This led to further definition and refinement of the QIs before presentation to the panel of experts. These revisions ensured each QI included the specification of a numerator, a denominator, a data source, a period of assessment (when applicable), an evidence grade, and supporting literature (Table 1). All QIs were classified as structure or process indicators. There were no outcome indicators proposed. Levels of evidence were defined in the typical number and letter system as follows: (I) intervention is useful and effective, (IIa) weight of evidence/opinion is in favor of usefulness/efficacy, (IIb) usefulness/efficacy is less well established by evidence/opinion, (III) intervention is not useful/effective and may be harmful; Level (A) Data from many large randomized clinical trials (RCTs), (B) Data from fewer smaller RCTs, careful analysis of non-randomized studies, observational registries, and (C) expert consensus(11).
Table 1.
Metric elaboration for tetralogy of Fallot illustrating the steps in achieving operationalization based on literature review
| CHD Condition | Tetralogy of Fallot Your Name: |
|---|---|
| Metric 1 | Minimum of yearly scheduled return visit with cardiologist who has ACHD expertise for patients with TOF repair and followed by an ACHD specialist. Question we are asking: Are ACHD cardiologists scheduling patients with TOF repair for yearly visits? |
| Metric operationalized | Scheduled return visit with a cardiologist with ACHD expertise at least yearly |
| Numerator | Patients who had or were recommended to have visit with a cardiologist with ACHD expertise at least every 12 months Note: “Had” or were “recommended to have” accounts for patient non-compliance. |
| Denominator | Patients with TOF repair followed by a cardiologist with ACHD expertise |
| Source of Data | Medical record |
| Rational | |
| • Evidence Level | IC |
| • Structure | |
| • Process | √ |
| • Outcome | |
| • Guideline support | Yes |
| • Period of Assessment | 12 months |
| • References |
|
RAND-UCLA Delphi Panel Methodology
We used the UCLA/RAND appropriateness method and modified Delphi process to arrive at the final set of QIs.(12,13) This methodology includes the extensive literature review described above. The candidate QIs (Table 2) were submitted to a 9-member panel of international ACHD experts for 2 rounds of rating for validity and feasibility on a scale of 1–9. Experts for the panel were chosen based on nominations from the following organizations: American College of Cardiology (ACC), American Heart Association, Canadian Adult Congenital Heart Network, International Society for Adult Congenital Heart Disease and the medical advisory board of the Adult Congenital Heart Association (ACHA). Only 2 initial panel invitees declined, one due to prior commitments and one due to travel distance from Europe. The final panel was composed of ACHD physicians from the US and Canada with backgrounds in adult cardiology, pediatric cardiology or both. The experts were highly experienced congenital cardiologists and had spent between 12–40 years in cardiology practice. Experts in catheterization and imaging were included but no electrophysiology or transplant specialists were on the panel due to the lack of these subspecialties among the nominees.
Table 2.
All candidate quality indicators and rating scores for round 1 and round 2 ratings by the expert panel separated by lesion. The table also includes categorization of indicators by process and structure within each lesion and the level of evidence of each indicator.
| Secundum Atrial Septal Defect (ASD) | Score Round 1 Median, Mean Absolute Deviation (range) | Score Round 2 Median, Mean Absolute Deviation (range) | Recommended/Rejected | ||
|---|---|---|---|---|---|
| General Description (level of evidence) | Validity | Feasibility | Validity | Feasibility | |
| Process Indicators | |||||
| Demonstration of shunting across ASD by echocardiogram (IC) | 7.0, 2.0 (2–9) | 9.0, 1.1 (4–9) | 7.0, 1.8 (2–9) | 8.0, 1.7 (2–9) | Recommended |
| Report of presence of absence of right ventricular enlargement in unrepaired ASD by echocardiogram (IC) | 7.0, 1.0 (4–9) | 9.0, .7 (7–9) | 8.0, .7 (6–9) | 9.0, .7 (7–9) | Recommended |
| Echocardiogram report of pulmonary artery pressure by tricuspid regurgitation velocity (IC) | 8.0, .6 (6–9) | 8.0, 1.3 (3–9) | 8.0, .3 (7–9) | 9.0, .7 (6–9) | Recommended |
| Pulmonary vein anatomy determined prior to intervention (IC) | 9.0, .4 (7–9) | 8.0, 1.2 (4–9) | 9.0, .4 (7–9) | 8.0, .8 (5–9) | Recommended |
| Patient had appropriate indication for cardiac catheterization (IC) | 5.0, 1.9 (2–9) | 8.0, 1.4 (4–9) | 8.5, 1.1 (4–9) | 8.5, 1.0 (6–9) | Recommended |
| Patient did not have appropriate indication for ASD closure (IIIB) | 7.0, 1.0 (6–9) | 8.0, .9 (6–9) | 8.0, .8 (6–9) | 8.0, 1.0 (6–9) | Recommended |
| Annual clinic surveillance with cardiologist with ACHD expertise for patients with ASD closure and pulmonary hypertension, arrhythmia, ventricular or valve dysfunction (IC) | 7.0, 1.1 (5–9) | 7.0, 2.1 (3–9) | 8.0, .7 (6–9) | 9.0, .2 (8–9) | Recommended |
| Appropriate counseling of SBE prophylaxis (IIaC) | 8.0, 1.1 (5–9) | 7.0, 1.0 (6–9) | 9.0, 1.3 (5–9) | 7.0, 1.0 (6–9) | Recommended |
| Cardioversion to attempt restoration of sinus rhythm if atrial fibrillation (IA) | 6.0, 1.4 (4–9) | 7.0, 1.2 (4–9) | 3.0, 1.0 (1–6) | 5.0, 2.3 (1–9) | Rejected |
| Anticoagulation if atrial fibrillation in patient with or without ASD closure (IA) | 8.0, 1.7 (3–9) | 9.0, .2 (8–9) | 7.0, 2.1 (1–8) | 8.0, 1.9 (1–9) | Rejected |
| Coarctation of Aorta (COA) | Score Round 1 Median, Mean Absolute Deviation (range) | Score Round 2 Median, Mean Absolute Deviation (range) | Recommended/Rejected | ||
|---|---|---|---|---|---|
| General Description (level of evidence) | Validity | Feasibility | Validity | Feasibility | |
| Process Indicators | |||||
| Minimum of annual return visit with cardiologist with ACHD expertise for patients with COA repair followed by ACHD expert. (IC) | 5.0,1.6 (3–9) | 9.0,.8 (7–9) | 9.0,.4 (7–9) | 9.0,.2 (8–9) | Recommended |
| Annual check of bilateral upper extremity blood pressures (BP) and either lower extremity BP (IC) | 8.0,.8 (5–9) | 8.0, .9 (5–9) | 8.0, .6 (7–9) | 8.0, .7 (6–9) | Recommended |
| Right upper extremity blood pressure measured at each visit (IB(1), IC(2)) | 8.0, .6 (7–9) | 9.0, .1 (8–9) | 9.0, .6 (7–9) | 9.0, .0 (9-9) | Recommended |
| Any transthoracic echocardiogram includes proximal descending aortic Doppler gradient (IB(1), IC(2)) | 7.0, 1.3 (2–9) | 8.0, 1.3 (5–9) | 7.0, .9 (5–9) | 9.0, .6 (6–9) | Recommended |
| MRI or CT evaluation of complete thoracic aorta at least every 5 years for patients with repaired COA (IC) | 8.0, 1.1 (5–9) | 9.0, .3 (8–9) | 9.0, .7 (7–9) | 9.0, .2 (8–9) | Recommended |
| Documentation of aortic valve morphology (IC) | 9.0, .4 (7–9) | 9.0, 1.2 (4–9) | 9.0, .6 (7–9) | 9.0, .2 (7–9) | Recommended |
| Measurement of the ascending aorta by echocardiogram, CT or MRI at least every 2 years for those with bicuspid aortic valve or enlarged ascending aorta (IB(1), IC(2)) | 7.0, 1.2 (3–9) | 8.0, .9 (6–9) | 8.0, .6 (7–9) | 9.0, .6 (7–9) | Recommended |
| Minimum of annual visit with cardiologist and at least every 3 year referral to a cardiologist with ACHD expertise for patients with repaired COA not followed by ACHD expert (IC) | 6.0, 1.7 (3–7) | 7.0, 1.1 (5–8) | 9.0, .4 (7–9) | 9.0, .4 (7–9) | Recommended |
| Structure Indicators | |||||
| Catheter-based intervention on the COA site should be performed by a CHD specialist with cardiac catheterization expertise (IC) | 9.0, 1.4 (3–9) | 9.0, 1.3 (3–9) | 9.0, .4 (7–9) | 9.0, .6 (6–9) | Recommended |
| Eisenmenger Syndrome (ES) | Score Round 1 Median, Mean Absolute Deviation (range) | Score Round 2 Median, Mean Absolute Deviation (range) | Recommended/Rejected | ||
|---|---|---|---|---|---|
| General Description (level of evidence) | Validity | Feasibility | Validity | Feasibility | |
| Process Indicators | |||||
| Minimum of annual return visit with cardiologist with ACHD expertise for patients with ES followed by ACHD expert (IC) | 8.0, .9 (6–9) | 8.0, .7 (7–9) | 8.0, .7 (7–9) | 9.0, .2 (8–9) | Recommended |
| Annual measurement of pulse oximetry (IC) | 8.0, .7 (7–9) | 9.0, .0 (9-9) | 9.0, .4 (7–9) | 9.0, .0 (9-9) | Recommended |
| Annual hemoglobin or hematocrit (IC) | 9.0, 1.1 (4–9) | 9.0, .0 (9-9) | 9.0, .6 (6–9) | 9.0, .0 (9-9) | Recommended |
| Annual iron panel (Iron, transferring saturation, total binding capacity, ferritin) (IC) | 7.0, 1.7 (3–9) | 9.0, .3 (7–9) | 7.0, 1.2 (5–9) | 9.0, .2 (7–9) | Recommended |
| Discussion about use of pulmonary vasodilators in ES patients WHO functional class III or worse at least every 24 months (IIaC) | 8.0, .9 (5–9) | 9.0, .8 (6–9) | 8.0, .9 (6–9) | 8.0, 1.2 (4–9) | Recommended |
| Annual creatinine as measure of renal function (IC) | 8.0, 1.1 (3–9) | 9.0, .1 (8–9) | 9.0, .6 (7–9) | 9.0, .2 (8–9) | Recommended |
| Annual assessment of functional capacity by either cardiopulmonary exercise test or 6 minute walk test (IC) | 6.0, 1.2 (3–9) | 9.0, .6 (6–9) | 8.0, .9 (6–9) | 9.0, .2 (8–9) | Recommended |
| Recommendation to avoid pregnancy addressed annually (IB) | 8.0, 1.1 (5–9) | 8.0, 1.1 (6–9) | 8.0, .8 (7–9) | 8.0, .7 (7–9) | Recommended |
| Annual recommendation for use of SBE prophylaxis (IB) | 9.0, .7 (6–9) | 9.0, .4 (6–9) | 9.0, .6 (6–9) | 9.0, .7 (6–9) | Recommended |
| Annual recommendation for influenza vaccine (IC) | 9.0, .7 (5–9) | 9.0, .7 (5–9) | 9.0, .5 (7–9) | 9.0, .3 (7–9) | Recommended |
| No phlebotomy in ES patients unless symptoms of hyperviscosity and erythrocytosis without anemia or iron deficiency (IC) | 8.0, .7 (5–9) | 7.0, 1.4 (4–9) | 8.0, .8 (6–9) | 7.0, 1.0 (6–9) | Recommended |
| Minimum of yearly scheduled visit or referral to cardiologist with ACHD expertise for patients with Eisenmenger Syndrome (ES) not followed by ACHD expert (IC) | 7.0, 1.1 (3–9) | 7.0, 1.1 (5–9) | 8.0, .8 (6–9) | 8.0, .7 (7–9) | Recommended |
| Annual measurement of uric acid (IC) | 6.0, 1.6 (3–9) | 9.0, .2 (8–9) | 5.0, 2.1 (1–9) | 9.0, 1.2 (2–9) | Rejected |
| Fontan (FON) | Score Round 1 Median, Mean Absolute Deviation (range) | Score Round 2 Median, Mean Absolute Deviation (range) | Recommended/Rejected | ||
|---|---|---|---|---|---|
| General Description (level of evidence) | Validity | Feasibility | Validity | Feasibility | |
| Process Indicator | |||||
| Minimum of yearly scheduled return visit with a cardiologist with ACHD expertise for patients with Fontan surgery followed by ACHD expert (IC) | 9.0, .7 (7–9) | 9.0, .3 (7–9) | 9.0, .7 (7–9) | 9.0, .1 (8–9) | Recommended |
| Oxygen saturation at rest at least annually (IIaC) | 8.0, 1.1 (6–9) | 9.0, .1 (8–9) | 9.0, .4 (7–9) | 9.0, .2 (7–9) | Recommended |
| Anticoagulation if atrial shunt, atrial thrombus, or atrial arrhythmia (IC) | 8.0, 1.4 (3–9) | 9.0, .8 (4–9) | 9.0, .3 (8–9) | 9.0, .5 (8–9) | Recommended |
| Pregnancy or contraception counseling by cardiologist with ACHD expertise annually (IC) | 7.0, 1.4 (3–9) | 7.0, 1.4 (5–9) | 9.0, .7 (7–9) | 9.0, .8 (7–9) | Recommended |
| Annual measurement of liver function (IIaC) | 7.0, .9 (5–9) | 9.0, .2 (7–9) | 8.0, .7 (7–9) | 9.0, .0 (9-9) | Recommended |
| Documentation of hepatitis C status (not graded)* | 9.0, 1.8 (2–9) | 8.0, 1.1 (4–9) | 9.0, .9 (3–9) | 9.0, .1 (8–9) | Recommended |
| Yearly visit with or referral to a cardiologist with ACHD expertise for patients with Fontan surgery not followed by ACHD expert (IC) | 8.0, .9 (5–9) | 7.0, 1.2 (4–9) | 9.0, .3 (7–9) | 9.0, .9 (6–9) | Recommended |
| Fontan patients with onset of atrial arrhythmia should have TEE or MRI to evaluate for thrombus (IC) | 5.0, 1.9 (2–9) | 8.0, .7 (7–9) | 3.0, 2.0 (1–8) | 8.0, 1.0 (3–9) | Rejected |
| Structure Indicator | |||||
| Annual and comprehensive echocardiogram (MRI or CT may substitute) read by cardiologist with CHD imaging expertise (IC) | 7.0, 1.2 (5–9) | 9.0, .6 (6–9) | 8.0, .8 (6–9) | 9.0, .2 (8–9) | Recommended |
| Diagnostic or interventional cardiac catheterization performed by CHD specialist with cardiac catheterization expertise (IC) | 9.0, .4 (7–9) | 8.0, 1.1 (6–9) | 9.0, .2 (8–9) | 9.0, .6 (7–9) | Recommended |
| D-Transposition of the Great Arteries (TGA) | Score Round 1 Median, Mean Absolute Deviation (range) | Score Round 2 Median, Mean Absolute Deviation (range) | Recommended/Rejected | ||
|---|---|---|---|---|---|
| General Description (level of evidence) | Validity | Feasibility | Validity | Feasibility | |
| Process Indicator | |||||
| Minimum of yearly scheduled return visit with cardiologist who has ACHD expertise for patients with TGA and atrial baffle repair followed by ACHD expert (IC) | 7.0, .6 (7–9) | 9.0, .3 (7–9) | 8.0, .7 (7–9) | 9.0, .0 (9-9) | Recommended |
| At least annual electrocardiogram (IB) | 9.0, 1.1 (5–9) | 9.0, .0 (9-9) | 9.0, .8 (7–9) | 9.0, .0 (9-9) | Recommended |
| Pacemaker for symptomatic bradyarrhythmia or sick sinus syndrome (IB) | 9.0, 1.3 (5–9) | 8.0, 1.0 (5–9) | 9.0, 1.0 (5–9) | 9.0, 1.1 (5–9) | Recommended |
| Pregnancy or contraception counseling by cardiologist with ACHD expertise annually (IC) | 7.0, 1.2 (3–9) | 8.0, 1.3 (5–9) | 9.0, .9 (7–9) | 8.0, .8 (6–9) | Recommended |
| Appropriate counseling regarding SBE prophylaxis (IIaB) | 9.0, 1.3 (4–9) | 9.0, 1.0 (5–9) | 9.0, 1.4 (4–9) | 9.0, .8 (7–9) | Recommended |
| Minimum of yearly scheduled visit with or referral to cardiologist with ACHD expertise for patients with TGA and atrial baffle repair not followed by ACHD expert (IC) | 7.0, 1.3 (5–9) | 7.0, 1.0 (5–9) | 9.0, .6 (7–9) | 7.0, .7 (6–9) | Recommended |
| Valve repair or replacement for moderate to severe systemic atrioventricular valve regurgitation with normal ejection fraction (IC) | 6.0, .6 (4–7) | 6.0, 1.2 (3–8) | 3.0, 1.2 (1–6) | 6.0, 1.4 (3–8) | Rejected |
| Structure Indicator | |||||
| Echocardiographic imaging in TGA with atrial baffle repair should be performed by CHD specialist with imaging expertise (IB) | 9.0, .9 (6–9) | 9.0, 1.2 (5–9) | 9.0, .6 (7–9) | 9.0, .7 (7–9) | Recommended |
| Diagnostic cardiac catheterization should be performed by CHD specialist with cardiac catheterization expertise (IC) | 9.0, .9 (6–9) | 8.0, 1.1 (5–9) | 9.0, .4 (7–9) | 9.0, .4 (7–9) | Recommended |
| Interventional cardiac catheterization in TGA with atrial baffle repair should be performed by CHD specialist with cardiac catheterization expertise (IC) | 8.0, 1.1 (5–9) | 8.0, 1.2 (5–9) | 9.0, .4 (7–9) | 9.0, .4 (7–9) | Recommended |
| Tetralogy of Fallot (TOF) | Score Round 1 Median, Mean Absolute Deviation (range) | Score Round 2 Median, Mean Absolute Deviation (range) | Recommended/Rejected | ||
|---|---|---|---|---|---|
| General Description (level of evidence) | Validity | Feasibility | Validity | Feasibility | |
| Process Indicator | |||||
| Annual return visit with cardiologist with ACHD expertise for patients with TOF repair followed by ACHD expert (IC) | 7.0, 1.2 (3–9) | 9.0, .3 (8–9) | 8.0, .3 (7–9) | 9.0, .1 (8–9) | Recommended |
| At least annual surveillance electrocardiogram (ECG) in patients with TOF repair (IC) | 8.0, 1.1 (5–9) | 9.0, .2 (8–9) | 9.0, .6 (7–9) | 9.0, .0 (9-9) | Recommended |
| Surveillance of width of QRS complex on 12-lead ECG in patients with TOF repair (IB) | 8.0, 1.0 (6–9) | 9.0, 1.1 (6–9) | 8.0, .7 (7–9) | 8.0, .9 (6–9) | Recommended |
| Appropriate endocarditis prophylaxis counseling in patients with repaired TOF (IIaB) | 7.0, 1.1 (5–9) | 9.0, .9 (6–9) | 9.0, .3 (7–9) | 9.0, .6 (7–9) | Recommended |
| Minimum of yearly scheduled visit with or referral to cardiologist with ACHD expertise for patients with repaired TOF and not followed by ACHD expert (IC) | 7.0, .9 (5–9) | 7.0, .8 (5–9) | 8.0, .6 (7–9) | 9.0, .7 (7–9) | Recommended |
| Internal cardiac defibrillator implanted in patients with documented sustained and/or resuscitated ventricular arrhythmia (IC) | 9.0, .6 (5–9) | 9.0, 1.3 (5–9) | 5.0, 2.3 (1–9) | 5.0, 2.3 (1–9) | Rejected |
| Transthoracic echocardiography at least every 12 months in patients with TOF repair (IB(1), IC(2)) | 7.0, 1.0 (3–9) | 9.0, .2 (8–9) | 5.0, 4.0 (1–9) | 5.0, 4.0 (1–9) | Rejected |
| Structure Indicator | |||||
| MRI assessment of right ventricular ejection fraction in adults with TOF repair at least every 5 years and interpreted by CHD specialist with imaging expertise (IC) | 7.0, 1.1 (3–8) | 7.0, 1.0 (6–9) | 8.0, .7 (7–9) | 9.0, .3 (7–9) | Recommended |
| Echocardiographic assessment of adults with TOF repair interpreted by CHD specialist with imaging expertise (IB(1), IC(2)) | 8.0, 1.1 (6–9) | 8.0, 1.0 (6–9) | 9.0, .4 (7–9) | 9.0, .4 (7–9) | Recommended |
| Diagnostic or interventional cardiac catheterization in TOF patients performed by CHD specialist with cardiac catheterization expertise (IC) | 7.0, 1.4 (3–9) | 9.0, 1.1 (6–9) | 9.0, .6 (7–9) | 9.0, .6 (7–9) | Recommended |
This indicator was recommended by the working group but is not covered in any of the sets of available guidelines
1. Warnes CA, Williams RG, Bashore TM et al. ACC/AHA 2008 guidelines for the management of adults with congenital heart disease: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines (Writing Committee to Develop Guidelines on the Management of Adults With Congenital Heart Disease). Developed in Collaboration With the American Society of Echocardiography, Heart Rhythm Society, International Society for Adult Congenital Heart Disease, Society for Cardiovascular Angiography and Interventions, and Society of Thoracic Surgeons. Journal of the American College of Cardiology 2008;52:e1–121.
2. Silversides CK, Kiess M, Beauchesne L et al. Canadian Cardiovascular Society 2009 Consensus Conference on the management of adults with congenital heart disease: outflow tract obstruction, coarctation of the aorta, tetralogy of Fallot, Ebstein anomaly and Marfan’s syndrome. Canadian Journal of Cardiology 2010;26:e80–97.
The expert panel completed two rounds of rating the quality indicators. All indicators were individually graded by each panelist for validity and feasibility on an ordinal scale of 1–9. A high validity score meant the indicator was clinically relevant, supported by available scientific evidence or expert consensus, and that adherence to it would be considered delivery of high quality care. A high validity score also indicated that a large proportion of the determinants of adherence to the QI were under the provider’s or clinic’s control and compliance with the indicator would confer health benefits. A high feasibility score meant that the data needed to determine eligibility and adherence to the indicator would be readily available in the average medical record (or should be available), the data would likely be unbiased, and estimates of performance based on medical records data are likely to be reliable.(13)
The first set of ratings were performed and returned to the primary investigators (AM/MG) by email. Median feasibility and validity scores and mean absolute deviations from the median were calculated for each indicator. The median was used to measure the central tendency of the panelists’ ratings while the mean absolute deviation from the median was used to measure the dispersion of ratings.(14) Indicators with high median ratings [(7–9) for validity and (4–9) for feasibility] and rated with agreement (or minimal dispersion of scores) according to the mean absolute deviation from the median score were considered ‘accepted’ while those with low ratings (1–3 for validity and/or feasibility) were rejected. Indicators with median ratings between these two ranges (validity median 4–6) and/or scored by panelists with disagreement or an indeterminate level of agreement based on the mean absolute deviation from the median (wide spread of scores) were listed for further discussion in the next rating round. We contacted each panelist after the first round to discuss their individual results in order to better understand outlier scores, and to address any specific questions or concerns of the panelists.
The second round of ratings by the experts occurred at the in-person Delphi panel meeting in May 2011 in Boston, MA. The participants discussed the QIs identified for further consideration after the first round of scoring. The panelists proposed minor additions, deletions or modifications to the set of indicators, particularly those indicators with low-moderate scores or those with significant disagreement among the panelists as assessed by the mean absolute deviation from the median score. Following the discussions, each panelist individually re-scored the QIs for validity and feasibility on the 9-point Delphi scale. These results were scored and analyzed in the same way as the first round of scoring. Those indicators with a median score of 7–9 on validity and 4–9 on feasibility and scored without significant disagreement were included in the final set of quality indicators, the others were discarded.
Expert Definitions
Refining the quality indicators required the development of definitions for the term “expert” in relation to multiple aspects of CHD. In the ACHD guidelines, many of the recommendations for care include the need for an ACHD expert or for an expert in clinical care, catheterization, or imaging. The guidelines, however, do not define the cited expertise. We therefore developed pertinent definitions for the purposes of this project. The ACHD expert definition was developed with the knowledge that a board certification for ACHD was being considered so this definition would need to be able to be revised in the future. The American Board of Medical Specialties announced the creation of a subspecialty certification in ACHD in December 2012, and the exam is expected to be available by 2015.
We based the expert definitions on two sources. First, we considered published guidelines including those for adult and pediatric cardiovascular training (15–17), and the Bethesda conference recommendations on what constitutes an ACHD regional center.(18) Second, we consulted the American Congenital Heart Association clinic directory, which includes self-reported data from self-identified ACHD centers. To determine what elements most commonly comprise the profile of self-reported ACHD experts we looked at the distribution of three variables extracted from the ACHA website: the number of years in ACHD practice; the percent time spent in ACHD practice and the total number of patient visits per center per year. For each of these variables we obtained a histogram of distribution based on available data as well as minimum, maximum, median and interquartile range data. These data and suggestions for expert definitions were presented to the expert panel for input, revision and adjustment. The final definitions were divided into clinical and technical expertise domains and are presented in Table 3.
Table 3.
Definitions used to qualify adult congenital heart disease (ACHD) and congenital heart disease (CHD) expertise.
| A cardiologist with ACHD expertise | A cardiologist who has received level 2 or level 3 ACHD training as per published definitions OR for those who have not received formal training A cardiologist who spends at least 20% over 9 years or 50% over 5 years of his/her time in ACHD practice. This definition uses the lowest interquartile range of the practice patterns extracted from Adult Congenital Heart Association. |
| CHD specialist with imaging or cardiac catheterization expertise |
|
Results
Quality Indicators and Results of Panel Ratings
The working groups and primary investigators proposed a total of 62 QIs for consideration. After the first round of scoring, none were rejected, 29 scored high enough on validity and feasibility to be accepted, and 33 warranted further discussions or modification. After all controversial indicators were discussed for a given condition; panelists rescored the indicators for that condition. This second round of scoring resulted in 55 (89%) indicators being retained including 8 for atrial septal defects, 9 for aortic coarctation, 12 for Eisenmenger, 9 for Fontan, 9 for D-transposition of the great arteries, and 8 for tetralogy of Fallot. All of the 55 retained QIs are recommended to assess quality of care. Six QIs were removed for low validity and one QI was removed due to substantial dispersion of ratings and thus disagreement among the panelists (Table 2).
One example of a recommended QI would be an annual visit with an ACHD cardiologist for patients with Fontan procedure. For patients in specialty ACHD clinics, this would include an annual visit. However, for patients followed in non-ACHD cardiology clinics, this would include the need for an annual referral to an ACHD expert. This QI passed, but other QIs were rejected. Rejection of a QI does not mean it is not considered to be important; it may mean that the evidence base was not as strong or that the data was not feasible to collect in a reasonable and accurate manner. For example, for patients with ASD, “cardioversion to attempt restoration of sinus rhythm if atrial fibrillation” was rejected after the second round of voting. As can be seen in Table 2, panel 1, in Round 2 the median validity score decreased from 6.0 to 3.0 and the median feasibility score decreased from 7.0 to 5.0 with an increase in deviation to 2.3. This should not be taken to mean that the panel did not perceive the restoration of sinus rhythm to be important in the management of atrial fibrillation in patients with ASD. It means that a significant number of panelists did not feel the evidence strongly supported which groups would benefit most and how this data would be reliably collected or specifically measured.
Type of Indicators and Supportive Evidence
The proposed indicators were either related to healthcare process (85%) or structure (15%). No pure outcome indicators were proposed or included. In total, the QIs addressed the broad categories of clinical testing (45%), appointments (20%), procedures (15%) and counseling (18%). The evidence level supporting each indicator was either determined by the published guidelines or assigned by two of the authors (AM/MG) based on available literature and evidence criteria.(11) The evidence supporting the measures was predominantly level IC (41/62) with two measures graded IA, 12 measures rating IB, 3 rating IIaB, 4 rating IIaC, and 1 rating IIIB. The total is greater than 62 as the evidence supporting some of the indicators was graded differently in different guideline documents or literature. Table 2 shows corresponding levels of evidence and metric type for QIs that were retained or rejected. Indicators common to multiple lesions are shown in Table 4.
Table 4.
Quality indicators common to several of the congenital heart disease (CHD) lesions
| Quality Indicators | CHD Lesions |
|---|---|
| Annual visit with adult CHD specialist | ASD closure (if pulmonary hypertension, arrhythmia, valve disease), COA (repaired), ES, Fontan, TGA, TOF |
| Oxygen saturation annually | ES, Fontan |
| Catheterization performed by CHD catheterization specialist | COA (interventional only), Fontan, TGA, TOF |
| Appropriate SBE prophylaxis counseling | ASD, ES, TGA, TOF |
| Pregnancy/contraception discussion annually | ES, Fontan, TGA |
| Annual 12 lead electrocardiogram | TGA, TOF |
| Transthoracic echocardiogram interpreted by cardiologist with CHD imaging expertise | Fontan, TGA, TOF |
ASD, atrial septal defect; COA, coarctation of the aorta; ES, Eisenmenger syndrome; TGA, d-loop transposition of the great arteries following atrial baffle repair; TOF, tetralogy of Fallot
Discussion
To our knowledge this is the first effort to develop a valid and feasible set of QIs to assess the quality of outpatient ACHD care. Although the ACHD population is growing and guidelines for care exist, prior to this effort, there were no established measures to assess quality of care using these guideline standards. (19) We demonstrated that published literature and guideline documents can be utilized to inform the development of candidate QIs for ACHD care. We engaged international experts applying the RAND-UCLA modified Delphi method to select a final set of QIs. This quality measure development effort constitutes an important step in facilitating the assessment and improvement of quality care for ACHD patients.
The National Quality Forum has recognized the need to improve health care in America leveraging input from multiple stakeholders.(20) Quality measurement provides the basis for understanding where improvements are most needed and for testing the effectiveness of quality improvement interventions. They further provide the basis for quality reporting.(21) Adults with CHD are one of the fastest growing groups in cardiology.(19) They are also one of a growing number of adult patient groups with chronic life-long diseases of childhood.(22) In spite of this, efforts aimed at quality measurement are scant.
Expert consensus guidelines are written to standardize care based on best available evidence. However, guidelines are applied prospectively and intended to be broad in scope and allow margins in applicability based on the supporting grade of evidence and the sub-groups of patients to which they are applied. Guidelines are flexible in order to accommodate clinical judgment and the decision to follow a guideline remains with the individual physician in conjunction with his/her patient.
In this effort, we took the next essential step to decreasing practice variation by developing valid and feasible quality indicators for the 6 stated conditions informed by clinical guidelines. In contrast to guidelines, quality indicators, provide specific measures that are applied retrospectively to a group of patients to assess the processes of care received or not received by those patients.(3) Quality indicators should be sufficiently supported by scientific evidence and/or expert consensus such that failure to implement them would be considered incongruous with the standards of care for a particular condition. Thus QIs provide metrics to evaluate and measure the quality of care provided and are often used as a basis for quality improvement initiatives. In interpreting the findings of this project, it is recognized that different QIs might have been recommended or rejected for particular conditions. The fact that one QI is retained or considered for some lesions but not others does not indicate that the retained QI may not be generalizable. For example, an annual 12 lead EKG is recommended for TOF and TGA but not other conditions. In the other lesions, although an EKG may be important, other QIs were chosen by the working groups or were more highly rated. QI consideration and recommendation is therefore a composite expression of what experts perceived to be most feasible and valid for specific lesion categories.
We found that for ACHD care, as with many other conditions such as coronary artery disease, the QIs centering on the structural and process dimensions of quality were favored over those representing the outcome dimension. It is not known if structure and process measures selected will correlate with outcome but we anticipate that outcome measures will be developed in the future as more consistent clinical data becomes available. At the current time, structure and process measures of quality would constitute important steps in shaping quality of care for ACHD patients.(23)
Quality improvement initiatives are increasingly recognized in cardiology as well as in other chronic disease models. QIs have been developed for the management of acute myocardial infarction and percutaneous coronary interventions using national expert panels and a two-step modified Delphi process similar to the one used in this project.(24,25) In both of these efforts, the majority of measures selected were related to structure and process rather than outcome. This is so in spite of the fact that there are significantly more studies evaluating medical and interventional outcomes for coronary disease than for congenital heart disease.
In pediatric cardiology, there are limited large randomized trials or long term outcome studies on which to base quality improvement initiatives. However, moving forward, efforts are centering on outcomes evaluation, particularly for surgical and interventional procedures. There has been pilot work measuring surgical technical performance in pediatric congenital heart surgery.(26) The components of the technical performance scores were developed for surgeries on four congenital conditions and were agreed upon by expert consensus opinion. For interventional cardiac catheterization, methods have been developed to risk adjust for case mix complexity thus allowing outcome comparisons between institutions and procedures.(27) A recent national effort by the ACC to develop a catheterization registry and evaluate procedural outcomes has successfully recruited over 50 centers to participate.(28) Outside of procedural outcomes, there have also been efforts aimed at other areas within pediatric cardiology focusing more on processes of care. One such effort is the quality measures working group of the ACC. One goal of the group is to develop quality metrics across eight clinical areas and then to potentially combine the metrics into an online ‘scorecard’ for the interested CHD community for internal quality improvement.(29)
Other chronic diseases have made significant progress in improving quality of care. The cystic fibrosis model as determined by the Cystic Fibrosis Foundation incorporates serial establishment of a patient registry, establishment of a research development program, and a quality improvement initiative that have been used for program accreditation. This has led to the development of a culture of group data collection and transparency when it comes to performance benchmarking.(30) Several efforts are underway in ACHD care that may enable electronic deployment of the ACHD metrics developed in this effort to improve knowledge translation, decision support and tracking for reporting purposes. We expect that consistently and comprehensively collected data can be organized between providers and practices facilitating local and national quality improvement efforts.
Limitations
The quality measure development process used here has inherent limitations. First, we relied on literature and practice guidelines that are mostly level of evidence C in the QI development. We also relied on expert panel opinions. A different panel of experts might have selected a different set of QIs. While this is a known and reasonable part of the RAND/UCLA method, it does impose some uncertainty in the QIs. The ACHD experts serving on the panel were nominated from multiple national and international congenital cardiology organizations. We sought to obtain a wide representation of expert practicing cardiologists caring for ACHD patients. The fact that no ACHD electrophysiologist or transplant specialist was included in our panel does not indicate that the importance of those subspecialties was not recognized or that QIs regarding those topics are not important or valid. It only indicates that, at the time of this panel, experts representing those specialized subgroups were not included among the nominees. Future panels may have different compositions of experts.
Some of the QIs will also be limited by the definition of ACHD experts. As board certification for adult congenital heart disease was only recently approved by the American Board of Medical Specialties, there were no established references or “gold standard” criteria to define the relevant expertise. The proposed “expert” working definitions are designed to change as certification testing becomes available and policies change in the future. In this first iteration of ACHD QIs, we sought to cover processes of care where the greatest amount of agreement exists at the current time based on the existing published data. Congenital heart disease is rapidly evolving and our populations are changing not only in demographics but also in terms of anatomical and surgical substrates. Like guideline documents, it is planned that these QIs will be updated on a regular basis, not only as more data becomes available but also for specific types and subgroups of patients. Finally, development of detailed measure specifications and field testing are needed to fully operationalize and further validate these QI metrics.
Conclusions
The elaboration of care guidelines for ACHD patients has constituted an important initiative in attempting to create standards of care for this group of patients. They underscore the need to achieve the highest possible quality of care. Recognizing the difference between guidelines and quality indicators, we leveraged published literature and guidelines to develop the first set of QIs for ACHD care. We learned that in our field at this point in time, both the literature and our expert panel gravitated towards indicators related to process and structure of care rather than outcomes. This will allow refinement of quality of care moving forward. The development of these quality indicators constitutes a pivotal point that will allow us to begin to measure quality of care being delivered. In future studies, this first iteration of a quality assessment tool will be tested, updated, refined and expanded as more data become available. With this project we have advanced the process of improving the quality of care for the growing group of ACHD patients.
Abbreviations
- QI
quality indicator
- ACHD
adult congenital heart disease
- TOF
tetralogy of Fallot
- ASD
atrial septal defect
- COA
coarctation of the aorta
- TGA
d-transposition of the great arteries
- RCT
randomized clinical trials
- ACC
American College of Cardiology
- ACHA
Adult Congenital Heart Association
Footnotes
Panel Co-Chairs and Members
Michelle Gurvitz MD, MS (Panel Co-Chair), Ariane Marelli, MD, MPH (Panel Co-Chair), Curt Daniels MD, William Davidson MD, Elyse Foster MD, Michael Landzberg MD, Daniel Murphy MD, Erwin Oechslin MD, Judith Therrien MD, Gary Webb MD, Roberta Williams MD
Working Group Members
Jamil Aboulhosn MD, Craig Broberg MD, Stephen Cook MD, Michael Earing MD, Joseph Kay MD, Paul Khairy MD, PhD, Karen Kuehl MD MPH, Andrew Mackie MD, MS, Alexander Opotowsky MD, MPH, Jo Ann Nieves, MSN, Disty Pearson PA-C, Candice Silversides MD, MS, Anne Marie Valente MD
Disclosure: This project was supported by Award Number 7K23HL095908-02 from the National Heart, Lung and Blood Institute. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Heart, Lung and Blood Institute or the National Institutes of Health. Dr. Marelli is funded by the Heart and Stroke Foundation of Canada and the Canadian Institute for Health Research
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
- 1.Blumenthal D. Part 1: Quality of care--what is it? N Engl J Med. 1996;335:891–4. doi: 10.1056/NEJM199609193351213. [DOI] [PubMed] [Google Scholar]
- 2.Donabedian A. Evaluating the quality of medical care. Milbank Mem Fund Q. 1966;44(Suppl):166–206. [PubMed] [Google Scholar]
- 3.Spertus JA, Eagle KA, Krumholz HM, Mitchell KR, Normand SL. American College of Cardiology and American Heart Association methodology for the selection and creation of performance measures for quantifying the quality of cardiovascular care. Circulation. 2005;111:1703–12. doi: 10.1161/01.CIR.0000157096.95223.D7. [DOI] [PubMed] [Google Scholar]
- 4.Connelly MS, Webb GD, Somerville J, et al. Canadian Consensus Conference on Adult Congenital Heart Disease 1996. Can J Cardiol. 1998;14:395–452. [PubMed] [Google Scholar]
- 5.Baumgartner H, Bonhoeffer P, De Groot NM, et al. ESC Guidelines for the management of grown-up congenital heart disease (new version 2010) European Heart Journal. 2010;31:2915–57. doi: 10.1093/eurheartj/ehq249. [DOI] [PubMed] [Google Scholar]
- 6.Silversides CK, Dore A, Poirier N, et al. Canadian Cardiovascular Society 2009 Consensus Conference on the management of adults with congenital heart disease: shunt lesions. Canadian Journal of Cardiology. 2010;26:e70–9. doi: 10.1016/s0828-282x(10)70354-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Silversides CK, Kiess M, Beauchesne L, et al. Canadian Cardiovascular Society 2009 Consensus Conference on the management of adults with congenital heart disease: outflow tract obstruction, coarctation of the aorta, tetralogy of Fallot, Ebstein anomaly and Marfan’s syndrome. Canadian Journal of Cardiology. 2010;26:e80–97. doi: 10.1016/s0828-282x(10)70355-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Silversides CK, Marelli A, Beauchesne L, et al. Canadian Cardiovascular Society 2009 Consensus Conference on the management of adults with congenital heart disease: executive summary. Canadian Journal of Cardiology. 2010;26:143–50. doi: 10.1016/s0828-282x(10)70352-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Silversides CK, Salehian O, Oechslin E, et al. Canadian Cardiovascular Society 2009 Consensus Conference on the management of adults with congenital heart disease: complex congenital cardiac lesions. Canadian Journal of Cardiology. 2010;26:e98–117. doi: 10.1016/s0828-282x(10)70356-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Warnes CA, Williams RG, Bashore TM, et al. ACC/AHA 2008 guidelines for the management of adults with congenital heart disease: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines (Writing Committee to Develop Guidelines on the Management of Adults With Congenital Heart Disease). Developed in Collaboration With the American Society of Echocardiography, Heart Rhythm Society, International Society for Adult Congenital Heart Disease, Society for Cardiovascular Angiography and Interventions, and Society of Thoracic Surgeons. Journal of the American College of Cardiology. 2008;52:e1–121. doi: 10.1016/j.jacc.2008.10.001. [DOI] [PubMed] [Google Scholar]
- 11.Gibbons RJ, Smith S, Antman E. American College of Cardiology/American Heart Association clinical practice guidelines: Part I: where do they come from? Circulation. 2003;107:2979–86. doi: 10.1161/01.CIR.0000063682.20730.A5. [DOI] [PubMed] [Google Scholar]
- 12.Brook RH, McGlynn EA, Cleary PD. Quality of health care. Part 2: measuring quality of care. N Engl J Med. 1996;335:966–70. doi: 10.1056/NEJM199609263351311. [DOI] [PubMed] [Google Scholar]
- 13.Brook RH. The RAND/UCLA appropriateness method. In: McCormack KA, MS, Siegel RA, editors. Clinical Practice Guideline Development: Methodology Perspectives. Rockville, MD: Agency for Healthcare Research and Policy; 1994. [Google Scholar]
- 14.Mangione-Smith R, DeCristofaro AH, Setodji CM, et al. The quality of ambulatory care delivered to children in the United States. N Engl J Med. 2007;357:1515–23. doi: 10.1056/NEJMsa064637. [DOI] [PubMed] [Google Scholar]
- 15.Beller GA, Bonow RO, Fuster V. ACC revised recommendations for training in adult cardiovascular medicine. Core Cardiology Training II (COCATS 2) (Revision of the 1995 COCATS training statement) J Am Coll Cardiol. 2002;39:1242–6. doi: 10.1016/s0735-1097(02)01795-3. [DOI] [PubMed] [Google Scholar]
- 16.Graham TP, Jr, Beekman RH, 3rd, Allen HD, et al. ACCF/AHA/AAP recommendations for training in pediatric cardiology. A report of the American College of Cardiology Foundation/American Heart Association/American College of Physicians Task Force on Clinical Competence (ACC/AHA/AAP Writing Committee to Develop Training Recommendations for Pediatric Cardiology) Circulation. 2005;112:2555–80. doi: 10.1161/CIRCULATIONAHA.105.170308. [DOI] [PubMed] [Google Scholar]
- 17.Child JS, Freed MD, Mavroudis C, Moodie DS, Tucker AL. Task force 9: training in the care of adult patients with congenital heart disease. J Am Coll Cardiol. 2008;51:389–93. doi: 10.1016/j.jacc.2007.11.017. [DOI] [PubMed] [Google Scholar]
- 18.Landzberg MJ, Murphy DJ, Jr, Davidson WR, Jr, et al. Task force 4: organization of delivery systems for adults with congenital heart disease. J Am Coll Cardiol. 2001;37:1187–93. doi: 10.1016/s0735-1097(01)01275-x. [DOI] [PubMed] [Google Scholar]
- 19.Marelli AJ, Mackie AS, Ionescu-Ittu R, Rahme E, Pilote L. Congenital heart disease in the general population: changing prevalence and age distribution. Circulation. 2007;115:163–72. doi: 10.1161/CIRCULATIONAHA.106.627224. [DOI] [PubMed] [Google Scholar]
- 20.Kizer KW. The National Quality Forum seeks to improve health care. Acad Med. 2000;75:320–1. doi: 10.1097/00001888-200004000-00005. [DOI] [PubMed] [Google Scholar]
- 21.McGlynn EA. Introduction and overview of the conceptual framework for a national quality measurement and reporting system. Med Care. 2003;41:I1–7. doi: 10.1097/00005650-200301001-00001. [DOI] [PubMed] [Google Scholar]
- 22.Van Cleave J, Gortmaker SL, Perrin JM. Dynamics of obesity and chronic health conditions among children and youth. JAMA. 2010;303:623–30. doi: 10.1001/jama.2010.104. [DOI] [PubMed] [Google Scholar]
- 23.Berwick DM, James B, Coye MJ. Connections between quality measurement and improvement. Med Care. 2003;41:I30–8. doi: 10.1097/00005650-200301001-00004. [DOI] [PubMed] [Google Scholar]
- 24.Tu JV, Khalid L, Donovan LR, Ko DT. Indicators of quality of care for patients with acute myocardial infarction. CMAJ. 2008;179:909–15. doi: 10.1503/cmaj.080749. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Ko DT, Wijeysundera HC, Zhu X, Richards J, Tu JV. Canadian quality indicators for percutaneous coronary interventions. Can J Cardiol. 2008;24:899–903. doi: 10.1016/s0828-282x(08)70696-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Larrazabal LA, del Nido PJ, Jenkins KJ, et al. Measurement of technical performance in congenital heart surgery: a pilot study. Ann Thorac Surg. 2007;83:179–84. doi: 10.1016/j.athoracsur.2006.07.031. [DOI] [PubMed] [Google Scholar]
- 27.Bergersen L, Gauvreau K, Foerster SR, et al. Catheterization for Congenital Heart Disease Adjustment for Risk Method (CHARM) JACC: Cardiovascular Interventions. 2011;4:1037–1046. doi: 10.1016/j.jcin.2011.05.021. [DOI] [PubMed] [Google Scholar]
- 28.IMPACT registry https://www.ncdr.com/webncdr/impact.
- 29.ACPC section. Quality Measures Working Group; http://www.cardiosource.org/ACC/ACC-Membership/Sections-Segments-Councils/Adult-Congenital-and-Pediatric-Cardiology-Member-Section. [Google Scholar]
- 30.Marshall BC, Penland CM, Hazle L, et al. Cystic fibrosis foundation: achieving the mission. Respir Care. 2009;54:788–95. doi: 10.4187/002013209790983223. discussion 795. [DOI] [PubMed] [Google Scholar]