Abstract
Background
The complexity of congenital heart disease has been primarily stratified on the basis of surgical technical difficulty, specific diagnoses, and associated outcomes. We report on the refinement and validation of a pediatric echocardiography complexity (PEC) score.
Methods and Results
The American College of Cardiology Quality Network assembled a panel from 12 centers to refine a previously published PEC score developed in a single institution. The panel refined complexity categories and included study modifiers to account for complexity related to performance of the echocardiogram. Each center submitted data using the PEC scoring tool on 15 consecutive inpatient and outpatient echocardiograms. Univariate and multivariate analyses were performed to assess for independent predictors of longer study duration. Among the 174 echocardiograms analyzed, 68.9% had underlying congenital heart disease; 44.8% were outpatient; 34.5% were performed in an intensive care setting; 61.5% were follow‐up; 46.6% were initial or preoperative; and 9.8% were sedated. All studies had an assigned PEC score. In univariate analysis, longer study duration was associated with several patient and study variables (age <2 years, PEC 4 or 5, initial study, preoperative study, junior or trainee scanner, and need for additional imaging). In multivariable analysis, a higher PEC score of 4 or 5 was independently associated with longer study duration after controlling for study variables and center variation.
Conclusions
The PEC scoring tool is feasible and applicable in a variety of clinical settings and can be used for correlation with diagnostic errors, allocation of resources, and assessment of physician and sonographer effort in performing, interpreting, and training in pediatric echocardiography.
Keywords: complexity score, congenital heart disease, pediatric echocardiography, resource use
Subject Categories: Echocardiography
Nonstandard Abbreviation and Acronym
- PEC
pediatric echocardiography complexity
Clinical Perspective.
What Is New?
A scoring system to classify nonsurgical categorization of complexity in congenital heart disease is important to improve diagnostic accuracy, standardize quality improvement initiatives, and allocate appropriate resources.
We developed a consensus‐based complexity score for pediatric echocardiography and included important anatomic‐, scan‐, and acuity‐related factors. Using study duration as the outcome measure, we noted that high complexity scores were independently associated with longer study duration.
What Are the Clinical Implications?
These findings will have important implications in tracking productivity, monitoring resource use, and improving diagnostic accuracy within and across centers.
Echocardiography is the primary imaging modality used in the diagnosis, management, and long‐term surveillance of congenital heart diseases (CHDs). With advances in imaging platforms and easier access to imaging studies, there has been a substantial increase in the use of echocardiograms in the pediatric population. 1 , 2 To ensure the effective use of echocardiograms in pediatrics and patients with CHD, appropriate use criteria have been developed for standardization of diagnostic testing and for imaging‐based quality improvement initiatives. 1 , 3 In addition, resources needed to perform echocardiograms remain defined only by volume and not by the anatomic complexity of the diagnosis or the environment in which an echocardiogram is performed. 4 The vast anatomic spectrum of CHD notwithstanding, additional study and situational factors add to the complexity of performing an echocardiogram in the pediatric population. 5 , 6
In a prior study, we reported deficiencies in the ability of the relative value unit system to adequately capture the range of effort involved in performing and interpreting pediatric echocardiograms given anatomic complexities and age‐related factors. 4 This study underscored the importance of developing an echocardiography‐specific complexity scoring system that can be applied in the clinical setting. Although current models of stratification of complexity in CHD are primarily based on surgical interventions and outcomes relating to mortality and length of stay, 7 , 8 an imaging‐specific scoring system is likely to pave the way for relevant, trackable metrics, such as standardized comparison productivity measures, resource allocation, and quality improvement initiatives. The overall goal of this study was to develop a validated tool, designated the pediatric echocardiography complexity (PEC) score, that may be widely applicable and could be incorporated in the stratification of pediatric echocardiography–related metrics across institutions.
METHODS
The data that support the findings of this study are available from the corresponding author on reasonable request. The study design comprised 2 parts: the first was a development phase for constructing a complexity scoring tool, and the second comprised validation of the tool in a multicenter, prospective, consecutive cohort of patients, which was subsequently analyzed.
Panel
The American College of Cardiology Quality Network assembled a panel of pediatric cardiologists and sonographers with clinical expertise in noninvasive imaging from 12 centers (please see list of coauthors).
Consensus‐Based Approach to the Development of the Echocardiography Complexity Score
We had previously developed an anatomy‐based cardiac complexity score that was used to evaluate appropriate relative value unit assignment in pediatric echocardiography. 4 Using this footprint, our expert panel convened 4 virtual meetings within a span of 9 months to further refine this complexity score.
Figure 1 shows the final PEC score instrument. By consensus, the panel agreed that the score should be focused on capturing complexity as it pertains to performance and interpretation of an echocardiogram rather than enumerating the spectrum of anatomic variants within groups of lesions. Our overarching goals were to ensure wide applicability of the scoring system and to enable data gathering that is relatively easy, which, in turn, would allow meaningful comparisons among groups of patients within and between institutions. PEC scores 1 to 6 represent increasing anatomic or study complexity. PEC 1 included patients undergoing a limited study for evaluation for function or effusion. PEC 2 and 3 included patients with structurally normal heart, with the latter including patients with acquired heart disease or cardiomyopathy. PEC 4 and 5 represent patients with underlying congenital heart disease, and PEC 6 includes all patients on mechanical cardiac support. With this in mind, the anatomic categories were made free‐text sections with flexibility for the end user to upgrade or downgrade complexity category based on the perceived anatomic complexity of the study performed. As a starting point, guidelines on what studies could be captured in the respective PEC scores were provided to the scorer (Figure 1).
Figure 1. Pediatric echocardiography complexity (PEC) scoring tool developed by consensus using representation from 12 centers and validated with submission of 15 random echocardiograms per center.
ASD indicates atrial septal defect; AV, atrioventricular; CICU, cardiac intensive care unit; DORV, double‐outlet right ventricle; ECMO, extracorporeal membrane oxygenation; ICU, intensive care unit; MAPCA, major aortopulmonary collateral artery; NICU, neonatal intensive care unit; OR, operating room; PA, pulmonary atresia; PICU, pediatric intensive care unit; TGA, transposition of the great arteries; TOF, tetralogy of Fallot; TTE, transthoracic echocardiogram; VAD, ventricular assist device; and VSD, ventricular septal defect.
Next, the panel discussed and identified study modifiers that would add to the complexity or effort of performing the echocardiogram as measured by study duration, level of competence (cognitive and technical), location and environment, poor acoustic windows, need for sedation, need for complete assessment (initial or preoperative study versus follow‐up study), sonographer experience, involvement of a trainee, and the need for advanced imaging or postprocessing.
Subsequently, the panel concluded that study duration was the best available outcome measure to validate the PEC score. Study duration was documented using the time stamp of the first and last clip of each study.
PEC Score Validation
Next, the PEC scoring tool was validated, prospectively, on a consecutive sample of patients by each of the panel members at their respective tertiary pediatric cardiology echocardiography laboratories. The participating member of the panel identified 15 consecutive transthoracic echocardiograms that were performed for clinical indications. A PEC score was assigned by the participating center using the guidelines provided for score assignment with free‐text option for alternate categorization. All patient identifiers were removed from data submitted by the participating centers. In addition, data are reported as aggregate for the entire cohort. The Institutional Review Board at the University of Michigan granted waiver for this study; no informed consent was required for this study.
Statistical Analysis
Data are reported as frequency with percentage for categorical variables and median with interquartile range or mean±SD for continuous variables, as appropriate. Univariate associations of patient and scan characteristics with higher PEC (4 and 5) scores were evaluated using χ2 test. Distributions of study duration by anatomic categories in studies with PEC 4 and 5 as well as across the centers were examined using Kruskal–Wallis test. Because of the skewness of the distribution of study duration, a natural log‐transformed study duration was also used in the further analysis. Univariate associations of patient and scan characteristics with longer (natural log‐transformed) study duration were examined using a 2‐sample t test or ANOVA. Variables significantly associated with longer (natural log‐transformed) study duration in univariate analysis (P<0.05) were considered to be candidates for the multivariable analysis. Multicollinearity among the candidate variables was evaluated using the variance inflation factor before multivariable analysis. Mixed‐effects linear regression accounting for center effect was then performed to determine the independent association of PEC score with longer (natural log‐transformed) study duration. In the model, PEC score and scan characteristics associated with longer (natural log‐transformed) study duration in univariate analysis were included as fixed effects as well as a random intercept for the center. Least squares means with SEs from the model were reported. All analyses were performed using SAS, version 9.4 (SAS Institute Inc, Cary, NC). P<0.05 was considered statistically significant.
RESULTS
Demographics and Study Characteristics
Of the 179 studies that were submitted, 5 were excluded because of incomplete data. Table 1 summarizes cohort characteristics. About half of the patients (51.7%) were aged <2 years. Outpatient studies comprised 44.8% of the cohort, and 34.5% of the studies were in an intensive care setting. A larger proportion (61.5%) of studies were follow‐up studies. Initial and preoperative studies combined were 46.6% of the cohort. Studies scanned by a sonographer with >5 years of scanning congenital patients comprised 66.7%; additional images were acquired by a senior sonographer or attending in 19% of the studies. More important, 75.8% of the studies that required additional imaging had underlying CHD (PEC 4 and 5). No studies were submitted with a PEC score of 6.
Table 1.
Cohort Characteristics (N=174)
| Characteristic | Value |
|---|---|
| Patient age, y | 1.3 (0.1–9) |
| Patient age <2 y | 90 (51.7) |
| Location | |
| Outpatient | 78 (44.8) |
| ICU | 60 (34.5) |
| Non‐ICU inpatient | 28 (16.1) |
| Other/not reported | 8 (4.6) |
| Type of study | |
| Initial | 67 (38.5) |
| Follow‐up | 107 (61.5) |
| Preoperative | 32 (18.4) |
| Initial or preoperative | 81 (46.6) |
| Sedated study | 17 (9.8) |
| Patient age <2 y | 15 (8.6) |
| Patient compliant | 132 (75.9) |
| Seniority of (initial) scanner | |
| Sonographer/senior sonographer | 116 (66.7) |
| Junior sonographer | 47 (27.0) |
| Trainee | 11 (6.3) |
| Readers' experience, y | |
| <5 | 16 (9.2) |
| 5–10 | 25 (14.4) |
| >10 | 133 (76.4) |
| Additional scanning by senior sonographer or cardiologist | 33 (19.0) |
| PEC 4 or 5 | 25/33 (75.8) |
| Advanced imaging (strain, 3D imaging) | 17 (9.8) |
| Study duration, min | 34 (24–55) |
Data are presented as number (percentage) for categorical variables and median (interquartile range) for continuous variables. ICU indicates intensive care unit; and PEC, pediatric echocardiography complexity.
PEC Score and Associated Scan and Patient Characteristics
All 174 submitted studies had an assigned PEC score. Figure 2 shows the distribution of PEC scores for the entire cohort; a large proportion (68.9%) of the studies had underlying CHD. Table 2 summarizes the spectrum of congenital defects that were seen in categories PEC 4 and 5. Exploring the correlation of patient and scan characteristics with PEC scores, age <2 years, reader's experience, intensive care unit setting, preoperative study, and need for advanced imaging, findings were correlated with higher PEC scores in univariate analysis (Table 3).
Figure 2. Pie graph showing the distribution of pediatric echocardiography complexity (PEC) scores for the entire cohort (n=174).
PEC 1 indicates limited study; PEC 2, normal anatomy; PEC 3, acquired heart disease/cardiomyopathy; PEC 4, nonanatomically complex congenital heart disease; PEC 5, anatomically complex congenital heart disease; and PEC 6, mechanical support. No studies were submitted with PEC 6.
Table 2.
Spectrum of Congenital Lesions in PEC Categories 4 and 5
| PEC score | Description | Value, N (%) |
|---|---|---|
| 4 (n=62) | Aortopulmonary shunt | 26 (41.9) |
| Conotruncal defect | 9 (14.5) | |
| Arch abnormality | 9 (14.5) | |
| Valve abnormality | 4 (6.5) | |
| Endocardial cushion defect | 4 (6.5) | |
| Venous abnormality | 1 (1.6) | |
| Single ventricle | 1 (1.6) | |
| Coronary anomaly | 1 (1.6) | |
| Cardiac malposition | 1 (1.6) | |
| Myocardial abnormality | 1 (1.6) | |
| Not specified | 5 (8.1) | |
| 5 (n=58) | Single ventricle | 20 (34.5) |
| Conotruncal defect | 16 (27.6) | |
| Complex 2 ventricle | 5 (8.6) | |
| Heterotaxy syndrome | 4 (6.9) | |
| Venous abnormality | 3 (5.2) | |
| Valve abnormality | 3 (5.2) | |
| Complex VSD | 2 (3.4) | |
| Arch abnormality | 2 (3.4) | |
| Endocardial cushion defect with mechanical valve | 1 (1.7) | |
| Coronary sinus abnormality and cardiomyopathy | 1 (1.7) | |
| Other | 1 (1.7) |
PEC indicates pediatric echocardiography complexity; and VSD, ventricular septal defect.
Table 3.
Correlation of Patient and Scan Characteristics With PEC Score (N=174)
| Characteristic | PEC score | P value* | |
|---|---|---|---|
| 1–3 (N=54) | 4 or 5 (N=120) | ||
| Patient age <2 y | 19 (35.2) | 71 (59.2) | 0.003† |
| Location | 0.04† , ‡ | ||
| Outpatient | 28 (51.9) | 50 (41.7) | |
| ICU | 13 (24.1) | 47 (39.2) | |
| Non‐ICU inpatient | 9 (16.7) | 19 (15.8) | |
| Other/not reported | 4 (7.4) | 4 (3.3) | |
| Type of study | 0.08 | ||
| Initial | 26 (48.1) | 41 (34.2) | |
| Follow‐up | 28 (51.9) | 79 (65.8) | |
| Preoperative | 3 (5.6) | 29 (24.2) | 0.003† |
| Initial or preoperative | 28 (51.9) | 53 (44.2) | 0.35 |
| Sedated study | 3 (5.6) | 14 (11.7) | 0.21 |
| Patient compliant | 44 (81.5) | 88 (73.3) | 0.25 |
| Seniority of (initial) scanner | 0.56 | ||
| Sonographer/senior sonographer | 35 (64.8) | 81 (67.5) | |
| Junior sonographer | 14 (25.9) | 33 (27.5) | |
| Trainee | 5 (9.3) | 6 (5.0) | |
| Additional scanning by senior sonographer or cardiologist | 8 (14.8) | 25 (20.8) | 0.35 |
| Advanced imaging | 9 (16.7) | 8 (6.7) | 0.04† |
Data are presented as number (percentage). ICU indicates intensive care unit; and PEC, pediatric echocardiography complexity.
P value from χ2 test.
Associated with higher complexity score.
Comparison was made as ICU vs all others.
PEC Score and Study Duration
Figure 3 shows the distribution of study duration across the PEC categories. Patients with underlying congenital heart disease (PEC 4 and 5) had longer study duration (P<0.01). Figure 4 shows the distribution of study duration in PEC 4 and 5 grouped by anatomic categories showing no significant trends for longer study duration with specific anatomic groups (P=0.52). Figure 5 represents the distribution of study duration across the 12 academic centers, showing significant differences across the participating centers (P<0.0001). In univariate analysis, patients aged <2 years, intensive care unit setting, initial study, preoperative study, junior or trainee scanner, need for additional imaging by senior sonographer or attending cardiologist, and higher PEC scores (PEC 4 and 5) were associated with longer study duration (Table 4). Two outliers of the study duration were noted: (1) center 2 documented 224 minutes for supracardiac totally anomalous pulmonary venous connection; and (2) center 7 documented 447 minutes for complex single ventricle with atrioventricular valve straddle. Even after exclusion of these outliers, the univariate associations of the variables listed above with longer study duration remained significant (data not shown).
Figure 3. Box‐and‐whisker plot showing the distribution of pediatric echocardiography complexity (PEC) scores and the associated study duration for each category.
Compared with PEC 1 to 3, study duration was significantly higher for PEC 4 or 5. *P<0.0001.
Figure 4. Box‐and‐whisker plot showing anatomic categories and their associated study duration for each category (P>0.05).
ASD indicates atrial septal defect; PDA, patent ductus arteriosus; and VSD, ventricular septal defect.
Figure 5. Scatterplot showing distribution of scan length for the 12 academic centers, with variations between centers (P<0.0001).
Table 4.
Correlation of Patient and Scan Characteristics With (Natural Log‐Transformed) Study Duration (N=174)
| Characteristic | Patients, N (%) | Mean±SD | P value* |
|---|---|---|---|
| Patient age, y | 0.045† | ||
| <2 | 90 (51.7) | 3.65±0.81 | |
| ≥2 | 84 (48.3) | 3.44±0.54 | |
| Location | 0.03† , ‡ | ||
| Outpatient | 78 (44.8) | 3.48±0.55 | |
| ICU | 60 (34.5) | 3.73±0.84 | |
| Non‐ICU inpatient | 28 (16.1) | 3.51±0.49 | |
| Other/not reported | 8 (4.6) | 3.11±1.20 | |
| Type of study | <0.0001† | ||
| Initial | 67 (38.5) | 3.88±0.68 | |
| Follow‐up | 107 (61.5) | 3.35±0.63 | |
| Preoperative | 0.0003† | ||
| Yes | 32 (18.4) | 3.95±0.74 | |
| No | 142 (81.6) | 3.46±0.65 | |
| Initial or preoperative | <0.0001† | ||
| Yes | 81 (46.6) | 3.83±0.66 | |
| No | 93 (53.4) | 3.31±0.63 | |
| Sedated study | 0.81 | ||
| Yes | 17 (9.8) | 3.51±0.53 | |
| No | 157 (90.2) | 3.56±0.71 | |
| Patient compliant | 0.46 | ||
| Yes | 132 (75.9) | 3.53±0.71 | |
| No | 42 (24.1) | 3.62±0.64 | |
| Seniority of scanner | <0.0001† | ||
| Sonographer/senior sonographer | 116 (66.7) | 3.37±0.66 | |
| Junior sonographer | 47 (27.0) | 3.89±0.64 | |
| Trainee | 11 (6.3) | 3.99±0.59 | |
| Additional scanning by senior sonographer or cardiologist | <0.0001† | ||
| Yes | 33 (19.0) | 4.20±0.61 | |
| No | 141 (81.0) | 3.40±0.62 | |
| Advanced imaging | 0.35 | ||
| Yes | 17 (9.8) | 3.40±0.54 | |
| No | 157 (90.2) | 3.57±0.71 | |
| PEC score | <0.0001† | ||
| 1–3 | 54 (31.0) | 3.21±0.69 | |
| 4 or 5 | 120 (69.0) | 3.71±0.64 |
ICU indicates intensive care unit; and PEC, pediatric echocardiography complexity.
P value from 2‐sample t‐test or ANOVA.
Associated with longer scan duration.
Comparison was made as ICU vs all others, and P value was from 2‐sample t‐test.
Independent Predictors of Study Duration
Because of multicollinearity among variables significantly correlated with PEC score, such as patient age, intensive care unit setting, preoperative study, and need for advanced imaging, these variables were not included in the multivariable analysis. While adjusting for initial or preoperative study, seniority of scanner, and centers in the multivariable model, higher PEC score (4 or 5) was independently associated with longer study duration (Table 5). Because study duration was significantly different across centers, the center effect was included in the model as a random intercept. Even after controlling for the centers in the model, the result was unchanged.
Table 5.
Factors Associated With (Natural Log‐Transformed) Study Duration (N=174)
| Variable | Unadjusted | Adjusted | ||
|---|---|---|---|---|
| Mean±SD | P value* | LSMean±SE | P value† | |
| Initial or preoperative | <0.0001‡ | <0.0001‡ | ||
| Yes | 3.83±0.66 | 3.81±0.07 | ||
| No | 3.31±0.63 | 3.29±0.07 | ||
| Seniority of (initial) scanner | <0.0001‡ | <0.0001‡ | ||
| Sonographer/senior sonographer | 3.37±0.66 | 3.30±0.06 | ||
| Junior sonographer or trainee | 3.91±0.63 | 3.80±0.08 | ||
| PEC score | <0.0001‡ | <0.0001‡ | ||
| 1–3 | 3.21±0.69 | 3.27±0.08 | ||
| 4 or 5 | 3.71±0.64 | 3.82±0.06 | ||
LSMean indicates least squares mean; and PEC, pediatric echocardiography complexity.
P value from 2‐sample t‐test.
P value from mixed‐effects linear regression accounting for center effect.
Independtly associated with longer study duration.
DISCUSSION
Important Findings and Implications
In this multicenter study, using our previous complexity score footprint, we found the PEC score to be a useful marker of complexity associated with performance of a pediatric echocardiogram using total study duration as an outcome measure. Not surprisingly, study modifiers, including initial scan, preoperative scan, and scan by junior sonographer or trainee, were associated with longer study duration. Most important, higher PEC score remained independently associated with longer study duration in multivariable analysis.
The development of an echocardiography‐specific complexity score inherently poses challenges related to the vast spectrum of anatomic variants of CHD. Added to this anatomic complexity are study‐related variables that are important for successful performance of an echocardiogram. Consequently, the complexity scoring system can incorporate numerous permutations of anatomy, study variables, and scanner variables. However, the overarching goal of this study was not to enumerate the gamut of anatomy that can influence the time and resources allocated to performing an echocardiogram but rather to condense these groups into comparable categories. The input of our expert panel added to this scoring tool by incorporating clinically meaningful study‐related variables, which further strengthened the complexity score stratification. This study provides insight into the important factors that should be taken into consideration for developing quality improvement metrics, for allocation of resources and distribution of sonographer responsibilities, for tracking diagnostic errors, and for planning trainee education. For instance, individual centers could apply the PEC score with modifications as appropriate for the workflow of their respective centers (eg, 4A for standard conotruncal defect that is an initial scan or preoperative scan, 4B for follow‐up scan on the same lesion) to anticipatorily allocate time for scan, sonographer assignment, and identification of potential teaching scans. With uniformity in scoring and tracking metrics, interinstitutional comparisons on productivity, workflow, and resource allocation can be made effectively. Taken together, the PEC scoring system, with its ease of application, could find an important place in establishing baseline variations in the performance of echocardiograms across the country and could aid with creating national standards.
As with all diagnostic studies, performance and interpretation are closely intertwined. This study only examined the factors influencing the performance of an echocardiogram. It is almost certain that additional factors will unravel as we incorporate the components of interpretation into diagnostic studies. Additional components that should also be considered when performing a transthoracic echocardiogram include the following: (1) review of patient data and study requisition, (2) prescan review with reader, (3) equipment setup, (4) patient preparation and positioning, (5) performance of the scan, (6) review of the study with reader after scan, (7) equipment clean up, and (8) data and preliminary report entry. 9 With sedated echocardiograms, the added time of waiting until sedation has taken effect also adds to the length of the study. All of these components need to be factored into the total time for a scan. However, aside from steps 4, 5, and 6, the time taken for the remainder of the components is largely constant. Hence, we focused on the factors that have the greatest influence on the length of the study, and our findings have applicability in many clinical settings.
Study Duration and Workflow
Study duration directly impacts scheduling time allotment for both outpatient and inpatient echocardiograms. Indeed, the Intersocietal Accreditation Commission standards specify performance time and allocation for noncomplicated and complicated studies. 10 Examining our own cohort of studies with the assumption that this is a reasonable representation of study distribution, one would expect to allocate more time and resources to ≈69% of the studies that were PEC 4 or 5. Although this could represent a significant portion of the studies that are performed on a day‐to‐day basis, identification of those studies with higher anatomic complexity scores (PEC 4 or 5), along with additional factors, such as initial or preoperative study, should alert the system for longer study duration and allow for implementation of the necessary time and structured support. It is not surprising that studies that (1) have an underlying congenital heart defect, (2) are performed as part of a preoperative workup, and (3) are performed by less experienced sonographers and trainees have longer study duration. Because these variables can be easily identified a priori, they can play a pivotal role in resource allocation. Indeed, our study did not show any significant skew in the study duration based on variations in anatomic findings within PEC 4 and 5, lending support to our finding that once studies are separated out as congenital versus noncongenital, the additive value of categorizing anatomic variables is not contributory. Similar to our findings, Banka et al reported that medium‐ and high‐complexity studies had the longest study duration, where the investigators examined resource use in a tertiary pediatric echocardiography laboratory. 9 Our study adds to this body of work with the validation of a score by a multi‐institutional panel.
Comparisons of physician effort and clinical productivity with echocardiograms have proven to be complex undertakings because of variations in workflow, the need for real‐time review and communication with ordering teams, imaging‐assisted interventions, and timely review for preoperative planning. 2 This study could serve as an initial step toward designing and validating models that can further understand the challenges with workflow in the echocardiography laboratory that affect productivity (both clinical and academic) and efficiency.
Study Duration and Staffing
Scheduling should also account for resources allocated, accounting for scanner availability, anticipating the need for assistance from a senior sonographer or attending, and prioritizing scans that are more time intensive, such as studies in high‐acuity care settings. This finding, again, points to the planning of these studies, ensuring that adequate support is available for additional acquisition of images and postprocessing. The score can also serve to track sonographer productivity, particularly in instances where senior sonographers invest time in assisting trainee sonographers and fellows, which are not accounted for in the total number of studies scanned. 2
Impact on Training
From a fellow and sonographer training standpoint, early identification of these studies provides an opportunity for exposure to higher‐complexity studies. Prioritizing clinically important questions to be obtained in young children and during sedated examinations is an important skill to hone; designation of such studies as good training scans will facilitate good education in a controlled manner, so that inordinately longer times are not spent on these younger patients.
Impact on Quality Improvement
Use of the PEC score in a multi‐institutional setting can systematically track diagnostic errors and pave the way for more nuanced root cause analysis. 5 , 11 In an ongoing multicenter collaborative study examining causes for diagnostic discrepancies, 76% of the diagnostic discrepancies were preventable, of which 47% were related to imaging factors (S.S. Natrajan et al, unpublished data, 2021–2022). Use of the PEC score in a similar cohort could clarify what specific imaging factors were contributory and what modifiable factors could be instituted as interventions. These interventions could be iterative with an established smart aim.
Limitations
Initial validation was performed by members in large academic centers who were intimately involved with the development of the PEC score. The ease of use of this tool will be better established if validated by an independent panel from a variety of clinical settings. The PEC score examines factors that affect the performance of an echocardiogram but does not include aspects of study interpretation. We acknowledge that both performance and interpretation are integral components of echocardiography, but the latter requires further examination in a separate study. Study duration was calculated on the basis of the time stamp on the first and last images of the study. We acknowledge that this method assumes that all the scanning time is contiguous without interruptions for other factors, such as involving an additional scanner. Indeed, 2 of the scans were considered outliers, with scan times of 224 and 447 minutes, which were likely not related to continuous scanning. Although the contributing institutions in this study are tertiary care centers with mostly similar workflow and trainee involvement, centers vary in their scanning protocols (eg, initial study, lesion‐specific follow‐up), in their use of scanning and analysis platforms, and in their approach to use of advanced imaging techniques, such as strain and 3D volume processing, which could add time to study duration; data specific to these factors were not captured. In addition, an extension of these data to nonacademic institutions should be done for further validation in a variety of clinical settings.
CONCLUSIONS
Pediatric echocardiography encompasses a large spectrum of anatomic and study‐related variables that impact the complexity and effort of performing a study. In this multi‐institutional study, we developed and validated an echocardiography‐specific complexity score (PEC score), which showed that higher complexity studies require longer study duration. Future studies investigating the application of the PEC score in developing metrics for resource use and productivity and tracking diagnostic errors will facilitate longitudinal comparisons within and across institutions.
Sources of Funding
None.
Disclosures
Dr Parthiban is a primary investigator for the echocardiography core laboratory for the ALTERRA study, conducted by Edwards Lifesciences. The remaining authors have no disclosures to report.
Acknowledgments
We would like to acknowledge Dr Kathy Jenkins for her expert guidance at the inception of this project.
This article was sent to John L. Jefferies, MD, MPH, Guest Editor, for review by expert referees, editorial decision, and final disposition.
For Sources of Funding and Disclosures, see page 10.
References
- 1. Sachdeva R, Valente AM, Armstrong AK, Cook SC, Han BK, Lopez L, Lui GK, Pickard SS, Powell AJ, Bhave NM, et al. ACC/AHA/ASE/HRS/ISACHD/SCAI/SCCT/SCMR/SOPE 2020 appropriate use criteria for multimodality imaging during the follow‐up care of patients with congenital heart disease: a report of the American College of Cardiology Solution set Oversight Committee and Appropriate use Criteria Task Force, American Heart Association, American Society of Echocardiography, Heart Rhythm Society, International Society for Adult Congenital Heart Disease, Society for Cardiovascular Angiography and Interventions, Society of Cardiovascular Computed Tomography, Society for Cardiovascular Magnetic Resonance, and Society of Pediatric Echocardiography. J Am Coll Cardiol. 2020;75:657–703. doi: 10.1016/j.jacc.2019.10.002 [DOI] [PubMed] [Google Scholar]
- 2. Srivastava S, Allada V, Younoszai A, Lopez L, Soriano BD, Fleishman CE, Van Hoever AM, Lai WW. Determinants of pediatric echocardiography laboratory productivity: analysis from the second survey of the American Society of Echocardiography Committee on Echocardiography Laboratory Productivity. J Am Soc Echocardiogr. 2016;29:1009–1015. doi: 10.1016/j.echo.2016.06.007 [DOI] [PubMed] [Google Scholar]
- 3. Sachdeva R, Samai C, Vincent RN. Pediatric appropriate use criteria for echocardiography: implications for clinical practice. Pediatrics. 2015;135:e1128–e1130. doi: 10.1542/peds.2014-3537 [DOI] [PubMed] [Google Scholar]
- 4. Balasubramanian S, Kipps AK, Smith SN, Tacy TA, Selamet Tierney ES. Pediatric echocardiography by work relative value units: is study complexity adequately captured? J Am Soc Echocardiogr. 2016;29:1084–1091. doi: 10.1016/j.echo.2016.05.015 [DOI] [PubMed] [Google Scholar]
- 5. Benavidez OJ, Gauvreau K, Geva T. Diagnostic errors in congenital echocardiography: importance of study conditions. J Am Soc Echocardiogr. 2014;27:616–623. doi: 10.1016/j.echo.2014.03.001 [DOI] [PubMed] [Google Scholar]
- 6. Stern KW, Gauvreau K, Geva T, Benavidez OJ. The impact of procedural sedation on diagnostic errors in pediatric echocardiography. J Am Soc Echocardiogr. 2014;27:949–955. doi: 10.1016/j.echo.2014.04.024 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Jacobs JP, Jacobs ML, Lacour‐Gayet FG, Jenkins KJ, Gauvreau K, Bacha E, Maruszewski B, Clarke DR, Tchervenkov CI, Gaynor JW, et al. Stratification of complexity improves the utility and accuracy of outcomes analysis in a multi‐institutional congenital heart surgery database: application of the Risk Adjustment in Congenital Heart Surgery (RACHS‐1) and Aristotle Systems in the Society of Thoracic Surgeons (STS) Congenital Heart Surgery Database. Pediatr Cardiol. 2009;30:1117–1130. doi: 10.1007/s00246-009-9496-0 [DOI] [PubMed] [Google Scholar]
- 8. Jenkins KJ, Gauvreau K. Center‐specific differences in mortality: preliminary analyses using the Risk Adjustment in Congenital Heart Surgery (RACHS‐1) method. J Thorac Cardiovasc Surg. 2002;124:97–104. doi: 10.1067/mtc.2002.122311 [DOI] [PubMed] [Google Scholar]
- 9. Banka P, Schaetzle B, Gauvreau K, Geva T. Determinants of resource utilization in a tertiary pediatric and congenital echocardiographic laboratory. Am J Cardiol. 2015;116:1139–1143. doi: 10.1016/j.amjcard.2015.07.013 [DOI] [PubMed] [Google Scholar]
- 10. IAC Standards & Guidelines for Pediatric and Congenital Echocardiography Accreditation. Intersocietal Accreditation Commission. Date published June 2023. Accessed February 10, 2023. The IAC Standards and Guidelines for Pediatric and Congenital Echocardiography Accreditation. https://intersocietal.org/
- 11. Behrle NI, O'Brien J, Kiaffas M, Noel‐MacDonnell J, Shirali G, Parthiban A. Reducing transthoracic echocardiographic diagnostic error in congenital heart disease. J Am Soc Echocardiogr. 2020;33:1156–1158. doi: 10.1016/j.echo.2020.03.025 [DOI] [PubMed] [Google Scholar]