Intraoperative Renal Resistive Index as an Acute Kidney Injury Biomarker: Development and Validation of an Automated Analysis Algorithm

Abstract

Objective

Intraoperative Doppler-determined renal resistive index (RRI) is a promising early acute kidney injury (AKI) biomarker. As RRI continues to be studied, its clinical usefulness and robustness in research settings will be linked to the ease, efficiency, and precision with which it can be interpreted. Therefore, we assessed the usefulness of computer vision technology as an approach to developing an automated RRI-estimating algorithm with equivalent reliability and reproducibility to human experts.

Design

Retrospective.

Setting

Single-center, university hospital.

Participants

Adult cardiac surgery patients from 7/1/2013–7/10/2014 with intraoperative transesophageal echocardiography-determined renal blood flow measurements.

Interventions

None.

Measurements and Main Results

Renal Doppler waveforms were retrospectively obtained and assessed by blinded human expert raters. Images (430) were evenly divided into development and validation cohorts. An algorithm for automated RRI analysis was built using computer vision techniques and tuned for alignment with experts using bootstrap resampling in the development cohort. This algorithm was then applied to the validation cohort for an unbiased assessment of agreement with human experts. Waveform analysis time per image averaged 0.144 seconds. Agreement was excellent by intraclass correlation coefficient (0.939; 95% CI [0.921, 0.953]) and in Bland-Altman analysis (mean difference [human–algorithm] −0.0015; 95% CI [−0.0054, 0.0024]), without evidence of systematic bias.

Conclusions

We confirmed the value of computer vision technology to develop an algorithm for RRI estimation from automatically processed intraoperative renal Doppler waveforms. This simple-to-use and efficient tool further adds to the clinical and research value of RRI, already the “earliest” among several early AKI biomarkers being assessed.

Introduction

Doppler-determined renal resistive index (RRI), an index of renal arterial pulsatility, is an emerging novel biomarker for acute kidney injury (AKI) that adds value to a routine cardiovascular ultrasound examination (e.g., intraoperative transesophageal echocardiography [TEE] or transabdominal ultrasound [TUS])^1–3. RRI is calculated using Doppler velocity measurements of intrarenal arterial blood flow (figure 1). Emerging evidence highlights the applications of RRI as a significant in both clinical and research settings; these require that waveform interpretation be precise, reproducible, and consistent among raters. This is particularly important given high acuity situations in which RRI may play an important role (e.g., intraoperative and critical care settings). Furthermore, as the value of RRI becomes more validated, its use in clinical practice will be directly linked to the ease and efficiency with which it can be obtained and interpreted.

Figure 1.

Calculation of the renal resistive index (RRI) using transesophageal echocardiography. V_sys = peak systolic velocity; V_dia = trough diastolic velocity.

Importantly, traditional AKI diagnosis and therapy is often delayed due to reliance on serum creatinine accumulation (i.e., up to 48 hours). This has prompted a highly publicized search for early AKI biomarkers, among which an abnormally high RRI appears to be, by several hours, the earliest indication of AKI. This early predictive advantage may be explained mechanistically: it has been proposed that RRI elevation may reflect increased intracapsular pressure (i.e., injury-related renal “compartment syndrome”)^4–6. In cardiac surgery patients the apparent intraoperative timing of primary renal insult has facilitated research into the promptness of RRI as an early AKI biomarker^{2, 7}. An elevated intraoperative (post-CPB) RRI by TEE in cardiac surgery patients is associated with subsequent AKI diagnosis^{1, 2}. Similarly, elevated RRI predicts AKI in critically ill patients soon after various surgical procedures^{3, 8–10}. The hope is that this earliest knowledge of AKI risk based on RRI elevation (before other biomarkers become positive) will translate into future development of reno-protective interventions in both surgical and nonsurgical patients.

Furthermore, the utility of RRI as a biomarker is not limited to AKI episodes, nor is it limited to renal pathology alone. For example, elevated RRI in patients with congestive heart failure has prognostic value for mortality risk and the future decline of both cardiac and renal function^11–13. Moreover, in the setting of chronic kidney disease, RRI elevation correlates with mortality risk¹⁴. As this intriguing biomarker continues to be tested for its potential mainstream clinical utility, using a development-validation approach we sought to assess the value of computer vision technology (i.e., the computerized processing and analysis of digital images) to develop an automated algorithm for processing renal Doppler waveforms to more easily and reproducibly estimate RRI.

Methods

Study Population

After Institutional Review Board approval, a retrospective review of our institutional TEE Database was performed. Subjects included patients ≥18 years old who underwent cardiac surgery between 7/1/2013 and 7/10/2014 at our institution for whom renal blood flow measurements obtained via TEE were documented. Renal Doppler images were excluded a priori if they were deemed to be of insufficient quality for automated analysis (i.e., significant artifact, distortion due to masking, partial waveforms).

Intraoperative Ultrasound Imaging

A comprehensive examination using a multiplane TEE probe (Philips X7-2t; Philips iE33, Andover, MA) is routine both prior to, and after, CPB at our institution. Acquisition of renal blood flow measurements is encouraged during examination and images from each examination are saved in an institutional database for future reference. The examination is performed by an adult cardiothoracic anesthesiology fellow along with an attending anesthesiologist certified in advanced peri-operative TEE by the National Board of Echocardiography (Advanced PTEeXAM). Left renal artery Doppler velocity images were obtained using a transgastric approach. Briefly, a midpapillary transgastric short-axis view is first obtained. The probe is then rotated approximately 180 degrees to the left to obtain a short-axis view of the descending aorta and then advanced to the origin of the left renal artery. This artery is followed by rotating the probe to the right until visualization of the left kidney occurs.¹⁵

Human Assessment of RRI

Seven cardiothoracic anesthesiologists certified in TEE reviewed de-identified renal Doppler images to determine RRI values. Inter-rater reliability for RRI determination among our investigators was assessed previously through evaluation of 78 images (intraclass correlation coefficient 0.77).² Renal arterial peak systolic and trough diastolic velocities were measured for up to 3 cardiac cycles. RRI was then calculated for each cycle (figure 1) and averaged for each image. These human expert values served as the gold standard for comparison with values generated by the automated algorithm.

Automated Assessment of RRI

We used computer vision techniques (OpenCV; Python v3.6.4) to develop an algorithm to estimate RRI from automatically processed renal Doppler waveform images. Details of the standard image processing procedures utilized are well described elsewhere^16–18. The algorithm passes each original renal Doppler waveform through several processing steps as shown in figure 2. Briefly, coordinates of the zero velocity baseline are obtained using a Hough line transform¹⁶. All non-waveform components are then removed using text recognition, color masking, and shape-matching algorithms (i.e., text and marker annotations, zero velocity baseline, electrocardiogram [ECG] tracing, velocity axis, non-dominant waves); the remaining image represents the isolated dominant waveform structure (figure 2A/B-ii). Dynamic brightening and iterative blurring (via a median filter) are then applied to achieve pre-specified pixel intensity and waveform uniformity targets; these serve as key image tuning parameters (figure 2A/B-iii). A strictly black and white outline of the dominant waveform is then generated using Otsu’s method (figure 2A/B-iv)¹⁷, and contours are identified (figure 2A/B-v)¹⁸. Artifact structures highlighted by the contour analysis are then removed through filtering of contour coordinates, leaving only the top surface contour of each viable waveform (figure 2A/B-vi). Finally, individual waveform characteristics, including frequency and amplitude, are used to guide identification of valid peaks and troughs (figure 2A/B-vi). These values are then used to estimate RRI for each wave, which are averaged for each image.

Figure 2.

Processing steps of the automated algorithm shown for: (A) a higher-quality image and (B) a lower-quality image. (i) Original image; (ii) isolated dominant waveform; (iii) image following brightening and iterative blurring; (iv) image following Otsu’s method; (v) all identified contours (blue points); (vi) filtered contours (blue points) and identified peaks (orange crosses) and troughs (orange X’s). Notably, both positive and negative waveforms are shown to reinforce the automated nature of the algorithm in identifying the location of the dominant waveform structure in relation to the zero velocity baseline.

Algorithm Tuning and Validation

Key to the usefulness of the above-mentioned algorithm is the validity of the estimated RRI values compared to human experts. Therefore, to assess the validity of this methodology, all eligible renal Doppler images were randomly (via random number generator) and evenly divided into development and validation cohorts. The images in the development cohort were used to calibrate the algorithm against human expert determinations, and the images in the validation cohort were then used to confirm, in an unbiased manner, agreement with the gold standard. The algorithm was tuned in the development cohort by systematically adjusting image tuning parameters (i.e., pixel intensity and waveform uniformity targets). To allow for the most robust and unbiased assessment of agreement in the tuning process, a collection of 45 unique parameter combinations (i.e., image tuning parameter sets) were assessed for agreement via bootstrap resampling. For each unique image tuning parameter set, 1000 bootstrap samples were generated and agreement between human experts and the algorithm was evaluated. Agreement was assessed by: (1) mean difference (the signed difference in RRI between human experts and the algorithm) and (2) intraclass correlation coefficient (ICC). ICC was calculated using a two-way random effects model for absolute agreement¹⁹. The optimal image tuning parameter set was selected in a two-step process. First, to guard against systematic bias compared to humans, we only selected parameter sets for which the bootstrapped 95% confidence interval of the mean difference contained zero. The image tuning parameter set with the highest bootstrapped ICC was then chosen for use in the validation cohort. Finally, testing of the selected parameter set was conducted through a single evaluation of images in the validation cohort using both the ICC and a Bland-Altman analysis of differences. Mean absolute algorithm error was determined as a percent of human expert values (absolute value of [human value – algorithm value] / human value * 100).

Results

After exclusion of images of insufficient quality for automated analysis (n = 66), 430 renal Doppler velocity waveform images from 318 unique patient procedures were identified and divided for analysis in the two cohorts. Patient and procedural characteristics were similar to those observed in other populations (table 1)²⁰. In the development cohort (n = 215 images), 38 of the 45 image tuning parameter sets produced a bootstrapped mean difference with a 95% confidence interval that overlapped zero (figure 3A). Among the identified parameter sets, ICC values ranged from 0.950 to 0.956, and the optimal algorithm with the highest ICC was selected for further assessment in the validation cohort (figure 3B).

Table 1.

Patient and Procedural Characteristics

	Patients (n = 318)
Age (years)	64 (56 – 72)
Female	116 (36%)
Body Mass Index (kg/m2)	28 (25 – 33)
Diabetes (n = 312)	227 (73%)
Hypertension (n = 312)	147 (47%)
History of stroke (n = 312)	293 (94%)
Peripheral Vascular Disease (n = 312)	300 (96%)
Left Ventricular Ejection Fraction (n = 281)
<45%	68 (24%)
>55%	139 (49%)
45–55%	74 (26%)
Procedure
Aortic	11 (3%)
CABG	106 (33%)
Combined	51 (16%)
Valve	100 (32%)
Other	50 (16%)

Abbreviations: CABG = coronary artery bypass grafting, CPB = cardiopulmonary bypass. Combined procedure indicates a combination of two or more of the listed categories. Values are presented as median (Q1 – Q3) or frequency (%). Effective sample size (due to missing data) is indicated beside each parameter name when applicable.

Figure 3.

Results from the algorithm parameter tuning process on the development cohort. (A) Step 1: bootstrapped mean difference and empirical 95% confidence intervals (CI) for all tested image tuning parameter sets (n = 45 parameter sets). Parameter sets with a 95% CI containing 0 (blue solid lines) were considered eligible for further testing while those with 95% CI excluding 0 (orange dashed lines) were eliminated from consideration. (B) Step 2: bootstrapped intraclass correlation coefficient (ICC) and empirical 95% confidence intervals for image tuning parameter sets eligible after step 1 (n = 38 parameter sets). The parameter set with the highest ICC (green line; indicated with arrow) was selected for testing on the validation cohort.

In testing the optimal algorithm in the validation cohort (n = 215 images), there was excellent agreement with blinded human experts (ICC 0.939; 95% CI [0.921, 0.953]; figure 4). Similarly, agreement was strong in Bland-Altman analysis of differences (mean difference [human – algorithm] −0.0015; 95% CI [−0.0054, 0.0024]; figure 5), suggesting no evidence of systematic bias. Furthermore, consistency in agreement across the range of RRI values was supported by the lack of association between mean RRI values and differences between human experts and the selected algorithm (p = 0.30). The 95% limits of agreement (an indication of spread of the differences between human experts and the algorithm) were acceptably narrow, ranging from −0.058 to 0.055 (figure 5). The mean absolute error for algorithm values compared to human experts was 3.39%. Analysis time averaged 0.144 seconds per image (95% CI [0.139, 0.149]).

Figure 4.

Scatter plot displaying renal resistive index (RRI) values rated by human experts compared to those rated by the automated algorithm in the validation cohort using the parameters selected through parameter tuning on the development cohort. The black line represents the identity line (complete agreement). Intraclass correlation coefficient (ICC) is displayed with a 95% confidence interval.

Figure 5.

Bland-Altman plot displaying agreement between renal resistive index (RRI) values rated by blinded human experts and the automated algorithm in the validation cohort using parameters selected in the parameter tuning process on the development cohort. For each image, mean RRI (average of human and algorithm values) is plotted against the difference in RRI (human value – algorithm value). The mean difference between human experts and the algorithm is represented by the solid blue line, with 95% confidence intervals (CI) for the mean difference shown as dashed blue lines. The upper and lower limits of agreement (mean difference ± 1.96 standard deviation (SD) units) are shown as dotted red lines. Notably, the 95% CI for the mean difference contains 0, indicating a lack of systematic bias in the algorithm when compared to human experts.

Discussion

Using computer vision techniques, we successfully developed and validated a novel algorithm that automatically processes intraoperative renal Doppler waveform images and estimates RRI. In validation testing, automated RRI assessment showed excellent agreement with human raters without evidence of systematic bias. Moreover, previously-reported strong inter-rater reliability for RRI determination among human experts was even exceeded by that of the algorithm (ICC 0.939 vs. 0.770)², suggesting the algorithm may even improve upon the current human expert gold standard. Furthermore, automated analysis substantially improved efficiency when compared to our prior standard approach to post-hoc image assessment (mean analysis time per image 0.144 seconds vs. 76 seconds)²¹. This gain in efficiency (more than a minute per image if the technology were to be available on an equivalent analysis platform) easily reaches the level of clinical relevance and substantially reduces provider interpretation time. Together, these findings suggest that automated intraoperative analysis of renal Doppler waveforms for RRI assessment is a feasible, reproducible, and accurate approach which aligns with the need to make this data promptly available and has potential to add clinical and research value to RRI as the earliest currently available AKI biomarker among those being evaluated.

Although no previous reports have specifically explored the automation of RRI assessment from Doppler waveforms, automated analysis of medical imaging is a burgeoning field. A number of previously published reports have documented efforts to develop semi- and fully-automated methods for assessment of a variety of both two-dimensional and three-dimensional metrics^22–26. Most similar to the present analysis is the work of Zolgharni and colleagues in which an automated assessment method was developed for aortic Doppler velocity-time-integral and peak-velocity metrics²³. In their analysis, similar processing steps were carried out to define Doppler wave contours, with comparably strong agreement between human and algorithmic results. While not directly integrated into the ultrasound machine software, the algorithm developed by Zolgharni et al. was implemented using a laptop with real-time data acquisition using a frame grabber. Approaches to further develop the study algorithm include integration as a built-in component of the TEE machine to allow real-time RRI analysis. Furthermore, the approach we describe to develop automated RRI determination in the current study could be generalized to assess other Doppler-determined metrics.

We observed some limitations of our automated algorithm, including the indiscriminate acceptance of waveforms for assessment compared to human experts. For example, humans exclude waveforms due to poor quality or high variability while the algorithm registers all complete waveforms as valid. Interestingly, this approach to characterizing RRI variability and its relationship to AKI have not been assessed. Additionally, when waveforms fall at the far-right end of an image (with no discernable systolic upstroke after a level diastolic plateau) the peak/trough-finding function in our algorithm would not accept the trough without evidence of a subsequent upstroke. In many cases such waves were acceptable to human experts and contributed to their estimation of the mean RRI from the image. Such concerns highlight the importance of human oversight of any RRI automated estimation tool, as previously recognized with similar technologies in other clinical settings (e.g., ST segment analysis)²⁷. Inclusion of additional data (e.g., ECG analysis to identify QRS complex timing) in future iterations may be a useful approach to continue to improve our algorithm. Other steps we envision to further optimize this approach involve improvement of algorithm heuristics to more closely resemble the problem-solving strategies of human experts. Initial implementation of this technology would allow for automated ‘suggestions’ by the algorithm for human expert users to accept or modify in the same way that current 3D software has evolved to semi-automate initial tracings for editing by users. Over time, analysis of expert modification of algorithm suggestions would additionally allow for implementation of machine learning processes. These will lead to both: (1) improved image processing and (2) assessment of algorithm confidence (e.g., to identify poor quality waveforms). The latter will help facilitate a transition to full automation with human expert intervention only in instances of reduced algorithmic confidence. Future development of our system will include work with larger datasets to incorporate algorithmic-learning methods that will allow for both identification of poor quality waves likely to be excluded by human experts, and a more dynamic assessment of confidence in trough identification at the end of an image frame (through ECG identification of QRS complexes and predicted length of R-R intervals, for example). Additionally, while our method for image processing utilized several dynamic parameters based on individual image properties (i.e. was not a “one-size-fits-all” approach), future study with larger datasets of Doppler images with expert-labeled contours will allow for implementation of a machine-learning based approach to image processing, further improving agreement with these experts. To further mitigate error, comparison of adjacent waveforms may reduce concerns over respiratory cycle artifact. Automatic assessment of Doppler waveforms in real time (i.e., using video input directly from ultrasound machines) would allow for immediate calculation of averaged Doppler metrics and inclusion of several-fold more waveforms than is typical, yielding more stable and robust estimates. Notably, the use of this algorithm in the real time setting represents the arena in which time savings is most likely due to efficiency of measurement. While RRI formula can already be incorporated into ultrasound machines for improved ease of assessment, the ability to measure multiple beats simultaneously and to provide real time data on these beats remains out of reach using the current caliper measurement technology. In this sense, the automated algorithm may serve as a platform for a more expanded assessment of renal perfusion through analysis not only of single beat values but also of beat-to-beat variability (i.e., due to respiration or cardiac arrhythmia) and, RRI range over time (i.e., minimum and maximum peak velocities) and overall central tendency (i.e., moving average). Finally, the results of this analysis must be interpreted with consideration of the sample of images utilized for its construction. While a large proportion of all identified images were of high enough quality for automated assessment, a non-insignificant number of images were excluded from analysis due to prohibitively poor quality. Thus, the excellent reproducibility shown in the present analysis does not fully address reproducibility issues that may arise during acquisition of Doppler velocity tracings (i.e., prohibitively poor-quality images with distortion or excessive artifact). Accurate interpretation of RRI relies, in part, on the clinician’s ability to obtain high quality visualizations of renal blood flow.

Nonetheless, the current algorithm performs well compared to human experts and represents the only report of automated RRI assessment from renal Doppler images, an important initiative given the growing interest in this earliest of early AKI biomarkers. Availability of such tools in clinical practice should allow inexperienced users to more accurately and rapidly determine RRI values from static renal Doppler images, potentially removing barriers to its use in everyday practice. Furthermore, standardizing RRI estimation to align and even become more consistent than determinations among human experts, will reduce inter-rater variability and facilitate more robust research from smaller datasets. The potential value of computer vision and other similar technologies to develop tools for quantifying waveforms extends beyond RRI and includes other Doppler waveform metrics such as volume time integrals, pressure gradients, mitral inflow patterns, etc.

In summary, we confirmed the value of computer vision technology in developing a valid automated algorithm for RRI determination from static renal Doppler images. Our algorithm shows excellent agreement with blinded human experts trained in TEE interpretation and provides an exponential improvement in processing efficiency. This algorithm represents an important step in automating renal Doppler image analysis; future work will focus on additional algorithm improvements to hasten access to high quality early AKI biomarker data.