Comparison of different invasive hemodynamic methods for AV delay optimization in patients with cardiac resynchronization therapy: Implications for clinical trial design and clinical practice

Abstract

Background

Reproducibility and hemodynamic efficacy of optimization of AV delay (AVD) of cardiac resynchronization therapy (CRT) using invasive LV dp/dt_max are unknown.

Method and results

25 patients underwent AV delay (AVD) optimisation twice, using continuous left ventricular (LV) dp/dt_max, systolic blood pressure (SBP) and pulse pressure (PP). We compared 4 protocols for comparing dp/dt_max between AV delays:

• Immediate absolute: mean of 10 s recording of dp/dt_max acquired immediately after programming the tested AVD,

• Delayed absolute: mean of 10 s recording acquired 30 s after programming AVD,

• Single relative: relative difference between reference AVD and the tested AVD,

• Multiple relative: averaged difference, from multiple alternations between reference and tested AVD.

We assessed for dp/dt_max, LVSBP and LVPP, test–retest reproducibility of the optimum.

Optimization using immediate absolute dp/dt_max had poor reproducibility (SDD of replicate optima = 41 ms; R² = 0.45) as did delayed absolute (SDD 39 ms; R² = 0.50). Multiple relative had better reproducibility: SDD 23 ms, R² = 0.76, and (p < 0.01 by F test).

Compared with AAI pacing, the hemodynamic increment from CRT, with the nominal AV delay was LVSBP 2% and LVdp/dt_max 5%, while CRT with pre-determined optimal AVD gave 6% and 9% respectively.

Conclusions

Because of inevitable background fluctuations, optimization by absolute dp/dt_max has poor same-day reproducibility, unsuitable for clinical or research purposes. Reproducibility is improved by comparing to a reference AVD and making multiple consecutive measurements. More than 6 measurements would be required for even more precise optimization — and might be advisable for future study designs. With optimal AVD, instead of nominal, the hemodynamic increment of CRT is approximately doubled.

1. Introduction

Test–retest reproducibility of the identified optimum is a sine qua non of any process of optimization of AV delay for cardiac resynchronization therapy (CRT) [1]. If immediately-successive attempts at optimization yield very different results, then either the optimum is really changing dramatically (in which case there is no point in trying to optimize) or the optimization process is not reliably identifying the optimum, because the protocol has not done enough to counteract the intrinsic baseline variability of the measured variable. An unreliable optimization process may cause physiological harm if it causes AV delay to be programmed to a worse value than nominal.

Invasive left ventricular (LV) dp/dt_max measurements are used to guide optimization of cardiac resynchronization therapy devices [2,3], because they directly assay the effects of AV delay on cardiac contraction. On the strength of their directness, they are often used as a gold standard against which other candidate variables for optimization can be compared [4,5], as well as a tool for assessing the impact of LV pacing site during CRT [6].

Early assessment of test–retest reproducibility (i.e. from separate data, not re-analysis of identical datasets) is a useful prelude to instituting a method into clinical practice or embarking on a clinical outcome trial. Without this, if an outcome trial does not show benefits from optimization, it is not possible to distinguish between AV delay being unimportant versus the tested method being unreproducible [7], even if the trial is meticulously conducted and large-scale.

A reliable method for assessing the impact of adjusting the settings of resynchronization devices is a pre-requisite for studies investigating the mechanism through which they deliver their beneficial effect.

1.1. Processing of LV dp/dt_max

Measurements of LV dp/dt_max at each AV delay setting may be expressed absolutely (immediately after changing the AV delay, or after a delay for stabilisation); or by comparison to the measurement of a reference delay (recorded immediately prior to changing to the tested AV delay, or after a delay for stabilisation). A wide variety of protocols have been described in the literature for acquiring LV dp/dt_max including the following:

• •measuring absolute dp/dt_max after a 30 second stabilisation period for a duration of 30 s [8].
• •waiting 20 s and then recording dp/dt_max for ≥ 1 respiratory cycle, and calculating the average dp/dt_max over that time period [3].
• •the relative value of dp/dt_max recorded after a 2 minute stabilisation period following the transition to the tested AV delay [9].
• •the average dp/dt_max derived from 10 consecutive paced beats, starting from the third beat after the new pacing mode was applied, compared to a baseline measurement taken at the end of the optimization sequence [10].
• •comparing each tested AV delay to a reference state recorded immediately prior to the transition to the tested AV delay and performing repeated measurements between the tested AV delay and the reference state [2,11].

1.2. Choice of invasive variable

Even though LV dp/dt_max is widely accepted as the gold standard, others have also been used, including aortic pulse pressure and LV systolic pressure [2]. It is unclear whether they are acceptable, as to our knowledge no studies have assessed the test–retest reproducibility of AV delay optimization performed using these measures.

Several questions about dp/dt_max optimization remain unknown:

• 1.What is the test–retest reproducibility of optimization by dp/dt_max using the current most widely-used processing method of delayed absolute?
• 2.Do alternative methods of processing dp/dt_max, such as subtraction from reference measurements, provide any advantage?
• 3.Does the process of peak slope calculation (dp/dt_max) give worthwhile improvement in precision of identification of the optimum, versus LV systolic pressure or LV pulse pressure?

In this study we address these questions by determining test–retest reproducibility of optimization in patients undergoing invasive optimization.

1.3. The biological determinants of reproducibility

Optimization relies on detection of potentially small hemodynamic differences between AV delay settings, in an environment of ever-present biological variability. If the genuine hemodynamic differences between AV delays (the “signal”) are large, optimization is easy; if they are small, optimization is difficult. Meanwhile, if biological variability (the “noise”) between one measurement and a repeat measurement shortly afterwards, is large, optimization is difficult; if it is small, optimization is easy. When embarking on optimization, it is therefore preferable to use an approach with high signal-to-noise ratio. In this study we assess head-to-head signal-to-noise ratios for three hemodynamic variables.

2. Methods

2.1. Subjects

25 patients who had cardiac resynchronization pacemakers (CRT-P) or cardiac resynchronization therapy defibrillators (CRT-D) implanted for clinical indications (NYHA II–IV heart failure, QRS > 120 ms or echocardiographic evidence of mechanical dyssynchrony, maximal medical therapy) were enrolled into this study. AV delay optimization was performed using invasive hemodynamic measurements. This was carried out up to 3 days post-implantation. 21 were male, 4 were female, mean age 64 years (min 34, max 78). Cause of heart failure was ischemic in 14, idiopathic dilated in 10, and chemotherapy in 1. The ejection fraction of the patients at the time of the study was 27 ± 5%. 11 patients were in NYHA class II, 13 were in NYHA III, and 1 was in NYHA IV. 19 patients were taking beta-blockers, 21 ACE inhibitors or AII antagonists, 10 spironolactone and 17 were taking other diuretics. An informed consent was obtained from each patient and the study protocol conforms to the ethical guidelines of the 1975 Declaration of Helsinki as reflected in a priori approval by the institution's human research committee.

2.2. Measurements

2.2.1. Data acquisition

While changing the AV delay of a patients' cardiac resynchronization therapy (CRT) device, we recorded beat-by-beat left ventricular (LV) systolic blood pressure, left ventricular pulse pressure (defined as peak minus trough during the cardiac cycle) and left ventricular dp/dt_max, all of which were measured invasively from within the left ventricle using a 0.014-inch-diameter high fidelity pressure wire (Radi wire St. Jude Medical, Inc., St. Paul, MN, USA). The pressure wire was introduced via the radial approach in 23 patients and via the femoral approach in the remaining 2 patients.

A continuous ECG signal was recorded using a Dynascope DS-7100 (Fukuda Denshi USA) monitor. Analog output signals were taken via National instruments DAQ-Card AI-16E-4 (National Instruments, Austin, TX) and acquired in digital form using Labview (National Instruments, Austin, TX). They were analysed off line with custom software based on the Matlab platform (MathWorks, Natick, MA) [12].

2.2.2. AV delay optimization

We tested a range of AV delays; potentially these were 40, 80, 120, 160, 200, 240, 280, 320 and 350 ms. In practice, for each subject we did not study AV delays beyond the point at which conduction became purely intrinsic. Intrinsic rhythm was identified using a 3 lead ECG recording and by comparing morphology with that occurring during AAI pacing. The heart rate was kept constant throughout the study with atrial pacing at a rate of 90 bpm. The interventricular delay was left at 0 ms or as close to this as the device allowed. We tested each range of AV delays twice so that it was possible to determine the test–retest reproducibility for each of the tested parameters.

In order to identify the ‘optimal’ AV delay for each tested sequence of AV delays we applied a quadratic equation to identify the peak of the parabola and therefore the AV delay associated with the highest value of the measured hemodynamic parameter [13]. However, if the longest tested AV delay resulted in the highest hemodynamic value, then this was taken as the ‘optimal’ AV delay since in this situation the quadratic equation cannot be applied as it requires data points on either side of the peak in order to be reliable.

2.2.3. Tested protocols for using LV dp/dt_max for optimization

We recorded beat-by-beat invasive dp/dt_max from the left ventricle while adjusting the AV delay of the CRT device. We tested four different protocols for processing the dp/dt_max signal. For each tested protocol we determined the test–retest reproducibility with regard to the AV delay determined as optimal. We tested the following four protocols (Fig. 1):

• (1)Immediate absolute LV dp/dt_max: the mean LV dp/dt_max recorded during a window of 10 s commenced immediately after the transition to the tested AV delay.
• (2)Delayed absolute LV dp/dt_max: the mean LV dp/dt_max recorded during a window of 10 s commenced 30 s after the transition to the tested AV delay, i.e. permitting some time for stabilisation before recording.
• (3)Single relative LV dp/dt_max: the relative change in LV dp/dt_max measured with a single transition between a reference AV delay (120 ms) and the tested AV delay. This was calculated as the difference of the mean of the 10 beats immediately before a transition and the mean of the 10 beats immediately after.
• (4)Multiple relative LV dp/dt_max: the mean relative change in LV dp/dt_max calculated from a total of six transitions. For example, when testing 160 ms, the AV delay was changed from 120 to 160 to 120 to 160 to 120 to 160 to 120 ms, allowing at least 20 beats in each state. This provided 6 transitions between the states, composed of three transitions from 120 to 160 ms and three “reverse” transitions from 160 to 120 ms whose blood pressure effect we reversed in sign [11,13].

Fig. 1.

Description of the four tested protocols for processing the dp/dt_max signal for AV delay optimization. (a) Immediate absolute LV dp/dt_max protocol: mean of a 10 second recording commenced immediately after a transition to the tested AV delay (not compared to a reference setting). (b) Delayed absolute LV dp/dt_max protocol: mean of a 10 second recording commenced after a 30 second stabilisation period following the transition to the tested AV delay. (c) Single relative LV dp/dt_max protocol: the relative change in LV dp/dt_max measured with a single transition between a reference AV delay (120 ms) and the tested AV delay. (4) Multiple relative LV dp/dt_max: the mean relative change in LV dp/dt_max calculated from a total of six transitions to and from the tested AV delay (three forward transitions to the tested AV delay and three back transitions to the reference AV delay, reversing the sign for the back transitions).

2.2.4. Comparison of different invasive hemodynamic parameters

We compared three different invasive hemodynamic variables that might be used to guide AV delay optimization: left ventricular systolic blood pressure (SBP), pulse pressure (PP), and left ventricular dp/dt_max. For this part of the study we used the multiple relative protocol (mean of 3 forward and 3 back transitions between tested and reference AV delay).

We compared these different hemodynamic parameters with respect to reproducibility and precision.

2.3. Definition of reproducibility

In this paper, reproducibility of the AV delay optimum refers to whether the optimization process carried out with separate datasets identifies a similar AV delay as optimal. For each patient, one optimum (AV_opt1) was calculated from one dataset, and a second optimum (AV_opt2) from a second dataset. The difference from the first to the second (AV_opt1–AV_opt2) was calculated for each patient. The mean value of this difference is not of interest as there is no reason for the second optimum to be consistently longer or shorter than the first. It is the standard deviation of difference (SDD) which is a reflection of reproducibility: narrow SDD indicates good reproducibility (repeated optimizations yield similar AV optima) while wide SDD indicates poor reproducibility.

For completeness we also present R² values between first and second AV optima, although it should be recognised that R² is just as much a reflection of the width of the spectrum of AV optima in the population studied, as of the reproducibility within individuals.

2.4. Scatter and curvature, the determinants of the precision of the optimum

The more curved the hemodynamic response, the more precisely the optimum AV delay can be identified [1]. Conversely the noisier (more scattered) the measurements at each setting, the less precisely the optimum AV delay can be identified [1]. Scatter (Fig. 2, panel a) is quantified as the standard deviation of repeated measurement for the same AV delay (for convenience this can be averaged across different AV delays). Curvature of the biological response is calculated as the quadratic coefficient of the parabola describing the curved response of hemodynamic data to changes in AV delay (Fig. 2 middle panel b) [13].

Fig. 2.

Method for calculating scatter and curvature which are the biological determinants of the reproducibility of the optimum AV delay. Data from one patient is shown in order to describe how Scatter and Curvature can be calculated. Scatter is the spontaneous variability in measurements and is calculated as the standard deviation of the repeated measurements (shown in panel a). Curvature expresses how quickly the underlying measurement declines with distance away from the optimum. It is approximately parabolic and it is possible to calculate the underlying curvature by measuring the change in the acute hemodynamic parameter occurring over a known change in AV delay (panel b). The scatter:curvature ratio is calculated for each tested hemodynamic parameter in order to compare precision when used for AVD optimisation. The ratio between scatter and curvature is the biological determinant of the test–retest variability of the optimum: higher ratios indicate greater variability (panel c). SD: standard deviation.

The ratio between scatter and curvature is the biological determinant of the test–retest variability of the optimum: higher ratios indicate greater variability [1]. We compared the scatter:curvature ratio for each tested hemodynamic parameter in order to compare precision when used for AVD optimisation (Fig. 2 panel c).

2.5. Statistics

The test–retest reproducibility of the apparently optimal AV delay was calculated as the standard deviation of the difference between the two optima calculated from independent datasets. The correlation between the two optima was expressed as R². We used the F-test for equality of variances to test whether test–retest reproducibility was significantly different between methods of optimization. Paired comparisons were made using Student's paired t test. A p value of < 0.05 was taken as statistically significant. Microsoft Excel and Statview 5.0 (SAS Institute Inc., Cary, NC) were used to collate and analyse the data.

3. Results

3.1. Assessment of four different protocols for AV delay optimization using left ventricular dp/dt_max

3.1.1. Test–retest reproducibility of the AV optimum

All 25 patients underwent the full reproducibility studies of the immediate protocols. 15 of them also underwent reproducibility of the delayed protocol.

The immediate absolute protocol had a standard deviation of difference between replicate optima (SDD) of 41 ms and R² = 0.45. This means that on 95% of occasions, a second optimization would give an AV optimum within ± 82 ms of the first AV optimum. This 164 ms wide span means that knowing the result of one optimization gives virtually no clinically useful information regarding the likely result of a second optimization.

The delayed absolute protocol had a similarly wide SDD of 40 ms and R² = 0.5.

Reproducibility was improved in the protocols that used alternations, where the immediate change in dp/dt_max was measured relative to its value at a reference AV delay. The single relative protocol gave SDD = 27 ms and R² = 0.77. The multiple relative also had better same day reproducibility, with SDD = 23 ms and R² = 0.79 (Fig. 3).

Fig. 3.

Same day reproducibility of AV delay optimization, using the four tested protocols for processing left ventricular dp/dt_max. Line of equivalence (dashed line) is displayed.

SDD was significantly lower (i.e. reproducibility of the AV optimum is significantly better) for the multiple relative protocol than with the immediate absolute protocol (p < 0.01 by F-test) or the delayed absolute protocol (p < 0.01).

3.1.2. Biological variability (scatter) of dp/dt_max between successive measurements

Variability in the apparent optimum arose from natural biological variability in dp/dt_max values. Scatter between successive measurements at the same AV delay setting, calculated as the standard deviation of difference, was: immediate absolute 57 mm Hg/s, delayed absolute 105 mm Hg/s, single relative 60 mm Hg/s and multiple relative 49 mm Hg/s. This means that, for example, if a patient has at a particular setting on a first optimization a dp/dt_max measured by the standard (delayed absolute) method of 900 mm Hg/s, on the next optimization one can be 95% confident that the dp/dt_max will be in the range 900 ± 1.96 × 105 mm Hg/s, i.e. approximately 700 to 1100 mm Hg/s.

3.1.3. Comparison of the four tested dp/dt_max protocols with regard to the AV delay identified as optimal

The single relative method showed moderate agreement with the multiple relative algorithm R² = 0.58, SDD = 36 ms with regard to the AV delay identified as optimal. However, the other two protocols showed poor agreement with the multiple relative algorithm. For the immediate absolute protocol SDD = 66 ms and R² = 0.18, while for the delayed absolute protocol SDD = 66 ms and R² = 0.34 (Fig. 4). It should be noted that it is not sustainably possible for two methods to agree with each other any better than they agree with themselves [14].

Fig. 4.

Agreement with each other with regard to AV delay determined as optimal, for the 4 different protocols tested for processing LV dp/dt_max.

3.2. Comparison of three different invasive measures for AV delay optimization

3.2.1. Test–retest reproducibility

We compared test–retest reproducibility of three different invasive hemodynamic measures. All three tested measures were recorded simultaneously and we used an identical multiple relative protocol (mean relative change obtained from a total of 6 transitions) to process the hemodynamic data. Using this protocol all three measures showed similar reproducibility (Fig. 5). For LV dp/dt_max SDD 23 ms and R² = 0.79, for LV systolic blood pressure SDD = 16 ms and R² = 0.84, and for LV pulse pressure SDD = 21 ms and R² = 0.76 (p = NS for all).

Fig. 5.

Test–retest agreement with regard to the AV delay determined as optimal for three tested invasive hemodynamic measures. We used the multiple relative protocol to process the hemodynamic signals.

3.2.2. Drivers of precision of optimization: measurement scatter and curvature of response

Overall, dp/dt_max had a standard deviation of 28.8 mm Hg/s, and curvature of response 0.0034 mm Hg/s/ms², giving a scatter:curvature ratio of 8538 ms². LV systolic blood pressure had a measurement of scatter of 2.54 mm Hg and curvature of response 0.00056 mm Hg/ms², giving a scatter:curvature ratio of 4532 ms². For LV pulse pressure scatter was 2.59 mm Hg and curvature was 0.00034 mm Hg/ms² giving a scatter:curvature ratio of 7602 ms². dp/dt_max has a different physical dimension from the pressures, but both scatter and curvature reflect this, so that the scatter:curvature ratio is in ms², directly comparable between the three measures.

3.3. Relative importance of AV delay optimization with regard to the overall hemodynamic improvement observed with CRT

In addition to determining optimal AV delay, we also measured the acute change in hemodynamics (dp/dt_max, LV SBP and LV PP) when changing from intrinsic rhythm to CRT with a nominal AV delay of 120 ms. We then calculated the percentage increase in the three measured hemodynamic parameters, occurring as a result of ‘turning’ on CRT with a nominal AV delay. We compared this to the hemodynamic improvement achieved with CRT with an ‘optimal’ AV delay.

To prevent bias [1], we measured the size of the advantage of the optimal setting using completely separate data from that used to determine the optimal setting. In this way we were able to estimate the relative contribution AV delay optimization had on the overall maximal hemodynamic response to CRT.

The mean absolute values for the three measured hemodynamic parameters during AAI pacing at a heart rate of 90 bpm were LV systolic blood pressure 126 mm Hg, LV dp/dt_max 1055 mm Hg/s and LV pulse pressure 124 mm Hg.

3.3.1. Increment achieved by CRT at nominal AV delay

The mean percentage changes from baseline for these measures, with the onset of CRT with the nominal AV delay (AVD 120 ms) were: LV systolic pressure 2%, LV dp/dt_max 5% and LV pulse pressure 3%.

3.3.2. Unbiased estimate of increment achieved by CRT at optimal AV delay

Simply selecting the highest value of a set of noisy numbers and comparing it to a reference value, will produce an artefactual positive bias (since the value selected tends to be the one with the most positive noise). To avoid this bias, we defined the AV optimum with one dataset, and made measurements of effect size with a separate dataset. The percentage increase from a baseline of AAI pacing to the separately-determined optimal AV delay was 6%, 9% and 6% for LV systolic BP, LV dp/dt_max and LV pulse pressure respectively.

3.3.3. Increment contributed by AV optimization expressed as a percentage of the increment contributed by CRT at nominal AV delay

The extra increment AV delay optimization contributed over and above the increment given by nominal-AVD CRT can be expressed as a percentage of the increment given by nominal-AV CRT. On this scale, a value of 0% would indicate that AV optimization contributes nothing extra beyond nominal AV, and 100% would indicate that AV optimization doubles the impact of CRT at nominal AV.

For dp/dt_max AAI to CRT 120 ms raised the value by 60 mm Hg/s, and then CRT 120 ms to optimal AV raised the value by a further 42 mm Hg/s, therefore the additional improvement from AV optimization, expressed as a multiple of the CRT 120 ms effect, was 42/60 i.e. an additional 70%. For LV SBP, AAI to 120 ms raised pressure by 2.7 mm Hg, and then 120 ms to optimal raised it a further 4.3 mm Hg, therefore the additional increment from AV optimization expressed as a multiple of the CRT 120 ms effect, was 4.3 / 2.7 = 160%. For LV PP, AAI to 120 ms added 3.3 mm Hg, and 120 to optimal AV added a further 4.1 mm Hg, therefore the additional increment from AV optimization expressed as a multiple of the CRT 120 ms effect, was 4.1 / 3.3 = 124%.

3.4. Magnitude of reduction in dp/dt_max as a result of programming a non-optimal AV delay

We assessed the hemodynamic consequence of progressively changing the AV delay away from optimal. If we used the same dataset to select the optimum, as the dataset used to measure the increment obtained from optimization, that increment would be artefactually biased upwards (since it would disproportionately often contain a positive noise value). To eliminate this bias we used one dataset to identify optimal AV delay, and a numerically-separate dataset to measure the hemodynamic consequences of programming non-optimal versus optimal AV delay.

We found that the magnitude of reduction in dp/dt progressively increased the further from optimal that the AV delay was programmed. Even relatively small changes away from optimal resulted in significant reductions in acute dp/dt_max (Fig. 6). For example programming an AV delay 40 ms longer or shorter than optimal resulted in a significant fall in acute dp/dt_max. The mean reduction in dp/dt_max as a result of programming an AV delay 40 ms longer than optimal was − 27 mm Hg/s that is 45% of the hemodynamic benefit obtained from turning CRT on compared to baseline of AAI pacing (AAI to nominal AV delay of 120 ms = mean increase of 60 mm Hg/s).

Fig. 6.

Hemodynamic consequence of progressively programming the AV delay longer or shorter than optimal. In order to minimise bias we used separate measurements to identify the optimal AV delay from those used to measure the hemodynamic consequences of programming a longer or shorter AV delay.

4. Discussion

This study is a critical analysis of the fundamental properties of dp/dt_max when used for AV delay optimization. The measurement of dp/dt_max has been widely used as an invasive gold standard marker of ventricular performance for research and clinical environments where different pacing modes or configurations are tested. Although these findings are immediately relevant to AV optimization in CRT devices they are relevant to any protocol where hemodynamics are measured as part of a clinical investigation of drug therapy, or perturbations of CRT.

We found that, first, attempting to optimize AV delay using absolute measurements of dp/dt_max gives unsatisfactorily large variability between repeat optimizations of AV delay. Relative measurements of dp/dt_max, i.e. comparing measurements at a tested setting with measurements at a reference setting immediately beforehand, produce a more reproducible optimum. Second, signal-to-noise ratio, namely the relative sensitivity of a physiological variable to changes in AV delay versus random scatter, is higher for systolic pressure than for dp/dt_max. Third, AV delay optimization using the more reproducible method approximately doubles the acute hemodynamic (dp/dt_max) increase available from CRT at a nominal 120 ms setting. The, effect sizes of optimization were measured from an independent dataset to prevent the positive bias which occurs when increment from optimization is measured from the same dataset as the optimum is selected.

4.1. Processing the dp/dt_max signal to improve precision of optimization

We found that simply using an absolute measurement of dp/dt_max, without comparison to a reference AV delay, yields AV optima with poor test–retest reproducibility. This is the case regardless of whether dp/dt_max is recorded immediately after a transition to the tested AV delay or if the onset of the recording is delayed by 30 s to allow for ‘stabilisation’ of the dp/dt_max trace. This is an important finding as both of these methods are used in clinical practice for AV delay optimization. If the reproducibility is poor it is extremely unlikely that the optimization process will result in identification of the true ‘optimal’ value and indeed may identify a value which is worse than not performing an optimization at all.

There was a trend to improvement in reproducibility of the optimum when multiple replicates were averaged for each optimization, but even with 6 replicates, taking 1 min per optimization, the SD of difference was 23 ms. This means that if an optimum was on one occasion found to be 140 ms, then on a second occasion, 95% of the time it would lie between 140 ms − 46 ms and 140 ms + 46 ms. This constitutes a range of uncertainty 92 ms wide. If a clinician requires better precision than this, and wishes to use dp/dt_max, then more than 6 replicates would be required.

Clinicians and researchers designing protocols for optimization should start with their desired level of precision of the optimum and then use standard methods to ensure that their protocols deliver it [1]. Many widely used optimization protocols, even if conducted carefully, cannot deliver reproducible optima [1].

4.2. Drivers of precision of optimization, and choice between pressure and dp/dt_max

The driver of reproducibility of the optimum is the signal-to-noise ratio, namely the size of the changes caused by alterations in AV delay in relation to the size of the changes occurring spontaneously.

In this study we quantified signal as the curvature parameter of the quadratic fit of the hemodynamic response to adjusting AV delay, and noise by the standard deviation of the measurements at an unchanged AV delay. The advantage of this approach is it allows technologies to be compared between protocols that cover different ranges of AV delays.

Systolic pressure appears to show advantages over LV dp/dt_max, which suggests that in larger studies, optimization by systolic pressure might be expected to provide a more reproducible optimum than optimization by dp/dt_max. The lower signal-to-noise ratio for dp/dt_max occurs because even though the signal in dp/dt_max is a larger percentage of mean value than the signal in BP, the noise is a much larger percentage of mean value than the corresponding noise in BP.

4.3. Source of “noise”

In this study we use the term noise to encompass all sources of variability between one measurement and another, of a physiological variable such as LV dp/dt_max or pressure. From a statistical and practical point of view, all sources of such variability conspire to impair precision of identification of the AV optimum. Separating the sources might be useful if it leads to identifying strategies for reducing specific sources of noise. Our study does not attempt to separate the sources, but rather to quantify the composite size of noise and examine its implications for studies of AV optimization (and related fields such as lead position optimization).

We have previously observed that biological beat-to-beat, breath-to-breath variability is substantial, and occurs concordantly in simultaneous measurements made over 50 cm apart in the circulation, using separate physical principles (invasive solid-state pressure wire in proximal aorta versus servo-pressure system in finger cuff at finger) and recorded on independent equipment (Figure 1 of Reference 18 [15]).

4.4. Incremental contribution of optimization: method for measuring without bias

A weakness of any study that makes measurements at several settings, adopts the setting giving the highest measurement as the optimum, and then re-uses the same measures to determine the “benefit” of this setting, is that such a protocol artefactually inflates the “benefit” of the optimum setting [1]. This is particularly important if the noise is large or the differences observed in the measured parameter between the tested settings are small. This is because – in those circumstances – the measurement at the apparently optimal setting is likely to include a markedly positive error component (which is responsible for making it the apparent optimum) and therefore averaging the increments at these apparent optima across all patients tends to accumulate a series of positive errors rather than cancelling out positive with negative errors.

To prevent this bias (Illusion 2 and Figure 1 in Reference [1]) it is important to measure the size of the advantage of the optimum using completely separate data from the data used to decide which setting is the optimum. By this bias-resistant approach, we found that beyond the initial increment from switching on CRT at a nominal setting of 120 ms, the additional increment from AV optimization was 69% (dp/dt_max) to 160% (LV SBP).

4.5. How can optimization be important and unimportant?

If these figures are representative of the general population of patients receiving CRT then reproducible AV optimization would be expected to have a discernible clinical endpoint effect.

Some AV optimization protocols, tested in adequately-powered, bias-resistant fashion via large, randomized, controlled, externally-monitored trials, delivered no statistically significant effect on endpoints. What has not been reported is whether these tested algorithms are reproducible.

For example the SMART-AV study was not conducting reproducible, quantitative optimization of physiological response, but instead used two different methods [7]. One was a selection of AV delay based on qualitative judgement of transmitral Doppler pattern, whose blinded test–retest reproducibility was not reported and may be poor [16]. The other did not utilise physiological responses.

4.6. Magnitude of reduction in dp/dt _max as a result of programming a non-optimal AV delay

This study indicates that even small changes in the programmed AV delay away from optimal significantly reduce acute hemodynamics. Nearly 50% of the beneficial effect obtained by turning CRT on, is lost if the AV delay is programmed 40 ms longer or shorter than optimal. This suggests that optimization algorithms need to have a narrower within-patient confidence interval than ± 40 ms.

4.7. Reproducibility of invasive dp/dt optimization in the context of poor reproducibility of non-invasive optimization strategies

This study addresses only invasive optimization, which is an approach often used in research studies. In routine clinical practice, if optimization is carried out, the approach is usually non-invasive.

The most commonly used non-invasive method for AV optimization is echocardiographic transmitral Doppler. However, the blinded test–retest reproducibility of the optimum on two separate optimisations, is not known. We have previously abandoned plans to conduct blinded test–retest reproducibility of transmitral Doppler optimization when we found that even in the idealised situation of a just a single optimization dataset printed onto large sheets of paper, there is poor agreement between experts, and indeed poor agreement within the same experts re-asked a few minutes later [16,17]. There is therefore no possibility that blinded test–retest reproducibility of transmitral Doppler optimization by the methods classically described is good enough for it to be used as a reference standard for testing another optimization method.

Narrow within-patient confidence intervals of the optimum are more easily obtained using automatically quantifiable variables such as pressure, because several replicate measurements can easily be made at each setting [18]. The test–retest reproducibility of the best invasive optimization methods described in this study (SDD of replicate optima for LV SBP = 16 ms) is similar to that reported using some non-invasive methods (SDD of replicate optima = 14 ms [15]).

4.8. Clinical implications

Even performing invasive, intraventricular measurements does not grant immunity to biological noise, which is important to recognise in the design of optimization protocols. Obtaining data invasively from the ventricle eliminates some sources of measurement error, such as probe movement disturbing consistency of aortic outflow tract velocity time integral (VTI) during Doppler optimization. However the invasive approach does not eliminate genuine biological variability, and this is not trivial in magnitude compared to the size of genuine differences being studied between settings.

Determining an optimum AV delay by reproducible methods, and then (from separate data) calculating the increment in hemodynamics beyond CRT at AV 120 ms, suggests that optimization of AV delay delivers hemodynamic benefits about as large as that of AV 120 ms CRT itself. This may at first seem to contradict clinical data which have demonstrated clear symptom, remodelling and mortality benefits of CRT but not of optimization. However the methods used in those optimization studies have not yet been demonstrated to have narrow test–retest reproducibility and therefore should not be assumed to be delivering incontrovertible, reliable optima.

4.9. Study limitations

This study was designed to compare test–retest reproducibility of the optimization protocols assessed, and not designed to assess outcome measures such as symptoms or mortality of utilising these optimization protocols. It has adequate size and precision to answer the questions being addressed.

Ensuring that optimization protocols deliver reproducible optima is a crucial first step that must precede the design of clinical trials of the efficacy of optimization, because they will need to be very large in order to adequately test for clinical benefits. For example, even if the clinical benefits of reproducible optimization are approximately half those of standard CRT, then it might still be worthwhile (since inconvenience to the patient would be small compared to CRT implantation itself) but the trials required to demonstrate this would need to be 4 times the size of those designed to show clinical benefits of CRT implantation.

In this study we kept heart rate fixed at an elevated rate of 90 bpm. We did not attempt to assess the effect of heart rate on reproducibility, because it is already known that at lower heart rates, the hemodynamic response to adjusting AV delay is less curved and therefore it is harder to discern the optimum [11]. Because the test–retest reproducibility of the optimum is critically dependent on this curvature [18], at lower heart rates the test–retest reproducibility of the optimum should be wider. It should be noted that pacing at an elevated fixed heart rate, supine and at rest may not reflect clinical practice and therefore the AV delay determined as optimal in these circumstances may not be the same as in an ambulatory upright patient with a variably changing heart rate.

No data editing was carried out, because unblinded post-processing by researchers risks distorting findings [19], and even if carried out without bias might be difficult to replicate in exactly the same way in future experiments. Our algorithm addresses the beats immediately before and immediately after the change in setting, i.e. with no beats omitted. The reason for this is that these beats have the smallest opportunity for “drift noise” to develop between them, and because the “signal” is greatest in that early post-transition window of time. These considerations turn out to be quantitatively more important than the greater “between-beat noise” in that time window [20,21].

5. Conclusions

For AV delay optimization of CRT, invasive measurement of dp/dt_max provides confidence that physiological responses are being measured directly but this gold standard, alone, is not sufficient to guarantee precise optimisation, i.e. reproducible optima determined between test and retest with independent data. Spontaneous biological variability over time (noise) is not necessarily trivial in comparison to the genuine differences between settings (signal).

Improved reproducibility of the optimum is achieved when dp/dt_max measurements are calculated relative to immediately preceding measurements at a standard reference AV delay, and – most importantly –multiple measurements are made. Even with the 6 replicate measurements we made at each AV delay, the level of uncertainty is wide: SDD = 23 ms; 95% confidence interval of difference between two optimizations in same patient = ± 46 ms. Any level of precision may be obtained if enough replicates are measured [18], because these values will fall with the square root of the number of replicates. Clinicians and researchers designing protocols for optimization could start from a desired level of precision of the optimum, and then design their optimisation protocol to deliver it [18].

Signal-to-noise ratio is higher for systolic pressure than for dp/dt_max, which may confer advantages for ensuring reproducible AV optimization. Reproducible AV optimization provides elevation in the hemodynamic increment achievable from CRT beyond what is achieved from a CRT with nominal AVD of 120 ms: by approximately twofold.