Abstract
Background
Two methods of estimating reader variability (RV) in QT measurements between 12 readers were compared.
Methods
Using data from 500 electrocardiograms (ECGs) analyzed twice by 12 readers, we bootstrapped 1000 datasets each for both methods. In grouped analysis design (GAD), the same 40 ECGs were read twice by all readers. In pairwise analysis design (PAD), 40 ECGs analyzed by each reader in a clinical trial were reanalyzed by the same reader (intra‐RV) and also by another reader (inter‐RV); thus, variability between each pair of readers was estimated using different ECGs.
Results
Inter‐RV (mean [95% CI]) between pairs of readers by GAD and PAD was 3.9 ms (2.1–5.5 ms) and 4.1 ms (2.6–5.4 ms), respectively, using ANOVA, 0 ms (–0.0 to 0.4 ms), and 0 ms (–0.7 to 0.6 ms), respectively, by actual difference between readers and 7.7 ms (6.2–9.8 ms) and 7.7 ms (6.6–9.1 ms), respectively, by absolute difference between readers. Intra‐RV too was comparable.
Conclusions
RV estimates by the grouped‐ and pairwise analysis designs are comparable.
Keywords: thorough QT study, reader variability, study design, bootstrapping, data simulation
The E14 guidance of the International Conference on Harmonization of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH) mandates that thorough QT/QTc (TQT) studies should be designed to detect a mean placebo‐adjusted drug‐induced QTc prolongation of about 5 ms with a one‐sided 95% upper confidence interval of >10 ms.1 This threshold of regulatory concern (mean QTc prolongation by 5 ms) is relatively small in comparison to inter‐ or intra‐reader variability in QT interval measurement in ECGs. Variability in QT interval measurement, estimated by the standard deviation (SD) of difference between two measurements on the same set of ECGs, ranged from 7 to 23 ms for intra‐reader variability2, 3 and 0.7 to 37 ms for inter‐reader variability in some published studies.4 Variability in QT interval measurement arises due to difficulty in identifying the end of the T wave since it gradually merges with the baseline; the variability is higher in ECGs where the T wave is inverted, or has a low amplitude.5 Reader variability can be reduced significantly with training and experience,2, 3, 4 and the ICH E14 guidance recommends that a central ECG laboratory with trained readers should be used to analyze ECGs from TQT studies.1 Moreover, the ICH E14 guidance also requires that a proportion of ECGs from a TQT study should be reanalyzed for quantifying inter‐ and intra‐reader variability.1
For a thorough QT study, there are two ways of assessing reader variability. In one method (grouped analysis design), for a TQT study where 10,000 ECGs analyzed by 6 readers, 100 study ECGs are randomly selected and analyzed twice by all 6 readers (100 ECGs × 6 readers × 2 reads = 1200 additional analyses) which amounts to 12% additional work, but uses only 1% of the total ECGs in the study. Alternatively (pairwise analysis design), 100 ECGs read by Reader 1 during the TQT study are read again by Reader 1 (for intra‐reader variability) and also by Reader 2 (for inter‐reader variability), another set of 100 ECGs read by Reader 2 in the TQT study are read again by Reader 2 and by also by Reader 3, and so on. Thus, the same amount of additional work (100 ECGs × 6 readers × 2 reads = 1200 analyses) will be performed for reader variability analysis, but will now include 600 ECGs (i.e., 6% of the ECGs from the TQT study). While the first method estimates variability between the entire group of readers using a single set of ECGs, the second method uses different sets of ECGs to estimate variability between each pair of readers, making it logistically cumbersome to manage and makes statistical analysis more complex. However, by including a larger sample of ECGs from the study it may provide a better estimate of reader variability. Whether the estimates of reader variability by these two methods are comparable has not been studied. We conducted this simulation study to compare these two methods of assessing reader variability of QT interval measurement using real data from a central ECG laboratory to assess whether they provide comparable estimates of intra‐ and inter‐reader variability.
MATERIALS AND METHODS
Original ECG Data Used for Simulation Study
The present analysis is based on a set of 500 digital ECGs, recorded at a sampling frequency of 1000 Hz, which had been previously read by 12 experienced readers in the central ECG laboratory. These readers were Board‐certified cardiologists, who had undergone further training in reading digital ECGs using the ECG annotation software. Each ECG had been analyzed twice by each reader. For the second read (Read II), ECGs were read in a different sequence, after an interval of 4 weeks, to prevent “measurement recall.” About 20% of the 500 ECGs included ECGs with morphological abnormalities such as bundle branch block, ventricular hypertrophy, and rhythm abnormalities.
ECG Interval Measurement
The ECGs were annotated manually using on‐screen digital calipers using commercially available software to place the fiducial marks (CalECG version 1.3, AMPS. LLC, New York, NY, USA). QT and RR measurements were made on five consecutive complexes in lead II; in case measurement in lead II was not possible, lead V5 was used. The QT interval was measured from the onset of the QRS complex to the end of the T wave which was determined as the point of return of the T wave to the isoelectric baseline. In ECGs with U waves obscuring the end of the T waves, the end of the T wave was marked as the nadir between the T and U waves.
Simulation of Reader Variability Datasets by Bootstrapping
Simulations were performed by bootstrapping using SAS version 9.2 (SAS Institute, Cary, NC, USA). Datasets were created by selecting 40 ECGs from the 500 original ECGs using Simple Random Sampling With Replacement (SRSWR) procedure. The PROC SURVEYSELECT function of SAS with METHOD = URS was used for the resampling process. ECGs repeated within a sample, if any, were considered as different ECGs.
Grouped Analysis Design
A set of 40 ECGs was randomly selected from the set of 500 ECGs in the original dataset. The values of Read I and Read II for the 12 readers were extracted to create a dataset (Fig. 1). Each dataset contained 40 ECGs × 12 readers × 2 reads = 960 observations. The same process was repeated so that 1000 datasets of 40 ECGs each along with the Read I and Read II values for all 12 readers were created.
Figure 1.
Study design for the grouped analysis design for estimation of intra‐ and inter‐reader variability. A single set of 40 ECGs was randomly selected. All readers (Readers 1 to 12) analyzed this set of 40 ECGs twice (Read I and Read II). QT interval measurements from Read I of all readers was used to estimate inter‐reader variability while Read I and Read II of the same reader was used to estimate intra‐reader variability.
Pairwise Analysis Design
A set of 480 ECGs was randomly selected from the set of 500 ECGs such that first 40 ECGs analyzed twice by Reader 1 and once by Reader 2, the next 40 ECGs analyzed twice by Reader 2, and once by Reader 3, and so on for the 12 readers (Fig. 2). Each dataset therefore contained 40 different ECGs * 12 reader pairs = 480 ECGs × 2 reads by Reader A and one by Reader B = 1440 observations. The same process was repeated so that 1000 datasets of 480 ECGs and 1440 QT interval measurements were created.
Figure 2.
Study design for the pairwise analysis design for estimation of intra‐ and inter‐reader variability. Forty ECGs read by Reader 1 during the thorough QT study (Read I) were randomly selected to create Set A. Set A was then analyzed again by Reader 1 (Read II) for estimation of intra‐reader variability. Set A was also analyzed by Reader 2 (Read III). Difference between Read I and Read III was used to estimate inter‐reader variability and difference between Read I and Read II was used to estimate intra‐reader variability. Twelve such sets of 40 ECG (Sets A to L) were created, 1 for each reader and analyzed as described above. Thus 480 ECGs were used to estimate reader variability.
Statistical Methods
For the grouped analysis design, the mean of all 12 readers was used as the reference for each ECG and difference between each reader's measurements and this mean for each of the 40 ECGS in a sample were used to estimate inter‐reader variability.6, 7 The actual and absolute differences between these QT measurements and the reference values for each dataset2 were calculated and their SDs noted as a measure of inter‐reader variability for that dataset. Similarly, for intra‐reader variability, the actual and absolute difference between Read I and Read II by each individual reader for each ECG was calculated and their SDs noted as a measure of intra‐reader variability for that dataset. The mean of the SDs was obtained for each dataset. Next, the values of the 2.5th and 97.5th percentile of the SDs of intra‐ and inter‐reader variability from the 1000 simulated datasets were obtained to provide the 95% confidence limits of reader variability.
For the pairwise analysis design, difference between measurements by a pair of readers on the same ECG was used to estimate inter‐reader variability. The absolute and actual differences for the entire dataset were noted and their SDs were used as an estimate of overall inter‐reader variability for that dataset. Intra‐reader variability was estimated as the absolute and actual difference between the two measurements (Read I and Read II) by the same reader on the same ECG and the SD was noted as a measure of intra‐reader variability for that dataset. The 2.5th and 97.5th percentiles of the SDs of the 1000 simulated datasets were obtained to provide the 95% confidence limits of intra‐ and inter‐reader variability.
Thus, inter‐reader variability in grouped analysis design was estimated by comparing each reader's QT interval values with the reference value (mean of all readers), in pairwise analysis design, it was estimated by the difference between QT values of a pair of readers. To differentiate whether any differences in results of the two methods were due to the study design or due to difference in the method used for statistical analysis, data from grouped analysis design were also analyzed in a pairwise manner analogous to the method used for the pairwise analysis design. Thus, although all readers in the grouped analysis design had analyzed the same set of 40 ECGs, the measurements by Reader 1 was compared with Reader 2, Reader 2 was compared with Reader 3, Reader 3 with Reader 4, and so on.
In addition, data were also analyzed by a previously described three‐way interaction ANOVA model8 which used ECG (ECGs 1 to 40), ECG lead and reader (Readers 1 to 12) as factors for estimation of inter‐reader variability and ECG, ECG lead and read (Read I or Read II) as factors for intra‐reader variability.8 The square root of the residual variance from this ANOVA model gives an estimate of the overall reader variability (similar to SD from the other methods).
RESULTS
Inter‐Reader Variability
The mean of the mean actual difference of inter‐reader variability for the 1000 datasets was 0 ms by grouped analysis design, pairwise analysis design, and for data from grouped analysis design analyzed by pairwise comparison. However, this does not mean that the reader variability was 0 as in actual differences the negative and positive values cancel each other. The 95% confidence limits ranged from −0.8 to 0.7 ms. The mean of the SDs of the actual difference was 8.2 ms (95% CI: 5.8–11.5 ms) for grouped analysis design, which was lower than 12 ms with the pairwise analysis design (95% CI: 9.3–15.6 ms). While the grouped analysis design compared each reader's value with a reference value which was the average of 12 readers’ measurements, pairwise analysis design compared differences in measurements between a pair of readers. Therefore, the difference in SDs between the grouped analysis design and pairwise analysis design could have occurred because of the statistical method rather than the study design. To explore this possibility, data for grouped analysis design were analyzed again by pairwise comparison. Now, the mean of the SDs of the actual difference was 12 ms (95% CI: 8.4–18.5 ms), which was similar to the results for pairwise analysis design (Table 1).
Table 1.
Estimates of Inter‐Reader Variability in QT Interval Measurement (Milliseconds) Obtained by Two Methods of Estimating Reader Variability in a Thorough QT Study
| Mean of 1000 Samples | SD of 1000 Samples | |||||||
|---|---|---|---|---|---|---|---|---|
| 95% | 95% | 95% | 95% | |||||
| Mean | SD | LCLa | UCLa | Mean | SD | LCLa | UCLa | |
| Actual Difference | ||||||||
| Grouped analysis design | 0 | 0.5 | −0.8 | 0.7 | 8.2 | 1.4 | 5.8 | 11.5 |
| Pairwise analysis design | 0 | 0.3 | −0.7 | 0.6 | 12 | 1.6 | 9.3 | 15.6 |
| Data from grouped analysis design analyzed as in pairwise analysis design | 0 | 0.2 | −0.5 | 0.4 | 12 | 2.5 | 8.4 | 18.5 |
| Absolute Difference | ||||||||
| Grouped analysis design | 5.3 | 0.6 | 4.3 | 6.7 | 6.2 | 1.4 | 3.9 | 9.6 |
| Pairwise analysis design | 7.7 | 0.6 | 6.6 | 9.1 | 9.2 | 1.7 | 6.5 | 12.8 |
| Data from grouped analysis design analyzed as in pairwise analysis design | 7.7 | 0.9 | 6.2 | 9.8 | 9.1 | 2.5 | 5.5 | 15.8 |
| Three‐way interaction ANOVA method | Not Applicable | |||||||
| Grouped analysis design | 4.2 | 0.5 | 3.3 | 5.1 | ||||
| Pairwise analysis design | 4.1 | 0.7 | 2.6 | 5.4 | ||||
| Data from grouped analysis design analyzed as in pairwise analysis design | 3.9 | 0.8 | 2.1 | 5.5 | ||||
Grouped analysis design compares each reader's measurements with a reference value (mean of all readers) while pairwise analysis design compares differences between measurements made by a pair of readers.
The 95% confidence limits are values of the 2.5th and 97.5th percentiles of the 1000 simulated samples.
The mean of the mean absolute difference of inter‐reader variability for the 1000 datasets for grouped analysis design was 5.3 ms (95% CI: 4.3–6.7 ms), which was lower than that for pairwise analysis design (mean 7.7 ms; 95% CI: 6.6–9.1 ms). However, when data of grouped analysis design were analyzed by pairwise comparison, the mean inter‐reader variability was 7.7 ms (95% CI: 6.2–9.8 ms) which was similar to that obtained in pairwise analysis design. The mean of the SD of the absolute difference in inter‐reader variability too showed similar results (Table 1).
Using the three‐way interaction ANOVA model, the inter‐reader variability estimates for grouped analysis design (mean 4.2 ms; 95% CI: 3.3–5.1 ms), pairwise analysis design (mean 4.1 ms; 95% CI: 2.6–5.4 ms), and data from grouped analysis design analyzed by pairwise comparison (mean 3.9 ms; 95% CI: 2.1–5.5 ms) were comparable (Table 1).
Intra‐Reader Variability
The mean of the mean actual difference of intra‐reader variability for the 1000 datasets was close to 0 ms for all three methods of analysis; the 95% confidence limits too were identical (Table 2). The mean of the SDs of the actual difference was 8.4 ms for both grouped and pairwise analysis designs, and 8.5 for data from grouped analysis design analyzed by pairwise comparison; the 95% CI for all three methods were approximately 6.0 to 12.0 ms.
Table 2.
Estimates of Intra‐Reader Variability in QT Interval Measurement (Milliseconds) Obtained by Two Methods of Estimating Reader Variability in a Thorough QT Study
| Mean of 1000 Samples | SD of 1000 Samples | |||||||
|---|---|---|---|---|---|---|---|---|
| 95% | 95% | 95% | 95% | |||||
| Mean | SD | LCLa | UCLa | Mean | SD | LCLa | UCLa | |
| Actual difference | ||||||||
| Grouped analysis design | 0 | 0.4 | −0.8 | 0.7 | 8.4 | 1.5 | 6.0 | 11.6 |
| Pairwise analysis design | −0.1 | 0.4 | ‒0.8 | 0.7 | 8.4 | 1.2 | 6.3 | 10.9 |
| Data from grouped analysis design analyzed as in pairwise analysis design | −0.1 | 0.4 | −0.8 | 0.7 | 8.5 | 1.6 | 6.1 | 12.0 |
| Absolute Difference | ||||||||
| Grouped analysis design | 5.1 | 0.5 | 4.2 | 6.2 | 6.6 | 1.6 | 4.1 | 10.1 |
| Pairwise analysis design | 5.1 | 0.4 | 4.4 | 6.0 | 6.6 | 1.3 | 4.4 | 9.2 |
| Data from grouped analysis design analyzed as in pairwise analysis design | 5.2 | 0.5 | 4.3 | 6.3 | 6.7 | 1.6 | 4.2 | 10.3 |
| Three‐way interaction ANOVA method | Not Applicable | |||||||
| Grouped analysis design | 4.3 | 0.5 | 3.6 | 5.4 | ||||
| Pairwise analysis design | 4.3 | 0.4 | 3.7 | 5.3 | ||||
| Data from grouped analysis design analyzed as in pairwise analysis design | 4.4 | 0.6 | 3.6 | 5.9 | ||||
The 95% confidence limits are values of the 2.5th and 97.5th percentiles of the 1000 simulated samples.
Similarly, the mean of the mean absolute difference of intra‐reader variability and its 95% confidence limits for the 1000 datasets was almost identical for all three methods. The mean of the SDs of the absolute difference were approximately 6.6 ms for three methods with the 95% CI ranging from 4.1 to 10.3 ms (Table 2).
Using the three‐way interaction ANOVA model, the intra‐reader variability estimates by the grouped analysis design (mean 4.3 ms; 95% CI: 3.6–5.4 ms), pairwise analysis design (mean 4.3 ms; 95% CI: 3.7–5.3 ms) and for data from grouped analysis design analyzed by pairwise comparison (mean 4.4 ms; 95% CI: 3.6–5.9 ms) were comparable (Table 2)
DISCUSSION
In the absence of a standardized method of assessing reader variability two study designs are commonly used in central ECG laboratories which employ large numbers of trained expert readers to perform measurement of various intervals in the ECG including the QT interval. We compared the estimates of inter‐ and intra‐reader variability obtained by these two study designs in this simulation study. In the grouped analysis design, a single set of 40 ECGs were read by all 12 readers twice and the same ECGs were used to compare variability between all the readers. In the pairwise analysis design, a set of 40 ECGs were read by reader 1 twice and once by reader 2, another set of 40 ECGs were read twice by reader 2 and once by reader 3, and so on for all 12 readers. Thus, although 40 different ECGs were used to estimate variability between each pair of readers in pairwise analysis design and a greater number of ECGs (480 different ECGs) were used to obtain reader variability estimates. We simulated 1000 samples of data by bootstrapping from an original dataset where 500 ECGs had each been analyzed twice by 12 trained readers in a core ECG laboratory. The ECGs in the present study included abnormal ECGs too, which may result in greater inter‐reader variability than is expected in TQT studies, most of which are performed in healthy volunteers with normal ECGs.5
There is wide variability in the calculation and reporting of reader variability in published studies. Some studies have reported the mean of actual differences in measurements between readers (or between 2 reads by the same reader for intra‐reader variability) for a set of ECGs.9, 10, 11, 12, 13 Assuming that the difference between readers is random, Reader A's QT measurements in some ECGs will be longer those of Reader B and shorter in others. Therefore, a mean difference of zero does not mean that their measurements are identical. In this case, the SD would provide a better estimate of reader variability, but this is reported in very few studies.2, 9, 13 On the other hand, if Reader A consistently made measurements which were longer than those of reader B, the mean of differences would be a positive number. Thus, systematic difference in measurement would be estimated by the mean difference, variability in measurements by the SD, while both together would provide estimate of inter‐reader variability.
Reader variability estimates for QT interval measurements for each of the 12 readers in a sample of 40 ECGs analyzed in the grouped analysis design (Table 3) were consistent with that reported by previous authors.12, 13 When we studied the actual difference as a measure of inter‐reader variability, we found that the mean of the mean actual difference for the 1000 samples by grouped analysis design and pairwise analysis design were 0 with the 95% confidence limits ranging from −0.8 to 0.7 ms, indicating absence of a systematic difference. However, the mean of the SDs of the 1000 samples was 8.2 ms for the grouped analysis design and 12.0 ms for the pairwise analysis design. This difference could have occurred due to two possible reasons—a difference in the study design, or a difference in the method of statistical analysis. In grouped analysis design, since all readers analyzed the same ECGs, and there is no gold standard for QT interval measurement (which is subjective) the average of QT measurements by all for each ECG was considered as the reference value, and the difference between individual readers’ values and the reference value was used to estimate inter‐reader variability. In pairwise analysis design, however, since each ECG is read by only one pair of readers, the difference between this pair of measurements is used to estimate inter‐reader variability. Therefore, intuitively, the estimate from grouped analysis design would be expected to be lower than that of pairwise analysis design due to the method used to compute reader variability. We therefore analyzed data from the grouped analysis design again such that QT measurements by Reader 1 were subtracted from those by Reader 2, measurements by reader 3 from those by Reader 2, and so on, despite the fact that the same 40 ECGs were analyzed by all 12 readers. We now found that the mean and SD of the SDs of 1000 samples from the grouped analysis design closely matched those by the pairwise analysis design, implying that when identical methods are used to compute inter‐reader variability, both study designs provided similar estimates.
Table 3.
Reader Variability Estimates for QT Interval Measurements Made by 12 Trained Readers in a Set of 40 ECGs
| Intra‐Reader Variability (ms) | Inter‐Reader Variability (ms) | |||
|---|---|---|---|---|
| Reader | Actual Difference | Absolute Difference | Actual Difference | Absolute Difference |
| Mean ± SD | Mean ± SD | Mean ± SD | Mean ± SD | |
| 1 | −0.7 ± 3 | 2.1 ± 2.2 | −3.6 ± 6.9 | 5.8 ± 5.1 |
| 2 | 0.4 ± 3.6 | 2.7 ± 2.3 | 1.9 ± 7.8 | 5.5 ± 5.8 |
| 3 | −1.0 ± 4.2 | 2.7 ± 3.4 | 0.4 ± 7.8 | 5.4 ± 5.6 |
| 4 | −0.9 ± 3.9 | 3.2 ± 2.4 | 4.1 ± 7.1 | 6.0 ± 5.5 |
| 5 | −0.6 ± 3.3 | 2.4 ± 2.3 | 0.7 ± 6 | 4.3 ± 4.3 |
| 6 | 1.1 ± 3.3 | 2.8 ± 2.2 | −2.5 ± 7 | 5.3 ± 5.2 |
| 7 | 0.7 ± 3.5 | 2.5 ± 2.6 | 0.3 ± 6.6 | 4.4 ± 4.9 |
| 8 | 0.8 ± 4.4 | 3.0 ± 3.4 | 4.4 ± 6.9 | 6.2 ± 5.3 |
| 9 | −0.6 ± 3.6 | 2.8 ± 2.3 | −1.8 ± 7 | 4.6 ± 5.5 |
| 10 | −1.0 ± 3.8 | 2.6 ± 3 | −2.6 ± 6.5 | 5.1 ± 4.6 |
| 11 | 0.3 ± 4.8 | 3.6 ± 3.2 | 2.1 ± 6.4 | 4.6 ± 4.9 |
| 12 | 0.1 ± 4.4 | 3.3 ± 2.9 | −3.9 ± 7.4 | 6.3 ± 5.5 |
Some studies have reported the mean and SD of the absolute differences between readers.2, 14, 15 Here, the mean would provide a reasonable estimate of inter‐reader variability.2, 15 Using absolute differences too, we found that the mean of means as well as SDs of the 1000 samples were higher for the pairwise analysis design compared to the grouped analysis design. However, when data from grouped analysis design were analyzed by pairwise comparisons, the estimates were similar.
Since both these methods are simplistic and do not take into account that the fact that 40 ECGs are read by the same reader, or that in the grouped analysis design, the same ECG is analyzed by 12 different readers, an ANOVA method has been described which uses the interaction between ECG, readers and the Lead used to measure QT interval to provide an overall estimate of reader variability.8 Using this method, the mean of SDs of the 1000 samples were comparable for grouped and pairwise analysis design and for data from the grouped analysis design analyzed by pairwise comparisons.
For estimation of intra‐reader variability the same reader analyzed the same ECG twice in the grouped and the pairwise analysis designs. Therefore, in both methods, actual or absolute differences between the two readings were computed and not the difference between readings and a reference value. Although the same 40 ECGs were used to measure intra‐reader variability for all readers and 40 different ECGs were used to estimate reader variability for each reader, our simulation study showed that the estimates of intra‐reader variability obtained by the grouped and the pairwise analysis designs were comparable for the actual difference, absolute difference and by ANOVA.
Limitations
Results in this study were based on QT interval measurements made by the threshold method in single ECG lead (lead II). If the QT interval is measured on the superimposed median beat or in another lead by the tangent method, the estimates of reader variability might differ from the estimates reported in this study. Second, 20% of ECGs in the present study had morphological ECG abnormalities like bundle branch blocks, ventricular hypertrophy or rhythm disturbances to make them representative of ECGs seen in actual clinical trials. These abnormalities may affect the estimates of reader variability reported in this study. Nevertheless, the estimates of reader variability obtained by the pairwise and grouped analysis designs would be affected to a similar extent. Therefore, reader variability estimates by the two study designs would still be comparable as long as the same ECGs are used in the comparison and the QT interval is measured by the same method in the same lead.
CONCLUSIONS
This simulation study shows that estimates of inter‐ and intra‐reader variability obtained by grouped and pairwise analysis design are comparable, provided that identical methods of computation are used. Both the grouped and the pairwise analysis designs have a place in estimating reader variability. The grouped analysis design is more suitable for estimating the reader variability for a group of readers in a core ECG laboratory as a periodic measure of quality control since it also permits benchmarking of individual readers and helps identify readers with outlier values who may require retraining.4 On the other hand, pairwise analysis design may be more suitable for estimating study‐specific reader variability in a thorough QT study, as it uses a higher proportion of ECGs from the study for variability estimation utilizing the same amount of resources (time and cost of reanalysis) making the estimates more representative of the study data, just as a larger sample is more likely to be representative of the population. However, it must be realized that depending on the method of computation used, estimates of inter‐reader variability by the pairwise analysis design may be on an average 50% greater than those with grouped analysis design when actual or absolute differences are reported, while estimates of intra‐reader variability will be identical. Finally, use of the ANOVA model gives comparable estimates of reader variability with both study designs and also avoids the ambiguity arising from use of actual and absolute differences.
Acknowledgments
The authors are grateful to Dr. Michael O'kelly, Senior Strategic Biostatistics Director, Quintiles Center for Statistics in Drug Development, for his technical assistance in the interpretation of the study results.
AUTHORS’ DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST
Employment: Vaibhav Salvi, Gopi Krishna Panicker, Vaibhav Kerkar, Mili Natekar, and Snehal Kothari are/were employees of Quintiles Cardiac Safety Services, Mumbai, India.
Consultant or Advisory Role: Dilip Karnad is Consultant to Quintiles Cardiac Safety Services, Mumbai, India.
Stock Ownership: None. Honoraria: None. Research Funding: None. Expert Testimony: None. Other Remuneration: None.
REFERENCES
- 1. International Conference on Harmonization of Technical Requirements for Registration of Pharmaceuticals for Human Use . The Clinical Evaluation of QT/QTc Interval Prolongation and Proarrhythmic Potential for Non‐Antiarrhythmic Drugs: E14. Geneva, Switzerland: International Conference on Harmonization of Technical Requirements for Registration of Pharmaceuticals for Human Use; 2005. Available at: http://www.fda.gov/downloads/RegulatoryInformation/Guidances/ucm129357.pdf. Accessed 8 Jun 2012.
- 2. Panicker GK, Karnad DR, Natekar M, et al. Intra‐ and inter‐reader variability in QT interval measurement by tangent and threshold methods in a central electrocardiogram laboratory. J Electrocardiol 2009;42:348–352. [DOI] [PubMed] [Google Scholar]
- 3. De Groote K, Suys B, Deleeck A, et al. How accurately can QT interval be measured in newborn infants? Eur J Pediatr 2003;162:875–879. [DOI] [PubMed] [Google Scholar]
- 4. Panicker GK, Karnad DR, Joshi R, et al. Z‐score for benchmarking reader competence in a central ECG laboratory. Ann Noninvasive Electrocardiol 2009;14:19–25. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Shetty S, Khan M, Salvi S, et al. Do electrocardiography characteristics predict variability in QT measurements in clinical trials? (Abstract) Indian Heart J 2005;57:493. [Google Scholar]
- 6. Rosillon D, Astruc B, Hulhoven R, et al. Effect of brivaracetam on cardiac repolarization–a thorough QT study. Curr Med Res Opin 2008;24:2327–2337. [DOI] [PubMed] [Google Scholar]
- 7. Hulhoven R, Rosillon D, Bridson WE, et al. Effect of levetiracetam on cardiac repolarization in healthy subjects: A single‐dose, randomized, placebo‐ and active‐controlled, four‐way crossover study. Clin Ther 2008;30:260–270. [DOI] [PubMed] [Google Scholar]
- 8. Natekar M, Mahajan V, Satra a O'Kelly M, et al. Use of ANOVA to estimate inter‐ and intra‐reader variability for a group of readers in thorough QT/QTc studies. Clin Pharmacol Ther 2008;83:489–491. [DOI] [PubMed] [Google Scholar]
- 9. Christov I, Dotsinsky I, Simova I, et al. Dataset of manually measured QT intervals in the electrocardiogram. Biomed Eng Online 2006;5:31–38. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Tutar HE, Ocal B, Imamoglu A, et al. Dispersion of QT and QTc interval in healthy children, and effects of sinus arrhythmia on QT dispersion. Heart 1998;80:77–79. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Kyriazis J, Pikounis V, Smirnioudis N. Use of the QTc interval and QTc dispersion in patients on haemodialysis: Assessment of reproducibility. Nephrol Dial Transplant 2004;19:516–517. [DOI] [PubMed] [Google Scholar]
- 12. The CSE Working Party . Recommendations for measurement standards in quantitative Electrocardiography. Eur Heart J 1985;6:815–825. [PubMed] [Google Scholar]
- 13. Willems JL, Arnaud P, van Bemmel JH, et al. Establishment of a reference library for evaluating computer ECG measurement programs. Comput Biomed Res 1985;18:439–457. [DOI] [PubMed] [Google Scholar]
- 14. Poulsen PL, Ebbehøj E, Arildsen H, et al. Increased QTc dispersion is related to blunted circadian blood pressure variation in normoalbuminuric type 1 diabetic patients. Diabetes 2001;50:837–842. [DOI] [PubMed] [Google Scholar]
- 15. Kittnar O, Paclt I, Mlcek M, et al. QT dispersion and electrical heart field morphology in patients treated with dosulepin. Physiol Res 2004;53:379–386. [PubMed] [Google Scholar]