How to Assess the Quality of Glucose Clamps? Evaluation of Clamps Performed With ClampArt, a Novel Automated Clamp Device

Abstract

Background:

There are no widely accepted parameters to assess the quality of glucose clamps. Thus, we selected different parameters describing clamp quality. These parameters were then evaluated in glucose clamps carried out with ClampArt, a novel CE-marked, state-of-the-art fully automated glucose clamp device employing continuous blood glucose (BG) measurements and minute-by-minute adaptations of glucose infusion rate (GIR).

Methods:

Thirty-nine glucose clamps were performed in 10 healthy and 29 subjects with type 1 diabetes (T1DM) (total duration 583 h). ClampArt-based BG measurements were compared with those obtained with a laboratory reference method. Clamp quality was assessed by 5 parameters: (1) difference (mg/dl) of all paired BG measurements of ClampArt versus reference method (“trueness”), (2) coefficient of variation (CV, %) of ClampArt’s BG measurements at target clamp level (“precision”), (3) mean absolute relative difference (MARD, %) at target clamp level (“accuracy”), (4) difference (mg/dl) between ClampArt and target BG (“control deviation”), and (5) percentage operational time (“utility”).

Results:

ClampArt-based BG measurements showed a trueness of 1.2 ± 2.5 mg/dl. CV and MARD at target BG were 5.5 ± 2.1% and 5.3 ± 2.3%, respectively. There were only small deviations from target level (1.2 ± 1.6 mg/dl). Operational time was as high as 95.4% ± 4.1% (means ± SD).

Conclusions:

The selected parameters seem to be adequate to characterize clamp quality. The novel, fully automated clamp device ClampArt achieves high clamp quality, which in future trials should be compared with other (automated and manual) clamp methods.

Clinical glucose clamp trials have frequently been used for the evaluation of the pharmacokinetic/pharmacodynamic (PK/PD) properties of different kinds of insulin formulations^1-12 or for the determination of insulin sensitivity.^13,14 Both automated clamps (by means of the Biostator)^1-12 and manual clamps have been used for insulin PK/PD assessments. The metabolic effect of the investigated insulin is characterized by the glucose infusion rate (GIR) needed to keep BG constant at a predefined target BG concentration during a euglycemic glucose clamp. In automated clamps BG is measured continuously and GIR is adapted every minute—far more often than in manual clamps, in which GIR adjustments are performed “only” every 3-10 minutes. Automated glucose clamps, therefore, should have a better control quality because of minute-by-minute adjustments of GIR, however, no head-to-head comparison between automated and manual clamps has been published so far. In addition, with automated clamps there is no room for bias caused by the clamper,¹⁵ whereas there is a theoretical possibility that with manual clamps a clamper (instinctively) modifies the GIR in anticipation of a certain PD effect or follows certain patterns in GIR-modifications rather than reacting to the actually measured BG levels. Until now the Biostator (developed in the early 1970s by Life Sciences Instruments, Elkhart, IN)¹⁶ and the STG-22^TM/STG-55^TM (a device developed by Nikkiso, Tokyo, Japan)^17-19 have been the only devices used for automated glucose clamps.

The Biostator uses outdated technology, and it is difficult to get spare parts, while the STG-22^TM has not been available in Europe. Thus, Profil in Neuss, Germany, has developed ClampArt, a novel fully automated glucose clamp device which received CE-mark in November 2012. This device is only used at the Profil sites in Germany, and no commercialization of the device is intended.

Despite the high standardization of glucose clamp trials there have been divergent results, for example, concerning the time-action profiles of insulin detemir and insulin glargine.⁶ One possible explanation for these differences could relate to differences in glucose clamp quality. Deviations from the BG target clamp level actually imply that a wrong GIR was used. Small deviations of short duration are inevitable—these should not lead to big differences in GIR. That might be a different matter, however, with bigger deviations of longer duration. The quality of glucose clamps has also hardly ever been published and there are no established parameters available.

Therefore, we selected and used own quality parameters that were investigated in the first clinical trial with ClampArt and compared to literature data, if available, for other clamp methods. Further clinical trials with ClampArt are currently ongoing.

Subjects and Methods

Here we present the first clinical trial comprising of 39 glucose clamps carried out with ClampArt. The study was conducted according to the International Conference on Harmonisation (ICH) and the national (German) Medical Device Law (MPG), was approved by the Ethics Committee (Ethics Committee of the Medical Association North-Rhine, Düsseldorf, Germany) and the national competent authority (Bundesinstitut für Arzneimittel und Medizinprodukte [BfArM], Bonn, Germany) and conducted from October 2011 to April 2013. All participants provided written informed consent prior to any study related activities.

To test the novel device under different circumstances there were 2 experimental settings for the glucose clamps: The first group of 10 subjects with T1DM underwent a 12-hour euglycemic glucose clamp (plus a 1- to 6-hour initial stabilization period) at a target BG clamp level of 100 mg/dl after a single subcutaneous (SC) dose of 0.2 units (U)/kg of insulin aspart (Novo Nordisk A/S, Denmark) in the abdominal region by means of a syringe. In the second setting 29 healthy subjects or subjects with T1DM were included. The clamps performed varied concerning (1) clamp duration (8 to 48 hours, including initial stabilization period), (2) target clamp level: 70 to 100 mg/dl, and (3) insulin dose: maximum dose of 0.6 U/kg insulin aspart or 0.4 U/kg NPH insulin (eg, Protaphane, Novo Nordisk A/S, Denmark) or up to 0.7 U/kg of a basal insulin analog (eg, insulin glargine, Sanofi, France). Clamps were stopped, if BG rose to levels > 100 mg/dl above the clamp level. To assess the quality of BG measurements of ClampArt venous blood samples were taken every 15 minutes and assessed with a reference laboratory method (Super GL, Dr Müller Gerätebau, Freital, Germany).

The Device

During glucose clamps the subject’s BG is kept at a predefined target level (“clamp level”). To obtain a good clamp quality, BG has to be determined frequently and the GIR needed to keep BG as close to the target level as possible has to be adapted frequently as well. Therefore the clamp device consists of a continuous BG measurement unit, a glucose delivery unit (infusion pumps) and a control unit (computer) (Figure 1). The algorithm implemented in the latter calculates the appropriate GIR based on the BG readings.

Figure 1.

Schematic presentation of a glucose clamp.

The ClampArt device (Figure 2) consists of 2 parts: the central unit based on a mobile cart and the infusion pumps at a remote docking station. The infusion pumps are controlled by the central unit of the ClampArt device via a cable connection. The central unit contains all the equipment for glucose measurements and a computer with the ClampArt software running. A touchscreen monitor on top of the device serves as the user interface. All important data are displayed on the monitor and stored electronically. The machine is licensed for the use by trained professionals who need to successfully complete a structured training program before they can safely operate the device. Usually, each trained study nurse takes care of 2 parallel clamps with ClampArt. In addition, a physician is on call for medical emergencies whenever clamps are performed in our institute. No technicians or other staff is needed to safely operate the ClampArt device.

Figure 2.

The ClampArt device.

For a continuous glucose measurement employing ClampArt a small amount of blood from a peripheral vein of the subject is pumped continuously to a glucose sensor. ClampArt automatically calculates the GIR to keep the BG at a predefined level every minute using the Biostator algorithm²⁰ and infuses the calculated amount of glucose.

One major improvement of ClampArt in comparison to the Biostator is the consideration of the blood dilution when determining the glucose concentration in the diluted solution at the measuring unit. This makes the measurement independent of fluctuations of the blood percentage in the solution.

Parameters of Clamp Quality and Utility

In the present study the following parameters (Figure 3) were evaluated to characterize the clamp quality during the clamps carried out with ClampArt:

Figure 3.

Parameters of clamp quality.

• Mean difference of all paired BG measurements of ClampArt (= device) versus reference method (ref. meth.): “trueness”

• Coefficient of variation (CV, %) of ClampArt’s BG measurements at the target BG level: 100 × (SD device/mean device): “precision”

• Mean absolute relative difference (MARD, %) at the target BG level: 100 × |(ref. meth.- device)/ref. meth.|: “accuracy”

• Mean difference (mg/dl) between ClampArt (device) and target BG level: “control deviation”

• Percentage operational time: 100 × (operational time / total time; “utility”), excluding any periods when the device was not in the regular operational mode, for example, caused by problems with tubing sets because of coagulation in the sampling line.

In addition an error grid²¹ of all paired data points of the clamps was applied.

Safety Parameters

Safety was assessed by physical examination, observation of vital signs, safety lab, ECG, and monitoring of adverse events (AEs)/serious adverse events (SAEs) and adverse device effects (ADEs)/serious adverse device effects (SADEs).

Results

Subjects’ demographics (age, gender, BMI, diabetes duration) as well as information on the clamps (kind and dose of insulin used, target clamp level) are provided in Table 1. In total 39 glucose clamps in 10 healthy subjects (5 males, age 39.1 ± 6.6 years, BMI 24.7 ± 2.4 kg/m²) and 29 subjects with T1DM (23 males, 42.0 ± 11.0 years, 24.7 ± 2.1 kg/m², diabetes duration 18.4 ± 11.3 years; means ± SD) were performed by means of ClampArt. Two clamps had to be terminated early due to technical problems: One was due to insufficient sensor preparation leading to huge drifts on the sensor signal (clamp no. 16, T1DM). The other one was caused by massive problems with tubing sets conditional of manufacturing (clamp no. 31, T1DM). The remaining 37 clamps resulted in a total clamp duration of approximately 583 hours. Typical examples of clamps performed with ClampArt are provided in Figure 4 (a clamp with insulin aspart and insulin glargine, respectively).

Table 1.

Demographics and Clamp Information.

Clamp no.	Subject	Age (years)	Gender (M/F)	BMI (kg/m2)	Diabetes duration (years)	Insulin used	Total dose (U)	Dose (U/kg)	Clamp level (mg/dl)
1	T1DM	31	F	21.6	23	Insulin aspart	12	0.21	100
2	T1DM	56	M	21.1	42	Insulin aspart	14	0.20	100
3	T1DM	31	F	26.6	12	Insulin aspart	16	0.20	100
4	T1DM	42	M	25.0	23	Insulin aspart	15	0.21	100
5	T1DM	25	M	24.0	10	Insulin aspart	15	0.20	100
6	T1DM	55	M	26.7	10	Insulin aspart	17	0.20	100
7	T1DM	48	M	27.2	15	Insulin aspart	17	0.20	100
8	T1DM	57	M	23.8	11	Insulin aspart	17	0.20	100
9	T1DM	42	M	25.5	11	Insulin aspart	18	0.20	100
10	T1DM	31	M	23.9	18	Insulin aspart	18	0.20	100
11	Healthy	25	F	25.7	—	NPH insulin	26	0.40	74
12	Healthy	45	M	25.6	—	NPH insulin	34	0.40	91
13	Healthy	40	F	24.7	—	NPH insulin	28	0.40	78
14	Healthy	42	M	26.7	—	NPH insulin	38	0.40	77
15	Healthy	36	M	25.2	—	NPH insulin	38	0.41	70
16	Healthy	41	F	29.1	—	NPH insulin	31	0.40	75
17	Healthy	46	M	24.9	—	NPH insulin	34	0.40	80
18	Healthy	31	M	21.4	—	NPH insulin	30	0.40	70
19	Healthy	44	F	22.4	—	NPH insulin	25	0.40	72
20	Healthy	41	F	21.6	—	NPH insulin	24	0.40	76
21	T1DM	31	M	26.9	3	Insulin glargine	56	0.60	100
22	T1DM	53	M	22.3	12	Insulin glargine	44	0.59	100
23	T1DM	43	M	22.5	17	Insulin glargine	46	0.60	100
24	T1DM	49	F	25.6	22	Insulin glargine	43	0.60	100
25	T1DM	47	M	27.1	15	Insulin glargine	55	0.60	100
26	T1DM	23	F	23.2	2	Insulin aspart	21	0.30	100
27	T1DM	49	F	24.3	32	Insulin aspart	23	0.30	100
28	T1DM	27	M	22.3	7	Insulin aspart	22	0.30	100
29	T1DM	55	M	25.8	17	Insulin aspart	25	0.30	100
30	T1DM	54	M	24.9	27	Insulin aspart	26	0.30	100
31	T1DM	35	M	26.1	30	Insulin glargine	59	0.67	100
32	T1DM	48	M	23.9	34	Insulin glargine	42	0.59	100
33	T1DM	28	M	21.1	4	Insulin aspart	22	0.30	100
34	T1DM	47	F	23.6	35	Insulin aspart	21	0.29	100
35	T1DM	38	M	25.1	31	Insulin aspart	24	0.31	100
36	T1DM	52	M	29.8	41	Insulin glargine	52	0.60	100
37	T1DM	53	M	27.3	8	Insulin glargine	25	0.30	100
38	T1DM	44	M	23.6	6	Insulin aspart	25	0.30	100
39	T1DM	25	M	26.2	9	Insulin aspart	49	0.60	100

Figure 4.

A ClampArt glucose clamp with insulin aspart (a; subject with T1DM, insulin dose 0.20 U/kg, clamp level 100 mg/dl) and insulin glargine (b; subject with T1DM, insulin dose 0.59 U/kg, clamp level 100 mg/dl).

Clamp Quality and Utility

Results related to the quality of the glucose clamps carried out with ClampArt are displayed in Table 2: Trueness of all paired BG measurements was 1.2 ± 2.5 mg/dl, and precision and accuracy were 5.5 ± 2.1% and 5.3 ± 2.3%, respectively. The control deviation was 1.2 ± 1.6 mg/dl and utility of ClampArt amounted to 95.4 ± 4.1%.

Table 2.

Parameters Referring to Clamp Quality and Utility

Clamp no.	Trueness of BG measurement (mg/dl)	CV at target level (%)	MARD at target level (%)	Mean diff. device vs target level (mg/dl)	Utility (%)
1	2.3	3.7	5.2	0.4	97.0
2	9.3	7.0	13.6	0.4	96.9
3	8.8	5.7	8.6	0.0	84.3
4	0.9	4.2	5.7	0.3	99.7
5	3.6	3.5	2.2	0.4	98.0
6	−0.1	4.7	4.5	0.7	96.7
7	−0.9	11.0	10.9	0.0	98.8
8	−4.3	4.3	6.6	1.2	100.0
9	4.7	5.1	6.7	1.0	95.0
10	2.4	9.7	8.9	0.1	98.7
11	0.8	5.6	4.6	1.8	90.3
12	1.0	2.6	5.4	0.1	98.5
13	0.9	6.2	4.5	2.5	98.4
14	0.7	4.9	4.5	0.3	96.6
15	−0.2	7.7	5.6	2.1	96.8
16	—	—	—	—	—
17	2.4	5.3	5.9	0.2	98.0
18	0.4	9.4	7.5	2.0	94.5
19	0.0	4.2	3.8	0.8	99.2
20	2.1	5.2	5.5	0.3	95.8
21	0.2	8.4	3.8	8.6	91.3
22	1.2	5.5	4.0	1.5	93.7
23	1.6	3.7	4.3	1.1	97.9
24	1.9	5.4	4.6	0.3	91.9
25	−0.1	4.3	3.3	1.4	98.3
26	0.4	8.5	4.4	3.3	83.5
27	1.7	3.0	3.3	0.5	94.5
28	1.3	2.3	4.9	0.4	97.9
29	2.9	6.1	4.5	0.1	94.9
30	−1.1	7.6	6.4	1.8	88.5
31	—	—	—	—	—
32	1.2	7.8	3.4	3.6	98.6
33	−1.1	6.2	4.9	2.5	89.5
34	3.3	6.6	4.9	1.9	94.7
35	−1.0	4.5	3.6	0.6	98.9
36	−1.5	3.8	3.2	1.3	91.2
37	−0.3	4.9	7.5	0.5	97.9
38	−0.3	3.5	3.6	0.9	98.6
39	−1.4	2.4	2.3	0.3	95.8
Mean	1.2	5.5	5.3	1.2	95.4
SD	2.5	2.1	2.3	1.6	4.1
Median	0.9	5.2	4.6	0.7	96.8
Interquartile range	2.3	-2.4	2.1	1.5	3.9

Trueness: mean (device vs ref. method); CV: 100 × (SD device/mean device); MARD: 100 × ([ref. meth. – device]/ref. meth.); utility: 100 × (operational time/total time).

Of all paired data points, 99.97% (n = 3732) were within zones A and B of the consensus error grid, 99.06% were in zone A (Figure 5).

Figure 5.

Parkes error grid of 37 glucose clamps performed with ClampArt in this study.

Safety

No SADEs occurred.

Discussion

Euglycemic glucose clamps are needed to assess the PD of novel insulin analogs or administration routes (SC, IV, inhaled, oral, etc).^15,22 Irrespective of the way they are conducted (manually or automated), a high glucose clamp quality is important to precisely and reproducible describe the time-action profiles of BG lowering agents. To achieve a high quality, that is, a GIR reflecting the correct PD effect, BG must be kept at the target clamp level as closely as possible. Thus, the smaller the BG deviations from target are, the higher the clamp quality. Unfortunately no common definition is available for the quality of glucose clamps. In addition, data on glucose clamp quality are most often not reported in respective publications. The following aspects concerning “clamp quality” should be taken into account when comparing different glucose clamp methods and clamp trials: (1) trueness, (2) precision, (3) accuracy, (4) control deviation, and (5) utility. Figure 3 illustrates 4 different results for trueness, precision, accuracy, and control deviation and relates these to the target clamp level and the reference laboratory method. Of note, only sufficient results for each of the 4 parameters end up in high quality underlying the importance of each single one.

Noteworthy the European Medicines Agency (EMA) has dedicated a separate chapter to the quality of clamps in its draft guideline on nonclinical and clinical development of similar biological medicinal products containing recombinant human insulin and insulin analogs.¹⁵ This is the first time a regulatory body addresses clamp quality in a guidance document. EMA proposes the calculation of mean values, root mean square deviation and CV of the BG concentrations to provide an estimate of the performance of the clamp study. We did not select mean values as these depend on the clamp level, but we chose the mean ± SD difference between the clamp device and the target level to reflect the “control deviation.” Instead of root mean squares we calculated the MARD, because this is an established parameter for the evaluation of the quality of continuous glucose monitoring systems (CGMS). As proposed in the guideline we also took the CV, and in addition to the guideline we analyzed “trueness” and “utility”—the latter is important for automated clamp devices.

In the following we briefly discuss the results in this study related to the individual quality parameters:

• a. Concerning trueness ClampArt in the present trial exhibited excellent results—there are, however, no published data on trueness for any other clamp study available.

• b. With respect to precision 1 author²³ has already proposed to take the CV of the BG concentrations and stated that values of the CV below 5% are considered an adequate criterion for a glucose clamp of sufficient quality. For the Biostator a mean CV of the BG concentration of 5.9 ± 0.5% was observed in a euglycemic clamp study in healthy people.²⁴ The present 39 ClampArt clamps resulted in a slightly lower CV at the target level of 5.5 ± 2.1%.

• c. Accuracy (as well as trueness and precision) refers to the quality of the measurement, not to the quality of control. Therefore, a comparison with other methods like CGMS can be drawn considering the following limitations: The use of a CGMS implies a lag phase between the interstitial fluid and the blood, which is not the case with ClampArt. In a head-to-head trial investigating the accuracy of 3 devices, Navigator, G4, and Enlite continuous subcutaneous glucose sensor²⁵ the overall MARD between sensor and reference values (YSI 2300 STAT Plus analyzer, YSI Life Sciences, Yellow Springs, OH) was between 10.8 ± 9.9% and 17.9 ± 15.8%, that is, much higher than with ClampArt (MARD 5.3 ± 2.3%).

Accuracy can also be evaluated referring to ISO standard 15197:2013²⁶ and an error grid. Applying the error grid technique²¹ to the present investigation, with ClampArt 99.97% of BG measurement results compared with the reference BG (SuperGL, n=3732 data pairs) are in zones A and B and 99.06% in Zone A. In comparison, the STG-55 device showed that 98.4% of data fell in Zones A and B.¹⁸

• d. The control deviation, that is, a possibly low mean difference between clamp device and target BG, is also influenced by the quality of the measurement and the target BG. Results for the control deviation were excellent for ClampArt with differences as low as 1.2 ± 1.6 mg/dl. This shows that the used algorithm is suited to minimize deviations from the target BG. However, a further reduction in control deviation might still be possible by increasing the quality of the measurement and further optimizing the algorithm which will be next steps in the further development of ClampArt.

• e. The utility time for the Biostator was 90.5 ± 5.9% of the total clamp duration in 1 study.²⁷ This was higher with ClampArt (95.4 ± 4.1%). A further reduction in downtime can be expected in the near future by a further optimization of the tubing sets used (it should be noted that 1 clamp had to be terminated early due to tubing issues).

Our study has important limitations that need to be acknowledged. Most important, this first study with ClampArt investigated the PD of subcutaneously injected insulin analogs only. Other formulations like IV or inhaled insulins as well as other BG lowering agents will still have to be investigated. It is certainly possible that with different PD effects glucose clamp quality will vary. Nevertheless, the quality parameters themselves can be applied to all glucose clamps, irrespective of the insulin or administration route investigated. The parameters can also be used for the description of the clamp quality of glucose clamps used for the investigation of insulin sensitivity (ie, glucose clamps in which intravenous insulin infusions are used).

As data on clamp quality are so sparse, and as proposals made by clamp groups²⁸ have hardly been implemented, head-to-head glucose clamp trials (Biostator, ClampArt, manual clamp) will be necessary to clarify whether 1 clamp method achieves better quality compared to the others.

A challenge for the future remains the development of a clear definition of the term “clamp quality”—which is also supported by regulatory authorities.¹⁵ Our hope and intention with respect to this publication are to point out the importance of glucose clamp quality in terms of quality of measurement, control and utility, and to initiate a discussion on the most suitable parameters for the evaluation of the quality of glucose clamps. Ideally this should result in a consensus of the respective groups conducting glucose clamps on how to measure clamp quality. These parameters should then be included in future publications of glucose clamp trials.