Abstract
The use of glucose sensors during clinical trials seems like a great idea at first glance. Continuous glucose monitoring (CGM) should allow the gathering of more detailed information about metabolic control, without requiring much additional effort. In principle, CGM can reduce the duration of such studies and the number of participants required. The aim of this commentary is to highlight some of the reasons why, in practice, at least some of these hopes have not been realized. It is not only that a new technology requires extensive training of the study personnel; the practical handling of the devices and the time and effort required to download and analyze the data are often grossly underestimated initially. In addition, one must select the best endpoints for describing the level of metabolic control in view of the overwhelming amount of information provided by CGM. Several measures and endpoints were proposed as (potential) parameters that would be more meaningful than the standard parameters currently used to describe glucose profiles. Unfortunately, most of these proposed parameters have not, as yet, been proven to be more meaningful. Calibration is another critical aspect of using CGM that must be addressed. How this procedure is handled in practice has a profound impact on the quality of the glucose recordings. Finally, shall the current measurement results be displayed to the study participant or not? CGM can help prevent severe hypoglycemic episodes, but this can profoundly affect the study outcome in a manner that is unrelated to basic aim of the study (e.g., comparing medications that are designed to control glycemia). Therefore, the use of CGM in clinical trials requires much more careful consideration than was initially thought.
Keywords: clinical trials, continuous interstitial glucose monitoring
The invention of practicably usable systems for continuous glucose monitoring (CGM) some years ago was not just a starting point for using this technology in daily practice with patients with diabetes and in scientific studies. It also raised the hope that clinical trials performed in the development process of new anti-diabetic drugs could benefit from CGM. These comments discuss some of the pros and cons of using CGM in such clinical trials.
Considering that a single day during the clinical development process of a new drug can cost up to 1–2 million US dollars (or euro equivalent), any chance to accelerate this process is welcome. Especially in late phase 2 and 3 studies, which are the most time consuming, and therefore expensive, CGM could help determine the efficacy of a new anti-diabetic drug when it comes to its blood glucose lowering effects. Additional information provided by CGM should allow for a study of shorter duration, provide a more in-depth understanding of the efficacy, and reduce the number of patients that have to be studied. These factors are important in saving costs. These savings should greatly outweigh the additional costs of using CGM. The use of CGM is less relevant in the early phases of clinical development, as these phases are usually shorter and quite often are run under in-house conditions. However, even under these conditions CGM might help by providing additional information. This explains why companies have been enthusiastic about implementing CGM in clinical trials since the first systems became available some years ago.
The reality was surprising and disappointing when it turned out that these hopes were only partially fulfilled, as is true with any new development. More time than expected was required to train the study personnel in the appropriate handling of the CGM systems (we will come back to the complexity of the technique as a major hurdle for CGM usage), data download, and analysis of the data with the software provided by the manufacturer or the investigator's own software. It simply takes time to explain the system to patients, to fix the sensor adequately for daily life conditions, and to calibrate the measurement appropriately (which requires well performed capillary blood glucose measurements). In addition, more or less complete system failures occurred, some data points were missing during rapid movements of the patients, and implausible measurements were recorded from time to time. Such time consuming efforts create obstacles for all the other activities that are required in the successful performance of a study (e.g., noting all information in the case report form [CRF], talking to the volunteers, dispensing the study drug). If an insufficient number of study nurses and physicians take care of all these jobs, then the CGM measurement will probably not receive adequate attention. The cost for the additional personnel required reduces the anticipated cost savings. Also, the time required to analyze the obtained data sets (which are quite large compared those from studies that use simple blood glucose measurements) also reduces the benefits of CGM use. In summary, one has to acknowledge that CGM systems are not “hit and run” systems and cannot be quickly and immediately incorporated into study protocols.
Today, five different CGM systems from four manufacturers have market approval (not identical for the US and EU markets) and can be used in clinical trials. Initially, the most frequently used CGM system was from Medtronic; nowadays it is the Guardian RT. Manufacturers of these systems are extremely interested in participating in such trials for the following reasons:
Patients, diabetes nurses, and physicians become acquainted with the given system and might also use it after the study has ended; moreover, the initial use and education was funded by somebody else (i.e., a pharmaceutical company)
Patients learn about the benefits of using a CGM in their daily lives
Use of sensors and systems in the studies generates a certain turnover of the product
Manufacturers of CGM systems have very limited resources for clinical studies (and in each study they can learn something about their system)
Publication of the study results is an excellent promotion for the given CGM system.
Clearly, one key issue with CGM in clinical trials regards which end point should be used? In such trials, the new anti-diabetic drug—and not the performance of the CGM system—should be evaluated. Thus, continuously monitored glucose levels can be used to provide much more information about the impact of the medication on the blood glucose profile over time. These data are more useful than a limited number of capillary blood glucose measurements per day and/or a single HbA1c value measurement. Before discussing the variety of end points that CGM systems offer, we should look at the (surrogate) end points that have been used most often in clinical trials:
Blood glucose: Usually on a given study day a glucose profile is made with 10–24 measurements. However, even with such a number of measurements, a complete picture cannot be achieved, and there is a high risk that the maximal postprandial glycemic excursions and nocturnal hypoglycemic events will be missed. If the blood glucose profile is measured during an in-house study-day the question is, does this adequately represent the daily life of a patient?
Glycated hemoglobin: This parameter provides information about the average metabolic control in the last months, but gives no hint about glycemic variability. Due to the inherent inertia of this parameter, clinical studies have to span several months to detect a significant or clinically relevant difference in the metabolic effect of the study drug versus the control. Until now, to guarantee comparability of the measurement results, all blood samples in a clinical study had to be measured in a central laboratory, which is logistically demanding and a massive economical burden. The recent invention of an optimal standard for HbA1c measurement partially relieves this burden, as it allows one to measure samples in different laboratories. However, the numbers generated with the new standard are considerably lower than that obtained with the old standard.
Frequency of hypoglycemic events: A reliable determination of hypoglycemic events is quite tricky without CGM, as patients with diabetes often have no reliable sensation of hypoglycemia-associated symptoms and therefore do not measure blood glucose for confirmation in all cases. Especially for nocturnal hypoglycemic events, the numbers obtained with spot capillary blood glucose measurements by the patients are unreliable for obvious reasons (patients do not measure blood glucose at 3 am and it has been shown recently that this “intervention” in itself has an impact on the glucose profile). In addition, many blood glucose meters are not very reliable when it comes to low blood glucose values (this argument holds true to a given extent for CGM systems also). Another issue is that the definitions employed for the characterization of hypoglycemic events differ significantly between clinical trials (even within the same clinical development program). In essence, the numbers provided for hypoglycemic events are difficult to compare between centers and studies.
Unfortunately, the huge amount of information provided by CGM also means there are many measures to calculate to describe properties of the recorded glucose profiles. Attempts have been made to standardize certain end points to enhance comparability between the different CGM systems. Such an agreement on certain end points is especially relevant for pharmaceutical companies. For them, an issue is that they want to convince the regulatory authorities that their compound has a benefit described by an end point that is unknown to the regulatory authorities. These regulatory bodies are accustomed to conventional end points (such as those listed earlier) and might not to be willing to readily accept parameters like MAGE, MIME, MODD, FAGE, %PRESS, or ADRR as new end points. When they are interested in considering these end points, they might critically ask for clinical studies that demonstrate that these are more relevant for patients with diabetes than the conventional end points. Unfortunately, such studies are lacking. This might be the main reason why, to my knowledge, no pivotal study that was essential for approval of a new drug application by the Food and Drug Administration used a CGM system.
From a regulatory point of view, another aspect is difficult to manage. What constitutes “raw data” and how can it be protected against data manipulation? Usually the data generated by the laboratory when a blood sample is measured are transferred to a CRF by a study nurse. An independent monitor subsequently checks if the correct data are entered onto the CRF before these data are fed into a database for statistical analysis after the database freeze. When it comes to CGM, the situation is trickier. The software built into the CGM systems “manipulates” the data on its own to reduce noise, dampen too rapid changes in glycemia, filter out implausible data, and so on. All these adjustments are made to provide more usable data to the patient and health care provider. However, from the regulatory point of view, the data have been manipulated. Unfortunately, each manufacturer of a CGM system has its own software approach. Some manufactures provide software for clinical trials that bypass all manipulation steps and give more or less raw data.
Calibration presents another challenge in data analysis. While using the CGM system, patients should enter results of capillary blood glucose measurements in the system according to the instructions of the manufacturer. Depending on the current calibration factor calculated by the CGM system, the registered glucose profile can subsequently be shifted upwards or downwards considerably. Thus, the precision of this calibration procedure, including the precision with which the blood glucose measurement was performed, has a profound impact on the quality of glucose recordings. In principle, it is also possible to recalibrate retrospectively the complete recorded data set, taking all calibration measurements into account at the same time. This can result in a better overall fit of the glucose profile to such reference measurements. Also, the quality of glucose recording with CGM systems is lower than that of blood glucose meters. The accuracy and precision of the measurement has been improved in the last years, from generation to generation of the sensors/systems; however, in quantitative terms, the quality of the systems as they are used in daily life remains unclear. The current efforts to standardize assessment of the measurement quality of CGM systems will be helpful in this regard.
Another topic of concern is that the amount of data provided by CGM is too sizeable to allow manual transfer onto a CRF: Thus, the data must be collected in an appropriate database. This in turn requires a special database system that monitors each and every step of the downloading procedure to guarantee that no changes to data occur during downloading and after storage.
One fundamental physiological aspect must also be mentioned. CGM systems measure glucose changes in the interstitial fluid in the abdominal subcutaneous tissue and not directly in the blood. Thus, changes in glycemia are monitored in two different compartments. It is still a matter of scientific debate whether the glucose changes in the dermis and the subcutaneous tissue run in parallel under each and every condition or not (alternate site testing [AST]–like phenomenon). This difference can be minimized with an initial calibration and more or less frequent recalibration with spot capillary blood glucose measurements. However, certain differences in the time and concentration domain (when it comes to rapid changes in glycemia) cannot be avoided. This fact is also relevant for clinical studies during the development of a new compound, as it hampers comparability between the results obtained with capillary blood glucose measurements performed in one study with those obtained with a CGM system in another study. This is not an issue as long as the outcomes of these studies are in line with each other; however, imagine that a positive effect was observed in one study but not the other. This can generate critical questions by the regulatory authorities when the package of all studies is presented at the end of the clinical development. This is something that pharmaceutical companies try to avoid by all means.
Another key question is, shall the CGM systems used in the clinical trials show the current glucose values on their display and warn the patients if a hypoglycemic or hyperglycemic event takes place? The advantage for the volunteers is obvious: they can optimize their metabolic control by having this additional information provided by the CGM system without running into the risk of acute metabolic deteriorations. However, from the point of view of a clinical study, this improved metabolic control is an issue, as it may not reflect the better pharmacological effect of the new compound in comparison to the control. Rather, the difference arises from the quality with which the CGM system can prevent, for example, hypoglycemic events. Thus, the study evaluates the performance of the CGM system from a technical point of view as well as which patients can use it optimally and—probably most importantly—how well they have been trained to transfer the information provided into appropriate therapeutic action. In principle, we would have to talk about sensor-augmented diabetes therapy under such circumstances. If you blind the display of the CGM system and switch off the warning system, you are more or less back to the situation of a conventional clinical study with the advantage of having much more data for subsequent data analysis. One can challenge this approach as being unethical, as optimal CGM use could potentially help avoid life-threatening events. However, until now, CGM systems have not been standard-of-care for all patients and therefore can be regarded as an augmented means for monitoring glycemia.
The rapid progress made with diabetes technology and, more specifically, with glucose sensors has greatly benefited the treatment of patients. However, this progress poses difficulties in clinical trials that run for a number of years because, within this period, different versions of sensor electrodes and even new CGM systems might become available. Additionally, new versions of software might emerge. In the end, the question may be, are the results obtained with the original CGM system used when the study began comparable to those obtained by a newer CGM system used at the end of the study, one or two years later? This becomes a serious issue when we think about long-term outcome studies with hard end points. Such studies require a duration of several years. Without having the results of such studies in our hands, we will never obtain good evidence when it comes to the prognostic relevance of glycemic variability or postprandial glycemic excursions.
To avoid the impression that I am opposed to the use of CGM in clinical trials, I would like to outline the details that have to be clarified before such a study is initiated. In addition to the points mentioned earlier, there are several good reasons to use CGM in clinical trials. A very important one is the chance to quantify glycemic variability. This is not yet an established standard; however, many diabetologists believe that a reduction in variability will become a standard parameter once it is demonstrated in clinical trials that a reduction in variability has an impact on hard end points. This is not only of great interest to the manufactures of CGM systems but also for companies that try to develop better, new anti-diabetic drugs. If such parameters can be established as relevant new end points for clinical trials, should they be primary or secondary end points?
This also holds true for the topic of reducing postprandial glycemic excursions. CGM is much better than spot measurements at monitoring such excursions. For example, the optimal time point for a postprandial capillary blood glucose measurement is not clear. The time point at which the maximal glycemic excursion takes place depends on a number of factors such as type of meal, type of antidiabetic treatment (some drugs delay gastric emptying), and level of preprandial glycemia. If glucose profiles over several days are recorded with CGM, a much more detailed analysis is possible: When do hypoglycemic events take place during the day? Always at 2 am? Are postprandial glycemic excursions more prominent after breakfast? By means of specialized computer programs, such (post hoc) analyses of downloaded data could be performed quite rapidly.
In summary, using CGM systems in clinical trials is an excellent idea, but CGM has not been established as a standard technique until now. As with each complex new technique, CGM requires experience not only of the people who actually run these studies, but of the pharmaceutical companies, which must be aware that the additional information provided by CGM does not come without a cost. CGM is not simply a more intensive self-monitoring blood glucose procedure; it is a world unto its own. Successful performance of clinical studies with CGM requires study sites that have experience with this technology and personnel who understand the details behind it. As was the case with mobile phone technology, which has advanced tremendously, one can envision future generations of CGM systems becoming easier to handle, more reliable, less expensive, more widely available with reimbursement. Most probably, after this takes place, CGM systems will be routinely used in daily practice and in all clinical studies; not only in those performed during drug development. In other words, even if the present is grey for CGM in clinical trials, the future is rosy!
Abbreviations
- AST
alternate site testing
- CGM
continuous glucose monitoring
- CRF
case report form