Abstract
Background:
User-developed automated insulin delivery systems, also referred to as do-it-yourself artificial pancreas systems (DIY APS), are in use by people living with type 1 diabetes. In this work, we evaluate, in silico, the DIY APS Loop control algorithm and compare it head-to-head with the bio-inspired artificial pancreas (BiAP) controller for which clinical data are available.
Methods:
The Python version of the Loop control algorithm called PyLoopKit was employed for evaluation purposes. A Python-MATLAB interface was created to integrate PyLoopKit with the UVa-Padova simulator. Two configurations of BiAP (non-adaptive and adaptive) were evaluated. In addition, the Tandem Basal-IQ predictive low-glucose suspend was used as a baseline algorithm. Two scenarios with different levels of variability were used to challenge the algorithms on the adult (n = 10) and adolescent (n = 10) virtual cohorts of the simulator.
Results:
Both BiAP and Loop improve, or maintain, glycemic control when compared with Basal-IQ. Under the scenario with lower variability, BiAP and Loop perform relatively similarly. However, BiAP, and in particular its adaptive configuration, outperformed Loop in the scenario with higher variability by increasing the percentage time in glucose target range 70-180 mg/dL (BiAP-Adaptive vs Loop vs Basal-IQ) (adults: 89.9% ± 3.2%* vs 79.5% ± 5.3%* vs 67.9% ± 8.3%; adolescents: 74.6 ± 9.5%* vs 53.0% ± 7.7% vs 55.4% ± 12.0%, where * indicates the significance of P < .05 calculated in sequential order) while maintaining the percentage time below range (adults: 0.89% ± 0.37% vs 1.72% ± 1.26% vs 3.41 ± 1.92%; adolescents: 2.87% ± 2.77% vs 4.90% ± 1.92% vs 4.17% ± 2.74%).
Conclusions:
Both Loop and BiAP algorithms are safe and improve glycemic control when compared, in silico, with Basal-IQ. However, BiAP appears significantly more robust to real-world challenges by outperforming Loop and Basal-IQ in the more challenging scenario.
Keywords: type 1 diabetes, automatic insulin delivery, artificial pancreas, bio-inspired technology, do-it-yourself, in silico trials
Introduction
Technological progress in the field of diabetes management, and in particular, the improvement of continuous glucose monitors, has enabled the development of semi-automated (hybrid) insulin delivery (AID) systems for blood glucose control, the so-called artificial pancreas, which consists of a subcutaneous continuous glucose sensor and a subcutaneous infusion pump that delivers insulin at a rate decided by a computer program (control algorithm) to maintain glucose levels within a target range.1,2
Clinical studies have shown that a hybrid AID can improve glycemic control compared with standard treatment. 3 The commercialization of the first hybrid AID, the Medtronic MiniMed 670G with SmartGuard (Medtronic, California), 4 occurred in 2018 and was followed by the commercialization of four more systems, the Tandem Control-IQ (Tandem Diabetes, San Diego, California), the Medtronic MiniMed 780G with SmartGuard, the CamAPS FX (CamDiab, Cambridge, UK), and the DBLG1 system (Diabeloop, Grenoble, France). In addition, other systems are currently being assessed and soon commercialized. 5
Several closed-loop control algorithms for AID have been described and clinically evaluated, including model predictive control, 6 proportional-integral-derivative (PID), 7 fuzzy logic, 8 and bio-inspired. 9 However, none of these controllers have yet shown a clear superiority when compared with others in a clinical setting.10,11
Although commercial solutions of hybrid AID systems now exist, the availability of continuous glucose monitoring (CGM) and insulin pumps motivated people with diabetes to form the #WeAreNotWaiting movement and develop do-it-yourself (DIY) AID systems, such as OpenAPS, AndroidAPS, and Loop.12,13 The DIWHY survey of DIY system users by the EU-funded Open Project highlighted key motivations as improved glycemic control, reducing long-term complications, reduced interaction with their therapy systems, and a lack of commercially available closed-loop systems in their country. 14
Data reported for these DIY systems suggest general safety and improved glycemic control. For the openAPS algorithm, this includes retrospective analysis of CGM15,16 and in silico trials by Toffanin et al. 17 A large observational study into real-world use of the loop system concluded it can be used safely and effectively by children and adults. 18
With increasing availability of commercial AID systems, people with type 1 diabetes have a choice of systems and comparative data may contribute to this, as well as serving to benchmark performance. In this article, we provide, to the best of our knowledge, the first in silico evaluation of the Loop control algorithm with the UVa-Padova simulator. 19 In addition, we provide an in silico head-to-head comparison of the Loop system with the clinically evaluated bio-inspired artificial pancreas (BiAP) control algorithm developed at Imperial College London (London, UK). 9 ,20-22 The baseline algorithm for this comparison is the Tandem Basal-IQ predictive low-glucose insulin suspend and bolus calculator system (Tandem Diabetes Care, San Diego, California). 22
Methods
Loop Controller Implementation
Loop control algorithm modulates temporary basal insulin rates and offers recommended meal and correction insulin bolus to reach a predefined target glucose range by accounting for current CGM readings and the effect of ingested meals and administered insulin doses on future glucose levels. Details about the control algorithm can be found in the Loop documentation. 23
In this work, to evaluate the Loop control algorithm, we used its open-source Python version called PyLoopKit, which can be downloaded from a GitHub repository. 24
PyLoopKit can parse either Tidepool issue reports or an input dictionary containing the profile settings, along with meal, insulin, and glucose history. For this work, we created a Python-MATLAB interface which allowed us to call PyLoopKit with the required inputs, handle storing the required history, and parse the output to the correct format for the UVa-Padova simulator. An open-source version of such interface can be downloaded from a GitHub repository. 25
To approximate the average user experience, settings were provided following Loop’s setup configuration recommendations, which included using subject-specific information provided by the simulator, such as insulin sensitivity factor (ISF), insulin-to-carbohydrate ratio (ICR), and basal insulin. As suggested by the Loop documentation, the maximum basal rate was set to 4 times the highest scheduled rate. Based on multiple in silico tests, the correction range was set to 100 to 110 mg/dL for the adult cohort and to 110 to 140 mg/dL for the adolescent cohort. This range was empirically selected by considering, for the two evaluated scenarios, a trade-off between maximizing the percentage time in the glycemic target range (70-180 mg/dL) and minimizing the percentage time in hypoglycemia (<70 mg/dL).
Loop allows a slow, medium, and fast absorption rate to be chosen for meals, but because version 3.2 of the UVa-Padova simulator does not include this option, the default medium rate of 180 minutes was used throughout our evaluation. We also used the recommended adult/adolescent rapid-acting exponential insulin model of 6 hours, with a 75-minute peak.
BiAP Controller Implementation
The BiAP is a hybrid AID system that includes a closed-loop control algorithm based on the mathematical modeling of the pancreatic beta-cell physiology. To recommend meal insulin and correction doses, BiAP incorporates an adaptive bolus calculator that learns to optimize the bolus calculator settings from the controller’s functioning, user’s behavior, and CGM outcomes. 26
The BiAP controller used in this work slightly differs from the one originally presented by the authors. 26 In particular, the employed beta-cell model has been modified to include the potentiation effect of sustained elevated glucose levels on insulin secretion. Details about the updated model can be found in Supplemental Appendix. It is worth remarking that, as demonstrated in Herrero et al, 9 the employed beta-cell model is mathematically equivalent to a proportional-derivative (PD) controller with two low-pass filters, and by extension, the updated version employed in the current work is equivalent to a PID controller with two low-pass filters.
The BiAP closed-loop algorithm modulates insulin delivery in two different ways. The beta-cell model, which includes an insulin feedback mechanism to account for insulin stacking, is used to deliver micro-boluses, every 5 minutes, to compensate for hyperglycemia. Then, a predictive low-glucose insulin suspend system (PLGS) is employed to minimize hypoglycemia by partially (ie, 50%), or totally, suspending basal insulin delivery. The BiAP controller was tuned using subject-specific information provided by the simulator, such as ISF, ICR, and basal insulin. Glucose targets for the beta-cell model and bolus calculator were set to 100 mg/dL, and the partial and total suspension thresholds for the PLGS were set to 100 mg/dL and 80 mg/dL, respectively.
Finally, with the aim of reducing post-prandial peaks, a super bolus strategy has been introduced. 27 With a super bolus, some of the basal insulin delivery is stopped or partially reduced and delivered instead as additional bolus insulin on top of a normal meal or correction bolus. In this case, basal insulin delivery is suspended for 60 minutes after an insulin bolus is delivered.
Two configurations of the BiAP controller were evaluated: with non-adaptive bolus calculator (BiAP) and with adaptive bolus calculator (BiAP-Adaptive). The BiAP control algorithm employed in this work was implemented in MATLAB 2020b (MathWorks, Natick, Massachusetts).
Figure 1 shows a block diagram of the BiAP control algorithm, and Figure 2 shows an example of the different insulin delivery strategies employed by Loop and BiAP. Loop modulates the basal insulin delivery rate in both directions with respect to the basal profile, while BiAP only reduces the basal insulin rate (by 50% or 100%) and, when necessary, delivers additional micro-boluses at 5-minute intervals.
Figure 1.
Block diagram of the BiAP control algorithm. Rounded rectangles represent inputs to the controller and the non-rounded ones correspond to different modules of the controller. Nodes indicate where different connectors meet. Abbreviation: BiAP, bio-inspired artificial pancreas.
Figure 2.
Comparison of the different insulin delivery strategies employed by Loop and BiAP. Upper graph shows the CGM profiles for Loop (blue) and BiAP (red) for the same 24-hour meal scenario. Note that the announcement of lunch is omitted. Middle graph displays the basal-bolus insulin delivery by Loop and the bottom graph by BiAP. Abbreviations: BiAP, bio-inspired artificial pancreas; CGM, continuous glucose monitoring.
Tandem Basal-IQ and Bolus Calculator Implementations
The Tandem Basal-IQ predictive low-glucose insulin suspend algorithm was implemented in MATLAB as described in the manufacturer’s user manual. 28 However, the basal resumption condition, which states that basal insulin delivery is resumed once the current CGM sensor reading increases compared with the previous reading, was modified to be consistent with the figures in the manual and from the observation of the functioning of the actual system. In particular, this condition was replaced by a condition that requires three consecutive CGM sensor readings to increase with respect to the previous reading before the basal is resumed.
Based on multiple in silico tests, the glucose threshold to suspend insulin was left to the default value of 80 mg/dL for the adult cohort and set to 110 mg/dL for the adolescent cohort. The Tandem bolus calculator was implemented in MATLAB based on the work presented by Buchanan et al. 29
In Silico Environment
The in silico head-to-head evaluation was performed on version 3.2 of the UVa-Padova simulator. 19 However, to create more challenging scenarios for the controllers, additional intra-day variability on insulin absorption, carbohydrate estimation, meal timing, and insulin sensitivity was introduced. Details about how this variability was introduced can be found in Herrero et al. 26 In addition, to account for additional variability in meal composition, the simulator meal library, which consists of 30 different meals, was supplemented with a further 16 new mixed-meal absorption profiles described by the authors in a previous work. 30
Two in silico scenarios were employed to challenge the evaluated controllers at different levels. Both scenarios consisted of three meals a day at 7 am, 1 pm, and 8 pm with 70, 100 and 80 g of carbohydrates, respectively. No snacks were provided to prevent any masking of hypoglycemic events. Similarly, no rescue carbohydrates were given to see the capacity of the controllers to minimize hypoglycemia.
In scenario 1, all meals were announced for meal bolus calculation, and there was a 50% chance of checking for a correction bolus at 2.5 hours after a meal. If glucose levels were above 200 mg/dL at this check, then the correction bolus recommended by the algorithm being tested was accepted. The occurrence of this check was generated randomly and then kept consistent across participants, scenarios, and algorithms.
Scenario 2 included additional challenges for the controllers, such as offsetting bolus calculator settings (ICR, ISF) and basal insulin profile settings (Table 1), and skipped or delayed meal announcement. A trigger for skipping and delaying meal announcement was randomly generated based on an average 2.5 meal announcements skipped and 2 meals delayed per week. 31 This was then used consistently across all tests and comparisons. As the Loop algorithm uses a carbohydrate absorption model, when the delayed meals were announced, they were set to the correct time retrospectively as is recommended practice in the Loop documentation. The design of scenario 2 aimed to attain glycemic outcomes (ie, percentage time in glucose targets) that are closer to the outcomes observed when evaluating open-loop and closed-loop insulin delivery strategies in clinical and real-world settings.4,32
Table 1.
Offsetting of the Bolus Calculator and Basal Insulin Profile Settings.
| ICR schedule | 3:00-11:00 | 11:00-17:00 |
| ICR offset | +20% | −20% |
| ISF schedule | 8:00-16:00 | 16:00-24:00 |
| ISF offset | +20% | −20% |
| Basal insulin schedule | 0:00-12:00 | 12:00-24:00 |
| Basal insulin offset | −40% | +20% |
Abbreviations: ICR, insulin-to-carbohydrate ratio; ISF, insulin sensitivity factor.
The simulation was performed on the adult (n = 10) and adolescent (n = 10) cohorts of the simulator over a 60-day period. The last 30 days of the simulation were used for evaluation purposes to allow comparison with the adaptive version of BiAP. Tuning of all the evaluated algorithms was kept consistent in all scenarios.
Evaluation Metrics and Statistical Analysis
Common glycemic control metrics for artificial pancreas clinical trials 33 were employed for evaluation purposes. These included mean blood glucose measured with the CGM in mg/dL (MEAN BG) and time-in-range metrics, including percentage time within the 70-180 mg/dL range (%TIR70-180), percentage time below 70 mg/dL (%TB70), percentage time below 54 mg/dL (%TB54), percentage time above 180 mg/dL (%TA180), and percentage time above 250 mg/dL (%TA250). Other glycemic metrics were low blood glucose index (LBGI), high blood glucose index (HBGI), risk index (RI), average insulin units per day (INSULIN), and mean absolute glucose (MAG). Results are reported as mean ± SD.
For the comparison of the different evaluated interventions, a one-way analysis of variance (ANOVA) test was used to reject the null hypothesis that all intervention means are equal (P values shown in the last row of Tables 2 –4), with post hoc tests (Tukey-Kramer with α = 0.05) to compare pairwise interventions. For the comparison of the effect of the BiAP adaptation mechanism over time (ie, week 1 vs week 8), a paired t test was used. The statistical MATLAB 2020b toolbox was used for this purpose. Statistical significance was set at P < .05 for both the ANOVA post hoc test and the t test.
Table 2.
Glycemic Outcomes for the Adult Cohort on Scenario 1.
| No. | Algorithm | Mean BG | %TIR70-180 | %TB70 | %TB54 | %TA180 | %TA250 |
|---|---|---|---|---|---|---|---|
| 1 | Basal-IQ | 135.3 ± 8.8 | 86.5 ± 7.1(4) | 1.40 ± 0.81(2) | 0.38 ± 0.50 | 12.07 ± 6.61 | 0.92 ± 1.37 |
| 2 | Loop | 136.7 ± 6.8(4) | 86.3 ± 4.6(4) | 0.38 ± 0.33(1) | 0.09 ± 0.09 | 13.31 ± 4.73(4) | 1.59 ± 1.12(4) |
| 3 | BiAP | 133.5 ± 5.9 | 89.8 ± 5.8 | 0.62 ± 0.71 | 0.12 ± 0.19 | 9.58 ± 5.29 | 0.47 ± 0.55 |
| 4 | BiAP-Adaptive | 128.4 ± 3.5(2) | 93.6 ± 3.0(1,2) | 0.83 ± 0.76 | 0.22 ± 0.23 | 5.52 ± 2.69(2) | 0.20 ± 0.26(2) |
| P value | .0368 | .0126 | .0133 | .1421 | .0075 | .0118 | |
| No. | Algorithm | LBGI | HBGI | RI | INSULIN | MAG | |
| 1 | Basal-IQ | 0.52 ± 0.28(2) | 2.79 ± 1.19 | 3.31 ± 1.37(4) | 45.8 ± 11.1 | 1.48 ± 0.18 | |
| 2 | Loop | 0.24 ± 0.10(1) | 3.01 ± 0.93(4) | 3.25 ± 0.90 | 45.3 ± 10.8 | 1.68 ± 0.20 | |
| 3 | BiAP | 0.30 ± 0.17 | 2.41 ± 0.87 | 2.71 ± 0.98 | 46.1 ± 11.1 | 1.52 ± 0.28 | |
| 4 | BiAP-Adaptive | 0.36 ± 0.17 | 1.74 ± 0.48(2) | 2.10 ± 0.59(1) | 47.8 ± 12.0 | 1.46 ± 0.22 | |
| P value | .0180 | .0181 | .0348 | .9649 | .1135 | ||
P values in the last row correspond to the ANOVA test, and the numbers within the superindex indicate the groups that are statistically different (P < .05) as a result of the post hoc analysis.
Abbreviations: ANOVA, analysis of variance; BG, blood glucose; BiAP, bio-inspired artificial pancreas; HBGI, high blood glucose index; INSULIN, average insulin units per day; LBGI, low blood glucose index; MAG, mean absolute glucose; RI, risk index.
Table 3.
Glycemic Outcomes for the Adult Cohort on Scenario 2.
| No. | Algorithm | Mean BG | %TIR70-180 | %TB70 | %TB54 | %TA180 | %TA250 |
|---|---|---|---|---|---|---|---|
| 1 | Basal-IQ | 162.2 ± 16.1(2,3,4) | 67.9 ± 8.3(2,3,4) | 3.41 ± 1.92(2,3,4) | 1.60 ± 1.31(3,4) | 28.69 ± 7.39(2,3,4) | 8.86 ± 5.49(2,3,4) |
| 2 | Loop | 146.9 ± 8.3(1) | 79.5 ± 5.3(1,4) | 1.72 ± 1.26(1) | 0.66 ± 0.68 | 18.83 ± 5.89(1,4) | 2.29 ± 1.36(1) |
| 3 | BiAP | 140.0 ± 6.4(1) | 85.8 ± 4.9(1) | 1.11 ± 0.87(1) | 0.43 ± 0.47(1) | 13.05 ± 4.85(1) | 1.03 ± 0.81(1) |
| 4 | BiAP-Adaptive | 135.8 ± 4.7(1) | 89.9 ± 3.2(1,2) | 0.89 ± 0.37(1) | 0.28 ± 0.27(1) | 9.22 ± 3.13(1,2) | 0.89 ± 0.62(1) |
| P value | <.00001 | <.000001 | <.0001 | .026 | <.000001 | <.000001 | |
| No. | Algorithm | LBGI | HBGI | RI | INSULIN | MAG | |
| 1 | Basal-IQ | 1.13 ± 0.75(2,3,4) | 7.67 ± 3.42(2,3,4) | 8.80 ± 3.98(2,3,4) | 43.8 ± 10.5 | 1.87 ± 0.49 | |
| 3 | Loop | 0.58 ± 0.33(1) | 4.28 ± 1.14(1) | 4.86 ± 1.04(1) | 44.3 ± 10.5 | 1.77 ± 0.23 | |
| 4 | BiAP | 0.41 ± 0.27(1) | 3.12 ± 0.87(1) | 3.53 ± 0.92(1) | 45.3 ± 10.9 | 1.60 ± 0.24 | |
| 5 | BiAP-Adaptive | 0.34 ± 0.12(1) | 2.54 ± 0.64(1) | 2.88 ± 0.70(1) | 46.3 ± 11.4 | 1.56 ± 0.22 | |
| P value | .0011 | <.000001 | <.000001 | .9572 | .1084 | ||
P values in the last row correspond to the ANOVA test, and the numbers within the superindex indicate the groups that are statistically different (P < .05) as a result of the post hoc analysis.
Abbreviations: ANOVA, analysis of variance; BG, blood glucose; BiAP, bio-inspired artificial pancreas; HBGI, high blood glucose index; INSULIN, average insulin units per day; LBGI, low blood glucose index; MAG, mean absolute glucose; RI, risk index.
Table 4.
Glycemic Outcomes for the Adolescent Cohort on Scenario 1.
| No. | Algorithm | Mean BG | %TIR70-180 | %TB70 | %TB54 | %TA180 | %TA250 |
|---|---|---|---|---|---|---|---|
| 1 | Basal-IQ | 163.9 ± 17.3 | 63.0 ± 14.3 | 2.73 ± 3.06 | 1.45 ± 2.29 | 34.29 ± 12.07 | 9.94 ± 10.73 |
| 2 | Loop | 175.9 ± 16.7(3,4) | 60.2 ± 10.5 | 1.25 ± 2.21 | 0.62 ± 1.57 | 38.53 ± 8.75(4) | 17.39 ± 9.73(4) |
| 3 | BiAP | 157.2 ± 14.2(2) | 68.0 ± 14.1 | 2.20 ± 3.11 | 1.02 ± 1.95 | 29.84 ± 11.80 | 7.63 ± 8.79 |
| 4 | BiAP-Adaptive | 147.0 ± 8.8(2) | 74.7 ± 13.1 | 2.80 ± 4.11 | 1.43 ± 2.88 | 22.50 ± 9.74(2) | 4.40 ± 4.92(2) |
| P value | <.001 | .080 | .68 | .81 | <.05 | <.05 | |
| No. | Algorithm | LBGI | HBGI | RI | INSULIN | MAG | |
| 1 | Basal-IQ | 1.12 ± 1.54 | 7.69 ± 3.76 | 8.81 ± 4.85 | 48.7 ± 17.4 | 2.11 ± 0.53 | |
| 2 | Loop | 0.58 ± 0.99 | 10.19 ± 3.84(4) | 10.77 ± 4.65 | 46.4 ± 15.8 | 2.58 ± 0.61 | |
| 3 | BiAP | 0.88 ± 1.42 | 6.52 ± 3.18 | 7.40 ± 4.27 | 49.8 ± 17.8 | 2.06 ± 0.55 | |
| 4 | BiAP-Adaptive | 1.11 ± 1.85 | 4.89 ± 2.10(2) | 6.00 ± 3.69 | 51.8 ± 18.5 | 1.97 ± 0.50 | |
| P value | .83 | <.01 | .11 | .91 | .08 | ||
P values in the last row correspond to the ANOVA test, and the numbers within the superindex indicate the groups that are statistically different (P < .05) as a result of the post hoc analysis.
Abbreviations: ANOVA, analysis of variance; BG, blood glucose; BiAP, bio-inspired artificial pancreas; HBGI, high blood glucose index; INSULIN, average insulin units per day; LBGI, low blood glucose index; MAG, mean absolute glucose; RI, risk index.
Results
Tables 2 and 3 show the glycemic outcomes for the adult cohort on scenarios 1 and 2, respectively. Similarly, Tables 4 and 5 show the glycemic outcomes for the adolescent cohort on scenarios 1 and 2, respectively.
Table 5.
Glycemic Outcomes for the Adolescent Cohort on Scenario 2.
| No. | Algorithm | Mean BG | %TIR70-180 | %TB70 | %TB54 | %TA180 | %TA250 |
|---|---|---|---|---|---|---|---|
| 1 | Basal-IQ | 172.2 ± 16.8(4) | 55.4 ± 12.0(4) | 4.17 ± 2.74 | 2.40 ± 2.22 | 40.47 ± 10.68(4) | 13.91 ± 10.30 |
| 2 | Loop | 176.2 ± 14.8(4) | 53.0 ± 7.7(4) | 4.90 ± 1.92 | 2.35 ± 1.64 | 42.10 ± 6.18(4) | 18.05 ± 9.48(4) |
| 3 | BiAP | 163.9 ± 13.7 | 61.2 ± 11.4(4) | 3.27 ± 2.58 | 1.57 ± 2.08 | 35.53 ± 10.10(4) | 9.78 ± 8.36 |
| 4 | BiAP-Adaptive | 147.9 ± 6.7(1,2) | 74.6 ± 9.5(1,2,3) | 2.87 ± 2.77 | 1.17 ± 1.69 | 22.48 ± 7.33(1,2,3) | 4.45 ± 4.24(2) |
| P value | <.001 | <.001 | .28 | .40 | <.0001 | <.01 | |
| No. | ALGORITHM | LBGI | HBGI | RI | INSULIN | MAG | |
| 1 | Basal-IQ | 1.76 ± 1.77 | 9.40 ± 3.64(4) | 11.16 ± 4.42(4) | 47.5 ± 16.6 | 2.12 ± 0.46 | |
| 3 | Loop | 1.69 ± 1.38 | 10.64 ± 3.46(4) | 12.33 ± 4.56(4) | 47.0 ± 16.5 | 2.63 ± 0.57(4) | |
| 4 | BiAP | 1.27 ± 1.53 | 7.74 ± 2.94 | 9.01 ± 3.76 | 48.6 ± 16.9 | 2.05 ± 0.47 | |
| 5 | BiAP-Adaptive | 0.99 ± 1.09 | 4.96 ± 1.63(1,2) | 5.95 ± 2.48(1,2) | 51.7 ± 17.8 | 1.93 ± 0.43(2) | |
| P value | .61 | <.01 | <.01 | .92 | <.05 | ||
P values in the last row correspond to the ANOVA, test and the numbers within the superindex indicate the groups that are statistically different (P < .05) as a result of the post-hoc analysis.
Abbreviations: ANOVA, analysis of variance; BG, blood glucose; BiAP, bio-inspired artificial pancreas; HBGI, high blood glucose index; INSULIN, average insulin units per day; LBGI, low blood glucose index; MAG, mean absolute glucose; RI, risk index.
In scenario 1, there is significant variance (from the one-way ANOVA test, indicating at least one intervention differs significantly from the group mean) in mean glucose and all time in ranges except %TB54 in the adult cohort and %TB70 and %TB54 in the adolescent cohort. Mean glucose and times above ranges were optimal with BiAP-adaptive in both cohorts, while %TIR70-180 was only optimal in the adult cohort. %TB70 was lowest with Loop, although in pairwise post hoc testing %TB70 with Loop was only significantly different to that with Basal-IQ in the adult cohort (0.38% ± 0.33% vs 1.4% ± 0.81%; P < .05).
For variability metrics in scenario 1, there is significant variance between AID approaches in LBGI, HBGI, and RI in the adult cohort, while for the adolescent cohort only show significant variance in HBGI. In post hoc testing, LBGI was significantly lower with Loop compared with Basal-IQ (0.52 ± 0.28 vs 2.79 ± 1.19; P < .05) in the adult cohort. No variance was seen in MAG or insulin delivered.
In scenario 2, in the adult cohort, there is significant variance in all metrics, but not in MAG and insulin delivered. In the adolescent cohort, there is significant variance in all metrics, but not in %TB70, %TB54, LBGI, and insulin delivered. In the adult cohort, when compared with Basal-IQ, both Loop and BiAP (non-adaptive and adaptive) were optimal in all significantly variant metrics with the exception of %T54, which was not different for Loop. In the adolescent cohort, only BiAP (non-adaptive and adaptive) was significantly different to Basal-IQ in mean glucose, %TIR70-180, %TA180, and HBGI. %TIR70-180 and %TA180 were optimal with BiAP-adaptive when compared with Loop (%TIR70-180: 89.9% ± 3.2% vs 79.5% ± 5.3%, P < .05 [adult]; 74.6 ± 9.5 vs 53.0 ± 7.7 [adolescent]; %TA180: 9.22% ± 3.13% vs 18.83% ± 5.89%, P < .01 [adult]; 22.48% ± 7.33% vs 42.10% ± 6.18% [adolescent]). For variability metrics in scenario 2, there is significant variance between AID approaches in LBGI (only adult cohort), HBGI, and RI, but the post hoc testing did not show significant differences between Loop and BiAP.
Figures 3 and 4 depict the 24-hour adult and adolescent population CGM averages, respectively, expressed as median [5th and 95th percentiles], corresponding to Loop and BiAP with adaptive bolus calculator (BiAP-Adaptive).
Figure 3.
Twenty-hour-hour adult population CGM averages expressed as median [5th and 95th percentiles] corresponding to Loop and BiAP with adaptive bolus calculator on scenario 2. Abbreviations: BiAP, bio-inspired artificial pancreas; CGM, continuous glucose monitoring.
Figure 4.
Twenty-hour-hour adolescent population CGM averages expressed as median [5th and 95th percentiles] corresponding to Loop and BiAP with adaptive bolus calculator on scenario 2.Abbreviations: BiAP, bio-inspired artificial pancreas; CGM, continuous glucose monitoring.
Table 6 presents, for scenario 2, the metric %TIR70-180, %TA180, and %TBR corresponding to weeks 1 and 8 for the BiAP with and without meal bolus adaptation and evaluated the adult and adolescent cohorts. Figures 5 and 6 display the temporal evolution of %TIR70-180 over the 8-week period for the two BiAP configurations evaluated on scenario 2 and the adult and adolescent cohorts, respectively. Note that, unlike the non-adaptive version, the adaptive version results in a %TIR70-180 and %TA180 significantly superior in week 8 when compared with week 1.
Table 6.
Comparison of Percentage Time in Target Corresponding to Week 1 and Week 8 Between BiAP Control With and Without Meal Bolus Adaptation Corresponding to Scenario 2 Evaluated on the Adult and Adolescent Cohorts.
| Adult cohort | ||||||
|---|---|---|---|---|---|---|
| BiAP-adaptation | BiAP (no adaptation) | |||||
| %TIR70-180 | %TA180 | %TB70 | %TIR70-180 | %TA180 | %TB70 | |
| Week 1 | 86.5 [81.5, 88.9] | 12.1 [9.9, 18.0] | 1.31 [0.45, 2.1] | 87.5 [80.6, 89.1] | 11.8 [9.6, 18.7] | 1.24 [0.69, 2.18] |
| Week 8 | 90.7 [87.0, 91.4]* | 8.9 [7.5, 12.3]* | 0.79 [0.40, 1.2] | 85.4 [81.9, 89.6] | 12.3 [9.7, 17.8] | 1.49 [0.2, 1.98] |
| Adolescent cohort | ||||||
| BiAP-adaptation | BiAP (no adaptation) | |||||
| %TIR70-180 | %TA180 | %TB70 | %TIR70-180 | %TA180 | %TB70 | |
| Week 1 | 63.1 [46.9, 69.8] | 34.9 [27.5, 43.9] | 3.2 [0.99, 4.3] | 62.0 [46.9, 69.1] | 33.9 [28.1, 45.8] | 2.75 [1.63, 3.76] |
| Week 8 | 79.6 [69.3, 84.9]* | 16.9[14.3, 27.2]* | 2.0 [0.69, 4.4] | 63.7 [69.3, 71.3] | 33.4 [28.2, 41.7] | 2.75 [1.98, 3.22] |
Results are presented as median [25th and 75th percentiles].
Abbreviation: BiAP, bio-inspired artificial pancreas.
P < .05.
Figure 5.
Temporal evolution of percentage time in glucose target range (70-180 mg/dL) corresponding to BiAP control with and without meal bolus adaptation evaluated on scenario 2 and the adult cohort. Results are presented as median [25th and 75th percentiles]. Abbreviations: BiAP, bio-inspired artificial pancreas; IQR, interquartile range.
Figure 6.
Temporal evolution of percentage time in glucose target range (70-180 mg/dL) corresponding to BiAP control with and without meal bolus adaptation evaluated on scenario 2 and the adolescent cohort. Results are presented as median [25th and 75th percentiles]. Abbreviations: BiAP, bio-inspired artificial pancreas; IQR, interquartile range.
Discussion
Overall, both Loop and BiAP, in its non-adaptive and adaptive configurations, improve glycemia control when compared with the Tandem Basal-IQ predictive low-glucose insulin suspend system. Such improvements are more relevant in the more challenging scenario 2. These differences can be explained by the fact that in scenario 2, the bolus calculator settings are more offset, and, in some cases, meal announcement is omitted or delayed. Hence, a closed-loop controller can better compensate for such uncertainties and perturbations.
In both scenarios and both cohorts, when compared with Loop, BiAP-Adaptive increases %TIR70-180 and reduces hyperglycemia (%TA180 and %TA250). Hence, adaptation is potent, even over a relatively short period (ie, 2 weeks), as can be observed in Figure 4. Furthermore, unlike Loop, both BiAP and BiAP-Adaptive reduce hypoglycemia in scenario 2 (adult cohort) compared with Tandem Basal-IQ.
It is worth remarking the effectiveness of the BiAP adaptation mechanism for the adolescent cohort, which might indicate that there is greater potential for improvement when starting with a lower percentage time in glucose range baseline.
The fact that the differences in performance between Loop and BiAP, and in particular its adaptive version, are more evident in the more challenging scenario 2 may be explained by the nature of the controllers. Loop is based on a model-predictive strategy and BiAP is primarily a reactive controller, similar to a PID; hence, Loop’s performance is closely tied to the goodness of the glucose forecasting models it uses. The BiAP suffers less from this problem because it does not rely on a long-term glucose prediction. Furthermore, as can be observed in Figure 2, BiAP tends to deliver insulin in response to a glucose increase in a shorter period when compared with Loop. This might also help to explain BiAP’s advantage over Loop because the former could be better suited to deal with the slow insulin pharmacokinetics.
Another reason that might partially explain the relative performance of Loop when compared with BiAP is the fact that Loop control algorithm bases its decision on a glucose forecast over a 6-hour window. Then, if a hypoglycemic event is predicted within the post-prandial period forecast, Loop will not suggest a meal bolus, which may lead to excessive hyperglycemia.
The BIAP-Adaptive shows a trend toward giving the most insulin but has lower, or equal, percentage time in hypoglycemia. This can be explained by a more optimized ICR, thanks to the adaptation mechanisms. This advantage might be less evident in a real-life scenario when meal carbohydrate content and absorption are even more challenging.
This study has several limitations. Although efforts have been put into generating realistic in silico simulations that yield glycemic results close to clinical trials from the literature, these scenarios are still not able to include all the variability and perturbations seen in a clinical or real-world setting. One example is the lack of a mathematical model that can realistically emulate the effect of physical exercise on glucose regulation.
As previously described, we decided not to include snack and rescue carbohydrates to challenge the controller with respect to hypoglycemia. However, this is not what would be expected in a real-life scenario. Hence, the provided results might underestimate the percentage time in hyperglycemia and overestimate hypoglycemia. Another aspect that can be affected by the inclusion of snack and rescue carbohydrates is the convergence speed of the adaptation mechanism of the bolus calculator in BiAP because these perturbations can mask or invalidate the adaptation. Hence, the provided results might overestimate the benefits of the adaptation mechanism when compared with a real-life setting.
Regarding the Tandem Basal-IQ algorithm, we have implemented it based on information provided in the user manual and scientific literature; however, the actual implementation of this algorithm in the commercial product might differ from our implementation, so these results should be interpreted with caution.
Finally, we have done everything possible to tune the controllers to optimize glycemic outcome. However, no formal method has been employed for this purpose. Instead, a mix of clinical and empirical knowledge was employed. If scenario 1 is considered, Loop can be tuned more aggressively to increase percentage time in range. However, if this is done, then, a clinically significant increase in percentage time below range (eg, >1%) in scenario 2 was observed.
Conclusion
Both Loop and BiAP closed-loop control algorithms are safe and improve, or maintain, glycemic control when compared with the evaluated baseline algorithm (Tandem Basal-IQ).
Under scenarios with less uncertainty and perturbations, BiAP and Loop perform relatively similarly. However, BiAP appears more robust to real-world challenges such as variability in the basal-bolus therapy settings and skipped or delayed meal announcements. The version of BiAP with bolus calculator setting adaptation appears capable of improving percentage time in range further without an increase in hypoglycemia. Therefore, BiAP has the potential to provide superior glycemic control to Loop in adult and adolescent populations with type 1 diabetes.
Supplemental Material
Supplemental material, sj-docx-1-dst-10.1177_19322968211060074 for An In Silico Head-to-Head Comparison of the Do-It-Yourself Artificial Pancreas Loop and Bio-Inspired Artificial Pancreas Control Algorithms by Ryan Armiger, Monika Reddy, Nick S. Oliver, Pantelis Georgiou and Pau Herrero in Journal of Diabetes Science and Technology
Footnotes
Authors’ Note: A preliminary version of this work was presented as an oral presentation at the 14th International Conference on Advanced Technologies & Treatments for Diabetes, Virtual, 3-5 June 2021.
Abbreviations: AID, automated insulin delivery; CGM, continuous glucose monitoring;IY, do-it-yourself; DIY APS, do-it-yourself artificial pancreas system; BiAP, bio-inspired artificial pancreas; ICR, insulin-to-carbohydrate ratio; ISF, insulin sensitivity factor; PLGS, predictive low-glucose insulin suspend system.
Declaration of Conflicting Interests: The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: N.O. has received research funding from Dexcom and Roche Diabetes, and is a member of advisory boards for Dexcom, Roche Diabetes, and Medtronic Diabetes.
Funding: The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project has received funding from the Imperial College London Diabetes Centre (Abu Dhabi, UAE).
ORCID iD: Pau Herrero 
https://orcid.org/0000-0002-7088-5807
Supplemental Material: Supplementary material for this article is available online.
References
- 1. Boughton CK, Hovorka R. Advances in artificial pancreas systems. Sci Transl Med. 2019;11(484):eaaw4949. [DOI] [PubMed] [Google Scholar]
- 2. Weaver KW, Hirsch IB. The hybrid closed-loop system: evolution and practical applications. Diabetes Technol Ther. 2018;20(S2):S2-S16. [DOI] [PubMed] [Google Scholar]
- 3. Breton MD, Kanapka LG, Beck RW, et al. A randomized trial of closed-loop control in children with type 1 diabetes. New Engl J Med. 2020;383(9):836-845. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Stone MP, Agrawal P, Chen X, et al. Retrospective analysis of 3-month real-world glucose data after the MiniMed 670G system commercial launch. Diabetes Technol Ther. 2018;20(10):689-692. [DOI] [PubMed] [Google Scholar]
- 5. Stone JY, Haviland N, Bailey TS. Review of a commercially available hybrid closed-loop insulin-delivery system in the treatment of Type 1 diabetes. Ther Deliv. 2018;9(2):77-87. [DOI] [PubMed] [Google Scholar]
- 6. Bequette BW. Algorithms for a closed-loop artificial pancreas: the case for model predictive control. J Diabetes Sci Technol. 2013;7(6):1632-1643. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Steil GM. Algorithms for a closed-loop artificial pancreas: the case for proportional-integral-derivative control. J Diabetes Sci Technol. 2013;7(6):1621-1631. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Nimri R, Phillip M. Artificial pancreas: fuzzy logic and control of glycemia. Curr Opin Endocrinol Diabetes Obes. 2014;21(4):251-256. [DOI] [PubMed] [Google Scholar]
- 9. Herrero P, El-Sharkawy M, Daniels J, et al. The bio-inspired artificial pancreas for type 1 diabetes control in the home: system architecture and preliminary results. J Diabetes Sci Technol. 2019;13(6):1017-1025. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Pinsker JE, Lee JB, Dassau E, et al. randomized crossover comparison of personalized MPC and PID control algorithms for the artificial pancreas. Diabetes Care. 2016;39(7):1135-1142. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Steil GM. Comment on Pinsker et al. Randomized crossover comparison of personalized MPC and PID control algorithms for the artificial pancreas. Diabetes Care 2016;39:1135-1142. Diabetes Care. 2017;40(1):e3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Oliver N, Reddy M, Marriott C, Walker T, Heinemann L. Open source automated insulin delivery: addressing the challenge. NPJ Digit Med. 2019;2(1):124. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Kesavadev J, Srinivasan S, Saboo B, Krishnan G. The do-it-yourself artificial pancreas: a comprehensive review. Diabetes Ther. 2020;11(6):1217-1235. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. The OPEN Project. https://open-diabetes.eu/. Accessed September 25, 2021.
- 15. Melmer A, Züger T, Lewis DM, Leibrand S, Stettler C, Laimer M. Glycaemic control in individuals with type 1 diabetes using an open source artificial pancreas system (OpenAPS). Diabetes Obes Metab. 2019;21(10):2333-2337. [DOI] [PubMed] [Google Scholar]
- 16. Petruzelkova L, Jiranova P, Soupal J, et al. Pre-school and school-aged children benefit from the switch from a sensor-augmented pump to an AndroidAPS hybrid closed loop: a retrospective analysis. Pediatr Diabetes. 2021;22:594-604. [DOI] [PubMed] [Google Scholar]
- 17. Toffanin C, Kozak M, Sumnik Z, Cobelli C, Petruzelkova L. In silico trials of an open-source Android-based artificial pancreas: a new paradigm to test safety and efficacy of do-it-yourself systems. Diabetes Technol Ther. 2020;22(2):112-120. [DOI] [PubMed] [Google Scholar]
- 18. Lum JW, Bailey RJ, Barnes-Lomen V, et al. A real-world prospective study of the safety and effectiveness of the loop open source automated insulin delivery system. Diabetes Technol Ther. 2021;23:367-375. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Man CD, Micheletto FLD, Breton M, Kovatchev B, Cobelli C. The UVA/PADOVA type 1 diabetes simulator: new features. J Diabetes Sci Technol. 2014;8(1):26-34. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Reddy M, Herrero P, El-Sharkawy M, et al. Feasibility study of a bio-inspired artificial pancreas in adults with type 1 diabetes. Diabetes Technol Ther. 2014;16(9):550-557. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Reddy M, Herrero P, El Sharkawy M, et al. Metabolic control with the bio-inspired artificial pancreas in adults with type 1 diabetes: a 24-hour randomised controlled crossover study. J Diabetes Sci Technol. 2016;10:408-413. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Müller L, Habif S, Leas S, Aronoff-Spencer E. Reducing hypoglycemia in the real world: a retrospective analysis of predictive low-glucose suspend technology in an ambulatory insulin-dependent cohort. Diabetes Technol Ther. 2019;21(9):478-484. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. LoopKit. https://loopkit.github.io/loopdocs/. Accessed September 25, 2021.
- 24. PyLoopKit. https://github.com/tidepool-org/PyLoopKit. Accessed June 21, 2021.
- 25. GitHub repository for interfacing PyLoopKit with the UVa-Padova simulator (v3.2). https://github.com/RyanArmiger/Loop-Simulator-Interface. Accessed June 21, 2021.
- 26. Herrero P, Bondia J, Adewuyi O, et al. Enhancing automatic closed-loop glucose control in type 1 diabetes with an adaptive meal bolus calculator—in silico evaluation under intra-day variability. Comput Methods Programs Biomed. 2017;146:125-131. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. Ziegler R, Freckmann G, Heinemann L. Boluses in insulin therapy: a commentary. J Diabetes Sci Technol. 2017;11(1):165-171. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Tandem t:slim with Basal-IQ user guide. https://www.tandemdiabetes.com. Accessed September 25, 2021.
- 29. Buchanan J, Zabinsky JA, Ferrara-Cook C, Adi S, Wong JC. Comparison of insulin pump bolus calculators reveals wide variation in dose recommendations. J Diabetes Sci Technol. 2021;15:1290-1296. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Liu C, Vehí J, Avari P, et al. Long-term glucose forecasting using a physiological model and deconvolution of the continuous glucose monitoring signal. Sensors. 2019;19(19):4338. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31. Norlander LM, Anderson S, Levy CJ, et al. Late and missed meal boluses with multiple daily insulin injections. Diabetes. 2018;67(suppl 1):992-P. doi: 10.2337/db18-992-P. [DOI] [Google Scholar]
- 32. Forlenza GP, Li Z, Buckingham BA, et al. Predictive low-glucose suspend reduces hypoglycemia in adults, adolescents, and children with type 1 diabetes in an at-home randomized crossover study: results of the PROLOG trial. Diabetes Care. 2018;41(10):2155-2161. [DOI] [PubMed] [Google Scholar]
- 33. Maahs DM, Buckingham BA, Castle JR, et al. Outcome measures for artificial pancreas clinical trials: a consensus report. Diabetes Care. 2016;39(7):1175-1179. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supplemental material, sj-docx-1-dst-10.1177_19322968211060074 for An In Silico Head-to-Head Comparison of the Do-It-Yourself Artificial Pancreas Loop and Bio-Inspired Artificial Pancreas Control Algorithms by Ryan Armiger, Monika Reddy, Nick S. Oliver, Pantelis Georgiou and Pau Herrero in Journal of Diabetes Science and Technology