Insulin Pump (Dose-to-Dose) Accuracy

Abstract

This article reviews the concept of using the proper methods for the proper task, in this case measuring dose-to-dose accuracy of continuous subcutaneous insulin infusion pumps.

How does anyone measure anything accurately and precisely? In this issue, Borot and colleagues report on the accuracy of the Jewel MEMS pump in comparison to current commercially available pumps. In addition, they include data on pilot clinical data. The article is informative and well written, but uses previously published methods that may not be telling the whole story.

How does anyone concretely know something that is inherently hard to measure at a deep and precise level? I believe the answer involves thinking about what it is you want to characterize in a deep way and then rethinking the process over and over again. I was invited to write this analysis/editorial because I have thought about the issue of insulin pump accuracy for quite some time. Let me explain. Before the Sansum/UCSB research team could use our artificial pancreas system (APS) in human clinical trials, I needed to obtain approval from the FDA. After the team submitted our Investigational Device Exemption for approval, I received a request from the agency to demonstrate the dose-to-dose accuracy of the OmniPod (Insulet, Billerica, MA). Their request was designed to demonstrate that the pump was delivering precisely what the APS requested of it. Using a precision balance housed in a sophisticated temperature/humidity controlled room was the accepted method to measure trumpet curves and average insulin pump delivery over time. It would not be an appropriate method to measure dose-to-dose accuracy at the lowest limit of a pump’s dosing range.

So I did what any good scientist or musician would have done and improvised. Improvisation combines using information that you have learned in the past with creativity in the moment. Another project that I was working on at the time was investigating what happens when insulin pumps are asked to pump against or with gravity.¹ I used graduated pipets and digital photography to demonstrate that when traditional tubed pumps are asked to pump against gravity, they tend to underdeliver, and when asked to with gravity, they tend to overdeliver. I attempted to use the same setup to demonstrate dose-to-dose accuracy, as requested by the FDA. After submitting the data set to the FDA, they requested that I use an additional method to further characterize the pump’s accuracy. A third project that I was working at the time was looking at the safety and efficacy of an investigational intravascular continuous glucose sensor. I was using a digital microscope to record what the sensor surface looked like after removal from the vessel. This digital microscope was used to design the second method,² which involved calculating the volume of each individual bolus using the diameter of a sphere. I believed and continue to believe that this is a superior method to measure dose-to-dose accuracy in insulin infusion pumps. An accompanying commentary article by Ochoa and Ziaie³ said of the new method:

The high resolution images acquired by this method allow for a much more reliable volume characterization for the lowest dosage levels. This measurement method could become the new standard for evaluating the bolus dose accuracy of future drug delivery patches.

If you have not seen what 0.05 units of insulin looks like, I would invite you to prime a pump or pod and deliver this amount. Take a close look. What you will find is that at these small volumes, due to hydrostatic forces, the bolus remains on the end of the catheter opening and hence makes it extremely difficult to measure dose-to-dose variability by having drops drip into a beaker on top of a balance or scale.

In response to my publication, Jahn and colleagues⁴ used the beaker/balance method in an attempt to measure dose-to-dose delivery accuracy for a number of pumps. This procedure was designed to measure pump dose accuracy over time and was never meant to be used for dose-to-dose measurements. I believe they used the wrong tool for the job. Prior to publication, the results were presented in poster form. Posters, while helpful, have a low density of information. It was clear from the poster that the setup methods used for traditional pumps and the methods used for the Omnipod. The Omnipod was actually placed inside of the measuring chamber. It was not until I read the article and looked at Figure 5, which showed their dose-to-dose results for individual pumps, that I realized they were not necessarily measuring what they thought they were measuring. They were using the wrong tool. I won’t go into too much detail, but suffice it to say the measurement results had episodic spikes, which due to the design of the Omnipod, could not have been due to large variability in delivery volumes. The Omnipod uses a paired SMA wire design for its motor drive and is designed to have paired dosing—if 1 pulse is slightly off in a positive direction, the next dose will cancel out any small error. It was also designed to fail in a safe manner, eliminating the potential for electromechanical discharge: meaning that it will not empty the contents of the entire pump (“runaway”). These large noise spikes occurred with a regular periodicity every 3 hours. I believe what they were actually measuring was an episodic change in environmental conditions. These uber-sensitive balances need to be housed in an isolated environment, to prevent electrostatic and temperature related interference or drift, which was not the case for the methods used to measure the Omnipod delivery.⁵ Since the Omnipod does not have any tubing, the investigators chose to place the pod inside the chamber. Another possibility was that they were measuring the movement or vibration of the pump during delivery and not the actual volume of the insulin being delivered.

Why is it so important to make these detailed comments on the methods of a previously published article? Once something makes it into the cannon of scientific literature, it is thought of as gospel and is often used by industry to differentiate their products. I will say that while the peer-review process may not have worked perfectly for the Jahn et al publication, I believe it has helped the Borot et al article move from a marketing device to a scientific article. Their conclusions are that all of the pumps meet the ISO standards. They all do so in their own unique manner. One would expect a MEMS device to deliver in a more accurate fashion since it does not have to deal with issues such as stiction and compliance seen in syringe-based pumps.

At what point does insulin dose-to-dose accuracy become clinically relevant? As previously reported,⁶ insulin pumps are more accurate and precise than pens, which are more accurate that syringes. I strongly believe that all currently approved pumps meet ISO standards and deliver insulin accurately enough. At some point, any improved accuracy in dosing becomes clinically irrelevant.

For example, lets look at a patient with a correction factor of 1:100, that is, all things being equal; 1 unit of insulin would lower their blood glucose concentration by 100 mg/dl. An injection 0.1units of insulin should lower the blood glucose by 10 mg/dl. Continuing this line of reasoning, an injection of 0.05 units should lower the blood glucose of this insulin sensitive patient by 5 mg/dl. Say the pump delivered 0.06 units instead. This would result in lowering their blood glucose by 6mg/dl or an overall difference of 1 mg/dl.

It would be ideal if these accuracy studies could be repeated using the digital photography/sphere technique. The only true way to determine whether or not these microdifferences in insulin delivery would be to do a detailed pk/pd studies to show whether or not any of these differences directly impact insulin absorption or action in any clinically significant way. One can always point to theoretical differentiators, but it really matters only if it affects the patient in a clinically significant way as I have demonstrated in a manuscript on the clinical relevance of interrupting basal insulin delivery:⁷ on average, people’s blood glucose will go up 1 mg/dl for every minute that their basal delivery is interrupted. Similar results were seen when insulin delivery was interrupted for 3 hours in association with physical activity.⁸

For those of you interested in the history of how standard measurements come into existence and how to precisely measure anything, I would recommend listening to the following Radiolab episode: http://www.radiolab.org/story/kg/. For those interested in the history of insulin pump accuracy, I would recommend Jackman et al⁹ and Birch et al.¹⁰