Comparing the #Loop and #OpenAPS Algorithms

It’s the numbers, silly…

The first thing that anyone dabbling in looping needs to consider is that you need to know your numbers. That means plenty of testing. And when I say testing, that’s basal testing, Insulin duration testing, I:C ratio testing and ISF testing. That means doing I:C and ISF at different times of day to confirm that you don’t get changes and therefore need to have them added differently.

Why’s this important? Well both Loop and OpenAPS work out your insulin adjustments using this data, and if it’s wrong, then both struggle to keep you in target and deliver a good result.

It’s not just a case of picking up an iPhone, bulding Loop and connecting it to the RileyLink and pump. If you’re moving from MDI, you need to read Walsh’s “Pumping Insulin” and understand and refine your basal rates. Why? Because if those are wrong then your ISF and IC numbers, typically will be wrong, and then whichever loop you choose will be all over the place.

You also need to make sure your Duration of Insulin Action is correct. Again, most of us when using MDI underestimate the amount of time it takes to get insulin out of our systems. The reality is that, at a dose of 0.1U/kg, it tends to hang around for 5-6 hours (based on clamp studies over a number of people).

If you don’t spend the time to work these factors out properly, you’ll find that the loop will run you up and down in a way that can be quite unpleasant. So before even considering closed loop, or open loop, spend the time getting these right. It’s worth it, and for the time being, it’s going to be necessary for commercial applications too.

The second similarity is setting maximum TBRs. In both cases, it’s crucial to remember that setting max TBRs that are too high could be lethal. Only set them to something that you could recover from. Again, this is true whether you are looking at a commercial or open source system.

Using the Loops… OpenAPS or Loop?

Let’s make one thing clear before we start. Neither is perfect and both rely on user input, so there is a margin for error in each. In my experience though, there are some key factors that differentiate the two.

Much of the detail of the Loop algorithm is discussed in the Loop guide, available from the Loop Github, while Dana Lewis Scott Leibrand have written up most of the detail regarding how OpenAPS and specifically oref0 works on the DIPYS blog.

From an end user perspective, there are a few key differences between the two that I notice in everyday life with the systems. They can be condensed down to two headlines. “Form” and “Function”.

Form

We all know by now, I hope, that the Loop algorithm runs in Loopkit, which is a fantastic piece of work from Nate Racklyeft and others to build an artificial pancreas framework that runs on the iPhone. What we should also be aware of is that the algorithm is designed around him and those who participated in building it. But that’s for later.

The form of Loop is excellent. It runs on an iPhone, uses the Dexcom G5 app (or G4 share) on the phone and handles everything in one easy package. It can be installed by anyone with access to a Mac and has a great GUI showing all the information you might need plus a bolus wizard that incorporates Insulin on Board (IOB) into the calculation, giving you the ability to manage your condition from the phone interface.

OpenAPS is designed to work with the pump and NightScout (although you can collect CGM data locally via the Medtronic pump CGM system), and as such requires a connection to a NightScout instance in some shape or form to receive some of the command information, such as “Eating Soon”. NightScout isn’t such a nice GUI as a native Swift interface on the iPhone, but it isn’t at all bad and does what is required. It also adds a level of interoperability with tools such as IFTTT, which are more difficult to enact with the self contained model that Loop has.

Both systems are also capable of using the Medtronic Enlite glucose monitoring system, which takes away the reliance on Dexcom locally.

From a hardware standpoint, early in the cycle, the decision was taken to put the OpenAPS system on a separate bit of hardware – phones have a bad habit of stopping processes, however Loop was built with the phone as the brain. From a use perspective, this has some interesting side effects. The most prominent is that when you are in an area which has a lot of BLE noise, or when you use a lot of bluetooth devices, signal loss can be a problem. I’m not the only one who has experienced RileyLink and often the Dexcom G5 disconnecting from the phone. That can be quite a frustration. It’s less of an issue with OpenAPS, but RF interference can still be a problem there too.

As I’ve previously mentioned, portability plays into Loop’s hands as you only need a RileyLink, although with the advent of the explorer board, I’m tempted to move off Dexcom G5 and use enlites with my 722 to get my OpenAPS rig down to a similar size level.

In terms of use, then, Loop is probably slightly easier to interact with.

Function

Function is where the big differences between the two systems can be found.

The Basics

Both the Loop algorithm and oref0 (the OpenAPS algorithm) have basic functionality that is simple and consistent across both. They observe the sensor glucose levels and based on a calculation looking out, predict future blood glucose based on insulin on board and then adjust current glucose levels to try and get future, predicted levels within the target range. Loop’s algo is detailed in the earlier linked guide, and uses the Insulin Absorption curves from Glucodyn, whilst the basic oref0 functionality can be found here, about halfway down the page.

Essentially, both use the bolus and temporary basal history, combined with DIA and published IOB curves, to calculate current net IOB and use this to determine a future glucose level.

Both have a “deviation function” to determine whether glucose change is in accordance with the earlier algorithm and then modify insulin delivery if it is not.

In Loop this is called “Retrospective Correction” and works by checking back every time it makes a prediction. It logs the prediction it makes, and every time (every five mins) it makes a projection, it pulls the previous value from 30 mins ago, and includes the variation of that value from where the current status is to work out how far out it is and adjust insulin and future glucose level accordingly, decaying the variation over 60 mins.

In oref0,  a term called “BG Impact” or BGI, which is simply the current insulin activity (first derivative of IOB) * insulin sensitivity, is first calculated. This represents the degree to which BG “should” be rising or falling, and can be directly compared to the delta between the current and last CGM reading, or an average delta over the last 15m or so. To calculate the deviation, oref0 does this comparison, between the 15m average delta in CGM readings and the predicted BGI. It then applies that deviation as an adjustment to the eventual BG prediction. This is based on the simple assumption that if BG is rising or falling more than expected over the last 15 minutes, that trend is likely to continue over the next 15-30 minutes, and the magnitude of the projected deviation is approximately equal to what was seen over the last 15 minutes.

As can be seen here, both are very similar. In its most basic implementation, oref0 also includes a function called Bolus Snooze, which stops the basic algorithm from issuing a low temp basal immediately after a meal bolus if glucose values don’t necessarily correspond to the input meal data.

Both also have the ability to set a temporary target glucose level to deal with either workouts (higher) or ahead of food (lower, which drives increased insulin to get the GLUT4 receptors ‘unlocked’).

In its predicted glucose model, Loop also includes a feature called “Glucose Momentum”, which takes the last 15 mins of readings and linearly projects them out for a further 30 mins with a decay to zero. This feature takes priority in the future value prediction, after which the rest of the algorithm is applied.

That’s where the basics (and commonalities) end. The next steps for each are different and serve different purposes.

The more advanced functionality

The key differences between advanced functionalities lie in the raison d’étre of the two systems. Loopkit was conceived as a way to manage diabetic life from an iPhone, using predictive techniques. OpenAPS is derived from the need to manage overnight glucose levels more effectively.

Loop

Loop’s advanced functionality is a bolus capability that allows a bolus to be provisioned based on net IOB and future predicted glucose levels. In its base install, you cannot issue a bolus if the future predicted levels drop below the target range at any point. This can be modified in the code. The carb data that the code uses is based on the Gary Scheiner Carb absorption curves and defaults to 120, 180 and 240 mins. This means that when you bolus, you need to guess the carb absorption time.

It’s great to be able to bolus from the phone, but this carb absorption guess can be troublesome and has been seen, alongside the ISF and DIA variations, to cause those new to Loop to struggle to get effective closed looping. If you get the carb absorption wrong, it either over-temps early on in the absorption, or under-temps early and has to play catch up. This ends up in either highs or lows that shouldn’t really be occurring. To fix this, you have to go back in and adjust your meal carb data, which is a frustrating overhead.

The other piece it doesn’t handle all that well is foods with high fat content and therefore variable absorption. Users of Loop have had to derive a model to combat this that involves entering two meals. One for the carbs with a shorter absorption time, and bolusing accordingly, and one for the “protein” content of the meal with a much longer absorption time, typically 5+ hours, with no upfront bolus. Whilst this works quite well, it’s also much more to think about, and not so far removed from pumping on its own, as you are trying to replicate a Dual-Wave or Combination bolus.

The other piece of the Loop set-up that has raised some cause for concern is the bolus calculation in the presence of “Glucose Momentum”. If the glucose level is rising or falling with a relatively high velocity, the predicted bolus can be incorrect. This is because the bolus value is partially based on the predicted glucose level.  As Momentum takes precedence in this calculation, there are times when you may be approaching a peak and it predicts a much higher future value, so when bolusing in these conditions, users need to be aware of this.

oref0

oref0’s advanced features are “Advanced Meal Assist” (AMA) and “Autosensitivity” (Autosens).

Autosens looks at the past twenty four hours of data and establishes whether Insulin Sensitivity has varied based on the past 24 hours of sensor data. It looks at each BG data point and calculates the delta (actual observed change) over the last 5 minutes. It then compares it to “BGI” (blood glucose impact, which is how much BG *should* be dropping from insulin alone), and assesses the “deviations” (differences between the delta and BGI). The output number is a ratio, and this is used to adjust ISF and basal rates. I’ve found this extremely useful when being ill, and also used the adjusted ISF value in bolusing calculations, which has been very successful so far.

The other where Autosens can be really useful is in seeing how certain foods result in absorptions that are not what is expected, potentially due to the fat content of the food. I’ve noticed this phenomenon is particularly strong in Burrito Salads from the likes of Chipotle and Chilango, where a salad bowl with limited carbs (around 44g) but containing avocado, cheese and sour cream, can drop ISF by up to 30%. Not something I was expecting!

AMA does exactly what it says on the tin. It makes meals easier. As opposed to Loop, you don’t assume a carb absorption rate at the start of a meal. Instead, you can calculate a standard absorption rate for your body (or use the default) and then oref0-AMA will manage insulin up and down based on the following:

To calculate carbs absorbed to date, sum up the 5m deviations, for each 5m period since the meal carbs were entered, to determine how much effect carb absorption has actually had on BG so far, and by using the IC ratio and ISF, determine how many carbs have been absorbed, and conversely how many g of COB are remaining.

Then extrapolate that the current deviation (the current impact of carb absorption) will continue into the future until most of those carbs have absorbed. To simplify the calculations, it makes sense to extrapolate the current rate until those projected deviations have accounted for half the remaining COB. (In reality, deviations tend to gradually return to zero as carb absorption slows down, but the simpler math should be sufficient for estimating the total area under that curve.) Then use the projected BGI and deviations for the next few hours to determine the projected minimum and eventual BG. If projected minimum BG is below targeted min_bg (and below current BG), a low-temp can be set as needed to help bring that up. If projected minimum BG is above target (or above current BG), a high-temp can be set as required to bring eventual BG down to target.

In practice this means that you should enter carbs fairly correctly (as you do for bolusing) but you can enter slightly less insulin and AMA will handle the rest for you. It needs triggering with eating soon and a pre-bolus for best effect. For mixed meals, i.e. reasonable amounts of carbs, fats and protein, this is a very effective algorithm, and I’ve found it to be very successful in keeping me on the straight and narrow. What I’d note is that in reality, with AMA in place, you can actually graze and as long as you’ve hit eating soon, and the food isn’t massively carby, it really manages glucose levels well.

The key thing about AMA is that it works on top of and alongside the rest of the algorithm, and as we all know, carb absorption isn’t linear or defined and many of the things we do day to day can muck up a predefined model. This is why I prefer AMA to Loop for meal management.

At the ADA2016 conference, AMA was described as:

[AMA] is considered by some experts to be one of the most advanced and aggressive algorithms in existence for managing post-meal BG, and may be more advanced than anything that has yet been tested in clinical trials.

I am confident that with faster insulins, the use of AMA will preclude the need to bolus.

Conclusions

There are notable differences between OpenAPS and Loop/Loopkit and users who are jumping into this world for the first time really should be aware of them.

A lot of people have started with Loop because it’s relatively easy to set up on and fits better into their lifestyle. Regardless of this, they need to ensure that they have their numbers correct before they get going or it all goes horribly wrong.

Once in operation, whilst Loop is determinedly portable and great to carry around, the Loop algorithm is not as advanced, and has a few foibles that I have mentioned. It also requires much more interaction in order to get good results, and for me, one of the main points of a hybrid closed loop is to reduce my thinking about diabetes. I’ve found that Loop, around meals, requires almost as much interaction and observation as Sugar Surfing.

The newer hardware for OpenAPS has made it as portable as Loop and the work that has been done into set-up scripts has made it easy to install. It’s still a little more difficult than Loop, but not much.

From my perspective, it’s much easier to manage glucose levels with OpenAPS than with Loop, and I have multiple rigs located around the house and at work, using Loop only when I need the portability and offline comms (I don’t use a G5 receiver so can’t plug it into my OpenAPS rig). With some of the developments for local mongo databases and nightscout, this offline difference will become a thing of the past for OpenAPS.

So what do I recommend? Well, as with all these things, I’d say it’s up to you. But for me, I use OpenAPS most of the time, because the functionality serves me better. Advanced Meal Assist is an astonishing piece of work that makes the whole model of living with diabetes much, much easier.

In the future? Well I”ll be taking OpenAPS portable on the Edison and Explorer board, and if someone wanted to port oref0 to Loopkit, I’d be the first one willing to test.

I’ve taken a look, and my skillset in Swift simply isn’t up to it, but there are a few people out there who will have little issue with it.

As for the commercial products? one or two of them look interesting, but with the ongoing development of OpenAPS and what’s likely to be coming, I’d like to see openly accessible pumps so I can choose my algorithm…