The art of CGM accuracy studies, part 2: The Freestyle Libre3

As I’m currently testing a Libre3, and have taken a look back at the various accuracy studies as a part of this to review MARD data, I thought it worth while in taking a quick look at the Libre3, and it’s much vaunted accuracy data.

“But that’s quite a cynical title”, you might be saying to yourself. “What could that have to do with Abbott?”

In the previous article on this topic, we discussed the Libre2 and Dexcom G7 studies, and how they applied gold standard techniques and had similar approaches and outcomes.

Between that and the Libre3, Abbott appear to have gone away and reviewed the data they received from the study, before assessing the impact of various data on the results.

Surprisingly good study results

Before we go down that route, let’s take a quick at the update to Libre3 accuracy. The numbers are very good. An overall Adult MARD of 7.6% was presented at ADA in the summer of 2022, as shown in the table below.

Now it has to be said that we heard from numerous sources that they were really surprised at the outcome. The word is that the Libre 3 hardware and algorithm are essentially the same as the Libre2, so it wasn’t clear what had changed.

But the clues are there, and what they paint is a less than rosy picture.

What really happened?

If you’ve ever watched Kamil’s video over at Nerdabetic on You Tube, the clue is hiding in plain sight, where he says that Abbott have no glucose manipulation data.

Read the late breaking summary from the ADA meeting this summer and the clue is hiding by omission.

I guess Abbott figured out from the data that if they omitted the glucose manipulation process, where, by raising and lowering the glucose levels artificially they test the sensor at high and low levels, they would see better MARD results. After all, the Libre 2 study published this summer showed a 9.2% MARD but included glucose manipulation..

So why such a big difference?

Well the devil is once again in the detail. This change in study detail results in around only 2% or 3% of readings Vs YSI being in the low, less accurate range, while in the Libre 2 version, we have around 7%. That difference seems to make a huge difference to the overall MARD, which suggests that when you’re below 70mg/dl or 3.9 mmol/l, the Libre3 isn’t quite as accurate as you’d expect.

What’s the point of this post?

Very simply, we don’t know whether the Libre3 is more accurate than the Libre2. All the analysis that was done with the Libre2 was not repeated with the Libre3. Instead, Abbott chose to remove the glucose manipulation testing, which removed a whole chunk of data from the study, and changed the weighting of the data within it. Far less low data was shown, and as a result a better MARD reported.

Once again, manipulation of the study method results in data that is real data, but doesn’t allow you to compare like with like, so obfuscates any comparisons.

This provides yet more evidence that a standardized approach is required to allow comparison of devices. It also demonstrates very clearly that many of the measures we use to assess CGM accuracy are, in and of themselves, flawed.

Either we need a standard approach, or we need everyone to be aware of how easily the data can be manipulated, as it effects the decisions that payers make.

In many ways, Abbott may have done everyone a huge favour by undertaking this study and openly demonstrating how easy it is to change results, without, ostensibly, doing anything wrong!