Title: User performance evaluation of 5 SMBG systems based on ISO 15197 and calculation of insulin dosing error 


Authors: Nina Jendrike, Annette Baumstark, Stefan Pleus, Ulrike Kamecke, Guido Freckmann


Background
For people with diabetes, accurate blood glucose (BG) values obtained with systems for self-monitoring of blood glucose (SMBG) are essential to ensure adequate therapeutic decisions. The International Organization for Standardization (ISO) 15197:2013 standard is widely accepted for the performance evaluation of SMBG systems. ISO 15197:2013 was harmonized with the regulations of the European Union as EN ISO 15197:2015. This harmonization had no impact on the requirements and procedures in ISO 15197:2013; changes were made to the foreword and an informative annex. ISO 15197:2013 also requires a user performance evaluation to show if intended lay-users are able to obtain accurate measurement results with a given system. According to ISO 15197:2013, system accuracy in the hand of lay-users is acceptable if at least 95% of results measured with the SMBG system are within ±15 mg/dl of comparison measurement results at BG concentrations <100 mg/dl and within ±15% at BG concentrations ≥100 mg/dl. 

Aims
This study aimed at assessing accuracy of five SMBG systems when used by lay-persons and when used by trained personnel based on testing procedures and acceptance criteria of ISO 15197:2013, clause 8. In addition, insulin dosing error was modeled. 

Method
The evaluation was performed with one test strip lot for each of the following five systems: A: Accu-Chek® Aviva Connect (Roche Diabetes Care GmbH), B: Contour® next One (Ascensia Diabetes Care Holdings AG), C: FreeStyle Freedom Lite (Abbott Diabetes Care Ltd.), D: GlucoMen® areo (A. Menarini Diagnostics S.r.l.), E: OneTouch® Verio (LifeScan Europe). 
For each system, data from 100 subjects were included for analysis. 
Before the measurement procedure with each SMBG system, subjects had the opportunity to familiarize themselves with the system (designated training meter), i.e. read the instructions for use and perform a limited number of training measurements with control solution. Subjects were asked to wash and dry their hands before the measurement procedure. Then, subjects collected a capillary blood sample from the fingertip and performed a BG measurement with the SMBG system. Subjects were allowed to repeat the measurement up to three times if they reported measurement mistakes or an error message occurred. Then, study personnel performed two additional SMBG measurements, and comparison measurements with a glucose oxidase (GOD) and a hexokinase (HK) method.
Accuracy was evaluated by comparing the SMBG measurement result with the respective mean result of the comparison measurements (GOD or HK, depending on the manufacturer’s labeling).  
Insulin dosing errors were estimated by calculating the difference between insulin doses based on SMBG measurement results and comparison measurement results (GOD method, indicated for four of the five systems as manufacturer’s comparison method in the respective labeling). For the calculation of insulin doses, a simple model with fixed therapeutic parameters was used that is similar to the calculations performed by automated bolus advisors, e.g., in some insulin pumps (insulin-to-carb ratio: 1/15 units/grams, insulin sensitivity: 1/25 units/(mg/dl), target BG: 100 mg/dl, carb content: 60g). 


Results
Four of the five systems showed acceptable accuracy with at least 95% of results within the defined limits (±15 mg/dl, ±15%): A: 96%/100% (lay-users/study personnel); B: 100%/100%; C: 95%/99.5%; D: 95%/100%. System E only showed 93% (lay users) and 88.5% (study personnel) of results within the required limits. Frequently observed lay-user errors were not checking the test strip’s expiry date, incorrect or premature blood application and mistakes in device handling.
The systems showed the following median modeled insulin dosing errors:  A: -0.4/-0.2 units (lay-users/study personnel); B: -0.3/-0.1 units; C: -0.8/-0.7 units; D: +0.4/+0.1 units; 
E: +0.6/+0.8 units. All 99% ranges were found between dosing errors of -2.7 and +3.4 units (lay-users) and -2.7 and +2.2 units (study personnel). 

Discussion
Accuracy acceptance criteria of ISO 15197:2013 were fulfilled by four systems, with only one system showing 100% of results within the defined limits in the hands of both, lay users and professionals. In the insulin dosing error model, none of the investigated systems showed median insulin dosing errors larger than ±0.8 units. The calculated dosing error tended to be larger when measurements were performed by lay-users. For comparable measurement accuracy in the hands of intended users and professionals, SMBG systems should be easy to use and resistant to errors.



This study was supported by Ascensia Diabetes Care Deutschland GmbH.

GF is general manager of the IDT (Institut für Diabetes-Technologie Forschungs- und Entwicklungsgesellschaft mbH an der Universität Ulm, Ulm, Germany), which carries out clinical studies on the evaluation of BG meters and medical devices for diabetes therapy on its own initiative and on behalf of various companies. GF/IDT have received speakers’ honoraria or consulting fees from Abbott, Ascensia, Bayer, Berlin-Chemie, Becton-Dickinson, Dexcom, LifeScan, Menarini Diagnostics, Novo Nordisk, Roche, Sanofi, Sensile and Ypsomed.
AB, SP, UK, NJ have disclosed that they are employees of IDT.