Title: User performance evaluation of 4 blood glucose monitoring systems based on ISO 15197 and calculated insulin dosing error

 
Authors: Guido Freckmann, Annette Baumstark, Stefan Pleus, Ulrike Kamecke, Nina Jendrike


Background
Accurate blood glucose (BG) values obtained by people with diabetes with systems for self-monitoring of blood glucose (SMBG) are essential to ensure adequate therapeutic decisions. The International Organization for Standardization (ISO) 15197:2013 is an established standard for the performance evaluation of SMBG systems. ISO 15197:2013 was harmonized with the regulations of the European Union as EN ISO 15197:2015. This harmonization had no impact on the requirements and procedures in ISO 15197:2013; changes were made to the foreword and an informative annex. ISO 15197:2013 also requires a user performance evaluation to show if intended lay-users are able to obtain accurate measurement results with a given system. According to ISO 15197:2013, system accuracy in the hand of lay-users is acceptable if at least 95% of results measured with the SMBG system are within ±15 mg/dl of comparison measurement results at blood glucose (BG) concentrations <100 mg/dl and within ±15% at BG concentrations ≥100 mg/dl. 

Aims
In this study, accuracy of four SMBG systems when used by lay-persons and when used by trained personnel was assessed based on testing procedures and acceptance criteria of ISO 15197:2013, clause 8. In addition, insulin dosing error was modeled. 

Method
The systems Accu-Chek Performa Connect [A], Contour Plus One [B], FreeStyle Optium Neo [C] and OneTouch SelectPlus [D] were evaluated with one test strip lot per system. 
For each system, data from 100 subjects were included for analysis. 
Subjects had the opportunity to familiarize themselves with the system (designated training meter), i.e. read the instructions for use and perform a limited number of training measurements with control solution, before the BG measurement. Before the measurement procedure, subjects were asked to wash and dry their hands. Then, subjects collected a capillary blood sample from the fingertip and performed a BG measurement with the SMBG system. Subjects were allowed to repeat the measurement up to three times if they reported measurement mistakes or an error message occurred. Then, study personnel performed two additional SMBG measurements as well as comparison measurements by using a glucose oxidase (GOD) and a hexokinase (HK) method.
Accuracy was evaluated by comparing the SMBG measurement result with the respective mean result of the comparison measurements (GOD or HK, depending on the manufacturer’s labeling).  
For the estimation of insulin dosing errors, the difference between insulin doses based on SMBG measurement results and comparison measurement results was calculated. For the calculation of insulin doses, a simple model with fixed therapeutic parameters was used that is similar to the calculations performed by automated bolus advisors, e.g., in some insulin pumps (insulin-to-carb ratio: 1/15 units/grams, insulin sensitivity: 1/25 units/(mg/dl), target BG: 100 mg/dl, carb content: 60g). 

Results
Three of the four systems fulfilled ISO 15197 user performance accuracy criteria with at least 95% of results within the defined limits (±15 mg/dl, ±15%): A: 96%/99% (lay-users/study personnel); B: 100%/100%; C: 98%/99.5%. System D showed acceptable accuracy when measurements were performed by study personnel (96%); however, when lay-users performed the measurements, only 92% of results were with the defined limits. Only system B fulfilled accuracy criteria when more stringent limits (±10 mg/dl, ±10%) were applied with 99%/100% (lay-users/study personnel) of results within these limits. Frequently observed lay-user errors were not checking the test strip’s expiry date and incorrect or premature blood application.
Median modeled insulin dosing errors were as follows:  A: -0.5/-0.4 units (lay-users/study personnel); B: -0.3/-0.2 units; C: +0.3/+0.2 units; D: -0.6/-0.7 units. The 99% ranges of all four systems were found between insulin dosing errors of -2.4 and +1.4 units (lay-users) and -2.5 and +1.4 units (study personnel). 


Discussion
In this study, three systems fulfilled accuracy acceptance criteria of ISO 15197:2013 in the hands of lay-users. None of the investigated systems showed median calculated insulin dosing errors larger than ±0.7 units. SMBG systems should be easy to use and resistant to errors to ensure comparable measurement accuracy in the hands of intended users and professionals.



This study was supported by Ascensia Diabetes Care Deutschland GmbH.

GF is general manager of the IDT (Institut für Diabetes-Technologie Forschungs- und Entwicklungsgesellschaft mbH an der Universität Ulm, Ulm, Germany), which carries out clinical studies on the evaluation of BG meters and medical devices for diabetes therapy on its own initiative and on behalf of various companies. GF/IDT have received speakers’ honoraria or consulting fees from Abbott, Ascensia, Bayer, Berlin-Chemie, Becton-Dickinson, Dexcom, LifeScan, Menarini Diagnostics, Novo Nordisk, Roche, Sanofi, Sensile and Ypsomed.
AB, SP, UK, NJ have disclosed that they are employees of IDT.