Background and Goal of Study: The Fragility Index (FI), which represents the number of patients responsible for a statistically significant finding, has been suggested as an aid for interpreting the robustness of results from clinical trials. A small FI indicates that the statistical significance of a trial depends on only a few events. Our objectives were to calculate the FI of statistically significant results from randomized controlled trials (RCT) of anesthesia and critical care interventions and to determine the frequency of distorted presentation of results or ‘spin’.

Materials and Methods: We systematically searched MEDLINE from 01 January 2007 to 22 February 2017 to identify RCTs exploring the effect of critical care medicine or anesthesia interventions. Studies were included if they randomized patients 1:1 into two parallel arms and reported at least one statistically significant (P<0.05) binary outcome (primary or secondary). Two reviewers independently assessed eligibility and extracted data. The FI was determined for the chosen outcome. We assessed the level of spin in negative trials and the presence of recommendations for clinical practice in positive trials.

Results and Discussion: We identified 166 eligible RCTs with a median sample size of 207 patients (interquartile range [IQR] 109 to 497). The median FI was 3 (IQR 1-7), which means that adding three events to one of the trials treatment arms eliminated its statistical significance. Further, 21 (13%) trials had a FI of zero as the statistically significant outcome was found non-significant when recalculating the P-value using a 2-sided Fisher exact test. High spin was identified in 42% (n=30) of negative RCTs while 21% (n=20) of positive RCTs provided recommendations. Lower levels of spin and recommendations were associated with publication in journals with high impact factors (P<0.001 for both).

Conclusion: Statistically significant results in anesthesia and critical care RCTs are often fragile, and study conclusions are frequently affected by spin. Routine calculation of the FI in medical literature may allow for better understanding of trials and therefore enhance the quality of reporting.

10 / The fragility and reliability of conclusions of anesthesia and critical care randomized trials with statistically significant findings: A systematic review