A great deal of work has been done on quantitative structure-activity relationship (QSAR) validation, e.g., on assessing the overall reliability of such a model. Unfortunately, even good QSAR models are liable to give bad predictions when extrapolated too far from the structural data from which they were built. Appreciation of that fact has led to innovative work on how to define predictive applicability domains, but such efforts generally only yield broad qualitative indications of predictive reliability. The degree of consensus among predictions from the components of an ensemble model is an attractive basis for obtaining more quantitative estimates of predictive uncertainty, but exploiting that information effectively is complicated by those predictions not being statistically independent. In the biometrics field, such interdependence is often handled by fitting "over-disperse" distributions such as the beta binomial to the data. Indeed, we recently1 showed that by judicious application of such an analysis to the degree of ensemble consensus, we could accurately estimate the predictive uncertainty for individual classifications. Here we have extended the logic underlying that methodology to continuous outputs from regression models. 1. Clark et al. Using beta binomials to estimate classification uncertainty for ensemble models. J Cheminfo 2014, 6, 34.

Assessing the reliability of individual regression predictions

251st National Meeting (2016)

The official archive and repository for all ACS Meetings & Expositions abstracts and select posters or presentations.