Machine learning models with uncertainty estimates for molecular properties are often used for chemical decision making tasks such as in experimental design or molecular discovery. These algorithms leverage statistical patterns in data, and often increase in performance with more available data. How effective are these models when the amount of data is low? This is a practical concern when building a new dataset or starting an experiment, due the resource intensive nature of experimentation and synthesis. Additionally, recent studies have demonstrated that, while accurate, machine learning models are often poorly calibrated, meaning the confidence of the model does not correlate with the accuracy. How do these observations hold in the low data regime and how does the calibration and accuracy impact experimental campaigns? In this talk, we look at trends in predictive performances for a diverse class of models, and a variety of featurization methods on small chemical datasets for both regression and classification tasks. We consider models such as ensemble decision trees, Graph Neural Networks (GNNs), Gaussian Processes (GP), and generalized linear models. To probe at the generalization capabilities of models and the quality of uncertainty estimates, we propose two in-silico experiments that can be performed on any dataset or model. We look at prediction trends for clustered out-of-distribution molecules, simulating the performance for novel chemistry classes in a discovery setting. For uncertainty, we examine its influence in performance in a sequential molecular selection task, guided by a Bayesian optimization algorithm.

Uncertainty and generalization of predictive models for chemistry in the low data regime

Fall 2022 abstracts, posters, and presentations will launch in mid-September.

ACS Fall 2022

The official archive and repository for all ACS Meetings & Expositions abstracts and select posters or presentations.