: ‘Overfitting’ is a phenomenon in which a mathematical model attains high performance on its training data by ‘memorizing’ its points. This is reflected in significant differences in the statistical performance of a model in predicting its training data and predicting external validation data unseen during the model’s training. Conventional wisdom suggests such models will necessarily internalize noise or spurious correlations within the training data, and that models which achieve parity between training and validation performance have a reached a more desirable state of generalization. Typically, such generalized models are observed to perform better on their external sets, while the overfit models perform worse. An empirical observation of QSA/PR (Quantitative Structure Activity/Property Relationship) models which, when more ‘overfit’, perform better on external data is made across numerous endpoints, suggesting a trend that is at odds with modeling best practices. This project analyzes possible causes for and attempts to reconcile the observed behavior with the theories of computational modeling. We consider the possibility of multiple signals within the data describing different mechanisms generating the endpoint, the influence of coverage within chemical space, and the relevance of hyperparameterization to model fitting and performance.

Analysis of overfitting in machine learning models

Fall 2022 abstracts, posters, and presentations will launch in mid-September.

ACS Fall 2022

The official archive and repository for all ACS Meetings & Expositions abstracts and select posters or presentations.