For anyone working with novel organic molecules, the ability to determine if a certain compound is synthesizable and how complex that synthesis might be are incredibly important tasks. Even scientists who will not be conducting syntheses themselves are finding the need to incorporate retrosynthetic analysis into their workflows.<br/>Advances in computing power, data source quality, and machine learning and artificial intelligence algorithms have dramatically improved the quality of retrosynthesis proposed by CASP tools, so scientists with many different needs and goals have begun to incorporate them into their own workflows. With many different needs, goals, and expectations from users, and with many different tools available, both commercial and open source, determining which CASP tool to use is an incredibly complex problem.<br/>Two major issues are apparent when trying to make comparisons between different CASP tools: first, the playing fields are not level. Tools will be trained on different reaction datasets and they will have access to different catalogs of commercial starting materials. Second, judging the output of the different CASP tools is not at all straightforward. Similar to judging two different human-proposed retrosyntheses, the comparisons are highly subjective. Work has been done to quantify organic syntheses, but an ideal synthesis in one application does not mean it is ideal in all situations. To confound this, judging retrosynthesis routes without actually conducting the forward synthesis is always going to be a step removed from the truth of if a molecule can be made by a given retrosynthesis or not.<br/>We will present a few different approaches we have taken to make comparisons between different CASP tools. We have considered benchmarking tools under similar conditions, with quantitative metrics and descriptive statistics to best make comparisons. We have also made qualitative comparisons of retrosynthesis routes with input from expert synthetic chemists. Overall, individual assessments are the best measure-of-fit for a CASP tool and a user, but hopefully by determining some consistent metrics, we can help people to make informed decisions about which tools to use, as well as provide metrics for measuring improvements made within the field.

Evaluating, benchmarking, and comparing computer-aided synthesis planning software tools

ACS Spring 2022

The official archive and repository for all ACS Meetings & Expositions abstracts and select posters or presentations.