Data scientists, like software engineers, use search engines, code repositories, tutorials, and question and answer sites for finding code snippets. The objective of this study is to understand what information can be extracted from data science experiment repositories for quicker availability of relevant information when data scientists search for information. In this paper, we investigated a set of notebooks to identify recurring data science techniques for efficient information retrieval and easy adaptation from online solutions to support their search during experimentation. From the manual annotation of 57 natural language processing notebooks, a taxonomy on 106 data science techniques was developed, grouped by data science workflow stages. The preliminary evaluation shows that our constructed taxonomy is relevant to retrieve information that data scientists are searching for. Future work will continue to investigate the creation of a context aware code snippet engine designed for data scientists. 

Towards a taxonomy for annotation of data science experiment repositories

Welcome to the 21st IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM 2021)! 

We hope that you enjoy the conference and that it will inspire you in your future research.

Should you require a certificate of participation please contact Alexander Serebrenik, the general chair of SCAM, at a.serebrenik@tue.nl.

For questions related to the Morressier virtual platform please contact support@morressier.com.