Conversion of CCDC CIF files to SciData JSON-LD files for data integration

Scientific data has been predominantly communicated through the publication of articles in scientific journals. Over the years these have been increasingly accessed via online databases. This gave rise to a massive accumulation of diverse scientific datasets from countless scientists under various scientific disciplines. Consequently, resulting in a wide array of organizational styles for data, even within the same focused research area. To manage, store, and share large files of assorted data, the need for a database that can establish relationships among data points while handling the magnitude of the data itself became paramount.

This research describes the way that SciData, a scientific data model, uses JavaScript Object Notation for Linked Data (JSON-LD) with the provided ontological framework for data organization. The data model paired with an ontological framework serves as a hybrid relational-graph database.

Educational crystallographic data files provided in Crystallographic Information Framework (CIF) files were provided by the Cambridge Crystallographic Data Centre (CCDC). This document format provides the association of a relationship among a set of values (tuples) to keywords (a data dictionary), in a semantic representation that is not ready for AI and Machine Learning. Conversion to JSON-LD makes the data AI ready via ingestion into a graph database.

A history of the iterative code development, optimization of code, and the final output of the data will be presented. Future development will focus on optimization for graph databases and semantic searching.