The Cambridge Structural Database — TEI-tagged view

Colin R. Groom, Ian J. Bruno, Matthew P. Lightfoot, Suzanna C. Ward · 2016 · chemistry · doi:10.1107/s2052520616003954 · TEI xml · tagged edition · index · concepts · authors
Where this record's text came from. 2026-09-07 schema-0.3 pass: every quote verified as an exact substring of the staged text and anchored at merge time; concept terms verified present; every edge endpoint an existing node.
Other fields define these differently the same word, defined another way in another discipline
InChI 2 fields · Cambridge Structural Database 2 fields · CSD 2 fields
Tags on this paper every tag the record carries, whether or not the words occur in the text below — a number is how many papers in the corpus carry that tag; a tag with no number is unique to this paper
Cambridge Structural Database2 CSD5 small molecule crystal structures crystallography2 X-ray2 neutron diffraction analyses aggregation and standardization of structures access to individual entries automatic and manual curation used without further processing CSD Communications primary data resource CIF2 Crystallographic Information Framework CCDC number accession identifier CSD refcode CSD-Xpedite processing system CSD-Editor deCIFer probabilistic Bayesian approach reliability score improbable features Digital Object Identifier DOI DataCite International Chemical Identifier Standard InChI3 chemical substance studied CellCheckCSD automated tool CSD-System CSD community web services federation of databases specialist data centres specialist data repositories system-or-tool-building173
Method, as described: a staged database-production workflow in which diffraction results are captured in CIF, deposited, automatically checked, enriched and validated by software and expert staff, assigned persistent identifiers, and exposed through community services

Colour legend

Acta Crystallographica Section B Structural Science, Crystal Engineering and Materials<title level="j">A work — the journal / venue title.
2016<date>A year or date.

Colin R. GroompersName, Ian J. BrunopersName, Matthew P. LightfootpersName, Suzanna C. WardpersName (2016date). “The Cambridge Structural Database.” Acta Crystallographica Section B Structural Science, Crystal Engineering and Materialstitle·j.

Object of study — the Cambridge Structural Databaseterm, its information content, curation workflow, identifiers, and access services

Research question — how the Cambridge Structural Databaseterm is created, maintained, populated, identified, linked, and made available as a reusable structural-chemistry resource

Contribution — a definitive account of the CSDterm as a comprehensive repository and of the informatics, expert-curation, identifier, and access mechanisms that sustain it

Cross-disciplinary analysis
content-tags & facets: phenomenon · claim-type · cross-domain-transfer · mechanism · claim-relation · replication-status · limitation — grounded in the full text
phenomenonthe evolution of crystal-structure communication from literature abstraction to direct digital deposition, curation, linking, and access — “The increasing digital availability of the basic results of a diffraction experiment has also had an impact on the overall nature of the CSD. Originally, it might best have been described as a secondary resource of data abstracted from the literature. Today, new source data files are almost always deposited directly into the CSD making it much more of a primary data resource. Indeed, many structures are published only and directly through the CSD as CSD Communications (previously known as Private Communications). Whilst the CCDC has never charged for deposition, curation, archiving or for access to the primary source data files, early versions of the CCDC’s informatics systems did have technical barriers that hindered access to structures by researchers. Here, we describe developments that ensure entries from the CSD are made available to the widest possible community through CCDC services and third party resources.”
claim-typeexistence — “The CSD contains all published organic and metal–organic small-molecule crystal structures whose structures have been determined using the technique of crystallography. In addition, it acts as a publication vehicle for structure determinations with no accompanying manuscript.”
method — evidence — “An illustration of the flow from sample to CSD entry. Solid lines are the elements of the process involving the CCDC. A crystalline sample ( a ) is put on a diffractometer ( b ) for an experiment that ultimately results in the experimentally determined coordinates of atoms in the crystal that are captured in a CIF file ( c ). The CIF file is deposited with the CCDC ( d ) and may be associated with a scientific article ( e ). On publication of the article, the structure is validated and enriched by CCDC editorial software and staff ( f ) to create an entry in the CSD ( g ). Structures are publicly accessible through CSD community services such as Access Structures ( h ). Standard identifiers such as DOIs and InChIs facilitate links between articles, structures and other resources. Software developed by the CCDC ( i ) enables the knowledge embedded in the CSD to be applied to a range of scientific problems including aiding in the determination and refinement of future structures through free resources such as CellCheckCSD ( j ). The flow of data is largely represented using lamotrigine (Sridhar & Ravikumar, 2009 ▸ ), the 500 000th entry in the CSD, CSD refcode: EFEMUX01; CCDC number: CCDC 749719; DOI: 10.5517/cct54hl. Image ( a ) is a crystal of p -aminobenzoic acid (Sullivan & Davey, 2015 ▸ ) courtesy of Rachel Sullivan and Roger Davey, University of Manchester, CSD refcode: AMBNAC10; CCDC 983122; DOI: 10.5517/cc1200mq. Image ( b ) courtesy of Andrew Bond, University of Cambridge.”
object of study — evidence — “This paper is the definitive article describing the creation, maintenance, information content and availability of the Cambridge Structural Database (CSD), the world’s repository of small molecule crystal structures.”
research question — evidence — “This paper is the definitive article describing the creation, maintenance, information content and availability of the Cambridge Structural Database (CSD), the world’s repository of small molecule crystal structures.”
contribution — evidence — “This paper is the definitive article describing the creation, maintenance, information content and availability of the Cambridge Structural Database (CSD), the world’s repository of small molecule crystal structures.”
mechanismaggregation and standardization Aggregating and standardizing structures facilitates access to individual entries and supports reuse without repeated user-side correction. — “The database provides value in two distinct ways. The first simply relates to the aggregation and standardization of structures, which facilitates access to individual entries. This brings value both to the data generators and consumers. A single archive of all structures allows crystallographers to avoid the inadvertent redetermination of structures and provides a mechanism by which they can archive the output of their work at a specialist data centre for their own future use. Of course, it also allows for the easy sharing of their work, massively increasing their sphere of influence. Such sharing has always been the norm for the crystallographic community and the use of this worldwide, standard, specialist discipline repository allows individuals to demonstrate their adherence to the new data sharing mandates of various funding bodies. As all entries are subject to both automatic and manual curation, they can usually be used without further processing. Indeed, one might argue that the financial cost of maintaining such a resource, although significant, is recovered many times over by removing the need for repeated correction by users.”mechanisms[0]
mechanismCIF syntax checking The deposition workflow checks CIF syntax automatically and requires depositors to fix problems before proceeding. — “During deposition, the CIF syntax is automatically checked based on the checks in enCIFer (Allen et al. , 2004 ▸ ) and the depositor is required to fix any issues before continuing with the deposition process. Depositors are strongly encouraged to deposit structure factors in line with the IUCr’s publication standards for crystal structures (IUCr, 2011 ▸ ). The embedding of reflection data into the CIF by structure refinement programs such as SHELXL (Sheldrick, 2015 ▸ ) greatly simplifies this process. As a consequence the amount of reflection data stored at CCDC has increased significantly and the majority of new depositions now include this data.”mechanisms[1]
mechanismdepositor metadata review Metadata extracted from CIF files and a JSmol representation are presented to depositors so they can check and enrich the shared record. — “During deposition, key metadata are extracted from the CIF and presented to the researcher giving them the opportunity to check and further enhance the data that is shared through the CSD. To aid with this process a three-dimensional representation of the structure is displayed to the depositor using JSmol (Hanson et al. , 2013 ▸ ).”mechanisms[2]
mechanismpersistent accession assignment Each deposited dataset receives a persistent CCDC number, while indexed structures receive CSD refcodes and related determinations may share a refcode family. — “On deposition, each dataset is assigned an accession identifier referred to as a ‘CCDC number’ in the format CCDC 1234567 (older entries have six digits). This uniquely identifies the data associated with a particular structure determination and persists for the lifetime of the dataset. CCDC numbers are communicated to the depositor once it has been confirmed that the dataset is not a duplicate submission, usually within seconds of deposition. Should the data require further investigation, CCDC Deposition Coordinators address any problems, minimizing the delay in providing the CCDC number to the depositor. A reference code (known as a CSD refcode and in the format ABCDEF) is assigned to structures as they are indexed into the CSD itself. Where possible, determinations of the same substance are assigned into a CSD refcode family (with a CSD refcode format of ABCDEF01). These codes are used as a common way of referring to structures extracted from the CSD-System.”mechanisms[3]
mechanismunified automated processing CSD-Xpedite combines unified storage, automation, modular extensibility, and an extensible master format to scale entry processing. — “CSD-Xpedite is built around Microsoft Dynamics CRM (Microsoft, 2016 a ▸ ) and SharePoint (Microsoft, 2016 b ▸ ). The system is designed to store all data in one unified system and many automated processes have been incorporated to reduce the number of manual interventions required to process entries through the CSD creation process. This scalable system allows for the ever increasing rate of deposition, and its modular nature allows extra functionality to be incorporated with minimal disruption. One important aspect of CSD-Xpedite is the use of a new extensible file format for internal storage of CSD entries. This format allows a fuller representation of the underlying data than in previous formats (BCCAB, ASER). Databases for use in the CSD-System are created from this master database.”mechanisms[4]
mechanismBayesian chemical assignment DeCIFer uses prior CSD entries and a probabilistic Bayesian approach to resolve disorder, detect bonds, assign bond types and charges, and infer missing hydrogen atoms. — “DeCIFer uses the information already in the CSD to interpret a new structure and add a chemical representation to the atomic coordinates in the CIF. The stages involved in the automatic creation of the CSD entry include resolution of any disorder in the structure, detection of bonds and determination of bond types and charges. Assignment of bond types, charges and the inference of missing H atoms uses a probabilistic Bayesian approach which allows all the existing entries in the CSD to be used to help assign the chemistry to new entries.”mechanisms[5]
mechanismimprobable-feature validation DeCIFer checks improbable chemical features and derives a reliability score that directs possible errors to expert review. — “The final step of the deCIFer process is to validate the assignment by looking at improbable features such as unprecedented bond types, unlikely oxidation states, unlikely metal–metal bonds and other empirical indicators such as non-planar double bonds. From this analysis a reliability score is calculated which highlights any possible errors in the representation of the structure which can then be reviewed and corrected during CSD entry validation. It is important to note that this reliability score simply gives a confidence value to the automatic treatment of the entry; this is not necessarily related to the ‘quality’ of the structure.”mechanisms[6]
mechanismidentifier-mediated interoperability DOIs, DataCite metadata, and InChIs link datasets to articles and third-party resources while making structures discoverable and independently citable. — “Discoverability of data by chemists and biologists is enabled by establishing links to datasets from services such as ChemSpider (Pence & Williams, 2010 ▸ ) and PubChem (Bolton et al. , 2008 ▸ ). The overlap between these resources and the CSD is identified by taking advantage of the International Chemical Identifier Standard (InChI) which provides a unique and canonical representation of the chemical substance studied (Heller et al. , 2015 ▸ ). Links from ChemSpider and PubChem have been established for over 52 000 compounds that could be reliably identified using InChIs as being in common between these resources and the CSD. InChIs have also been used to identify correspondences between CSD entries and ligands bound to macromolecules in structures archived in the Protein Data Bank (PDB; Berman et al. , 2000 ▸ ). This lookup is enabled by a CCDC web service that identifies the best-representative CSD entry for a given molecule and provides access to its coordinates. This is a particularly useful resource for structural biologists refining or investigating the structure of a protein ligand. Representative structures are freely available for approximately 1500 PDB ligands in a Chemical Component Model file provided by the PDB (wwPDB, 2015 ▸ ).”mechanisms[7]
supersedes the previous published description of the CSD by Allen (2002) — “This ‘new knowledge’ relates primarily to the geometry of molecules and the interactions they make. A knowledge of these factors underpins huge areas of both fundamental and applied science (Wong et al. , 2010 ▸ ). They form the basis of our understanding of the energetics of molecular conformation – from bond lengths and angles, through to torsional preferences. They also teach us about the fundamentals of molecular recognition, be it small molecules interacting with small molecules in a lattice or with a protein. Such appreciation is vital for materials sciences and pharmaceutical research and development. Perhaps the most quantitative assessment that can be made of the value of the resource is that the previous published description of the CSD (Allen, 2002 ▸ ), which this article supersedes, has received over 10 000 citations.”claim_relations[0]
builds-on the CIF standard for interchange of crystallographic data — “Just as the database and software evolved so too did the way in which crystal structure data is communicated. In the early days atomic coordinates were published as tables in scientific articles. These were transcribed manually for inclusion in data files and databases. In the 1990s, a file format called CIF (now the Crystallographic Information Framework) was proposed as a standard for the interchange of crystallographic data. CIF is now ubiquitously used to capture the results of a diffraction experiment and enables streamlined publication of results alongside journal articles and in data repositories (Hall et al. , 1991 ▸ ; Brown & McMahon, 2002 ▸ ; Bernstein et al. , 2016 ▸ ; Hall & McMahon, 2016 ▸ ).”claim_relations[1]
qualifies the expectation that scientific data be freely, openly and instantly available — “There is now an expectation in science that data is freely, openly and instantly available. With modern informatics techniques this is trivial to achieve, but is often done at the expense of data quality, completeness, integrity and with questionable sustainability. History has shown us that all these are best achieved by specialist data centres, maintaining specialist data repositories. The challenge of finding a universally agreed ‘perfect’ funding model, if one exists, remains (Bruno & Groom, 2014 ▸ ).”claim_relations[2]
senseCambridge Structural Database as the world repository for small-molecule crystal structures — “This paper is the definitive article describing the creation, maintenance, information content and availability of the Cambridge Structural Database (CSD), the world’s repository of small molecule crystal structures.”senses[0]
senseCSD coverage as published organic and metal–organic structures determined by crystallography — “The CSD contains all published organic and metal–organic small-molecule crystal structures whose structures have been determined using the technique of crystallography. In addition, it acts as a publication vehicle for structure determinations with no accompanying manuscript.”senses[1]
sensedatabase value as both entry-level access and collective knowledge extraction — “The database provides value in two distinct ways. The first simply relates to the aggregation and standardization of structures, which facilitates access to individual entries. This brings value both to the data generators and consumers. A single archive of all structures allows crystallographers to avoid the inadvertent redetermination of structures and provides a mechanism by which they can archive the output of their work at a specialist data centre for their own future use. Of course, it also allows for the easy sharing of their work, massively increasing their sphere of influence. Such sharing has always been the norm for the crystallographic community and the use of this worldwide, standard, specialist discipline repository allows individuals to demonstrate their adherence to the new data sharing mandates of various funding bodies. As all entries are subject to both automatic and manual curation, they can usually be used without further processing. Indeed, one might argue that the financial cost of maintaining such a resource, although significant, is recovered many times over by removing the need for repeated correction by users.”senses[2]
senseCSD as a primary data resource rather than only a secondary literature abstraction — “The increasing digital availability of the basic results of a diffraction experiment has also had an impact on the overall nature of the CSD. Originally, it might best have been described as a secondary resource of data abstracted from the literature. Today, new source data files are almost always deposited directly into the CSD making it much more of a primary data resource. Indeed, many structures are published only and directly through the CSD as CSD Communications (previously known as Private Communications). Whilst the CCDC has never charged for deposition, curation, archiving or for access to the primary source data files, early versions of the CCDC’s informatics systems did have technical barriers that hindered access to structures by researchers. Here, we describe developments that ensure entries from the CSD are made available to the widest possible community through CCDC services and third party resources.”senses[3]
senseCIF as both an interchange standard and a capture format for diffraction results — “Just as the database and software evolved so too did the way in which crystal structure data is communicated. In the early days atomic coordinates were published as tables in scientific articles. These were transcribed manually for inclusion in data files and databases. In the 1990s, a file format called CIF (now the Crystallographic Information Framework) was proposed as a standard for the interchange of crystallographic data. CIF is now ubiquitously used to capture the results of a diffraction experiment and enables streamlined publication of results alongside journal articles and in data repositories (Hall et al. , 1991 ▸ ; Brown & McMahon, 2002 ▸ ; Bernstein et al. , 2016 ▸ ; Hall & McMahon, 2016 ▸ ).”senses[4]
senseCCDC number as a lifetime-persistent deposited-dataset identifier — “On deposition, each dataset is assigned an accession identifier referred to as a ‘CCDC number’ in the format CCDC 1234567 (older entries have six digits). This uniquely identifies the data associated with a particular structure determination and persists for the lifetime of the dataset. CCDC numbers are communicated to the depositor once it has been confirmed that the dataset is not a duplicate submission, usually within seconds of deposition. Should the data require further investigation, CCDC Deposition Coordinators address any problems, minimizing the delay in providing the CCDC number to the depositor. A reference code (known as a CSD refcode and in the format ABCDEF) is assigned to structures as they are indexed into the CSD itself. Where possible, determinations of the same substance are assigned into a CSD refcode family (with a CSD refcode format of ABCDEF01). These codes are used as a common way of referring to structures extracted from the CSD-System.”senses[5]
sensereliability score as confidence in automatic treatment rather than structural quality — “The final step of the deCIFer process is to validate the assignment by looking at improbable features such as unprecedented bond types, unlikely oxidation states, unlikely metal–metal bonds and other empirical indicators such as non-planar double bonds. From this analysis a reliability score is calculated which highlights any possible errors in the representation of the structure which can then be reviewed and corrected during CSD entry validation. It is important to note that this reliability score simply gives a confidence value to the automatic treatment of the entry; this is not necessarily related to the ‘quality’ of the structure.”senses[6]
senseDOI as a dataset-linking, formal-citation, and contributor-credit mechanism — “At the point a structure can be made public a Digital Object Identifier (DOI) is associated with the deposited dataset (580 000 are currently available). This DOI allows a third party to link to a structure summary page without needing to know details of linking services provided by the CCDC. In addition the DOI provides the basis for a more formal citation of the deposited dataset in line with the spirit of the Joint Declaration of Data Citation Principles (Data Citation Synthesis Group, 2014 ▸ ). The metadata provided to the DOI registration agency DataCite (DataCite, 2016 ▸ ) indicates key elements of a dataset citation including contributor, a title and publication year. This enables citation of the dataset independently of the associated article and thus helps ensure recognition of the specific contribution made by the crystallographer, who may not always be included in the article’s author list.”senses[7]
senseInChI as the canonical substance representation used for cross-resource matching — “Discoverability of data by chemists and biologists is enabled by establishing links to datasets from services such as ChemSpider (Pence & Williams, 2010 ▸ ) and PubChem (Bolton et al. , 2008 ▸ ). The overlap between these resources and the CSD is identified by taking advantage of the International Chemical Identifier Standard (InChI) which provides a unique and canonical representation of the chemical substance studied (Heller et al. , 2015 ▸ ). Links from ChemSpider and PubChem have been established for over 52 000 compounds that could be reliably identified using InChIs as being in common between these resources and the CSD. InChIs have also been used to identify correspondences between CSD entries and ligands bound to macromolecules in structures archived in the Protein Data Bank (PDB; Berman et al. , 2000 ▸ ). This lookup is enabled by a CCDC web service that identifies the best-representative CSD entry for a given molecule and provides access to its coordinates. This is a particularly useful resource for structural biologists refining or investigating the structure of a protein ligand. Representative structures are freely available for approximately 1500 PDB ligands in a Chemical Component Model file provided by the PDB (wwPDB, 2015 ▸ ).”senses[8]
senseopen scientific data as a balance among availability, quality, completeness, integrity, and sustainability — “There is now an expectation in science that data is freely, openly and instantly available. With modern informatics techniques this is trivial to achieve, but is often done at the expense of data quality, completeness, integrity and with questionable sustainability. History has shown us that all these are best achieved by specialist data centres, maintaining specialist data repositories. The challenge of finding a universally agreed ‘perfect’ funding model, if one exists, remains (Bruno & Groom, 2014 ▸ ).”senses[9]
notethe previous published description of the CSD by Allen (2002) — The article states the supersession categorically. — “This ‘new knowledge’ relates primarily to the geometry of molecules and the interactions they make. A knowledge of these factors underpins huge areas of both fundamental and applied science (Wong et al. , 2010 ▸ ). They form the basis of our understanding of the energetics of molecular conformation – from bond lengths and angles, through to torsional preferences. They also teach us about the fundamentals of molecular recognition, be it small molecules interacting with small molecules in a lattice or with a protein. Such appreciation is vital for materials sciences and pharmaceutical research and development. Perhaps the most quantitative assessment that can be made of the value of the resource is that the previous published description of the CSD (Allen, 2002 ▸ ), which this article supersedes, has received over 10 000 citations.”claim_relations[0]
notethe CIF standard for interchange of crystallographic data — The modern communication and deposition workflow is presented as enabled by the established CIF standard. — “Just as the database and software evolved so too did the way in which crystal structure data is communicated. In the early days atomic coordinates were published as tables in scientific articles. These were transcribed manually for inclusion in data files and databases. In the 1990s, a file format called CIF (now the Crystallographic Information Framework) was proposed as a standard for the interchange of crystallographic data. CIF is now ubiquitously used to capture the results of a diffraction experiment and enables streamlined publication of results alongside journal articles and in data repositories (Hall et al. , 1991 ▸ ; Brown & McMahon, 2002 ▸ ; Bernstein et al. , 2016 ▸ ; Hall & McMahon, 2016 ▸ ).”claim_relations[1]
weightthe CIF standard for interchange of crystallographic data — 0.95 — “Just as the database and software evolved so too did the way in which crystal structure data is communicated. In the early days atomic coordinates were published as tables in scientific articles. These were transcribed manually for inclusion in data files and databases. In the 1990s, a file format called CIF (now the Crystallographic Information Framework) was proposed as a standard for the interchange of crystallographic data. CIF is now ubiquitously used to capture the results of a diffraction experiment and enables streamlined publication of results alongside journal articles and in data repositories (Hall et al. , 1991 ▸ ; Brown & McMahon, 2002 ▸ ; Bernstein et al. , 2016 ▸ ; Hall & McMahon, 2016 ▸ ).”claim_relations[1]
notethe expectation that scientific data be freely, openly and instantly available — The authors accept the availability expectation while explicitly qualifying it with quality, integrity, and sustainability costs. — “There is now an expectation in science that data is freely, openly and instantly available. With modern informatics techniques this is trivial to achieve, but is often done at the expense of data quality, completeness, integrity and with questionable sustainability. History has shown us that all these are best achieved by specialist data centres, maintaining specialist data repositories. The challenge of finding a universally agreed ‘perfect’ funding model, if one exists, remains (Bruno & Groom, 2014 ▸ ).”claim_relations[2]
weightthe expectation that scientific data be freely, openly and instantly available — 0.95 — “There is now an expectation in science that data is freely, openly and instantly available. With modern informatics techniques this is trivial to achieve, but is often done at the expense of data quality, completeness, integrity and with questionable sustainability. History has shown us that all these are best achieved by specialist data centres, maintaining specialist data repositories. The challenge of finding a universally agreed ‘perfect’ funding model, if one exists, remains (Bruno & Groom, 2014 ▸ ).”claim_relations[2]
noteCambridge Structural Database —same-as→ CSD — The parenthetical naming equivalence is categorical. — “This paper is the definitive article describing the creation, maintenance, information content and availability of the Cambridge Structural Database (CSD), the world’s repository of small molecule crystal structures.”concept_edges[0]
relationCambridge Structural Database —same-as→ CSD — CSD is the parenthetical short name of the Cambridge Structural Database. — “This paper is the definitive article describing the creation, maintenance, information content and availability of the Cambridge Structural Database (CSD), the world’s repository of small molecule crystal structures.”concept_edges[0]
noteX-ray —part-of→ CSD — The paper categorically states that the database contains this analysis class. — “Specifically, the CSD contains both X-ray and neutron diffraction analyses from a single-crystal study or a powder study where cell parameters, atomic coordinates and refinement are reported. To ensure comprehensive coverage of single-crystal data, cell parameters and all available data are included even if no coordinates are available. Powder structures without coordinates are available from the International Centre for Diffraction Data (ICDD; Kabekkodu et al. , 2002 ▸ ).”concept_edges[1]
relationX-ray —part-of→ CSD — X-ray diffraction analyses are contained in the CSD. — “Specifically, the CSD contains both X-ray and neutron diffraction analyses from a single-crystal study or a powder study where cell parameters, atomic coordinates and refinement are reported. To ensure comprehensive coverage of single-crystal data, cell parameters and all available data are included even if no coordinates are available. Powder structures without coordinates are available from the International Centre for Diffraction Data (ICDD; Kabekkodu et al. , 2002 ▸ ).”concept_edges[1]
noteneutron diffraction analyses —part-of→ CSD — The paper categorically states that the database contains this analysis class. — “Specifically, the CSD contains both X-ray and neutron diffraction analyses from a single-crystal study or a powder study where cell parameters, atomic coordinates and refinement are reported. To ensure comprehensive coverage of single-crystal data, cell parameters and all available data are included even if no coordinates are available. Powder structures without coordinates are available from the International Centre for Diffraction Data (ICDD; Kabekkodu et al. , 2002 ▸ ).”concept_edges[2]
relationneutron diffraction analyses —part-of→ CSD — Neutron diffraction analyses are contained in the CSD. — “Specifically, the CSD contains both X-ray and neutron diffraction analyses from a single-crystal study or a powder study where cell parameters, atomic coordinates and refinement are reported. To ensure comprehensive coverage of single-crystal data, cell parameters and all available data are included even if no coordinates are available. Powder structures without coordinates are available from the International Centre for Diffraction Data (ICDD; Kabekkodu et al. , 2002 ▸ ).”concept_edges[2]
noteCIF —same-as→ Crystallographic Information Framework — The naming equivalence is explicit and categorical. — “Just as the database and software evolved so too did the way in which crystal structure data is communicated. In the early days atomic coordinates were published as tables in scientific articles. These were transcribed manually for inclusion in data files and databases. In the 1990s, a file format called CIF (now the Crystallographic Information Framework) was proposed as a standard for the interchange of crystallographic data. CIF is now ubiquitously used to capture the results of a diffraction experiment and enables streamlined publication of results alongside journal articles and in data repositories (Hall et al. , 1991 ▸ ; Brown & McMahon, 2002 ▸ ; Bernstein et al. , 2016 ▸ ; Hall & McMahon, 2016 ▸ ).”concept_edges[3]
relationCIF —same-as→ Crystallographic Information Framework — CIF is the name used for the Crystallographic Information Framework. — “Just as the database and software evolved so too did the way in which crystal structure data is communicated. In the early days atomic coordinates were published as tables in scientific articles. These were transcribed manually for inclusion in data files and databases. In the 1990s, a file format called CIF (now the Crystallographic Information Framework) was proposed as a standard for the interchange of crystallographic data. CIF is now ubiquitously used to capture the results of a diffraction experiment and enables streamlined publication of results alongside journal articles and in data repositories (Hall et al. , 1991 ▸ ; Brown & McMahon, 2002 ▸ ; Bernstein et al. , 2016 ▸ ; Hall & McMahon, 2016 ▸ ).”concept_edges[3]
noteCCDC number —same-as→ accession identifier — The paper uses an explicit referred-to-as equivalence. — “On deposition, each dataset is assigned an accession identifier referred to as a ‘CCDC number’ in the format CCDC 1234567 (older entries have six digits). This uniquely identifies the data associated with a particular structure determination and persists for the lifetime of the dataset. CCDC numbers are communicated to the depositor once it has been confirmed that the dataset is not a duplicate submission, usually within seconds of deposition. Should the data require further investigation, CCDC Deposition Coordinators address any problems, minimizing the delay in providing the CCDC number to the depositor. A reference code (known as a CSD refcode and in the format ABCDEF) is assigned to structures as they are indexed into the CSD itself. Where possible, determinations of the same substance are assigned into a CSD refcode family (with a CSD refcode format of ABCDEF01). These codes are used as a common way of referring to structures extracted from the CSD-System.”concept_edges[4]
relationCCDC number —same-as→ accession identifier — The accession identifier is referred to as a CCDC number. — “On deposition, each dataset is assigned an accession identifier referred to as a ‘CCDC number’ in the format CCDC 1234567 (older entries have six digits). This uniquely identifies the data associated with a particular structure determination and persists for the lifetime of the dataset. CCDC numbers are communicated to the depositor once it has been confirmed that the dataset is not a duplicate submission, usually within seconds of deposition. Should the data require further investigation, CCDC Deposition Coordinators address any problems, minimizing the delay in providing the CCDC number to the depositor. A reference code (known as a CSD refcode and in the format ABCDEF) is assigned to structures as they are indexed into the CSD itself. Where possible, determinations of the same substance are assigned into a CSD refcode family (with a CSD refcode format of ABCDEF01). These codes are used as a common way of referring to structures extracted from the CSD-System.”concept_edges[4]
noteaggregation and standardization of structures —causes→ access to individual entries — Facilitates is a flat positive causal claim; weight reflects strong but not necessary causation. — “The database provides value in two distinct ways. The first simply relates to the aggregation and standardization of structures, which facilitates access to individual entries. This brings value both to the data generators and consumers. A single archive of all structures allows crystallographers to avoid the inadvertent redetermination of structures and provides a mechanism by which they can archive the output of their work at a specialist data centre for their own future use. Of course, it also allows for the easy sharing of their work, massively increasing their sphere of influence. Such sharing has always been the norm for the crystallographic community and the use of this worldwide, standard, specialist discipline repository allows individuals to demonstrate their adherence to the new data sharing mandates of various funding bodies. As all entries are subject to both automatic and manual curation, they can usually be used without further processing. Indeed, one might argue that the financial cost of maintaining such a resource, although significant, is recovered many times over by removing the need for repeated correction by users.”concept_edges[5]
relationaggregation and standardization of structures —causes→ access to individual entries — Aggregation and standardization of structures facilitates access to individual entries. — “The database provides value in two distinct ways. The first simply relates to the aggregation and standardization of structures, which facilitates access to individual entries. This brings value both to the data generators and consumers. A single archive of all structures allows crystallographers to avoid the inadvertent redetermination of structures and provides a mechanism by which they can archive the output of their work at a specialist data centre for their own future use. Of course, it also allows for the easy sharing of their work, massively increasing their sphere of influence. Such sharing has always been the norm for the crystallographic community and the use of this worldwide, standard, specialist discipline repository allows individuals to demonstrate their adherence to the new data sharing mandates of various funding bodies. As all entries are subject to both automatic and manual curation, they can usually be used without further processing. Indeed, one might argue that the financial cost of maintaining such a resource, although significant, is recovered many times over by removing the need for repeated correction by users.”concept_edges[5]
weightaggregation and standardization of structures —causes→ access to individual entries — 0.9 — “The database provides value in two distinct ways. The first simply relates to the aggregation and standardization of structures, which facilitates access to individual entries. This brings value both to the data generators and consumers. A single archive of all structures allows crystallographers to avoid the inadvertent redetermination of structures and provides a mechanism by which they can archive the output of their work at a specialist data centre for their own future use. Of course, it also allows for the easy sharing of their work, massively increasing their sphere of influence. Such sharing has always been the norm for the crystallographic community and the use of this worldwide, standard, specialist discipline repository allows individuals to demonstrate their adherence to the new data sharing mandates of various funding bodies. As all entries are subject to both automatic and manual curation, they can usually be used without further processing. Indeed, one might argue that the financial cost of maintaining such a resource, although significant, is recovered many times over by removing the need for repeated correction by users.”concept_edges[5]
noteautomatic and manual curation —causes→ used without further processing — The outcome is explicitly hedged by usually, so the gradable edge is below a flat-assertion weight. — “The database provides value in two distinct ways. The first simply relates to the aggregation and standardization of structures, which facilitates access to individual entries. This brings value both to the data generators and consumers. A single archive of all structures allows crystallographers to avoid the inadvertent redetermination of structures and provides a mechanism by which they can archive the output of their work at a specialist data centre for their own future use. Of course, it also allows for the easy sharing of their work, massively increasing their sphere of influence. Such sharing has always been the norm for the crystallographic community and the use of this worldwide, standard, specialist discipline repository allows individuals to demonstrate their adherence to the new data sharing mandates of various funding bodies. As all entries are subject to both automatic and manual curation, they can usually be used without further processing. Indeed, one might argue that the financial cost of maintaining such a resource, although significant, is recovered many times over by removing the need for repeated correction by users.”concept_edges[6]
relationautomatic and manual curation —causes→ used without further processing — Automatic and manual curation means entries can usually be used without further processing. — “The database provides value in two distinct ways. The first simply relates to the aggregation and standardization of structures, which facilitates access to individual entries. This brings value both to the data generators and consumers. A single archive of all structures allows crystallographers to avoid the inadvertent redetermination of structures and provides a mechanism by which they can archive the output of their work at a specialist data centre for their own future use. Of course, it also allows for the easy sharing of their work, massively increasing their sphere of influence. Such sharing has always been the norm for the crystallographic community and the use of this worldwide, standard, specialist discipline repository allows individuals to demonstrate their adherence to the new data sharing mandates of various funding bodies. As all entries are subject to both automatic and manual curation, they can usually be used without further processing. Indeed, one might argue that the financial cost of maintaining such a resource, although significant, is recovered many times over by removing the need for repeated correction by users.”concept_edges[6]
weightautomatic and manual curation —causes→ used without further processing — 0.7 — “The database provides value in two distinct ways. The first simply relates to the aggregation and standardization of structures, which facilitates access to individual entries. This brings value both to the data generators and consumers. A single archive of all structures allows crystallographers to avoid the inadvertent redetermination of structures and provides a mechanism by which they can archive the output of their work at a specialist data centre for their own future use. Of course, it also allows for the easy sharing of their work, massively increasing their sphere of influence. Such sharing has always been the norm for the crystallographic community and the use of this worldwide, standard, specialist discipline repository allows individuals to demonstrate their adherence to the new data sharing mandates of various funding bodies. As all entries are subject to both automatic and manual curation, they can usually be used without further processing. Indeed, one might argue that the financial cost of maintaining such a resource, although significant, is recovered many times over by removing the need for repeated correction by users.”concept_edges[6]
auditreliability score —operationalised-as→ improbable features — Audit 2026-09-28: DOUBTFUL - read against its span and kept, but the reliability score is CALCULATED from an analysis of improbable features; operationalised-as is loose. — “The final step of the deCIFer process is to validate the assignment by looking at improbable features such as unprecedented bond types, unlikely oxidation states, unlikely metal–metal bonds and other empirical indicators such as non-planar double bonds. From this analysis a reliability score is calculated which highlights any possible errors in the representation of the structure which can then be reviewed and corrected during CSD entry validation. It is important to note that this reliability score simply gives a confidence value to the automatic treatment of the entry; this is not necessarily related to the ‘quality’ of the structure.”concept_edges[7]
notereliability score —operationalised-as→ improbable features — Calculated from empirical indicators is a direct operationalization statement. — “The final step of the deCIFer process is to validate the assignment by looking at improbable features such as unprecedented bond types, unlikely oxidation states, unlikely metal–metal bonds and other empirical indicators such as non-planar double bonds. From this analysis a reliability score is calculated which highlights any possible errors in the representation of the structure which can then be reviewed and corrected during CSD entry validation. It is important to note that this reliability score simply gives a confidence value to the automatic treatment of the entry; this is not necessarily related to the ‘quality’ of the structure.”concept_edges[7]
relationreliability score —operationalised-as→ improbable features — The reliability score is calculated from analysis of improbable features. — “The final step of the deCIFer process is to validate the assignment by looking at improbable features such as unprecedented bond types, unlikely oxidation states, unlikely metal–metal bonds and other empirical indicators such as non-planar double bonds. From this analysis a reliability score is calculated which highlights any possible errors in the representation of the structure which can then be reviewed and corrected during CSD entry validation. It is important to note that this reliability score simply gives a confidence value to the automatic treatment of the entry; this is not necessarily related to the ‘quality’ of the structure.”concept_edges[7]
weightreliability score —operationalised-as→ improbable features — 0.95 — “The final step of the deCIFer process is to validate the assignment by looking at improbable features such as unprecedented bond types, unlikely oxidation states, unlikely metal–metal bonds and other empirical indicators such as non-planar double bonds. From this analysis a reliability score is calculated which highlights any possible errors in the representation of the structure which can then be reviewed and corrected during CSD entry validation. It is important to note that this reliability score simply gives a confidence value to the automatic treatment of the entry; this is not necessarily related to the ‘quality’ of the structure.”concept_edges[7]
noteCellCheckCSD —is-a→ automated tool — The is-a statement is explicit and categorical. — “CellCheckCSD is an automated tool which uses the data from the CSD to check unit cells during data collection and can be used to check that the structure is novel rather than the starting material, a by-product or another previously determined structure.”concept_edges[8]
relationCellCheckCSD —is-a→ automated tool — CellCheckCSD is an automated tool. — “CellCheckCSD is an automated tool which uses the data from the CSD to check unit cells during data collection and can be used to check that the structure is novel rather than the starting material, a by-product or another previously determined structure.”concept_edges[8]
relationCSD-Xpedite → processing system — CSD-Xpedite is introduced as a new processing system. — “Due to the rising number of structures, depositions and transactions a new processing system, CSD-Xpedite , has been developed, automating most informatics processes required to manage depositions and process crystal structures into entries in the CSD. Fig. 6 ▸ shows how data flows through the system and where various users interact with the system.”unhoused_relations[0]
relationCSD-Editor → CSD-Xpedite — CSD-Editor is a key component in CSD-Xpedite. — “A key component in CSD-Xpedite is CSD-Editor , an interactive tool for processing structures such that they can enter into the CSD. This uses many of the visualization and menu options available in the CSD-System program Mercury (Macrae et al. , 2006 ▸ ). Deposited CIFs can be easily viewed alongside any associated article. Errors and warnings are displayed for each structure to allow expert structural chemistry editors to concentrate their efforts on the challenging scientific parts of the process.” — “A key component in CSD-Xpedite is CSD-Editor , an interactive tool for processing structures such that they can enter into the CSD.”unhoused_relations[1]
relationautomated processes → manual interventions — Automated processes reduce the number of manual interventions required to process entries. — “CSD-Xpedite is built around Microsoft Dynamics CRM (Microsoft, 2016 a ▸ ) and SharePoint (Microsoft, 2016 b ▸ ). The system is designed to store all data in one unified system and many automated processes have been incorporated to reduce the number of manual interventions required to process entries through the CSD creation process. This scalable system allows for the ever increasing rate of deposition, and its modular nature allows extra functionality to be incorporated with minimal disruption. One important aspect of CSD-Xpedite is the use of a new extensible file format for internal storage of CSD entries. This format allows a fuller representation of the underlying data than in previous formats (BCCAB, ASER). Databases for use in the CSD-System are created from this master database.”unhoused_relations[2]
weightautomated processes → manual interventions — 0.95 — “CSD-Xpedite is built around Microsoft Dynamics CRM (Microsoft, 2016 a ▸ ) and SharePoint (Microsoft, 2016 b ▸ ). The system is designed to store all data in one unified system and many automated processes have been incorporated to reduce the number of manual interventions required to process entries through the CSD creation process. This scalable system allows for the ever increasing rate of deposition, and its modular nature allows extra functionality to be incorporated with minimal disruption. One important aspect of CSD-Xpedite is the use of a new extensible file format for internal storage of CSD entries. This format allows a fuller representation of the underlying data than in previous formats (BCCAB, ASER). Databases for use in the CSD-System are created from this master database.”unhoused_relations[2]
relationDOI → citation of the deposited dataset — A DOI provides the basis for formal citation of the deposited dataset. — “At the point a structure can be made public a Digital Object Identifier (DOI) is associated with the deposited dataset (580 000 are currently available). This DOI allows a third party to link to a structure summary page without needing to know details of linking services provided by the CCDC. In addition the DOI provides the basis for a more formal citation of the deposited dataset in line with the spirit of the Joint Declaration of Data Citation Principles (Data Citation Synthesis Group, 2014 ▸ ). The metadata provided to the DOI registration agency DataCite (DataCite, 2016 ▸ ) indicates key elements of a dataset citation including contributor, a title and publication year. This enables citation of the dataset independently of the associated article and thus helps ensure recognition of the specific contribution made by the crystallographer, who may not always be included in the article’s author list.”unhoused_relations[3]
weightDOI → citation of the deposited dataset — 0.95 — “At the point a structure can be made public a Digital Object Identifier (DOI) is associated with the deposited dataset (580 000 are currently available). This DOI allows a third party to link to a structure summary page without needing to know details of linking services provided by the CCDC. In addition the DOI provides the basis for a more formal citation of the deposited dataset in line with the spirit of the Joint Declaration of Data Citation Principles (Data Citation Synthesis Group, 2014 ▸ ). The metadata provided to the DOI registration agency DataCite (DataCite, 2016 ▸ ) indicates key elements of a dataset citation including contributor, a title and publication year. This enables citation of the dataset independently of the associated article and thus helps ensure recognition of the specific contribution made by the crystallographer, who may not always be included in the article’s author list.”unhoused_relations[3]
relationInChI → correspondences between resources — InChIs identify overlap and correspondence between CSD entries and records in external resources. — “Discoverability of data by chemists and biologists is enabled by establishing links to datasets from services such as ChemSpider (Pence & Williams, 2010 ▸ ) and PubChem (Bolton et al. , 2008 ▸ ). The overlap between these resources and the CSD is identified by taking advantage of the International Chemical Identifier Standard (InChI) which provides a unique and canonical representation of the chemical substance studied (Heller et al. , 2015 ▸ ). Links from ChemSpider and PubChem have been established for over 52 000 compounds that could be reliably identified using InChIs as being in common between these resources and the CSD. InChIs have also been used to identify correspondences between CSD entries and ligands bound to macromolecules in structures archived in the Protein Data Bank (PDB; Berman et al. , 2000 ▸ ). This lookup is enabled by a CCDC web service that identifies the best-representative CSD entry for a given molecule and provides access to its coordinates. This is a particularly useful resource for structural biologists refining or investigating the structure of a protein ligand. Representative structures are freely available for approximately 1500 PDB ligands in a Chemical Component Model file provided by the PDB (wwPDB, 2015 ▸ ).”unhoused_relations[4]
weightInChI → correspondences between resources — 0.95 — “Discoverability of data by chemists and biologists is enabled by establishing links to datasets from services such as ChemSpider (Pence & Williams, 2010 ▸ ) and PubChem (Bolton et al. , 2008 ▸ ). The overlap between these resources and the CSD is identified by taking advantage of the International Chemical Identifier Standard (InChI) which provides a unique and canonical representation of the chemical substance studied (Heller et al. , 2015 ▸ ). Links from ChemSpider and PubChem have been established for over 52 000 compounds that could be reliably identified using InChIs as being in common between these resources and the CSD. InChIs have also been used to identify correspondences between CSD entries and ligands bound to macromolecules in structures archived in the Protein Data Bank (PDB; Berman et al. , 2000 ▸ ). This lookup is enabled by a CCDC web service that identifies the best-representative CSD entry for a given molecule and provides access to its coordinates. This is a particularly useful resource for structural biologists refining or investigating the structure of a protein ligand. Representative structures are freely available for approximately 1500 PDB ligands in a Chemical Component Model file provided by the PDB (wwPDB, 2015 ▸ ).”unhoused_relations[4]
relationfederation of databases → methods for searching them — Growth of federated repositories will require improved search methods. — “We expect to see a continued push towards the federation of databases containing molecular structures (Sali et al. , 2015 ▸ ) and the development of links between repositories. As these individual repositories grow, methods for searching them will also need to be improved.”unhoused_relations[5]
weightfederation of databases → methods for searching them — 0.7 — “We expect to see a continued push towards the federation of databases containing molecular structures (Sali et al. , 2015 ▸ ) and the development of links between repositories. As these individual repositories grow, methods for searching them will also need to be improved.”unhoused_relations[5]
no type for thisCSD-Xpedite → processing system — The paper introduces CSD-Xpedite appositionally as a new processing system. — “Due to the rising number of structures, depositions and transactions a new processing system, CSD-Xpedite , has been developed, automating most informatics processes required to manage depositions and process crystal structures into entries in the CSD. Fig. 6 ▸ shows how data flows through the system and where various users interact with the system.”unhoused_relations[0]
no type for thisCSD-Editor → CSD-Xpedite — CSD-Editor is a key component in CSD-Xpedite. — “A key component in CSD-Xpedite is CSD-Editor , an interactive tool for processing structures such that they can enter into the CSD. This uses many of the visualization and menu options available in the CSD-System program Mercury (Macrae et al. , 2006 ▸ ). Deposited CIFs can be easily viewed alongside any associated article. Errors and warnings are displayed for each structure to allow expert structural chemistry editors to concentrate their efforts on the challenging scientific parts of the process.”unhoused_relations[1]
no type for thisautomated processes → manual interventions — Automated processes reduce the number of manual interventions required for CSD entry processing. — “CSD-Xpedite is built around Microsoft Dynamics CRM (Microsoft, 2016 a ▸ ) and SharePoint (Microsoft, 2016 b ▸ ). The system is designed to store all data in one unified system and many automated processes have been incorporated to reduce the number of manual interventions required to process entries through the CSD creation process. This scalable system allows for the ever increasing rate of deposition, and its modular nature allows extra functionality to be incorporated with minimal disruption. One important aspect of CSD-Xpedite is the use of a new extensible file format for internal storage of CSD entries. This format allows a fuller representation of the underlying data than in previous formats (BCCAB, ASER). Databases for use in the CSD-System are created from this master database.”unhoused_relations[2]
no type for thisDOI → citation of the deposited dataset — A DOI provides the basis for formal citation of a deposited dataset independently of its associated article. — “At the point a structure can be made public a Digital Object Identifier (DOI) is associated with the deposited dataset (580 000 are currently available). This DOI allows a third party to link to a structure summary page without needing to know details of linking services provided by the CCDC. In addition the DOI provides the basis for a more formal citation of the deposited dataset in line with the spirit of the Joint Declaration of Data Citation Principles (Data Citation Synthesis Group, 2014 ▸ ). The metadata provided to the DOI registration agency DataCite (DataCite, 2016 ▸ ) indicates key elements of a dataset citation including contributor, a title and publication year. This enables citation of the dataset independently of the associated article and thus helps ensure recognition of the specific contribution made by the crystallographer, who may not always be included in the article’s author list.”unhoused_relations[3]
no type for thisInChI → correspondences between resources — InChI matching is used to identify overlaps and correspondences between the CSD and external chemistry and structural-biology resources. — “Discoverability of data by chemists and biologists is enabled by establishing links to datasets from services such as ChemSpider (Pence & Williams, 2010 ▸ ) and PubChem (Bolton et al. , 2008 ▸ ). The overlap between these resources and the CSD is identified by taking advantage of the International Chemical Identifier Standard (InChI) which provides a unique and canonical representation of the chemical substance studied (Heller et al. , 2015 ▸ ). Links from ChemSpider and PubChem have been established for over 52 000 compounds that could be reliably identified using InChIs as being in common between these resources and the CSD. InChIs have also been used to identify correspondences between CSD entries and ligands bound to macromolecules in structures archived in the Protein Data Bank (PDB; Berman et al. , 2000 ▸ ). This lookup is enabled by a CCDC web service that identifies the best-representative CSD entry for a given molecule and provides access to its coordinates. This is a particularly useful resource for structural biologists refining or investigating the structure of a protein ligand. Representative structures are freely available for approximately 1500 PDB ligands in a Chemical Component Model file provided by the PDB (wwPDB, 2015 ▸ ).”unhoused_relations[4]
no type for thisfederation of databases → methods for searching them — As federated molecular-structure repositories grow, their search methods will need improvement. — “We expect to see a continued push towards the federation of databases containing molecular structures (Sali et al. , 2015 ▸ ) and the development of links between repositories. As these individual repositories grow, methods for searching them will also need to be improved.”unhoused_relations[5]
limitationThe repository covers published structures, but the paper reports an estimate that only 15% of determined structures are published. — “Although the CSD contains all published crystal structures, it has been estimated that only 15% of structures determined are published (UK National Crystallography Service, 2014 ▸ ). Automatic links in software used during structure determination (Dolomanov et al. , 2009 ▸ ), the ease with which structures can be deposited, attribution of credit in the form of a DOI and continued demonstration of the value to science of depositing crystal structures (Berman et al. , 2015 ▸ ) may help close this gap.”limitations[0]
limitationEarlier CCDC informatics systems created technical barriers that hindered researcher access to structures. — “The increasing digital availability of the basic results of a diffraction experiment has also had an impact on the overall nature of the CSD. Originally, it might best have been described as a secondary resource of data abstracted from the literature. Today, new source data files are almost always deposited directly into the CSD making it much more of a primary data resource. Indeed, many structures are published only and directly through the CSD as CSD Communications (previously known as Private Communications). Whilst the CCDC has never charged for deposition, curation, archiving or for access to the primary source data files, early versions of the CCDC’s informatics systems did have technical barriers that hindered access to structures by researchers. Here, we describe developments that ensure entries from the CSD are made available to the widest possible community through CCDC services and third party resources.”limitations[1]
limitationAs federated molecular-structure repositories grow, their search methods will need improvement. — “We expect to see a continued push towards the federation of databases containing molecular structures (Sali et al. , 2015 ▸ ) and the development of links between repositories. As these individual repositories grow, methods for searching them will also need to be improved.”limitations[2]
limitationInstant open availability can come at the expense of data quality, completeness, integrity, and sustainability. — “There is now an expectation in science that data is freely, openly and instantly available. With modern informatics techniques this is trivial to achieve, but is often done at the expense of data quality, completeness, integrity and with questionable sustainability. History has shown us that all these are best achieved by specialist data centres, maintaining specialist data repositories. The challenge of finding a universally agreed ‘perfect’ funding model, if one exists, remains (Bruno & Groom, 2014 ▸ ).”limitations[3]
limitationThe paper leaves the sustainable funding model for specialist data repositories unresolved. — “There is now an expectation in science that data is freely, openly and instantly available. With modern informatics techniques this is trivial to achieve, but is often done at the expense of data quality, completeness, integrity and with questionable sustainability. History has shown us that all these are best achieved by specialist data centres, maintaining specialist data repositories. The challenge of finding a universally agreed ‘perfect’ funding model, if one exists, remains (Bruno & Groom, 2014 ▸ ).”limitations[4]
Content tags
content-observations: finding · quantity · variable · definition · key-term — grounded in the full text
findingThe CSD contains all published organic and metal–organic small-molecule crystal structures determined by crystallography and also publishes determinations without an accompanying manuscript. — “The CSD contains all published organic and metal–organic small-molecule crystal structures whose structures have been determined using the technique of crystallography. In addition, it acts as a publication vehicle for structure determinations with no accompanying manuscript.”content_tags.findings[0]
findingThe database aggregates and standardizes structures, improving access and usually permitting reuse without further processing after curation. — “The database provides value in two distinct ways. The first simply relates to the aggregation and standardization of structures, which facilitates access to individual entries. This brings value both to the data generators and consumers. A single archive of all structures allows crystallographers to avoid the inadvertent redetermination of structures and provides a mechanism by which they can archive the output of their work at a specialist data centre for their own future use. Of course, it also allows for the easy sharing of their work, massively increasing their sphere of influence. Such sharing has always been the norm for the crystallographic community and the use of this worldwide, standard, specialist discipline repository allows individuals to demonstrate their adherence to the new data sharing mandates of various funding bodies. As all entries are subject to both automatic and manual curation, they can usually be used without further processing. Indeed, one might argue that the financial cost of maintaining such a resource, although significant, is recovered many times over by removing the need for repeated correction by users.”content_tags.findings[1]
findingThe authors characterize the CSD as a complete, rapidly updated record of published structures. — “A further vital property of the CSD is its comprehensive and up-to-date nature. As it represents the complete record of published structures and is updated within a few moments of a new publication, users can have confidence that there are no published structures of relevance of which they are unaware.”content_tags.findings[2]
findingBy 2015 the online deposition service was the main route, with most structures deposited before publication and most submitted through the service. — “The method for deposition into the CSD has evolved since the advent of the CIF format in the 1990s, when email depositions dominated. In 2009 the CCDC launched an online web-based tool which is now the main route for deposition. In 2015, 90% of structures were deposited with the CCDC prior to publication and 85% were submitted through this service. A key benefit of this early deposition is that at this point the crystallographer who generated the data is likely to be the depositor and be in a position both to provide the richest data and to respond to any issues most effectively.”content_tags.findings[3]
findingDeCIFer combines stored CSD information with Bayesian inference to assign chemical representations to new crystallographic entries. — “DeCIFer uses the information already in the CSD to interpret a new structure and add a chemical representation to the atomic coordinates in the CIF. The stages involved in the automatic creation of the CSD entry include resolution of any disorder in the structure, detection of bonds and determination of bond types and charges. Assignment of bond types, charges and the inference of missing H atoms uses a probabilistic Bayesian approach which allows all the existing entries in the CSD to be used to help assign the chemistry to new entries.”content_tags.findings[4]
findingAutomatic deCIFer processing is followed by manual inspection by expert structural chemists. — “Although the automatic deCIFer process is used as much as possible structures are still manually viewed by expert structural chemists before they are added to the CSD.”content_tags.findings[5]
findingCommunity services provide free access to the complete structure collection, while advanced and specialist applications are provided through other products. — “The CSD community web services detailed earlier provide free access to the entire collection of structures. As well as these services there are a number of other avenues to explore and exploit the data ranging from free lookup tools such as CellCheckCSD (Wood, 2011 ▸ ) to advanced search, analysis and validation tools in the CSD-System (Bruno et al. , 1997 ▸ , 2002 ▸ , 2004 ▸ ; Macrae et al. , 2008 ▸ ). More specialist applications are provided as part of the CSD-Enterprise suite.”content_tags.findings[6]
findingInChI matching had established more than 52,000 reliable links between compounds in the CSD and external chemical resources. — “Discoverability of data by chemists and biologists is enabled by establishing links to datasets from services such as ChemSpider (Pence & Williams, 2010 ▸ ) and PubChem (Bolton et al. , 2008 ▸ ). The overlap between these resources and the CSD is identified by taking advantage of the International Chemical Identifier Standard (InChI) which provides a unique and canonical representation of the chemical substance studied (Heller et al. , 2015 ▸ ). Links from ChemSpider and PubChem have been established for over 52 000 compounds that could be reliably identified using InChIs as being in common between these resources and the CSD. InChIs have also been used to identify correspondences between CSD entries and ligands bound to macromolecules in structures archived in the Protein Data Bank (PDB; Berman et al. , 2000 ▸ ). This lookup is enabled by a CCDC web service that identifies the best-representative CSD entry for a given molecule and provides access to its coordinates. This is a particularly useful resource for structural biologists refining or investigating the structure of a protein ligand. Representative structures are freely available for approximately 1500 PDB ligands in a Chemical Component Model file provided by the PDB (wwPDB, 2015 ▸ ).”content_tags.findings[7]
quantity90% — share of structures deposited with CCDC before publication in 2015 — “The method for deposition into the CSD has evolved since the advent of the CIF format in the 1990s, when email depositions dominated. In 2009 the CCDC launched an online web-based tool which is now the main route for deposition. In 2015, 90% of structures were deposited with the CCDC prior to publication and 85% were submitted through this service. A key benefit of this early deposition is that at this point the crystallographer who generated the data is likely to be the depositor and be in a position both to provide the richest data and to respond to any issues most effectively.”content_tags.quantities[0]
quantity85% — share of structures submitted through the online deposition service in 2015 — “The method for deposition into the CSD has evolved since the advent of the CIF format in the 1990s, when email depositions dominated. In 2009 the CCDC launched an online web-based tool which is now the main route for deposition. In 2015, 90% of structures were deposited with the CCDC prior to publication and 85% were submitted through this service. A key benefit of this early deposition is that at this point the crystallographer who generated the data is likely to be the depositor and be in a position both to provide the richest data and to respond to any issues most effectively.”content_tags.quantities[1]
quantity580 000 — deposited datasets with an associated DOI when the article was written — “At the point a structure can be made public a Digital Object Identifier (DOI) is associated with the deposited dataset (580 000 are currently available). This DOI allows a third party to link to a structure summary page without needing to know details of linking services provided by the CCDC. In addition the DOI provides the basis for a more formal citation of the deposited dataset in line with the spirit of the Joint Declaration of Data Citation Principles (Data Citation Synthesis Group, 2014 ▸ ). The metadata provided to the DOI registration agency DataCite (DataCite, 2016 ▸ ) indicates key elements of a dataset citation including contributor, a title and publication year. This enables citation of the dataset independently of the associated article and thus helps ensure recognition of the specific contribution made by the crystallographer, who may not always be included in the article’s author list.”content_tags.quantities[2]
quantity530 000 — CSD entries available in the Thomson Reuters Data Citation Index — “Metadata submitted by the CCDC to DataCite when generating a DOI is openly accessible and facilitates interoperability with third party systems to improve the discoverability of data. Examples where other parties have taken advantage of this include the Thomson Reuters Data Citation Index (Thomson Reuters, 2016 ▸ ), where data is currently available for 530 000 CSD entries, and the prototype RDA/WDS Data-Literature Interlinking service (Burton et al. , 2015 ▸ ). In order to support this interoperability, additional metadata items are made public including the DOIs of associated articles and the chemical name of the substance studied.”content_tags.quantities[3]
quantityover 52 000 — compounds reliably linked between ChemSpider or PubChem and the CSD using InChIs — “Discoverability of data by chemists and biologists is enabled by establishing links to datasets from services such as ChemSpider (Pence & Williams, 2010 ▸ ) and PubChem (Bolton et al. , 2008 ▸ ). The overlap between these resources and the CSD is identified by taking advantage of the International Chemical Identifier Standard (InChI) which provides a unique and canonical representation of the chemical substance studied (Heller et al. , 2015 ▸ ). Links from ChemSpider and PubChem have been established for over 52 000 compounds that could be reliably identified using InChIs as being in common between these resources and the CSD. InChIs have also been used to identify correspondences between CSD entries and ligands bound to macromolecules in structures archived in the Protein Data Bank (PDB; Berman et al. , 2000 ▸ ). This lookup is enabled by a CCDC web service that identifies the best-representative CSD entry for a given molecule and provides access to its coordinates. This is a particularly useful resource for structural biologists refining or investigating the structure of a protein ligand. Representative structures are freely available for approximately 1500 PDB ligands in a Chemical Component Model file provided by the PDB (wwPDB, 2015 ▸ ).”content_tags.quantities[4]
quantityapproximately 1500 — PDB ligands with freely available representative structures — “Discoverability of data by chemists and biologists is enabled by establishing links to datasets from services such as ChemSpider (Pence & Williams, 2010 ▸ ) and PubChem (Bolton et al. , 2008 ▸ ). The overlap between these resources and the CSD is identified by taking advantage of the International Chemical Identifier Standard (InChI) which provides a unique and canonical representation of the chemical substance studied (Heller et al. , 2015 ▸ ). Links from ChemSpider and PubChem have been established for over 52 000 compounds that could be reliably identified using InChIs as being in common between these resources and the CSD. InChIs have also been used to identify correspondences between CSD entries and ligands bound to macromolecules in structures archived in the Protein Data Bank (PDB; Berman et al. , 2000 ▸ ). This lookup is enabled by a CCDC web service that identifies the best-representative CSD entry for a given molecule and provides access to its coordinates. This is a particularly useful resource for structural biologists refining or investigating the structure of a protein ligand. Representative structures are freely available for approximately 1500 PDB ligands in a Chemical Component Model file provided by the PDB (wwPDB, 2015 ▸ ).”content_tags.quantities[5]
quantity15% — estimated share of determined crystal structures that are published — “Although the CSD contains all published crystal structures, it has been estimated that only 15% of structures determined are published (UK National Crystallography Service, 2014 ▸ ). Automatic links in software used during structure determination (Dolomanov et al. , 2009 ▸ ), the ease with which structures can be deposited, attribution of credit in the form of a DOI and continued demonstration of the value to science of depositing crystal structures (Berman et al. , 2015 ▸ ) may help close this gap.”content_tags.quantities[6]
variableCIF syntax (machine-checked input whose issues must be fixed before deposition continues) — “During deposition, the CIF syntax is automatically checked based on the checks in enCIFer (Allen et al. , 2004 ▸ ) and the depositor is required to fix any issues before continuing with the deposition process. Depositors are strongly encouraged to deposit structure factors in line with the IUCr’s publication standards for crystal structures (IUCr, 2011 ▸ ). The embedding of reflection data into the CIF by structure refinement programs such as SHELXL (Sheldrick, 2015 ▸ ) greatly simplifies this process. As a consequence the amount of reflection data stored at CCDC has increased significantly and the majority of new depositions now include this data.”content_tags.variables[0]
variablechemical identity (curated metadata needed for reliable search and reuse of a structure) — “DeCIFer uses the information already in the CSD to interpret a new structure and add a chemical representation to the atomic coordinates in the CIF. The stages involved in the automatic creation of the CSD entry include resolution of any disorder in the structure, detection of bonds and determination of bond types and charges. Assignment of bond types, charges and the inference of missing H atoms uses a probabilistic Bayesian approach which allows all the existing entries in the CSD to be used to help assign the chemistry to new entries.”content_tags.variables[1]
variablereliability score (confidence indicator used to highlight possible automatic-representation errors for review) — “The final step of the deCIFer process is to validate the assignment by looking at improbable features such as unprecedented bond types, unlikely oxidation states, unlikely metal–metal bonds and other empirical indicators such as non-planar double bonds. From this analysis a reliability score is calculated which highlights any possible errors in the representation of the structure which can then be reviewed and corrected during CSD entry validation. It is important to note that this reliability score simply gives a confidence value to the automatic treatment of the entry; this is not necessarily related to the ‘quality’ of the structure.”content_tags.variables[2]
variableInChI (canonical substance identifier used to identify overlap and correspondence across resources) — “Discoverability of data by chemists and biologists is enabled by establishing links to datasets from services such as ChemSpider (Pence & Williams, 2010 ▸ ) and PubChem (Bolton et al. , 2008 ▸ ). The overlap between these resources and the CSD is identified by taking advantage of the International Chemical Identifier Standard (InChI) which provides a unique and canonical representation of the chemical substance studied (Heller et al. , 2015 ▸ ). Links from ChemSpider and PubChem have been established for over 52 000 compounds that could be reliably identified using InChIs as being in common between these resources and the CSD. InChIs have also been used to identify correspondences between CSD entries and ligands bound to macromolecules in structures archived in the Protein Data Bank (PDB; Berman et al. , 2000 ▸ ). This lookup is enabled by a CCDC web service that identifies the best-representative CSD entry for a given molecule and provides access to its coordinates. This is a particularly useful resource for structural biologists refining or investigating the structure of a protein ligand. Representative structures are freely available for approximately 1500 PDB ligands in a Chemical Component Model file provided by the PDB (wwPDB, 2015 ▸ ).”content_tags.variables[3]
definitionCambridge Structural Database — “This paper is the definitive article describing the creation, maintenance, information content and availability of the Cambridge Structural Database (CSD), the world’s repository of small molecule crystal structures.”content_tags.definitions[0]
definitionCIF — “Just as the database and software evolved so too did the way in which crystal structure data is communicated. In the early days atomic coordinates were published as tables in scientific articles. These were transcribed manually for inclusion in data files and databases. In the 1990s, a file format called CIF (now the Crystallographic Information Framework) was proposed as a standard for the interchange of crystallographic data. CIF is now ubiquitously used to capture the results of a diffraction experiment and enables streamlined publication of results alongside journal articles and in data repositories (Hall et al. , 1991 ▸ ; Brown & McMahon, 2002 ▸ ; Bernstein et al. , 2016 ▸ ; Hall & McMahon, 2016 ▸ ).”content_tags.definitions[2]
definitionCCDC number — “On deposition, each dataset is assigned an accession identifier referred to as a ‘CCDC number’ in the format CCDC 1234567 (older entries have six digits). This uniquely identifies the data associated with a particular structure determination and persists for the lifetime of the dataset. CCDC numbers are communicated to the depositor once it has been confirmed that the dataset is not a duplicate submission, usually within seconds of deposition. Should the data require further investigation, CCDC Deposition Coordinators address any problems, minimizing the delay in providing the CCDC number to the depositor. A reference code (known as a CSD refcode and in the format ABCDEF) is assigned to structures as they are indexed into the CSD itself. Where possible, determinations of the same substance are assigned into a CSD refcode family (with a CSD refcode format of ABCDEF01). These codes are used as a common way of referring to structures extracted from the CSD-System.”content_tags.definitions[3]
definitionCSD refcode — “On deposition, each dataset is assigned an accession identifier referred to as a ‘CCDC number’ in the format CCDC 1234567 (older entries have six digits). This uniquely identifies the data associated with a particular structure determination and persists for the lifetime of the dataset. CCDC numbers are communicated to the depositor once it has been confirmed that the dataset is not a duplicate submission, usually within seconds of deposition. Should the data require further investigation, CCDC Deposition Coordinators address any problems, minimizing the delay in providing the CCDC number to the depositor. A reference code (known as a CSD refcode and in the format ABCDEF) is assigned to structures as they are indexed into the CSD itself. Where possible, determinations of the same substance are assigned into a CSD refcode family (with a CSD refcode format of ABCDEF01). These codes are used as a common way of referring to structures extracted from the CSD-System.”content_tags.definitions[4]
definitionreliability score — “The final step of the deCIFer process is to validate the assignment by looking at improbable features such as unprecedented bond types, unlikely oxidation states, unlikely metal–metal bonds and other empirical indicators such as non-planar double bonds. From this analysis a reliability score is calculated which highlights any possible errors in the representation of the structure which can then be reviewed and corrected during CSD entry validation. It is important to note that this reliability score simply gives a confidence value to the automatic treatment of the entry; this is not necessarily related to the ‘quality’ of the structure.”content_tags.definitions[5]
definitionInChI — “Discoverability of data by chemists and biologists is enabled by establishing links to datasets from services such as ChemSpider (Pence & Williams, 2010 ▸ ) and PubChem (Bolton et al. , 2008 ▸ ). The overlap between these resources and the CSD is identified by taking advantage of the International Chemical Identifier Standard (InChI) which provides a unique and canonical representation of the chemical substance studied (Heller et al. , 2015 ▸ ). Links from ChemSpider and PubChem have been established for over 52 000 compounds that could be reliably identified using InChIs as being in common between these resources and the CSD. InChIs have also been used to identify correspondences between CSD entries and ligands bound to macromolecules in structures archived in the Protein Data Bank (PDB; Berman et al. , 2000 ▸ ). This lookup is enabled by a CCDC web service that identifies the best-representative CSD entry for a given molecule and provides access to its coordinates. This is a particularly useful resource for structural biologists refining or investigating the structure of a protein ligand. Representative structures are freely available for approximately 1500 PDB ligands in a Chemical Component Model file provided by the PDB (wwPDB, 2015 ▸ ).”content_tags.definitions[6]
definitionCellCheckCSD — “CellCheckCSD is an automated tool which uses the data from the CSD to check unit cells during data collection and can be used to check that the structure is novel rather than the starting material, a by-product or another previously determined structure.”content_tags.definitions[7]
Concepts
terms this paper carries, each with the sentence it was cut from — the layer that joins one paper to another. Terms with no quote yet are on CONCEPTS.html.
conceptCambridge Structural Database — not defined in this paper's own words; the curated global repository of small-molecule crystal structures — “This paper is the definitive article describing the creation, maintenance, information content and availability of the Cambridge Structural Database (CSD), the world’s repository of small molecule crystal structures.”concepts[0]
conceptCSD — not defined in this paper's own words; the abbreviation for the Cambridge Structural Database — “This paper is the definitive article describing the creation, maintenance, information content and availability of the Cambridge Structural Database (CSD), the world’s repository of small molecule crystal structures.”concepts[1]
conceptsmall molecule crystal structures — not defined in this paper's own words; the structural records described as the CSD's repository content — “This paper is the definitive article describing the creation, maintenance, information content and availability of the Cambridge Structural Database (CSD), the world’s repository of small molecule crystal structures.”concepts[2]
conceptcrystallography — not defined in this paper's own words; the structure-determination technique defining the CSD's core coverage — “The CSD contains all published organic and metal–organic small-molecule crystal structures whose structures have been determined using the technique of crystallography. In addition, it acts as a publication vehicle for structure determinations with no accompanying manuscript.”concepts[3]
conceptX-ray — not defined in this paper's own words; one diffraction mode represented in CSD analyses — “Specifically, the CSD contains both X-ray and neutron diffraction analyses from a single-crystal study or a powder study where cell parameters, atomic coordinates and refinement are reported. To ensure comprehensive coverage of single-crystal data, cell parameters and all available data are included even if no coordinates are available. Powder structures without coordinates are available from the International Centre for Diffraction Data (ICDD; Kabekkodu et al. , 2002 ▸ ).”concepts[4]
conceptneutron diffraction analyses — not defined in this paper's own words; a diffraction-analysis class included in the CSD — “Specifically, the CSD contains both X-ray and neutron diffraction analyses from a single-crystal study or a powder study where cell parameters, atomic coordinates and refinement are reported. To ensure comprehensive coverage of single-crystal data, cell parameters and all available data are included even if no coordinates are available. Powder structures without coordinates are available from the International Centre for Diffraction Data (ICDD; Kabekkodu et al. , 2002 ▸ ).”concepts[5]
conceptaggregation and standardization of structures — not defined in this paper's own words; the database operation that facilitates entry-level access — “The database provides value in two distinct ways. The first simply relates to the aggregation and standardization of structures, which facilitates access to individual entries. This brings value both to the data generators and consumers. A single archive of all structures allows crystallographers to avoid the inadvertent redetermination of structures and provides a mechanism by which they can archive the output of their work at a specialist data centre for their own future use. Of course, it also allows for the easy sharing of their work, massively increasing their sphere of influence. Such sharing has always been the norm for the crystallographic community and the use of this worldwide, standard, specialist discipline repository allows individuals to demonstrate their adherence to the new data sharing mandates of various funding bodies. As all entries are subject to both automatic and manual curation, they can usually be used without further processing. Indeed, one might argue that the financial cost of maintaining such a resource, although significant, is recovered many times over by removing the need for repeated correction by users.”concepts[6]
conceptaccess to individual entries — not defined in this paper's own words; the access outcome facilitated by aggregation and standardization — “The database provides value in two distinct ways. The first simply relates to the aggregation and standardization of structures, which facilitates access to individual entries. This brings value both to the data generators and consumers. A single archive of all structures allows crystallographers to avoid the inadvertent redetermination of structures and provides a mechanism by which they can archive the output of their work at a specialist data centre for their own future use. Of course, it also allows for the easy sharing of their work, massively increasing their sphere of influence. Such sharing has always been the norm for the crystallographic community and the use of this worldwide, standard, specialist discipline repository allows individuals to demonstrate their adherence to the new data sharing mandates of various funding bodies. As all entries are subject to both automatic and manual curation, they can usually be used without further processing. Indeed, one might argue that the financial cost of maintaining such a resource, although significant, is recovered many times over by removing the need for repeated correction by users.”concepts[7]
conceptautomatic and manual curation — not defined in this paper's own words; the combined curation regime applied to entries — “The database provides value in two distinct ways. The first simply relates to the aggregation and standardization of structures, which facilitates access to individual entries. This brings value both to the data generators and consumers. A single archive of all structures allows crystallographers to avoid the inadvertent redetermination of structures and provides a mechanism by which they can archive the output of their work at a specialist data centre for their own future use. Of course, it also allows for the easy sharing of their work, massively increasing their sphere of influence. Such sharing has always been the norm for the crystallographic community and the use of this worldwide, standard, specialist discipline repository allows individuals to demonstrate their adherence to the new data sharing mandates of various funding bodies. As all entries are subject to both automatic and manual curation, they can usually be used without further processing. Indeed, one might argue that the financial cost of maintaining such a resource, although significant, is recovered many times over by removing the need for repeated correction by users.”concepts[8]
conceptused without further processing — not defined in this paper's own words; the usual reuse condition claimed for curated entries — “The database provides value in two distinct ways. The first simply relates to the aggregation and standardization of structures, which facilitates access to individual entries. This brings value both to the data generators and consumers. A single archive of all structures allows crystallographers to avoid the inadvertent redetermination of structures and provides a mechanism by which they can archive the output of their work at a specialist data centre for their own future use. Of course, it also allows for the easy sharing of their work, massively increasing their sphere of influence. Such sharing has always been the norm for the crystallographic community and the use of this worldwide, standard, specialist discipline repository allows individuals to demonstrate their adherence to the new data sharing mandates of various funding bodies. As all entries are subject to both automatic and manual curation, they can usually be used without further processing. Indeed, one might argue that the financial cost of maintaining such a resource, although significant, is recovered many times over by removing the need for repeated correction by users.”concepts[9]
conceptCSD Communications — not defined in this paper's own words; the CSD route for direct publication of structures — “The increasing digital availability of the basic results of a diffraction experiment has also had an impact on the overall nature of the CSD. Originally, it might best have been described as a secondary resource of data abstracted from the literature. Today, new source data files are almost always deposited directly into the CSD making it much more of a primary data resource. Indeed, many structures are published only and directly through the CSD as CSD Communications (previously known as Private Communications). Whilst the CCDC has never charged for deposition, curation, archiving or for access to the primary source data files, early versions of the CCDC’s informatics systems did have technical barriers that hindered access to structures by researchers. Here, we describe developments that ensure entries from the CSD are made available to the widest possible community through CCDC services and third party resources.”concepts[10]
conceptprimary data resource — not defined in this paper's own words; the contemporary role of the CSD after direct source-data deposition — “The increasing digital availability of the basic results of a diffraction experiment has also had an impact on the overall nature of the CSD. Originally, it might best have been described as a secondary resource of data abstracted from the literature. Today, new source data files are almost always deposited directly into the CSD making it much more of a primary data resource. Indeed, many structures are published only and directly through the CSD as CSD Communications (previously known as Private Communications). Whilst the CCDC has never charged for deposition, curation, archiving or for access to the primary source data files, early versions of the CCDC’s informatics systems did have technical barriers that hindered access to structures by researchers. Here, we describe developments that ensure entries from the CSD are made available to the widest possible community through CCDC services and third party resources.”concepts[11]
conceptCIF — not defined in this paper's own words; the standard file format used to capture and interchange crystallographic data — “Just as the database and software evolved so too did the way in which crystal structure data is communicated. In the early days atomic coordinates were published as tables in scientific articles. These were transcribed manually for inclusion in data files and databases. In the 1990s, a file format called CIF (now the Crystallographic Information Framework) was proposed as a standard for the interchange of crystallographic data. CIF is now ubiquitously used to capture the results of a diffraction experiment and enables streamlined publication of results alongside journal articles and in data repositories (Hall et al. , 1991 ▸ ; Brown & McMahon, 2002 ▸ ; Bernstein et al. , 2016 ▸ ; Hall & McMahon, 2016 ▸ ).”concepts[12]
conceptCrystallographic Information Framework — not defined in this paper's own words; the expanded name of CIF — “Just as the database and software evolved so too did the way in which crystal structure data is communicated. In the early days atomic coordinates were published as tables in scientific articles. These were transcribed manually for inclusion in data files and databases. In the 1990s, a file format called CIF (now the Crystallographic Information Framework) was proposed as a standard for the interchange of crystallographic data. CIF is now ubiquitously used to capture the results of a diffraction experiment and enables streamlined publication of results alongside journal articles and in data repositories (Hall et al. , 1991 ▸ ; Brown & McMahon, 2002 ▸ ; Bernstein et al. , 2016 ▸ ; Hall & McMahon, 2016 ▸ ).”concepts[13]
conceptCCDC number — not defined in this paper's own words; a persistent accession identifier for a deposited dataset — “On deposition, each dataset is assigned an accession identifier referred to as a ‘CCDC number’ in the format CCDC 1234567 (older entries have six digits). This uniquely identifies the data associated with a particular structure determination and persists for the lifetime of the dataset. CCDC numbers are communicated to the depositor once it has been confirmed that the dataset is not a duplicate submission, usually within seconds of deposition. Should the data require further investigation, CCDC Deposition Coordinators address any problems, minimizing the delay in providing the CCDC number to the depositor. A reference code (known as a CSD refcode and in the format ABCDEF) is assigned to structures as they are indexed into the CSD itself. Where possible, determinations of the same substance are assigned into a CSD refcode family (with a CSD refcode format of ABCDEF01). These codes are used as a common way of referring to structures extracted from the CSD-System.”concepts[14]
conceptaccession identifier — not defined in this paper's own words; the identifier class to which a CCDC number belongs — “On deposition, each dataset is assigned an accession identifier referred to as a ‘CCDC number’ in the format CCDC 1234567 (older entries have six digits). This uniquely identifies the data associated with a particular structure determination and persists for the lifetime of the dataset. CCDC numbers are communicated to the depositor once it has been confirmed that the dataset is not a duplicate submission, usually within seconds of deposition. Should the data require further investigation, CCDC Deposition Coordinators address any problems, minimizing the delay in providing the CCDC number to the depositor. A reference code (known as a CSD refcode and in the format ABCDEF) is assigned to structures as they are indexed into the CSD itself. Where possible, determinations of the same substance are assigned into a CSD refcode family (with a CSD refcode format of ABCDEF01). These codes are used as a common way of referring to structures extracted from the CSD-System.”concepts[15]
conceptCSD refcode — not defined in this paper's own words; the reference code assigned when a structure is indexed into the CSD — “On deposition, each dataset is assigned an accession identifier referred to as a ‘CCDC number’ in the format CCDC 1234567 (older entries have six digits). This uniquely identifies the data associated with a particular structure determination and persists for the lifetime of the dataset. CCDC numbers are communicated to the depositor once it has been confirmed that the dataset is not a duplicate submission, usually within seconds of deposition. Should the data require further investigation, CCDC Deposition Coordinators address any problems, minimizing the delay in providing the CCDC number to the depositor. A reference code (known as a CSD refcode and in the format ABCDEF) is assigned to structures as they are indexed into the CSD itself. Where possible, determinations of the same substance are assigned into a CSD refcode family (with a CSD refcode format of ABCDEF01). These codes are used as a common way of referring to structures extracted from the CSD-System.”concepts[16]
conceptCSD-Xpedite — not defined in this paper's own words; the processing system that automates management of depositions and creation of CSD entries — “Due to the rising number of structures, depositions and transactions a new processing system, CSD-Xpedite , has been developed, automating most informatics processes required to manage depositions and process crystal structures into entries in the CSD. Fig. 6 ▸ shows how data flows through the system and where various users interact with the system.”concepts[17]
conceptprocessing system — not defined in this paper's own words; the system class assigned to CSD-Xpedite — “Due to the rising number of structures, depositions and transactions a new processing system, CSD-Xpedite , has been developed, automating most informatics processes required to manage depositions and process crystal structures into entries in the CSD. Fig. 6 ▸ shows how data flows through the system and where various users interact with the system.”concepts[18]
conceptCSD-Editor — not defined in this paper's own words; the interactive structure-processing component of CSD-Xpedite — “A key component in CSD-Xpedite is CSD-Editor , an interactive tool for processing structures such that they can enter into the CSD. This uses many of the visualization and menu options available in the CSD-System program Mercury (Macrae et al. , 2006 ▸ ). Deposited CIFs can be easily viewed alongside any associated article. Errors and warnings are displayed for each structure to allow expert structural chemistry editors to concentrate their efforts on the challenging scientific parts of the process.”concepts[19]
conceptdeCIFer — not defined in this paper's own words; the automatic program for assigning chemical representations to crystallographic coordinates — “DeCIFer uses the information already in the CSD to interpret a new structure and add a chemical representation to the atomic coordinates in the CIF. The stages involved in the automatic creation of the CSD entry include resolution of any disorder in the structure, detection of bonds and determination of bond types and charges. Assignment of bond types, charges and the inference of missing H atoms uses a probabilistic Bayesian approach which allows all the existing entries in the CSD to be used to help assign the chemistry to new entries.”concepts[20]
conceptprobabilistic Bayesian approach — not defined in this paper's own words; the inference approach deCIFer uses to assign chemistry — “DeCIFer uses the information already in the CSD to interpret a new structure and add a chemical representation to the atomic coordinates in the CIF. The stages involved in the automatic creation of the CSD entry include resolution of any disorder in the structure, detection of bonds and determination of bond types and charges. Assignment of bond types, charges and the inference of missing H atoms uses a probabilistic Bayesian approach which allows all the existing entries in the CSD to be used to help assign the chemistry to new entries.”concepts[21]
conceptreliability score — not defined in this paper's own words; confidence in the automatic treatment of an entry, not necessarily structure quality — “The final step of the deCIFer process is to validate the assignment by looking at improbable features such as unprecedented bond types, unlikely oxidation states, unlikely metal–metal bonds and other empirical indicators such as non-planar double bonds. From this analysis a reliability score is calculated which highlights any possible errors in the representation of the structure which can then be reviewed and corrected during CSD entry validation. It is important to note that this reliability score simply gives a confidence value to the automatic treatment of the entry; this is not necessarily related to the ‘quality’ of the structure.”concepts[22]
conceptimprobable features — not defined in this paper's own words; empirical anomalies used to validate automatic chemical assignment — “The final step of the deCIFer process is to validate the assignment by looking at improbable features such as unprecedented bond types, unlikely oxidation states, unlikely metal–metal bonds and other empirical indicators such as non-planar double bonds. From this analysis a reliability score is calculated which highlights any possible errors in the representation of the structure which can then be reviewed and corrected during CSD entry validation. It is important to note that this reliability score simply gives a confidence value to the automatic treatment of the entry; this is not necessarily related to the ‘quality’ of the structure.”concepts[23]
conceptDigital Object Identifier — not defined in this paper's own words; an identifier associated with a public deposited dataset to support linking and citation — “At the point a structure can be made public a Digital Object Identifier (DOI) is associated with the deposited dataset (580 000 are currently available). This DOI allows a third party to link to a structure summary page without needing to know details of linking services provided by the CCDC. In addition the DOI provides the basis for a more formal citation of the deposited dataset in line with the spirit of the Joint Declaration of Data Citation Principles (Data Citation Synthesis Group, 2014 ▸ ). The metadata provided to the DOI registration agency DataCite (DataCite, 2016 ▸ ) indicates key elements of a dataset citation including contributor, a title and publication year. This enables citation of the dataset independently of the associated article and thus helps ensure recognition of the specific contribution made by the crystallographer, who may not always be included in the article’s author list.”concepts[24]
conceptDOI — not defined in this paper's own words; the abbreviation for Digital Object Identifier — “At the point a structure can be made public a Digital Object Identifier (DOI) is associated with the deposited dataset (580 000 are currently available). This DOI allows a third party to link to a structure summary page without needing to know details of linking services provided by the CCDC. In addition the DOI provides the basis for a more formal citation of the deposited dataset in line with the spirit of the Joint Declaration of Data Citation Principles (Data Citation Synthesis Group, 2014 ▸ ). The metadata provided to the DOI registration agency DataCite (DataCite, 2016 ▸ ) indicates key elements of a dataset citation including contributor, a title and publication year. This enables citation of the dataset independently of the associated article and thus helps ensure recognition of the specific contribution made by the crystallographer, who may not always be included in the article’s author list.”concepts[25]
conceptDataCite — not defined in this paper's own words; the DOI registration agency receiving openly accessible CCDC metadata — “Metadata submitted by the CCDC to DataCite when generating a DOI is openly accessible and facilitates interoperability with third party systems to improve the discoverability of data. Examples where other parties have taken advantage of this include the Thomson Reuters Data Citation Index (Thomson Reuters, 2016 ▸ ), where data is currently available for 530 000 CSD entries, and the prototype RDA/WDS Data-Literature Interlinking service (Burton et al. , 2015 ▸ ). In order to support this interoperability, additional metadata items are made public including the DOIs of associated articles and the chemical name of the substance studied.”concepts[26]
conceptInternational Chemical Identifier Standard — not defined in this paper's own words; the identifier standard used to match chemical substances across resources — “Discoverability of data by chemists and biologists is enabled by establishing links to datasets from services such as ChemSpider (Pence & Williams, 2010 ▸ ) and PubChem (Bolton et al. , 2008 ▸ ). The overlap between these resources and the CSD is identified by taking advantage of the International Chemical Identifier Standard (InChI) which provides a unique and canonical representation of the chemical substance studied (Heller et al. , 2015 ▸ ). Links from ChemSpider and PubChem have been established for over 52 000 compounds that could be reliably identified using InChIs as being in common between these resources and the CSD. InChIs have also been used to identify correspondences between CSD entries and ligands bound to macromolecules in structures archived in the Protein Data Bank (PDB; Berman et al. , 2000 ▸ ). This lookup is enabled by a CCDC web service that identifies the best-representative CSD entry for a given molecule and provides access to its coordinates. This is a particularly useful resource for structural biologists refining or investigating the structure of a protein ligand. Representative structures are freely available for approximately 1500 PDB ligands in a Chemical Component Model file provided by the PDB (wwPDB, 2015 ▸ ).”concepts[27]
conceptInChI — not defined in this paper's own words; a unique canonical representation used for cross-resource substance matching — “Discoverability of data by chemists and biologists is enabled by establishing links to datasets from services such as ChemSpider (Pence & Williams, 2010 ▸ ) and PubChem (Bolton et al. , 2008 ▸ ). The overlap between these resources and the CSD is identified by taking advantage of the International Chemical Identifier Standard (InChI) which provides a unique and canonical representation of the chemical substance studied (Heller et al. , 2015 ▸ ). Links from ChemSpider and PubChem have been established for over 52 000 compounds that could be reliably identified using InChIs as being in common between these resources and the CSD. InChIs have also been used to identify correspondences between CSD entries and ligands bound to macromolecules in structures archived in the Protein Data Bank (PDB; Berman et al. , 2000 ▸ ). This lookup is enabled by a CCDC web service that identifies the best-representative CSD entry for a given molecule and provides access to its coordinates. This is a particularly useful resource for structural biologists refining or investigating the structure of a protein ligand. Representative structures are freely available for approximately 1500 PDB ligands in a Chemical Component Model file provided by the PDB (wwPDB, 2015 ▸ ).”concepts[28]
conceptchemical substance studied — not defined in this paper's own words; the substance represented canonically by an InChI — “Discoverability of data by chemists and biologists is enabled by establishing links to datasets from services such as ChemSpider (Pence & Williams, 2010 ▸ ) and PubChem (Bolton et al. , 2008 ▸ ). The overlap between these resources and the CSD is identified by taking advantage of the International Chemical Identifier Standard (InChI) which provides a unique and canonical representation of the chemical substance studied (Heller et al. , 2015 ▸ ). Links from ChemSpider and PubChem have been established for over 52 000 compounds that could be reliably identified using InChIs as being in common between these resources and the CSD. InChIs have also been used to identify correspondences between CSD entries and ligands bound to macromolecules in structures archived in the Protein Data Bank (PDB; Berman et al. , 2000 ▸ ). This lookup is enabled by a CCDC web service that identifies the best-representative CSD entry for a given molecule and provides access to its coordinates. This is a particularly useful resource for structural biologists refining or investigating the structure of a protein ligand. Representative structures are freely available for approximately 1500 PDB ligands in a Chemical Component Model file provided by the PDB (wwPDB, 2015 ▸ ).”concepts[29]
conceptCellCheckCSD — not defined in this paper's own words; an automated unit-cell novelty checking tool — “CellCheckCSD is an automated tool which uses the data from the CSD to check unit cells during data collection and can be used to check that the structure is novel rather than the starting material, a by-product or another previously determined structure.”concepts[30]
conceptautomated tool — not defined in this paper's own words; the method class assigned to CellCheckCSD — “CellCheckCSD is an automated tool which uses the data from the CSD to check unit cells during data collection and can be used to check that the structure is novel rather than the starting material, a by-product or another previously determined structure.”concepts[31]
conceptCSD-System — not defined in this paper's own words; the search and analysis suite for extracting knowledge from CSD data — “The CSD-System enables knowledge to be gained from the collection of data through powerful two-dimensional/three-dimensional search capabilities, extensive geometry analysis tools, inter- and intramolecular interaction analysis, generation of high impact graphics and the ability to delve further into the data using a Python-based applications programming interface.”concepts[32]
conceptCSD community web services — not defined in this paper's own words; the free public-access services for the CSD structure collection — “The CSD community web services detailed earlier provide free access to the entire collection of structures. As well as these services there are a number of other avenues to explore and exploit the data ranging from free lookup tools such as CellCheckCSD (Wood, 2011 ▸ ) to advanced search, analysis and validation tools in the CSD-System (Bruno et al. , 1997 ▸ , 2002 ▸ , 2004 ▸ ; Macrae et al. , 2008 ▸ ). More specialist applications are provided as part of the CSD-Enterprise suite.”concepts[33]
conceptfederation of databases — not defined in this paper's own words; the anticipated connected future of molecular-structure repositories — “We expect to see a continued push towards the federation of databases containing molecular structures (Sali et al. , 2015 ▸ ) and the development of links between repositories. As these individual repositories grow, methods for searching them will also need to be improved.”concepts[34]
conceptspecialist data centres — not defined in this paper's own words; institutions argued to sustain data quality, completeness, integrity, and availability — “There is now an expectation in science that data is freely, openly and instantly available. With modern informatics techniques this is trivial to achieve, but is often done at the expense of data quality, completeness, integrity and with questionable sustainability. History has shown us that all these are best achieved by specialist data centres, maintaining specialist data repositories. The challenge of finding a universally agreed ‘perfect’ funding model, if one exists, remains (Bruno & Groom, 2014 ▸ ).”concepts[35]
conceptspecialist data repositories — not defined in this paper's own words; curated disciplinary repositories maintained by specialist data centres — “There is now an expectation in science that data is freely, openly and instantly available. With modern informatics techniques this is trivial to achieve, but is often done at the expense of data quality, completeness, integrity and with questionable sustainability. History has shown us that all these are best achieved by specialist data centres, maintaining specialist data repositories. The challenge of finding a universally agreed ‘perfect’ funding model, if one exists, remains (Bruno & Groom, 2014 ▸ ).”concepts[36]

Full text — tagged

This paper is the definitive article describing the creation, maintenance, information content and availability of the Cambridge Structural Databaseterm (CSDterm), the world’s repository of small molecule crystal structuresterm.

Keywords: Cambridge Structural Databaseterm, CIFterm archive, open data, crystal structure database The value of sharing crystal structures The ongoing stewardship of the Cambridge Structural Databaseterm (CSDterm) has been the core activity of the Cambridge Crystallographic Data Centre (CCDC) since its inception in 1965. The CCDC is committed to providing a permanent archive of crystal structures and making these available to all. This non-profit, charitable organization is overseen by an international board of trustees drawn from the community it serves.

The CSDterm contains all published organic and metal–organic small-molecule crystal structures whose structures have been determined using the technique of crystallographyterm. In addition, it acts as a publication vehicle for structure determinations with no accompanying manuscript.

Specifically, the CSDterm contains both X-rayterm and neutron diffraction analysesterm from a single-crystal study or a powder study where cell parameters, atomic coordinates and refinement are reported. To ensure comprehensive coverage of single-crystal data, cell parameters and all available data are included even if no coordinates are available. Powder structures without coordinates are available from the International Centre for Diffraction Data (ICDD; Kabekkodu et al. , 2002 ▸ ).

The CSDterm covers all organic and metal–organic structures, where organic is generally taken to mean a carbon-containing molecule. The CSDterm also contains boron compounds containing one or more B—H or B—OH bond and borazines and ring compounds containing any two of the following elements: N, P, S, Se and Te. Purely inorganic structures that do not fit the criteria above are added to the Inorganic Crystal Structure Database (ICSD; Belsky et al. , 2002 ▸ ) produced by FIZ Karlsruhe or the Metals Database (CRYSTMET; White et al. , 2002 ▸ ) for metals and alloys. Peptides and polysaccharides of up to 24 residues and mono-, di- and tri-nucleotides are included in the CSDterm, higher oligomers are covered by the Nucleic Acids Database (Coimbatore Narayanan et al. , 2014 ▸ ) with the Protein Data Bank (PDB; Berman et al. , 2000 ▸ ) curating and sharing structural data of larger biological macromolecules. In all cases, these guidelines are relaxed where there is clear scientific merit in including a structure in multiple resources.

The database provides value in two distinct ways. The first simply relates to the aggregation and standardization of structuresterm, which facilitates access to individual entriesterm. This brings value both to the data generators and consumers. A single archive of all structures allows crystallographers to avoid the inadvertent redetermination of structures and provides a mechanism by which they can archive the output of their work at a specialist data centreterm for their own future use. Of course, it also allows for the easy sharing of their work, massively increasing their sphere of influence. Such sharing has always been the norm for the crystallographic community and the use of this worldwide, standard, specialist discipline repository allows individuals to demonstrate their adherence to the new data sharing mandates of various funding bodies. As all entries are subject to both automatic and manual curationterm, they can usually be used without further processingterm. Indeed, one might argue that the financial cost of maintaining such a resource, although significant, is recovered many times over by removing the need for repeated correction by users.

A further vital property of the CSDterm is its comprehensive and up-to-date nature. As it represents the complete record of published structures and is updated within a few moments of a new publication, users can have confidence that there are no published structures of relevance of which they are unaware.

The second distinct benefit of the database comes from the study of the collection of entries. This was perhaps best articulated by the founder of the CSDterm, Dr Olga Kennard, who, recounting a discussion with JD Bernal commented, ‘We had a passionate belief that the collective use of data would lead to the discovery of new knowledge which transcends the results of individual experiments’ (Kennard, 1997 ▸ ). The motivations behind the determination of crystal structures do differ, the most common probably being the confirmation of a molecule’s chemical identityterm·var. However, the use they are put to once in the database usually bears no similarity to these motivations. Two illustrative examples are perhaps the establishment of the ability of C—H groups to act as hydrogen-bond donors (Taylor & Kennard, 1984 ▸ ) and ‘structure correlation’ – the linking of three-dimensional geometry to reaction pathways (Bürgi & Dunitz, 1983 ▸ , 1997 ▸ ).

This ‘new knowledge’ relates primarily to the geometry of molecules and the interactions they make. A knowledge of these factors underpins huge areas of both fundamental and applied science (Wong et al. , 2010 ▸ ). They form the basis of our understanding of the energetics of molecular conformation – from bond lengths and angles, through to torsional preferences. They also teach us about the fundamentals of molecular recognition, be it small molecules interacting with small molecules in a lattice or with a protein. Such appreciation is vital for materials sciences and pharmaceutical research and development. Perhaps the most quantitative assessment that can be made of the value of the resource is that the previous published description of the CSDterm (Allen, 2002 ▸ ), which this article supersedes, has received over 10 000 citations.

The Cambridge Structural Databaseterm In 2015 the number of entries in the CSDterm surpassed 800 000 (Fig. 1 ▸ ). This is twice the number of entries in the database less than a decade ago. Comparing statistics based on the database as it was then allows us to see what has changed in the last decade – and what has not. Table 1 ▸ shows that the proportion of structures which are organic or metal–organic structures (which we classify as structures containing a transition metal, lanthanide, actinide, or Al, Ga, In, Tl, Ge, Sn, Pb, Sb, Bi, Po) has remained fairly constant. What has changed is the complexity of the structures being published: the average number of atoms per structure and the average molecular weight have increased (Fig. 2 ▸ ), as has the proportion of structures that are polymeric or that have resolved disorder (Fig. 3 ▸ ).

Growth of the CSDterm since 1972, the red bar shows structures added annually.

Development of the CSDterm over a 10-year timespan from 2006 to 2015 The increase in the average molecular weight of CSDterm entries since 1965, with ( a ) average formula weight per Z ′ of metal–organic structures, ( b ) average molecular weight of heaviest component of metal–organic structures, ( c ) average formula weight per Z ′ of organic structures and ( d ) average molecular weight of the heaviest component of organic structures.

The increase in resolved disorder in CSDterm entries since 1965.

Another significant change is in the number of new structures published per year, which in 2015 was almost twice the number published during 2006. Structures in the CSDterm are associated with over 400 000 articles from 1600 publication sources. Table 2 ▸ shows the top 20 publication sources currently represented in the CSDterm; these account for 67% of the entries in the database.

Publication sources for CSDterm entries The increase in volume and complexity of structures deposited into the CSDterm over the past decade has presented both administrative and scientific challenges. This article will describe how these have been addressed, but to do this we must first look back at the journey that led us to where we are today.

The development of the CSDterm What today we call the Cambridge Structural Databaseterm began life as ‘a computer-based file containing both bibliographic information and numerical data abstracted from the literature and relevant to molecular crystal structures, as obtained by diffraction methods’ (Kennard & Watson, 1970 b ▸ ). Work compiling this file began in 1965 and contents were made available through the series of printed volumes, ‘ Molecular Structures and Dimensions ’ (Kennard & Watson, 1970 a ▸ ). Over time the file developed into a more structured database and the software used to generate and check data evolved into interactive applications enabling chemical searching and analysis of three-dimensional structural data. Together these provided the foundations for the rich suite of CSD-based applications available today.

Just as the database and software evolved so too did the way in which crystal structure data is communicated. In the early days atomic coordinates were published as tables in scientific articles. These were transcribed manually for inclusion in data files and databases. In the 1990s, a file format called CIFterm (now the Crystallographic Information Frameworkterm) was proposed as a standard for the interchange of crystallographic data. CIFterm is now ubiquitously used to capture the results of a diffraction experiment and enables streamlined publication of results alongside journal articles and in data repositories (Hall et al. , 1991 ▸ ; Brown & McMahon, 2002 ▸ ; Bernstein et al. , 2016 ▸ ; Hall & McMahon, 2016 ▸ ).

Advances in science and technology have enabled many of the processes involved in generating and maintaining a database to be automated. However, in order to ensure accuracy and quality, the oversight and input of expert scientists remains as much a requirement today as it did in the early days. Of particular importance is the assignment of rich metadata, including the chemical identityterm·var of the substance studied during the diffraction experiment. Without this, the ease with which the data can be reused is greatly limited. Subsequent sections of this paper describe our informatics systems that combine technology and expertise in a way that minimizes the cost of curation without compromising on the reusability of the data.

The increasing digital availability of the basic results of a diffraction experiment has also had an impact on the overall nature of the CSDterm. Originally, it might best have been described as a secondary resource of data abstracted from the literature. Today, new source data files are almost always deposited directly into the CSDterm making it much more of a primary data resourceterm. Indeed, many structures are published only and directly through the CSDterm as CSD Communicationsterm (previously known as Private Communications). Whilst the CCDC has never charged for deposition, curation, archiving or for access to the primary source data files, early versions of the CCDC’s informatics systems did have technical barriers that hindered access to structures by researchers. Here, we describe developments that ensure entries from the CSDterm are made available to the widest possible community through CCDC services and third party resources.

Deposition and retrieval of data The method for deposition into the CSDterm has evolved since the advent of the CIFterm format in the 1990s, when email depositions dominated. In 2009 the CCDC launched an online web-based tool which is now the main route for deposition. In 2015, 90% of structures were deposited with the CCDC prior to publication and 85% were submitted through this service. A key benefit of this early deposition is that at this point the crystallographer who generated the data is likely to be the depositor and be in a position both to provide the richest data and to respond to any issues most effectively.

During deposition, the CIF syntaxterm·var is automatically checked based on the checks in enCIFer (Allen et al. , 2004 ▸ ) and the depositor is required to fix any issues before continuing with the deposition process. Depositors are strongly encouraged to deposit structure factors in line with the IUCr’s publication standards for crystal structures (IUCr, 2011 ▸ ). The embedding of reflection data into the CIFterm by structure refinement programs such as SHELXL (Sheldrick, 2015 ▸ ) greatly simplifies this process. As a consequence the amount of reflection data stored at CCDC has increased significantly and the majority of new depositions now include this data.

In 2015 the checkCIF / PLATON service (Spek, 2009 ▸ ) was integrated into the CCDC’s deposition process (Fig. 4 ▸ ). This allows the researcher to generate validation reports and embed validation responses into the ‘structure of record’ during deposition to the CSDterm. Reviewers and publishers can read the checkCIF report alongside the deposited data to aid peer review of submitted papers.

The validation step of the CSDterm Deposition process showing the integration with checkCIF (CCDC, 2016 b ▸ ).

During deposition, key metadata are extracted from the CIFterm and presented to the researcher giving them the opportunity to check and further enhance the data that is shared through the CSDterm. To aid with this process a three-dimensional representation of the structure is displayed to the depositor using JSmol (Hanson et al. , 2013 ▸ ).

On deposition, each dataset is assigned an accession identifierterm referred to as a ‘CCDC numberterm’ in the format CCDC 1234567 (older entries have six digits). This uniquely identifies the data associated with a particular structure determination and persists for the lifetime of the dataset. CCDC numbersterm are communicated to the depositor once it has been confirmed that the dataset is not a duplicate submission, usually within seconds of deposition. Should the data require further investigation, CCDC Deposition Coordinators address any problems, minimizing the delay in providing the CCDC numberterm to the depositor. A reference code (known as a CSD refcodeterm and in the format ABCDEF) is assigned to structures as they are indexed into the CSDterm itself. Where possible, determinations of the same substance are assigned into a CSD refcodeterm family (with a CSD refcodeterm format of ABCDEF01). These codes are used as a common way of referring to structures extracted from the CSD-Systemterm.

CCDC numbersterm are used in manuscripts to indicate the location of the data that supports results described in that article. They are used as the basis for providing links directly to the structure from within the article when published. Prior to publication, data is stored in a confidential data archive and is only available to referees and publishers during the peer review process. Providing access to the structure of record, reflection data and validation reports helps ensure the accuracy and integrity of published science. Authors and depositors are able to revise data stored at CCDC up to the point of publication and retain a consistent CCDC numberterm. About one quarter of all submissions are revised during the deposition process. In 2015 alone, over 67 465 unique CCDC numbersterm were assigned and data was deposited by over 10 000 unique depositors.

For structures associated with a journal article, computational workflows and processes with the major publishers handle the flow of data during the publication process. The publication of an article referencing a dataset results in the immediate public release of the corresponding structure. This is automatically triggered by feeds from the publisher or, failing that, by the identification of a manuscript with a CCDC numberterm. Structures deposited with the intention of accompanying a publication are held securely in trust for a period of 1 year. If no publication is identified within that year, authors are contacted to confirm that the structure should be published as a CSD Communicationterm. If the structure is still intended for another publication then the embargo period is extended for another year.

A structure published as a CSD Communicationterm is freely available through the CSDterm within seconds of CCDC numberterm assignment. As the general appreciation of the value of data sharing has increased so has the popularity of publishing structures directly through the CSDterm. In fact, although only 2% of structures in the CSDterm are CSD Communicationsterm, 2015 saw over 4400 CSD Communicationsterm published, making it likely that this will soon become the most popular way in which to publish crystal structures.

At the point of publication, entries are available to anyone through free CSD-Community web services. These services allow anyone to access published structures either via the CCDC website or by following links from other resources. They provide an interactive visualization of the three-dimensional structure through JSmol (Hanson et al. , 2013 ▸ ), a two-dimensional chemical diagram and key metadata associated with the entry (Fig. 5 ▸ ). Individual data files can also be downloaded and used by anyone wishing to investigate or explore the entries in more detail.

Screenshot of the CSDterm community Access Structures results page. This is the ‘landing page’ for many referring services based, for example on DOIterm or CCDC numberterm (CCDC, 2016 a ▸ ).

The creation of the CSDterm Processing entries Due to the rising number of structures, depositions and transactions a new processing systemterm, CSD-Xpediteterm , has been developed, automating most informatics processes required to manage depositions and process crystal structures into entries in the CSDterm. Fig. 6 ▸ shows how data flows through the system and where various users interact with the system.

An illustration of the flow from sample to CSDterm entry. Solid lines are the elements of the process involving the CCDC. A crystalline sample ( a ) is put on a diffractometer ( b ) for an experiment that ultimately results in the experimentally determined coordinates of atoms in the crystal that are captured in a CIFterm file ( c ). The CIFterm file is deposited with the CCDC ( d ) and may be associated with a scientific article ( e ). On publication of the article, the structure is validated and enriched by CCDC editorial software and staff ( f ) to create an entry in the CSDterm ( g ). Structures are publicly accessible through CSDterm community services such as Access Structures ( h ). Standard identifiers such as DOIsterm and InChIsterm facilitate links between articles, structures and other resources. Software developed by the CCDC ( i ) enables the knowledge embedded in the CSDterm to be applied to a range of scientific problems including aiding in the determination and refinement of future structures through free resources such as CellCheckCSDterm ( j ). The flow of data is largely represented using lamotrigine (Sridhar & Ravikumar, 2009 ▸ ), the 500 000th entry in the CSDterm, CSD refcodeterm: EFEMUX01; CCDC numberterm: CCDC 749719; DOIterm: 10.5517/cct54hl. Image ( a ) is a crystal of p -aminobenzoic acid (Sullivan & Davey, 2015 ▸ ) courtesy of Rachel Sullivan and Roger Davey, University of Manchester, CSD refcodeterm: AMBNAC10; CCDC 983122; DOIterm: 10.5517/cc1200mq. Image ( b ) courtesy of Andrew Bond, University of Cambridge.

CSD-Xpediteterm is built around Microsoft Dynamics CRM (Microsoft, 2016 a ▸ ) and SharePoint (Microsoft, 2016 b ▸ ). The system is designed to store all data in one unified system and many automated processes have been incorporated to reduce the number of manual interventions required to process entries through the CSDterm creation process. This scalable system allows for the ever increasing rate of deposition, and its modular nature allows extra functionality to be incorporated with minimal disruption. One important aspect of CSD-Xpediteterm is the use of a new extensible file format for internal storage of CSDterm entries. This format allows a fuller representation of the underlying data than in previous formats (BCCAB, ASER). Databases for use in the CSD-Systemterm are created from this master database.

A key component in CSD-Xpediteterm is CSD-Editorterm , an interactive tool for processing structures such that they can enter into the CSDterm. This uses many of the visualization and menu options available in the CSD-Systemterm program Mercury (Macrae et al. , 2006 ▸ ). Deposited CIFsterm can be easily viewed alongside any associated article. Errors and warnings are displayed for each structure to allow expert structural chemistry editors to concentrate their efforts on the challenging scientific parts of the process.

Assignment of chemical identityterm·var An important aspect of creating the CSDterm is the assignment of the chemical representation to structures so that these can be reliably searched and analysed using structure-based methodologies such as substructure search. In the past, the chemical representation was assigned by scientific editors visualizing the structure and consulting any associated article. A program called deCIFerterm (BrunopersName et al. , 2011 ▸ ) now helps automatically assign ‘chemistry’ to structures.

DeCIFerterm uses the information already in the CSDterm to interpret a new structure and add a chemical representation to the atomic coordinates in the CIFterm. The stages involved in the automatic creation of the CSDterm entry include resolution of any disorder in the structure, detection of bonds and determination of bond types and charges. Assignment of bond types, charges and the inference of missing H atoms uses a probabilistic Bayesian approachterm which allows all the existing entries in the CSDterm to be used to help assign the chemistry to new entries.

The final step of the deCIFerterm process is to validate the assignment by looking at improbable featuresterm such as unprecedented bond types, unlikely oxidation states, unlikely metal–metal bonds and other empirical indicators such as non-planar double bonds. From this analysis a reliability scoreterm is calculated which highlights any possible errors in the representation of the structure which can then be reviewed and corrected during CSDterm entry validation. It is important to note that this reliability scoreterm simply gives a confidence value to the automatic treatment of the entry; this is not necessarily related to the ‘quality’ of the structure.

With ‘chemistry’ assigned, an internally developed diagram generation algorithm takes the three-dimensional coordinates of an entry and ‘flattens’ these, producing a two-dimensional representation with minimal overlaps of atoms and bonds. This procedure also takes advantage of the many two-dimensional diagrams already available for related entries, often drawn previously by scientific editors, which increases both the quality and consistency of diagrams.

Although the automatic deCIFerterm process is used as much as possible structures are still manually viewed by expert structural chemists before they are added to the CSDterm.

Using the CSDterm Identifying and linking digital objects At the point a structure can be made public a Digital Object Identifierterm (DOIterm) is associated with the deposited dataset (580 000 are currently available). This DOIterm allows a third party to link to a structure summary page without needing to know details of linking services provided by the CCDC. In addition the DOIterm provides the basis for a more formal citation of the deposited dataset in line with the spirit of the Joint Declaration of Data Citation Principles (Data Citation Synthesis Group, 2014 ▸ ). The metadata provided to the DOIterm registration agency DataCiteterm (DataCiteterm, 2016 ▸ ) indicates key elements of a dataset citation including contributor, a title and publication year. This enables citation of the dataset independently of the associated article and thus helps ensure recognition of the specific contribution made by the crystallographer, who may not always be included in the article’s author list.

Metadata submitted by the CCDC to DataCiteterm when generating a DOIterm is openly accessible and facilitates interoperability with third party systems to improve the discoverability of data. Examples where other parties have taken advantage of this include the Thomson Reuters Data Citation Index (Thomson Reuters, 2016 ▸ ), where data is currently available for 530 000 CSDterm entries, and the prototype RDA/WDS Data-Literature Interlinking service (Burton et al. , 2015 ▸ ). In order to support this interoperability, additional metadata items are made public including the DOIsterm of associated articles and the chemical name of the substance studied.

Discoverability of data by chemists and biologists is enabled by establishing links to datasets from services such as ChemSpider (Pence & Williams, 2010 ▸ ) and PubChem (Bolton et al. , 2008 ▸ ). The overlap between these resources and the CSDterm is identified by taking advantage of the International Chemical Identifier Standardterm (InChIterm) which provides a unique and canonical representation of the chemical substance studiedterm (Heller et al. , 2015 ▸ ). Links from ChemSpider and PubChem have been established for over 52 000 compounds that could be reliably identified using InChIsterm as being in common between these resources and the CSDterm. InChIsterm have also been used to identify correspondences between CSDterm entries and ligands bound to macromolecules in structures archived in the Protein Data Bank (PDB; Berman et al. , 2000 ▸ ). This lookup is enabled by a CCDC web service that identifies the best-representative CSDterm entry for a given molecule and provides access to its coordinates. This is a particularly useful resource for structural biologists refining or investigating the structure of a protein ligand. Representative structures are freely available for approximately 1500 PDB ligands in a Chemical Component Model file provided by the PDB (wwPDB, 2015 ▸ ).

Accessibility and efficiency The CSD community web servicesterm detailed earlier provide free access to the entire collection of structures. As well as these services there are a number of other avenues to explore and exploit the data ranging from free lookup tools such as CellCheckCSDterm (Wood, 2011 ▸ ) to advanced search, analysis and validation tools in the CSD-Systemterm (BrunopersName et al. , 1997 ▸ , 2002 ▸ , 2004 ▸ ; Macrae et al. , 2008 ▸ ). More specialist applications are provided as part of the CSD-Enterprise suite.

CellCheckCSDterm is an automated toolterm which uses the data from the CSDterm to check unit cells during data collection and can be used to check that the structure is novel rather than the starting material, a by-product or another previously determined structure.

The CSD-Systemterm enables knowledge to be gained from the collection of data through powerful two-dimensional/three-dimensional search capabilities, extensive geometry analysis tools, inter- and intramolecular interaction analysis, generation of high impact graphics and the ability to delve further into the data using a Python-based applications programming interface.

Visualizing three-dimensional structural data can be a powerful way to teach chemistry concepts (Battle & Allen, 2012 ▸ ; Henderson et al. , 2011 ▸ ; Battle et al. , 2011 ▸ , and references therein). Through the CSDterm community services, educators worldwide are able to access all 800 000 entries in the CSDterm, but the enormity of the database means it is not a simple process to identify the key structures that are most appropriate for the class. To this end a CSDterm teaching database (Battle et al. , 2010 ▸ ) has been compiled. This is a collection of around 700 carefully selected crystal structures, targeted to represent a diverse range of chemistry and to allow teachers to demonstrate key chemical concepts and principles.

To support the free archiving services and free universal access to all individual structures, users of client installed software ( i.e. the CSD-Systemterm) are asked to contribute to the running costs of the CCDC. In many cases this is done at a national level, allowing unfettered use, with the modest contribution levels set according to their economic status. These contributions fund the maintenance of the CSDterm, whereas larger contributions from for-profit users of the system fund its ongoing development (BrunopersName & GroompersName, 2014 ▸ ).

Future Since its very inception, X-rayterm and neutron crystallographyterm has been the method of choice for the elucidation of the full three-dimensional structure of molecules (Wilkins, 2013 ▸ ). However, techniques involving electron diffraction (see for example Yun et al. , 2015 ▸ ), atomic force microscopy (see for example Gross et al. , 2009 ▸ ), free electron lasers (Barty et al. , 2013 ▸ ) and NMR crystallographyterm (Baias et al. , 2013 ▸ ) have already shown their potential. In some cases, this will involve the capture of molecular structures not in a crystalline lattice, so systems have been designed to allow for this. Developments are also already underway to allow effective treatment of predicted crystal structures (Reilly et al. , 2016 ▸ ).

Although the CSDterm contains all published crystal structures, it has been estimated that only 15% of structures determined are published (UK National Crystallographyterm Service, 2014 ▸ ). Automatic links in software used during structure determination (Dolomanov et al. , 2009 ▸ ), the ease with which structures can be deposited, attribution of credit in the form of a DOIterm and continued demonstration of the value to science of depositing crystal structures (Berman et al. , 2015 ▸ ) may help close this gap.

We expect to see a continued push towards the federation of databasesterm containing molecular structures (Sali et al. , 2015 ▸ ) and the development of links between repositories. As these individual repositories grow, methods for searching them will also need to be improved.

There is now an expectation in science that data is freely, openly and instantly available. With modern informatics techniques this is trivial to achieve, but is often done at the expense of data quality, completeness, integrity and with questionable sustainability. History has shown us that all these are best achieved by specialist data centresterm, maintaining specialist data repositoriesterm. The challenge of finding a universally agreed ‘perfect’ funding model, if one exists, remains (BrunopersName & GroompersName, 2014 ▸ ).

The authors of this article were unable to write a concluding paragraph that expresses the continued value of the CSDterm better than those who had the inspiration to found it, Olga Kennard and J. D. Bernal (1901–1971). We therefore conclude with an excerpt from a historical memoir presented in 1995. ‘ I think that the great ocean of truth is still in front of us and that we will continue to discover new aspects of this truth. Some of them will be discovered through the insight of outstanding individuals and some through the insight which Bernal predicted could be gained by more ordinary mortals through the analysis and transformation of the pebbles of information which have accumulated over the past decades. We have the tools and resources to do this and Bernal’s inspiration is still with us ’ (Kennard, 1995 ▸ ).

Acknowledgments It is a pleasure to acknowledge the work of all CCDC staff and trustees, past and present, who have contributed so much to the creation, maintenance and evolution of the CSDterm. The current CSDterm team includes Rebecca Anderson, Philip Andrews, Marjia Billah, Caroline Davies, Yu Gan, Stephen Holgate, Kamila Orzechowska, Clare Tovee and Seth Wiggin. In addition to referenced software, they use systems currently written, maintained and managed by Stewart Adcock, Jason Cole, Neil Feeder, Humberto Gloria, Coran Hoskin, Oliver Korb, Abel Lovas, Clare Macrae, Stefano Mazzucco, Simon Miller, Darryl Morris, Zenno Neagu, Florian Piesche, Michael Platings, Thomas Pleasance, Fergus Prone, Christin Schärfer, Greg Shields, James Smaldon, Richard Sykes, Robin Taylor, Beth Thomas, Matthew Towler and Pete Wood. The computational infrastructure of the CCDC is maintained by Matthew Burling, Jordan Layfield and Keith Taylor. Finance, business, administration and management of the CCDC are performed by Maggie Brown, Paul Davie, Daniel Giddings, Vanda Sewell, Margaux Toral, Bing-Bing Waterman and Jian Yang. Users of the database are helped by the scientific and support activities of Sarah Amery, David Bardwell, Maria Brandl, Mathew Bryant, Simon Cottrell, Levente Csurka, Erin Davis, Ilenia Giangreco, Scott Gothe, Alicia Higueruelo, Paul Hudson, Timo Krotzky, Steve Maginn, Andrew Maloney, Patrick McCabe, Tom Parry, Elna Pidcock, Juliette Pradon, Louisa Quegan, Chris Radoux, Murray Read, Anthony Reilly, Ghazala Sadiq, Paul Sanschagrin, Michele Santullo, Amy Sarjeant, Shyam Vyas and Robert Wilkinson. Guiding the direction of the CCDC on behalf of the communities it serves are trustees Paul Raithby, Judith Currano, Christer Aakeröy, David Martinsen, John Overington and Martin Stahl. Maintaining the completeness, currency and efficient population of the CSDterm is only possible because of the assistance and support of the many scientific publishers we work with. Finally, we would particularly like to thank the 317 991 authors of the papers describing the crystal structures in the CSDterm, whose approach to data sharing is truly exemplary.

References

Interpretive notes.
• Highlighted spans are tagged entities: persName (author), title (venue), placeName (place), date (year), and term (a key concept: domain / ai / method).
• Eponymous or retrospective terms (e.g. “Nash equilibrium”, “segregation”) do not appear in the original source and so are not highlighted — a deliberate choice (see Report 4).
• Some entries begin with a word in capitals. EVIDENCE CORRECTED means the finding still stands and it was our quotation that was wrong — it has been replaced. RETRACTED means the entry itself was wrong; it stays on the page, marked, with its reason, rather than being deleted. WITHDRAWN AND REPLACED means a corrected version of it appears in the entries below. Each mark carries its own reason, because the reasons differ from one entry to the next. An entry we have WITHDRAWN is removed from this page and kept in the record, with its reason. A line reading Audit and a date is a later re-reading of the entry against its whole sentence: DOUBTFUL means it was kept, and the reason says what is loose about it.
• Quotation marks inside the tagger's sentence. Where a tag is the tagger's own sentence, a phrase in quotation marks inside it is borrowed from the paper — the tagger's wording around it is not. The paper's own sentence is the evidence quote shown on the same row, so the borrowing can be checked against the thing borrowed from without leaving this page. Nothing inside quotation marks anywhere on this page is ours: if it is quoted, it is the paper's. Whose words a concept carries. A concept entry records a term the paper itself uses, so the term, the sense given for it, and its evidence are the paper's own words, quoted where the paper fixes them. This is the opposite of a sense entry in the tag table, which is the tagger's sentence about how the paper uses a term. Where a concept entry could not be grounded in the paper's words, the entry says so on its own row.
• Counts: 1 work · 1 date.
Meta-Tagging Project · record groom2016-cambridge-structural-database.
© 2026 Shir Sivroni