Crystallography Open Database – an open-access collection of crystal structures

Crystallography Open Database – an open-access collection of crystal structures S. Gražulis Daniel Chateigner Robert T. Downs Alex Yokochi M. Quirós Luca Lutterotti E. Manakova Justas Butkus Peter Moeck A. Le Bail

Tag layer of the DHSS Meta-Tagging Project. The article text is reproduced from the staged copy named in sourceDesc and is not altered; the tags are held separately in standOff so that overlapping and nested spans are preserved.

Crystallography Open Database – an open-access collection of crystal structures 2009 Journal of Applied Crystallography 10.1107/s0021889809016690 grazulis2009-crystallography-open-database-an-o grazulis2009-crystallography-open-database-an-o.txt, the project's staged copy f32be7cb1afd5f2ac23e578bac82fa7bb812ecc2 cc-by materials-science

the paper's own words

the tag layer; a sentence written by the tagger about the paper, always anchored to a quotation but not itself a quotation

0 spans located by searching the stored text for the quote's exact words, allowing any whitespace between them an open-access collection of published crystal structures, and the deposition machinery - identifiers, versioning, duplicate detection, validation - that lets a community keep one Quotation withheld because this record is not cleared for redistribution. how to gather structural data that was historically split across three specialised databases into one searchable open collection, and keep it correct while it grows Quotation withheld because this record is not cleared for redistribution. the COD itself at ~80,000 entries with near-full coverage of IUCr publications, plus a deposition pipeline that is described in enough detail to be re-implemented: permanent identifiers, Subversion revisions written into the file header, a four-condition duplicate rule ending in a human Quotation withheld because this record is not cleared for redistribution. method Quotation withheld because this record is not cleared for redistribution. a scientific record split across databases by historical accident rather than by anything about the objects, and the identity problems that appear when the pieces are brought back together Quotation withheld because this record is not cleared for redistribution. system-or-tool-building system-or-tool-building Quotation withheld because this record is not cleared for redistribution. a permanent identifier is what makes citation possible and therefore must never change; a file that can never change cannot be corrected. The COD keeps the number fixed and versions the file underneath it, then writes both the revision and the repository address INTO the file, so that a copy taken away still says which state of which record it is two numbers because one cannot do both jobs Quotation withheld because this record is not cleared for redistribution. two measurements can agree to 0.5 A and 1.2 degrees and still be separate scientific facts; the COD only calls them duplicates if they also came from the same paper and the same pressure and temperature, and even then a person decides. The object does not determine its own key identity that is partly bibliographic Quotation withheld because this record is not cleared for redistribution. each CIF is given a full copy of the available bibliographic information rather than a pointer to it, so that the file can be processed and stored independently - the cost is duplication, and the benefit is that a file removed from the database is still interpretable a record made self-contained on purpose Quotation withheld because this record is not cleared for redistribution. a structure on hold is present in the database with its cell constants, composition, symmetry and authorship visible and its COD number issued, while the coordinates - the part that could be scooped - are withheld until publication or one year, whichever comes first an embargo that splits the record rather than hiding it Quotation withheld because this record is not cleared for redistribution. the main functional limitation is the absence of a substructure search engine, because the chemical connectivity is not represented in the stored data Quotation withheld because this record is not cleared for redistribution. chemical and hydrogen bond information is preserved only if the original file carried it, and is never inferred from the structure Quotation withheld because this record is not cleared for redistribution. the deposition procedure has two manual steps - number assignment and error checking - and the number of determined structures is growing exponentially Quotation withheld because this record is not cleared for redistribution. bibliographic information may fall back as far as the names of the directories the files arrived in Quotation withheld because this record is not cleared for redistribution. one centralised repository is the authoritative source, and decentralisation is described as possible rather than done Quotation withheld because this record is not cleared for redistribution. persistent URLs for deposited data are planned rather than in place Quotation withheld because this record is not cleared for redistribution. software version control - Subversion, revision numbers, change logs, checkout of prior states - carried from programming practice into the curation of a scientific data record Quotation withheld because this record is not cleared for redistribution. bulk distribution protocols from system administration - rsync, tarballs, anonymous repository checkout - used so that the database can be taken away entire rather than queried a record at a time Quotation withheld because this record is not cleared for redistribution. every change to every file recorded in a central Subversion repository, with the revision and repository address written into the file header by software eliminates: a silent edit - without the revision in the file, a corrected record and its predecessor are indistinguishable once either has been copied away · outcome: held Quotation withheld because this record is not cleared for redistribution. a four-condition duplicate scan - cell constants, angles, formula and same paper, plus pressure and temperature when given - with every candidate flagged for a person eliminates: numeric closeness alone deciding identity, and an automatic merge removing a second independent determination of the same compound · outcome: held Quotation withheld because this record is not cleared for redistribution. the whole collection, the database dumps and the web scripts published for anonymous checkout, so that any user can reconstitute the database and its logs locally eliminates: the maintainers being the only witnesses to what the database contains and what was changed in it · outcome: held Quotation withheld because this record is not cleared for redistribution. each CIF checked for syntax by TWO independent parsers - the IUCr 'vcif' tool and a COD-written Perl parser eliminates: syntax errors in a deposited CIF, and the use of two independent parsers means an error has to be missed twice to get through · outcome (stated by the tagger, no quotation): held Quotation withheld because this record is not cleared for redistribution. validation that required data items - cell parameters, symmetry, bibliography - are present before deposition eliminates: depositions missing cell parameters, symmetry or bibliography - and note the verb: the CIFs 'can be' validated. The capability is described; the paper does not say it was run · outcome (stated by the tagger, no quotation): absent-declared-unreported Quotation withheld because this record is not cleared for redistribution. a fallback chain for bibliographic information ending, as a last resort, at the directory names the files arrived in eliminates: unattributed structures - though the paper is candid that the last resort is the directory name, which is the weakest link in the chain it describes · outcome (stated by the tagger, no quotation): held Quotation withheld because this record is not cleared for redistribution. each CIF given a full copy of the bibliographic information rather than a reference to it, so it can be processed independently eliminates: a record that cannot be interpreted once separated from the collection it arrived in · outcome (stated by the tagger, no quotation): held Quotation withheld because this record is not cleared for redistribution. an on-hold deposition status releasing metadata and a COD number while withholding coordinates until publication or one year eliminates: depositors withholding structures altogether for fear of being scooped, at the cost of a delay the policy bounds · outcome (stated by the tagger, no quotation): held Quotation withheld because this record is not cleared for redistribution. chemical and hydrogen bond information preserved from the source file and never inferred from the structure eliminates: a structure that cannot be identified as the compound it is, and bond information silently re-derived from geometry rather than taken from the depositor · outcome (stated by the tagger, no quotation): held Quotation withheld because this record is not cleared for redistribution. the claim that several dozen contributors are enough to provide qualified peer review for incoming structures eliminates: unreviewed structures reaching the database - conditional on adequate software and on enough volunteers, neither of which the paper claims to have · outcome (stated by the tagger, no quotation): absent-declared-unreported Quotation withheld because this record is not cleared for redistribution. COD COD Quotation withheld because this record is not cleared for redistribution. duplicates duplicates Quotation withheld because this record is not cleared for redistribution. revision number revision number Quotation withheld because this record is not cleared for redistribution. cell constants cell constants Quotation withheld because this record is not cleared for redistribution. CIF CIF Quotation withheld because this record is not cleared for redistribution. substructure substructure Quotation withheld because this record is not cleared for redistribution. open-access open-access Quotation withheld because this record is not cleared for redistribution. mirrors mirrors Quotation withheld because this record is not cleared for redistribution. fragmentation fragmentation Quotation withheld because this record is not cleared for redistribution. two entries being the SAME STRUCTURE - AS THE COD DEFINES IT: cell constants within 0.5 A, angles within 1.2 degrees, formulas matching, same paper, same pressure and temperature, and then a human agrees Quotation withheld because this record is not cleared for redistribution. two entries being the SAME STRUCTURE - AS THE COD DELIBERATELY DOES NOT: a redetermination at higher precision or under different conditions gets a NEW identifier rather than replacing the old one Quotation withheld because this record is not cleared for redistribution. a COD record's identity - THE COD NUMBER: permanent, assigned once, and never changed even when the file it names is corrected Quotation withheld because this record is not cleared for redistribution. a COD record's identity - THE REVISION NUMBER: changes on every edit, is written into the file header by software, and is what makes any previous state recoverable Quotation withheld because this record is not cleared for redistribution. a deposited structure's visibility - UNDER THE ON-HOLD FLAG: cell constants, composition, symmetry and authorship are public and searchable Quotation withheld because this record is not cleared for redistribution. a deposited structure's visibility - THE COORDINATES UNDER THE SAME FLAG: withheld until publication, or release by the authors, or one year Quotation withheld because this record is not cleared for redistribution. critiques → the historical three-way split of structural data across inorganic, metals and organic databases, which reflected how laboratories used to specialise rather than anything about the compounds Quotation withheld because this record is not cleared for redistribution. extends → the IUCr's 2007 decision to make CIFs from its journals freely available to all databases, which is what let the COD download new structures routinely Quotation withheld because this record is not cleared for redistribution. qualifies → its own functionality - without a representation of chemical connectivity there is no substructure search, which the authors call the main limitation Quotation withheld because this record is not cleared for redistribution. qualifies → its own architecture - one centralised authoritative repository today, with decentralisation named as possible rather than implemented Quotation withheld because this record is not cleared for redistribution. A structure is a potential duplicate if its cell constants and angles fall within set tolerances, its formula matches and both come from the same paper; all potential duplicates are reviewed manually. A structure is a potential duplicate if its cell constants and angles fall within set tolerances, its formula matches and both come from the same paper; all potential duplicates are reviewed manually. Quotation withheld because this record is not cleared for redistribution. a redetermination of the same compound is given a new identifier rather than replacing the old entry a redetermination of the same compound is given a new identifier rather than replacing the old entry Quotation withheld because this record is not cleared for redistribution. a permanent identifier and a mutable file need a second number, and the COD writes the revision and repository address into the file header a permanent identifier and a mutable file need a second number, and the COD writes the revision and repository address into the file header Quotation withheld because this record is not cleared for redistribution. chemical and hydrogen bonding is preserved only when the source file carried it and is never inferred from the structure chemical and hydrogen bonding is preserved only when the source file carried it and is never inferred from the structure Quotation withheld because this record is not cleared for redistribution. An 'on hold' deposit shows cell constants, composition, symmetry and authorship and receives a COD number. An 'on hold' deposit shows cell constants, composition, symmetry and authorship and receives a COD number. Quotation withheld because this record is not cleared for redistribution. The lack of a substructure search engine is one of the COD's main functional limitations. The lack of a substructure search engine is one of the COD's main functional limitations. Quotation withheld because this record is not cleared for redistribution. the whole collection, its dumps and its scripts are available for anonymous checkout, so any user can reconstitute the database and read its deposition logs the whole collection, its dumps and its scripts are available for anonymous checkout, so any user can reconstitute the database and read its deposition logs Quotation withheld because this record is not cleared for redistribution. cell constants within 0.5 A and angles within 1.2 degrees of an existing entry, matching summary formulas, both structures published in the same paper, and identical pressure and temperature where specified - then reviewed manually duplicates Quotation withheld because this record is not cleared for redistribution. a common group of atoms chemically linked in the same way - the paper's definition of chemical similarity, and the thing the database cannot search for substructure Quotation withheld because this record is not cleared for redistribution. angstroms - the cell-constant tolerance inside which two entries become duplicate candidates 0.5 Quotation withheld because this record is not cleared for redistribution. degrees - the cell-angle tolerance in the same rule 1.2 Quotation withheld because this record is not cleared for redistribution. the length of a COD identifier, assigned once and never changed seven-digit Quotation withheld because this record is not cleared for redistribution. revision number revision number Quotation withheld because this record is not cleared for redistribution.

The full text of this article is not redistributable, so it is not included here. The evidence sentences are withheld with it; the tag layer and its grounding status remain. The article itself is at https://doi.org/10.1107/s0021889809016690.