Automatic Chemical Design Using a Data-Driven Continuous Representation of Molecules

Automatic Chemical Design Using a Data-Driven Continuous Representation of Molecules Rafael Gómez-Bombarelli Jennifer N. Wei David Duvenaud et al.

Tag layer of the DHSS Meta-Tagging Project. The article text is reproduced from the staged copy named in sourceDesc and is not altered; the tags are held separately in standOff so that overlapping and nested spans are preserved.

Automatic Chemical Design Using a Data-Driven Continuous Representation of Molecules 2018 ACS Central Science 10.1021/acscentsci.7b00572 1610.02415 gomez-bombarelli2018-molecular-design gomez-bombarelli2018-molecular-design.txt, the project's staged copy 5bdff675ec5f9fea17eeb21ab783945f5dffd6b1 arxiv-open chemistry

the paper's own words

the tag layer; a sentence written by the tagger about the paper, always anchored to a quotation but not itself a quotation

0 spans located by searching the stored text for the quote's exact words, allowing any whitespace between them Chemical space, and how to search it by turning molecules into vectors that can be optimised with gradients. Quotation withheld because this record is not cleared for redistribution. Molecular space is discrete and unstructured, so gradients cannot guide a search through it - can it be made continuous instead? Quotation withheld because this record is not cleared for redistribution. An encoder/decoder/predictor trained jointly, so that molecules become points in a continuous space where optimisation and interpolation are ordinary operations. Quotation withheld because this record is not cleared for redistribution. method Quotation withheld because this record is not cleared for redistribution. automated molecular design in chemical space Quotation withheld because this record is not cleared for redistribution. system-or-tool-building system-or-tool-building Quotation withheld because this record is not cleared for redistribution. mechanisms encoding molecules into a continuous latent space so new molecules are generated by decoding/interpolating vectors (and properties optimized by moving in that space) Quotation withheld because this record is not cleared for redistribution. mechanisms an information bottleneck forces the network to keep only what is statistically salient - the compression is what makes the vector a representation rather than a copy Quotation withheld because this record is not cleared for redistribution. mechanisms the variational penalty fills in the space: adding noise to the encoding forces the decoder to handle points between molecules, so the space has no dead areas Quotation withheld because this record is not cleared for redistribution. mechanisms training the property predictor JOINTLY is what aligns the geometry with the chemistry - otherwise moving in the space would not move the property Quotation withheld because this record is not cleared for redistribution. mechanisms invalid outputs are filtered rather than prevented - the decoder is allowed to be wrong and a cheminformatics toolkit discards the failures Quotation withheld because this record is not cleared for redistribution. optimization in molecular space is hard because the search space is large, discrete, and unstructured what this limits: any claim of coverage over chemical space: Quotation withheld Quotation withheld because this record is not cleared for redistribution. Decoded molecules can be graph-valid but chemically undesirable. what this limits: the generated molecules: THE AUTOENCODER SOMETIMES PRODUCED MOLECULES THAT ARE FORMALLY VALID AS GRAPHS BUT CONTAIN MOIETIES THAT ARE NOT DESIRABLE BECAUSE OF STABILITY OR SYNTHETIC CONSTRAINTS Quotation withheld because this record is not cleared for redistribution. SMILES syntax fragility means the decoder still emits invalid molecules despite the variational constraint. what this limits: the decoder's output: THE CHARACTER-BY-CHARACTER NATURE OF THE SMILES REPRESENTATION AND THE FRAGILITY OF ITS INTERNAL SYNTAX CAN STILL RESULT IN THE OUTPUT OF INVALID strings Quotation withheld because this record is not cleared for redistribution. Requiring the decoder to learn SMILES validity makes the learning problem harder than it needs to be. what this limits: the learning problem as posed: FORCING THE DECODER TO PRODUCE VALID SMILES STRINGS MAKES THE LEARNING PROBLEM UNNECESSARILY HARD SINCE THE DECODER MUST ALSO IMPLICITLY LEARN WHICH STRINGS ARE VALID SMILES Quotation withheld because this record is not cleared for redistribution. Decoding succeeds only near known molecules; random latent points almost always fail. what this limits: any claim about generating novel molecules: THE PROBABILITY OF DECODING FROM A POINT IN LATENT SPACE IS DEPENDENT ON HOW CLOSE THIS POINT IS TO THE LATENT REPRESENTATIONS OF OTHER MOLECULES - a decoding rate of 73-79% NEAR known molecules Quotation withheld because this record is not cleared for redistribution. Property-prediction accuracy is only comparable to graph convolutions for some properties and is not fully optimised. what this limits: the property-prediction comparison: PERFORMANCE FOR ELECTRONIC PROPERTIES ARE SIMILAR TO GRAPH CONVOLUTIONS FOR SOME PROPERTIES, and accuracy COULD BE IMPROVED Quotation withheld because this record is not cleared for redistribution. The graph-output alternative the authors prefer is not yet solvable. what this limits: the graph-output alternative the authors prefer: BUILDING A NEURAL NETWORK WHICH CAN OUTPUT ARBITRARY GRAPHS IS AN OPEN PROBLEM Quotation withheld because this record is not cleared for redistribution. The decoder still emits chemically invalid strings, because SMILES syntax is fragile and the model works character by character. duplicate of: limitations[2] · what this limits: the same scope as limitations[2] - SMILES syntax fragility still yields invalid output Quotation withheld because this record is not cleared for redistribution. Without the variational constraint the latent space contains dead regions that decode to nothing valid. what this limits: the latent space without the variational constraint: IT MAY BE SPARSE AND MAY CONTAIN LARGE DEAD AREAS, WHICH DECODE TO INVALID SMILES STRINGS Quotation withheld because this record is not cleared for redistribution. The choice of string encoding is load-bearing and only partly explained - InChI performed substantially worse and the reason is offered as a presumption. what this limits: every result in the paper, which is conditional on the encoding: WE ALSO TESTED InChI AS AN ALTERNATIVE STRING REPRESENTATION, BUT FOUND IT TO PERFORM SUBSTANTIALLY WORSE THAN SMILES, PRESUMABLY DUE TO A MORE COMPLEX SYNTAX THAT INCLUDES COUNTING AND ARITHMETIC Quotation withheld because this record is not cleared for redistribution. a variational autoencoder mapping molecules (SMILES strings) into a continuous latent space for automatic chemical design Quotation withheld because this record is not cleared for redistribution. the variational autoencoder with recurrent encoder and decoder, taken from Bowman et al.'s work on English sentences and applied to molecules as strings Quotation withheld because this record is not cleared for redistribution. the idea that a low-dimensional continuous code supports interpolation and analogy, imported as the reason a molecular latent space should be navigable Quotation withheld because this record is not cleared for redistribution. only CANONICALISED SMILES strings used for training eliminates: the same molecule appearing as several different strings - SMILES is not unique, so an uncanonicalised set would teach the model that one compound is many, and inflate apparent diversity in the latent space the paper is about · outcome (stated by the tagger, no quotation): held Quotation withheld because this record is not cleared for redistribution. variational autoencoder variational autoencoder Quotation withheld because this record is not cleared for redistribution. latent representation latent representation Quotation withheld because this record is not cleared for redistribution. SMILES SMILES Quotation withheld because this record is not cleared for redistribution. InChI InChI Quotation withheld because this record is not cleared for redistribution. autoencoder autoencoder Quotation withheld because this record is not cleared for redistribution. information bottleneck information bottleneck Quotation withheld because this record is not cleared for redistribution. chemical space chemical space Quotation withheld because this record is not cleared for redistribution. gradient-based optimization gradient-based optimization Quotation withheld because this record is not cleared for redistribution. inverse design challenge inverse design challenge Quotation withheld because this record is not cleared for redistribution. Virtual screening Virtual screening Quotation withheld because this record is not cleared for redistribution. encoder network encoder network Quotation withheld because this record is not cleared for redistribution. decoder network decoder network Quotation withheld because this record is not cleared for redistribution. property prediction task property prediction task Quotation withheld because this record is not cleared for redistribution. latent space latent space Quotation withheld because this record is not cleared for redistribution. dead areas dead areas Quotation withheld because this record is not cleared for redistribution. doubly probabilistic model doubly probabilistic model Quotation withheld because this record is not cleared for redistribution. spherical interpolation spherical interpolation Quotation withheld because this record is not cleared for redistribution. slerp slerp Quotation withheld because this record is not cleared for redistribution. synthetic accessibility score synthetic accessibility score Quotation withheld because this record is not cleared for redistribution. Quantitative Estimation of Drug-likeness Quantitative Estimation of Drug-likeness Quotation withheld because this record is not cleared for redistribution. Gaussian process model Gaussian process model Quotation withheld because this record is not cleared for redistribution. joint training for property prediction joint training for property prediction Quotation withheld because this record is not cleared for redistribution. teacher forcing teacher forcing Quotation withheld because this record is not cleared for redistribution. spherical interpolation → slerp The parenthetical introduces slerp as the short name for spherical interpolation. This is an abbreviation/short-form relation: it is not class membership, containment, identity of two independently named concepts, distinction, contrast, operationalisation, causation, or renaming across fields. Quotation withheld because this record is not cleared for redistribution. representation - DISCRETE: the molecule as a symbol string, which a chemist can read and a machine can turn back into a structure Quotation withheld because this record is not cleared for redistribution. representation - CONTINUOUS: the molecule as a real-valued vector, on which arithmetic and gradients are defined Quotation withheld because this record is not cleared for redistribution. representation - LATENT: specifically the compressed code inside the bottleneck, which is a representation because it discards, not because it depicts Quotation withheld because this record is not cleared for redistribution. representation - as used across images, text, speech and music, where the same low-dimensional continuous code supports interpolation and analogy Quotation withheld because this record is not cleared for redistribution. inverse design challenge - searching chemistry by mapping molecules into continuous neural representations Quotation withheld because this record is not cleared for redistribution. Virtual screening - computationally assaying large libraries and experimentally testing selected leads Quotation withheld because this record is not cleared for redistribution. encoder network - converts each molecular string to a fixed-dimensional vector Quotation withheld because this record is not cleared for redistribution. decoder network - converts vectors back into molecular strings Quotation withheld because this record is not cleared for redistribution. property prediction task - predicts molecular properties from the latent vector during joint training Quotation withheld because this record is not cleared for redistribution. latent space - the encoder's vector space, functioning as a continuous molecular representation Quotation withheld because this record is not cleared for redistribution. dead areas - sparse latent-space regions that decode into invalid SMILES strings Quotation withheld because this record is not cleared for redistribution. doubly probabilistic model - stochastic in both encoding noise and decoder output sampling Quotation withheld because this record is not cleared for redistribution. spherical interpolation - follows a circular arc to remain in high-probability latent-space regions Quotation withheld because this record is not cleared for redistribution. slerp - the paper's short form for spherical interpolation Quotation withheld because this record is not cleared for redistribution. synthetic accessibility score - SAS, a property used to compare generated molecule sets Quotation withheld because this record is not cleared for redistribution. Quantitative Estimation of Drug-likeness - QED, ranging from zero to one with higher values more drug-like Quotation withheld because this record is not cleared for redistribution. Gaussian process model - a lightweight smooth-function model used to smooth the optimization landscape Quotation withheld because this record is not cleared for redistribution. joint training for property prediction - organizes latent-space molecules by property value Quotation withheld because this record is not cleared for redistribution. teacher forcing - feeds the previous sampled character to the GRU, improving valid outputs but complicating training Quotation withheld because this record is not cleared for redistribution. method: computational-model a second reading DIFFERING from the recorded value. Recorded: system-or-tool-building — note: a second reading, verbatim-verified, disagreeing with the value on the record. Not applied - a person decides. Quotation withheld because this record is not cleared for redistribution. phenomenon: the discreteness of chemical space as an obstacle to searching it a second reading DIFFERING from the recorded value. Recorded: automated molecular design in chemical space — note: a second reading, verbatim-verified, disagreeing with the value on the record. Not applied - a person decides. Quotation withheld because this record is not cleared for redistribution. builds-on → autoencoders (deep generative models, AI) Quotation withheld because this record is not cleared for redistribution. builds-on → Bowman et al. (ref 20), variational autoencoders with recurrent encoder/decoder for English sentences Quotation withheld because this record is not cleared for redistribution. contradicts → existing discrete search methods for chemical space (genetic algorithms and discrete interpolation techniques, refs 10-18) Quotation withheld because this record is not cleared for redistribution. builds-on → Bowman et al., variational autoencoders for written English sentences Quotation withheld because this record is not cleared for redistribution. contradicts → fixed-library and genetic-algorithm approaches to molecular search Quotation withheld because this record is not cleared for redistribution. information bottleneck —part-of→ autoencoder information bottleneck autoencoder strength of the relation: 1 Quotation withheld because this record is not cleared for redistribution. SMILES —contrasted-with→ InChI SMILES InChI strength of the relation: 1 Quotation withheld because this record is not cleared for redistribution. information bottleneck —causes→ latent representation information bottleneck latent representation strength of the relation: 1 Audit 2026-09-28: DOUBTFUL - read against its full span and kept, but endpoint by anaphora: 'This bottleneck' = the information bottleneck (previous sentence); 'latent representation' is written 'compressed representation'. Quotation withheld because this record is not cleared for redistribution. encoder network —part-of→ autoencoder encoder network autoencoder the relation itself is separately evidenced · strength of the relation: 1 · note: The paper explicitly says the autoencoder is comprised of the encoder and decoder networks. Quotation withheld because this record is not cleared for redistribution. decoder network —part-of→ autoencoder decoder network autoencoder the relation itself is separately evidenced · strength of the relation: 1 · note: The paper explicitly says the autoencoder is comprised of the encoder and decoder networks. Quotation withheld because this record is not cleared for redistribution. hundreds of thousands — training set size of existing molecules Quotation withheld because this record is not cleared for redistribution. < 9 — size regime of one demonstration molecule set Quotation withheld because this record is not cleared for redistribution. 10^8 synthesised vs 10^23–10^60 possible — the size of the search problem the method addresses Quotation withheld because this record is not cleared for redistribution. An autoencoder maps SMILES-encoded molecules to a continuous vector space. Quotation withheld because this record is not cleared for redistribution. EVIDENCE CORRECTED (RE-GROUNDED) · Continuous representations allow gradient-based optimization to guide the search for optimized functional compounds. EVIDENCE CORRECTED: RE-GROUNDED. The published string was in NEITHER copy of the paper. Ours read 'THESE representations'; the paper says 'CONTINUOUS representations', and continuity is the property the whole method rests on - it is what makes gradient-based optimization possible at all. The demonstrative dropped the reason. Quotation withheld because this record is not cleared for redistribution. Once molecules are points in a continuous space, generation becomes arithmetic - decode a random vector, perturb a known molecule, interpolate between two. Quotation withheld because this record is not cleared for redistribution. The encoding matters: SMILES works and InChI works substantially worse, apparently because its syntax involves counting and arithmetic. Quotation withheld because this record is not cleared for redistribution. The search problem is one of scale: about 10^8 substances have ever been made, against an estimated 10^23 to 10^60 drug-like possibilities. Quotation withheld because this record is not cleared for redistribution. Genetic generation of compounds requires manually specified heuristics for mutation and crossover. Quotation withheld because this record is not cleared for redistribution. continuous representation continuous representation Quotation withheld because this record is not cleared for redistribution. property values property values Quotation withheld because this record is not cleared for redistribution. string representation string representation Quotation withheld because this record is not cleared for redistribution. SMILES strings SMILES strings Quotation withheld because this record is not cleared for redistribution. latent representation latent representation Quotation withheld because this record is not cleared for redistribution. information bottleneck information bottleneck Quotation withheld because this record is not cleared for redistribution. variational autoencoder variational autoencoder Quotation withheld because this record is not cleared for redistribution. encoder encoder Quotation withheld because this record is not cleared for redistribution. decoder decoder Quotation withheld because this record is not cleared for redistribution. predictor predictor Quotation withheld because this record is not cleared for redistribution. mean mean Quotation withheld because this record is not cleared for redistribution. regression regression Quotation withheld because this record is not cleared for redistribution. regression regression Quotation withheld because this record is not cleared for redistribution.

The full text of this article is not redistributable, so it is not included here. The evidence sentences are withheld with it; the tag layer and its grounding status remain. The article itself is at https://doi.org/10.1021/acscentsci.7b00572.