Generalization without Systematicity: On the Compositional Skills of Sequence-to-Sequence Recurrent Networks

Generalization without Systematicity: On the Compositional Skills of Sequence-to-Sequence Recurrent Networks Brenden M. Lake Marco Baroni

Tag layer of the DHSS Meta-Tagging Project. The article text is reproduced from the staged copy named in sourceDesc and is not altered; the tags are held separately in standOff so that overlapping and nested spans are preserved.

Generalization without Systematicity: On the Compositional Skills of Sequence-to-Sequence Recurrent Networks 2018 ICML / arXiv 1711.00350 lake2018-generalization-systematicity lake2018-generalization-systematicity.txt, the project's staged copy e04f874749847e2c737cb8046d899feebe0e49c5 arxiv-open cognitive-psychology

the paper's own words

the tag layer; a sentence written by the tagger about the paper, always anchored to a quotation but not itself a quotation

0 spans located by searching the stored text for the quote's exact words, allowing any whitespace between them null-result Quotation withheld because this record is not cleared for redistribution. systematic compositional generalization in sequence-to-sequence networks Quotation withheld because this record is not cleared for redistribution. simulation simulation Quotation withheld because this record is not cleared for redistribution. mechanisms A STIPULATED EQUIVALENCE CUTS THE TEST SPACE: commands are declared distributionally identical - defined as occurring in exactly the same composed commands - and that declaration is what makes testing the rest redundant. The sameness asserted is a property of the training distribution, not of the actions Quotation withheld because this record is not cleared for redistribution. mechanisms THE DIFFERENCE BETWEEN TWO ITEMS IS EXPLAINED BY EXPOSURE RATHER THAN CAPACITY, and hedged twice while doing it: one primitive's action is seen many times inside other directed actions, which APPARENTLY suffices, PROBABLY because the model gets direct evidence of how it is used in context - while the other action occurs only with its own primitive command and does not generalise from that minimal context Quotation withheld because this record is not cleared for redistribution. mechanisms A NEAR-PERFECT SUCCESS IS DISQUALIFIED ON THE SHAPE OF ITS ERRORS RATHER THAN THEIR NUMBER: all 45 errors are conjunctions containing one of just two components (22 and 23 cases), both of which the model handles correctly on their own - and the authors say it is HARD TO CONCEIVE OF someone who understood the longer command but not the shorter one inside it. The error DISTRIBUTION, not the error rate, is what is read as measuring the capacity Quotation withheld because this record is not cleared for redistribution. mechanisms A DELIBERATELY UNREALISTIC MANIPULATION IS ADDED AS A DIAGNOSTIC, WITH BOTH OUTCOMES NAMED IN ADVANCE: an oracle supplies the correct output length, and the authors state before running it that a large improvement would mean the decoder was merely terminating early while a small one would mean the failure is deeper. The control is designed to be uninformative about performance and informative about cause about this quote: The span contains a LOST-HYPHEN FUSION - 'overallbest' for 'overall-best'. A line-break hyphen was dropped by the extraction and the two halves fused. The quote is an exact substring and anchors correctly; the damage is in what a reader of the tag sees. None of the four shapes an internal tool looks for covers this, so it is annotated by hand. Seven such fusions are in this file. Quotation withheld because this record is not cleared for redistribution. mechanisms A RIVAL EXPLANATION OF THE FAILURE IS ELIMINATED BY A SEPARATE MEASUREMENT: the network preferred its own output to the target by log-likelihood, so the errors are not search failures in the decoder - the model is not failing to FIND the right answer, it does not prefer it about this quote: The span contains a LOST-HYPHEN FUSION - 'selfgenerated' for 'self-generated'. A line-break hyphen was dropped by the extraction and the two halves fused. The quote is an exact substring and anchors correctly; the damage is in what a reader of the tag sees. None of the four shapes an internal tool looks for covers this, so it is annotated by hand. Seven such fusions are in this file. Quotation withheld because this record is not cleared for redistribution. mechanisms THE SUFFICIENCY OF THE TRAINING DATA IS ARGUED SEPARATELY FROM THE RESULT AND MARKED AS IMPORTANT: the authors state that the training data of both experiments provide enough evidence to learn the composition rules that would afford the correct generalisations, and then spell out, per experiment, what was demonstrated during training Quotation withheld because this record is not cleared for redistribution. mechanisms THE DEFLATIONARY READING OF THE RESULT IS STATED BY THE AUTHORS THEMSELVES AND THEN ANSWERED: the finding can be seen as a trivial restatement of the principle that train and test should come from the same distribution - and the reply is that human learners DO generalise to data governed by rules they have learned, so the triviality does not cover the case Quotation withheld because this record is not cleared for redistribution. mechanisms AN ALTERNATIVE EXPLANATION OF A NULL IS ELIMINATED BY GRADED DOSING RATHER THAN ARGUMENT: composed examples of the new primitive are stepped 8 -> 16 -> 32 and accuracy climbs 38.3 -> 77.8 -> 88.4, appearing GRADUALLY instead of switching on, which rules out the reading that the network had simply lacked evidence Quotation withheld because this record is not cleared for redistribution. mechanisms THE CAPACITY IS OPERATIONALISED AS A BEHAVIOURAL SIGNATURE, NOT AS AN ACCURACY: what is expected of a systematically compositional device is ALL-OR-NOTHING rule-based behaviour, so a network that generalises powerfully but needs more positive examples for it to emerge fails the definition WHILE SUCCEEDING AT THE TASK Quotation withheld because this record is not cleared for redistribution. mechanisms THE AUTHORS NAME THEIR OWN TEST'S POSSIBLE UNFAIRNESS BEFORE ANYONE ELSE CAN: they say they tested systematicity in its purest form, then concede the setup is arguably too opaque for a computational model which could lack evidence that the two actions are the same sort of thing - and the concession is what motivates the follow-up experiment Quotation withheld because this record is not cleared for redistribution. mechanisms THE CONTROL ITEM IS MATCHED ON THE TEST AND DELIBERATELY UNMATCHED ON THE TRAINING: a real adjective seen in 80 different constructions is tested on exactly the same eight constructions as the invented one and scores 8 of 8 against 1 of 8 - so the comparison isolates breadth of training context while holding the test fixed Quotation withheld because this record is not cleared for redistribution. mechanisms THE POSITIVE HALF OF A NEGATIVE RESULT IS INSISTED ON RATHER THAN CONCEDED: the authors explicitly stress that the high zero-shot accuracy is an important positive result in controlled experiments, and add that understanding the mechanisms behind it is future work - refusing to let the headline failure absorb the finding that sits beside it Quotation withheld because this record is not cleared for redistribution. the SCAN benchmark grammar lacks recursion and generates only a finite command set Quotation withheld because this record is not cleared for redistribution. The machine-translation demonstration is small-scale and its result is described by the authors as preliminary. Quotation withheld because this record is not cleared for redistribution. The authors concede the Experiment 3 setup (a primitive seen only in isolation) may be unfairly opaque for a computational model that has no evidence the new action belongs to the same class as the others. Quotation withheld because this record is not cleared for redistribution. The benchmark's name is an acronym whose first letter is its own admission of simplification. what this limits: any result reported as SCAN accuracy: the authors state the data set is called SCAN because it is Quotation withheld Quotation withheld because this record is not cleared for redistribution. The criterion the models are measured against comes from an argument the authors call controversial. what this limits: the claim that these networks lack systematicity: the standard comes from a set of papers the authors describe as influential and controversial, in which neural networks are argued to be associative devices that cannot capture systematic compositionality Quotation withheld because this record is not cleared for redistribution. Failure is defined by what an idealised agent would find trivial, not by a measured human baseline. what this limits: the description of these failures as catastrophic: the differences are said to be of the sort that would be trivial for an agent equipped with an algebraic mind Quotation withheld because this record is not cleared for redistribution. the systematicity failure is reframed using machine learning's own assumption that training and test data come from the same distribution Quotation withheld because this record is not cleared for redistribution. the network re-trained with varying numbers of DISTINCT examples while the number of training presentations was held constant at 100K eliminates: diversity being confounded with volume - more distinct examples otherwise means more gradient steps, and the two cannot be separated · outcome (stated by the tagger, no quotation): held Quotation withheld because this record is not cleared for redistribution. SCAN SCAN Quotation withheld because this record is not cleared for redistribution. CommAI Navigation tasks CommAI Navigation tasks Quotation withheld because this record is not cleared for redistribution. systematic compositionality systematic compositionality Quotation withheld because this record is not cleared for redistribution. algebraic mind algebraic mind Quotation withheld because this record is not cleared for redistribution. mix-and-match mix-and-match Quotation withheld because this record is not cleared for redistribution. SCAN - AN ACRONYM THAT ADMITS ITS OWN REDUCTION: the S is for Simplified, because the data set is Quotation withheld; the expansion carries the scope limitation and the four letters that get cited do not Quotation withheld because this record is not cleared for redistribution. systematic compositionality - A CONTESTED PHILOSOPHICAL CRITERION USED AS A BENCHMARK: it comes from papers the authors themselves call influential and controversial, in which neural networks are argued to be associative devices that cannot capture it, so a SCAN score inherits an argument it cannot display Quotation withheld because this record is not cleared for redistribution. a catastrophic failure - RELATIVE TO AN ALGEBRAIC MIND: the models' errors are called catastrophic because the differences involved would be trivial for an agent equipped with an algebraic mind, which is a construct rather than a measured comparison group Quotation withheld because this record is not cleared for redistribution. builds-on → sequence-to-sequence learning methods Quotation withheld because this record is not cleared for redistribution. builds-on → the CommAI Navigation tasks (Mikolov et al., 2016) Quotation withheld because this record is not cleared for redistribution. builds-on → the attentional sequence-to-sequence model of Bahdanau et al. (2015) Quotation withheld because this record is not cleared for redistribution. extends → three decades of prior tests of compositional generalization in neural networks in the systematicity debate Quotation withheld because this record is not cleared for redistribution. RNNs generalize zero-shot when training and test commands differ only slightly, using mix-and-match strategies, but fail spectacularly when generalization needs systematic compositional skills. about this quote: QUOTE REPAIRED: the extractor split a hyphen-space and the quote was recorded faithfully with the damage in it, which made it unanchorable. The repaired string is an EXACT SUBSTRING of the staged text - that is why it was accepted, not because a rule produced it. The text as originally recorded is kept in evidence as extracted. Quotation withheld because this record is not cleared for redistribution. A proof-of-concept in neural machine translation suggests lack of systematicity partly explains data-hungriness. about this quote: QUOTE REPAIRED: the extractor split a hyphen-space and the quote was recorded faithfully with the damage in it, which made it unanchorable. The repaired string is an EXACT SUBSTRING of the staged text - that is why it was accepted, not because a rule produced it. The text as originally recorded is kept in evidence as extracted. Quotation withheld because this record is not cleared for redistribution. SCAN domain SCAN domain about this quote: QUOTE REPAIRED: the extractor split a hyphen-space and the quote was recorded faithfully with the damage in it, which made it unanchorable. The repaired string is an EXACT SUBSTRING of the staged text - that is why it was accepted, not because a rule produced it. The text as originally recorded is kept in evidence as extracted. Quotation withheld because this record is not cleared for redistribution. systematic compositionality systematic compositionality about this quote: QUOTE REPAIRED: the extractor split a hyphen-space and the quote was recorded faithfully with the damage in it, which made it unanchorable. The repaired string is an EXACT SUBSTRING of the staged text - that is why it was accepted, not because a rule produced it. The text as originally recorded is kept in evidence as extracted. Quotation withheld because this record is not cleared for redistribution. mean mean Quotation withheld because this record is not cleared for redistribution. standard-error standard-error Quotation withheld because this record is not cleared for redistribution.

The full text of this article is not redistributable, so it is not included here. The evidence sentences are withheld with it; the tag layer and its grounding status remain. The article itself is at https://arxiv.org/abs/1711.00350.