Natural Language Processing (Almost) from Scratch

Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen · Journal of Machine Learning Research · 2011 · linguistics · record collobert2011-nlp-scratch
tag layer · index · concepts · authors · עברית

Where this record's text came from. faceted 2026-08 by reading the locally held text (pdftext); every evidence string verified as an exact substring of that text before merging. The multi-task-learning result is effectively an internal null result, but the paper's overall contribution shape is a method, so claim type is 'method' and the MTL null is recorded as a stated limitation instead. The paper's own Section 7 ('Critical Discussion') anticipates criticism of both goal and means; that self-critique is broader than a single quotable limitation and is not listed separately. The comparison with Brown clusters ("Our embeddings perform at least as well as Brown Clusters") was not encoded as a claim_relation because the available relation types do not cover a benchmark comparison. 2026-08-14, 2026-08-15, 2026-08-17 and 2026-08-25 schema-0.3 passes: every quote verified as an exact substring of the staged text and anchored at merge time; concept terms verified present; every edge endpoint an existing node. Update, 15 September 2026: EXCEPTION, measured with the merge gate's own predicate (Paper.locate): 2 quote(s) on this record are not found as one exact string in the current staged text - content_tags.statistics[0].evidence, content_tags.statistics[1].evidence. The sentence above does not hold for them as written. Where a row records why - the quote is grounded in another copy of the paper, joins two passages with an ellipsis, or differs from the staged text by extraction noise - that note stays on the row.

The quality of the text these quotes were cut from. symbol-font-not-extractable: 172 '(cid:NNN)' glyph-index markers remain in a staged text of 109,238 characters, where the PDF's embedded font had no ToUnicode map. THE QUOTES ON THIS RECORD ARE THE PAPER'S AND THE STAGED COPY IS THE DAMAGED ONE - which is the opposite of the usual case and is why it went unnoticed: one of its 59 live quotes is not an exact substring of this text, and it was located in it by aligning the quote's WORD SEQUENCE under exactly this damage - the words are there, the characters inside them are not. Not repaired in place: editing the staged text moves every offset after the edit and re-hashes every anchor on the record. Measured at 113aae60fefc3eb1, re-measured unchanged at 54b63482778ddf56.

2 quotes on this page were cut from a different copy of this article. This project holds two extractions of this paper: the plain text every other quote here is anchored in, and the publisher’s own HTML edition, which we are not licensed to republish. They are verbatim in the second and cannot be found in the first — not because the tagging is wrong, but because the extraction damaged the passage: a page number dropped into the middle of a number, a formula moved out of its sentence, a ligature where two letters belong. The record names the copy for each one. The quote is the paper’s; the extraction is not.

How to read this page. The article itself is not reproduced here — this paper is not cleared for redistribution, so what follows is the tag layer alone. The evidence sentences are withheld with the article. The table at the foot lists every tag the record carries; each row names the facet it belongs to, and the whose words column says whether the tag records the paper's own wording or the tagger's sentence about the paper. Every tag links to each other paper in the corpus carrying it. The machine-readable edition of the same thing is the TEI file beside this one, where the tags live in standOff, and the article is at its publisher via the link below. When we have changed our mind. Some entries in the table begin with a word in capitals and the reason for it. EVIDENCE CORRECTED — the finding still stands, but we had quoted the wrong sentence from the paper, and the quotation has been replaced with the right one. RETRACTED — the entry itself was wrong; it stays on the page, marked, with the reason, rather than being deleted. WITHDRAWN AND REPLACED — the entry was wrong, and a corrected version of it appears in the entries below. Each mark carries its own reason, in brackets, because the reasons differ from one entry to the next. An entry we have WITHDRAWN is removed from this page and kept in the record, with its reason. A line reading Audit and a date is a later re-reading of the entry against its whole sentence: DOUBTFUL means it was kept, and the reason says what is loose about it. Quotation marks inside the tagger's sentence. Where a tag is the tagger's own sentence, a phrase in quotation marks inside it is borrowed from the paper — the tagger's wording around it is not. The paper's own sentence is the evidence quote for that row; on this page it is withheld with the article, and the article is at its publisher. Nothing inside quotation marks anywhere on this page is ours: if it is quoted, it is the paper's. Whose words a concept carries. A concept entry records a term the paper itself uses, so the term, the sense given for it, and its evidence are the paper's own words, quoted where the paper fixes them. This is the opposite of a sense entry in the tag table, which is the tagger's sentence about how the paper uses a term. Where a concept entry could not be grounded in the paper's words, the entry says so on its own row.

Other fields define these differently the same word, defined another way in another discipline
standard-deviation 3 fields · confidence-interval 2 fields · confidence intervals 2 fields

Tags on this paper

Every tag the record carries, each linking to every other paper in the corpus that carries it. A tag appears here whether or not its words occur in the article — the spans below are only the ones a sentence in this paper licenses. Of the 10 concept tags on this paper, 5 are grounded in a quoted sentence.

neural network natural language processing word embedding sequence labeling feature learning benchmark internal representations linguistic knowledge lookup table a priori NLP knowledge THE FIELD'S GOAL IS POSED AS A QUESTION AND THEN REPORTED TO HAVE NO AGREED FORM: will a program ever convert English text into a data structure that UNAMBIGUOUSLY AND COMPLETELY DESCRIBES THE MEANING - and AMONG NUMEROUS PROBLEMS, NO CONSENSUS HAS EMERGED ABOUT THE FORM OF SUCH A DATA STRUCTURE. The obstacle named first is not difficulty but the absence of agreement on what the answer would look like THE FIELD'S TASKS ARE NAMED AS REDUCED OBJECTIVES ADOPTED BECAUSE THE REAL QUESTION IS UNRESOLVED: UNTIL SUCH FUNDAMENTAL PROBLEMS ARE RESOLVED, computer scientists MUST SETTLE FOR REDUCED OBJECTIVES - extracting SIMPLER REPRESENTATIONS describing RESTRICTED ASPECTS of the textual information. What the field measures itself on is described as a substitute, in the paper's second sentence THE SAME REPRESENTATIONS ARE GIVEN TWO DIFFERENT WARRANTS, AND ONE OF THEM IS A BELIEF: they are OFTEN MOTIVATED BY SPECIFIC APPLICATIONS, for instance bag-of-words for information retrieval - and they CAN ALSO BE MOTIVATED BY OUR BELIEF THAT THEY CAPTURE SOMETHING MORE GENERAL about natural language. An instrumental justification and an epistemic one, offered for the same objects THE ANNOTATIONS EXIST IN ORDER TO MAKE SYSTEMS COMPARABLE, AND THE BENCHMARKS ARE CREDITED WITH DRIVING THE RESEARCH: text corpora HAVE BEEN MANUALLY ANNOTATED with such data structures IN ORDER TO COMPARE THE PERFORMANCE of various systems, and THE AVAILABILITY OF STANDARD BENCHMARKS HAS STIMULATED RESEARCH. The ground truth's purpose is comparison, and the comparison is what made the field move THE METHOD IS DEFINED BY WHAT IT REFUSES TO USE, AND THE COST IS STATED IN THE SAME SENTENCE: the versatility is achieved BY TRYING TO AVOID TASK-SPECIFIC ENGINEERING AND THEREFORE DISREGARDING A LOT OF PRIOR KNOWLEDGE - Quotation withheld. What is given up is named as the price of what is gained THE GROUND TRUTH ITSELF COMES IN TWO VOCABULARIES, AND THE FIX IS TO PICK ONE: the ground truth for three of the tasks IS PROVIDED USING TWO DIFFERENT TAGGING SCHEMES, and IN ORDER TO ELIMINATE THIS ADDITIONAL SOURCE OF VARIATIONS the authors DECIDED TO USE THE MOST EXPRESSIVE scheme FOR ALL TASKS. Two naming systems for the same annotations, normalised by choosing the more expressive - not the more common THE CHOICE OF HYPER-PARAMETERS IS CONCEDED TO BE ONE OF MANY EQUALLY GOOD ONES, WITH THE UNEXPLORED SPACE NAMED: although they ended up with A PARTICULAR CHOICE, MANY OTHER CHOICES ARE ALMOST EQUALLY AS GOOD - ALTHOUGH PERHAPS THERE ARE OTHERS THAT ARE BETTER, AS WE COULD NOT DO A FULL GRID SEARCH. The reported configuration is presented as a sample rather than an optimum THE SCALE OF THE EXPERIMENT IS ATTRIBUTED TO THE HARDWARE RATHER THAN TO THE SCIENCE, WRYLY: very long training times MAKE SUCH STRATEGIES NECESSARY FOR THE FORESEEABLE FUTURE - and IF WE HAD BEEN GIVEN COMPUTERS TEN TIMES FASTER, WE PROBABLY WOULD HAVE FOUND USES FOR DATASETS TEN TIMES BIGGER. The design is described as a function of the machines available TWO NEGATIVE RESULTS FROM OTHER GROUPS ARE CITED SIDE BY SIDE: one group obtained results SOMEWHAT BELOW THE STATE-OF-THE-ART using joint decoding for two tasks, and another DESCRIBED A NEGATIVE RESULT AT THE SAME JOINT TASK. Failures are gathered as evidence about a practice rather than about the groups that attempted it part-of-speech tagging chunking named entity recognition semantic role labeling unlabeled data benchmarks task-specific engineering almost from scratch unified architecture syntactic information semantic information unlabeled training data benchmark tasks ad-hoc features confidence-interval confidence intervals standard-deviation standard deviation system-or-tool-building

Legend

relation to other work	`claim_relations`	how this paper stands to another: agrees, contradicts, extends, qualifies
claim type	`claim_type`	the shape of the claim: existence, mechanism, method, critique, prediction, null
concept edge	`concept_edges`	an asserted relation between two concepts, with the sentence asserting it
concept	`concepts`	an idea the paper works with, quoted where the paper uses it
content tag	`content_tags`	a finding, definition, quantity, statistic or variable named in the paper
control	`controls`	an experiment run to eliminate a rival explanation
limitation	`limitations`	what the paper says its evidence cannot support
mechanism	`mechanisms`	how the reported effect is said to come about
method	`method`	how the paper produced its evidence
phenomenon	`phenomenon`	the effect or regularity under study
sense	`senses`	a distinct use of a term, quoted where the paper fixes it

The paper

The full text is not reproduced here. This article is not cleared for redistribution, so the page carries the tag layer only. Its evidence sentences are withheld; the grounding status remains with each tag, and the article itself is at https://www.jmlr.org/papers/v12/collobert11a.html. The reading view exists for the papers whose licence permits it.

Every tag on this page

claim_relations 4 · claim_type 1 · concept_edges 1 · concepts 5 · content_tags 17 · controls 2 · limitations 9 · mechanisms 9 · method 1 · phenomenon 1 · senses 3

Relations on this page carry a quotation for the claim. Where a relation was separately evidenced — and so tested against the vocabulary for its type — the row says so; none of 1 relation here carry that mark. The rest are the tagger’s reading of the sentence quoted beside them.

facet	tag	field	conf.	whose words
claim_type	method Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`claim_type`		the paper's words
phenomenon	neural sequence labelling across NLP benchmark tasks Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`phenomenon`		the paper's words
method	system-or-tool-building Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: system-or-tool-building	`method`		the paper's words
mechanisms	mechanisms Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: THE FIELD'S GOAL IS POSED AS A QUESTION AND THEN REPORTED TO HAVE NO AGREED FORM: will a program ever convert English text into a data structure that UNAMBIGUOUSLY AND COMPLETELY DESCRIBES THE MEANING - and AMONG NUMEROUS PROBLEMS, NO CONSENSUS HAS EMERGED ABOUT THE FORM OF SUCH A DATA STRUCTURE. The obstacle named first is not difficulty but the absence of agreement on what the answer would look like	`mechanisms[0]`	0.95	the tagger's words
mechanisms	mechanisms Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: THE FIELD'S TASKS ARE NAMED AS REDUCED OBJECTIVES ADOPTED BECAUSE THE REAL QUESTION IS UNRESOLVED: UNTIL SUCH FUNDAMENTAL PROBLEMS ARE RESOLVED, computer scientists MUST SETTLE FOR REDUCED OBJECTIVES - extracting SIMPLER REPRESENTATIONS describing RESTRICTED ASPECTS of the textual information. What the field measures itself on is described as a substitute, in the paper's second sentence	`mechanisms[1]`	0.95	the tagger's words
mechanisms	mechanisms Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: THE SAME REPRESENTATIONS ARE GIVEN TWO DIFFERENT WARRANTS, AND ONE OF THEM IS A BELIEF: they are OFTEN MOTIVATED BY SPECIFIC APPLICATIONS, for instance bag-of-words for information retrieval - and they CAN ALSO BE MOTIVATED BY OUR BELIEF THAT THEY CAPTURE SOMETHING MORE GENERAL about natural language. An instrumental justification and an epistemic one, offered for the same objects	`mechanisms[2]`	0.95	the tagger's words
mechanisms	mechanisms Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. about this quote: RE-SLICED, AND THE GUARD THAT CAUGHT IT WAS TWENTY MINUTES OLD. My end key was "...Natural Language Processing (NLP) and" - ending on "and" - because the page furniture ("Keywords: ... arXiv 1. Introduction") is spliced into that sentence and its real continuation is on the far side of the header. The dangling-word check refused it at drafting time. also in: THE ANNOTATIONS EXIST IN ORDER TO MAKE SYSTEMS COMPARABLE, AND THE BENCHMARKS ARE CREDITED WITH DRIVING THE RESEARCH: text corpora HAVE BEEN MANUALLY ANNOTATED with such data structures IN ORDER TO COMPARE THE PERFORMANCE of various systems, and THE AVAILABILITY OF STANDARD BENCHMARKS HAS STIMULATED RESEARCH. The ground truth's purpose is comparison, and the comparison is what made the field move	`mechanisms[3]`	0.85	the tagger's words
mechanisms	mechanisms Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: THE METHOD IS DEFINED BY WHAT IT REFUSES TO USE, AND THE COST IS STATED IN THE SAME SENTENCE: the versatility is achieved BY TRYING TO AVOID TASK-SPECIFIC ENGINEERING AND THEREFORE DISREGARDING A LOT OF PRIOR KNOWLEDGE - Quotation withheld. What is given up is named as the price of what is gained	`mechanisms[4]`	0.95	the tagger's words
mechanisms	mechanisms Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: THE GROUND TRUTH ITSELF COMES IN TWO VOCABULARIES, AND THE FIX IS TO PICK ONE: the ground truth for three of the tasks IS PROVIDED USING TWO DIFFERENT TAGGING SCHEMES, and IN ORDER TO ELIMINATE THIS ADDITIONAL SOURCE OF VARIATIONS the authors DECIDED TO USE THE MOST EXPRESSIVE scheme FOR ALL TASKS. Two naming systems for the same annotations, normalised by choosing the more expressive - not the more common	`mechanisms[5]`	0.95	the tagger's words
mechanisms	mechanisms Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: THE CHOICE OF HYPER-PARAMETERS IS CONCEDED TO BE ONE OF MANY EQUALLY GOOD ONES, WITH THE UNEXPLORED SPACE NAMED: although they ended up with A PARTICULAR CHOICE, MANY OTHER CHOICES ARE ALMOST EQUALLY AS GOOD - ALTHOUGH PERHAPS THERE ARE OTHERS THAT ARE BETTER, AS WE COULD NOT DO A FULL GRID SEARCH. The reported configuration is presented as a sample rather than an optimum	`mechanisms[6]`	0.95	the tagger's words
mechanisms	mechanisms Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: THE SCALE OF THE EXPERIMENT IS ATTRIBUTED TO THE HARDWARE RATHER THAN TO THE SCIENCE, WRYLY: very long training times MAKE SUCH STRATEGIES NECESSARY FOR THE FORESEEABLE FUTURE - and IF WE HAD BEEN GIVEN COMPUTERS TEN TIMES FASTER, WE PROBABLY WOULD HAVE FOUND USES FOR DATASETS TEN TIMES BIGGER. The design is described as a function of the machines available	`mechanisms[7]`	0.95	the tagger's words
mechanisms	mechanisms Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: TWO NEGATIVE RESULTS FROM OTHER GROUPS ARE CITED SIDE BY SIDE: one group obtained results SOMEWHAT BELOW THE STATE-OF-THE-ART using joint decoding for two tasks, and another DESCRIBED A NEGATIVE RESULT AT THE SAME JOINT TASK. Failures are gathered as evidence about a practice rather than about the groups that attempted it	`mechanisms[8]`	0.85	the tagger's words
limitations	avoiding task-specific features meant forgoing a large body of linguistic knowledge Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`limitations[0]`		the tagger's words
limitations	The system is only 'almost' from scratch: words are lowercased and capitalization encoded, and learning from letter sequences was left out of scope. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: the from-scratch claim: IDEALLY, AN EVEN MORE RAW INPUT WOULD BE TO LEARN FROM LETTER SEQUENCES RATHER THAN WORDS - words are lowercased and capitalisation encoded separately	`limitations[1]`		the tagger's words
limitations	Week-long training times made a proper hyperparameter search infeasible. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: every hyperparameter-dependent result: TRAINING TIMES FOR SUCH LARGE SCALE SYSTEMS ARE COUNTED IN WEEKS, [SO] IT IS NOT FEASIBLE TO TRY MANY COMBINATIONS	`limitations[2]`		the tagger's words
limitations	Multi-task learning gave no or only marginal gains over separately trained per-task architectures. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: the multi-task claim: NO (OR ONLY MARGINAL) IMPROVEMENTS WERE OBTAINED with the unified architecture over separately trained ones	`limitations[3]`		the tagger's words
limitations	The language model structure is probably too restrictive to capture hierarchical syntax; only the word embeddings are exploited downstream. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: any syntactic reading of the language model: ITS STRUCTURE IS PROBABLY TOO RESTRICTIVE FOR SUCH GOALS, and only the word embeddings are exploited	`limitations[4]`		the tagger's words
limitations	The benchmark scores are treated as indirect measurements of something the paper does not measure directly, and the claim about it is a posit. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: the benchmark tables reported here: the authors view the benchmarks as indirect measurements of the relevance of the internal representations, and posit that those representations are more general than any of the benchmarks	`limitations[5]`	1.0	the tagger's words
limitations	The paper names the failure mode its own design is exposed to. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: the comparison of these results against other systems: the authors note there is a great temptation to optimize a system's performance for a specific benchmark	`limitations[6]`	1.0	the tagger's words
limitations	The pre-processing the architecture does keep is named, with a presumably attached to removing it. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: the claim that this system works from raw input: the authors state they did some pre-processing, namely lowercasing and encoding capitalization as a separate feature, and say that with enough unlabeled data they could presumably learn without it	`limitations[7]`	1.0	the tagger's words
limitations	Task-specific engineering is added back in a later section, so the headline architecture and the best numbers are not the same system. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: the best figures quoted from this paper: a later section evaluates how much further improvement comes from incorporating standard NLP task-specific engineering, which the main architecture avoids	`limitations[8]`	1.0	the tagger's words
controls	the word embedding dimension held constant at 50 across the tasks, so the language models of the earlier section could be reused unchanged Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. eliminates: a task-to-task difference that is really a difference in embedding capacity · outcome (stated by the tagger, no quotation): held	`controls[0]`	0.85	the tagger's words
controls	performance also reported with the embeddings FROZEN during task-specific training, beside the figures where they are allowed to move Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. eliminates: attributing to the pre-trained embeddings a gain that actually comes from fine-tuning them on the task · outcome (stated by the tagger, no quotation): held	`controls[1]`	0.85	the tagger's words
concepts	benchmark Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: benchmark	`concepts[5]`		the paper's words
concepts	internal representations Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: internal representations	`concepts[6]`		the paper's words
concepts	linguistic knowledge Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: linguistic knowledge	`concepts[7]`		the paper's words
concepts	lookup table Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: lookup table	`concepts[8]`		the paper's words
concepts	a priori NLP knowledge Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: a priori NLP knowledge	`concepts[9]`		the paper's words
senses	'almost from scratch' - THE PAPER NAMES ITS OWN HEDGE AND GIVES THE REASON: it calls the approach almost from scratch to emphasise a reduced but still important reliance on a priori NLP knowledge, while the body says 'from scratch' twenty-seven times and 'almost from scratch' once Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`senses[0]`	1.0	the tagger's words
senses	a benchmark - AS AN INDIRECT MEASUREMENT, NOT A GOAL: the scores stand in for the relevance of internal representations that nobody measures directly, and the authors posit those representations are more general than the benchmarks that stand in for them Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`senses[1]`	1.0	the tagger's words
senses	the cost of the design - LINGUISTIC KNOWLEDGE, DELIBERATELY SET ASIDE: avoiding task-specific engineered features is stated to have led the authors to ignore a large body of linguistic knowledge, so the architecture's generality and its ignorance are the same decision Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`senses[2]`	1.0	the tagger's words
claim_relations	contradicts → the reliance on task-specific hand-engineered features in NLP Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. differs in: THE PARAMETER IS WHERE THE FEATURES COME FROM. The span: 'Quotation withheld' 'Instead of' names the axis of disagreement in the same clause as the disagreement.	`claim_relations[0]`		the tagger's words
claim_relations	builds-on → the convolutional / Time Delay Neural Network approach of Waibel et al. (1989) Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`claim_relations[1]`		the tagger's words
claim_relations	extends → Conditional Random Fields (Lafferty et al., 2001) Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`claim_relations[2]`		the tagger's words
claim_relations	builds-on → graph transformer networks (Bottou et al., 1997; Le Cun et al., 1998) Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`claim_relations[3]`		the tagger's words
concept_edges	internal representations —operationalised-as→ benchmark Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. strength of the relation: 1 also in: internal representations · benchmark	`concept_edges[0]`		the tagger's words
content_tags	852 — unlabeled corpus used for language modeling Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`content_tags.quantities[0]`		the tagger's words
content_tags	4 — benchmark tasks (POS, CHUNK, NER, SRL) Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`content_tags.quantities[1]`		the tagger's words
content_tags	912,344 — POS training-set size (WSJ sections) Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`content_tags.quantities[2]`		the tagger's words
content_tags	one unified network handles many NLP tasks without task-specific engineering Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. about this quote: QUOTE RECOVERED FROM THE PAPER'S OWN CHARACTERS. The recorded quote was LETTERS-ONLY IDENTICAL to this stretch of the staged text but not an exact substring of it - spaces, hyphens, ligature glyphs or mathematical operators differed, so it could never anchor. This is the file's own text between the same two letters: not produced by a rule about the damage, copied out of the file. Round-trip checked (the recovered span is letters-only equal to the original) and length-checked. The text as originally recorded is kept in evidence as extracted.	`content_tags.findings[0]`		the tagger's words
content_tags	the system learns internal representations from mostly unlabeled data Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`content_tags.findings[1]`		the tagger's words
content_tags	transferring unsupervised representations improves supervised benchmarks Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`content_tags.findings[2]`		the tagger's words
content_tags	avoiding task-specific engineering deliberately disregards prior knowledge Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`content_tags.findings[3]`		the tagger's words
content_tags	internal representations Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: internal representations	`content_tags.variables[0]`		the tagger's words
content_tags	unlabeled training data Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: unlabeled training data	`content_tags.variables[1]`		the tagger's words
content_tags	benchmark tasks Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: benchmark tasks	`content_tags.variables[2]`		the tagger's words
content_tags	ad-hoc features Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: ad-hoc features	`content_tags.variables[3]`		the tagger's words
content_tags	almost from scratch Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. about this quote: QUOTE RECOVERED FROM THE PAPER'S OWN CHARACTERS. The recorded quote was LETTERS-ONLY IDENTICAL to this stretch of the staged text but not an exact substring of it - spaces, hyphens, ligature glyphs or mathematical operators differed, so it could never anchor. This is the file's own text between the same two letters: not produced by a rule about the damage, copied out of the file. Round-trip checked (the recovered span is letters-only equal to the original) and length-checked. The text as originally recorded is kept in evidence as extracted. also in: almost from scratch	`content_tags.definitions[0]`		the tagger's words
content_tags	Part-Of-Speech tagging Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: Part-Of-Speech tagging	`content_tags.definitions[1]`		the tagger's words
content_tags	unified architecture Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: unified architecture	`content_tags.definitions[2]`		the tagger's words
content_tags	syntactic information Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: syntactic information	`content_tags.definitions[4]`		the tagger's words
content_tags	semantic information Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: semantic information	`content_tags.definitions[5]`		the tagger's words
content_tags	standard-deviation Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: standard-deviation	`content_tags.statistics[2]`		the tagger's words

Interpretive choices

Tags that could not be placed in the text:

content_tags.statistics[1] — quotation withheld — the page has already reproduced as much of this paper as the rights budget allows
NOT MACHINE-VOUCHED: not-in-our-copy - The quote is in the original web page we fetched for this paper, but not in the text we tag against, which was extracted differently. Checked with ligatures, hyphens, spacing and punctuation set aside.

What was deliberately not tagged, and why, is recorded per paper by the tagger. Where a paper has no such record, this page says so rather than implying that nothing was declined.