Efficient Estimation of Word Representations in Vector Space

Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean · ICLR Workshop · 2013 · linguistics · record mikolov2013-word2vec
tag layer · index · concepts · authors · עברית

Where this record's text came from. faceted 2026-08 by reading the locally held text (pdftext); every evidence string verified as an exact substring of that text before merging. The paper is a method contribution plus a new benchmark ('To measure quality of the word vectors, we define a comprehensive test set that contains five types of semantic questions, and nine types of syntactic questions.'), which the single-value method field cannot show alongside system-or-tool-building. No limitations section; the quoted limits are drawn from §4.1, §4.5 and §5. One further self-noted caveat ('the accuracy metric that assumes exact match ... would score only about 60%') was not recorded because the sentence is split across a page break by a table in the extracted text and could not be quoted verbatim in one piece. 2026-08-14 and 2026-08-25 schema-0.3 passes: every quote verified as an exact substring of the staged text and anchored at merge time; concept terms verified present; every edge endpoint an existing node. Update, 15 September 2026: EXCEPTION, measured with the merge gate's own predicate (Paper.locate): 1 quote(s) on this record are not found as one exact string in the current staged text - content_tags.findings[0].evidence. The sentence above does not hold for them as written. Where a row records why - the quote is grounded in another copy of the paper, joins two passages with an ellipsis, or differs from the staged text by extraction noise - that note stays on the row.

1 quote on this page breaks a word across a line. The PDF this text was extracted from hyphenated a word at the end of a printed line — adversar- ial, repre- sentations — and the extractor kept the hyphen and the space. The quotes keep them because a quote is reproduced exactly or it proves nothing: the break is the typesetter’s, not the author’s. A search of this page will not find the affected words.

1 quote on this page was cut from a different copy of this article. This project holds two extractions of this paper: the plain text every other quote here is anchored in, and the publisher’s own HTML edition, which we are not licensed to republish. It is verbatim in the second and cannot be found in the first — not because the tagging is wrong, but because the extraction damaged the passage: a page number dropped into the middle of a number, a formula moved out of its sentence, a ligature where two letters belong. The record names the copy for each one. The quote is the paper’s; the extraction is not.

How to read this page. The article itself is not reproduced here — this paper is not cleared for redistribution, so what follows is the tag layer alone. The evidence sentences are withheld with the article. The table at the foot lists every tag the record carries; each row names the facet it belongs to, and the whose words column says whether the tag records the paper's own wording or the tagger's sentence about the paper. Every tag links to each other paper in the corpus carrying it. The machine-readable edition of the same thing is the TEI file beside this one, where the tags live in standOff, and the article is at its publisher via the link below. When we have changed our mind. Some entries in the table begin with a word in capitals and the reason for it. EVIDENCE CORRECTED — the finding still stands, but we had quoted the wrong sentence from the paper, and the quotation has been replaced with the right one. RETRACTED — the entry itself was wrong; it stays on the page, marked, with the reason, rather than being deleted. WITHDRAWN AND REPLACED — the entry was wrong, and a corrected version of it appears in the entries below. Each mark carries its own reason, in brackets, because the reasons differ from one entry to the next. An entry we have WITHDRAWN is removed from this page and kept in the record, with its reason. A line reading Audit and a date is a later re-reading of the entry against its whole sentence: DOUBTFUL means it was kept, and the reason says what is loose about it. Quotation marks inside the tagger's sentence. Where a tag is the tagger's own sentence, a phrase in quotation marks inside it is borrowed from the paper — the tagger's wording around it is not. The paper's own sentence is the evidence quote for that row; on this page it is withheld with the article, and the article is at its publisher. Nothing inside quotation marks anywhere on this page is ours: if it is quoted, it is the paper's. Whose words a concept carries. A concept entry records a term the paper itself uses, so the term, the sense given for it, and its evidence are the paper's own words, quoted where the paper fixes them. This is the opposite of a sense entry in the tag table, which is the tagger's sentence about how the paper uses a term. Where a concept entry could not be grounded in the paper's words, the entry says so on its own row.

Other fields define these differently the same word, defined another way in another discipline
semantic 2 fields

Tags on this paper

Every tag the record carries, each linking to every other paper in the corpus that carries it. A tag appears here whether or not its words occur in the article — the spans below are only the ones a sentence in this paper licenses. Of the 7 concept tags on this paper, 4 are grounded in a quoted sentence.

word embedding distributional semantics skip-gram vector space model word vectors cosine distance CBOW learn distributed word vectors with a shallow model - CBOW predicts a word from its context, skip-gram predicts the context from the word - so words in similar contexts get similar vectors word vector representations semantic syntactic similarity architecture Continuous Bag-of-Words Continuous Skip-gram Model model architecture correlation Spearman system-or-tool-building

Legend

relation to other work	`claim_relations`	how this paper stands to another: agrees, contradicts, extends, qualifies
claim type	`claim_type`	the shape of the claim: existence, mechanism, method, critique, prediction, null
concept edge	`concept_edges`	an asserted relation between two concepts, with the sentence asserting it
concept	`concepts`	an idea the paper works with, quoted where the paper uses it
content tag	`content_tags`	a finding, definition, quantity, statistic or variable named in the paper
control	`controls`	an experiment run to eliminate a rival explanation
limitation	`limitations`	what the paper says its evidence cannot support
mechanism	`mechanisms`	how the reported effect is said to come about
method	`method`	how the paper produced its evidence
phenomenon	`phenomenon`	the effect or regularity under study
sense	`senses`	a distinct use of a term, quoted where the paper fixes it
transfer	`transfers`	a method or concept carried in from another field

The paper

The full text is not reproduced here. This article is not cleared for redistribution, so the page carries the tag layer only. Its evidence sentences are withheld; the grounding status remains with each tag, and the article itself is at https://arxiv.org/abs/1301.3781. The reading view exists for the papers whose licence permits it.

Every tag on this page

claim_relations 4 · claim_type 1 · concept_edges 1 · concepts 4 · content_tags 7 · controls 1 · limitations 8 · mechanisms 1 · method 1 · phenomenon 1 · senses 3 · transfers 1

Relations on this page carry a quotation for the claim. Where a relation was separately evidenced — and so tested against the vocabulary for its type — the row says so; none of 1 relation here carry that mark. The rest are the tagger’s reading of the sentence quoted beside them.

facet	tag	field	conf.	whose words
claim_type	method Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`claim_type`		the paper's words
phenomenon	distributed vector representations of words (word embeddings) Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`phenomenon`		the paper's words
method	system-or-tool-building Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: system-or-tool-building	`method`		the paper's words
mechanisms	mechanisms Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: learn distributed word vectors with a shallow model - CBOW predicts a word from its context, skip-gram predicts the context from the word - so words in similar contexts get similar vectors	`mechanisms[0]`		the tagger's words
limitations	for many languages the available training corpora are only a few billion words or less (a data constraint) Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: the results for lower-resource languages: Quotation withheld	`limitations[0]`		the tagger's words
limitations	The exact-match accuracy metric makes 100% unreachable because the models have no morphological information and synonyms count as errors. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: the accuracy figures against their own ceiling: REACHING 100% ACCURACY IS LIKELY TO BE IMPOSSIBLE, AS THE CURRENT MODELS DO NOT HAVE ANY INPUT INFORMATION ABOUT WORD MORPHOLOGY	`limitations[1]`		the tagger's words
limitations	The test set excludes multi-word entities. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: everything the evaluation can speak about: WE HAVE INCLUDED IN OUR TEST SET ONLY SINGLE TOKEN WORDS, THUS MULTI-WORD ENTITIES ARE NOT PRESENT (SUCH AS NEW YORK) - what falls outside is invisible, not wrong	`limitations[2]`		the tagger's words
limitations	Relationship accuracy in the qualitative examples still leaves much room for improvement. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: the qualitative relationship examples: ACCURACY IS QUITE GOOD, ALTHOUGH THERE IS CLEARLY A LOT OF ROOM FOR FURTHER IMPROVEMENTS	`limitations[3]`		the tagger's words
limitations	On the Microsoft Sentence Completion Challenge the Skip-gram model alone is worse than LSA similarity; it only wins in combination with RNNLMs. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: the Sentence Completion result specifically: THE SKIP-GRAM MODEL ITSELF DOES NOT PERFORM ON THIS TASK BETTER THAN LSA SIMILARITY - it contributes only in combination	`limitations[4]`		the tagger's words
limitations	The benchmark's question count is combinatorial: questions are formed by pairing items from short hand-made lists, so the number of questions is far larger than the number of independent facts being tested. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: the size of this test set as a measure of evidence: a list of 68 hand-listed American cities and their states was expanded into about 2,500 questions by pairing entries at random, so questions within a category are not independent of one another	`limitations[5]`	1.0	the tagger's words
limitations	In the semantic half of a semantic benchmark, a correct-meaning answer is scored wrong unless it is the exact expected token. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: any accuracy figure from this test set: the answer must be the exact expected token, and synonyms are counted as mistakes	`limitations[6]`	1.0	the tagger's words
limitations	The connection between the benchmark and any practical use is asserted as a belief, not shown. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: this accuracy as evidence that the vectors are useful: the authors state a belief that usefulness for certain applications should be positively correlated with the metric	`limitations[7]`	1.0	the tagger's words
transfers	distributed vector representations of words, taken from neural language models into large-scale computational linguistics and simplified until they could be trained on billions of words Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`transfers[0]`	0.85	the tagger's words
controls	the same data used to train a previously published recurrent neural network language model, kept as the comparison point Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. eliminates: a speed or quality gain that comes from a different or larger corpus rather than from the model - the RNNLM baseline saw the same data · outcome (stated by the tagger, no quotation): held	`controls[0]`	0.85	the tagger's words
concepts	skip-gram Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: skip-gram	`concepts[2]`		the paper's words
concepts	word vectors Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: word vectors	`concepts[4]`		the paper's words
concepts	cosine distance Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: cosine distance	`concepts[5]`		the paper's words
concepts	CBOW Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: CBOW	`concepts[6]`		the paper's words
senses	semantic - AS FIVE HAND-CHOSEN RELATION TYPES: the semantic half of the test set is five relations (common capital city, all capital cities, currency, city-in-state, Man-Woman), so 'captures semantics' means 'answers analogies of those five kinds' Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`senses[0]`	1.0	the tagger's words
senses	similarity - AS COSINE DISTANCE AFTER A VECTOR OFFSET, with the three input words removed from the search, so the answer is the nearest remaining word rather than the nearest word Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`senses[1]`	1.0	the tagger's words
senses	the analogy relation - AS ARITHMETIC ON VECTORS: a relation between two words is the difference of their vectors, and it is applied to a third word by addition Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`senses[2]`	1.0	the tagger's words
claim_relations	builds-on → neural-network language models Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`claim_relations[0]`		the tagger's words
claim_relations	extends → the two-step NNLM architecture of [13, 14] Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`claim_relations[1]`		the tagger's words
claim_relations	builds-on → the authors' earlier two-step word-vector training work [13, 14] Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`claim_relations[2]`		the tagger's words
claim_relations	builds-on → the model-complexity comparison framework of [18] Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`claim_relations[3]`		the tagger's words
concept_edges	CBOW —contrasted-with→ Skip-gram Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. strength of the relation: 1 also in: CBOW · Skip-gram	`concept_edges[0]`		the tagger's words
content_tags	1.6 billion — training corpus size; high-quality vectors learned in under a day Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`content_tags.quantities[0]`		the tagger's words
content_tags	< 1 day — training time for high-quality word vectors Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`content_tags.quantities[1]`		the tagger's words
content_tags	The learned vectors capture linear regularities: vector(King) - vector(Man) + vector(Woman) is closest to vector(Queen). Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. about this quote: QUOTE REPAIRED: the extractor split a hyphen-space and the quote was recorded faithfully with the damage in it, which made it unanchorable. The repaired string is an EXACT SUBSTRING of the staged text - that is why it was accepted, not because a rule produced it. The text as originally recorded is kept in evidence as extracted.	`content_tags.findings[1]`		the tagger's words
content_tags	model architecture Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. about this quote: QUOTE REPAIRED: the extractor split a hyphen-space and the quote was recorded faithfully with the damage in it, which made it unanchorable. The repaired string is an EXACT SUBSTRING of the staged text - that is why it was accepted, not because a rule produced it. The text as originally recorded is kept in evidence as extracted. also in: model architecture	`content_tags.variables[0]`		the tagger's words
content_tags	Continuous Bag-of-Words Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: Continuous Bag-of-Words	`content_tags.definitions[0]`		the tagger's words
content_tags	Continuous Skip-gram Model Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: Continuous Skip-gram Model	`content_tags.definitions[1]`		the tagger's words
content_tags	correlation Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: correlation	`content_tags.statistics[0]`		the tagger's words

Interpretive choices

Tags that could not be placed in the text:

content_tags.findings[0] — quotation withheld — the page has already reproduced as much of this paper as the rights budget allows
NOT MACHINE-VOUCHED: declared-ellipsis - An elided quote: its 2 pieces ('We propose two novel model architectures' ... 'ccuracy at much lower computational cost') are each in our copy of the paper (with line-break hyphens, spacing and punctuation set aside), and the quote marks the cut with an ellipsis, so the joined string is in no copy. Located mechanically, piece by piece.

What was deliberately not tagged, and why, is recorded per paper by the tagger. Where a paper has no such record, this page says so rather than implying that nothing was declined.