phenomenon · claim-type · cross-domain-transfer · mechanism · claim-relation · replication-status · limitation — grounded in the full text| phenomenon | word vector representations from co-occurrence statistics— “The statistics of word occurrences in a corpus is the primary source of information available to all unsupervised methods for learning word representations, and although many such methods now exist, the question still remains as to how meaning is generated from these statistics, and how the resulting word vectors might represent that… | |
| claim-type | method— “The result is a new global logbilinear regression model that combines the advantages of the two major model families in the literature: global matrix factorization and local context window methods.” | |
| method — evidence | — “We propose a specific weighted least squares model that trains on global word-word co-occurrence counts and thus makes efficient use of statistics.” | |
| transfer | from information retrieval / linear algebra (LSA, matrix factorization): global matrix factorization (from latent semantic analysis) combined with local context windows to learn word vectors— “combines the advantages of the two major model families in the literature: global matrix factorization and local context window methods” | transfers[0] |
| mechanism | learn word vectors from the global co-occurrence matrix via a log-bilinear model on ratios of co-occurrence probabilities, so vector differences encode meaning— “The above argument suggests that the appropriate starting point for word vector learning should be with ratios of co-occurrence probabilities rather than the probabilities themselves” | mechanisms[0] |
| builds-on | global matrix factorization (LSA) and local context-window methods— “combines the advantages of the two major model families in the literature: global matrix factorization and local context window methods” | claim_relations[0] |
| contradicts | Baroni et al. (2014)— “Currently, prediction-based models garner substantial support; for example, Baroni et al. (2014) argue that these models perform better across a range of tasks. In this work we argue that the two classes of methods are not dramatically different at a fundamental level since they both probe the underlying co-occurrence statistics of the… | claim_relations[1] |
| extends | skip-gram and ivLBL window-based models— “Therefore, in this subsection we show how these models are related to our proposed model, as defined in Eqn. (8).” | claim_relations[2] |
| sense | what the paper calls its own object - WORD VECTORS AND WORD REPRESENTATIONS, NEVER EMBEDDINGS: it uses those two terms 24 and 11 times; 'word embeddings' appears exactly once and describes another group's method rather than this paper's output, and 'distributional semantics' does not appear at all— “The result, GloVe, is a new global log-bilinear regression model for the unsupervised learning of word representations that outperforms other models on word analogy, word similarity, and named entity recognition tasks” | senses[0] |
| sense | 'word embeddings' IN THIS PAPER - SOMEONE ELSE'S OBJECT: the single occurrence attributes the term to Levy et al., alongside vLBL and ivLBL, as a rival PPMI-based approach; the authors never apply the word to what they themselves produce— “proposed closely-related vector log-bilinear models, vLBL and ivLBL, and Levy et al. (2014) proposed explicit word embeddings based on a PPMI metric” | senses[1] |
| sense | the analogy evaluation - AS DIMENSIONS OF DIFFERENCE, NOT DISTANCE: it probes the vector space by examining the various dimensions of difference between vectors rather than the scalar distance between them, which is why a model can win on similarity and lose on analogy— “introduced a new evaluation scheme based on word analogies that probes the finer structure of the word vector space by examining not the scalar distance between word vectors, but rather their various dimensions of difference” | senses[2] |
| sense | the context window - SYMMETRIC OR ASYMMETRIC, AND THE CHOICE MOVES ONE SUBTASK ONLY: a window extending both ways is called symmetric and one extending only left asymmetric, and the syntactic subtask specifically improves with small asymmetric windows— “A context window that extends to the left and right of a target word will be called symmetric, and one which extends only to the left will be called asymmetric” | senses[3] |
| control | holding corpus, vocabulary size, vector dimension and context window fixed across GloVe and word2vec, and plotting accuracy against training time — partial— “In all cases, we train 300-dimensional vectors on the same 6B token corpus (Wikipedia 2014 + Gigaword 5) with the same 400,000 word vocabulary, and use a symmetric context window of size 10.” — rules out that the accuracy difference between GloVe and word2vec comes from differing hyperparameters or training budget rather than from the model itself— “We control for the main sources of variation that we identified in Sections 4.4 and 4.5 by setting the vector length, context window size, corpus, and vocabulary size to the configuration mentioned in the previous subsection.” | controls[0] |
| audit | context window —causes→ syntactic subtask — Audit 2026-09-28: DOUBTFUL - read against its full span and kept, but the span says performance on the syntactic subtask is better for small context windows; causes is loose.— “Performance is better on the syntactic subtask for small and asymmetric context windows, which aligns with the intuition that syntactic information is mostly drawn from the immediate context and can depend strongly on word order” | concept_edges[1] |
| weight | context window —causes→ syntactic subtask — 0.8— “Performance is better on the syntactic subtask for small and asymmetric context windows, which aligns with the intuition that syntactic information is mostly drawn from the immediate context and can depend strongly on word order” | concept_edges[1] |
| outcome | control — partial— “We control for the main sources of variation that we identified in Sections 4.4 and 4.5 by setting the vector length, context window size, corpus, and vocabulary size to the configuration mentioned in the previous subsection.”— “In all cases, we train 300-dimensional vectors on the same 6B token corpus (Wikipedia 2014 + Gigaword 5) with the same 400,000 word vocabulary, and use a symmetric context window of size 10.” | controls[0] |
| for which use | limitation — the explanatory claim about vector arithmetic: THE ORIGIN OF THESE REGULARITIES HAS REMAINED OPAQUE— “the origin of these regularities has remained opaque” | limitations[0] |
| for which use | limitation — every result produced with the weighting function: WE OFFER ONLY EMPIRICAL MOTIVATION FOR CHOOSING THE VALUE 3/4 - the exponent is fitted, not derived | limitations[1] |
| for which use | limitation — the head-to-head comparison with word2vec: A RIGOROUS QUANTITATIVE COMPARISON ... IS COMPLICATED BY THE EXISTENCE OF MANY PARAMETERS THAT HAVE A STRONG EFFECT ON PERFORMANCE | limitations[2] |
| for which use | limitation — every reported comparison, and it is the confound item 2 names: WE SET ANY UNSPECIFIED PARAMETERS TO THEIR DEFAULT VALUES, ASSUMING THAT THEY ARE CLOSE TO OPTIMAL, THOUGH WE ACKNOWLEDGE THAT THIS SIMPLIFICATION SHO[uld be noted] Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | limitations[3] |
| for which use | limitation — the scalability demonstration specifically: TO DEMONSTRATE THE SCALABILITY OF THE MODEL, WE ALSO TRAINED IT ON A MUCH LARGER SIXTH CORPUS, CONTAINING 840 BILLION TOKENS OF WEB DATA, BUT IN THIS [case the conditions differ] Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | limitations[4] |
| for which use | limitation — the claim that this model outperforms word2vec: unspecified parameters of the compared systems were left at default values on the assumption they are close to optimal, which the authors call a simplification that a more thorough analysis should relax Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | limitations[5] |
| for which use | limitation — results that depend on the weighting function: the authors state they offer only empirical motivation for the value 3/4, noting a similar fractional power scaling was found elsewhere Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | limitations[6] |
| for which use | limitation — any interpretation of the two vector sets as different kinds of thing: the paper states the distinction between a word and a context word is arbitrary and that the two roles may be exchanged Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | limitations[7] |
| for which use | limitation — a comparison across the corpora reported here: for the 840-billion-token web corpus the vocabulary was not lowercased, and the authors state the results are therefore not directly comparable Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | limitations[8] |
| for which use | limitation — the claim about how the model's cost scales: it requires assuming the number of co-occurrences of two words can be modelled as a power-law function of that pair's frequency rank Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | limitations[9] |
| replication | untested | |
| limitation | the origin of the linguistic regularities captured by word-vector arithmetic was not understood Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | limitations[0] |
| limitation | The chosen exponent in the weighting function is justified only empirically, not derived. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | limitations[1] |
| limitation | The head-to-head comparison with word2vec is confounded by many performance-affecting parameters. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | limitations[2] |
| limitation | Unspecified parameters were left at defaults, a simplification the authors say a more thorough analysis should relax. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | limitations[3] |
| limitation | The largest-corpus (840B token) run used a different preprocessing, so its results are not comparable with the reported ones. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | limitations[4] |
| limitation | The comparison that establishes the headline result runs the competing method at default settings, assumed near-optimal, and the paper says the assumption should be relaxed. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | limitations[5] |
| limitation | The exponent that shapes the weighting function has empirical motivation only. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | limitations[6] |
| limitation | The distinction the model is built on is declared arbitrary by the authors. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | limitations[7] |
| limitation | The largest corpus in the paper was processed differently, so its numbers do not sit on the same scale as the rest. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | limitations[8] |
| limitation | The complexity result rests on an assumed distribution rather than a measured one. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | limitations[9] |
finding · quantity · variable · definition · key-term — grounded in the full text| finding | GloVe is a new global log-bilinear regression model combining the advantages of global matrix factorization and local context window methods. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.findings[0] |
| evidence status note | finding — QUOTE REPAIRED: the extractor split a hyphen-space and the quote was recorded faithfully with the damage in it, which made it unanchorable. The repaired string is an EXACT SUBSTRING of the staged text - that is why it was accepted, not because a rule produced it. The text as originally recorded is kept in evidence as extracted. | content_tags.findings[0] |
| finding | Only in the ratio of co-occurrence probabilities does noise from non-discriminative words cancel out. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.findings[1] |
| quantity | 75% — accuracy on the word analogy task (GloVe headline result) Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.quantities[0] |
| quantity | 6 billion tokens — corpus size used to build the co-occurrence statistics Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.quantities[1] |
| variable | word-word co-occurrence counts (independent) Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.variables[0] |
| evidence status note | variable — QUOTE REPAIRED: the extractor split a hyphen-space and the quote was recorded faithfully with the damage in it, which made it unanchorable. The repaired string is an EXACT SUBSTRING of the staged text - that is why it was accepted, not because a rule produced it. The text as originally recorded is kept in evidence as extracted. | content_tags.variables[0] |
| definition | GloVe Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.definitions[0] |
| evidence status note | definition — QUOTE REPAIRED: the extractor split a hyphen-space and the quote was recorded faithfully with the damage in it, which made it unanchorable. The repaired string is an EXACT SUBSTRING of the staged text - that is why it was accepted, not because a rule produced it. The text as originally recorded is kept in evidence as extracted. | content_tags.definitions[0] |
| definition | word analogy task Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.definitions[1] |
| key-terms | word, vectors, co-occurrence, corpus, semantic, matrix, analogy, GloVe |
| concept | word vector Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[4] |
| concept | context word Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[5] |
| concept | word analogy Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[6] |
| concept | context window Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[7] |
| concept | syntactic subtask Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[8] |
| correlation r | as written: correlation coefficient Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.statistics[0] |
| regression β | as written: regression model Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.statistics[1] |
| evidence status note | regression — QUOTE REPAIRED: the extractor split a hyphen-space and the quote was recorded faithfully with the damage in it, which made it unanchorable. The repaired string is an EXACT SUBSTRING of the staged text - that is why it was accepted, not because a rule produced it. The text as originally recorded is kept in evidence as extracted. | content_tags.statistics[1] |
| claim type | method — a second reading reaching the same value; recorded: method Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | alternatives.claim_type |
| method | system-or-tool-building — a second reading reaching the same value; recorded: system-or-tool-building Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | alternatives.method |
| phenomenon | why linear regularities emerge in word vector spaces learned from co-occurrence statistics — a second reading DIFFERING from the recorded value; recorded: word vector representations from co-occurrence statistics Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | alternatives.phenomenon |
When we have changed our mind. Some entries in the table begin with a word in capitals and the reason for it. EVIDENCE CORRECTED — the finding still stands, but we had quoted the wrong sentence from the paper, and the quotation has been replaced with the right one. RETRACTED — the entry itself was wrong; it stays on the page, marked, with the reason, rather than being deleted. WITHDRAWN AND REPLACED — the entry was wrong, and a corrected version of it appears in the entries below. Each mark carries its own reason, in brackets, because the reasons differ from one entry to the next. An entry we have WITHDRAWN is removed from this page and kept in the record, with its reason. A line reading Audit and a date is a later re-reading of the entry against its whole sentence: DOUBTFUL means it was kept, and the reason says what is loose about it. Quotation marks inside the tagger's sentence. Where a tag is the tagger's own sentence, a phrase in quotation marks inside it is borrowed from the paper — the tagger's wording around it is not. The paper's own sentence is the evidence quote for that row; on this page it is withheld with the article, and the article is at its publisher. Nothing inside quotation marks anywhere on this page is ours: if it is quoted, it is the paper's. Whose words a concept carries. A concept entry records a term the paper itself uses, so the term, the sense given for it, and its evidence are the paper's own words, quoted where the paper fixes them. This is the opposite of a sense entry in the tag table, which is the tagger's sentence about how the paper uses a term. Where a concept entry could not be grounded in the paper's words, the entry says so on its own row.
Tag layer © the Meta-Tagging Project, CC BY 4.0. The paper text is © its authors/publisher
and is linked, not reproduced. Excerpts shown are short quotations for identification and verification.
· back to index
· concepts · authors
· עברית
© 2026 Shir Sivroni · ORCID 0009-0000-2597-2824