phenomenon · claim-type · cross-domain-transfer · mechanism · claim-relation · replication-status · limitation — grounded in the full text| phenomenon | distributed vector representations of words (word embeddings)— “The main goal of this paper is to introduce techniques that can be used for learning high-quality word vectors from huge data sets with billions of words, and with millions of words in the vocabulary.” | |
| claim-type | method— “We propose two novel model architectures for computing continuous vector representations of words from very large data sets.” | |
| method — evidence | — “In this section, we propose two new model architectures for learning distributed representations of words that try to minimize computational complexity.” | |
| transfer | from neural network language modelling: distributed vector representations of words, taken from neural language models into large-scale computational linguistics and simplified until they could be trained on billions of words— “Probably the most successful concept is to use distributed representations of words [10].” | transfers[0] |
| mechanism | learn distributed word vectors with a shallow model - CBOW predicts a word from its context, skip-gram predicts the context from the word - so words in similar contexts get similar vectors— “The CBOW architecture predicts the current word based on the context, and the Skip-gram predicts surrounding words given the current word” | mechanisms[0] |
| builds-on | neural-network language models— “techniques based on different types of neural networks” | claim_relations[0] |
| extends | the two-step NNLM architecture of [13, 14]— “In this work, we directly extend this architecture, and focus just on the first step where the word vectors are learned using a simple model.” | claim_relations[1] |
| builds-on | the authors' earlier two-step word-vector training work [13, 14]— “The new architectures directly follow those proposed in our earlier work [13, 14], where it was found that neural network language model can be successfully trained in two steps: first, continuous word vectors are learned using simple model, and then the N-gram NNLM is trained on top of these distributed representations of words.” | claim_relations[2] |
| builds-on | the model-complexity comparison framework of [18]— “Similar to [18], to compare different model architectures we define first the computational complexity of a model as the number of parameters that need to be accessed to fully train the model.” | claim_relations[3] |
| sense | semantic - AS FIVE HAND-CHOSEN RELATION TYPES: the semantic half of the test set is five relations (common capital city, all capital cities, currency, city-in-state, Man-Woman), so 'captures semantics' means 'answers analogies of those five kinds'— “To measure quality of the word vectors, we define a comprehensive test set that contains five types of semantic questions, and nine types of syntactic questions. Two examples from each category are shown in Table 1. Overall, there are 8869 semantic and 10675 syntactic questions” | senses[0] |
| sense | similarity - AS COSINE DISTANCE AFTER A VECTOR OFFSET, with the three input words removed from the search, so the answer is the nearest remaining word rather than the nearest word— “Then, we search in the vector space for the word closest to X measured by cosine distance, and use it as the answer to the question (we discard the input question words during this search).” | senses[1] |
| sense | the analogy relation - AS ARITHMETIC ON VECTORS: a relation between two words is the difference of their vectors, and it is applied to a third word by addition— “we can simply compute vector X = vector(”biggest”) − vector(”big”) + vector(”small”). Then, we search in the vector space for the word closest to X measured by cosine distance, and use it as the answer to the question” | senses[2] |
| control | the same data used to train a previously published recurrent neural network language model, kept as the comparison point — held — rules out a speed or quality gain that comes from a different or larger corpus rather than from the model - the RNNLM baseline saw the same data— “We used these data to provide a comparison to a previously trained recurrent neural network language model that took about 8 weeks to train on a single CPU.” | controls[0] |
| edge | CBOW —contrasted-with→ Skip-gram— “The CBOW architecture predicts the current word based on the context, and the Skip-gram predicts surrounding words given the current word” | concept_edges[0] |
| outcome (stated by the tagger, no quotation) | control — held | controls[0] |
| for which use | limitation — the results for lower-resource languages: THE EXISTING CORPORA FOR MANY LANGUAGES CONTAIN ONLY A FEW BILLIONS OF WORDS OR LESS— “the existing corpora for many languages contain only a few billions of words or less” | limitations[0] |
| for which use | limitation — the accuracy figures against their own ceiling: REACHING 100% ACCURACY IS LIKELY TO BE IMPOSSIBLE, AS THE CURRENT MODELS DO NOT HAVE ANY INPUT INFORMATION ABOUT WORD MORPHOLOGY— “This also means that reaching 100% accuracy is likely to be impossible, as the current models do not have any input information about word morphology.” | limitations[1] |
| for which use | limitation — everything the evaluation can speak about: WE HAVE INCLUDED IN OUR TEST SET ONLY SINGLE TOKEN WORDS, THUS MULTI-WORD ENTITIES ARE NOT PRESENT (SUCH AS NEW YORK) - what falls outside is invisible, not wrong— “We have included in our test set only single token words, thus multi-word entities are not present (such as New York).” | limitations[2] |
| for which use | limitation — the qualitative relationship examples: ACCURACY IS QUITE GOOD, ALTHOUGH THERE IS CLEARLY A LOT OF ROOM FOR FURTHER IMPROVEMENTS— “As it can be seen, accuracy is quite good, although there is clearly a lot of room for further improvements” | limitations[3] |
| for which use | limitation — the Sentence Completion result specifically: THE SKIP-GRAM MODEL ITSELF DOES NOT PERFORM ON THIS TASK BETTER THAN LSA SIMILARITY - it contributes only in combination— “While the Skip-gram model itself does not perform on this task better than LSA similarity, the… | limitations[4] |
| for which use | limitation — the size of this test set as a measure of evidence: a list of 68 hand-listed American cities and their states was expanded into about 2,500 questions by pairing entries at random, so questions within a category are not independent of one another | limitations[5] |
| for which use | limitation — any accuracy figure from this test set: the answer must be the exact expected token, and synonyms are counted as mistakes Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | limitations[6] |
| for which use | limitation — this accuracy as evidence that the vectors are useful: the authors state a belief that usefulness for certain applications should be positively correlated with the metric Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | limitations[7] |
| replication | untested | |
| limitation | for many languages the available training corpora are only a few billion words or less (a data constraint) Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | limitations[0] |
| limitation | The exact-match accuracy metric makes 100% unreachable because the models have no morphological information and synonyms count as errors. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | limitations[1] |
| limitation | The test set excludes multi-word entities. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | limitations[2] |
| limitation | Relationship accuracy in the qualitative examples still leaves much room for improvement. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | limitations[3] |
| limitation | On the Microsoft Sentence Completion Challenge the Skip-gram model alone is worse than LSA similarity; it only wins in combination with RNNLMs. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | limitations[4] |
| limitation | The benchmark's question count is combinatorial: questions are formed by pairing items from short hand-made lists, so the number of questions is far larger than the number of independent facts being tested. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | limitations[5] |
| limitation | In the semantic half of a semantic benchmark, a correct-meaning answer is scored wrong unless it is the exact expected token. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | limitations[6] |
| limitation | The connection between the benchmark and any practical use is asserted as a belief, not shown. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | limitations[7] |
finding · quantity · variable · definition · key-term — grounded in the full text| finding | Two novel architectures compute continuous word vectors from very large data sets, with large accuracy improvements at much lower computational cost. Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.findings[0] |
| not machine-vouched | finding — declared-ellipsis - An elided quote: its 2 pieces ('We propose two novel model architectures' ... 'ccuracy at much lower computational cost') are each in our copy of the paper (with line-break hyphens, spacing and punctuation set aside), and the quote marks the cut with an ellipsis, so the joined string is in no copy. Located mechanically, piece by piece. | content_tags.findings[0] |
| finding | The learned vectors capture linear regularities: vector(King) - vector(Man) + vector(Woman) is closest to vector(Queen). Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.findings[1] |
| evidence status note | finding — QUOTE REPAIRED: the extractor split a hyphen-space and the quote was recorded faithfully with the damage in it, which made it unanchorable. The repaired string is an EXACT SUBSTRING of the staged text - that is why it was accepted, not because a rule produced it. The text as originally recorded is kept in evidence as extracted. | content_tags.findings[1] |
| quantity | 1.6 billion words — training corpus size; high-quality vectors learned in under a day Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.quantities[0] |
| quantity | < 1 day — training time for high-quality word vectors Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.quantities[1] |
| variable | model architecture (independent) Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.variables[0] |
| evidence status note | variable — QUOTE REPAIRED: the extractor split a hyphen-space and the quote was recorded faithfully with the damage in it, which made it unanchorable. The repaired string is an EXACT SUBSTRING of the staged text - that is why it was accepted, not because a rule produced it. The text as originally recorded is kept in evidence as extracted. | content_tags.variables[0] |
| definition | Continuous Bag-of-Words Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.definitions[0] |
| definition | Continuous Skip-gram Model Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.definitions[1] |
| key-terms | word, vector, representations, semantic, syntactic, similarity, architecture, CBOW, Skip-gram |
| concept | skip-gram Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[2] |
| concept | word vectors Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[4] |
| concept | cosine distance Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[5] |
| concept | CBOW Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | concepts[6] |
| correlation r | as written: Spearman Quotation withheld — this paper is not ours to reproduce, and the page has already quoted as much of it as the rights layer allows. The tag, its field and its confidence stand; the sentence is at the publisher. | content_tags.statistics[0] |
When we have changed our mind. Some entries in the table begin with a word in capitals and the reason for it. EVIDENCE CORRECTED — the finding still stands, but we had quoted the wrong sentence from the paper, and the quotation has been replaced with the right one. RETRACTED — the entry itself was wrong; it stays on the page, marked, with the reason, rather than being deleted. WITHDRAWN AND REPLACED — the entry was wrong, and a corrected version of it appears in the entries below. Each mark carries its own reason, in brackets, because the reasons differ from one entry to the next. An entry we have WITHDRAWN is removed from this page and kept in the record, with its reason. A line reading Audit and a date is a later re-reading of the entry against its whole sentence: DOUBTFUL means it was kept, and the reason says what is loose about it. Quotation marks inside the tagger's sentence. Where a tag is the tagger's own sentence, a phrase in quotation marks inside it is borrowed from the paper — the tagger's wording around it is not. The paper's own sentence is the evidence quote for that row; on this page it is withheld with the article, and the article is at its publisher. Nothing inside quotation marks anywhere on this page is ours: if it is quoted, it is the paper's. Whose words a concept carries. A concept entry records a term the paper itself uses, so the term, the sense given for it, and its evidence are the paper's own words, quoted where the paper fixes them. This is the opposite of a sense entry in the tag table, which is the tagger's sentence about how the paper uses a term. Where a concept entry could not be grounded in the paper's words, the entry says so on its own row.
Tag layer © the Meta-Tagging Project, CC BY 4.0. The paper text is © its authors/publisher
and is linked, not reproduced. Excerpts shown are short quotations for identification and verification.
· back to index
· concepts · authors
· עברית
© 2026 Shir Sivroni · ORCID 0009-0000-2597-2824