| phenomenon | cultural dimensions in word-embedding spaces— “We demonstrate that derived dimensions of word embedding vector spaces correspond closely to “cultural dimensions,” such as race, class, and gender, which individuals use in everyday life to classify and categorize objects in the world.” |
| claim-type | method— “In this paper, we present a novel method for applying word embedding models to the sociological analysis of culture.” |
| method — evidence | — “We demonstrate the utility of a new methodological tool, neural-network word embedding models, for large-scale text analysis, revealing how these models produce richer insights into cultural associations and categories than possible with prior methods.” |
| transfer | from linguistics (natural language processing): word-embedding vector-space models (Mikolov/Pennington), applied to measure cultural meaning (class, gender, race)— “embeddings are high-dimensional vector-space models of text in which each unique word in the corpus is represented as a vector in a shared vector space” [checked against the archived source, not this rendering] | transfers[0] |
| mechanism | building cultural dimensions as averaged vector-differences between antonym word pairs, then projecting words onto them— “we select word pairs such that the difference between each word in a pair is a “step” along the dimension of interest.” | mechanisms[0] |
| builds-on | word2vec (CBOW/skip-gram) and GloVe (from NLP)— “primarily utilize the CBOW and skip-gram models” | claim_relations[0] |
| extends | interpretive text analysis of culture (adding reproducibility)— “suffers from clear limitations in reproducibility” | claim_relations[1] |
| extends | Bolukbasi et al. (2016) and Caliskan et al. (2017) on bias in word embeddings— “Their analysis, however, like Bolukbasi et al. (2016) and Caliskan et al. (2017) remains couched in the analysis of bias rather than cultural categories in general.” | claim_relations[2] |
| contradicts | Bolukbasi et al. (2016) proposal to de-bias word embeddings— “Thus, it is rarely in the interest of cultural analyst to “de-bias” a word embedding model as Bolukbasi et al.” | claim_relations[3] |
| extends | Pierre Bourdieu's relational field theory / correspondence analysis— “Vector-space representations of culture both resonate and contend with more contemporary relational field theory developed by Pierre Bourdieu.” | claim_relations[4] |
| extends | Osgood's semantic differential method— “Like Bourdieu’s low-dimensional correspondence analysis of survey responses, our use of word embeddings shares much in common with Osgood’s semantic differential method, which similarly rates words along cultural dimensions.” | claim_relations[5] |
| sense | meaning - OPERATIONALISED AS GEOMETRY: semantic relations between words represented as geometric relationships between vectors, which the paper calls a relational model of meaning— “Word embeddings represent semantic relations between words as geometric relationships between vectors in a high-dimensional space, operationalizing a relational model of meaning” | senses[0] |
| sense | semantic change - measured as CHANGE IN POSITION: a word's rate of semantic change taken to be the change in its overall position in the vector space— “models to investigate how a word’s rate of semantic change, measured as change in the word’s overall position in the vector space” | senses[1] |
| sense | cultural categories - held by the intersectionality literature to be inseparable: their meaning changes as they overlap, so they cannot be isolated and understood independently— “The fundamental insight of the intersectionality literature is that cultural categories, particularly those of identity, cannot be isolated and understood independently” | senses[2] |
| control | the interpretation space sampled SYSTEMATICALLY - 1,775 antonym pairs taken from WordNet - rather than by selecting salient dimensions, with the reported dimensions restricted to those at the smallest angle to the focal ones — held — rules out CHERRY-PICKING BY THE ANALYST. The paper names the charge it is answering: that researchers 'conveniently cherry-pick explanations to confirm…'. An unsupervised model yields far more readable structure than any paper reports, so the choice of what to report is where the freedom lives— “To avoid this temptation, we attempt to systematically sample from the space of possible interpretations by extracting a list of 1775 antonym pairs from WordNet” | controls[0] |
| edge | semantic relations —operationalised-as→ word embeddings— “Word embeddings represent semantic relations between words as geometric relationships between vectors in a high-dimensional space, operationalizing a relational model of meaning” | concept_edges[0] |
| outcome (stated by the tagger, no quotation) | control — held | controls[0] |
| for which use | limitation — any application to a small corpus: WORD EMBEDDINGS ... REQUIRE A RELATIVELY LARGE BODY OF TEXT IF THE OUTPUT VECTOR SPACE IS TO CAPTURE SUBTLE AND COMPLEX ASSOCIATIONS— “A current limitation for the application of word embeddings is that they require a relatively large body of text if the output vector space is to capture subtle and complex associations of greatest interest to analysts of culture.” | limitations[0] |
| for which use | limitation — the absence of race from the historical analysis: UNDERTAKING THIS TASK EFFECTIVELY WOULD REQUIRE A DEEPER ENGAGEMENT WITH LINGUISTIC SHIFTS IN RACIAL TERMINOLOGY THAN IS FEASIBLE WITHIN THE SPACE CONSTRAINTS - the dimension is omitted for a vocabulary reason— “Undertaking this task effectively would require a deeper engagement with linguistic shifts in racial terminology than is feasible within the space constraints of this paper.” | limitations[1] |
| for which use | limitation — the survey validation specifically: OUR SAMPLE CANNOT BE SAID TO BE REPRESENTATIVE OF THE GENERAL POPULATION OF THE UNITED STATES— “Although our sample cannot be said to be representative of the general population of the United States, responses to basic demographic questions indicate wide diversity in age, gender, and racial composition in the sample.” | limitations[2] |
| for which use | limitation — every substantive claim about culture: WORD EMBEDDINGS MODELS, BY DISTILLING INFORMATION FROM LOCAL WORD CO-OCCURRENCES, DO NOT PROVIDE DIRECT INFORMATION ABOUT MATERIAL, ON-THE-GROUND REALITIES— “Once again, we caution that word embeddings models, by distilling information from local word co-occurrences, do not provide direct information about material, on-the-ground realities.” | limitations[3] |
| evidence status note | limitation — THE QUOTE IS AN ASSUMPTION, NOT A FINDING - the sentence opens 'If we assume that the texts' before it, and a page number ('29') sits between that opening and the quote.— “our word embedding model are observations drawn from an independent and identically distributed (i.i.d.) population of cultural observations” | limitations[4] |
| for which use | limitation — the confidence or credible intervals reported for distances and projections here: they follow from assuming the underlying texts are i.i.d. observations drawn from a population of cultural observations— “our word embedding model are observations drawn from an independent and identically distributed (i.i.d.) population of cultural observations” | limitations[4] |
| for which use | limitation — the comparison between close reading and embedding-based analysis: close reading is credited with advancing the field and criticised for clear limitations in reproducibility— “in which the researcher draws insights from a holistic deep reading of text, has done much to advance sociological understandings of culture, but suffers from clear limitations in reproducibility” | limitations[5] |
| not machine-vouched | transfer — not-in-our-copy - The quote is in the original web page we fetched for this paper, but not in the text we tag against, which was extracted differently. Checked with ligatures, hyphens, spacing and punctuation set aside.— “embeddings are high-dimensional vector-space models of text in which each unique word in the corpus is represented as a vector in a shared vector space” [checked against the archived source, not this rendering] | transfers[0] |
| replication | untested |
| limitation | The method needs a large corpus; subtle and complex cultural associations cannot be recovered from small bodies of text.— “A current limitation for the application of word embeddings is that they require a relatively large body of text if the output vector space is to capture subtle and complex associations of greatest interest to analysts of culture.” | limitations[0] |
| limitation | Race could not be traced historically because the terminology shifted across the century, so the historical analysis was restricted to gender and class.— “Undertaking this task effectively would require a deeper engagement with linguistic shifts in racial terminology than is feasible within the space constraints of this paper.” | limitations[1] |
| limitation | The validating survey used a Mechanical Turk sample that is not representative of the US population.— “Although our sample cannot be said to be representative of the general population of the United States, responses to basic demographic questions indicate wide diversity in age, gender, and racial composition in the sample.” | limitations[2] |
| limitation | Embedding projections capture semantic association in text, not material or on-the-ground social realities.— “Once again, we caution that word embeddings models, by distilling information from local word co-occurrences, do not provide direct information about material, on-the-ground realities.” | limitations[3] |
| limitation | The confidence intervals depend on treating books and newspapers as independent draws from a population of cultural observations, which is stated as a condition rather than shown.— “our word embedding model are observations drawn from an independent and identically distributed (i.i.d.) population of cultural observations” | limitations[4] |
| limitation | The method the paper displaces is criticised on reproducibility, which is the ground on which the new one is offered.— “in which the researcher draws insights from a holistic deep reading of text, has done much to advance sociological understandings of culture, but suffers from clear limitations in reproducibility” | limitations[5] |