Latent Dirichlet Allocation

Latent Dirichlet Allocation David M. Blei Andrew Y. Ng Michael I. Jordan

Tag layer of the DHSS Meta-Tagging Project. The article text is reproduced from the staged copy named in sourceDesc and is not altered; the tags are held separately in standOff so that overlapping and nested spans are preserved.

Latent Dirichlet Allocation 2003 Journal of Machine Learning Research blei2003-lda blei2003-lda.txt, the project's staged copy 78613f588f26bd4f36d1cedcc749bee882240fa3 open-access digital-humanities

the paper's own words

the tag layer; a sentence written by the tagger about the paper, always anchored to a quotation but not itself a quotation

0 spans located by searching the stored text for the quote's exact words, allowing any whitespace between them method Quotation withheld because this record is not cleared for redistribution. topic modelling of document collections Quotation withheld because this record is not cleared for redistribution. system-or-tool-building system-or-tool-building Quotation withheld because this record is not cleared for redistribution. mechanisms a generative probabilistic model: each document is a finite mixture over latent topics, and each topic a distribution over words Quotation withheld because this record is not cleared for redistribution. The bag-of-words (exchangeability) assumption can split words that belong to a single phrase across different topics. what this limits: every topic assignment over multi-word names and phrases: THE BAG-OF-WORDS ASSUMPTION ALLOWS WORDS THAT SHOULD BE GENERATED BY THE SAME TOPIC (E.G., 'WILLIAM RANDOLPH HEARST FOUNDATION') TO BE ALLOCATED TO SEVERAL DIFFERENT TOPICS Quotation withheld because this record is not cleared for redistribution. The text-classification results are presented as preliminary and needing further support. what this limits: the text-classification result: ALTHOUGH THESE RESULTS NEED FURTHER SUBSTANTIATION, THEY SUGGEST THAT THE TOPIC-BASED REPRESENTATION ... MAY BE USEFUL AS A FAST FILTERING ALGORITHM Quotation withheld because this record is not cleared for redistribution. Exact posterior inference in LDA cannot be done; only approximate inference is available. what this limits: any topic solution treated as the posterior: EXACT INFERENCE IS INTRACTABLE FOR LDA, and only approximate inference algorithms are available Quotation withheld because this record is not cleared for redistribution. The authors disclaim any epistemological claim about the variables they call topics. what this limits: any interpretation of a fitted topic as a theme or subject: the authors state they use the word topics to exploit text-oriented intuitions and make no epistemological claims about these latent variables beyond their utility in representing probability distributions on sets of words Quotation withheld because this record is not cleared for redistribution. The text vocabulary is stated to be an aid to intuition, and the model is not tied to text. what this limits: any humanities reading of this model's output: the authors state the language of text collections is used to guide intuition and that the LDA model is not necessarily tied to text Quotation withheld because this record is not cleared for redistribution. Documents as well as words are assumed exchangeable, which the authors note is less often stated. what this limits: any use of a topic model over a corpus with an order that matters, such as a diachronic study: the methods assume documents are exchangeable, so the specific ordering of documents in a corpus can be neglected, which the authors note is less often stated formally Quotation withheld because this record is not cleared for redistribution. the document-and-words structure of latent Dirichlet allocation carried over to collaborative filtering, with a user standing in for a document and the movies chosen standing in for its words Quotation withheld because this record is not cleared for redistribution. density estimation evaluated by likelihood on a held-out test set, with the corpora treated as unlabelled eliminates: a model rewarded for memorising the documents it was fitted to · outcome (stated by the tagger, no quotation): held Quotation withheld because this record is not cleared for redistribution. latent multinomial variables latent multinomial variables Quotation withheld because this record is not cleared for redistribution. exchangeability exchangeability Quotation withheld because this record is not cleared for redistribution. bag-of-words bag-of-words Quotation withheld because this record is not cleared for redistribution. perplexity perplexity Quotation withheld because this record is not cleared for redistribution. topic topic Quotation withheld because this record is not cleared for redistribution. a topic - A LATENT MULTINOMIAL VARIABLE, NAMED TO BORROW A MEANING IT DOES NOT CLAIM: the authors say they call these variables topics in order to exploit text-oriented intuitions, and in the same sentence make no epistemological claims about them beyond representing probability distributions over words; the word appears about a hundred times in the body and the disclaimer once, in a footnote Quotation withheld because this record is not cleared for redistribution. the language of text collections - A GUIDE TO INTUITION, NOT A DOMAIN: words, documents and corpora are used throughout because they help intuition, while the model itself is stated not to be necessarily tied to text and is applied to collaborative filtering and other collections Quotation withheld because this record is not cleared for redistribution. exchangeability - TWO ASSUMPTIONS, ONE OF THEM USUALLY LEFT UNSAID: words within a document are exchangeable, which is the bag-of-words assumption everyone states, and DOCUMENTS are exchangeable too, which the authors say is less often stated formally - the second is what removes time from a corpus Quotation withheld because this record is not cleared for redistribution. claim_type: method a second reading reaching the same value. Recorded: method — note: A BLIND second reading by the project on 2026-08-25: the record's own claim type/method/phenomenon were not looked at until every value here was written and substring-verified. Not applied - a person decides. Sampling frame: the 504 staged records carrying no alternatives block, so this never re-measures the 202. All 11 records with 130+ tags were taken as a census and 19 more spread one per discipline across 22 disciplines - the dense non-neuroscience cell is the one the 202 do not contain (inside them, 94-100% of the densest bands are neuroscience; among these 11, one is)., Quotation withheld because this record is not cleared for redistribution. method: probabilistic-modelling a second reading DIFFERING from the recorded value. Recorded: system-or-tool-building — note: A BLIND second reading by the project on 2026-08-25: the record's own claim type/method/phenomenon were not looked at until every value here was written and substring-verified. Not applied - a person decides. Sampling frame: the 504 staged records carrying no alternatives block, so this never re-measures the 202. All 11 records with 130+ tags were taken as a census and 19 more spread one per discipline across 22 disciplines - the dense non-neuroscience cell is the one the 202 do not contain (inside them, 94-100% of the densest bands are neuroscience; among these 11, one is)., Quotation withheld because this record is not cleared for redistribution. phenomenon: latent topic structure in collections of discrete data such as text corpora a second reading DIFFERING from the recorded value. Recorded: topic modelling of document collections — note: A BLIND second reading by the project on 2026-08-25: the record's own claim type/method/phenomenon were not looked at until every value here was written and substring-verified. Not applied - a person decides. Sampling frame: the 504 staged records carrying no alternatives block, so this never re-measures the 202. All 11 records with 130+ tags were taken as a census and 19 more spread one per discipline across 22 disciplines - the dense non-neuroscience cell is the one the 202 do not contain (inside them, 94-100% of the densest bands are neuroscience; among these 11, one is)., Quotation withheld because this record is not cleared for redistribution. builds-on → hierarchical Bayesian modeling Quotation withheld because this record is not cleared for redistribution. contradicts → LSI (latent semantic indexing) methodology Quotation withheld because this record is not cleared for redistribution. extends → Hofmann (1999) probabilistic latent semantic indexing (pLSI) model Quotation withheld because this record is not cleared for redistribution. contradicts → Hofmann's pLSI treatment of documents differs in: THE PARAMETER IS THE LEVEL AT WHICH THE MODEL IS PROBABILISTIC. The span: 'While Hofmann's work is a useful step toward probabilistic modeling of text, it is incomplete Quotation withheld.' The contradiction is not global - pLSI is granted as a useful step - and it turns on one named thing. Quotation withheld because this record is not cleared for redistribution. builds-on → de Finetti (1990) representation theorem for exchangeable random variables Quotation withheld because this record is not cleared for redistribution. builds-on → Nigam et al. (2000) mixture of unigrams model Quotation withheld because this record is not cleared for redistribution. topic —operationalised-as→ latent multinomial variables topic latent multinomial variables strength of the relation: 1 Quotation withheld because this record is not cleared for redistribution. bag-of-words —same-as→ exchangeability bag-of-words exchangeability strength of the relation: 1 Quotation withheld because this record is not cleared for redistribution. bag-of-words —renamed-across-fields→ exchangeability bag-of-words exchangeability the relation itself is separately evidenced · strength of the relation: 1 · note: ADDS the better-typed edge beside the record's 'same-as'. The same-as is defensible and is left alone; this one says WHY they are the same - it is one assumption renamed between fields. · the relation stated: ONE assumption under two field vocabularies - information retrieval calls it bag-of-words, probability theory calls it exchangeability Quotation withheld because this record is not cleared for redistribution. LDA is a generative probabilistic model for text corpora: a three-level hierarchical Bayesian model in which each item is a finite mixture over an underlying set of topics. Quotation withheld because this record is not cleared for redistribution. Inference uses efficient approximate variational methods and an EM algorithm; results in document modeling, text classification and collaborative filtering are compared with a mixture-of-unigrams model and probabilistic LSI. Quotation withheld because this record is not cleared for redistribution. latent Dirichlet allocation (LDA) latent Dirichlet allocation (LDA) Quotation withheld because this record is not cleared for redistribution. topic topic Quotation withheld because this record is not cleared for redistribution. tf-idf scheme tf-idf scheme about this quote: QUOTE RECOVERED FROM THE PAPER'S OWN CHARACTERS. The recorded quote was LETTERS-ONLY IDENTICAL to this stretch of the staged text but not an exact substring of it - spaces, hyphens, ligature glyphs or mathematical operators differed, so it could never anchor. This is the file's own text between the same two letters: not produced by a rule about the damage, copied out of the file. Round-trip checked (the recovered span is letters-only equal to the original) and length-checked. The text as originally recorded is kept in evidence as extracted. Quotation withheld because this record is not cleared for redistribution. Bayesian Bayesian about this quote: QUOTE REPAIRED: the extractor split a unicode and the quote was recorded faithfully with the damage in it, which made it unanchorable. The repaired string is an EXACT SUBSTRING of the staged text - that is why it was accepted, not because a rule produced it. The text as originally recorded is kept in evidence as extracted. Quotation withheld because this record is not cleared for redistribution. mean mean Quotation withheld because this record is not cleared for redistribution.

The full text of this article is not redistributable, so it is not included here. The evidence sentences are withheld with it; the tag layer and its grounding status remain. The article itself is at https://jmlr.org/papers/v3/blei03a.html.