Visualization of Categorization: How to See the Wood and the Trees

Visualization of Categorization: How to See the Wood and the Trees Ophir Münz-Manor Itay Marienberg-Milikowsky

Tag layer of the DHSS Meta-Tagging Project. The article text is reproduced from the staged copy named in sourceDesc and is not altered; the tags are held separately in standOff so that overlapping and nested spans are preserved.

Visualization of Categorization: How to See the Wood and the Trees 2023 Digital Humanities Quarterly 17(3) munzmanor2023-visualization-categorization munzmanor2023-visualization-categorization.txt, the project's staged copy 2c0fd8a7e2a761855119d27ea7425760e5ce2a1e open-access digital-humanities

the paper's own words

the tag layer; a sentence written by the tagger about the paper, always anchored to a quotation but not itself a quotation

0 spans located by searching the stored text for the quote's exact words, allowing any whitespace between them Figurative language in a corpus of late-antique Hebrew liturgical poetry, investigated from both a manual and a computational point of view. Quotation withheld because this record is not cleared for redistribution. How much room is there for sample-based research in a computationally-based annotation process, and can a particular concept of categorization support it? Quotation withheld because this record is not cleared for redistribution. Proposes a playful approach to the epistemological status of categories, in which they are returned to and re-examined critically rather than fixed at the start of annotation. Quotation withheld because this record is not cleared for redistribution. method Quotation withheld because this record is not cleared for redistribution. a hermeneutical circle in which visualizing the annotations sends the analyst back to re-assess the categories that produced them Quotation withheld because this record is not cleared for redistribution. practice-report practice-report Quotation withheld because this record is not cleared for redistribution. mechanisms the original annotations are carried into the digital tool untouched, a rule the authors adopt because whether a word is figurative is a hermeneutical decision that differs between annotators and within one annotator at a later date Quotation withheld because this record is not cleared for redistribution. mechanisms the corpus's figurative share is correlated against two further corpora newly annotated for figurative language - the Psalms and a ninth-century poet - which is what supplies an answer to the question the dissertation could not answer, small in comparison to what Quotation withheld because this record is not cleared for redistribution. mechanisms a per-genre ratio of figurative to non-figurative words, set against the corpus-wide rate, which is what converts a qualitative impression about a genre into a checkable quantity Quotation withheld because this record is not cleared for redistribution. mechanisms annotation counts are aggregated into ten-percent chunks of each text, which normalises length and is what makes a positional pattern comparable across poems of different sizes Quotation withheld because this record is not cleared for redistribution. mechanisms individual tags can be withdrawn from the chart as well as added, so a pattern is exposed by subtracting annotations rather than only by displaying them Quotation withheld because this record is not cleared for redistribution. mechanisms a tagset too large and too hierarchical to grasp in the annotation tool itself is given an interactive network-oriented representation in the separate visualisation tool Quotation withheld because this record is not cleared for redistribution. mechanisms the chart collapses and uncollapses level by level, and a level's size is drawn as the number of children it has, which is what makes the inner hierarchy of the tagset readable as shape Quotation withheld because this record is not cleared for redistribution. mechanisms the annotations of two or more annotators are viewed side by side, which is what turns the scholar's attention to the individual and subjective interpretation of each annotator instead of to a single reading of the text Quotation withheld because this record is not cleared for redistribution. mechanisms the variation between annotators is not collapsed into a gold or a silver standard; the multiplicity is visualised instead, so the disagreement is left as a question about the correct annotation rather than answered Quotation withheld because this record is not cleared for redistribution. Annotation is stated to be hermeneutical and unstable across annotators and across time, including within the same annotator. what this limits: the annotations as a fixed measurement: the paper states the same annotator may annotate differently at a later date Quotation withheld because this record is not cleared for redistribution. The original annotation phase had no well-defined tagset and the category definitions were flexible. what this limits: the dissertation-era annotations as a defined category system: the paper says no well-defined tagset existed Quotation withheld because this record is not cleared for redistribution. The examples analysed in the original work were not selected for representativity. what this limits: treating the analysed examples as a sample of the corpus: they were chosen for interest, not representativity Quotation withheld because this record is not cleared for redistribution. Moving from many annotated texts to a statement about the corpus is said to involve an almost inevitable reduction. what this limits: corpus-level generalisations from this workflow: the paper states some Quotation withheld because this record is not cleared for redistribution. The paper states that adopting the computational option carries a price rather than delivering objectivity. what this limits: reading the digital re-annotation as more objective than the manual one: the paper denies this Quotation withheld because this record is not cleared for redistribution. The paper argues that excessive textual anchoring of speculation and hypothesis carries a cost. what this limits: textual anchoring as a cost-free virtue: this paper argues excessive anchoring has a price Quotation withheld because this record is not cleared for redistribution. The exploratory visualization phase is declared unpredictable in duration and in whether it yields anything. what this limits: planning or costing the exploratory phase: the paper says its length and fruitfulness cannot be predicted Quotation withheld because this record is not cleared for redistribution. The tool interoperates with one annotation system only. what this limits: use of the tool outside CATMA: at the time of writing Quotation withheld because this record is not cleared for redistribution. A quantitative and a qualitative reading of the same genre disagree, and the paper holds both to be valid rather than resolving them. what this limits: the figurative-density figure as the genre's figurative character: the paper holds the qualitative reading equally valid Quotation withheld because this record is not cleared for redistribution. The price of a category system is named as an effect on the annotator, not on the data: students become addicted to it and are described as ending in a different intellectual position from the one they started in. what this limits: systematic annotation as a neutral instrument: the paper states the practice changes the annotator's disposition, describing students as moving from one critical position to another through annotating Quotation withheld because this record is not cleared for redistribution. The stated end state of the process is that the interpretation itself becomes rigid. what this limits: annotation followed by query and visualization as making interpretation legible: the paper states the same sequence ends with metadata becoming data and the interpretation becoming rigid Quotation withheld because this record is not cleared for redistribution. A claim in the author's own earlier dissertation is identified as incorrect by the quantitative measure, and the paper says so in print, naming the source: the genre was described as low on figurative language and measures at more than twice the corpus density. what this limits: the qualitative characterisation of a genre's figurative language: the paper reports its own earlier published assertion about the Hosha'not as incorrect once the density was counted Quotation withheld because this record is not cleared for redistribution. The reason the human reading was wrong is named, and it is not carelessness: the words are figurative but the poem does not sound figurative, so impression and count come apart. what this limits: figurative density as a proxy for how figurative a text reads: the paper states the same poem can be high in figurative words and low in figurative effect, and explains why Quotation withheld because this record is not cleared for redistribution. comparing a genre's figurative density against the overall figurative density of the whole corpus eliminates: that a genre's apparently high figurative density is the corpus-wide rate rather than a property of that genre · outcome: held Quotation withheld because this record is not cleared for redistribution. annotation annotation Quotation withheld because this record is not cleared for redistribution. tagset tagset Quotation withheld because this record is not cleared for redistribution. modeling modeling Quotation withheld because this record is not cleared for redistribution. in-line annotation in-line annotation Quotation withheld because this record is not cleared for redistribution. inter-annotator agreement inter-annotator agreement Quotation withheld because this record is not cleared for redistribution. visualization of multiplicity visualization of multiplicity Quotation withheld because this record is not cleared for redistribution. close reading close reading Quotation withheld because this record is not cleared for redistribution. distant reading distant reading Quotation withheld because this record is not cleared for redistribution. text classification text classification Quotation withheld because this record is not cleared for redistribution. CATMA CATMA Quotation withheld because this record is not cleared for redistribution. ViS-À-ViS ViS-À-ViS Quotation withheld because this record is not cleared for redistribution. category system category system Quotation withheld because this record is not cleared for redistribution. systematic annotation systematic annotation Quotation withheld because this record is not cleared for redistribution. figurative density figurative density Quotation withheld because this record is not cleared for redistribution. Hosha’not Hosha’not Quotation withheld because this record is not cleared for redistribution. categorization - built upward from the data, by collecting instances until a category that contains them is defined Quotation withheld because this record is not cleared for redistribution. categorization - built downward by formalization, by removing actual data until an abstract category remains Quotation withheld because this record is not cleared for redistribution. reading - distant reading, which the paper says is better understood as a metaphor, since meaning is extracted without reading the text Quotation withheld because this record is not cleared for redistribution. annotation - a close reading practice, based on sensitive attention to every detail in the text Quotation withheld because this record is not cleared for redistribution. annotation - a modelling operation that prepares the object for future, as-yet-unknown questions Quotation withheld because this record is not cleared for redistribution. inter-annotator variation - the driving force of interpretation, to be visualized rather than collapsed into a standard Quotation withheld because this record is not cleared for redistribution. the price of a category system - not inaccuracy but ATTRACTION: solidity, precision and caution are named as what makes it seductive, and the seduction is the cost Quotation withheld because this record is not cleared for redistribution. figurative density - a counted ratio of figurative to non-figurative words, which for one genre is 25% against a corpus figure of 11% Quotation withheld because this record is not cleared for redistribution. figurative density - the aesthetic impression of figurativeness, which the paper states can be LOW in the same poem whose counted density is high Quotation withheld because this record is not cleared for redistribution. distinguishes-from → Franco Moretti, distant reading differs in: WHETHER EACH TEXT IS INTERPRETED ON THE WAY. The method reaches the same place - a claim about a whole corpus - and the paper refuses the label anyway: 'this is NOT another version of Franco Moretti's distant reading, because, Quotation withheld.' The distinction is in the procedure, not in the output, which is why it is invisible in the result Quotation withheld because this record is not cleared for redistribution. builds-on → Rieder and Röhle (2012) differs in: WHETHER GOING COMPUTATIONAL BUYS OBJECTIVITY. The move is described as widely PERCEIVED as 'a move in the direction of objectivity, scientification, and the like', and then: 'However, as we shall see, and as other scholars have already argued, the picture is more complex, since, even in such a case, this move has a price'. The borrowed claim is that a method change costs something rather than only adding rigour Quotation withheld because this record is not cleared for redistribution. qualifies → Piper differs in: WHICH COMPUTATIONAL PRACTICE THE ARGUMENT IS PAIRED WITH. The agreement is stated first - 'Our approach here is NOT VERY DIFFERENT FROM PIPER'S' - and the difference is the instrument: projects relying on 'the systematic annotation of texts, rather than on machine learning algorithms, which - despite being based on pre-existing annotated texts - emphasize the distance from the text itself'. Same argument, different tooling, and the reason given is proximity to the text Quotation withheld because this record is not cleared for redistribution. extends → Marienberg-Milikowsky et al. (2022) differs in: WHAT TO DO WITH DISAGREEMENT BETWEEN ANNOTATORS. The prior work argued variation is the driving force of interpretation; this paper turns that into a tool behaviour: 'Instead of collapsing the variation into a gold or a silver standard, ViS-À-ViS advocates for the visualization of multiplicity ... instead of providing an answer concerning the "correct" annotation, it RAISES A QUESTION about it'. Inter-annotator disagreement stops being noise to be resolved and becomes the displayed result Quotation withheld because this record is not cleared for redistribution. annotation —is-a→ modeling annotation modeling strength of the relation: 1 Quotation withheld because this record is not cleared for redistribution. inter-annotator agreement —contrasted-with→ visualization of multiplicity inter-annotator agreement visualization of multiplicity strength of the relation: 1 Quotation withheld because this record is not cleared for redistribution. ViS-À-ViS —distinct-from→ CATMA ViS-À-ViS CATMA strength of the relation: 1 Audit 2026-09-28: DOUBTFUL - read against its full span and kept, but endpoint by anaphora: 'This reality' = CATMA's basic visualisation capabilities (previous sentence). Quotation withheld because this record is not cleared for redistribution. close reading —contrasted-with→ distant reading close reading distant reading differs in: THE AXIS IS WHETHER THE TEXT IS ACTUALLY READ. 'unlike close reading, where the scholar actually reads the text closely, distant reading - regardless of its specific history - is BETTER UNDERSTOOD AS A METAPHOR.' The contrast is not two techniques on a scale of distance; one names an activity and the other is said to name none. · strength of the relation: 1 Quotation withheld because this record is not cleared for redistribution. Visualising the annotations repeatedly forced the categories to be revised. about this quote: QUOTE RECOVERED FROM THE PAPER'S OWN CHARACTERS. The recorded quote was LETTERS-ONLY IDENTICAL to this stretch of the staged text but not an exact substring of it - spaces, hyphens, ligature glyphs or mathematical operators differed, so it could never anchor. This is the file's own text between the same two letters: not produced by a rule about the damage, copied out of the file. Round-trip checked (the recovered span is letters-only equal to the original) and length-checked. The text as originally recorded is kept in evidence as extracted. Quotation withheld because this record is not cleared for redistribution. Categorisation and visualisation form a hermeneutic circle: each changes the other. Quotation withheld because this record is not cleared for redistribution. The first annotation pass had a working notion of categories but no defined tagset — the tagset came later. about this quote: QUOTE RECOVERED FROM THE PAPER'S OWN CHARACTERS. The recorded quote was LETTERS-ONLY IDENTICAL to this stretch of the staged text but not an exact substring of it - spaces, hyphens, ligature glyphs or mathematical operators differed, so it could never anchor. This is the file's own text between the same two letters: not produced by a rule about the damage, copied out of the file. Round-trip checked (the recovered span is letters-only equal to the original) and length-checked. The text as originally recorded is kept in evidence as extracted. Quotation withheld because this record is not cleared for redistribution. The initial scheme was flat: four devices with no hierarchy between them. Quotation withheld because this record is not cleared for redistribution. 25% — figurative density of the Hosha'not genre: one word in four Quotation withheld because this record is not cleared for redistribution. 11% — figurative density of the whole corpus, the comparison the 25% stands out against Quotation withheld because this record is not cleared for redistribution.

The full text of this article is not redistributable, so it is not included here. The evidence sentences are withheld with it; the tag layer and its grounding status remain. The article itself is at .