A Mathematical Theory of Communication

A Mathematical Theory of Communication Claude E. Shannon

Tag layer of the DHSS Meta-Tagging Project. The article text is reproduced from the staged copy named in sourceDesc and is not altered; the tags are held separately in standOff so that overlapping and nested spans are preserved.

A Mathematical Theory of Communication 1948 The Bell System Technical Journal 10.1002/j.1538-7305.1948.tb01338.x shannon1948-communication shannon1948-communication.txt, the project's staged copy 3fe4a713ba8937e6ae2723de74bb2eee7be7b9f2 open-access information-theory

the paper's own words

the tag layer; a sentence written by the tagger about the paper, always anchored to a quotation but not itself a quotation

0 spans located by searching the stored text for the quote's exact words, allowing any whitespace between them existence Quotation withheld because this record is not cleared for redistribution. information transmission over noisy channels Quotation withheld because this record is not cleared for redistribution. mathematical-proof mathematical-proof Quotation withheld because this record is not cleared for redistribution. mechanisms quantify information as the logarithm of the number of possible messages (entropy); a noisy channel has a definite capacity C that bounds reliable transmission Quotation withheld because this record is not cleared for redistribution. the theory deliberately excludes the semantic/meaning aspects of communication what this limits: everything the theory is ever used to say about content: THESE SEMANTIC Quotation withheld - meaning is declared out of scope in the founding paper Quotation withheld because this record is not cleared for redistribution. Meaning is explicitly excluded from the theory's scope. duplicate of: limitations[0] · what this limits: the same scope as limitations[0] - meaning is excluded by construction, not by oversight Quotation withheld because this record is not cleared for redistribution. The continuous-case results are deliberately not developed with full generality or measure-theoretic rigour. what this limits: the continuous-case results: WE WILL NOT ATTEMPT, IN THE CONTINUOUS CASE, TO OBTAIN OUR RESULTS WITH THE GREATEST GENERALITY, OR WITH THE EXTREME RIGOR OF PURE MATHEMATICS Quotation withheld because this record is not cleared for redistribution. The existence proof of the noisy-channel coding theorem does not yield a usable construction. what this limits: any practical reading of the noisy-channel coding theorem: AN ATTEMPT TO OBTAIN A GOOD APPROXIMATION TO IDEAL CODING BY FOLLOWING THE METHOD OF THE PROOF IS GENERALLY IMPRACTICAL - existence, not construction Quotation withheld because this record is not cleared for redistribution. For non-white noise the capacity cannot be computed explicitly, only bounded. what this limits: capacity results for arbitrary noise: IT DOES NOT APPEAR THAT THE MAXIMIZING PROBLEM INVOLVED IN DETERMINING THE CHANNEL CAPACITY C CAN BE S[olved] explicitly Quotation withheld because this record is not cleared for redistribution. Source rates relative to a fidelity criterion have only been computed in a handful of simple cases. what this limits: the rate-versus-fidelity theory: THE ACTUAL CALCULATION OF RATES HAS BEEN CARRIED OUT IN ONLY A FEW VERY SIMPLE CASES Quotation withheld because this record is not cleared for redistribution. Shannon states that the uniqueness theorem for H, and the assumptions behind it, are in no way necessary to the theory; the justification is deferred to the implications of the definitions. what this limits: the axiomatic derivation of H as the theory's foundation: the paper says the theorem is not necessary and that the definitions are justified by their implications instead Quotation withheld because this record is not cleared for redistribution. The 50% redundancy figure for English is bounded to statistical structure within about eight letters. what this limits: the 50% redundancy of English as an unqualified figure: it excludes statistical structure beyond about eight letters Quotation withheld because this record is not cleared for redistribution. entropy itself - the quantity H, its form, and its identification with the H of Boltzmann's theorem, carried out of statistical mechanics and made the measure of information in a communication channel Quotation withheld because this record is not cleared for redistribution. entropy entropy Quotation withheld because this record is not cleared for redistribution. channel capacity channel capacity Quotation withheld because this record is not cleared for redistribution. bit bit Quotation withheld because this record is not cleared for redistribution. redundancy redundancy Quotation withheld because this record is not cleared for redistribution. information information Quotation withheld because this record is not cleared for redistribution. meaning meaning Quotation withheld because this record is not cleared for redistribution. uncertainty uncertainty Quotation withheld because this record is not cleared for redistribution. equivocation equivocation Quotation withheld because this record is not cleared for redistribution. communication communication Quotation withheld because this record is not cleared for redistribution. information - the engineering sense: which message was selected from a known set, with meaning explicitly set aside Quotation withheld because this record is not cleared for redistribution. information - the significant aspect for design: one message drawn from a set of possible messages, any of which the system must handle Quotation withheld because this record is not cleared for redistribution. information - a rate at which a source produces, rather than a quantity a message contains Quotation withheld because this record is not cleared for redistribution. H - one quantity under three names: the amount of choice involved in a selection, and the uncertainty of the outcome Quotation withheld because this record is not cleared for redistribution. rate of transmission - not the transmitted bits minus the errors; the naive subtraction is shown to credit a channel that carries nothing Quotation withheld because this record is not cleared for redistribution. H - a single quantity serving as the measure of three things at once: information, choice and uncertainty Quotation withheld because this record is not cleared for redistribution. redundancy - the fraction of a message fixed by the structure of the language rather than freely chosen Quotation withheld because this record is not cleared for redistribution. builds-on → Nyquist, "Certain Factors Affecting Telegraph Speed" / "Certain Topics in Telegraph Transmission Theory" and Hartley, "Transmission of Information" Quotation withheld because this record is not cleared for redistribution. extends → the communication theory of Nyquist and Hartley Quotation withheld because this record is not cleared for redistribution. builds-on → Hartley's logarithmic measure of information Quotation withheld because this record is not cleared for redistribution. builds-on → N. Wiener, Cybernetics and The Interpolation, Extrapolation and Smoothing of Stationary Time Series Quotation withheld because this record is not cleared for redistribution. replicates → R. M. Fano's coding method (Technical Report No. 65, Research Laboratory of Electronics, M.I.T.) Quotation withheld because this record is not cleared for redistribution. entropy —same-as→ uncertainty entropy uncertainty strength of the relation: 1 Audit 2026-09-28: DOUBTFUL - read against its span and kept, but the span is a question about measuring choice and uncertainty; it does not itself equate entropy with uncertainty. Quotation withheld because this record is not cleared for redistribution. EVIDENCE CORRECTED (RE-GROUNDED) · redundancy —operationalised-as→ entropy redundancy entropy EVIDENCE CORRECTED: RE-GROUNDED. The old span named only one endpoint. This sentence names both and IS the definition - redundancy is one minus relative entropy, which is exactly what the edge records. — strength of the relation: 0.8 Quotation withheld because this record is not cleared for redistribution. 5 — components of a communication system Quotation withheld because this record is not cleared for redistribution. 2^N — N binary devices store N bits Quotation withheld because this record is not cleared for redistribution. 3 — discrete, continuous, and mixed systems Quotation withheld because this record is not cleared for redistribution. semantic meaning is irrelevant to the engineering problem Quotation withheld because this record is not cleared for redistribution. the fundamental problem is reproducing a message at another point Quotation withheld because this record is not cleared for redistribution. the logarithmic function is the natural measure of information Quotation withheld because this record is not cleared for redistribution. base-2 units of information are called bits Quotation withheld because this record is not cleared for redistribution. set of possible messages set of possible messages Quotation withheld because this record is not cleared for redistribution. logarithmic base logarithmic base Quotation withheld because this record is not cleared for redistribution. the channel the channel Quotation withheld because this record is not cleared for redistribution. bit bit Quotation withheld because this record is not cleared for redistribution. communication system communication system Quotation withheld because this record is not cleared for redistribution. information source information source Quotation withheld because this record is not cleared for redistribution. discrete channel discrete channel Quotation withheld because this record is not cleared for redistribution. sample-size sample-size Quotation withheld because this record is not cleared for redistribution. EVIDENCE CORRECTED (RE-GROUNDED) · standard-deviation standard-deviation EVIDENCE CORRECTED: RE-GROUNDED. The old span quoted the ensemble expression itself, from the web page we fetched, and reached the page as 'f /#28 ai /; t/#29/= /+ ∞ ∑ n/= ...'. The grounding copy states the same fact in prose with no debris at all, and that clause - not the formula - is what attests 'standard deviation'. A TEXT EXTRACTION CANNOT GROUND A CLAIM WHOSE EVIDENCE IS AN EQUATION, so the claim is grounded on the sentence that says it in words. Quotation withheld because this record is not cleared for redistribution.

The full text of this article is not redistributable, so it is not included here. The evidence sentences are withheld with it; the tag layer and its grounding status remain. The article itself is at https://doi.org/10.1002/j.1538-7305.1948.tb01338.x.