Information Processing in Living Systems Gašper Tkačik William Bialek

Tag layer of the DHSS Meta-Tagging Project. The article text is reproduced from the staged copy named in sourceDesc and is not altered; the tags are held separately in standOff so that overlapping and nested spans are preserved.

 Information Processing in Living Systems 2016 Annual Review of Condensed Matter Physics / arXiv 10.1146/annurev-conmatphys-031214-014803 1412.8752 tkacik2016-information-living tkacik2016-information-living.txt, the project's staged copy a1132285c72349147f5b8fd868e36c6b684c9fce arxiv-open biology

the paper's own words

the tag layer; a sentence written by the tagger about the paper, always anchored to a quotation but not itself a quotation

 0 spans located by searching the stored text for the quote's exact words, allowing any whitespace between them Quotation withheld because this record is not cleared for redistribution. Quotation withheld because this record is not cleared for redistribution. review Quotation withheld because this record is not cleared for redistribution. THE PAPER'S QUESTION IS WHETHER AN INTUITION CAN BE FORMALISED, AND IT IS ASKED IN THOSE WORDS: in all these cases INTUITION TELLS US that information is FLOWING from the environment to a representation internal to the organism - CAN THIS INTUITION BE FORMALIZED? IF SO, CAN IT BE CONNECTED, QUANTITATIVELY, TO EXPERIMENTAL DATA? The scare quotes on `flowing` are the paper's, and they mark exactly what has to be earned Quotation withheld because this record is not cleared for redistribution. THE ABSTRACT VOCABULARY IS SHOWN TO BE NEEDED ONLY WHERE A CONCRETE ONE IS MISSING: in a ferromagnet what carries information over long distances is the ORDER PARAMETER - and PRECISELY BECAUSE WE CAN IDENTIFY THE ORDER PARAMETER, WE DON'T NEED TO SPEAK ABSTRACTLY ABOUT INFORMATION. Rather than turning CARRY INFORMATION into a precise mathematical statement, one just computes correlation functions. The general term is a substitute for a specific one that is absent Quotation withheld because this record is not cleared for redistribution. AND THE CONDITIONS UNDER WHICH THE ABSTRACT VOCABULARY BECOMES NECESSARY ARE SPELLED OUT: for most biological systems finding something like an order parameter REMAINS A DISTANT HOPE, SYMMETRIES ARE ABSENT, and EVEN LOCALITY IS NOT MUCH OF A GUIDE - UNDER THESE CONDITIONS, some MORE GENERAL APPROACHES to formalizing the intuition WOULD BE USEFUL. The need for the general term is derived from three specific things the field does not have Quotation withheld because this record is not cleared for redistribution. A DISCIPLINARY TRADITION IS NAMED AS THE REASON FOR A TRADE-OFF, AND THE COST IS STATED IN THE SAME SENTENCE: Quotation withheld What is being traded, and on whose authority, are both on the page Quotation withheld because this record is not cleared for redistribution. WHICH PARTS OF A TRANSFERRED FORMALISM ARE USEFUL DEPENDS ON THE DESTINATION FIELD, AND THE MISMATCH IS STATED: the ideas MOST WIDELY USED in thinking about biological systems are NOT NECESSARILY THOSE THAT HAVE BEEN MOST USEFUL in the enormously successful application of information theory to communication technology - SO OUR PERSPECTIVE HERE IS SOMEWHAT DIFFERENT from the standard text's Quotation withheld because this record is not cleared for redistribution. THE MEASURE IS FORCED BY POSTULATES RATHER THAN CHOSEN, AND THE POSTULATES ARE LISTED FIRST: ANY ACCEPTABLE MEASURE OF INFORMATION MUST OBEY SOME CONSTRAINTS - it must grow with the number of equally likely answers; it must ADD across independent parts; and it must be the weighted sum along the paths of a tree of choices - and REMARKABLY, THESE POSTULATES ALLOW ONLY ONE POSSIBLE MEASURE OF INFORMATION, THE ENTROPY. A definition derived from desiderata, not stipulated about this quote: EXTENDED: the first cut stopped four words short of "the entropy" - the answer the tag is about. Widening rather than weakening. The span ends on a comma because the next token is a footnote marker glued to a displayed formula ("answers,1 S[p] ="). Quotation withheld because this record is not cleared for redistribution. TWO SEEMINGLY DIFFERENT QUESTIONS TURN OUT TO HAVE THE SAME ANSWER, WHICH IS WHAT LICENSES ONE WORD FOR BOTH: A SEEMINGLY VERY DIFFERENT QUESTION concerns the amount of SPACE required to write down the answer - and THE ANSWER IS AGAIN THE ENTROPY. One formal object serving as the answer to a question about knowledge and to a question about storage Quotation withheld because this record is not cleared for redistribution. AN AMBIGUITY IN THE UNITS IS NAMED, RESOLVED BY CONVENTION, AND THE NAME IS ATTACHED TO THE CONVENTION: THERE IS SOME AMBIGUITY ABOUT UNITS - the alphabet is a choice and the constant is arbitrary - BUT if a binary alphabet is chosen and the constant set to one, ALL OF THE AMBIGUITY IS RESOLVED, and measured in these units they ARE CALLED BITS. The familiar name names a settled convention, not a quantity Quotation withheld because this record is not cleared for redistribution. THE SAME QUANTITY IS GIVEN TWO DESCRIPTIONS OF OPPOSITE VALENCE AND ONE IS PREFERRED: Shannon's theorem MAKES PRECISE THE VAGUE STATEMENT made in many statistical mechanics classes, that entropy is related to our LACK of information about the microscopic state - and THE MORE POSITIVE VERSION IS THAT THE ENTROPY IS THE INFORMATION WE WOULD GAIN upon learning the full state. One object, a lack and a gain, and the author says which framing he prefers Quotation withheld because this record is not cleared for redistribution. A PREDICTION IS RATED BY WHAT IT DID NOT NEED RATHER THAN BY HOW WELL IT FIT: the measured input/output relation agreed REMARKABLY with the theory, ESPECIALLY CONSIDERING THAT THERE ARE NO FREE PARAMETERS - and the result is credited with having INSPIRED THE COMMUNITY TO TAKE THESE IDEAS MORE SERIOUSLY. The absence of adjustable quantities is what the agreement is worth about this quote: EXTENDED BACKWARDS so the span carries the method as well as the verdict. NOTE ON HOW THIS NEARLY WENT WRONG: my first attempt at the start key was "By MEASURING natural scenes" and the paper says "By SAMPLING natural scenes". I typed the verb from memory instead of slicing it, the uniqueness assertion caught it, and the correct word came from the file. Slice, never type - including the words you are certain of. Quotation withheld because this record is not cleared for redistribution. what this limits: the level of biological detail in every claim: WE FOLLOW THE PHYSICS TRADITION OF EMPHASIZING THE UNITY OF THEORETICAL IDEAS AT THE EXPENSE OF SOME OF THE BIOLOGICAL COMPLEXITY Quotation withheld because this record is not cleared for redistribution. what this limits: every entropy or mutual-information estimate: THE DIFFICULTY OF ACTUALLY ESTIMATING THE ENTROPY OF A PROBABILITY DISTRIBUTION FROM A LIMITED SET OF SAMPLES Quotation withheld because this record is not cleared for redistribution. what this limits: the sufficiency of the information-theoretic framing itself: ITS UNIVERSALITY IS ITS STRENGTH, BUT IT ALSO LEAVES ONE WITH AN UNCOMFORTABLE FEELING THAT THERE IS MORE TO THE STORY - not all bits are equal Quotation withheld because this record is not cleared for redistribution. what this limits: the maximum-entropy protein-folding results: THERE ARE MANY OPEN QUESTIONS ABOUT THE SUFFICIENCY OF THE AVAILABLE DATA AND THE APPROXIMATIONS Quotation withheld because this record is not cleared for redistribution. what this limits: the scope of the review: the links to statistical mechanics and field theory ARE BEYOND THE SCOPE OF THIS REVIEW Quotation withheld because this record is not cleared for redistribution. what this limits: the scope of information-theoretic claims about biology: the authors suggest it is not widely appreciated what the theory does and does not cover, perhaps because Shannon titled his work a mathematical theory of communication Quotation withheld because this record is not cleared for redistribution. what this limits: the use of information-theoretic measures here: they are adopted because one may be unsure which features are most relevant and may not know the biologically meaningful metric, and these measures are completely general Quotation withheld because this record is not cleared for redistribution. what this limits: the claim that information-theoretic clustering recovers meaning: the authors write that it is hard to resist saying so, which reports the temptation rather than making the claim Quotation withheld because this record is not cleared for redistribution. Quotation withheld because this record is not cleared for redistribution. mutual information Quotation withheld because this record is not cleared for redistribution. utility Quotation withheld because this record is not cleared for redistribution. efficient representation Quotation withheld because this record is not cleared for redistribution. efficient coding Quotation withheld because this record is not cleared for redistribution. sequence logos Quotation withheld because this record is not cleared for redistribution. NO BOUNDARY CAN BE DRAWN - the CONCEPTUAL form of the negative relation / This is a claim about the concepts themselves, not about this study's data. No edge type can express it: all eight assert that a relation HOLDS. The boundary fails because of a TIMESCALE - the response adjusts so fast that the two cannot be told apart - so the distinction is real in principle and unusable in practice. Quotation withheld because this record is not cleared for redistribution. THREE PAPERS, THREE DIFFERENT VERDICTS ON WHETHER A FORMALISM CAN BE SHOWN TO CAPTURE THE INFORMAL NOTION IT REPLACES - and no type in the vocabulary can express a disagreement about what KIND OF CLAIM something is. This paper reports that a theorem DID make a vague statement precise - the one made in many statistical mechanics classes - and then states the more positive version of it. turing1936-computable-numbers argues the identification rather than proving it; shor1997-factoring says making it a theorem would weaken it. 'same-as' and 'distinct-from' are both false here: the three are not making one claim or two, they are ruling differently on whether the identification is the sort of thing a proof can settle. All eight edge types assert that a relation holds between two concepts. Recorded per record, each row quoting only its own paper; the cross-paper half is this paraphrase, checkable by opening the other two., and the third case for an earlier note of ours - which by the rule that one is an observation and three make a proposal is the point at which the gap should be put to Shir. Quotation withheld because this record is not cleared for redistribution. Quotation withheld because this record is not cleared for redistribution. Quotation withheld because this record is not cleared for redistribution. Quotation withheld because this record is not cleared for redistribution. Quotation withheld because this record is not cleared for redistribution. mutual information utility strength of the relation: 1 Audit 2026-09-28: DOUBTFUL - read against its full span and kept, but the span says not all bits MAY be of equal value or utility; hedged. Quotation withheld because this record is not cleared for redistribution. Quotation withheld because this record is not cleared for redistribution. Quotation withheld because this record is not cleared for redistribution. Quotation withheld because this record is not cleared for redistribution. Quotation withheld because this record is not cleared for redistribution. Quotation withheld because this record is not cleared for redistribution. Quotation withheld because this record is not cleared for redistribution. Quotation withheld because this record is not cleared for redistribution. transcription factor concentration Quotation withheld because this record is not cleared for redistribution. linear correlation coefficient Quotation withheld because this record is not cleared for redistribution. mutual information Quotation withheld because this record is not cleared for redistribution. channel capacity Quotation withheld because this record is not cleared for redistribution. entropy Quotation withheld because this record is not cleared for redistribution. small-noise approximation Quotation withheld because this record is not cleared for redistribution. Bayesian about this quote: QUOTE REPAIRED: the extractor split a hyphen-space and the quote was recorded faithfully with the damage in it, which made it unanchorable. The repaired string is an EXACT SUBSTRING of the staged text - that is why it was accepted, not because a rule produced it. The text as originally recorded is kept in evidence as extracted. Quotation withheld because this record is not cleared for redistribution. correlation about this quote: QUOTE RECOVERED FROM THE PAPER'S OWN CHARACTERS. The recorded quote was LETTERS-ONLY IDENTICAL to this stretch of the staged text but not an exact substring of it - spaces, hyphens, ligature glyphs or mathematical operators differed, so it could never anchor. This is the file's own text between the same two letters: not produced by a rule about the damage, copied out of the file. Round-trip checked (the recovered span is letters-only equal to the original) and length-checked. The text as originally recorded is kept in evidence as extracted. Quotation withheld because this record is not cleared for redistribution. mean about this quote: QUOTE REPAIRED: the extractor split a ligature and the quote was recorded faithfully with the damage in it, which made it unanchorable. The repaired string is an EXACT SUBSTRING of the staged text - that is why it was accepted, not because a rule produced it. The text as originally recorded is kept in evidence as extracted. Quotation withheld because this record is not cleared for redistribution.

The full text of this article is not redistributable, so it is not included here. The evidence sentences are withheld with it; the tag layer and its grounding status remain. The article itself is at https://doi.org/10.1146/annurev-conmatphys-031214-014803.