The importance of mixed selectivity in complex cognitive tasks

The importance of mixed selectivity in complex cognitive tasks Rigotti Barak Warden Wang Daw Miller Fusi

Tag layer of the DHSS Meta-Tagging Project. The article text is reproduced from the staged copy named in sourceDesc and is not altered; the tags are held separately in standOff so that overlapping and nested spans are preserved.

The importance of mixed selectivity in complex cognitive tasks 2013 Nature 10.1038/nature12160 Rigotti2013-the-importance-of-mixed-selectivity-in-compl Rigotti2013-the-importance-of-mixed-selectivity-in-compl.txt, the project's staged copy e6237998590f96fb1bfda87fa5c9bb75222d6c21 not-redistributable neuroscience

the paper's own words

the tag layer; a sentence written by the tagger about the paper, always anchored to a quotation but not itself a quotation

0 spans located by searching the stored text for the quote's exact words, allowing any whitespace between them mixed selectivity in prefrontal cortex neurons and the dimensionality of the population representation it produces Quotation withheld because this record is not cleared for redistribution. what functional role the heterogeneous, hard-to-interpret mixed selectivity of PFC neurons plays Quotation withheld because this record is not cleared for redistribution. reinterprets seemingly disordered tuning as the signature of a high-dimensional code, measures that dimensionality through classifier capacity, and shows it collapses when the animal errs Quotation withheld because this record is not cleared for redistribution. mechanism Quotation withheld because this record is not cleared for redistribution. mixed selectivity in prefrontal cortex Quotation withheld because this record is not cleared for redistribution. electrophysiology electrophysiology Quotation withheld because this record is not cleared for redistribution. mechanisms on error trials the non-linear mixed selectivity component is disrupted, dimensionality collapses, and downstream readout neurons can no longer be driven to the correct output Quotation withheld because this record is not cleared for redistribution. mechanisms excessive variability from sources that are not task-relevant enters the non-linear mixing and changes the dimensionality status: the paper's own words: may be caused by Quotation withheld because this record is not cleared for redistribution. mechanisms diverse non-linear selectivity to mixtures of task variables raises the dimension of the representation, which is what gives linear readouts their expanded repertoire of input-output functions Quotation withheld because this record is not cleared for redistribution. mechanisms strongly non-linear mixed selectivity neurons are more sensitive to task-irrelevant variability, which is why the most useful component of the code is also the most fragile Quotation withheld because this record is not cleared for redistribution. dimensionality is estimated by counting how many binary classifications a linear classifier can implement, since that number grows exponentially with dimension Quotation withheld because this record is not cleared for redistribution. the estimate is converted by d = log2(Nc), valid when the number of inputs is large enough Quotation withheld because this record is not cleared for redistribution. only non-linear mixed selectivity can raise dimensionality: pure selectivity and linear mixing generate low-dimensional representations Quotation withheld because this record is not cleared for redistribution. classical selectivity is removed by adding noise that equalises average responses while preserving differences between individual conditions Quotation withheld because this record is not cleared for redistribution. averaging over conditions to test classical selectivity is itself what discards the distributed information Quotation withheld because this record is not cleared for redistribution. the proposed cause of the error-trial collapse: non-linear mixing is more sensitive than pure selectivity to task-irrelevant variability, so it is the most useful and the most fragile component Quotation withheld because this record is not cleared for redistribution. non-simultaneously recorded neurons are combined into pseudo-simultaneous population vectors, discarding temporal correlations Quotation withheld because this record is not cleared for redistribution. Errors could only be analysed in the recall task because the recognition task yielded too few of them. what this limits: the error analysis, which exists only for the recall task: WE TESTED THIS PREDICTION BY ANALYSING THE ERRORS MADE BY THE MONKEYS IN THE RECALL TASK (THERE WERE TOO FEW ERRORS IN THE RECOGNITION TASK) Quotation withheld because this record is not cleared for redistribution. Non-simultaneously recorded neurons were combined into pseudo-simultaneous population patterns, neglecting temporal correlations. what this limits: every population-level correlation and dimensionality claim: NEURONS THAT WERE NOT RECORDED SIMULTANEOUSLY WERE COMBINED AS PSEUDO-SIMULTANEOUS POPULATION ACTIVITY PATTERNS Quotation withheld because this record is not cleared for redistribution. The argument is framed for linear readouts; whether it holds for more complex non-linear readouts is left open. what this limits: the argument as stated, which assumes linear readouts: IT IS LEGITIMATE TO ASK WHETHER THESE CONSIDERATIONS WOULD STILL BE VALID IF WE CONSIDER MORE COMPLEX NON-LINEAR READOUTS Quotation withheld because this record is not cleared for redistribution. Dimensionality expansion by single-neuron dendritic non-linearities is not ruled out. what this limits: any claim that dimensionality expansion happens at the population level rather than within cells: OUR RESULTS DO NOT EXCLUDE THE FUNCTIONAL IMPORTANCE OF SUCH DENDRITIC PROCESSES Quotation withheld because this record is not cleared for redistribution. the measured dimensionality is bounded above by the number of experimental conditions, and the recorded data reach that maximum what this limits: the dimensionality number itself: ACCORDINGLY, THE DIMENSIONALITY IS BOUNDED BY THE TOTAL NUMBER OF DISTINCT CONDITIONS c - it cannot exceed the experiment's design Quotation withheld because this record is not cleared for redistribution. dimensionality is notoriously difficult to estimate in the presence of noise, which is why an indirect method was needed what this limits: the dimensionality estimate and the indirect method used to get it: AS THIS IS NOTORIOUSLY DIFFICULT IN THE PRESENCE OF NOISE, WE ADOPTED A NOVEL STRATEGY THAT EXPLOITS THE RELATION BETWEEN DIMENSIONALITY AND THE PERFORMANCE of a linear classifier Quotation withheld because this record is not cleared for redistribution. non-linear transformations implemented in dendrites could expand dimensionality invisibly, leaving firing-rate representations low-dimensional what this limits: the same scope as limitations[2], with the mechanism: SOME OF THE TRANSFORMATIONS WHICH INCREASE THE DIMENSIONALITY of the representation could be implemented in dendrites and would not appear in firing rates Quotation withheld because this record is not cleared for redistribution. high dimensionality is not strictly necessary for rich dynamics or complex tasks; the claim is that it simplifies circuit design what this limits: the strength of the paper's own claim: Quotation withheld Quotation withheld because this record is not cleared for redistribution. the causal account of the collapse is offered explicitly as speculation what this limits: the causal account of the collapse, labelled as speculation by the authors: IT IS TEMPTING TO SPECULATE ABOUT THE CAUSES OF THIS DIMENSIONALITY COLLAPSE Quotation withheld because this record is not cleared for redistribution. temporal correlations are discarded by the pseudo-simultaneous construction what this limits: every population-level result: NEURONS THAT WERE NOT RECORDED SIMULTANEOUSLY WERE COMBINED AS PSEUDO-SIMULTANEOUS POPULATION ACTIVITY PATTERNS, and THE TEMPORAL CORRELATIONS THAT WERE NEGLECTED are lost by that construction Quotation withheld because this record is not cleared for redistribution. the error analysis rests on 121 neurons with enough error trials, and only in the recall task what this limits: the error analyses only: THESE ANALYSES WERE CARRIED OUT ON A SUBSET DATASET OF 121 NEURONS THAT WERE RECORDED IN AS MANY CORRECT AS ERROR TRIALS DURING THE RECALL TASK Quotation withheld because this record is not cleared for redistribution. the analysis of the recorded population was designed from machine-learning theory about how learning principles operate in neuronal circuits Quotation withheld because this record is not cleared for redistribution. the coding strategy of support vector machines is used as the interpretive frame for what mixed selectivity is doing Quotation withheld because this record is not cleared for redistribution. the perceptron learning rule is imported as the supervised procedure by which a readout neuron's weights would be adjusted Quotation withheld because this record is not cleared for redistribution. the kernel-machine insight that projecting into a high-dimensional space lets a linear classifier implement more functions, imported as the interpretation of PFC tuning Quotation withheld because this record is not cleared for redistribution. counting implementable dichotomies as a measure of representational capacity, used here as the dimensionality estimator Quotation withheld because this record is not cleared for redistribution. the observation that firing rates are less variable on faster-reaction-time trials, used as the analogy for variability driving the error-trial collapse Quotation withheld because this record is not cleared for redistribution. the decoder's performance was checked against the possibility that it had saturated eliminates: that the correspondence between dimensionality and performance is a ceiling effect of the population decoder rather than a property of the representation · outcome (stated by the tagger, no quotation): held Quotation withheld because this record is not cleared for redistribution. dimensionality here is a property of CONDITION space: the minimal number of coordinate axes needed to place one firing-rate vector per experimental condition. It is not a statement about time courses across cells Quotation withheld because this record is not cleared for redistribution. high-dimensional is therefore a bounded claim in this paper - the ceiling is the number of conditions, here 24, and the data reach it Quotation withheld because this record is not cleared for redistribution. mixed selectivity means tuning to a mixture of task aspects; only the NON-LINEAR kind raises dimensionality, and the paper keeps linear and non-linear mixing strictly apart Quotation withheld because this record is not cleared for redistribution. classically selective is defined by a significant difference in averages across a task aspect - and the paper's point is that this test discards the information it is looking for Quotation withheld because this record is not cleared for redistribution. pure selectivity means responding to one aspect at a time; it is the foil, simulated rather than recorded Quotation withheld because this record is not cleared for redistribution. claim_type: mechanism a second reading reaching the same value. Recorded: mechanism — note: a second reading reaching the same value from a different sentence. Not applied - a person decides. Quotation withheld because this record is not cleared for redistribution. method: reanalysis of 237 lateral PFC neurons from two monkeys in a sequence memory task, with selectivity-removal manipulations and a dimensionality estimate from the number of implementable binary classifications a second reading DIFFERING from the recorded value. Recorded: electrophysiology — note: a second reading, verbatim-verified, disagreeing with the value on the record. Not applied - a person decides. Quotation withheld because this record is not cleared for redistribution. phenomenon: high-dimensional population coding in prefrontal cortex a second reading DIFFERING from the recorded value. Recorded: mixed selectivity in prefrontal cortex — note: a second reading, verbatim-verified, disagreeing with the value on the record. Not applied - a person decides. Quotation withheld because this record is not cleared for redistribution. builds-on → the monkey object sequence memory dataset of ref [3] Quotation withheld because this record is not cleared for redistribution. builds-on → machine-learning-inspired circuit theory, refs [4]-[10] Quotation withheld because this record is not cleared for redistribution. extends → prior linear-readout decoding studies, refs [17], [18] Quotation withheld because this record is not cleared for redistribution. reinterprets → the heterogeneous, seemingly disordered tuning that reverse-engineering approaches treat as noise Quotation withheld because this record is not cleared for redistribution. extends → previous demonstrations that a linear readout suffices to decode particular task aspects Quotation withheld because this record is not cleared for redistribution. supports → model circuits that expand dimensionality by random non-linear mixing Quotation withheld because this record is not cleared for redistribution. qualifies → its own claim, by allowing that dendritic non-linearities could hide the expansion Quotation withheld because this record is not cleared for redistribution. contradicts → the grandmother-cell account of how high dimensionality could arise Quotation withheld because this record is not cleared for redistribution. non-linear mixed selectivity —causes→ dimensionality non-linear mixed selectivity dimensionality strength of the relation: 0.95 · note: and the same quote carries the negative half: pure and linear mixed selectivity can only generate low-dimensional representations Quotation withheld because this record is not cleared for redistribution. linear mixed selectivity —contrasted-with→ non-linear mixed selectivity linear mixed selectivity non-linear mixed selectivity strength of the relation: 1 Quotation withheld because this record is not cleared for redistribution. dimensionality —operationalised-as→ binary classifications dimensionality binary classifications strength of the relation: 1 · note: an abstract property measured through a downstream tool's capacity - d = log2(Nc) Quotation withheld because this record is not cleared for redistribution. t-test t-test Quotation withheld because this record is not cleared for redistribution. cross-validation cross-validation Quotation withheld because this record is not cleared for redistribution. confidence interval confidence interval Quotation withheld because this record is not cleared for redistribution. resampling resampling Quotation withheld because this record is not cleared for redistribution. principal component analysis principal component analysis Quotation withheld because this record is not cleared for redistribution. standard error of the mean standard error of the mean Quotation withheld because this record is not cleared for redistribution. cross-validated decoding accuracy cross-validated decoding accuracy Quotation withheld because this record is not cleared for redistribution. number of implementable binary classifications number of implementable binary classifications Quotation withheld because this record is not cleared for redistribution. Every task-relevant aspect can still be decoded from the PFC population after classical selectivity has been removed. Quotation withheld because this record is not cleared for redistribution. During the two-object delay the recorded PFC population supports the maximum possible number of binary classifications, i.e. it attains the maximal dimensionality of 24 set by the 24 experimental conditions. Quotation withheld because this record is not cleared for redistribution. Representations based on pure selectivity generate fewer than 8 dimensions. Quotation withheld because this record is not cleared for redistribution. The dimensionality of the PFC representation drops on trials where the monkey makes an error, so dimensionality tracks behavioural success. Quotation withheld because this record is not cleared for redistribution. On error trials the identities of the two cues could still be decoded with high accuracy. Quotation withheld because this record is not cleared for redistribution. The dimensionality collapse that predicts the animal's errors is specifically due to a weakening of the non-linear component of mixed selectivity. Quotation withheld because this record is not cleared for redistribution. The authors rule out grandmother-cell-like coding by verifying that PFC representations are rather dense. Quotation withheld because this record is not cleared for redistribution. Removing classical selectivity costs the most decoding accuracy in the early epochs of the trial. Quotation withheld because this record is not cleared for redistribution. every task-relevant aspect can still be decoded after classical selectivity to it has been removed from every neuron Quotation withheld because this record is not cleared for redistribution. the recovered accuracy grows as the task becomes more complex and more cues are held in memory Quotation withheld because this record is not cleared for redistribution. the recorded representations reach the maximal dimensionality allowed by the design, d = 24 Quotation withheld because this record is not cleared for redistribution. simulated pure-selectivity populations fall orders of magnitude short of the recorded ones Quotation withheld because this record is not cleared for redistribution. dimensionality collapses on error trials, as predicted before the test Quotation withheld because this record is not cleared for redistribution. cue identity is decoded just as well on error trials, so the collapse is not a failure to encode or remember the stimuli Quotation withheld because this record is not cleared for redistribution. the collapse is due specifically to the non-linear component of mixed selectivity Quotation withheld because this record is not cleared for redistribution. the dense, non-sparse character of the representations rules out a grandmother-cell explanation Quotation withheld because this record is not cleared for redistribution. mixed selectivity mixed selectivity Quotation withheld because this record is not cleared for redistribution. dimensionality dimensionality Quotation withheld because this record is not cleared for redistribution. non-linear mixed selectivity non-linear mixed selectivity Quotation withheld because this record is not cleared for redistribution. classical selectivity classical selectivity Quotation withheld because this record is not cleared for redistribution. grandmother cell grandmother cell Quotation withheld because this record is not cleared for redistribution. dimensionality of a neural representation dimensionality of a neural representation Quotation withheld because this record is not cleared for redistribution. mixed selectivity mixed selectivity Quotation withheld because this record is not cleared for redistribution. classical selectivity classical selectivity Quotation withheld because this record is not cleared for redistribution. dimensionality of the neural representation dimensionality of the neural representation Quotation withheld because this record is not cleared for redistribution. population decoding accuracy population decoding accuracy Quotation withheld because this record is not cleared for redistribution. noise level noise level Quotation withheld because this record is not cleared for redistribution. task type task type Quotation withheld because this record is not cleared for redistribution. 237 — number of lateral PFC (area 46) neurons analysed, recorded in two monkeys Quotation withheld because this record is not cleared for redistribution. 24 — number of distinct experimental conditions (combinations of cue 1, cue 2 and task type) Quotation withheld because this record is not cleared for redistribution. less than 4% — difference between the number of correct and error trials for the 121 neurons used in the error analysis Quotation withheld because this record is not cleared for redistribution. 100–150 trials — block length over which recognition and recall trials were interleaved Quotation withheld because this record is not cleared for redistribution. p < 0.05 — significance of the difference between the two task types in an example neuron's response to object C as first cue Quotation withheld because this record is not cleared for redistribution. close to 100% — maximal cross-validated decoding accuracy for all task-relevant aspects before selectivity removal Quotation withheld because this record is not cleared for redistribution. 1000 — number of train/test partitions over which decoding accuracy was averaged, with 95% confidence intervals shown Quotation withheld because this record is not cleared for redistribution. d = 12 — near-maximal dimensionality reached on correct recall-task trials Quotation withheld because this record is not cleared for redistribution. 237 lateral PFC neurons (area 46) from two monkeys — the reanalysed sample Quotation withheld because this record is not cleared for redistribution. 24 conditions, from 4 objects as cue 1, 3 remaining as cue 2, and 2 task types — the design that sets the ceiling on measurable dimensionality Quotation withheld because this record is not cleared for redistribution. d = log2(Nc) — the conversion from implementable classifications to dimensionality Quotation withheld because this record is not cleared for redistribution. 121 neurons with enough error trials, correct-error trial counts differing by less than 4% — the error-analysis subset Quotation withheld because this record is not cleared for redistribution. blocks of 100 to 150 trials — the interleaving of recognition and recall Quotation withheld because this record is not cleared for redistribution.

The full text of this article is not redistributable, so it is not included here. The evidence sentences are withheld with it; the tag layer and its grounding status remain. The article itself is at https://doi.org/10.1038/nature12160.