TengpersName, TianpersName, PoeppelpersName (2016date). “Testing multi-scale processing in the auditory system.” Scientific Reportstitle·j.
Object of study — How the auditory systemterm·key integrates sound over time - whether on one continuous timescaleterm·dom or on two distinct ones.
Research question — Natural sound carries information on several timescalesterm·dom at once - does one integration process handle all of them, or are there separate windows for detail and for pattern?
Contribution — Shows that performance on fine detail does not improve with longer stimuli - which the older integration models predict it should - and supports two concurrent temporal windowsterm·dom instead.
phenomenon · claim-type · cross-domain-transfer · mechanism · claim-relation · replication-status · limitation — grounded in the full text| phenomenon | auditory temporal integration across timescales — “The temporal integration of sound is a fundamental property of hearing.” | |
| claim-type | mechanism — “Here we use a paradigm requiring processing on relatively ‘local’ and ‘global’ scales and provide evidence suggesting that the auditory system extracts fine-detail acoustic information using short temporal windows and uses long temporal windows to abstract global acoustic patterns.” | |
| method — evidence | — “Here, to explore mechanisms of temporal integration and test certain predictions of the different integration models, we investigated in a series of four psychophysical experiments the perceptual performance at shorter and longer scales by manipulating the timescale of acoustic information while biasing the auditory processes implementing local or global task demands.” | |
| object of study — evidence | — “Integrating information over time, as we define it operationally here, serves as a basic information accumulation process in the auditory system to guarantee that enough information is supplied for extracting meaningful regularities on a certain timescale.” | |
| research question — evidence | — “Therefore, by hypothesis, a single integration process may not suffice for the variety of auditory tasks that the system successfully solves.” | |
| contribution — evidence | — “Behavioral task performance that requires processing fine-detail information does not improve with longer stimulus length, contrary to predictions of previous temporal integration models such as the multiple-looks and the spectro-temporal excitation pattern model.” | |
| mechanism | sampled acoustic information is passed to a second, representational stage of limited capacity, so what reaches perception is not the full set of samples — “In a second stage, the sampled information is channeled to a representational stage with limited information capacity and probably with a time constant of hundreds of milliseconds.” | mechanisms[0] |
| mechanism | when too much detail is squeezed into that limited-capacity representation the system smooths or summarises it, destroying the local detail that the task requires — “However, the limited information capacity of the representation stage forces the auditory system to smooth or summarize all information.” | mechanisms[1] |
| mechanism | inserting gaps lowers the amount of information per unit time, letting local detail be represented precisely — “After intervals are inserted between segments, information per unit time decreases, so local details can be represented in a more precise manner.” | mechanisms[2] |
| mechanism | stretching a stimulus with gaps pushes segments outside the single long temporal window, so the global pattern can no longer be sampled — “As the long temporal window is around hundreds of milliseconds, inserting intervals stretches the stimuli and makes few segments fall within one long temporal window, so a global pattern of the stretched stimuli cannot be sampled.” | mechanisms[3] |
| mechanism | two windows run concurrently: about 30 ms for parsing acoustic structure at high resolution, about 200 ms for extracting the global pattern — “one temporal window, of ~30 ms or less in which acoustic structure is parsed and analyzed with relatively high temporal resolution; and another temporal window of ~200 ms in which acoustic information is extracted at a more global scale” | mechanisms[4] |
| mechanism | the two models are separated by a prediction, not by plausibility: more stimulus should mean better detail discrimination under multiple-looks and STEP, and need not under 2TWI — “In contrast, the 2TWI model suggests that increased stimulus length may not necessarily increase performance, because representation and processing of local details might function best only at a time range of tens of milliseconds” | mechanisms[5] |
| mechanism | the older account is leaky integration with a time constant of hundreds of milliseconds, under which performance rises with duration and plateaus above 200 ms — “Previous models of temporal integration typically assume leaky integration with a time constant of hundreds of milliseconds, based on studies such as loudness summation 15 , 16 , signal detection in noise 17 , 18 , and temporal integration at threshold 13 , 19 , 20 , 21 , 22 , in which listeners’ performance increases continuously with sound duration and typically achieves a plateau above 200 ms.” | mechanisms[6] |
| mechanism | multiple-looks resolves the paradox by sampling fast and concatenating - so the long timescale is BUILT from short samples rather than being independent — “The multiple-looks model suggests that temporal integration is a process of multiple timescales instead of one, but the longer timescale is assembled on the basis of concatenated short samples.” | mechanisms[7] |
| contradicts | the multiple-looks model (Viemeister) and the spectro-temporal excitation pattern (STEP) model (Moore) — “However, we find that the performance of discrimination between two stimuli with different fine-grained information decreases, which is inconsistent with the prediction of a multiple-looks model and a STEP model.” | claim_relations[0] |
| builds-on | the multiple-looks and STEP models — “Inspired by the multiple-looks and STEP models, we suggest a slightly different temporal integration process to account for the present results.” | claim_relations[1] |
| replicates | McDermott et al. and Piazza et al. — “The results of Experiment 1 are consistent with recent findings by McDermott et al . 46 and Piazza et al . 47 , who also showed that increased stimulus length helps the auditory system perceive global statistical patterns but can impede access to local acoustic detail.” | claim_relations[2] |
| extends | Poeppel (2001, 2003) multi-time resolution hypothesis for speech — “Such a multi-time resolution hypothesis, proposed by Poeppel (2001, 2003) for speech is probably a general mechanism for audition, not just a specific model for speech perception as was originally argued.” | claim_relations[3] |
| contradicts | the multiple-looks and spectro-temporal excitation pattern models — “Behavioral task performance that requires processing fine-detail information does not improve with longer stimulus length, contrary to predictions of previous temporal integration models such as the multiple-looks and the spectro-temporal excitation pattern model.” | claim_relations[4] |
| builds-on | the two-temporal-windows integration model — “Although the two-temporal-windows integration model (2TWI) shares some basic features with the multiple-looks and STEP models, there are significant differences, as well.” | claim_relations[5] |
| extends | previous local-global paradigms — “Our approach is conceptually analogous to previous local-global studies” | claim_relations[6] |
| sense | integration - as a single leaky accumulator with one time constant, under which more sound always means better performance — “Previous models of temporal integration typically assume leaky integration with a time constant of hundreds of milliseconds, based on studies such as loudness summation 15 , 16 , signal detection in noise 17 , 18 , and temporal integration at threshold 13 , 19 , 20 , 21 , 22 , in which listeners’ performance increases continuously with sound duration and typically achieves a plateau above 200 ms.” | senses[0] |
| sense | integration - as fast discrete sampling then concatenation, where the long scale is assembled from short samples — “The multiple-looks model suggests that temporal integration is a process of multiple timescales instead of one, but the longer timescale is assembled on the basis of concatenated short samples.” | senses[1] |
| sense | integration - as two independent concurrent windows, where the long scale is not built from the short one and each does its own job — “In contrast, the 2TWI model suggests that increased stimulus length may not necessarily increase performance, because representation and processing of local details might function best only at a time range of tens of milliseconds” | senses[2] |
| control | a separate experiment (Experiment 4) varying the inter-stimulus interval systematically, with d-prime found not to vary across ISIs — held — rules out auditory MEMORY as the explanation of the main results - the same-different procedure requires holding the first stimulus, so a memory effect would mimic the multi-scale processing the paper claims — “Although memory requirements may come into play in our experiment due to the characteristics of the same-different procedure, the results of this control experiment demonstrate that memory cannot explain the main results obtained in the previous experiments.” | controls[0] |
| audit | resolution-integration paradox —causes→ multiple-looks — Audit 2026-09-28: DOUBTFUL - read against its full span and kept, but the model was proposed 'to resolve this paradox'; causes is loose. — “To resolve this paradox, Viemeister (1991) proposed a ‘multiple-looks’ model: the continuous sound is sampled discretely at a high rate (e.g. with integration windows of ~5 ms), and the samples are stored and concatenated/integrated for further processing.” | concept_edges[1] |
| weight | resolution-integration paradox —causes→ multiple-looks — 0.9 — “To resolve this paradox, Viemeister (1991) proposed a ‘multiple-looks’ model: the continuous sound is sampled discretely at a high rate (e.g. with integration windows of ~5 ms), and the samples are stored and concatenated/integrated for further processing.” | concept_edges[1] |
| edge | temporal window —part-of→ temporal integration — “The findings of the present experiments suggest a more complicated temporal integration process in the auditory system: the acoustic segments are analyzed within a short temporal window for extracting local details and concurrently acoustic information is integrated within a larger temporal window to form a global percept.” | concept_edges[2] |
| outcome (stated by the tagger, no quotation) | control — held | controls[0] |
| for which use | limitation — the local-task results at the extreme durations: THE EFFECT OF INCREASED STIMULUS LENGTH CANNOT BE OBSERVED PROBABLY BECAUSE OF A CEILING EFFECT — “Therefore, the effect of increased stimulus length cannot be observed probably because of a ceiling effect.” | limitations[0] |
| for which use | limitation — every sensitivity estimate: responses from a same-different paradigm were treated as yes-no, and PARTICIPANTS ... MAY ADAPT A DIFFERENCING STRATEGY - the measurement model may not match the task performed — “Participants in our experiments may adapt a “differencing” strategy as in the same-different task, since there is no obvious perceptual token in our stimuli as well as stimuli of different segment durations and stimulus lengths were grouped and presented randomly in each task.” | limitations[1] |
| for which use | limitation — the segment-number result, which reverses between experiments: WE SUSPECT THAT THE DISCREPANCY IS CAUSED BY THE FACT THAT WE FIXED THE FIRST AND THE LAST SEGMENTS IN EXPERIMENT 3A - the differing parameter is named, post hoc — “We suspect that the discrepancy is caused by the fact that we fixed the first and the last segments in Experiment 3A, so differences between two stimuli within the ‘different’ pair are much smaller in 3-segment stimuli (only one segment is different) than in 5-segment stimuli (three segments are different).” | limitations[2] |
| for which use | limitation — the evidential context of the whole claim: THIS MULTI-SCALE PROCESS ... IS NOT WIDELY INVESTIGATED IN THE LITERATURE, AND EXISTING MODELS OF TEMPORAL INTEGRATION ARE MAINLY BUILT UPON DETECTION OR REC[ognition] — “However, this multi-scale process in the auditory system is not widely investigated in the literature, and existing models of temporal integration are mainly built upon detection or recognition tasks on a single timescale.” | limitations[3] |
| for which use | limitation — the cortical attribution specifically: THESE FINDINGS SUPPORT THE HYPOTHESIS OF A DUAL-SCALE PROCESSING LIKELY IMPLEMENTED IN THE AUDITORY CORTEX - psychophysics cannot establish the implementation — “These findings support the hypothesis of a dual-scale processing likely implemented in the auditory cortex.” | limitations[4] |
| for which use | limitation — the design's fit to the question: OUR APPROACH IS CONCEPTUALLY ANALOGOUS TO PREVIOUS LOCAL-GLOBAL STUDIES - borrowed rather than purpose-built — “Our approach is conceptually analogous to previous local-global studies” — “Our approach is conceptually analogous to previous local-global studies 39 , 40 .” | limitations[5] |
| replication | untested | |
| limitation | At short (15-20 ms) and long (50-100 ms) segment durations the local-task data may be at floor or ceiling, so the effect of stimulus length could not be observed there. — “Therefore, the effect of increased stimulus length cannot be observed probably because of a ceiling effect.” | limitations[0] |
| limitation | Responses from the same-different paradigm were analysed as if from a yes-no paradigm, and participants may have adopted a differencing strategy. — “Participants in our experiments may adapt a “differencing” strategy as in the same-different task, since there is no obvious perceptual token in our stimuli as well as stimuli of different segment durations and stimulus lengths were grouped and presented randomly in each task.” | limitations[1] |
| limitation | Experiment 3A gives a result opposite in direction to Experiment 1B on segment number, attributed post hoc to fixing the first and last segments. — “We suspect that the discrepancy is caused by the fact that we fixed the first and the last segments in Experiment 3A, so differences between two stimuli within the ‘different’ pair are much smaller in 3-segment stimuli (only one segment is different) than in 5-segment stimuli (three segments are different).” | limitations[2] |
| limitation | The literature it enters is thin - the multi-scale process itself is described as not widely investigated. — “However, this multi-scale process in the auditory system is not widely investigated in the literature, and existing models of temporal integration are mainly built upon detection or recognition tasks on a single timescale.” | limitations[3] |
| limitation | The two-window sizes carry a physiological commitment the psychophysics alone cannot establish. — “These findings support the hypothesis of a dual-scale processing likely implemented in the auditory cortex.” | limitations[4] |
| limitation | The design is borrowed from existing local-global paradigms rather than built for the question. — “Our approach is conceptually analogous to previous local-global studies” | limitations[5] |
finding · quantity · variable · definition · key-term — grounded in the full text| finding | Making a sound longer improved listeners' ability to judge its overall global pattern but did not improve their ability to compare its fine acoustic detail. — “However, longer stimuli only improved integration in a global task , but did not increase performance on extracting fine-grained information in a local task .” | content_tags.findings[0] |
| finding | Lengthening the individual segments had the opposite effect from lengthening the whole stimulus: it helped the local task and hurt the global task. — “Furthermore, increasing segment duration increases participants’ performance in the local task but decreases the performance in the global task .” | content_tags.findings[1] |
| finding | With longer stimuli listeners shifted their decision criterion, increasingly calling two genuinely different sounds 'the same'. — “Furthermore, in Experiment 1B, the criterion in the local task increased with stimulus length, which indicates that subjects became worse at analyzing local details and tended to consider two sounds in the ‘different’ pair to be the same.” | content_tags.findings[2] |
| finding | Inserting silent gaps between segments raised sensitivity in the local task while lowering it in the global task. — “Figure 3b shows that d prime value s increased as the IOIs increased in the local task (Experiment 3A) but decreased in the global task (Experiment 3B).” | content_tags.findings[3] |
| finding | Sensitivity did not change across inter-stimulus intervals for a given stimulus length. — “Figure 4 illustrates that d prime value s did not change across different ISIs for a given stimulus length.” | content_tags.findings[4] |
| finding | In the global task, sensitivity increased as the number of segments (stimulus length) increased. — “Figure 2b summarizes the results: as the number of segments (i.e. stimulus length) increased, d prime value s increased.” | content_tags.findings[5] |
| finding | Discriminating sounds on detailed local spectrotemporal information did not improve with increasing stimulus length. — “The performance on the local task shows that differentiating sounds based on detailed, local spectrotemporal information does not increase with increasing stimulus length.” | content_tags.findings[6] |
| finding | Experiment 3A shows that working-memory load cannot explain the effect of stimulus length in the local task. — “The results from Experiment 3A also demonstrate that the working memory load cannot explain the effect of stimulus length in the local task of Experiment 1.” | content_tags.findings[7] |
| finding | The pattern replicates earlier reports that longer sounds aid global statistical perception while blocking access to local acoustic detail. — “The results of Experiment 1 are consistent with recent findings by McDermott et al . 46 and Piazza et al . 47 , who also showed that increased stimulus length helps the auditory system perceive global statistical patterns but can impede access to local acoustic detail.” | content_tags.findings[8] |
| finding | Fine-detail performance does not improve with longer stimuli - the prediction that separates the old models from the new one fails for the old ones. — “Behavioral task performance that requires processing fine-detail information does not improve with longer stimulus length, contrary to predictions of previous temporal integration models such as the multiple-looks and the spectro-temporal excitation pattern model.” | content_tags.findings[9] |
| finding | Hearing a sound as one unitary event takes more than hundreds of milliseconds. — “Moreover, the perceptual construction of putatively ‘unitary’ auditory events requires more than hundreds of milliseconds.” | content_tags.findings[10] |
| finding | Temporal-resolution studies report perceptual thresholds of 2-30 ms. — “In those studies, perceptual thresholds from 2–30 ms are the norm.” | content_tags.findings[11] |
| finding | The timescales are not arbitrary - speech carries phonemic, syllabic and intonational information at three separate ranges. — “For example, speech conveys linguistic information at several scales: 20–80 ms for phonemic information, 100–300 ms for syllabic information, and more than 1000 ms for intonation information 1 .” | content_tags.findings[12] |
| quantity | 4.482 — ceiling d-prime value imposed by adding a half artificial incorrect trial to all-correct blocks — “Therefore, the maximum d-prime value for all correct responses is 4.482 (instead of infinity) and the minimum d-prime value is 0.” | content_tags.quantities[0] |
| quantity | 1600 — trials presented in the local task of Experiment 1A, and the same number again in the global task — “In Experiment 1A, 1600 trials in total were presented in the local task and the same number in the global task .” | content_tags.quantities[1] |
| quantity | Twenty-two — participants recruited across Experiments 1A and 1B — “Twenty-two participants, who were all right-handed and had no known hearing deficit, took part in the experiment.” | content_tags.quantities[2] |
| quantity | ~30 ms and ~200 ms — the two integration windows proposed by the 2TWI model — “one temporal window, of ~30 ms or less in which acoustic structure is parsed and analyzed with relatively high temporal resolution; and another temporal window of ~200 ms in which acoustic information is extracted at a more global scale” | content_tags.quantities[3] |
| quantity | 20–80 ms / 100–300 ms / >1000 ms — phonemic, syllabic and intonational timescales in speech — “For example, speech conveys linguistic information at several scales: 20–80 ms for phonemic information, 100–300 ms for syllabic information, and more than 1000 ms for intonation information 1 .” | content_tags.quantities[4] |
| quantity | 2–30 ms — perceptual thresholds in the resolution literature - the half of the paradox the integration models cannot accommodate — “In those studies, perceptual thresholds from 2–30 ms are the norm.” | content_tags.quantities[5] |
| quantity | 10–30 ms and 500–700 ms — the two timescales carrying information in bird song — “Bird song, similarly, also includes information on one timescale around 10–30 ms and another one around 500–700 ms 5 .” | content_tags.quantities[6] |
| variable | stimulus length (independent) — “In Experiment 1A, two factors, stimulus length and segment duration , were manipulated.” | content_tags.variables[0] |
| variable | segment duration (independent) — “In Experiment 1A, two factors, stimulus length and segment duration , were manipulated.” | content_tags.variables[1] |
| variable | inter-stimulus interval (ISI) (independent) — “The factors, segment number (i.e. overall stimulus length) and inter-stimulus interval (ISI), were manipulated.” | content_tags.variables[2] |
| variable | d prime (dependent) — “Responses were transformed into the sensitivity index, d prime, and criterion, denoted as C, as in a simple yes-no paradigm using the Palamedes toolbox 44 .” | content_tags.variables[3] |
| variable | working memory load (control) — “By inserting intervals into the stimuli, we kept the working memory load constant and stretched the stimuli to make the task more memory demanding, but the participants’ performance became better rather than worse.” | content_tags.variables[4] |
| variable | frequency difference (control) — “The frequency difference between two stimuli at the onset was the same as the frequency difference at the offset, so the participants could not rely on spectral cues to judge the trend of the frequency shift.” | content_tags.variables[5] |
| variable | sweep direction (control) — “In the global task , the sweep directions of corresponding segments between two stimuli were always different because we wanted to prevent participants from making judgments based on acoustic details.” | content_tags.variables[6] |
| variable | task demands (independent) — “Here, to explore mechanisms of temporal integration and test certain predictions of the different integration models, we investigated in a series of four psychophysical experiments the perceptual performance at shorter and longer scales by manipulating the timescale of acoustic information while biasing the auditory processes implementing local or global task demands.” | content_tags.variables[7] |
| variable | behavioral task performance (dependent) — “Behavioral task performance that requires processing fine-detail information does not improve with longer stimulus length, contrary to predictions of previous temporal integration models such as the multiple-looks and the spectro-temporal excitation pattern model.” | content_tags.variables[8] |
| definition | temporal integration — “Integrating information over time, as we define it operationally here, serves as a basic information accumulation process in the auditory system to guarantee that enough information is supplied for extracting meaningful regularities on a certain timescale.” | content_tags.definitions[0] |
| definition | inter-onset interval (IOI) — “Two additional factors, the number of segments and inter-onset interval (IOI), were included in this experiment. The IOI is defined here as the duration between the onset of one segment and the onset of the next segment (see Fig. 3a ).” | content_tags.definitions[1] |
| definition | local task — “In the local task , the sweep directions of each corresponding segment were the same between two stimuli in the ‘same’ pair, whereas the sweep directions of each corresponding segment in the ‘different’ pair were opposite.” | content_tags.definitions[2] |
| definition | the multiple-looks model — “To resolve this paradox, Viemeister (1991) proposed a ‘multiple-looks’ model: the continuous sound is sampled discretely at a high rate (e.g. with integration windows of ~5 ms), and the samples are stored and concatenated/integrated for further processing. The multiple-looks model suggests that temporal integration is a process of multiple timescales instead of one, but the longer timescale is assembled on the basis of concatenated short samples.” [ATTRIBUTED TO A CITED AUTHOR, NOT THIS PAPER] the paper’s own words: “Viemeister (1991) proposed” | content_tags.definitions[4] |
| definition | the resolution-integration paradox — “This incompatibility has led to the formulation of the resolution-integration paradox” | content_tags.definitions[5] |
| definition | two-temporal-windows integration model — “one temporal window, of ~30 ms or less in which acoustic structure is parsed and analyzed with relatively high temporal resolution; and another temporal window of ~200 ms in which acoustic information is extracted at a more global scale” | content_tags.definitions[6] |
| key-terms | temporal integration, temporal window, multiple-looks, spectro-temporal excitation pattern, resolution-integration paradox, leaky integration, local, global, auditory system, timescale |
| concept | temporal integration — not defined in this paper's own words; accumulating acoustic information over time until there is enough of it to extract a regularity at a given timescale — “The temporal integration of sound is a fundamental property of hearing. Integrating information over time, as we define it operationally here, serves as a basic information accumulation process in the auditory system to guarantee that enough information is supplied for extracting meaningful regularities on a certain timescale.” | concepts[0] |
| concept | temporal window — not defined in this paper's own words; the span of time over which the system accumulates before reading out - here two of them, ~30 ms and ~200 ms, running concurrently — “one temporal window, of ~30 ms or less in which acoustic structure is parsed and analyzed with relatively high temporal resolution; and another temporal window of ~200 ms in which acoustic information is extracted at a more global scale” | concepts[1] |
| concept | multiple-looks — not defined in this paper's own words; Viemeister's account: sample the sound fast, store the samples, concatenate them - so long timescales are built out of short ones — “To resolve this paradox, Viemeister (1991) proposed a ‘multiple-looks’ model: the continuous sound is sampled discretely at a high rate (e.g. with integration windows of ~5 ms), and the samples are stored and concatenated/integrated for further processing.” | concepts[2] |
| concept | resolution-integration paradox — not defined in this paper's own words; the field's own contradiction: integration studies imply a slow accumulator, resolution studies imply a fast one, and both are well evidenced — “This incompatibility has led to the formulation of the resolution-integration paradox” | concepts[3] |
| concept | leaky integration — not defined in this paper's own words; the older single-process account, with a time constant of hundreds of milliseconds and performance rising continuously with duration — “Previous models of temporal integration typically assume leaky integration with a time constant of hundreds of milliseconds, based on studies such as loudness summation 15 , 16 , signal detection in noise 17 , 18 , and temporal integration at threshold 13 , 19 , 20 , 21 , 22 , in which listeners’ performance increases continuously with sound duration and typically achieves a plateau above 200 ms.” | concepts[4] |
| concept | spectro-temporal excitation pattern — not defined in this paper's own words; Moore's internal representation of a sound as a weighted vector of samples or 'intelligent looks' — “For example, in a task comparing local details of sounds, the multiple-looks model and STEP would predict that, as stimulus length increases and hence more local details are available, the auditory system would form more distinct representations.” | concepts[5] |
| concept | timescale — not defined in this paper's own words; the temporal grain at which a kind of information lives - phonemes, syllables and intonation each occupying a different one — “For example, speech conveys linguistic information at several scales: 20–80 ms for phonemic information, 100–300 ms for syllabic information, and more than 1000 ms for intonation information 1 . Bird song, similarly, also includes information on one timescale around 10–30 ms and another one around 500–700 ms 5 .” | concepts[6] |
| d-prime (sensitivity index) C | as written: the sensitivity index, d prime, and criterion, denoted as C — “Responses were transformed into the sensitivity index, d prime, and criterion, denoted as C, as in a simple yes-no paradigm using the Palamedes toolbox 44 .” | content_tags.statistics[0] |
| ANOVA | as written: two-way repeated measures ANOVA — “A two-way repeated measures ANOVA with the factors task and segment duration was conducted at each level of stimulus length, followed by a one-way repeated measures ANOVA to evaluate the main effect of segment duration within each task at each level of stimulus length.” | content_tags.statistics[1] |
| two-alternative forced choice d-prime | as written: two-interval two-alternative-forced choice paradigm and d prime values — “In Experiment 2, we used a two-interval two-alternative-forced choice paradigm and d prime values were computed accordingly.” | content_tags.statistics[2] |
| claim type | existence — a second reading DIFFERING from the recorded value; recorded: mechanism — “These findings support the hypothesis of a dual-scale processing likely implemented in the auditory cortex.” | alternatives.claim_type |
| phenomenon | temporal integration of sound on two concurrent timescales, local and global — a second reading DIFFERING from the recorded value; recorded: auditory temporal integration across timescales — “Here we use a paradigm requiring processing on relatively ‘local’ and ‘global’ scales and provide evidence suggesting that the auditory system extracts fine-detail acoustic information using short temporal windows and uses long temporal windows to abstract global acoustic patterns.” | alternatives.phenomenon |
Full text — tagged
The reference list is not part of the text we hold for this paper. Our copy was extracted from the publisher's page and the bibliography did not survive that extraction, so the article below ends before its references. Nothing has been removed by us.
Testing multi-scale processing in the auditory systemterm·key Xiangbin TengpersName Xiangbin TengpersName 1 Department of Psychology, New York University, New York, NY, USA 1, a , Xing TianpersName Xing TianpersName 2 New York University Shanghai, Shanghai, 200122 China 3 NYU-ECNU Institute of Brain and Cognitive Science at NYU Shanghai, Shanghai, 200122 China 2, 3 , David PoeppelpersName David PoeppelpersName 1 Department of Psychology, New York University, New York, NY, USA 4 Department of Neuroscience, Max-Planck Institute, Frankfurt, Germany 1, 4 1 Department of Psychology, New York University, New York, NY, USA 2 New York University Shanghai, Shanghai, 200122 China 3 NYU-ECNU Institute of Brain and Cognitive Science at NYU Shanghai, Shanghai, 200122 China 4 Department of Neuroscience, Max-Planck Institute, Frankfurt, Germany a Email: xiangbin.tengpersName@gmail.com Received 2016 Mar 1; Accepted 2016 Sep 6; Collection date 2016.
Copyright © 2016, The Author(s) This work is licensed under a Creative Commons Attribution 4.0 International License. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in the credit line; if the material is not included under the Creative Commons license, users will need to obtain permission from the license holder to reproduce the material. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ PMCID: PMC5054370 PMID: 27713546 Abstract Natural sounds contain information on multiple timescalesterm·dom, so the auditory systemterm·key must analyze and integrate acoustic information on those different scales to extract behaviorally relevant information. However, this multi-scale process in the auditory systemterm·key is not widely investigated in the literature, and existing models of temporal integrationterm·dom are mainly built upon detection or recognition tasks on a single timescaleterm·dom. Here we use a paradigm requiring processing on relatively ‘localterm·key’ and ‘globalterm·key’ scales and provide evidence suggesting that the auditory systemterm·key extracts fine-detail acoustic information using short temporal windowsterm·dom and uses long temporal windowsterm·dom to abstract globalterm·key acoustic patterns. Behavioral task performanceterm·var that requires processing fine-detail information does not improve with longer stimulus lengthterm·var, contrary to predictions of previous temporal integrationterm·dom models such as the multiple-looksterm·dom and the spectro-temporal excitation patternterm·mth model. Moreover, the perceptual construction of putatively ‘unitary’ auditory events requires more than hundreds of milliseconds. These findings support the hypothesis of a dual-scale processing likely implemented in the auditory cortex.
Natural sounds, music, and vocal sounds have a rich temporal structure over multiple timescalesterm·dom 1 , 2 , 3 , 4 , 5 , 6 , and behaviorally relevant acoustic information is usually carried on more than one timescaleterm·dom. For example, speech conveys linguistic information at several scales: 20–80 ms for phonemic information, 100–300 ms for syllabic information, and more than 1000 ms for intonation information 1 . Bird song, similarly, also includes information on one timescaleterm·dom around 10–30 ms and another one around 500–700 ms 5 . Therefore, auditory processing requires sounds to be integrated temporally at different scales to extract critical perceptual information.
The temporal integrationterm·dom of sound is a fundamental property of hearing. Integrating information over time, as we define it operationally here, serves as a basic information accumulation process in the auditory systemterm·key to guarantee that enough information is supplied for extracting meaningful regularities on a certain timescaleterm·dom. It has been demonstrated in various experiments, using different types of signals, that perceptual thresholds typically decrease as sound duration increases 7 , 8 . Such a trading relationship between sound duration and perceptual performance suggests that the auditory systemterm·key accumulates information over time to get better estimates of auditory signal characteristics. Visual perception, especially in dynamic contexts, also incorporates such temporal integrationterm·dom processes 9 . As natural sounds contain information on multiple timescalesterm·dom, fine-grained (informally speaking, temporally ‘localterm·key’) information usually exists on a shorter timescaleterm·dom and requires a fast integration process, while more ‘globalterm·key’ information occupies a larger timescaleterm·dom and a longer integration window is consequently needed. Therefore, by hypothesis, a single integration process may not suffice for the variety of auditory tasks that the system successfully solves.
The existing literature argues that the auditory systemterm·key works over several distinct time ranges - instead of in a unitary manner across the (intuitively simpler and conceptually appealing) continuum of temporal variation 8 , 10 , 11 , 12 , 13 , 14 . Previous models of temporal integrationterm·dom typically assume leaky integrationterm·dom with a time constant of hundreds of milliseconds, based on studies such as loudness summation 15 , 16 , signal detection in noise 17 , 18 , and temporal integrationterm·dom at threshold 13 , 19 , 20 , 21 , 22 , in which listeners’ performance increases continuously with sound duration and typically achieves a plateau above 200 ms. These models, however, are not compatible with the range of results that demonstrate the high temporal resolution of the auditory systemterm·key, such as in modulation detection 23 , gap detection 24 , and non-simultaneous masking 25 . In those studies, perceptual thresholds from 2–30 ms are the norm. This incompatibility has led to the formulation of the resolution-integration paradoxterm·def 8 , 26 .
To resolve this paradox, Viemeister (1991) proposed a ‘multiple-looksterm·dom’ model: the continuous sound is sampled discretely at a high rate (e.g. with integration windows of ~5 ms), and the samples are stored and concatenated/integrated for further processing. The multiple-looks modelterm·def suggests that temporal integrationterm·dom is a process of multiple timescalesterm·dom instead of one, but the longer timescaleterm·dom is assembled on the basis of concatenated short samples. Moore (2003) further relates this concept to the internal representation of sounds, calculated as spectro-temporal excitation patternsterm·mth (STEP): the internal representation can be treated as a vector of ‘intelligent looks’ or samples of auditory stimuli, with some ‘looks’ weighted more than others for different tasks. Perceptual tasks can be implemented on this type of internal representation 27 .
Recently, based on the results of speech perception experiments as well as neurophysiological studies, one line of argumentation proposes that there are two main temporal windowsterm·dom working concurrently in the auditory systemterm·key 28 , 29 , 30 , 31 , 32 , 33 : one temporal windowterm·dom, of ~30 ms or less in which acoustic structure is parsed and analyzed with relatively high temporal resolution; and another temporal windowterm·dom of ~200 ms in which acoustic information is extracted at a more globalterm·key scale. Although such a ‘two-timescale integration model’ has a specific physiological constraint on the size of temporal windowsterm·dom, it is supported by a growing body of neural evidence 30 , 34 , 35 , 36 , 37 , 38 . Its core idea builds on the multiple-looks modelterm·def and STEP, which is that temporal integrationterm·dom involves a process of sampling or integrating information discretely on a small timescaleterm·dom (e.g, tens of milliseconds) and on a large timescaleterm·dom (e.g. hundreds of milliseconds), instead of simply integrating information continuously, as suggested by the leaky-integration model.
Although the two-temporal-windows integration modelterm·def (2TWI) shares some basic features with the multiple-looksterm·dom and STEP models, there are significant differences, as well. For example, 2TWI model suggests that the auditory systemterm·key may process information differently on different scales 32 . The processing may be modulated by acoustic properties on different timescalesterm·dom or by task requirements so that extraction and representation of acoustic information is more efficient. This leads to predictions different from the ones derived from the multiple-looksterm·dom and STEP models. For example, in a task comparing localterm·key details of sounds, the multiple-looks modelterm·def and STEP would predict that, as stimulus lengthterm·var increases and hence more localterm·key details are available, the auditory systemterm·key would form more distinct representations. Therefore, the performance, e.g. of differentiating two sounds, should increase with stimulus lengthterm·var. In contrast, the 2TWI model suggests that increased stimulus lengthterm·var may not necessarily increase performance, because representation and processing of localterm·key details might function best only at a time range of tens of milliseconds 29 , 32 . On the other hand, the 2TWI model further suggests that representation for globalterm·key scales is best only for a time range in the hundreds of milliseconds, while the multiple-looksterm·dom and STEP models would predict the perception of globalterm·key information increases with increased number of ‘samples’ or ‘looks’ of information.
Here, to explore mechanisms of temporal integrationterm·dom and test certain predictions of the different integration models, we investigated in a series of four psychophysical experiments the perceptual performance at shorter and longer scales by manipulating the timescaleterm·dom of acoustic information while biasing the auditory processes implementing localterm·key or globalterm·key task demandsterm·var. Our approach is conceptually analogous to previous local-global studies 39 , 40 . In Experiment 1A, a same-different task is employed to test how various localterm·key and globalterm·key scales affect participants’ performance on abstracting fine-grained, ‘localterm·key’ spectrotemporal information as well as more ‘globalterm·key’ acoustic patterns. In Experiment 1B, by holding fixed the localterm·key scale, we further test how stimulus lengthterm·var modulates participants’ ability to extract fine-detailed and globalterm·key information. Experiment 2 assesses whether the frequency differenceterm·var at the onset or offset between two stimuli can affect the findings. In Experiment 3, we first measure how much time the auditory systemterm·key needs to effectively process particular acoustic details (Experiment 3A) and then test whether the integration process functions best over a limited time (Experiment 3B). In Experiment 4, by varying intervals between two stimuli, we ask whether memory factors may affect the results, as the results of the same-different task may be modulated by memory requirements. Table 1 provides an overview of the logic for each experiment and the tasked used in each experiment.
Table 1. An overview of the current study.
Motivation Task Experiment 1A We varied both the segment durationterm·var and the stimulus lengthterm·var to investigate how acoustic information on the localterm·key and globalterm·key scales affects auditory processing on both the localterm·key and globalterm·key tasks. We predict that the auditory systemterm·key requires tens of milliseconds to resolve information on the localterm·key scale while hundreds of milliseconds are required for the globalterm·key scale. This may reveal a two-scale processing mechanism in the auditory systemterm·key. Same-different Experiment 1B We fixed the segment durationterm·var at 30 ms while varying the stimulus lengthterm·var to further demonstrate the modulation effect of the stimulus lengthterm·var observed in Experiment 1A on both the localterm·key and globalterm·key tasks. Same-different Experiment 2 We controlled the globalterm·key frequency shift range of the stimuli and used a different paradigm. By doing this, we can test whether the frequency shift range changes our interpretation of our results in Experiment 1. Two-alternative-forced choice Experiment 3A The effect of stimulus lengthterm·var in the local taskterm·def of Experiment 1 could be caused by the capacity limit of information processing or the increased working memory loadterm·var. We test these two possibilities by inserting intervals between segments to decrease the amount of information per unit of time while keeping the memory load constant. Same-different Experiment 3B By inserting gaps between segments, we test whether the auditory systemterm·key can store all segments and integrate acoustic details over a longer time range. Same-different Experiment 4 When we varied the stimulus lengthterm·var in the local taskterm·def, this lengthened the amount of time participants needed to remember information about the first stimulus in the stimulus pair. If memory degraded during that time, this may have been one factor contributing to the effect of stimulus lengthterm·var in the local taskterm·def. To examine this hypothesis, the interval between two stimuli within a pair was varied in Experiment 4. Same-different General Method Participants We conducted all experiments in accordance with procedures approved by the NYU committee on Activities Involving Human Subjects. All participants were from the New York University community and surrounding areas and provided written informed consent before doing experiments. All participants were compensated financially for the time they spent doing experimental tasks.
Experiment 1 Twenty-two participants, who were all right-handed and had no known hearing deficit, took part in the experiment. Ten participated Experiment 1A (age range 22 to 33 years; 6 females) and ten participated Experiment 1B (age range 20 to 32 years; 7 females). The data from two subjects were excluded from Experiment 1B because of their failure to follow the instructions.
Experiment 2 Twelve listeners participated in this experiment. We excluded the data from two participants because of their failure to comply with the experimental instructions. The remaining 10 participants (age range 18 to 31 years; 6 female) were right-handed and had no reported hearing loss.
Experiment 3 Ten and eleven participants, who were right-handed and had no known hearing deficit took part in Experiment 3A (age range 20 to 32 years; 6 female) and Experiment 3B (age range 19 to 32 years; 5 female), respectively. One participant from Experiment 3B did not complete the experiment and therefore was removed from further analysis.
Experiment 4 Ten listeners (age range 18 to 31 years; 7 female) participated in the experiment. Nine participants were right-handed and one participant was left-handed according to participant self-report. The participants had no reported hearing loss.
Stimuli and apparatus We generated auditory stimuli and presented them to participants using MATLAB (The MathWorks, Natick, MA) and the Psychophysics Toolbox extensions 41 , 42 , 43 . The stimuli were inspired by similar auditory signals used in an fMRI study by Boemio et al . 35 and a magnetoencephalography (MEG) study by Luo & PoeppelpersName (2012). All stimuli were created by concatenating narrow-band frequency-modulated segments, each consisting of a sum of 200 frequency-modulated sinusoids. The amplitude and onset phase of each sinusoid were randomized, and the frequency range was centered at a frequency and varied with the bandwidth of 100 Hz. Each sinusoid was generated individually and all sinusoids were added together to create a narrow-band frequency modulated segment. Therefore, all generated segments spanned the range from 1000 to 2000 Hz with a mean frequency of 1500 Hz and had a bandwidth of 100 Hz, thereby remaining within the critical band at the corresponding frequency. The segments swept linearly upward or downward, randomly over the same frequency range. Each segment in each stimulus was individually generated. Figure 1a,b provide spectrograms of the stimuli.
Figure 1. Spectrograms of stimuli and results of Experiments 1A and 1B.
( a ) An example of spectrograms for the ‘different’ pair of stimuli is plotted separately for the local taskterm·def and the globalterm·key task . The dashed arrow schematizes the overall frequency change across stimulus duration. ( b ) Results of Experiment 1A are plotted as a function of stimulus lengthterm·var (x-axis) and d primeterm·var (y-axis). The marker shapes represent different segment durationsterm·var. Longer stimuli do not increase participants’ performance in the local taskterm·def (left), but facilitate recognition of the globalterm·key sound pattern across different segment durationsterm·var in globalterm·key task (right ). Larger segment durationterm·var increases subjects’ performance in the local taskterm·def (left) but decrease the performance in the globalterm·key task (right) . ( c ) The results of Experiment 1B using only segment durationterm·var of 30 ms demonstrate the opposite effect of the stimulus lengthterm·var on the localterm·key and globalterm·key tasks. ( d ) Criteria ( C ) in the localterm·key and globalterm·key tasks as a function of stimulus lengthterm·var. The criterion in the local taskterm·def increases with stimulus lengthterm·var, which indicates that subjects become worse at analyzing localterm·key details and tend to consider two sounds in the ‘different’ pair as the same.
All stimuli were presented at 16 bit, 44.1 kHz sampling resolution using headphones (Sennheiser HD 380 Professional, Sennheiser Electronic Corporation, Wedemark, Germany). The amplitude was kept at a comfortable level (~60 dB, SPL).
Analysis Procedures Responses were transformed into the sensitivity index, d prime, and criterion, denoted as Cterm·stat, as in a simple yes-no paradigm using the Palamedes toolbox 44 .
In Experiments 1, 3 and 4, we used a same-different paradigm. Correct responses to ‘different’ trials were marked as hit and incorrect responses to ‘same’ trials as false alarm. Participants in our experiments may adapt a “differencing” strategy as in the same-different task, since there is no obvious perceptual token in our stimuli as well as stimuli of different segment durationsterm·var and stimulus lengthsterm·var were grouped and presented randomly in each task. However, without assuming complex behavior models, treating responses in the same-different paradigm as from a yes-no paradigm can appropriately capture the sensitivity of participants and the behavioral effects caused by experimental modulations, which is the main focus of our study. When a participant responded correctly to all ‘same’ or ‘different’ trials within a block, a half artificial incorrect same or different trial was added to the block 45 . Therefore, the maximum d-prime value for all correct responses is 4.482 (instead of infinity) and the minimum d-prime value is 0.
In Experiment 2, we used a two-interval two-alternative-forced choice paradigm and d prime valuesterm·stat were computed accordingly.
Experiment 1 Both the multi-look and the STEP models predict that longer stimuli provide more localterm·key details than similar shorter ones, and hence that perceptual performance increases with stimulus lengthterm·var. In contrast, the 2TWI model hypothesizes that the implementation of different time scales can be modulated by the task demandsterm·var. Increasing stimulus lengthterm·var does therefore not necessarily enhance perceptual performance on localterm·key processing tasks. In Experiment 1A, we manipulate the implementation of localterm·key and globalterm·key scales in different tasks to investigate how local-scale and global-scale processes vary with the timescaleterm·dom of localterm·key acoustic details and stimulus lengthterm·var. In Experiment 1B, we further examine how access to fine-temporal-detail information changes with increases in overall stimulus lengthterm·var.
Method Stimuli and apparatus In Experiment 1A, two factors, stimulus lengthterm·var and segment durationterm·var , were manipulated. We varied both segment durationterm·var and stimulus lengthterm·var and chose specific combinations of number of segments and segment durationterm·var to fix overall stimulus lengthterm·var at around 90, 150, 220 and 300 ms. We confined the maximum stimulus lengthterm·var to be 300 ms because, as we observed in pilot data, participants’ performance achieved a ceiling effect or floor effect when stimulus lengthterm·var exceeded 300 ms. The minimum number of segments within a stimulus was set at 3, and the duration of segments was varied at 15, 20, 30, 50, 75 and 100 ms. Therefore, for stimuli of ~90 ms total duration, three segment durationsterm·var were included, 15, 20 and 30 ms; for stimuli of around 150 ms total duration, 15, 20, 30 and 50 ms segments were included; for stimuli of around 220 ms, 15, 20, 30, 50, and 75 ms; and for stimuli of 300 ms, 15, 20, 30, 50, 75 and 100 ms.
Two tasks, a local taskterm·def and a globalterm·key task , were run in Experiment 1A ( Fig. 1a ). On each trial, a pair of stimuli was presented. For both tasks, we generated ‘same’ and ‘different’ pairs. In the local taskterm·def , the sweep directionsterm·var of each corresponding segment were the same between two stimuli in the ‘same’ pair, whereas the sweep directionsterm·var of each corresponding segment in the ‘different’ pair were opposite. In the globalterm·key task , the sweep directionsterm·var of corresponding segments between two stimuli were always different because we wanted to prevent participants from making judgments based on acoustic details. In the ‘same’ pair, both stimuli had a mean frequency either increasing from 1300 Hz to 1700 Hz over the stimulus duration or decreasing from 1700 Hz to 1300 Hz; in the ‘different’ pair, one stimulus had a mean frequency increasing from 1300 Hz to 1700 Hz over the stimulus lengthterm·var, and the other one had a mean frequency decreasing from 1700 Hz to 1300 Hz.
In Experiment 1B, segment durationterm·var was fixed at 30 ms and the factor ‘stimulus lengthterm·var’ (equivalent to the number of segments) was manipulated in each condition, as Experiment 1A showed that overall stimulus lengthterm·var markedly affected performance of the local taskterm·def at this segment durationterm·var. We used the same manipulations as in Experiment 1A: in the local taskterm·def , we generated ‘same’ and ‘different’ pairs of stimuli with varying stimulus lengthterm·var: 30, 90, 150, 210, 270, and 330 ms. In the globalterm·key task , we only selected stimulus lengthterm·var: 90, 150, 210, 270, and 330 ms, because we could not manipulate globalterm·key pattern when there was only one segment. The sweep directionsterm·var of corresponding segments between two stimuli were always the same in the globalterm·key task (different from Experiment 1A). In the ‘same’ pair, both stimuli had a mean frequency increasing with the number of segments from 1300 Hz to 1700 Hz, or had a mean frequency decreasing with the number of segments from 1700 Hz to 1300 Hz; in the ‘different’ pair, one stimulus had a mean frequency increasing with the number of segments from 1300 Hz to 1700 Hz, and the other one had a mean frequency decreasing from 1700 Hz to 1300 Hz. Because we wanted to examine how acoustic information is integrated over time to build up a globalterm·key pattern, we made the mean frequency increase with the stimulus lengthterm·var instead of fixing the range of mean frequency for each stimulus, as we did in Experiment 1A.
Procedure and design The experiment employed a same-different task. On each trial, two stimuli were presented sequentially, with an inter-stimulus interval uniformly distributed from 400 to 600 ms. Participants were prompted to make a response 100 ms after the second interval. The local taskterm·def was to judge whether two stimuli were the same or not. In the globalterm·key task , the participants were asked to judge whether the directions of the mean frequency shift were the same between two stimuli. The localterm·key and globalterm·key tasks were presented in two separate blocks, and trials within a block were presented in a pseudorandom order. The order of blocks was counterbalanced across participants. In each block, for trials with the same number of segments, there were 40 same trials and 40 different trials for each condition. In Experiment 1A, 1600 trials in total were presented in the local taskterm·def and the same number in the globalterm·key task . In Experiment 1B, 480 trials in total were presented in the local taskterm·def and 400 trials in the globalterm·key task .
Before the formal experiment, three trials for each stimulus lengthterm·var and segment durationterm·var in each task were presented (with feedback) to familiarize the participants with the stimuli and tasks.
Results and Discussion Figure 1b shows the results of Experiment 1A, mean d primeterm·var value s as a function of stimulus lengthterm·var (mean ± 1 SEM). As illustrated in the left panel, stimulus lengthterm·var does not modulate sensitivity, as quantified by d primeterm·var value s, across different segment durationsterm·var except the segment durationterm·var of 30 ms in the local taskterm·def - in which participants needed to compare fine-detailed information between two stimuli. In contrast, d primeterm·var value s increased with increasing stimulus lengthterm·var in the globalterm·key task in which participants needed to integrate the acoustic information to perceive the globalterm·key sound pattern. Furthermore, increasing segment durationterm·var increases participants’ performance in the local taskterm·def but decreases the performance in the globalterm·key task .
Figure 1c shows the d primeterm·var value s for Experiment 1B, which demonstrate, again counterintuitively, that increased stimulus lengthterm·var negatively modulated performance in the local taskterm·def . The response criteria, depicted in Fig. 1d , indicate that at increased stimulus lengthterm·var participants tend to smooth out localterm·key details and treat two different localterm·key stimuli as the same.
In Experiment 1A, we first evaluated how stimulus lengthterm·var affected performance on each task. As the number of stimulus lengthterm·var levels varies across different segment durationsterm·var (e.g. there are four levels of stimulus lengthterm·var for segment durationterm·var of 15 ms while there are only two levels for 75 ms), we conducted a one-way repeated measures ANOVAterm·stat for each task and each segment durationterm·var. We found that stimulus lengthterm·var in the local taskterm·def significantly affected participants’ performance only when segment durationterm·var is 30 ms (15 ms: F (3,27) = 0.529, p = 0.666; 20 ms: F (3,27) = 0.419, p = 0.741; 30 ms: F (3,27) = 14.137, p < 0.001; 50 ms: F (2,18) = 1.902, p = 0.178; 75 ms: F (3,27) = 0.210, p = 0.658); in the globalterm·key task , the main effects of stimulus lengthterm·var are significant for all levels of segment durationterm·var except for 75 ms (15 ms: F (3,27) = 35.66, p < 0.001; 20 ms: F (3,27) = 42.89, p < 0.001; 30 ms: F (3,27) = 48.28, p < 0.001; 50 ms: F (2,18) = 15.43, p < 0.001; 75 ms: F (3,27) = 0.149, p = 0.709).
Next we examined the effect of segment durationterm·var on different tasks. A two-way repeated measures ANOVAterm·stat with the factors task and segment durationterm·var was conducted at each level of stimulus lengthterm·var, followed by a one-way repeated measures ANOVAterm·stat to evaluate the main effect of segment durationterm·var within each task at each level of stimulus lengthterm·var. The interaction between task and segment durationterm·var is significant for all levels of stimulus lengthterm·var (300 ms: F (5,45) = 143.84, p < 0.001; ~220 ms: F (4,36) = 74.61, p < 0.001; ~150 ms: F (3,27) = 35.13, p < 0.001; ~90 ms: F (2,18) = 14.24, p < 0.001). The main effect for segment durationterm·var is significant when stimulus lengthterm·var is ~220 ms (F (4,36) = 2.738, p = 0.044) and 150 ms: (F (3,27) = 4.285, p = 0.013) and ~90 ms: (F (2,18) = 25.13, p < 0.001). The main effect for task is not significant across all stimulus lengthsterm·var (300 ms: F (1,9) = 0.66, p = 0.437; ~220 ms: F (1,9) = 1.883, p = 0.203; ~150 ms: F (1,9) = 1.103, p = 0.321; ~90 ms: F (1,9) = 0.173, p = 0.687). In post-hoc comparisons, the main effect for segment durationterm·var is significant in both tasks at all levels of stimulus lengthterm·var ( local taskterm·def , 300 ms: F (5,45) = 93.33, p < 0.001; ~220 ms: F (4,36) = 54.10, p < 0.001; ~150 ms: F (3,27) = 33.02, p < 0.001; ~90 ms: F (2,18) = 25.85, p < 0.001; globalterm·key task , 300 ms: F (5,45) = 57.83, p < 0.001; ~220 ms: F (4,36) = 41.88, p < 0.001; ~150 ms: F (3,27) = 19.44, p < 0.001; ~90 ms: F (2,18) = 3.708, p = 0.045). Linear trend contrasts showed an upward trend for the local taskterm·def and a downward trend for the globalterm·key task , except when stimulus lengthterm·var was ~90 ms, F (1,9) = 2.474, p = 0.150.
In Experiment 1B, a two-way repeated measures ANOVAterm·stat on d primeterm·var values with the factors task and number of segments revealed a significant effect of the number of segments, F (4,36) = 14.88, p < 0.001, and task, F (1,9) = 5.684, p = 0.041, and a significant interaction between task and number of segments, F (4,36) = 38.54, p < 0.001. A one-way repeated measures ANOVAterm·stat showed a significant effect of the number of segments for both the local taskterm·def , F (5,45) = 64.43, p < 0.001, and the globalterm·key task , F (4,36) = 25.13, p < 0.001. Linear trend contrasts showed a downward trend for the local taskterm·def , F (1,9) = 99.44, p < 0.001, and an upward trend for the globalterm·key task , F (1,9) = 25.47, p = 0.001. We further investigated how the subjects’ response criterion varied with stimulus lengthterm·var. A two-way repeated measures ANOVAterm·stat on criterion (C) with the factors of task and number of segments revealed a significant interaction between task and number of segments, F (4,36) = 18.81, p < 0.001. A one-way repeated measures ANOVAterm·stat showed a significant effect of the number of segments for the local taskterm·def , F (5,45) = 7.47, p < 0.001, but not for the globalterm·key task , F (4,36) = 2.27, p = 0.081. Linear trend contrasts showed a downward trend for the local taskterm·def , F (1,9) = 11.18, p = 0.009.
The performance on the local taskterm·def shows that differentiating sounds based on detailed, localterm·key spectrotemporal information does not increase with increasing stimulus lengthterm·var. According to the multiple-looksterm·dom and STEP models, longer stimuli provide a larger amount of acoustic information for the auditory systemterm·key to draw more samples or looks. The auditory systemterm·key should form more distinct representations of stimuli and lead to better performance. However, the results in both Experiments 1A and 1B suggest that the auditory systemterm·key fails to (be able to) utilize the extra information provided in the longer stimuli, since performance did not increase. The performance of the local taskterm·def remained at the lowest level across different stimulus lengthsterm·var when segment durationterm·var was short (15 or 20 ms). One possible explanation for the floor effect is that the auditory systemterm·key cannot resolve detailed information at such short localterm·key scales. A ceiling effect may also explain that stimulus lengthterm·var has no effect when segment durationsterm·var are 50, 75 or 100 ms. At those scales, the listeners can resolve the detailed spectrotemporal information and perceive the sweep directionsterm·var of the segments. Therefore, the effect of increased stimulus lengthterm·var cannot be observed probably because of a ceiling effect. In contrast, when segment durationterm·var was 30 ms, stimulus lengthterm·var exerted a clear negative influence on the local taskterm·def performance, as shown by the main effect of stimulus lengthterm·var in Experiment 1A and again in Experiment 1B. Furthermore, in Experiment 1B, the criterion in the local taskterm·def increased with stimulus lengthterm·var, which indicates that subjects became worse at analyzing localterm·key details and tended to consider two sounds in the ‘different’ pair to be the same. Therefore, these results suggest a more complicated temporal integrationterm·dom process in the auditory systemterm·key, one that might be consistent with the 2TWI model.
In Experiment 1A, the scale of tens of milliseconds seems to apply to perception of fine-grained information in the local taskterm·def . Performance is poor when segment durationterm·var is only 15 ms (mean of d primeterm·var value below 1) and becomes very good (and then levels off) when segment durationterm·var is 50 ms (mean d primeterm·var value greater than 3). To generate a good estimation of localterm·key acoustic details, the auditory systemterm·key may need to integrate information over tens of milliseconds. The range around 30–50 ms seems to be a lower bound for a timescaleterm·dom that facilitates access to such detailed information. These results are consistent with the fact that speech perception requires fine-detailed decoding of phonemic information, which also falls on a scale of tens of milliseconds 1 .
On the other hand, the globalterm·key task shows the reverse pattern, which is predicted by the multiple-looksterm·dom and STEP models: participants’ performance on the globalterm·key task increases with increasing stimulus duration, arguably because stimuli of longer duration provide more acoustic information, and a globalterm·key pattern can be better recognized as more information is integrated. In contrast, longer segment durationterm·var causes worse performance in the globalterm·key task . When the localterm·key scale increases, each stimulus contains fewer segments. Therefore, less information is available for extracting a globalterm·key pattern.
The results of Experiment 1 are consistent with recent findings by McDermott et al . 46 and Piazza et al . 47 , who also showed that increased stimulus lengthterm·var helps the auditory systemterm·key perceive globalterm·key statistical patterns but can impede access to localterm·key acoustic detail.
Experiment 2 In Experiments 1A and 1B, participants could in principle use a strategy of monitoring the spectral change of a single segment between two stimuli to respond in the globalterm·key task . That is, subjects could make their responses based on the frequency differenceterm·var between the onsets or the offsets of stimuli. To address this potential confound, in Experiment 2 we kept the frequency differenceterm·var between two stimuli at the onset and the offset the same and made all segments different between two stimuli. Only a segment durationterm·var of 30 ms is only used in this experiment (as well as in Experiments 3 and 4), because previous behavioral studies 48 found that frequency modulation sweeps of 30 ms ensure good perception of sweep directionterm·var and, therefore, good access to localterm·key details. Furthermore, as shown in Experiment 1A, at the segment durationterm·var of 30 ms, participants’ performance did not reveal floor or ceiling effects, so a modulatory effect of stimulus lengthterm·var could be well observed.
Method Stimuli and apparatus We generated two groups of stimulus pairs, with 3, 5, 7, 9 and 11 segments. The factor segment number (similar to stimulus lengthterm·var because we fixed the segment durationterm·var at 30 ms) was manipulated in two conditions. In condition 1 , each pair of stimuli included one stimulus with mean frequency shifting from 1300 Hz to 1700 Hz, regardless of the number of segments, and the other stimulus with mean frequency constant at 1500 Hz ( Fig. 2a ). In condition 2 , each pair of stimuli included one stimulus with mean frequency shifting from 1700 Hz to 1300 Hz and the other stimulus with mean frequency kept at 1500 Hz ( Fig. 2a ). The stimuli within a pair always had segments of opposite sweep directionsterm·var. The frequency differenceterm·var between two stimuli at the onset was the same as the frequency differenceterm·var at the offset, so the participants could not rely on spectral cues to judge the trend of the frequency shift. Furthermore, because each segment swept in different directions, participants could not monitor the spectral change of a single segment between two stimuli to generate a response.
Figure 2. Stimulus spectrograms and results of Experiment 2.
( a ) Examples of spectrograms for the stimulus pair, in which one of the stimuli has flat globalterm·key mean frequency and the globalterm·key frequency of the other stimulus shifts upwards. Results of Experiment 2 ( b ) are plotted as a function of stimulus lengthterm·var (x-axis). Although the frequency differenceterm·var between the two stimuli is kept constant, the ability to recognize globalterm·key frequency trends increases as a function of stimulus lengthterm·var.
Procedure and design We used a two-interval two-alternative-forced choice paradigm and block design for each condition. Participants reported which interval shifted upward in condition 1 , and which interval shifted downward in condition 2 . For each number of segments, 30 trials were presented (total of 720 trials).
Results and Discussion Figure 2b summarizes the results: as the number of segments (i.e. stimulus lengthterm·var) increased, d primeterm·var value s increased. A two-way repeated measures ANOVAterm·stat revealed that the main effect was significant for number of segments F (4,36) = 23.07, p < 0.001, but not for shifting direction F (1,36) = 2.35, p = 0.160. There was no interaction F (4,36) = 1.83, p = 0.145. The results show that, when the frequency differencesterm·var of onset and offset between two stimuli were kept the same for all stimulus lengthterm·var, it became very difficult for participants to identify which interval had a frequency shift when sound duration was short (<150 ms). Therefore, this experiment rules out the possibility that the results of Experiment 1 were caused by the frequency differenceterm·var between the stimuli.
Experiment 3 In the local taskterm·def , longer stimulus lengthterm·var did not increase participants’ performance. One explanation is that more segments require more processing time to represent localterm·key details. Within a short period of time, the acoustic details of only several segments can be fully accessed. When the amount of information extends beyond the capacity of processing within a certain time window, the auditory systemterm·key ‘summarizes’ the localterm·key details and forms a representation of a globalterm·key pattern. Another explanation is that increased stimulus lengthterm·var increases the amount of information, namely the number of segments, carried in the working memory. The increased working memory loadterm·var may potentially degraded participants’ performance in the local taskterm·def. In this experiment, we keep the number of segments constant while inserting intervals between segments, to assess whether localterm·key details can be recovered by providing more processing time. If the working memory loadterm·var plays a key role here, inserting intervals while keeping the memory load consistent will not change the participants’ performance.
We hypothesize that acoustic information can only be integrated within a limited temporal range. By keeping constant the information provided by stimuli (the number of segments) while increasing stimulus lengthterm·var by inserting gaps between segments, we can test whether the auditory systemterm·key can store all segments and integrate acoustic details over a longer time range. If the acoustic information can be integrated without a time constraint, the stretched stimuli will not affect the performance in the globalterm·key task . Otherwise, the performance will decrease as the size of intervals increases.
In Experiment 3A, we examine the amount of additional time the auditory systemterm·key needs to recover fine-detailed information. In Experiment 3B, we ask whether temporal integrationterm·dom functions best on a time scale of hundreds of milliseconds. We fix both the number of segments and segment durationterm·var while manipulating overall stimulus lengthterm·var by inserting gaps between segments.
Method Stimuli and apparatus Two additional factors, the number of segments and inter-onset interval (IOI)term·def, were included in this experiment. The IOI is defined here as the duration between the onset of one segment and the onset of the next segment (see Fig. 3a ).
Figure 3. Spectrogram of stimuli and results of Experiment 3A and 3B.
( a ) The left panel shows the spectrograms of the original stimuli for the localterm·key and globalterm·key tasks, and the right panel shows the spectrograms of the stimuli after the intervals are inserted. ( b ) The results as a function of inter-onset intervals (x-axis) and d primeterm·var value (y-axis). The dashed line represents results of Experiment 3A, the local taskterm·def and the solid line represents results of Experiment 3B, the globalterm·key task . As the inter-onset interval increases, it facilitates access to the fine-detailed information, while the ability to recognize the globalterm·key sound pattern decreases because acoustic segments fall out of the putative temporal integrationterm·dom window.
In Experiment 3A, segment durationterm·var was fixed at 30 ms and stimuli with 3 and 5 segments were used. Six levels were used for the factor IOI: 50, 60, 80, 110, 155, and 230 ms. Half of trials were ‘same’ and half were ‘different’ pairings. A large IOI may make the first and the last segments more salient than middle segments, because no segment precedes the first segment and no segment follows the last segment. Participants may just pay attention to the first or the last segment to do the tasks instead of resolving acoustic details of the whole stimulus. To avoid this potential confound, the sweep directionsterm·var of the segments in the beginning and the end of the stimuli were the same between two stimuli in both ‘same’ and ‘different’ pairs; the segments in the middle of the stimuli were the same in the ‘same’ pair, and different in the ‘different’ pair.
In Experiment 3B, stimuli with 5 and 7 segments were used. The same six levels were used for the factor IOI. The stimuli within both ‘same’ and ‘different’ pairs always had segments of opposite sweep directionsterm·var. For ‘same’ pairs, regardless of the number of segments, the stimuli either had a mean frequency increasing with the number of segments from 1300 Hz to 1700 Hz or a mean frequency decreasing from 1700 Hz to 1300 Hz. For ‘different’ pairs, the stimuli had opposite directions of frequency shifting trend.
Procedure and design Stimuli with different number of segments were presented in separate blocks, and in each block the order of IOIs was randomized. The order of the blocks was counterbalanced across participants. For each IOI, the stimuli were the same for 30 trials, and different for 30 trials. In total, 720 trials were presented for Experiment 3A and Experiment 3B, respectively. Around ten trials for each number of segments at each IOI were presented with feedback for training purposes before the experiment.
Results and Discussion Figure 3b shows that d primeterm·var value s increased as the IOIs increased in the local taskterm·def (Experiment 3A) but decreased in the globalterm·key task (Experiment 3B). A two-way mixed factor ANOVAterm·stat with task (the local taskterm·def in Experiment 3A and the globalterm·key task in Experiment 3B) as a between-subject factor and IOI as a within-subject factor revealed significant main effects of task, F (1,18) = 6.37, p = 0.021, and IOI, F (5,90) = 5.739, p < 0.001, and a significant interaction, F (5,90) = 55.13, p < 0.001.
In Experiment 3A, d primeterm·var value s increased as the IOIs increased. A two-way repeated measures ANOVAterm·stat shows that the main effects of the number of segments F (1,9) = 10.80, p = 0.001 and the IOI, F (5,45) = 43.89, p < 0.001 were significant and that the interaction between the number of segments and the IOIs was also significant, F (5,45) = 2.83, p = 0.027. In the post-hoc test, we found significant upward linear trend for both 3 segments, F (1,9) = 72.22, p < 0.001, and 5 segments, F (1,9) = 48.30, p < 0.001.
In Experiment 3B, d primeterm·var value s decreased as the IOIs increased. A two-way repeated measures ANOVAterm·stat showed that the main effect was significant for IOI, F (5,45) = 14.99, p < 0.001, but not for the number of segments, F (1,9) = 2.34, p = 0.161. No significant interaction effect was found, F (5,45) = 0.88, p = 0.504. Therefore, we combined the data from trials with different number of segments into one dataset, and the linear trend contrasts showed a downward trend F (1,9) = 32.87, p < 0.001.
The results from Experiment 3A suggest that the auditory systemterm·key requires time to process and represent fine-grained information. The intervals between segments and, of course, the duration of segments are both important for extracting fine-grained information from the stimuli. Similar results have been found in a speech perception study 49 in which inserting intervals into compressed speech increased intelligibility. Contrary to the results of Experiment 1B, it seems that participants in this experiment are better at the local taskterm·def using 5-segment stimuli than 3-segment stimuli. We suspect that the discrepancy is caused by the fact that we fixed the first and the last segments in Experiment 3A, so differences between two stimuli within the ‘different’ pair are much smaller in 3-segment stimuli (only one segment is different) than in 5-segment stimuli (three segments are different).
The results from Experiment 3A also demonstrate that the working memory loadterm·var cannot explain the effect of stimulus lengthterm·var in the local taskterm·def of Experiment 1. By inserting intervals into the stimuli, we kept the working memory loadterm·var constant and stretched the stimuli to make the task more memory demanding, but the participants’ performance became better rather than worse. Therefore, the effect of stimulus lengthterm·var is related more to properties of auditory processing than to effects of the working memory loadterm·var.
Importantly, in Experiment 3B, the ability to recognize the globalterm·key sound pattern decreases as IOI increases. Grouping acoustic details and abstracting a globalterm·key pattern seem only to work well within a particular, restricted time range. Only if the necessary information falls within a short time range (e.g. hundreds of milliseconds), can the auditory systemterm·key integrate acoustic information and abstract the globalterm·key trend. Otherwise, the stimuli start to sound like a sequence of individual acoustic segments instead of a coherent, integrated sound pattern, so localterm·key details become salient and could potentially affect the integration process.
Experiment 4 In the local taskterm·def of Experiment 1, the same-different judgment of two stimuli based on fine-grained information may be influenced by memory constraints. That is, when we varied stimulus lengthterm·var, this lengthened the amount of time participants needed to remember information about the first stimulus in the stimulus pair. If memory degraded during that time, this may have been one factor contributing to the effect of stimulus lengthterm·var. To examine this hypothesis, the interval between two stimuli within a pair was varied in Experiment 4. If memory plays an important role in the local taskterm·def of Experiments 1A and 1B, performance should decrease as the interval between two stimuli increases.
Method Stimuli and apparatus The factors, segment number (i.e. overall stimulus lengthterm·var) and inter-stimulus interval (ISI)term·var, were manipulated. We generated stimulus pairs with 1, 3, 5, and 11 segments. ISIs were varied across four levels: 500, 700, 1000 and 1500 ms.
Procedure and design The procedure was the same as in the local taskterm·def of Experiment 1. Pairs of stimuli with different segments and ISIs were pseudo-randomly presented. All stimuli were presented in one block. For each number of segments and each ISI, 30 trials were presented, for a total of 960 trials. Around ten trials for each number of segments at each ISI were presented with feedback for training purposes before the formal experiment.
Results and discussion Figure 4 illustrates that d primeterm·var value s did not change across different ISIs for a given stimulus lengthterm·var. A two-way repeated measures ANOVAterm·stat shows that the main effect was significant for number of segments F (3,27) = 94.51, p < 0.001, but not for ISIs F (3,27) = 0.06, p = 0.982. There was not enough evidence to show a significant interaction effect F (9,81) = 1.76, p = 0.090. These results are consistent with previous work in which the interval between stimuli is varied systematically to examine the effect of auditory memory; no significant effect of interval between stimuli is found 50 . Although memory requirements may come into play in our experiment due to the characteristics of the same-different procedure, the results of this control experiment demonstrate that memory cannot explain the main results obtained in the previous experiments.
Figure 4. Results of Experiment 4.
The d primeterm·var values do not vary with inter-stimulus intervals, which suggests that memory factors do not affect the results in Experiment 1. Different line types represent stimuli with different number of segments.
General Discussion We report the findings from four behavioral studies designed to investigate how auditory information on different timescalesterm·dom is integrated. Experiment 1 showed that the auditory systemterm·key can extract fine-temporal-detail information as well as integrate acoustic chunks to form a globalterm·key percept. However, longer stimuli only improved integration in a globalterm·key task , but did not increase performance on extracting fine-grained information in a local taskterm·def . Experiment 2 suggested that potential frequency differenceterm·var cues between stimuli did not affect the results of temporal integrationterm·dom obtained in Experiment 1. Experiment 3A showed that decoding and representing the detailed acoustic information required processing time. Experiment 3B suggested that temporal integrationterm·dom functions best within a limited time range. Experiment 4, finally, controlled for the possibility that memory factors affected the conclusions of Experiment 1.
How the auditory systemterm·key extracts fine-grained information and integrates acoustic information over time remains a key issue for auditory processing. Recent studies by McDermott et al . 46 , 51 and Piazza et al . 47 argue that the auditory systemterm·key uses a form of statistical summation to integrate acoustic information over time to generate auditory percepts, but the temporal range of putative statistical summation processes in these studies was not clearly articulated. Durations of the stimuli used in McDermott et al . 46 range from 40 ms to more than 2 seconds, and results similar to our findings were found: when discrimination depends on the details of the stimuli, performance decreases with sound duration; when discrimination depends on the long-term statistics of the stimuli, performance increases with sound duration. In Piazza et al . 47 , the sound duration used is more than two seconds, which is a different time range than that used in the present study. In both studies, short-term memory may be involved in the tasks, so the summation process cannot be defined with certainty as a characteristic of the auditory systemterm·key independent from additional memory factors.
In Experiment 1 of the present study, the stimulus lengthterm·var was varied from 30 ms to 330 ms. The data from Experiment 1A demonstrated that access to the fine-grained information of the stimuli improves as segment durationterm·var increased over tens of milliseconds - but deteriorates with increased stimulus lengthterm·var. The results of Experiment 1B showed that the representation of the globalterm·key pattern increased to about 200 ms. These results are consistent with the psychophysical studies that reported two time constants in the auditory systemterm·key 8 . Moreover, these results suggest that auditory processing is different on these two timescalesterm·dom. On the timescaleterm·dom of hundreds of milliseconds, acoustic information can be integrated and well represented while the representation of fine-grained information is compromised; on the timescaleterm·dom of tens of milliseconds, the acoustic information is processed but arguably not ‘fully’ represented to subserve subsequent perceptual analysis.
The results cannot be explained in a straightforward manner by models assuming a standard process such as a leaky integrator, because the stimuli in the present study have a multiplexed temporal structure, and the task does not require integration of either the envelope or the sound energy. Although the multiple-looks modelterm·def implies that the auditory systemterm·key samples acoustic information on a short timescaleterm·dom, the details of the sampling process are not specified. Furthermore, as stimulus lengthterm·var increases, there should be more opportunities for sampling acoustic information. However, we find that the performance of discrimination between two stimuli with different fine-grained information decreases, which is inconsistent with the prediction of a multiple-looksterm·dom model and a STEP model. The findings of the present experiments suggest a more complicated temporal integrationterm·dom process in the auditory systemterm·key: the acoustic segments are analyzed within a short temporal windowterm·dom for extracting localterm·key details and concurrently acoustic information is integrated within a larger temporal windowterm·dom to form a globalterm·key percept.
In Experiment 3, access to fine-detail acoustic information is recovered if sound segments are followed by a silent interval of hundred milliseconds. This suggests that the representation of fine acoustic information can only be well-formed within a temporal windowterm·dom of this range. The results enrich explanations of a recent study of speech perception 49 , in which compressed speech becomes more intelligible after inserting intervals between segments, and also echo earlier findings (for a review, see Massaro, 1972). It is worth bearing in mind that, in a range of neurophysiological studies, ~150–300 ms are around the length of one cycle of neural oscillations in the theta band (~3–7 Hz) that are found in speech perception and other auditory tasks 30 , 31 , 34 , 36 , 37 , 52 , 53 . Experiment 3 provides behavioral evidence that the auditory systemterm·key chunks incoming information into units of the appropriate temporal granularity 29 .
Inspired by the multiple-looksterm·dom and STEP models, we suggest a slightly different temporal integrationterm·dom process to account for the present results. We hypothesize that two stages of temporal integrationterm·dom exist in the auditory systemterm·key. In the first stage, the acoustic information is sampled using two temporal windowsterm·dom, a localterm·key one in the tens-of-milliseconds range and a globalterm·key one in the hundreds-of-milliseconds range. Detailed information is sampled by the localterm·key temporal windowterm·dom so that fast fluctuations and fine-grained information can be analyzed in a high-time-resolution manner; in contrast, acoustic patterns or statistics, such as pitch and loudness, are better abstracted using the globalterm·key temporal windowterm·dom so that robust estimation of these acoustic attributes can be achieved. In a second stage, the sampled information is channeled to a representational stage with limited information capacity and probably with a time constant of hundreds of milliseconds. The information at this representational stage supports making responses and memory storage. Because it is unlikely that all detailed information can be represented or stored, the representation of sounds may not simply be an entire set of acoustic samples. The information is ‘smoothed’ or sparsely coded. Attention driven by task demandsterm·var at this stage can bias representation so that only selected information is represented.
Such a proposed integration model can explain four main findings in the present study: Longer stimulus lengthterm·var does not increase performance in the local taskterm·def as demonstrated by Experiment 1. Although the localterm·key details can be well sampled as suggested by the multiple-looks modelterm·def and STEP using a short temporal windowterm·dom, the representation of the localterm·key details is not a vector of samples, as STEP suggests. When the segment size is fixed, a longer stimulus means more samples of segment directions are presented. However, the limited information capacity of the representation stage forces the auditory systemterm·key to smooth or summarize all information. The details are lost at the representation stage, if too much information is squeezed into the representation. In contrast, when there are only a few segments in the stimuli, the amount of detailed information is low and the information can be well represented, since no ‘smoothing’ or ‘summarizing’ process is implemented. Therefore, increasing stimulus lengthterm·var does not improve performance, and the performance in the local taskterm·def is best at short stimulus lengthterm·var.
Globalterm·key perception is best in the range of hundreds of milliseconds. The auditory systemterm·key employs a long temporal windowterm·dom to abstract information on a globalterm·key scale at the first stage and, depending on task requirements, represents globalterm·key patterns in this representation stage. This contrasts with models suggesting that a globalterm·key process is increasingly accumulating information derived from a short integration scale. This can explain a sharp change of performance at around 150 ms of stimulus lengthterm·var in the globalterm·key task of Experiment 1B. As stimulus lengthterm·var increases to a point, a globalterm·key pattern emerges and is sampled by the long temporal windowterm·dom. When the segment durationterm·var is varied, the small segment durationterm·var (<50 ms) does not affect the globalterm·key percept because of the direct sampling on a globalterm·key scale. When segment durationterm·var increases above 75 ms, although each segment still carries information of globalterm·key trend, localterm·key directions of segments are sampled by long temporal windowterm·dom so the recognition of globalterm·key pattern decreases.
An interval is needed for access to localterm·key details shown by Experiment 3. Given a certain amount of time, the representational capacity is limited. After intervals are inserted between segments, information per unit time decreases, so localterm·key details can be represented in a more precise manner. Therefore, the performance in the local taskterm·def of Experiment 3A gets better as the length of intervals increases.
Inserting intervals decreases performance in a globalterm·key task as in Experiment 3B. As the long temporal windowterm·dom is around hundreds of milliseconds, inserting intervals stretches the stimuli and makes few segments fall within one long temporal windowterm·dom, so a globalterm·key pattern of the stretched stimuli cannot be sampled.
Conclusion Given the results of the present study, the closest heuristic model that accounts for the data is a two-timescale model: one temporal windowterm·dom works on short timescaleterm·dom (<50 ms) to resolve and integrate fine-grained information; concurrently another window works on a long timescaleterm·dom (~200 ms) to extract the globalterm·key pattern of sound segments. Such a multi-time resolution hypothesis, proposed by PoeppelpersName (2001, 2003) for speech is probably a general mechanism for audition, not just a specific model for speech perception as was originally argued. Our results thus build on the previous models of temporal integrationterm·dom and shed light on the multiple timescalesterm·dom of auditory processing.
Additional Information How to cite this article : TengpersName, X. et al . Testing multi-scale processing in the auditory systemterm·key. Sci. Rep. 6 , 34390; doi: 10.1038/srep34390 (2016).
Acknowledgments We thank Jeff Walker for his technical support. We also thank Jess Rowland, Keith Doelling, Gwyneth Lewis, and Adeen Flinker for many helpful comments on the manuscript. This study was supported by 2R01 DC05660 to D.P., and National Natural Science Foundation of China 31500914 and Major Program of Science and Technology Commission of Shanghai Municipality (STCSM) 15JC1400104, and Program of Introducing Talents of Discipline to Universities, Base B16018 to X.T.
Footnotes Author Contributions X.T. designed research, performed the experiments, analyzed data and wrote the manuscript. X.T. and D.P. contributed to discussion and improved the manuscript.