Operant conditioning

Staddon, Niv · Scholarpedia · 2008 · behavioural-psychology · 10.4249/scholarpedia.2318 · record Staddon2008-operant-conditioning
tag layer · index · concepts · authors · עברית
Where this record's text came from. faceted 2026-08 by reading the locally held text (html); every evidence string verified as an exact substring of that text before merging. limitations is empty by design: the article states limits of the field and of other people's theories (e.g. 'There are limits, therefore, to what can be learned simply by studying supposedly reversible steady-state behavior in individual organisms.' and 'In all these theories, however, nothing is said about the shaping of the response itself, or response topography.') but never states a limit of its own coverage, so nothing qualified under the 'its own limits' rule. The single claim_relation is the article's own verdict against Herrnstein's matching law, named one sentence earlier ('The most well-known molar relationship is the matching law, first stated by Richard Herrnstein in 1961.'). Method is tagged review because this is a Scholarpedia survey article; it never labels its own method, so the method evidence is the sentence in which it classifies the existing theoretical literature. 2026-08-12 and 2026-08-15 schema-0.3 passes: every quote verified as an exact substring of the staged text and anchored at merge time; concept terms verified present; every edge endpoint an existing node.
This record’s own method is review, and the tags below follow from that. A review’s findings, quantities and statistics are things stated IN this paper, and some of them report results obtained elsewhere. No tag on this page names a different author. Measured across every review in this corpus, 81% of a review’s tags carry no citation at all — most of them are the review speaking. The corpus does not adjudicate the rest: read an unmarked tag as a claim this paper makes, not necessarily one it made first.
How to read this page. The article itself is not reproduced here — this paper is not cleared for redistribution, so what follows is the tag layer alone. The evidence sentences are withheld with the article. The table at the foot lists every tag the record carries; each row names the facet it belongs to, and the whose words column says whether the tag records the paper's own wording or the tagger's sentence about the paper. Every tag links to each other paper in the corpus carrying it. The machine-readable edition of the same thing is the TEI file beside this one, where the tags live in standOff, and the article is at its publisher via the link below. When we have changed our mind. Some entries in the table begin with a word in capitals and the reason for it. EVIDENCE CORRECTED — the finding still stands, but we had quoted the wrong sentence from the paper, and the quotation has been replaced with the right one. RETRACTED — the entry itself was wrong; it stays on the page, marked, with the reason, rather than being deleted. WITHDRAWN AND REPLACED — the entry was wrong, and a corrected version of it appears in the entries below. Each mark carries its own reason, in brackets, because the reasons differ from one entry to the next. An entry we have WITHDRAWN is removed from this page and kept in the record, with its reason. A line reading Audit and a date is a later re-reading of the entry against its whole sentence: DOUBTFUL means it was kept, and the reason says what is loose about it. Quotation marks inside the tagger's sentence. Where a tag is the tagger's own sentence, a phrase in quotation marks inside it is borrowed from the paper — the tagger's wording around it is not. The paper's own sentence is the evidence quote for that row; on this page it is withheld with the article, and the article is at its publisher. Nothing inside quotation marks anywhere on this page is ours: if it is quoted, it is the paper's. Whose words a concept carries. A concept entry records a term the paper itself uses, so the term, the sense given for it, and its evidence are the paper's own words, quoted where the paper fixes them. This is the opposite of a sense entry in the tag table, which is the tagger's sentence about how the paper uses a term. Where a concept entry could not be grounded in the paper's words, the entry says so on its own row.

Tags on this paper

Every tag the record carries, each linking to every other paper in the corpus that carries it. A tag appears here whether or not its words occur in the article — the spans below are only the ones a sentence in this paper licenses. Of the 9 concept tags on this paper, all 9 are grounded in a quoted sentence. All carry the tagger's gloss of what the term means in this paper, which is not something the article can license.
operant conditioning instrumental conditioning ontogenetic selection phylogenetic selection Pavlovian conditioning reinforcer punisher contingency natural selection Reinforcement acts as a selective agent on emitted behaviour, its strength set by how soon the reinforcer follows the response, how often the pairing occurs, and how well the response predicts the reinforcer The predictive properties of the situation build, through innate food-related response tendencies, the repertoire of activities out of which reinforcement can then select A food-predicting stimulus recruits built-in mechanisms that bias the animal's repertoire towards food-getting acts, which is why behaviour appears before any operant reinforcement Arousal has a bitonic effect because increasing predictiveness simultaneously raises response vigour and narrows the repertoire, so past a point the effective response drops out Matching emerges because reinforcement probability on a VI schedule falls as responses accumulate and rises with time away, so any reward-following rule produces it reinforcement acts as a selective agent through three quantities: how soon it follows the response, how often, and how well the response predicts it contingency is about predictiveness, not pairing: adding free food WEAKENS lever pressing, because the lever stops predicting the food the act closest in time to the reinforcer is the one strengthened - the principle of postremity, and the origin of the Law of Effect the elicited/emitted distinction: reflexive behaviour is triggered by a stimulus, operant behaviour is emitted before any stimulus is attached to it response strength Law of Effect assignment of credit Yerkes-Dodson law interval schedule ratio schedule matching law habitual behavior response rate reinforcement delay, rate and contingency review

Legend

second readingalternativesan independent re-reading of a facet, recorded whether or not it agrees
relation to other workclaim_relationshow this paper stands to another: agrees, contradicts, extends, qualifies
claim typeclaim_typethe shape of the claim: existence, mechanism, method, critique, prediction, null
concept edgeconcept_edgesan asserted relation between two concepts, with the sentence asserting it
conceptconceptsan idea the paper works with, quoted where the paper uses it
content tagcontent_tagsa finding, definition, quantity, statistic or variable named in the paper
contributioncontributionwhat the paper claims to add
limitationlimitationswhat the paper says its evidence cannot support
mechanismmechanismshow the reported effect is said to come about
methodmethodhow the paper produced its evidence
object of studyobject_of_studywhat the paper is about — the thing it investigates
phenomenonphenomenonthe effect or regularity under study
research questionresearch_questionthe question the paper sets itself
sensesensesa distinct use of a term, quoted where the paper fixes it
transfertransfersa method or concept carried in from another field

The paper

The full text is not reproduced here. This article is not cleared for redistribution, so the page carries the tag layer only. Its evidence sentences are withheld; the grounding status remains with each tag, and the article itself is at https://doi.org/10.4249/scholarpedia.2318. The reading view exists for the papers whose licence permits it.

Every tag on this page

alternatives 3 · claim_relations 3 · claim_type 1 · concept_edges 6 · concepts 9 · content_tags 28 · contribution 1 · limitations 3 · mechanisms 9 · method 1 · object_of_study 1 · phenomenon 1 · research_question 1 · senses 4 · transfers 4

Relations on this page carry a quotation for the claim. Where a relation was separately evidenced — and so tested against the vocabulary for its type — the row says so; 1 of 6 relations here carries that mark. The rest are the tagger’s reading of the sentence quoted beside them.

This record holds no controls. A layer is empty when the tagger recorded nothing in it — which is not the same as the paper having nothing to record.

facettagfieldconf.whose words
object_of_studyOperant conditioning as a process and as an experimental paradigm - how behaviour is selected by its consequences.
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
object_of_studythe paper's words
research_questionWhat is the relation between behaviour and its consequences, and how far does the analogy with natural selection actually carry?
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
research_questionthe paper's words
contributionSets out operant conditioning as ontogenetic selection - behaviour varied then selected by reinforcement - and flags that the variation half of the analogy has gone unstudied.
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
contributionthe paper's words
claim_typecritique
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
claim_typethe paper's words
phenomenonoperant conditioning
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
phenomenonthe paper's words
methodreview
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
also in: review
methodthe paper's words
mechanismsmechanisms
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
mechanisms[0]the tagger's words
mechanismsmechanisms
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
mechanisms[1]the tagger's words
mechanismsmechanisms
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
mechanisms[2]the tagger's words
mechanismsmechanisms
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
mechanisms[3]the tagger's words
mechanismsmechanisms
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
mechanisms[4]the tagger's words
mechanismsmechanisms
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
duplicate of: mechanisms[0]
mechanisms[5]1.0the tagger's words
mechanismsmechanisms
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
mechanisms[6]1.0the tagger's words
mechanismsmechanisms
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
mechanisms[7]1.0the tagger's words
mechanismsmechanisms
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
mechanisms[8]1.0the tagger's words
limitationsThe theoretical account is incomplete by the article's own statement - the empirical space is mapped, the mechanism is not.
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
what this limits: the mechanistic account: EXACTLY HOW ALL THIS WORKS IS STILL NOT UNDERSTOOD IN FULL THEORETICAL DETAIL, though the empirical space is mapped
limitations[0]1.0the tagger's words
limitationsA century of research has studied selection and not variation: where the first successful response comes from is unexamined.
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
what this limits: the scope of the whole research programme: OMITTED IS ANY DISCUSSION OF WHERE THE SUCCESSFUL RESPONSE COMES FROM IN THE FIRST PLACE - selection is studied, variation is not
limitations[1]1.0the tagger's words
limitationsThe framework that organised the field is described as not really a theory.
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
what this limits: any use of the Skinnerian framework as an explanatory theory: SKINNER'S 'THEORY' WAS MORE AN ORGANIZING FRAMEWORK THAN A TRUE THEORY
limitations[2]1.0the tagger's words
transfersThe credit-assignment problem is taken over from AI and made the central question that an operant conditioning process must answer
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
transfers[0]the tagger's words
transfersDarwin's natural selection is imported as the model for operant conditioning, with reinforcement as the selective agent acting on a variable repertoire of emitted acts
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
transfers[1]the tagger's words
transfersThe reinforcement-learning framework is adopted as the normative account within which both Pavlovian and operant behaviour are interpreted
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
transfers[2]the tagger's words
transfersDarwinian selection from a varied population, imported to explain how one behaviour is picked out of an emitted repertoire by reinforcement
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
transfers[3]1.0the tagger's words
conceptsoperant conditioning
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
concepts[0]1.0the paper's words
conceptsinstrumental conditioning
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
concepts[1]1.0the paper's words
conceptsontogenetic selection
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
concepts[2]1.0the paper's words
conceptsphylogenetic selection
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
concepts[3]1.0the paper's words
conceptsPavlovian conditioning
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
concepts[4]1.0the paper's words
conceptsreinforcer
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
also in: reinforcer
concepts[5]1.0the paper's words
conceptspunisher
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
also in: punisher
concepts[6]1.0the paper's words
conceptscontingency
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
also in: contingency
concepts[7]0.85the paper's words
conceptsnatural selection
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
concepts[8]1.0the paper's words
sensesselection - ontogenetic: consequences shaping behaviour within one animal's lifetime
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
senses[0]1.0the tagger's words
sensesselection - phylogenetic: consequences shaping a species over evolutionary time
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
senses[1]1.0the tagger's words
sensesconditioning - operant: the animal has control, and outcomes are contingent on its behaviour
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
senses[2]1.0the tagger's words
sensesconditioning - Pavlovian: the animal only observes, and no response is required for the outcome
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
senses[3]1.0the tagger's words
alternativesclaim_type: mechanism
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
a second reading DIFFERING from the recorded value. Recorded: critique — note: a second reading, verbatim-verified, disagreeing with the value on the record. Not applied - a person decides.
alternatives.claim_typethe tagger's words
alternativesmethod: review
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
a second reading reaching the same value. Recorded: review — note: a second reading reaching the same value from a different sentence. Not applied - a person decides.
alternatives.methodthe tagger's words
alternativesphenomenon: selection of behaviour by its consequences within the lifetime of an individual
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
a second reading DIFFERING from the recorded value. Recorded: operant conditioning — note: a second reading, verbatim-verified, disagreeing with the value on the record. Not applied - a person decides.
alternatives.phenomenonthe tagger's words
claim_relationscontradicts → Herrnstein's matching law (Richard Herrnstein, 1961), as a general law relating response rate to reinforcement rate
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
claim_relations[0]the tagger's words
claim_relationsbuilds-on → Thorndike's Law of Effect
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
claim_relations[1]0.9the tagger's words
claim_relationscontradicts → Skinner's framework as a theory in the ordinary sense
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
claim_relations[2]1.0the tagger's words
concept_edgesinstrumental conditioning —same-as→ operant conditioning
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
strength of the relation: 1
concept_edges[0]the tagger's words
concept_edgesontogenetic selection —same-as→ operant conditioning
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
strength of the relation: 1
concept_edges[1]the tagger's words
concept_edgesEVIDENCE CORRECTED (RE-GROUNDED) · operant conditioning —distinct-from→ Pavlovian conditioning
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
EVIDENCE CORRECTED: RE-GROUNDED. The old span states the difference but names the wrong term for one end - that '…in CLASSICAL conditioning no…' response is required, unlike operant conditioning. The edge's endpoint is Pavlovian conditioning, and the span says CLASSICAL, which is the same thing under another name and therefore attests nothing for that endpoint. The replacement names both terms as the edge does AND gives the basis of the distinction: 'BETWEEN PAVLOVIAN…' conditioning, resting on whether the animal only observes the relations between events. Both spans carry a differentiation cue - 'unlike' and 'distinct' - so this is not a trade of strength for form; the new one simply uses the vocabulary the edge does. — strength of the relation: 1 Audit 2026-09-28: DOUBTFUL - read against its full span and kept, but endpoint by synonym: the span writes 'classical conditioning', which the previous sentence introduces as Pavlovian conditioning; the contrast is stated.
concept_edges[2]the tagger's words
concept_edgesreinforcer —contrasted-with→ punisher
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
strength of the relation: 1
also in: reinforcer · punisher
concept_edges[4]the tagger's words
concept_edgesreinforcer —part-of→ contingency
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
strength of the relation: 0.9 Audit 2026-09-28: DOUBTFUL - read against its span and kept, but the span describes reinforcement contingencies; part-of is loose.
concept_edges[5]the tagger's words
concept_edgesoperant conditioning —distinct-from→ Pavlovian conditioning
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
the relation itself is separately evidenced · strength of the relation: 1 · the relation stated: The distinction between Pavlovian and operant conditioning therefore rests on
concept_edges[6]0.9the tagger's words
content_tagsAdding free food pellets that arrive independently of lever pressing makes the rat press less, not more.
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
content_tags.findings[0]the tagger's words
content_tagsMatching can be seen as a result of the negative-feedback properties of the concurrent variable-interval schedule in which it is measured.
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
content_tags.findings[1]the tagger's words
content_tagsMatching depends on the animal's switching history: pigeons trained on each option separately and only then given the choice abandon matching and respond exclusively to the richer schedule.
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
content_tags.findings[2]the tagger's words
content_tagsThat steady-state behaviour is often reversible does not mean the animal's state is equally reversible.
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
content_tags.findings[3]the tagger's words
content_tagsThe field concentrated almost entirely on how reinforcement strengthens behaviour and neglected where the reinforced behaviour comes from in the first place — the problem of behavioural variation.
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
content_tags.findings[4]the tagger's words
content_tagsResearch of the last two decades has convincingly shown that both stimulus-response and action-outcome control structures exist.
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
content_tags.findings[5]the tagger's words
content_tagsIn autoshaping, a pigeon is likely to keep pecking the key even when pecking removes the food.
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
content_tags.findings[6]the tagger's words
content_tagsMuch evidence suggests dopamine provides basal ganglia targets with a reward prediction error that can influence learning and action selection.
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
content_tags.findings[7]the tagger's words
content_tagsBecause simple operant behaviour occurs even in bacteria, the essentials of operant conditioning need not depend on specific neural structures.
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
content_tags.findings[8]the tagger's words
content_tagsBehaviour guided by consequences is universal across animals, from protists to humans.
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
content_tags.findings[9]1.0the tagger's words
content_tagsAdding reward that is NOT contingent on the response reduces responding - the animal tracks predictiveness, not frequency of pairing.
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
content_tags.findings[10]1.0the tagger's words
content_tagsThe field's own gap, stated plainly: research has studied how responses are strengthened and almost never where they come from.
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
content_tags.findings[11]1.0the tagger's words
content_tagsEarly neo-behaviourists were led to an exponential learning curve and a learning principle of the form V(t+1) = A(1-V(t)).
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
content_tags.findings[12]1.0the tagger's words
content_tagsassignment of credit
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
content_tags.definitions[3]the tagger's words
content_tagsYerkes-Dodson law
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
content_tags.definitions[4]the tagger's words
content_tagsinterval schedule
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
content_tags.definitions[5]the tagger's words
content_tagsratio schedule
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
content_tags.definitions[6]the tagger's words
content_tagsmatching law
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
also in: matching law
content_tags.definitions[7]the tagger's words
content_tagshabitual behavior
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
content_tags.definitions[9]the tagger's words
content_tagsoperant conditioning
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
content_tags.definitions[10]1.0the tagger's words
content_tagsontogenetic selection
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
content_tags.definitions[12]1.0the tagger's words
content_tagsreinforcer
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
also in: reinforcer
content_tags.definitions[14]1.0the tagger's words
content_tagspunisher
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
also in: punisher
content_tags.definitions[15]1.0the tagger's words
content_tagsresponse rate
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
also in: response rate
content_tags.variables[0]the tagger's words
content_tagsreinforcement delay, rate and contingency
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
content_tags.variables[1]1.0the tagger's words
content_tagsresponse strength
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
content_tags.variables[2]1.0the tagger's words
content_tagstwice — How much faster rats press the lever on the richer RI15 schedule than on the leaner RI30 schedule, illustrating matching
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
content_tags.quantities[0]the tagger's words
content_tagsV(t+1) = A(1-V(t)) — the neo-behaviourist exponential learning rule, with V response strength and A a learning rate below one
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
content_tags.quantities[1]1.0the tagger's words

Interpretive choices

Tags that could not be placed in the text:

What was deliberately not tagged, and why, is recorded per paper by the tagger. Where a paper has no such record, this page says so rather than implying that nothing was declined.

© 2026 Shir Sivroni · ORCID 0009-0000-2597-2824