End-to-End Training of Deep Visuomotor Policies

Sergey Levine, Chelsea Finn, Trevor Darrell, Pieter Abbeel · JMLR / arXiv · 2016 · robotics · record levine2016-visuomotor
tag layer · index · concepts · authors · עברית

Where this record's text came from. faceted 2026-08 by reading the locally held text (pdftext); every evidence string verified as an exact substring of that text before merging. Long JMLR-format article whose body is heavily interleaved with LaTeX-to-text artefacts ((cid:xx) tokens, figure-caption fragments, page headers), so several otherwise apt sentences (e.g. the comparison with Levine and Koltun 2014, broken by a page header) were unusable as verbatim evidence and cleaner sentences were chosen instead. method is recorded as system-or-tool-building because the paper's empirical work is real-robot (PR2) task learning plus MuJoCo simulated baselines rather than any of the listed study designs; the simulated comparisons alone would map to 'simulation'. The Mordatch and Todorov relation is labelled 'extends' because the paper positions its own formulation as covering the unknown-dynamics, stochastic-policy case that the cited work does not; the available relation vocabulary has no 'differentiates-from' option. 2026-08-14, 2026-08-15 and 2026-08-25 schema-0.3 passes: every quote verified as an exact substring of the staged text and anchored at merge time; concept terms verified present; every edge endpoint an existing node.

Some quotes on this page contain (cid:NN). The PDF this text was extracted from encodes 3 characters in a way that names no letter, so the extractor could not say what they were. The quotes keep them because a quote is reproduced exactly or it proves nothing — the marks are the source’s, not the tagging’s.

How to read this page. The article itself is not reproduced here — this paper is not cleared for redistribution, so what follows is the tag layer alone. The evidence sentences are withheld with the article. The table at the foot lists every tag the record carries; each row names the facet it belongs to, and the whose words column says whether the tag records the paper's own wording or the tagger's sentence about the paper. Every tag links to each other paper in the corpus carrying it. The machine-readable edition of the same thing is the TEI file beside this one, where the tags live in standOff, and the article is at its publisher via the link below. When we have changed our mind. Some entries in the table begin with a word in capitals and the reason for it. EVIDENCE CORRECTED — the finding still stands, but we had quoted the wrong sentence from the paper, and the quotation has been replaced with the right one. RETRACTED — the entry itself was wrong; it stays on the page, marked, with the reason, rather than being deleted. WITHDRAWN AND REPLACED — the entry was wrong, and a corrected version of it appears in the entries below. Each mark carries its own reason, in brackets, because the reasons differ from one entry to the next. An entry we have WITHDRAWN is removed from this page and kept in the record, with its reason. A line reading Audit and a date is a later re-reading of the entry against its whole sentence: DOUBTFUL means it was kept, and the reason says what is loose about it. Quotation marks inside the tagger's sentence. Where a tag is the tagger's own sentence, a phrase in quotation marks inside it is borrowed from the paper — the tagger's wording around it is not. The paper's own sentence is the evidence quote for that row; on this page it is withheld with the article, and the article is at its publisher. Nothing inside quotation marks anywhere on this page is ours: if it is quoted, it is the paper's. Whose words a concept carries. A concept entry records a term the paper itself uses, so the term, the sense given for it, and its evidence are the paper's own words, quoted where the paper fixes them. This is the opposite of a sense entry in the tag table, which is the tagger's sentence about how the paper uses a term. Where a concept entry could not be grounded in the paper's words, the entry says so on its own row.

Other fields define these differently the same word, defined another way in another discipline
deep learning 11 fields · convolutional neural networks 3 fields · perception 3 fields · standard-deviation 3 fields · supervised learning 3 fields · linear regression 2 fields

Tags on this paper

Every tag the record carries, each linking to every other paper in the corpus that carries it. A tag appears here whether or not its words occur in the article — the spans below are only the ones a sentence in this paper licenses. Of the 7 concept tags on this paper, 4 are grounded in a quoted sentence.

deep learning reinforcement learning robot control visuomotor policy guided policy search pose estimation supervised learning guided policy search: convert reinforcement-learning policy search into supervised learning of a deep network, with a trajectory-optimizer acting as the teacher visuomotor policies policy search convolutional neural networks end-to-end sensorimotor torques perception Bregman ADMM (BADMM) raw image observations motor torques cost function full state mean mean of regression linear regression standard-deviation standard deviation system-or-tool-building

Legend

relation to other work	`claim_relations`	how this paper stands to another: agrees, contradicts, extends, qualifies
claim type	`claim_type`	the shape of the claim: existence, mechanism, method, critique, prediction, null
concept edge	`concept_edges`	an asserted relation between two concepts, with the sentence asserting it
concept	`concepts`	an idea the paper works with, quoted where the paper uses it
content tag	`content_tags`	a finding, definition, quantity, statistic or variable named in the paper
control	`controls`	an experiment run to eliminate a rival explanation
limitation	`limitations`	what the paper says its evidence cannot support
mechanism	`mechanisms`	how the reported effect is said to come about
method	`method`	how the paper produced its evidence
phenomenon	`phenomenon`	the effect or regularity under study
sense	`senses`	a distinct use of a term, quoted where the paper fixes it
transfer	`transfers`	a method or concept carried in from another field

The paper

The full text is not reproduced here. This article is not cleared for redistribution, so the page carries the tag layer only. Its evidence sentences are withheld; the grounding status remains with each tag, and the article itself is at https://arxiv.org/abs/1504.00702. The reading view exists for the papers whose licence permits it.

Every tag on this page

claim_relations 5 · claim_type 1 · concept_edges 1 · concepts 4 · content_tags 18 · controls 1 · limitations 8 · mechanisms 1 · method 1 · phenomenon 1 · senses 3 · transfers 1

Relations on this page carry a quotation for the claim. Where a relation was separately evidenced — and so tested against the vocabulary for its type — the row says so; none of 1 relation here carry that mark. The rest are the tagger’s reading of the sentence quoted beside them.

facet	tag	field	conf.	whose words
claim_type	method Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`claim_type`		the paper's words
phenomenon	visuomotor policy learning Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`phenomenon`		the paper's words
method	system-or-tool-building Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: system-or-tool-building	`method`		the paper's words
mechanisms	mechanisms Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: guided policy search: convert reinforcement-learning policy search into supervised learning of a deep network, with a trajectory-optimizer acting as the teacher	`mechanisms[0]`		the tagger's words
limitations	training requires positioning objects at consistent positions Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: the training procedure as described: it requires Quotation withheld	`limitations[0]`		the tagger's words
limitations	The method does not generalise to dramatically different visual settings, especially under occluding distractors. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: the generalisation claim: OUR CURRENT METHOD DOES NOT GENERALIZE TO DRAMATICALLY DIFFERENT SETTINGS, ESPECIALLY WHEN VISUAL DISTRACTO[rs occlude]	`limitations[1]`		the tagger's words
limitations	The approach requires the full system state to be observable during training, which restricts the range of applicable tasks. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: the range of tasks the method can address: THE REQUIREMENT TO OBSERVE THE FULL STATE DURING TRAINING LIMITS THE TASKS TO WHICH THE METHOD CAN BE APPLIED	`limitations[2]`		the tagger's words
limitations	Tasks with freely moving objects would need heavier instrumentation such as motion capture. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: any extension to freely moving objects: those TASKS ... REQUIRE MORE EXTENSIVE INSTRUMENTATION, SUCH AS MOTION CAPTURE	`limitations[3]`		the tagger's words
limitations	The spatial feature-point representation makes simplifying assumptions about feature presence and uniqueness. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: every result produced through the visual encoder: THE FEATURE POINT REPRESENTATION ... ASSUMES THAT THE LEARNED FEATURES ARE PRESENT AT ALL TIMES, AND ONLY ONE INSTANCE OF EACH FEATURE IS EVER PRESENT IN THE IMAGE	`limitations[4]`		the tagger's words
limitations	The method needs the full state observable during training, and the authors say exactly when that is minor and when it is not. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: applying this method to a new task: the full state must be observable during training, which the authors call minor when objects can be positioned consistently and not minor for tasks manipulating freely moving objects	`limitations[5]`	1.0	the tagger's words
limitations	The architecture comparison is not run on the target task but on a pretraining task, which the authors describe as a reasonable proxy. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: the architecture comparison in this paper: it is measured on the pose-estimation pretraining task, described by the authors as a reasonable proxy for the visuomotor challenges	`limitations[6]`	1.0	the tagger's words
limitations	The training-time observability the method depends on is bought with a physical constraint on the world, stated plainly: providing the full state 'simply requires positioning objects in one of several known positions for each trial during training'. The assumption is not about the algorithm but about how the laboratory is arranged. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`limitations[7]`	0.95	the tagger's words
transfers	deep convolutional networks trained end-to-end from perception to control for robotic manipulation Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`transfers[0]`		the tagger's words
controls	two baselines that pre-train the vision layers for pose prediction instead of training end-to-end, one with the SAME architecture as the full policy Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. eliminates: that the improvement comes from the network architecture rather than from training the vision and control layers jointly end-to-end · outcome: held	`controls[0]`	0.9	the tagger's words
concepts	visuomotor policy Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: visuomotor policy	`concepts[3]`		the paper's words
concepts	guided policy search Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: guided policy search	`concepts[4]`		the paper's words
concepts	pose estimation Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: pose estimation	`concepts[5]`		the paper's words
concepts	supervised learning Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: supervised learning	`concepts[6]`	0.95	the paper's words
senses	architecture quality - MEASURED ON A PROXY: accuracy on the pose-estimation pretraining task, chosen to isolate the architectures from other confounding factors Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`senses[0]`	1.0	the tagger's words
senses	failure at insertion - defined by a threshold taken from the object itself: since the peg is 0.5 units long, a larger distance means the controller cannot insert Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`senses[1]`	1.0	the tagger's words
senses	observable - split by phase, and the split is the method's central assumption: the full system state is observable AT TRAINING TIME BUT NOT AT TEST TIME, which is what lets a trajectory-centric RL algorithm supply supervision the deployed policy never gets Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`senses[2]`	0.95	the tagger's words
claim_relations	builds-on → convolutional neural networks (deep learning) Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`claim_relations[0]`		the tagger's words
claim_relations	builds-on → reinforcement learning Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`claim_relations[1]`		the tagger's words
claim_relations	builds-on → Bregman ADMM (BADMM) (Wang and Banerjee, 2014) Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`claim_relations[2]`		the tagger's words
claim_relations	extends → Levine and Abbeel, 2014; Levine et al., 2015 (the authors' own conference papers) Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`claim_relations[3]`		the tagger's words
claim_relations	extends → Mordatch and Todorov (2014) Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`claim_relations[4]`		the tagger's words
concept_edges	guided policy search —operationalised-as→ supervised learning Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. strength of the relation: 0.9 · note: THE METHOD'S WHOLE MOVE IS A CONVERSION BETWEEN PROBLEM CLASSES, stated as such: '…search TRANSFORMS policy search INTO A SUPERVISED LEARNING…' The hard problem is not solved; it is turned into an easier one whose labels another algorithm supplies. Audit 2026-09-28: DOUBTFUL - read against its full span and kept, but the span says guided policy search TRANSFORMS policy search into a supervised learning problem; operationalised-as is loose. also in: guided policy search · supervised learning	`concept_edges[0]`	0.95	the tagger's words
content_tags	92,000 — size of the CNN policy network Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`content_tags.quantities[0]`		the tagger's words
content_tags	7 — depth of the policy CNN Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`content_tags.quantities[1]`		the tagger's words
content_tags	tens of minutes — real-world data needed to train a policy Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`content_tags.quantities[2]`		the tagger's words
content_tags	2 — prior papers this article extends Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`content_tags.quantities[3]`		the tagger's words
content_tags	end-to-end training improves consistency and generalization Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. about this quote: QUOTE RECOVERED FROM THE PAPER'S OWN CHARACTERS. The recorded quote was LETTERS-ONLY IDENTICAL to this stretch of the staged text but not an exact substring of it - spaces, hyphens, ligature glyphs or mathematical operators differed, so it could never anchor. This is the file's own text between the same two letters: not produced by a rule about the damage, copied out of the file. Round-trip checked (the recovered span is letters-only equal to the original) and length-checked. The text as originally recorded is kept in evidence as extracted.	`content_tags.findings[0]`		the tagger's words
content_tags	the policy maps raw camera images directly to motor torques Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`content_tags.findings[2]`		the tagger's words
content_tags	full system state is available at training but not at test time Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`content_tags.findings[3]`		the tagger's words
content_tags	raw image observations Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: raw image observations	`content_tags.variables[0]`		the tagger's words
content_tags	motor torques Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: motor torques	`content_tags.variables[1]`		the tagger's words
content_tags	cost function Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: cost function	`content_tags.variables[2]`		the tagger's words
content_tags	full state Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: full state	`content_tags.variables[3]`		the tagger's words
content_tags	guided policy search Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: guided policy search	`content_tags.definitions[0]`		the tagger's words
content_tags	visuomotor policies Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. about this quote: QUOTE RECOVERED FROM THE PAPER'S OWN CHARACTERS. The recorded quote was LETTERS-ONLY IDENTICAL to this stretch of the staged text but not an exact substring of it - spaces, hyphens, ligature glyphs or mathematical operators differed, so it could never anchor. This is the file's own text between the same two letters: not produced by a rule about the damage, copied out of the file. Round-trip checked (the recovered span is letters-only equal to the original) and length-checked. The text as originally recorded is kept in evidence as extracted. also in: visuomotor policies	`content_tags.definitions[1]`		the tagger's words
content_tags	policy search Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: policy search	`content_tags.definitions[2]`		the tagger's words
content_tags	Bregman ADMM (BADMM) Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: Bregman ADMM (BADMM)	`content_tags.definitions[3]`		the tagger's words
content_tags	mean Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. about this quote: DAMAGED GLYPHS IN THIS QUOTE: (cid:107). These are raw PDF font glyph indices that leaked out in place of the characters they stand for - usually a mathematical operator. The quote is verbatim to the staged text and its anchor is sound; the STAGED TEXT is what is damaged. NOT repaired: the cid-to-character mapping is defined by the font embedded in that particular PDF, so guessing it would produce a clean and unverifiable quote. Repair needs the source PDF and is a re-staging job. also in: mean	`content_tags.statistics[1]`		the tagger's words
content_tags	regression Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: regression	`content_tags.statistics[2]`		the tagger's words
content_tags	standard-deviation Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: standard-deviation	`content_tags.statistics[3]`		the tagger's words

Interpretive choices

Tags that could not be placed in the text:

none — every tag was located in the text

What was deliberately not tagged, and why, is recorded per paper by the tagger. Where a paper has no such record, this page says so rather than implying that nothing was declined.