A theory of learning from different domains

Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, Jennifer Wortman Vaughan · Machine Learning · 2009 · machine-learning · 10.1007/s10994-009-5152-4 · record ben-david2009-a-theory-of-learning-from-differen
tag layer · index · concepts · authors · עברית
Where this record's text came from. 2026-08-21 schema-0.3 pass: every quote verified as an exact substring of the staged text and anchored at merge time; concept terms verified present; every edge endpoint an existing node.
Some quotes on this page contain (cid:NN). The PDF this text was extracted from encodes 1 character in a way that names no letter, so the extractor could not say what they were. The quotes keep them because a quote is reproduced exactly or it proves nothing — the marks are the source’s, not the tagging’s.
How to read this page. The article itself is not reproduced here — this paper is not cleared for redistribution, so what follows is the tag layer alone. The evidence sentences are withheld with the article. The table at the foot lists every tag the record carries; each row names the facet it belongs to, and the whose words column says whether the tag records the paper's own wording or the tagger's sentence about the paper. Every tag links to each other paper in the corpus carrying it. The machine-readable edition of the same thing is the TEI file beside this one, where the tags live in standOff, and the article is at its publisher via the link below. When we have changed our mind. Some entries in the table begin with a word in capitals and the reason for it. EVIDENCE CORRECTED — the finding still stands, but we had quoted the wrong sentence from the paper, and the quotation has been replaced with the right one. RETRACTED — the entry itself was wrong; it stays on the page, marked, with the reason, rather than being deleted. WITHDRAWN AND REPLACED — the entry was wrong, and a corrected version of it appears in the entries below. Each mark carries its own reason, in brackets, because the reasons differ from one entry to the next. An entry we have WITHDRAWN is removed from this page and kept in the record, with its reason. A line reading Audit and a date is a later re-reading of the entry against its whole sentence: DOUBTFUL means it was kept, and the reason says what is loose about it. Quotation marks inside the tagger's sentence. Where a tag is the tagger's own sentence, a phrase in quotation marks inside it is borrowed from the paper — the tagger's wording around it is not. The paper's own sentence is the evidence quote for that row; on this page it is withheld with the article, and the article is at its publisher. Nothing inside quotation marks anywhere on this page is ours: if it is quoted, it is the paper's. Whose words a concept carries. A concept entry records a term the paper itself uses, so the term, the sense given for it, and its evidence are the paper's own words, quoted where the paper fixes them. This is the opposite of a sense entry in the tag table, which is the tagger's sentence about how the paper uses a term. Where a concept entry could not be grounded in the paper's words, the entry says so on its own row.
Other fields define these differently the same word, defined another way in another discipline
adaptability 2 fields

Tags on this paper

Every tag the record carries, each linking to every other paper in the corpus that carries it. A tag appears here whether or not its words occur in the article — the spans below are only the ones a sentence in this paper licenses. Of the 10 concept tags on this paper, all 10 are grounded in a quoted sentence. All carry the tagger's gloss of what the term means in this paper, which is not something the article can license.
H-divergence L1 divergence ideal joint hypothesis adaptability labeling function distribution divergence VC dimension source domain target domain hypothesis classes a distance defined by the classifier you were going to use measure the difference by trying to detect it the third term is the assumption the half of the difference that is assumed away an intractable quantity approximated by a surrogate mathematical-proof
Method, as described: a VC-style uniform convergence argument, with the domain distance defined as a supremum over the hypothesis class and computed by training a classifier to separate unlabeled source from unlabeled target instances

Legend

relation to other workclaim_relationshow this paper stands to another: agrees, contradicts, extends, qualifies
claim typeclaim_typethe shape of the claim: existence, mechanism, method, critique, prediction, null
concept edgeconcept_edgesan asserted relation between two concepts, with the sentence asserting it
conceptconceptsan idea the paper works with, quoted where the paper uses it
content tagcontent_tagsa finding, definition, quantity, statistic or variable named in the paper
contributioncontributionwhat the paper claims to add
controlcontrolsan experiment run to eliminate a rival explanation
limitationlimitationswhat the paper says its evidence cannot support
mechanismmechanismshow the reported effect is said to come about
methodmethodhow the paper produced its evidence
object of studyobject_of_studywhat the paper is about — the thing it investigates
phenomenonphenomenonthe effect or regularity under study
research questionresearch_questionthe question the paper sets itself
sensesensesa distinct use of a term, quoted where the paper fixes it
transfertransfersa method or concept carried in from another field

The paper

The full text is not reproduced here. This article is not cleared for redistribution, so the page carries the tag layer only. Its evidence sentences are withheld; the grounding status remains with each tag, and the article itself is at https://doi.org/10.1007/s10994-009-5152-4. The reading view exists for the papers whose licence permits it.

Every tag on this page

claim_relations 4 · claim_type 1 · concept_edges 2 · concepts 10 · content_tags 12 · contribution 1 · controls 5 · limitations 6 · mechanisms 5 · method 1 · object_of_study 1 · phenomenon 1 · research_question 1 · senses 6 · transfers 2

Relations on this page carry a quotation for the claim. Where a relation was separately evidenced — and so tested against the vocabulary for its type — the row says so; 2 of 2 relations here carry that mark. The rest are the tagger’s reading of the sentence quoted beside them.

facettagfieldconf.whose words
object_of_studywhen a classifier trained on one distribution can be expected to work on another, and how to weight source against target data when a little labelled target data exists
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
object_of_studythe paper's words
research_questionwhat quantity, estimable from unlabeled samples, bounds the target error of a source-trained classifier
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
research_questionthe paper's words
contributiona classifier-induced divergence that can be estimated from finite unlabeled samples, and a bound in which the target error is the source error plus that divergence plus the error of the ideal joint hypothesis
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
contributionthe paper's words
claim_typemechanism
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
claim_typethe paper's words
phenomenondomain adaptation - training where the labels are and deploying where they are not, with a theory whose usable half is the half that can be computed without labels
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
phenomenonthe paper's words
methodmathematical-proof
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
methodthe paper's words
mechanismsthe L1 divergence takes a supremum over all measurable subsets and is rejected as unestimable and over-strict; the H-divergence takes it over the hypothesis class. So the distance between two FIXED distributions is a function of H - a richer class sees the domains as further apart, and the bound loosens because the model got more expressive rather than because the data moved
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
mechanisms[0]0.95the tagger's words
mechanismslabel every unlabeled source instance 0 and every unlabeled target instance 1, train a classifier to tell them apart, and read the divergence off its error. If the domains are indistinguishable to H the divergence is zero. The measurement IS the adversary, which is why the whole domain-adversarial literature follows from this paragraph
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
mechanisms[1]0.95the tagger's words
mechanismstarget error is bounded by source error plus divergence plus lambda. The first two come from unlabeled data. Lambda is the error of the best joint hypothesis and needs labelled target data to evaluate - the thing the setting is defined by not having. THE BOUND IS USABLE EXACTLY WHEN ITS UNCHECKABLE TERM IS SMALL
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
mechanisms[2]0.95the tagger's words
mechanismstwo domains can differ in their labeling functions as well as in their input distributions. One clause - `since we expect the labeling function difference to be small in practice` - removes the first from the theory. Everything downstream measures distributional shift and inherits the assumption without restating it
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
mechanisms[3]0.95the tagger's words
mechanismscomputing the H-divergence needs the minimum-error hypothesis for the source-versus-target problem, which is intractable for most classes. What is used instead is the error of a hypothesis minimising a convex upper bound - so the reported divergence is an approximation of an approximation, and the paper says so
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
mechanisms[4]0.95the tagger's words
limitationsthe bound is only useful when the ideal joint hypothesis has small error, and that error cannot be estimated without labelled target data
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
limitations[0]0.95the tagger's words
limitationsthe difference between the two domains' labeling functions is assumed small rather than measured
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
limitations[1]0.95the tagger's words
limitationsminimising error to compute the H-divergence is computationally intractable for most hypothesis classes, so a convex surrogate is used
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
limitations[2]0.95the tagger's words
limitationscommon divergences such as L1 and Kullback-Leibler cannot be accurately estimated from finite samples in a distribution-free setting
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
limitations[3]0.95the tagger's words
limitationswith enough labelled target data the source data is never preferred, no matter how much of it there is - the method's value is bounded by target-label scarcity
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
limitations[4]0.95the tagger's words
limitationsthe practical recommendation holds only given a reasonable measure of complexity
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
limitations[5]0.95the tagger's words
transfersa divergence built for change detection in data streams, taken into statistical learning theory as the domain-distance term of a generalisation bound
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
transfers[0]0.95the tagger's words
transfersbinary classification itself used as a measuring instrument - a discriminator trained to separate two unlabeled samples, with its error read as a distance
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
transfers[1]0.95the tagger's words
controlsthe H-divergence checked against the L1 divergence on both counts that matter - estimability from finite samples and tightness of the resulting bound
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
eliminates: a new measure adopted for convenience - it is shown never to be larger than the one it replaces, so the bound cannot have been loosened to make it work · outcome: held
controls[0]0.95the tagger's words
controlsthe bound's behaviour examined at both extremes of lambda rather than only where it is convenient
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
eliminates: a bound that is quietly vacuous when adaptation is impossible - the paper states plainly that a large lambda means no classifier serves both domains · outcome: held
controls[1]0.95the tagger's words
controlsthe intractability of computing the H-divergence stated, with the convex surrogate named as an approximation rather than presented as the quantity
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
eliminates: a reported divergence being read as the defined one - the gap between them is declared at the point the substitution is made · outcome: held
controls[2]0.95the tagger's words
controlssource domains at varying distance compared against target-only training as labelled target data increases
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
eliminates: the claim that source data always helps - with enough target labels it never does, whatever its volume · outcome: failed
controls[3]0.95the tagger's words
controlsthe relation to Crammer et al. (2008) worked out, with the conditions under which the two measures coincide
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
eliminates: that the two divergence measures differ in the cases that matter - the conditions under which they coincide are what would settle it · outcome (stated by the tagger, no quotation): absent-stated-about-other
controls[4]0.95the tagger's words
conceptsH-divergence
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
also in: H-divergence
concepts[0]0.95the paper's words
conceptsL1 divergence
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
also in: L1 divergence
concepts[1]0.95the paper's words
conceptsideal joint hypothesis
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
concepts[2]0.95the paper's words
conceptsadaptability
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
also in: adaptability
concepts[3]0.95the paper's words
conceptslabeling function
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
concepts[4]0.95the paper's words
conceptsdistribution divergence
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
concepts[5]0.95the paper's words
conceptsVC dimension
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
also in: VC dimension
concepts[6]0.95the paper's words
conceptssource domain
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
also in: source domain
concepts[7]0.95the paper's words
conceptstarget domain
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
also in: target domain
concepts[8]0.95the paper's words
conceptshypothesis classes
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
concepts[9]0.95the paper's words
sensesdivergence between domains - AS L1: a supremum over all measurable subsets, which cannot be accurately estimated from finite samples and unnecessarily inflates the bound
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
senses[0]0.95the tagger's words
sensesdivergence between domains - AS H-DIVERGENCE: a supremum over the hypothesis class only, estimable from finite unlabeled samples and never larger than L1
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
senses[1]0.95the tagger's words
sensesthe domain difference - THE PART MEASURED: the divergence between the two marginal distributions
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
senses[2]0.95the tagger's words
sensesthe domain difference - THE PART ASSUMED AWAY: the difference between the two labeling functions, expected to be small in practice and not measured
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
senses[3]0.95the tagger's words
senseslambda - AS A TERM IN THE BOUND: the combined error of the hypothesis that minimises source plus target error
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
senses[4]0.95the tagger's words
senseslambda - AS AN ASSUMPTION: adaptability itself. If it is large there is no classifier good on both domains and no hope from source training; if small, the H-divergence carries the bound - and which of the two holds cannot be checked without labelled target data
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
senses[5]0.95the tagger's words
claim_relationscritiques → the use of L1 and Kullback-Leibler divergence in domain adaptation, on the grounds that neither can be accurately estimated from finite samples in a distribution-free setting
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
claim_relations[0]0.95the tagger's words
claim_relationsextends → Kifer et al. 2004's divergence for change detection, carried into a learning bound
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
claim_relations[1]0.95the tagger's words
claim_relationsdistinguishes-from → Crammer et al. 2008, whose bound is less general and does not handle positive H-delta-H divergence or non-uniform weighting
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
claim_relations[2]0.95the tagger's words
claim_relationsqualifies → its own bound, by stating that it is informative only when the ideal joint hypothesis has small error - a quantity the setting cannot supply
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
claim_relations[3]0.95the tagger's words
concept_edgesH-divergence —distinct-from→ L1 divergence
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
differs in: what set the supremum runs over, and therefore whether the distance can be estimated · the relation itself is separately evidenced · strength of the relation: 0.95 · differs in (stated by the tagger, no quotation): what set the supremum runs over, and therefore whether the distance can be estimated · the relation stated: the H-divergence is estimable from finite samples for classes of finite VC dimension and is never larger than the L1 divergence, because its supremum runs over the hypothesis class rather than over all measurable subsets
concept_edges[0]0.95the tagger's words
concept_edgeslabeling function —distinct-from→ distribution divergence
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
differs in: which of the two ways a domain can shift the theory actually measures · the relation itself is separately evidenced · strength of the relation: 0.95 · differs in (stated by the tagger, no quotation): which of the two ways a domain can shift the theory actually measures · the relation stated: two domains can differ in their labeling functions or in their distributions; the labeling difference is expected to be small in practice and the paper measures the other one Audit 2026-09-28: DOUBTFUL - read against its full span and kept, but the span sets the labeling-function difference aside to focus on distribution divergence; distinct-from is loose.
concept_edges[2]0.95the tagger's words
content_tagsA classifier-induced divergence estimable from unlabeled samples, with the empirical source error, characterizes a source-trained classifier's target error, assuming some hypothesis performs well in both domains.
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
content_tags.findings[0]0.95the tagger's words
content_tagswhen the ideal joint hypothesis has large error, no classifier performs well on both domains and source training cannot help
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
content_tags.findings[1]0.95the tagger's words
content_tagsthe H-divergence is computed by training a classifier to distinguish source from target instances and reading it off the error
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
content_tags.findings[2]0.95the tagger's words
content_tagsthe H-divergence is never larger than the L1 divergence and is generally smaller when H has finite VC dimension
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
content_tags.findings[3]0.95the tagger's words
content_tagsthe difference between the two domains' labeling functions is assumed small rather than measured
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
content_tags.findings[4]0.95the tagger's words
content_tagscomputing the H-divergence exactly is intractable for most hypothesis classes; convex surrogates are used to approximate it
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
content_tags.findings[5]0.95the tagger's words
content_tagswith enough labelled target data, target-only training is always preferred no matter how much source data is available
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
content_tags.findings[6]0.95the tagger's words
content_tagsa divergence whose supremum runs over the hypothesis class rather than over all measurable subsets - estimable from finite unlabeled samples when H has finite VC dimension, and never larger than L1
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
also in: H-divergence
content_tags.definitions[0]0.95the tagger's words
content_tagswhat the ideal joint hypothesis embodies - whether any single classifier can perform well on both domains at once
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
also in: adaptability
content_tags.definitions[1]0.95the tagger's words
content_tagsVC dimension
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
also in: VC dimension
content_tags.statistics[0]0.95the tagger's words
content_tagsdistribution divergence
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
content_tags.variables[0]0.95the tagger's words
content_tagslabeling function
Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.
content_tags.variables[1]0.95the tagger's words

Interpretive choices

Tags that could not be placed in the text:

What was deliberately not tagged, and why, is recorded per paper by the tagger. Where a paper has no such record, this page says so rather than implying that nothing was declined.

© 2026 Shir Sivroni · ORCID 0009-0000-2597-2824