A survey of transfer learning

A survey of transfer learning Karl R. Weiss Taghi M. Khoshgoftaar Dingding Wang

Tag layer of the DHSS Meta-Tagging Project. The article text is reproduced from the staged copy named in sourceDesc and is not altered; the tags are held separately in standOff so that overlapping and nested spans are preserved.

A survey of transfer learning 2016 Journal Of Big Data 10.1186/s40537-016-0043-6 weiss2016-a-survey-of-transfer-learning weiss2016-a-survey-of-transfer-learning.txt, the project's staged copy d694ed6df017b78b36fa823211262f024c135317 cc-by machine-learning

the paper's own words

the tag layer; a sentence written by the tagger about the paper, always anchored to a quotation but not itself a quotation

0 spans located by searching the stored text for the quote's exact words, allowing any whitespace between them the transfer learning literature - solutions, applications, and the terminology used to describe them Quotation withheld because this record is not cleared for redistribution. what to do when the assumption that training and test data share a domain and a distribution does not hold Quotation withheld because this record is not cleared for redistribution. a formal definition of transfer learning and an explicit account of where the literature's own terms conflict Quotation withheld because this record is not cleared for redistribution. critique Quotation withheld because this record is not cleared for redistribution. improving a target learner using information from a different but related source domain Quotation withheld because this record is not cleared for redistribution. review review Quotation withheld because this record is not cleared for redistribution. mechanisms WHY TRANSFER LEARNING EXISTS AT ALL: ordinary machine learning assumes train and test come from one domain with one distribution, and the method is what you do when that assumption is false and the target data is expensive to collect Quotation withheld because this record is not cleared for redistribution. mechanisms NEGATIVE TRANSFER IS DEFINED BY COMPARISON, NOT BY DAMAGE: a learner trained on the target alone is compared against one trained with the source added, and transfer is negative when the plain learner wins Quotation withheld because this record is not cleared for redistribution. mechanisms PSEUDO LABELS CORRECT A DISTRIBUTION USING GUESSES AT THE THING BEING CORRECTED FOR: with no labeled target data, labels are estimated and then used to fix the conditional distribution - which the paper itself calls problematic Quotation withheld because this record is not cleared for redistribution. instance transfer reweights source examples to correct MARGINAL distribution differences, and works best precisely when the CONDITIONAL distribution is already the same Quotation withheld because this record is not cleared for redistribution. the two feature approaches differ in direction: one bends the source onto the target, the other maps both into a shared latent space Quotation withheld because this record is not cleared for redistribution. parameter transfer shares model parameters or combines multiple reweighted source learners into an ensemble for the target Quotation withheld because this record is not cleared for redistribution. source classifiers are weighted by how close their conditional distribution is to the target, summed, and used to manufacture pseudo labels for unlabelled target data Quotation withheld because this record is not cleared for redistribution. THE FIELD IS DECLARED TERMINOLOGICALLY INCONSISTENT AND THE PAPER SAYS SO IN ITS OWN VOICE: `Quotation withheld` A survey opening by stipulating the vocabulary because the literature has none. Quotation withheld because this record is not cleared for redistribution. THE SAME CONFIGURATION, RENAMED IN THE NEXT SENTENCE: `Quotation withheld.` The sentence immediately before it - quoted separately on this record - gives that exact configuration to Daume and Chattopadhyay as SUPERVISED. The full four-scheme enumeration spans several sentences with a journal running header splicing one of them, so it is carried across four tags on this record rather than asserted from one span. Quotation withheld because this record is not cleared for redistribution. AND THE PAPER'S REPAIR IS TO REFUSE THE VOCABULARY ALTOGETHER: `Quotation withheld.` It does not pick a winner and does not coin a fifth term - it describes the configuration and drops the names. THE SAME MOVE CONSORT MADE IN 2010, replacing `intention to treat` with `a more explicit request for information` (schulz2010). Quotation withheld because this record is not cleared for redistribution. THE LITERATURE'S VOCABULARY IS INCONSISTENT AND THE PAPER SAYS SO: transfer learning and domain adaptation are used for similar processes, and the supervised / semi-supervised / unsupervised labels mean different things to different authors Quotation withheld because this record is not cleared for redistribution. Some surveyed solutions are tied to their application and are stated not to generalise to others Quotation withheld because this record is not cleared for redistribution. Correcting the conditional distribution rests on pseudo labels, which the paper names as problematic because the correction uses estimates of the very labels it lacks Quotation withheld because this record is not cleared for redistribution. No single repository exists for the surveyed solutions, so the software behind the literature is scattered - the paper asks for one Quotation withheld because this record is not cleared for redistribution. heterogeneous solutions mostly assume the domain distributions are the SAME, and need further adaptation steps when they are not Quotation withheld because this record is not cleared for redistribution. transferring on a defined relationship between domains is named as the least used of the four categories Quotation withheld because this record is not cleared for redistribution. transfer learning transfer learning Quotation withheld because this record is not cleared for redistribution. domain adaptation domain adaptation Quotation withheld because this record is not cleared for redistribution. negative transfer negative transfer Quotation withheld because this record is not cleared for redistribution. positive transfer positive transfer Quotation withheld because this record is not cleared for redistribution. feature space feature space Quotation withheld because this record is not cleared for redistribution. domain D domain D Quotation withheld because this record is not cleared for redistribution. task T task T Quotation withheld because this record is not cleared for redistribution. label space label space Quotation withheld because this record is not cleared for redistribution. predictive function predictive function Quotation withheld because this record is not cleared for redistribution. supervised transfer learning supervised transfer learning Quotation withheld because this record is not cleared for redistribution. semi-supervised transfer learning semi-supervised transfer learning Quotation withheld because this record is not cleared for redistribution. inductive transfer learning inductive transfer learning Quotation withheld because this record is not cleared for redistribution. transductive transfer learning transductive transfer learning Quotation withheld because this record is not cleared for redistribution. unsupervised transfer learning unsupervised transfer learning Quotation withheld because this record is not cleared for redistribution. terminology inconsistencies terminology inconsistencies Quotation withheld because this record is not cleared for redistribution. pseudo labels pseudo labels Quotation withheld because this record is not cleared for redistribution. marginal distribution marginal distribution Quotation withheld because this record is not cleared for redistribution. conditional distribution conditional distribution Quotation withheld because this record is not cleared for redistribution. source domain source domain Quotation withheld because this record is not cleared for redistribution. target domain target domain Quotation withheld because this record is not cleared for redistribution. context feature bias context feature bias Quotation withheld because this record is not cleared for redistribution. Transfer Categories Transfer Categories Quotation withheld because this record is not cleared for redistribution. asymmetric feature transformation asymmetric feature transformation Quotation withheld because this record is not cleared for redistribution. symmetric feature transformation symmetric feature transformation Quotation withheld because this record is not cleared for redistribution. feature augmentation method feature augmentation method Quotation withheld because this record is not cleared for redistribution. informed supervised transfer learning informed supervised transfer learning Quotation withheld because this record is not cleared for redistribution. transductive transfer learning transductive transfer learning Quotation withheld because this record is not cleared for redistribution. supervised transfer learning - DAUME AND CHATTOPADHYAY'S SENSE: abundant labeled source data and LIMITED labeled target data Quotation withheld because this record is not cleared for redistribution. semi-supervised transfer learning - DAUME AND CHATTOPADHYAY'S SENSE: abundant labeled source and NO labeled target. Gong and Blitzer use this same phrase for what Daume calls SUPERVISED - the two labels are swapped between literatures Quotation withheld because this record is not cleared for redistribution. unsupervised transfer learning - PAN'S SENSE: NO labeled source and NO labeled target, where the same phrase elsewhere in the literature means abundant labeled source and no labeled target. THREE GROUPS, ONE VOCABULARY, THREE MAPPINGS Quotation withheld because this record is not cleared for redistribution. transfer learning - the paper's response to its own finding: it refuses the contested labels entirely and states the data conditions each time instead of naming them Quotation withheld because this record is not cleared for redistribution. `supervised transfer learning` - DAUME AND CHATTOPADHYAY'S SENSE: `abundant labeled source data and LIMITED labeled target data`. Gong and Blitzer give that same configuration to SEMI-SUPERVISED, and Cook and Feuz call it INFORMED SUPERVISED. Quotation withheld because this record is not cleared for redistribution. `semi-supervised transfer learning` - GONG AND BLITZER'S SENSE: `abundant labeled source data and LIMITED labeled target data`. THAT IS THE CONFIGURATION Daume and Chattopadhyay CALL SUPERVISED - see senses[4] and the `daume` mechanism on this record, which quote their sentence. One name over two configurations and one configuration under two names, in consecutive sentences. Quotation withheld because this record is not cleared for redistribution. `supervised` / `unsupervised` - COOK AND FEUZ MOVE THE AXIS: the pair refers to `the presence or absence of labeled data in the source domain` and a SECOND pair, `informed` / `uninformed`, carries the target. Two orthogonal axes where the other schemes have one. Quotation withheld because this record is not cleared for redistribution. `domain adaptation` - AS STIPULATED HERE, because the field does not agree: `the process of adapting one or more source domains for the means of transferring information to improve the performance of a target learner`, by bringing the source's distribution closer to the target's. Quotation withheld because this record is not cleared for redistribution. domain adaptation —part-of→ transfer learning domain adaptation transfer learning strength of the relation: 0.9 Quotation withheld because this record is not cleared for redistribution. negative transfer —distinct-from→ positive transfer negative transfer positive transfer strength of the relation: 1 Audit 2026-09-28: DOUBTFUL - read against its full span and kept, but endpoint by anaphora: 'the definition above' = negative transfer (previous sentences); distinct-from is loose. Quotation withheld because this record is not cleared for redistribution. feature space —part-of→ domain D feature space domain D strength of the relation: 1 Quotation withheld because this record is not cleared for redistribution. label space —part-of→ task T label space task T strength of the relation: 1 Quotation withheld because this record is not cleared for redistribution. predictive function —part-of→ task T predictive function task T strength of the relation: 1 Quotation withheld because this record is not cleared for redistribution. in the muscle-fatigue test EVERY transfer learning approach beat the non-transfer baseline, with the two proposed methods ranking first and second Quotation withheld because this record is not cleared for redistribution. an often-cited domain adaptation method is reported as requiring only ten lines of Perl - the simplicity is the claim Quotation withheld because this record is not cleared for redistribution. Homogeneous transfer learning is directly applicable to big data environments. Quotation withheld because this record is not cleared for redistribution. the four transfer categories enumerated by what is carried across transfer categories Quotation withheld because this record is not cleared for redistribution. context feature bias defined by its consequence - it makes the conditional distributions differ context feature bias Quotation withheld because this record is not cleared for redistribution.

The full text of this article is not redistributable, so it is not included here. The evidence sentences are withheld with it; the tag layer and its grounding status remain. The article itself is at https://doi.org/10.1186/s40537-016-0043-6.