A Survey on Deep Learning in Medical Image Analysis

Geert Litjens, Thijs Kooi, Babak Ehteshami Bejnordi, Arnaud Arindra Adiyoso Setio · Medical Image Analysis / arXiv · 2017 · medicine · 10.1016/j.media.2017.07.005 · record litjens2017-medical-imaging-survey
tag layer · index · concepts · authors · עברית

Where this record's text came from. faceted 2026-08 by reading the locally held text (pdftext); every evidence string verified as an exact substring of that text before merging. Extraction is usable prose in the introduction, methods overview and discussion, but the long overview tables (Tables 10-11 and similar) are heavily garbled by column interleaving and some non-ASCII author names are mangled ('C¸ ic¸ek' appears as 'C? ic?ek'); no quotes were taken from tables or from mangled passages. A second candidate claim shape is 'critique', since a large part of the discussion is a critical inventory of unsolved problems ('It is clear that applying deep learning algorithms to medical image analysis presents several unique challenges.'), but the paper's own stated aim is first to show the pervasion of the field, so 'existence' was chosen. Only one explicit relation to named prior work about this paper's own contribution was found; the hundreds of other citations are surveyed rather than positioned against. 2026-08-14 and 2026-08-17 schema-0.3 passes: every quote verified as an exact substring of the staged text and anchored at merge time; concept terms verified present; every edge endpoint an existing node.

Some words in these quotes contain the wrong character. The PDF this text was extracted from substituted 3 characters inside a word — a currency sign or a stray accent where a letter pair belongs, as in signi¢cantly for significantly, or a single ligature glyph, as in conﬁdence. They are valid characters in the wrong place, so they survive every encoding check. The quotes keep them for the same reason as above, and a search of this page will not find the affected words.

2 quotes on this page break a word across a line. The PDF this text was extracted from hyphenated those words at the end of a printed line — adversar- ial, repre- sentations — and the extractor kept the hyphen and the space. The quotes keep them because a quote is reproduced exactly or it proves nothing: the break is the typesetter’s, not the author’s. A search of this page will not find the affected words.

This record’s own method is review, and the tags below follow from that. A review’s findings, quantities and statistics are things stated IN this paper, and some of them report results obtained elsewhere. No tag on this page names a different author. Measured across every review in this corpus, 81% of a review’s tags carry no citation at all — most of them are the review speaking. The corpus does not adjudicate the rest: read an unmarked tag as a claim this paper makes, not necessarily one it made first.

How to read this page. The article itself is not reproduced here — this paper is not cleared for redistribution, so what follows is the tag layer alone. The evidence sentences are withheld with the article. The table at the foot lists every tag the record carries; each row names the facet it belongs to, and the whose words column says whether the tag records the paper's own wording or the tagger's sentence about the paper. Every tag links to each other paper in the corpus carrying it. The machine-readable edition of the same thing is the TEI file beside this one, where the tags live in standOff, and the article is at its publisher via the link below. When we have changed our mind. Some entries in the table begin with a word in capitals and the reason for it. EVIDENCE CORRECTED — the finding still stands, but we had quoted the wrong sentence from the paper, and the quotation has been replaced with the right one. RETRACTED — the entry itself was wrong; it stays on the page, marked, with the reason, rather than being deleted. WITHDRAWN AND REPLACED — the entry was wrong, and a corrected version of it appears in the entries below. Each mark carries its own reason, in brackets, because the reasons differ from one entry to the next. An entry we have WITHDRAWN is removed from this page and kept in the record, with its reason. A line reading Audit and a date is a later re-reading of the entry against its whole sentence: DOUBTFUL means it was kept, and the reason says what is loose about it. Quotation marks inside the tagger's sentence. Where a tag is the tagger's own sentence, a phrase in quotation marks inside it is borrowed from the paper — the tagger's wording around it is not. The paper's own sentence is the evidence quote for that row; on this page it is withheld with the article, and the article is at its publisher. Nothing inside quotation marks anywhere on this page is ours: if it is quoted, it is the paper's. Whose words a concept carries. A concept entry records a term the paper itself uses, so the term, the sense given for it, and its evidence are the paper's own words, quoted where the paper fixes them. This is the opposite of a sense entry in the tag table, which is the tagger's sentence about how the paper uses a term. Where a concept entry could not be grounded in the paper's words, the entry says so on its own row.

Other fields define these differently the same word, defined another way in another discipline
deep learning 11 fields · handcrafted features 2 fields · statistical-significance 2 fields · training data 2 fields · convolutional neural network 2 fields · reference standard 2 fields

Tags on this paper

Every tag the record carries, each linking to every other paper in the corpus that carries it. A tag appears here whether or not its words occur in the article — the spans below are only the ones a sentence in this paper licenses. Of the 10 concept tags on this paper, 5 are grounded in a quoted sentence.

deep learning convolutional neural network medical imaging image segmentation computer-aided diagnosis reference standard label noise selective sampling within-class heterogeneity binary task THE SAME TECHNICAL PROBLEM IS REPORTED TO HAVE DIFFERENT SEVERITY IN TWO FIELDS, AND THE CONTRAST IS DRAWN EXPLICITLY: EVEN WHEN DATA IS ANNOTATED BY DOMAIN EXPERT, LABEL NOISE CAN BE A SIGNIFICANT LIMITING FACTOR - WHEREAS IN COMPUTER VISION THE NOISE IN THE LABELING OF IMAGES IS TYPICALLY RELATIVELY LOW. The word `label` names something with a different reliability depending on who produced it A REFERENCE STANDARD IS BUILT WITH CONSENSUS DELIBERATELY NOT FORCED, AND THE DISAGREEMENT THEN OUTNUMBERS THE AGREEMENT THREE TO ONE: nodules were annotated by four radiologists INDEPENDENTLY, the readers REVIEWED EACH OTHERS ANNOTATIONS BUT NO CONSENSUS WAS FORCED - and the number they DID NOT UNANIMOUSLY AGREE ON to be a nodule WAS THREE TIMES LARGER THAN THE NUMBER THEY DID FULLY AGREE ON THE CONSEQUENCE FOR ANYTHING TRAINED ON IT IS STATED, AND THE FIX IS AN OPEN PROBLEM: training a system on such data REQUIRES CAREFUL CONSIDERATION OF HOW TO DEAL WITH NOISE AND UNCERTAINTY IN THE REFERENCE STANDARD - one COULD THINK OF solutions like incorporating labeling uncertainty directly in the loss function, BUT THIS IS STILL AN OPEN CHALLENGE. The uncertainty is acknowledged and has nowhere to go A BINARY PRESENTATION OF THE TASK IS CALLED A GROSS SIMPLIFICATION, AND THE REASON IS HETEROGENEITY ON BOTH SIDES: classification is often presented as A BINARY TASK - normal versus abnormal, object versus background - HOWEVER THIS IS OFTEN A GROSS SIMPLIFICATION AS BOTH CLASSES CAN BE HIGHLY HETEROGENEOUS. Not one class hiding structure but both THE `NORMAL` CATEGORY IS SHOWN TO CONTAIN THINGS THAT ARE NOT NORMAL, INCLUDING ARTIFACTS OF THE INSTRUMENT: the normal category OFTEN CONSISTS OF COMPLETELY NORMAL TISSUE **BUT ALSO** several categories of BENIGN FINDINGS, WHICH CAN BE RARE, and MAY OCCASIONALLY INCLUDE A WIDE VARIETY OF IMAGING ARTIFACTS. The residual class is defined by what it is not, and collects the measurement's own failures THE PERFORMANCE PATTERN IS PREDICTED FROM THE CATEGORY'S STRUCTURE: systems end up GOOD AT EXCLUDING THE MOST COMMON NORMAL SUBCLASSES BUT FAIL MISERABLY ON SEVERAL RARE ONES. A single accuracy figure over a heterogeneous class hides exactly the failures that matter, and the shape of the class predicts the shape of the failure THE CORRECT FIX IS NAMED AND DECLINED ON COST, IN ONE SENTENCE: a STRAIGHTFORWARD SOLUTION would be to turn the system into a MULTI-CLASS one by providing DETAILED ANNOTATIONS OF ALL POSSIBLE SUBCLASSES - OBVIOUSLY THIS AGAIN COMPOUNDS THE ISSUE OF LIMITED AVAILABILITY OF EXPERT TIME FOR ANNOTATING AND IS THEREFORE OFTEN SIMPLY NOT FEASIBLE. The right answer is known and unaffordable THE WORKAROUNDS ARE REPORTED TO FAIL FOR THE SAME REASON THE PROBLEM EXISTS: strategies that incorporate intelligence in the training process - selective sampling, hard negative mining - TYPICALLY FAIL WHEN THERE IS SUBSTANTIAL NOISE IN THE REFERENCE STANDARD - and ADDITIONAL METHODS FOR DEALING WITH WITHIN-CLASS HETEROGENEITY WOULD BE HIGHLY WELCOME. The noise in the labels defeats the corrections for the labels A STAIN IS OFFERED AS A SUBSTITUTE FOR EXPERTISE: in histopathology one can sometimes use SPECIFIC IMMUNOHISTOCHEMICAL STAINS to highlight regions of interest, REDUCING THE NEED FOR EXPERT EXPERIENCE - and non-expert labels can be leveraged VIA CROWD-SOURCING. Two ways to replace a judgement: a reagent and a crowd A BENCHMARK IS REPORTED TO HAVE SATURATED, SO THAT IMPROVEMENTS CAN NO LONGER BE ATTRIBUTED: since 2014 the performance on the benchmark HAS SATURATED and IT IS DIFFICULT TO ASSESS WHETHER THE SMALL INCREASES IN PERFORMANCE CAN REALLY BE ATTRIBUTED TO 'BETTER' AND MORE SOPHISTICATED ARCHITECTURES. The scare quotes on `better` are the paper's, and the measuring instrument has stopped discriminating A PERFORMANCE CRITERION IS DECLARED INSUFFICIENT AND SUPPLEMENTED BY AN ARTICULATION CRITERION, JUSTIFIED BY LEGAL ACCOUNTABILITY: these methods have often been described as BLACK BOXES, and especially in medicine WHERE ACCOUNTABILITY IS IMPORTANT AND CAN HAVE SERIOUS LEGAL CONSEQUENCES, IT IS OFTEN NOT ENOUGH TO HAVE A GOOD PREDICTION SYSTEM - the system ALSO HAS TO BE ABLE TO ARTICULATE ITSELF in a certain way. What counts as success acquires a second component from outside the field A SURVEY DE-DUPLICATES ITS OWN POPULATION BY A JUDGEMENT IT NEVER DEFINES, AND SUPPLIES A WORKED EXAMPLE INSTEAD OF A CRITERION: when largely overlapping work had been reported in multiple publications, ONLY THE PUBLICATION DEEMED MOST IMPORTANT WAS INCLUDED - A TYPICAL EXAMPLE HERE WAS ARXIV PREPRINTS THAT WERE SUBSEQUENTLY PUBLISHED, or conference contributions later expanded into journal papers. What `most important` means is stated nowhere, the survey's whole 308-paper population rests on it, and the example is doing the work a definition would have done convolutional networks medical image analysis image classification segmentation object detection feature extraction pattern recognition supervised GOFAI handcrafted features feature learning training data decision boundary discriminant features imaging tasks Bayesian regression regression model statistical-significance review

Legend

relation to other work	`claim_relations`	how this paper stands to another: agrees, contradicts, extends, qualifies
claim type	`claim_type`	the shape of the claim: existence, mechanism, method, critique, prediction, null
concept	`concepts`	an idea the paper works with, quoted where the paper uses it
content tag	`content_tags`	a finding, definition, quantity, statistic or variable named in the paper
control	`controls`	an experiment run to eliminate a rival explanation
limitation	`limitations`	what the paper says its evidence cannot support
mechanism	`mechanisms`	how the reported effect is said to come about
method	`method`	how the paper produced its evidence
phenomenon	`phenomenon`	the effect or regularity under study
sense	`senses`	a distinct use of a term, quoted where the paper fixes it
transfer	`transfers`	a method or concept carried in from another field

The paper

The full text is not reproduced here. This article is not cleared for redistribution, so the page carries the tag layer only. Its evidence sentences are withheld; the grounding status remains with each tag, and the article itself is at https://doi.org/10.1016/j.media.2017.07.005. The reading view exists for the papers whose licence permits it.

Every tag on this page

claim_relations 2 · claim_type 1 · concepts 5 · content_tags 17 · controls 1 · limitations 7 · mechanisms 12 · method 1 · phenomenon 1 · senses 3 · transfers 1

This record holds no relations between concepts. A layer is empty when the tagger recorded nothing in it — which is not the same as the paper having nothing to record.

facet	tag	field	conf.	whose words
claim_type	existence Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`claim_type`		the paper's words
phenomenon	deep learning in medical image analysis Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`phenomenon`		the paper's words
method	review Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: review	`method`		the paper's words
mechanisms	mechanisms Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: THE SAME TECHNICAL PROBLEM IS REPORTED TO HAVE DIFFERENT SEVERITY IN TWO FIELDS, AND THE CONTRAST IS DRAWN EXPLICITLY: EVEN WHEN DATA IS ANNOTATED BY DOMAIN EXPERT, LABEL NOISE CAN BE A SIGNIFICANT LIMITING FACTOR - WHEREAS IN COMPUTER VISION THE NOISE IN THE LABELING OF IMAGES IS TYPICALLY RELATIVELY LOW. The word `label` names something with a different reliability depending on who produced it	`mechanisms[0]`	0.95	the tagger's words
mechanisms	mechanisms Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. about this quote: THIS COPY MOVES TWO WORDS INTO THE QUOTE: the 'In this' after 'is' belongs at the start of the next sentence, before 'dataset pulmonary nodules'. also in: A REFERENCE STANDARD IS BUILT WITH CONSENSUS DELIBERATELY NOT FORCED, AND THE DISAGREEMENT THEN OUTNUMBERS THE AGREEMENT THREE TO ONE: nodules were annotated by four radiologists INDEPENDENTLY, the readers REVIEWED EACH OTHERS ANNOTATIONS BUT NO CONSENSUS WAS FORCED - and the number they DID NOT UNANIMOUSLY AGREE ON to be a nodule WAS THREE TIMES LARGER THAN THE NUMBER THEY DID FULLY AGREE ON	`mechanisms[1]`	0.95	the tagger's words
mechanisms	mechanisms Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: THE CONSEQUENCE FOR ANYTHING TRAINED ON IT IS STATED, AND THE FIX IS AN OPEN PROBLEM: training a system on such data REQUIRES CAREFUL CONSIDERATION OF HOW TO DEAL WITH NOISE AND UNCERTAINTY IN THE REFERENCE STANDARD - one COULD THINK OF solutions like incorporating labeling uncertainty directly in the loss function, BUT THIS IS STILL AN OPEN CHALLENGE. The uncertainty is acknowledged and has nowhere to go	`mechanisms[2]`	0.95	the tagger's words
mechanisms	mechanisms Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: A BINARY PRESENTATION OF THE TASK IS CALLED A GROSS SIMPLIFICATION, AND THE REASON IS HETEROGENEITY ON BOTH SIDES: classification is often presented as A BINARY TASK - normal versus abnormal, object versus background - HOWEVER THIS IS OFTEN A GROSS SIMPLIFICATION AS BOTH CLASSES CAN BE HIGHLY HETEROGENEOUS. Not one class hiding structure but both	`mechanisms[3]`	0.95	the tagger's words
mechanisms	mechanisms Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: THE `NORMAL` CATEGORY IS SHOWN TO CONTAIN THINGS THAT ARE NOT NORMAL, INCLUDING ARTIFACTS OF THE INSTRUMENT: the normal category OFTEN CONSISTS OF COMPLETELY NORMAL TISSUE BUT ALSO several categories of BENIGN FINDINGS, WHICH CAN BE RARE, and MAY OCCASIONALLY INCLUDE A WIDE VARIETY OF IMAGING ARTIFACTS. The residual class is defined by what it is not, and collects the measurement's own failures	`mechanisms[4]`	0.95	the tagger's words
mechanisms	mechanisms Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. about this quote: RE-SLICED off a damage combination I had not seen before today and which NONE of my guards catch: the sentence reads "systems that are ex- 25 tremely good at excluding" - a line-break hyphen RETAINED (shape 11) with a PAGE NUMBER inserted at the break. The page-number guard needs a lowercase letter before the digit and finds "- "; the hyphen guards look for a fused or a surviving hyphen, not one with a number wedged into it. So the sentence's most important word is destroyed and every check passes. The span now starts after it. also in: THE PERFORMANCE PATTERN IS PREDICTED FROM THE CATEGORY'S STRUCTURE: systems end up GOOD AT EXCLUDING THE MOST COMMON NORMAL SUBCLASSES BUT FAIL MISERABLY ON SEVERAL RARE ONES. A single accuracy figure over a heterogeneous class hides exactly the failures that matter, and the shape of the class predicts the shape of the failure	`mechanisms[5]`	0.85	the tagger's words
mechanisms	mechanisms Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: THE CORRECT FIX IS NAMED AND DECLINED ON COST, IN ONE SENTENCE: a STRAIGHTFORWARD SOLUTION would be to turn the system into a MULTI-CLASS one by providing DETAILED ANNOTATIONS OF ALL POSSIBLE SUBCLASSES - OBVIOUSLY THIS AGAIN COMPOUNDS THE ISSUE OF LIMITED AVAILABILITY OF EXPERT TIME FOR ANNOTATING AND IS THEREFORE OFTEN SIMPLY NOT FEASIBLE. The right answer is known and unaffordable	`mechanisms[6]`	0.95	the tagger's words
mechanisms	mechanisms Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: THE WORKAROUNDS ARE REPORTED TO FAIL FOR THE SAME REASON THE PROBLEM EXISTS: strategies that incorporate intelligence in the training process - selective sampling, hard negative mining - TYPICALLY FAIL WHEN THERE IS SUBSTANTIAL NOISE IN THE REFERENCE STANDARD - and ADDITIONAL METHODS FOR DEALING WITH WITHIN-CLASS HETEROGENEITY WOULD BE HIGHLY WELCOME. The noise in the labels defeats the corrections for the labels	`mechanisms[7]`	0.95	the tagger's words
mechanisms	mechanisms Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: A STAIN IS OFFERED AS A SUBSTITUTE FOR EXPERTISE: in histopathology one can sometimes use SPECIFIC IMMUNOHISTOCHEMICAL STAINS to highlight regions of interest, REDUCING THE NEED FOR EXPERT EXPERIENCE - and non-expert labels can be leveraged VIA CROWD-SOURCING. Two ways to replace a judgement: a reagent and a crowd	`mechanisms[8]`	0.95	the tagger's words
mechanisms	mechanisms Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: A BENCHMARK IS REPORTED TO HAVE SATURATED, SO THAT IMPROVEMENTS CAN NO LONGER BE ATTRIBUTED: since 2014 the performance on the benchmark HAS SATURATED and IT IS DIFFICULT TO ASSESS WHETHER THE SMALL INCREASES IN PERFORMANCE CAN REALLY BE ATTRIBUTED TO 'BETTER' AND MORE SOPHISTICATED ARCHITECTURES. The scare quotes on `better` are the paper's, and the measuring instrument has stopped discriminating	`mechanisms[9]`	0.95	the tagger's words
mechanisms	mechanisms Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: A PERFORMANCE CRITERION IS DECLARED INSUFFICIENT AND SUPPLEMENTED BY AN ARTICULATION CRITERION, JUSTIFIED BY LEGAL ACCOUNTABILITY: these methods have often been described as BLACK BOXES, and especially in medicine WHERE ACCOUNTABILITY IS IMPORTANT AND CAN HAVE SERIOUS LEGAL CONSEQUENCES, IT IS OFTEN NOT ENOUGH TO HAVE A GOOD PREDICTION SYSTEM - the system ALSO HAS TO BE ABLE TO ARTICULATE ITSELF in a certain way. What counts as success acquires a second component from outside the field	`mechanisms[10]`	0.95	the tagger's words
mechanisms	mechanisms Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. about this quote: The span carries this file's surviving line-break hyphens - “publica- tions”, “contri- butions” - quoted verbatim, because a quote is the staged file's bytes. AND THE SPAN IS IN TEXT THAT WAS NOT STAGED UNTIL 2026-08-17: this record held 64% of its source and stopped mid-clause, so these sentences were never available to any earlier pass. also in: A SURVEY DE-DUPLICATES ITS OWN POPULATION BY A JUDGEMENT IT NEVER DEFINES, AND SUPPLIES A WORKED EXAMPLE INSTEAD OF A CRITERION: when largely overlapping work had been reported in multiple publications, ONLY THE PUBLICATION DEEMED MOST IMPORTANT WAS INCLUDED - A TYPICAL EXAMPLE HERE WAS ARXIV PREPRINTS THAT WERE SUBSEQUENTLY PUBLISHED, or conference contributions later expanded into journal papers. What `most important` means is stated nowhere, the survey's whole 308-paper population rests on it, and the example is doing the work a definition would have done	`mechanisms[11]`	0.95	the tagger's words
limitations	open challenges and future-research directions remain in the field Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: the survey's own account of the field: it offers Quotation withheld rather than a settled picture	`limitations[0]`		the tagger's words
limitations	The survey deliberately excludes traditional (non-deep) feature-learning approaches applied to medical images. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: any comparison between deep and traditional methods drawn from this survey: WE FOCUS PARTICULARLY ON SUCH DEEP MODELS, AND DO NOT INCLUDE THE MORE TRADITIONAL FEATURE LEARNING APPROACHES THAT HAVE BEEN APPLIED TO MEDICAL IMAGES	`limitations[1]`		the tagger's words
limitations	Coverage has a hard cut-off date, so later work is not included. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: every count and trend in the survey: THE LAST UPDATE TO THE INCLUDED PAPERS WAS ON FEBRUARY 1, 2017	`limitations[2]`		the tagger's words
limitations	Completeness of the literature search is an expectation rather than a guarantee. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: the completeness of the literature search behind those counts: WE EXPECT THE SEARCH TERMS USED TO COVER MOST, IF NOT ALL, OF THE WORK INCORPORATING DEEP LEARNING METHODS	`limitations[3]`		the tagger's words
limitations	On a widely used dataset the disputed annotations outnumber the agreed ones three to one. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. about this quote: THIS COPY MOVES TWO WORDS INTO THE QUOTE: the 'In this' after 'is' belongs at the start of the next sentence, before 'dataset pulmonary nodules'. · what this limits: any accuracy reported on the LIDC-IDRI dataset: no consensus was forced between the four radiologists, and the number of nodules they did not unanimously agree on was three times larger than the number they did fully agree on	`limitations[4]`	1.0	the tagger's words
limitations	Label noise is worse in this field than in the field the methods came from, and the survey says so. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: any transfer of a computer-vision result to medical imaging: label noise can be a significant limiting factor even with domain-expert annotation, whereas in computer vision the labelling noise is typically relatively low	`limitations[5]`	1.0	the tagger's words
limitations	Training on such data is stated to be an open problem, not a solved preprocessing step. Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. what this limits: any system trained on a disputed reference standard: the authors state that handling noise and uncertainty in the reference standard requires careful consideration and that incorporating labelling uncertainty in the loss function is still an open challenge	`limitations[6]`	1.0	the tagger's words
transfers	deep convolutional networks, developed in computer vision, adopted as the methodology of choice for medical image analysis (classification, detection, segmentation) Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`transfers[0]`		the tagger's words
controls	pre-stated exclusion of papers not reporting results on medical image data or using only standard feed-forward networks with handcrafted features, plus de-duplication where the same work appeared in several publications Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. eliminates: a survey padded by work that is not deep learning on medical images, and double-counting of one piece of work reported more than once · outcome (stated by the tagger, no quotation): held	`controls[0]`	0.85	the tagger's words
concepts	reference standard Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: reference standard	`concepts[5]`		the paper's words
concepts	label noise Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: label noise	`concepts[6]`		the paper's words
concepts	selective sampling Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: selective sampling	`concepts[7]`		the paper's words
concepts	within-class heterogeneity Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: within-class heterogeneity	`concepts[8]`		the paper's words
concepts	binary task Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: binary task	`concepts[9]`		the paper's words
senses	the reference standard - AN ANNOTATION SET WITH ITS DISAGREEMENT LEFT IN: four radiologists annotated independently, reviewed each other, and NO CONSENSUS WAS FORCED, so what every accuracy on this dataset is measured against is a set whose disputed part is three times its agreed part - the standard is not a ground truth but a record of expert disagreement Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. about this quote: THIS COPY MOVES TWO WORDS INTO THE QUOTE: the 'In this' after 'is' belongs at the start of the next sentence, before 'dataset pulmonary nodules'.	`senses[0]`	1.0	the tagger's words
senses	label noise - A FIELD-RELATIVE QUANTITY: the survey contrasts medical imaging, where it can be a significant limiting factor even with domain-expert annotation, against computer vision, where it is typically relatively low - so the same word names a small nuisance in one literature and a limiting factor in the other, and the architectures move between them Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`senses[1]`	1.0	the tagger's words
senses	the binary task - A STATED SIMPLIFICATION: normal versus abnormal is called a gross simplification because both classes are highly heterogeneous, which is why systems can excel on common normal subclasses and fail on rare ones; the two-class label is a choice about the data, not a property of it Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`senses[2]`	1.0	the tagger's words
claim_relations	builds-on → convolutional neural networks (deep learning) Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`claim_relations[0]`		the tagger's words
claim_relations	extends → Shen et al. (2017), the earlier dedicated review of deep learning in medical image analysis Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`claim_relations[1]`		the tagger's words
content_tags	300 — papers summarized in the survey Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`content_tags.quantities[0]`		the tagger's words
content_tags	1970s-1990s — early rule-based image analysis period Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`content_tags.quantities[1]`		the tagger's words
content_tags	8 — clinical domains reviewed Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`content_tags.quantities[2]`		the tagger's words
content_tags	convolutional networks became the method of choice for medical images Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`content_tags.findings[0]`		the tagger's words
content_tags	the field shifted from human-designed to computer-trained systems Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`content_tags.findings[1]`		the tagger's words
content_tags	most surveyed contributions appeared in the last year Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above.	`content_tags.findings[2]`		the tagger's words
content_tags	the next step is to let computers learn features automatically Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. about this quote: QUOTE RECOVERED FROM THE PAPER'S OWN CHARACTERS. The recorded quote was LETTERS-ONLY IDENTICAL to this stretch of the staged text but not an exact substring of it - spaces, hyphens, ligature glyphs or mathematical operators differed, so it could never anchor. This is the file's own text between the same two letters: not produced by a rule about the damage, copied out of the file. Round-trip checked (the recovered span is letters-only equal to the original) and length-checked. The text as originally recorded is kept in evidence as extracted.	`content_tags.findings[3]`		the tagger's words
content_tags	training data Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: training data	`content_tags.variables[0]`		the tagger's words
content_tags	decision boundary Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: decision boundary	`content_tags.variables[1]`		the tagger's words
content_tags	discriminant features Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: discriminant features	`content_tags.variables[2]`		the tagger's words
content_tags	imaging tasks Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: imaging tasks	`content_tags.variables[3]`		the tagger's words
content_tags	GOFAI Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: GOFAI	`content_tags.definitions[0]`		the tagger's words
content_tags	handcrafted features Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: handcrafted features	`content_tags.definitions[1]`		the tagger's words
content_tags	feature learning Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. about this quote: QUOTE RECOVERED FROM THE PAPER'S OWN CHARACTERS. The recorded quote was LETTERS-ONLY IDENTICAL to this stretch of the staged text but not an exact substring of it - spaces, hyphens, ligature glyphs or mathematical operators differed, so it could never anchor. This is the file's own text between the same two letters: not produced by a rule about the damage, copied out of the file. Round-trip checked (the recovered span is letters-only equal to the original) and length-checked. The text as originally recorded is kept in evidence as extracted. also in: feature learning	`content_tags.definitions[3]`		the tagger's words
content_tags	Bayesian Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. about this quote: QUOTE REPAIRED: the extractor split a hyphen-space and the quote was recorded faithfully with the damage in it, which made it unanchorable. The repaired string is an EXACT SUBSTRING of the staged text - that is why it was accepted, not because a rule produced it. The text as originally recorded is kept in evidence as extracted. also in: Bayesian	`content_tags.statistics[0]`		the tagger's words
content_tags	regression Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. also in: regression	`content_tags.statistics[1]`		the tagger's words
content_tags	statistical-significance Quotation withheld — this record is not cleared for redistribution. The sentence is in the paper at the link above. about this quote: QUOTE REPAIRED: the extractor split a hyphen-space and the quote was recorded faithfully with the damage in it, which made it unanchorable. The repaired string is an EXACT SUBSTRING of the staged text - that is why it was accepted, not because a rule produced it. The text as originally recorded is kept in evidence as extracted. also in: statistical-significance	`content_tags.statistics[2]`		the tagger's words

Interpretive choices

Tags that could not be placed in the text:

none — every tag was located in the text

What was deliberately not tagged, and why, is recorded per paper by the tagger. Where a paper has no such record, this page says so rather than implying that nothing was declined.