Calculation of signal detection theory measures

Calculation of signal detection theory measures Stanislaw Todorov

Tag layer of the DHSS Meta-Tagging Project. The article text is reproduced from the staged copy named in sourceDesc and is not altered; the tags are held separately in standOff so that overlapping and nested spans are preserved.

Calculation of signal detection theory measures 1999 Behavior Research Methods, Instruments, & Computers 10.3758/bf03207704 stanislaw1999-calculation-of-signal-detection-th stanislaw1999-calculation-of-signal-detection-th.txt, the project's staged copy 4104a1ba189cac18389db2f020a9209a6f7260f4 open-access quantitative-psychology

the paper's own words

the tag layer; a sentence written by the tagger about the paper, always anchored to a quotation but not itself a quotation

0 spans located by searching the stored text for the quote's exact words, allowing any whitespace between them method Quotation withheld because this record is not cleared for redistribution. signal detection theory measures of sensitivity and response bias Quotation withheld because this record is not cleared for redistribution. review review Quotation withheld because this record is not cleared for redistribution. mechanisms Signal detection theory is underused because textbooks describe the theory but omit the computational methods needed to implement it. Quotation withheld because this record is not cleared for redistribution. mechanisms Moving the criterion shifts hit and false-alarm rates together, so a liberal criterion inflates both; only reducing the overlap of the signal and noise distributions can raise hits while lowering false alarms. Quotation withheld because this record is not cleared for redistribution. mechanisms Ordinary least squares yields biased ROC slope and intercept estimates because it assumes only one variable carries error, while both rate z scores are measured with sampling error. Quotation withheld because this record is not cleared for redistribution. Textbooks describe the theory and then decline to give the calculation - the paper quotes one concluding a nine-page treatment with the statement that the measures can be calculated by a procedure it will not discuss. Fewer than half the studies that could use SDT do. a theory is under-used because its methods are omitted, not doubted Quotation withheld because this record is not cleared for redistribution. The two assumptions d-prime rests on cannot be tested in the task where d-prime is most used; rating tasks are required instead. what this limits: a d-prime computed from a yes/no task: the paper states the d-prime assumptions cannot actually be tested in yes/no tasks and that rating tasks are required to test them Quotation withheld because this record is not cleared for redistribution. One of the two assumptions is singled out as particularly doubtful for some yes/no tasks. what this limits: d-prime as a pure sensitivity measure in a yes/no task: the paper states the assumptions may not be tenable and that equality of the signal and noise standard deviations is particularly suspect Quotation withheld because this record is not cleared for redistribution. The common bias measure carries its own assumption about how responses are generated. what this limits: a beta value reported as response bias: its use assumes Quotation withheld because this record is not cleared for redistribution. The paper's case that the barrier is now gone rests on a comparison rather than a measurement - that software makes the analysis no harder than a t test. Quotation withheld because this record is not cleared for redistribution. signal detection theory, carried into psychology to quantify discrimination between two stimulus types - and it brought its vocabulary with it, so psychological stimuli are still called 'signal' and 'noise' Quotation withheld because this record is not cleared for redistribution. d' assumptions d' assumptions Quotation withheld because this record is not cleared for redistribution. response bias response bias Quotation withheld because this record is not cleared for redistribution. signal detection theory signal detection theory Quotation withheld because this record is not cleared for redistribution. signal signal Quotation withheld because this record is not cleared for redistribution. noise noise Quotation withheld because this record is not cleared for redistribution. recognition memory recognition memory Quotation withheld because this record is not cleared for redistribution. medical diagnosis medical diagnosis Quotation withheld because this record is not cleared for redistribution. the d' assumptions - NAMED AND UNTESTABLE IN PLACE: normality and equal standard deviations, which the paper says cannot be tested in the yes/no task itself Quotation withheld because this record is not cleared for redistribution. discriminability versus response bias - the distinction the paper says is so critical that it is illustrated with five examples from widely disparate areas of study Quotation withheld because this record is not cleared for redistribution. signal / noise - in this paper NOT perceptual stimuli but whichever two categories a domain must discriminate: old/new items, lies/truths, guilty/innocent defendants, diseased/well patients. The words are inherited from perception and the referents are not Quotation withheld because this record is not cleared for redistribution. extends → Grier (1971) Quotation withheld because this record is not cleared for redistribution. contradicts → Macmillan and Creelman (1996) Quotation withheld because this record is not cleared for redistribution. signal —distinct-from→ noise signal noise differs in: NOTHING FIXED - the contrast is whatever the domain supplies. Across the seven applications listed in one sentence it is old against new, lie against truth, guilty against innocent, diseased against well. The pair names a CONTRAST STRUCTURE, not two things · strength of the relation: 0.9 Quotation withheld because this record is not cleared for redistribution. d' (sensitivity index) d' (sensitivity index) Quotation withheld because this record is not cleared for redistribution. A' (nonparametric sensitivity measure) A' (nonparametric sensitivity measure) Quotation withheld because this record is not cleared for redistribution. beta (likelihood-ratio response bias measure) beta (likelihood-ratio response bias measure) Quotation withheld because this record is not cleared for redistribution. regression (ordinary least squares) regression (ordinary least squares) Quotation withheld because this record is not cleared for redistribution. correlation correlation Quotation withheld because this record is not cleared for redistribution. ROC curve fitting (slope and intercept in z space) ROC curve fitting (slope and intercept in z space) Quotation withheld because this record is not cleared for redistribution. The major contribution of signal detection theory to psychology is that it separates response bias from sensitivity. Quotation withheld because this record is not cleared for redistribution. Despite wide acceptance of the theory, fewer than half of the studies to which signal detection theory applies actually use it. Quotation withheld because this record is not cleared for redistribution. Fitting ROC data by ordinary least squares gives biased slope and intercept estimates, because in rating tasks both the hit-rate and the false-alarm-rate z scores carry sampling error rather than only one of them. Quotation withheld because this record is not cleared for redistribution. In the worked example OLS gave the misleading impression that the signal and noise distributions have equal standard deviations. Quotation withheld because this record is not cleared for redistribution. The violated equal-variance assumption explains why the d' values from different criteria varied so widely. Quotation withheld because this record is not cleared for redistribution. Lookup tables are an inaccurate route to SDT measures because they force hit and false-alarm rates to be rounded, usually to two significant digits, and some published tables contain wrong entries. Quotation withheld because this record is not cleared for redistribution. criterion criterion Quotation withheld because this record is not cleared for redistribution. response bias response bias Quotation withheld because this record is not cleared for redistribution. d' d' Quotation withheld because this record is not cleared for redistribution. the d' assumptions the d' assumptions Quotation withheld because this record is not cleared for redistribution. receiver operating characteristic (ROC) curve receiver operating characteristic (ROC) curve Quotation withheld because this record is not cleared for redistribution. ROC area ROC area Quotation withheld because this record is not cleared for redistribution. m-alternative forced-choice (mAFC) task m-alternative forced-choice (mAFC) task Quotation withheld because this record is not cleared for redistribution. loglinear approach loglinear approach Quotation withheld because this record is not cleared for redistribution. hits hits Quotation withheld because this record is not cleared for redistribution. false alarms false alarms Quotation withheld because this record is not cleared for redistribution. signal distribution signal distribution Quotation withheld because this record is not cleared for redistribution. noise distribution noise distribution Quotation withheld because this record is not cleared for redistribution. over 2,000 — citations to Green and Swets (1966) recorded in the Social Sciences Citation Index, used to argue SDT is widely accepted Quotation withheld because this record is not cleared for redistribution. .9332 — hit rate in the illustrative Figure 1 example, the proportion of the signal distribution above the criterion Quotation withheld because this record is not cleared for redistribution. 50 signal trials and 25 noise trials — size of the hypothetical rating-task data set used for the worked computational example Quotation withheld because this record is not cleared for redistribution. d' = 1.56 — sensitivity computed for the second criterion of the worked example (hit rate .92, false-alarm rate .44) Quotation withheld because this record is not cleared for redistribution. A' = .84 — nonparametric sensitivity for the same pair of rates in the worked example Quotation withheld because this record is not cleared for redistribution. slope of 1.28 and an intercept of 1.52 — ROCFIT's fitted z-space ROC parameters for the example rating data, from which Az is computed Quotation withheld because this record is not cleared for redistribution. r = .90 — correlation between hit-rate and false-alarm-rate z scores in the example, shown still to leave the two OLS slopes far apart Quotation withheld because this record is not cleared for redistribution.

The full text of this article is not redistributable, so it is not included here. The evidence sentences are withheld with it; the tag layer and its grounding status remain. The article itself is at https://doi.org/10.3758/bf03207704.