Gains in accuracy from averaging ratings of abnormality

Richard G. Swensson; Jill L. King; David Gur; Walter F. Good

doi:10.1117/12.349630

24 May 1999 Gains in accuracy from averaging ratings of abnormality

Richard G. Swensson, Jill L. King, David Gur, Walter F. Good

Proceedings Volume 3663, Medical Imaging 1999: Image Perception and Performance; (1999) https://doi.org/10.1117/12.349630
Event: Medical Imaging '99, 1999, San Diego, CA, United States

Abstract

Six radiologists used continuous scales to rate 529 chest-film cases for likelihood of five separate types of abnormalities (interstitial disease, nodules, pneumothorax, alveolar infiltrates and rib fractures) in each of six replicated readings, yielding 36 separate ratings of each case for the five abnormalities. Analyses for each type of abnormality estimated the relative gains in accuracy (area below the ROC curve) obtained by averaging the case-ratings across: (1) six independent replications by each reader (30% gain), (2) six different readers within each replication (39% gain) or (3) all 36 readings (58% gain). Although accuracy differed among both readers and abnormalities, ROC curves for the median ratings showed similar relative gains in accuracy. From a latent-variable model for these gains, we estimate that about 51% of a reader's total decision variance consisted of random (within-reader) errors that were uncorrelated between replications, another 14% came from that reader's consistent (but idiosyncratic) responses to different cases, and only about 35% could be attributed to systematic variations among the sampled cases that were consistent across different readers.

Citation Download Citation

Richard G. Swensson, Jill L. King, David Gur, and Walter F. Good "Gains in accuracy from averaging ratings of abnormality", Proc. SPIE 3663, Medical Imaging 1999: Image Perception and Performance, (24 May 1999); https://doi.org/10.1117/12.349630

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available