Using heterogeneous annotation and visual information for the benchmarking of image retrieval system

Henning Müller; Paul Clough; William Hersh; Thomas Deselaers; Thomas M. Lehmann; Bruno Janvier; Antoine Geissbuhler

doi:10.1117/12.660259

16 January 2006 Using heterogeneous annotation and visual information for the benchmarking of image retrieval systems

Henning Müller, Paul Clough, William Hersh, Thomas Deselaers, Thomas M. Lehmann, Bruno Janvier, Antoine Geissbuhler

Proceedings Volume 6061, Internet Imaging VII; 606105 (2006) https://doi.org/10.1117/12.660259
Event: Electronic Imaging 2006, 2006, San Jose, California, United States

Abstract

Many image retrieval systems, and the evaluation methodologies of these systems, make use of either visual or textual information only. Only few combine textual and visual features for retrieval and evaluation. If text is used, it is often relies upon having a standardised and complete annotation schema for the entire collection. This, in combination with high-level semantic queries, makes visual/textual combinations almost useless as the information need can often be solved using just textual features. In reality, many collections do have some form of annotation but this is often heterogeneous and incomplete. Web-based image repositories such as FlickR even allow collective, as well as multilingual annotation of multimedia objects. This article describes an image retrieval evaluation campaign called ImageCLEF. Unlike previous evaluations, we offer a range of realistic tasks and image collections in which combining text and visual features is likely to obtain the best results. In particular, we offer a medical retrieval task which models exactly the situation of heterogenous annotation by combining four collections with annotations of varying quality, structure, extent and language. Two collections have an annotation per case and not per image, which is normal in the medical domain, making it difficult to relate parts of the accompanying text to corresponding images. This is also typical of image retrieval from the web in which adjacent text does not always describe an image. The ImageCLEF benchmark shows the need for realistic and standardised datasets, search tasks and ground truths for visual information retrieval evaluation.

Citation Download Citation

Henning Müller, Paul Clough, William Hersh, Thomas Deselaers, Thomas M. Lehmann, Bruno Janvier, and Antoine Geissbuhler "Using heterogeneous annotation and visual information for the benchmarking of image retrieval systems", Proc. SPIE 6061, Internet Imaging VII, 606105 (16 January 2006); https://doi.org/10.1117/12.660259

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available