The doctoral dissertations of the former Helsinki University of Technology (TKK) and Aalto University Schools of Technology (CHEM, ELEC, ENG, SCI) published in electronic format are available in the electronic publications archive of Aalto University - Aaltodoc.
Aalto

Interactive Image Retrieval Using Self-Organizing Maps

Markus Koskela

Dissertation for the degree of Doctor of Science in Technology to be presented with due permission of the Department of Computer Science and Engineering for public examination and debate in Auditorium T2 at Helsinki University of Technology (Espoo, Finland) on the 14th of November, 2003, at 12 o'clock noon.

Overview in PDF format (ISBN 951-22-6765-9)   [3491 KB]
Dissertation is also available in print (ISBN 951-22-6764-0)

Abstract

Digital image libraries are becoming more common and widely used as visual information is produced at a rapidly growing rate. Creating and storing digital images is nowadays easy and getting more affordable all the time as the needed technologies are maturing and becoming eligible for general use. As a result, the amount of data in visual form is increasing and there is a strong need for effective ways to manage and process it. In many settings, the existing and widely adopted methods for text-based indexing and information retrieval are inadequate for these new purposes.

Content-based image retrieval addresses the problem of finding images relevant to the users' information needs from image databases, based principally on low-level visual features for which automatic extraction methods are available. Due to the inherently weak connection between the high-level semantic concepts that humans naturally associate with images and the low-level visual features that the computer is relying upon, the task of developing this kind of systems is very challenging. A popular method to improve retrieval performance is to shift from single-round queries to navigational queries where a single retrieval instance consists of multiple rounds of user-system interaction and query reformulation. This kind of operation is commonly referred to as relevance feedback and can be considered as supervised learning to adjust the subsequent retrieval process by using information gathered from the user's feedback.

In this thesis, an image retrieval system named PicSOM is presented, including detailed descriptions of using multiple parallel Self-Organizing Maps (SOMs) for image indexing and a novel relevance feedback technique. The proposed relevance feedback technique is based on spreading the user responses to local SOM neighborhoods by a convolution with a kernel function. A broad set of evaluations with different image features, retrieval tasks, and parameter settings demonstrating the validity of the retrieval method is described. In particular, the results establish that relevance feedback with the proposed method is able to adapt to different retrieval tasks and scenarios.

Furthermore, a method for using the relevance assessments of previous retrieval sessions or potentially available keyword annotations as sources of semantic information is presented. With performed experiments, it is confirmed that the efficiency of semantic image retrieval can be substantially increased by using these features in parallel with the standard low-level visual features.

This thesis consists of an overview and of the following 7 publications:

  1. Laaksonen J., Koskela M., Laakso S. and Oja E., 2000. Pic-SOM – content-based image retrieval with self-organizing maps. Pattern Recognition Letters 21, No. 13-14, pages 1199-1207. © 2000 Elsevier Science. By permission.
  2. Laaksonen J., Oja E., Koskela M. and Brandt S., 2000. Analyzing low-level visual features using content-based image retrieval. Proceedings of the 7th International Conference on Neural Information Processing (ICONIP 2000) (invited paper). Taejon, Korea, Vol. 2, pages 1333-1338.
  3. Laaksonen J., Koskela M., Laakso S. and Oja E., 2001. Self-Organising Maps as a relevance feedback technique in content-based image retrieval. Pattern Analysis & Applications 4, No. 2-3, pages 140-152. © 2001 Springer-Verlag. By permission.
  4. Koskela M., Laaksonen J. and Oja E., 2001. Comparison of techniques for content-based image retrieval. Proceedings of the 12th Scandinavian Conference on Image Analysis (SCIA 2001). Bergen, Norway, pages 579-586. © 2001 Norsk forening for bildebehandling og mønstergjenkjenning (NOBIM). By permission.
  5. Laaksonen J., Koskela M. and Oja E., 2002. PicSOM – self-organizing image retrieval with MPEG-7 content descriptors. IEEE Transactions on Neural Networks: Special Issue on Intelligent Multimedia Processing 13, No. 4, pages 841-853. © 2002 IEEE. By permission.
  6. Koskela M., Laaksonen J. and Oja E., 2002. Implementing relevance feedback as convolutions of local neighborhoods on Self-Organizing Maps. Proceedings of the International Conference on Artificial Neural Networks (ICANN 2002). Madrid, Spain, pages 981-986. © 2002 Springer-Verlag. By permission.
  7. Koskela M. and Laaksonen J., 2003. Using long-term learning to improve efficiency of content-based image retrieval. Proceedings of the Third International Workshop on Pattern Recognition in Information Systems (PRIS 2003). Angers, France, pages 72-79. © 2003 ICEIS Press. By permission.

Keywords: content-based image retrieval (CBIR), Self-Organizing Map (SOM), relevance feedback, MPEG-7

This publication is copyrighted. You may download, display and print it for Your own personal use. Commercial use is prohibited.

© 2003 Helsinki University of Technology

Last update 2011-05-26