Searching Large-Scale Image Collections

Citation

Aly, Mohamed Alaa El-Dien Mahmoud Hussein (2011) Searching Large-Scale Image Collections. Dissertation (Ph.D.), California Institute of Technology. doi:10.7907/VRGJ-4J54. https://resolver.caltech.edu/CaltechTHESIS:04252011-145432540

Abstract

Searching quickly and accurately in a large collection of images has become an increasingly important problem. The ultimate goal is to make visual search possible: allow users to search using images in addition to typing text. The typical approach is to index all the images of interest (e.g., images of landmarks, books, or DVDs) in a database and let users question the system with query images. Such a database can reach billions of images, and this poses challenges in terms of memory and computational requirements and recognition performance. In this work we provide an in depth study of systems used for searching large-scale image collections. Specifically, we provide a thorough comparison of the two leading image search approaches: Full Representation (FR) vs. Bag of Words (BoW). We derive theoretical estimates of how the memory and computational cost scale with the number of images in the database, and empirically evaluate the performance and run time on four real-world datasets. Our experiments suggest that FR provides better recognition performance than BoW, though it requires more memory. Therefore, we address these shortcomings by presenting novel methods that increase the recognition performance of BoW and decrease the memory requirements of FR. Finally, we present a novel way to parallelize FR on multiple machines and scale up database sizes to 100 million images with interactive run time.

Thesis Files