deep-CBIRImageRetrieval

deep-CBIRImageRetrieval - Image Retrieval: Current...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Image Retrieval: Current Techniques, Image Retrieval: Current Techniques, Promising Directions, and Open Issues Yong Rui, Thomas Huang and Shih­Fu Chang Published in the Journal of Visual Communication and Image Representation. Presented by: Deepak Bote Presentation Outline Presentation Outline History of image retrieval – Issues faced Solution – Content­based image retrieval Feature extraction Multidimensional indexing Current Systems Open issues Conclusion History of Image Retrieval History of Image Retrieval Traditional text­based image search engines Manual annotation of images Use text­based retrieval methods E.g. Water lilies Flowers in a pond <Its biological name> Limitations of text­based approach Limitations of text­based approach Problem of image annotation Problem of human perception Large volumes of databases Valid only for one language – with image retrieval this limitation should not exist Subjectivity of human perception Too much responsibility on the end­user Problem of deeper (abstract) needs Queries that cannot be described at all, but tap into the visual features of images. Outline Outline History of image retrieval – Issues faced Solution – Content­based image retrieval Feature extraction Multidimensional indexing Current Systems Open issues Conclusion What is CBIR? What is CBIR? Images have rich content. This content can be extracted as various content features: Mean color , Color Histogram etc… Take the responsibility of forming the query away from the user. Each image will now be described by its own features. CBIR – A sample search query CBIR – A sample search query User wants to search for, say, many rose images He submits an existing rose picture as query. He submits his own sketch of rose as query. The system will extract image features for this query. It will compare these features with that of other images in a database. Relevant results will be displayed to the user. Sample Query Sample Query Sample CBIR architecture Sample CBIR architecture Outline Outline History of image retrieval – Issues faced Solution – Content­based image retrieval Feature extraction Multidimensional indexing Current Systems Open issues Conclusion Feature Extraction Feature Extraction What are image features? Primitive features Mean color (RGB) Color Histogram Semantic features General features Color Layout, texture etc… Domain specific features Face recognition, fingerprint matching etc… Mean Color Mean Color Pixel Color Information: R, G, B Mean component (R,G or B)= Sum of that component for all pixels Number of pixels Pixel Histogram Histogram Frequency count of each individual color Most commonly used color feature representation Image Corresponding histogram Color Layout Color Layout Need for Color Layout Global color features give too many false positives How it works: Divide whole image into sub­blocks Extract features from each sub­block Can we go one step further? Divide into regions based on color feature concentration This process is called segmentation. Example: Color layout Example: Color layout ** Image adapted from Smith and Chang : Single Color Extraction and Image Query Texture Texture Texture – innate property of all surfaces Clouds, trees, bricks, hair etc… Refers to visual patterns of homogeneity Does not result from presence of single color Most accepted classification of textures based on psychology studies – Tamura representation • Coarseness • Linelikeness • Contrast • Regularity • Directionality • Roughness Segmentation issues Segmentation issues Considered as a difficult problem Not reliable Segments regions, but not objects Different requirements from segmentation: Shape extraction: High Accuracy required Layout features: Coarse segmentation may be enough Presentation Outline Presentation Outline History of image retrieval – Issues faced Solution – Content­based image retrieval Feature extraction Multidimensional indexing Current Systems Open issues Conclusion Problem of high dimensions Problem of high dimensions Mean Color = RGB = 3 dimensional vector Color Histogram = 256 dimensions Effective storage and speedy retrieval needed Traditional data­structures not sufficient R­trees, SR­Trees etc… 2­dimensional space 2­dimensional space Point A D2 D1 3­dimensional space 3­dimensional space Now, imagine… Now, imagine… An N­dimensional box!! We want to conduct a nearest neighbor query. R­trees are designed for speedy retrieval of results for such purposes Designed by Guttmann in 1984 Presentation Outline Presentation Outline History of image retrieval – Issues faced Solution – Content­based image retrieval Feature extraction Multidimensional indexing Current Systems Open issues Conclusion IBM’s QBIC IBM’s QBIC QBIC – Query by Image Content First commercial CBIR system. Model system – influenced many others. Uses color, texture, shape features Text­based search can also be combined. Uses R*­trees for indexing QBIC – Search by color QBIC – Search by color ** Images courtesy : Yong Rao QBIC – Search by shape QBIC – Search by shape ** Images courtesy : Yong Rao QBIC – Query by sketch QBIC – Query by sketch ** Images courtesy : Yong Rao Virage Virage Developed by Virage inc. Like QBIC, supports queries based on color, layout, texture Supports arbitrary combinations of these features with weights attached to each This gives users more control over the search process VisualSEEk VisualSEEk Research prototype – University of Columbia Mainly different because it considers spatial relationships between objects. Global features like mean color, color histogram can give many false positives Matching spatial relationships between objects and visual features together result in a powerful search. ISearch – my own system ISearch – my own system ISearch – my own system ISearch – my own system ISearch – my own system ISearch – my own system Feature selection in ISearch Feature selection in ISearch Database Admin facility in ISearch Database Admin facility in ISearch Presentation Outline Presentation Outline History of image retrieval – Issues faced Solution – Content­based image retrieval Feature extraction Multidimensional indexing Current Systems Open issues Conclusion Open issues Open issues Gap between low level features and high­level concepts Human in the loop – interactive systems Retrieval speed – most research prototypes can handle only a few thousand images. A reliable test­bed and measurement criterion, please! Presentation Outline Presentation Outline History of image retrieval – Issues faced Solution – Content­based image retrieval Feature extraction Multidimensional indexing Current Systems Open issues Conclusion Conclusion Conclusion Satisfactory progress, but still… A long way to go…!! Acknowledgements Acknowledgements Dr. Padma Mundur Mr. Yong Rao Mr. Sumit Jain, Software Engineer, KPIT Cummins, India Mr. Ajay Joglekar, Software Engineer, Veritas India. References References Y. Rui, T. S. Huang, and S.­F. Chang, “Image retrieval: Current techniques, promising directions, and open issues” S. Jain, A. Joglekar, and D. Bote, ISearch: A Content­based Image Retrieval (CBIR) Engine, as Bachelor of Computer Engineering final year thesis, Pune University, 2002 THANK YOU!!! Questions? ...
View Full Document

Ask a homework question - tutors are online