2016-079 - Large-scale video content retrieval through text query

Description:

2016-079 Large-Scale Video Content Retrieval Through Text Query

Abstract

Video is one of the most popular communication tools today. The ability to search complex events in video based on text queries in large scale without using any video metadata such as titles, human provided keywords or video captions is largely unmet; large-scale content-based semantic search in video is a fundamental problem in multimedia analysis and retrieval. Videos shared may have either no text or only a few words with little relevance to the visual content. Current methods include searching user-generated metadata, often text-based (e.g., titles, descriptions, captions). These methods are unscalable, slow, inconsistent and index videos based on raw concept detection. This technology addresses these long-standing challenges to improve video search experiences through understanding the meaning that lies in the video content. Researchers at CMU have built a system to label videos automatically with textual representations, in a very large scale, for effective search and retrieval.

Benefit

Advances the text retrieval method for video retrieval
- Supports video summarization, visualization, search & recommendations
Scalable to big data collections required for real world applications
Uses a single CPU core
Fast-It only takes 0.2 seconds on a single CPU core to search a collection of 100 million internet videos
Free of user-generated metadata
Zero example search (0Ex) -does not require thousands of example videos to detect multimedia events. 0Ex resembles a real-world scenario in which users start the search without any example
Increased accuracy due to use of concept adjustment & concise optimization frameworks with interpretations