PROPOSAL

Competitor for the Video Browser Showdown

Supervisor: Björn Þór Jónsson
Semester: Fall 2019

The goal of this project is to integrate Exquisitor with other pieces of existing technology and turn into a competitor for a live video retrieval competition. The project is suitable for 3-4 well-qualified MSc students.

The ​Video Browser Showdown​ (VBS) is a live competition​ for video search andretrieval, held at the International Conference on Multimedia Modeling (MMM). In VBS, the competition tasks involve finding videos from a collection of 1000 hours of videos. The tasks are either based on a textual or a visual description, and the competing systems are judged based on speed, accuracy and recall, depending on the task. The goal is to develop a ​new competitor​ for VBS, based on existing pieces of technology developed at ITU and UvA. These are: * The Exquisitor image browser (see figure), developed at ITU and UvA, which usesrelevance feedback and high-dimensional indexing to rapidly find relevant images [1]. * The O​3​ media server, and the corresponding P​3​ photo browser, developed at ITU, whichuse a novel data model to filter media and present contents [2]. * A video engine, developed at UvA, to process, index and search for videos based onvisual content.

The research question to answer is: ​how well do the video exploration concepts of these ITU/UvA tools address the workloads of VBS compared to existing tools?​ The task of the student group will be to integrate (and extend) the components into a single system, develop the communication with the VBS competition system, test the entire software stack with real users using competition data and workloads from VBS 2018, and prepare for VBS 2019. The project is suitable for 3-4 well-qualified MSc students. Participation in VBS includes an international conference paper. And the presentation of the MSc project should be exceptionally visual and interesting! References * [1]Jan Zahálka, Stevan Rudinac, Björn Þór Jónsson, Dennis C. Koelma, Marcel Worring. ​Blackthorn:Large-Scale Interactive Multimodal Learning​. IEEE Transactions on Multimedia (TMM), 20(3), March2018. * [2]Snorri Gíslason, Björn Þór Jónsson, Laurent Amsaleg. ​Integration of Exploration and Search: A CaseStudy of the M​3​ Model​. Proceedings of the International Conference on Multimedia Modeling (MMM),Thessaloniki, Greece, January 2019.