Image-based Video Search Engine: Keyframe Extraction

Image-based Video Search Engine: Keyframe Extraction

Bos, Robert (TU Delft Electrical Engineering, Mathematics and Computer Science)
Zheng, Leo (TU Delft Electrical Engineering, Mathematics and Computer Science)

Dauwels, J.H.G. (mentor)
Llombart, Nuria (graduation committee)
Bol, E.W. (graduation committee)

Delft University of Technology

Electrical Engineering

Bachelor graduation project Electrical Engineering

2022-06-20

In this report, the analysis and design of a system that extracts keyframes from videos is detailed. The need for such a sub-module stems from the similarity of frames in a video. To aid in reducing the computation time of the content based video search engine, the Keyframe Extraction Module reduces the amount of frames by discarding frames that are similar in information. Determining what frames can be considered similar is one of the main challenges, as there are many ways of assigning values to how much frames differ. In the past decades, many research has been done on keyframe extraction and video summarization and many methods are proposed to form keyframe selections, varying in what is considered salient information and varying in computation time. The most challenging part of the design is that there is a time constraint present, which called for a proper analysis in what methods are suitable. After all, this limitation when creating video summaries is often not a large topic in research papers.This report will cover Shot Detection techniques along with various Keyframe Extraction methods that can be categorized in clustering, visual content, fixed selection and uniform sampling. Furthermore, evaluation methods like the Fidelity measure for the performance of particular methods are also addressed, as determining how well a keyframe selection is is not trivial. It is concluded that out of the techniques analyzed, a combination of VSUMM clustering and histogram matching along with histogram-based shot based detection with a CFAR threshold and pre-sampling is most suitable for the general case under a time constraint. Future work could include looking at hierarchical clustering methods and optimizing the Shot Boundary Detection module following the most recent papers.

Image Retrieval
Video search
Keyframe Extraction
Shot based detection
Video summarization
VSUMM

http://resolver.tudelft.nl/uuid:1f2f8fdc-dca9-46e0-a669-1d4f7c5170a5

Student theses

bachelor thesis

© 2022 Robert Bos, Leo Zheng