Image Search Engine for Digital History: A deep learning approach
M.R. van Geerenstein (TU Delft - Mechanical Engineering)
P.G. van Mastrigt (TU Delft - Electrical Engineering, Mathematics and Computer Science)
L.A.H. Vergroesen (TU Delft - Electrical Engineering, Mathematics and Computer Science)
J.H.G. Dauwels – Mentor (TU Delft - Signal Processing Systems)
A. Nanetti – Graduation committee member (Nanyang Technological University)
More Info
expand_more
Github repository containing the source code and documentation for this thesis.
https://github.com/EHM-Search-Engines/ISEDH-Deep-LearningOther than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
This research investigates and describes an image search engine for digital history using deep learning technologies. It is part of the Engineering Historical Memory research, contributing to a multilingual and transcultural approach to decode-encode the treasure of human experience and transmit it to the next generation of world citizens. The engine provides a new way to search in online (historical) digital libraries using content-based image retrieval and makes linguistic metadata redundant. State-of-the-art deep learning methodologies in computer vision have been investigated and tested. These methodologies include both template-based matching and feature-based matching. A VGG16 Convolutional Neural Network based approach, called D2-Net, is concluded to provide the best basis. D2-Net is then further analyzed, improved, and optimized to run on a large dataset of more than 12k image combinations related to history, heritage, and art. The final implementation shows promising results with a precision of 0.96 and a recall of 0.44 on a challenging testing dataset. Future improvements include speed improvement and model training.