Assisting Experts in Image Description for Visually Impaired People

Assisting Experts in Image Description for Visually Impaired People

Vollebregt, Frank (TU Delft Electrical Engineering, Mathematics and Computer Science)

Lofi, C. (mentor)
Cockx, J.G.H. (graduation committee)

Delft University of Technology

Computer Science | Software Technology

2023-02-23

There are an estimated 253 million blind and visually impaired people in the world. To grant them access to text publications that contain images, experts are employed to write image descriptions. There is both a societal and a legislative pressure to supply image descriptions to all new and archived publications within a number of years, yet the number of available experts is limited. The image description task for images in textual context is complex, since a fitting description combines salient elements from both the image as well as the context into a description, which also differs depending on the publisher and published medium.

Because of this complex nature, current automated systems are unable to reliably produce desirable results. Instead, this thesis focuses on developing software to assist the experts in their general image description task in order to improve their efficiency. Specifically, we use existing, commercially available automated tools to generate alternative representations of the data. To analyse the system, we develop a user interface to present all of the available data and design an experiment with a small group of experts to investigate the system's applicability and perceived usefulness. We find that such a system has great potential to assist the experts, but that it might be desirable to focus on a solution aimed towards a smaller subset of publications, so that domain-specific information sources can be exploited to improve the information quality.

visually impaired people
Image description
Expert tool
Assistive technology
accessibility

http://resolver.tudelft.nl/uuid:400c1593-48f6-496d-afd2-675af653bfaf

https://github.com/frankvollebregt/imagedescriptions Github repository with the code used

Student theses

master thesis

© 2023 Frank Vollebregt