Efficient Temporal Action Localization via Vision-Language Modelling: An Empirical Study on the STALE Model's Efficiency and Generalizability in Resource-constrained Environments

Temporal Action Localization (TAL) aims to localize the start and end times of actions in untrimmed videos and classify the corresponding action types. TAL plays an important role in understanding video. Existing TAL approaches heavily rely on deep learning and require large-scale data and expensive training processes. Recent advances in...

End-to-End Learning with Multiple Modalities for System-Optimised Renewables Nowcasting

With the increasing penetration of renewable power sources such as wind and solar, accurate short-term, nowcasting renewable power prediction is becoming increasingly important. This paper investigates the multi-modal (MM) learning and end-to-end (E2E) learning for nowcasting renewable power as an intermediate to energy management systems. MM...

Multi-Modal End-to-End Learning for Real-Time Monitoring of Sustainable Energy Systems

The growth of renewable energy technologies is leading to energy systems that are more reliant than ever on renewables such as Wind and Photovoltaic (PV) power. Despite their benefits in terms of sustainability, their ubiquity poses challenges in maintaining grid stability given their intermittency, emphasising the prediction of power...

Modelling Human Word Learning and Recognition Using Visually Grounded Speech

Many computational models of speech recognition assume that the set of target words is already given. This implies that these models learn to recognise speech in a biologically unrealistic manner, i.e. with prior lexical knowledge and explicit supervision. In contrast, visually grounded speech models learn to recognise speech without prior...

Multimodal learning analytics on sustained attention by measuring ambient noise

In this research, a learner’s sustained attention in the remote learning context will be studied by collecting data from different sensors. By combining the results of these sensors in a multi-modal analytics tool, the estimation of the learner’s sustained attention can hopefully be improved. This research will mainly focus on microphone...

Design of Human Computer Interface for blind children to learn programming languages

As the modern world progressing towards digital technologies, there is an increasing need for school students to learn programming skills so that they fit into the labor market now and in the future. Tangible programming toys and block-based languages are popular among younger children to learn to code. Students in middle and high school in...

Learning state representations for robotic control: Information disentangling and multi-modal learning

Representation learning is a central topic in the field of deep learning. It aims at extracting useful state representations directly from raw data. In deep learning, state representations are usually used for classification or inferences. For example, image embedding that provides similarity metrics can be used for face recognition. Recent...