Human-in-the-Loop Rule Discovery for Micropost Event Detection

Human-in-the-Loop Rule Discovery for Micropost Event Detection

Bhardwaj, Akansha (University of Fribourg)
Yang, J. (TU Delft Web Information Systems)
Cudré-Mauroux, Philippe (University of Fribourg)

2022

Platforms such as Twitter are increasingly being used for real-world event detection. Recent work often leverages event-related keywords for training machine learning based event detection models. These approaches make strong assumptions on the distribution of the relevant microposts containing the keyword – referred to as the expectation – and use it as a posterior regularization parameter during model training. Such approaches are, however, limited by the informativeness of the keywords and by the accuracy of the expectation estimation for keywords. In this work, we introduce a human-in-the-loop approach to jointly discover informative rules for model training while estimating their expectation. Our approach iteratively leverages the crowd to estimate both rule-specific expectation and the disagreement between the crowd and the model in order to discover new rules that are most beneficial for model training. To identify such rules, we introduce a hybrid human-machine workflow that engages human workers in rule discovery through an interactive hypothesis creation and testing interface and leverages automatic methods for suggesting useful rules for human verification. We empirically demonstrate the merits of our approach, on multiple real-world datasets and show that our approach improves the state of the art by a margin of 25.63% in terms of AUC.

Event Detection
Human-in-the-loop AI
Rules in Machine Learning
Interactive Machine Learning

http://resolver.tudelft.nl/uuid:bcf8a1f7-265f-4d1b-8c44-d0da36e35e48

https://doi.org/10.1109/TKDE.2022.3208345

2023-07-26

1558-2191

IEEE Transactions on Knowledge & Data Engineering, 35 (8), 8100-8111

Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Institutional Repository

journal article

© 2022 Akansha Bhardwaj, J. Yang, Philippe Cudré-Mauroux