Towards Improving Retrieval for the Verification of Natural Numerical Claims
D. Prabhu (TU Delft - Electrical Engineering, Mathematics and Computer Science)
Avishek Anand – Mentor (Leibniz Universität)
V. Viswanathan – Graduation committee member (TU Delft - Web Information Systems)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
Verification of numerical claims is critical as they tend to be more believable despite being fake and have previously demonstrated the potential to cause catastrophic impacts on society. While there currently exist several automatic fact verification pipelines, only a handful focus on natural numerical claims. A typical human fact-checker first retrieves relevant evidence addressing the different numerical aspects of the claim and then reasons about them to predict the veracity of the claim. Hence, the retrieval thought process of a human fact-checker is a crucial skill that forms the foundation of the verification process. Emulating a real-world setting is essential to aid in the development of automated methods that encompass such skills. Hence, we introduce QuanTemp++: a dataset consisting of natural numerical claims, an open domain corpus, and the corresponding evidence relevance and veracity labels. Given this dataset, we also aim to characterize the retrieval performance of key query planning paradigms, especially those of decomposition as they have shown promising results in other tasks. Finally, we observe their effect on the outcome of the verification pipeline and draw insights.