Print Email Facebook Twitter Nessy Title Nessy: A Neuro-Symbolic System for Label Noise Reduction Author Smirnova, Alisa (University of Fribourg) Yang, J. (TU Delft Web Information Systems) Yang, Dingqi (University of Macau) Cudre-Mauroux, Philippe (University of Fribourg) Date 2022 Abstract Noisy labels represent one of the key issues in supervised machine learning. Existing work for label noise reduction mainly takes a probabilistic approach that infers true labels from data distributions in low-level feature spaces. Such an approach is not only limited by its capability to learn high-quality data representations, but also by the low predictive power of data distributions in inferring true classes. To address those problems, we introduce Nessy, a neuro-symbolic system that integrates deep probabilistic modeling and symbolic knowledge for label noise reduction. Our deep probabilistic model infers the true classes of data instances with noisy labels by exploiting data distributions in an underlying latent feature representation space. For data instances where inference is not reliable enough, Nessy extracts symbolic rules and ranks them according to several utility metrics. Top-ranking rules are injected into the deep probabilistic model via expectation regularization, i.e., via a posterior regularization term constraining the class distribution in the objective function. In a real deployment over multiple relation extraction tasks, we demonstrate that Nessy is able to significantly improve the state of the art, by 7% accuracy and 10.7% AUC on average. Subject Data miningData modelsDeep learningDeep probabilistic modeldistant supervisionFeature extractionneuro-symbolic systemsNoise measurementnoise reductionProbabilistic logicrelation extractionTraining data To reference this document use: http://resolver.tudelft.nl/uuid:49600693-899c-426a-9ce3-46da8b80f08b DOI https://doi.org/10.1109/TKDE.2022.3199570 Embargo date 2023-07-24 ISSN 1041-4347 Source IEEE Transactions on Knowledge & Data Engineering, 35 (8), 8300-8311 Bibliographical note Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public. Part of collection Institutional Repository Document type journal article Rights © 2022 Alisa Smirnova, J. Yang, Dingqi Yang, Philippe Cudre-Mauroux Files PDF Nessy_A_Neuro_Symbolic_Sy ... uction.pdf 2.83 MB Close viewer /islandora/object/uuid:49600693-899c-426a-9ce3-46da8b80f08b/datastream/OBJ/view