Towards an evolutionary-based approach for natural language processing

Towards an evolutionary-based approach for natural language processing

Manzoni, Luca (University of Trieste)
Jakobovic, Domagoj (University of Zagreb)
Mariot, L. (TU Delft Cyber Security)
Picek, S. (TU Delft Cyber Security)
Castelli, Mauro (New University of Lisbon)

2020

Tasks related to Natural Language Processing (NLP) have recently been the focus of a large research endeavor by the machine learning community. The increased interest in this area is mainly due to the success of deep learning methods. Genetic Programming (GP), however, was not under the spotlight with respect to NLP tasks. Here, we propose a first proof-of-concept that combines GP with the well established NLP tool word2vec for the next word prediction task. The main idea is that, once words have been moved into a vector space, traditional GP operators can successfully work on vectors, thus producing meaningful words as the output. To assess the suitability of this approach, we perform an experimental evaluation on a set of existing newspaper headlines. Individuals resulting from this (pre-)training phase can be employed as the initial population in other NLP tasks, like sentence generation, which will be the focus of future investigations, possibly employing adversarial co-evolutionary approaches.

Genetic programming
Natural language processing
Next word prediction

http://resolver.tudelft.nl/uuid:59110064-7964-4b1a-9dd3-eb19a3387007

https://doi.org/10.1145/3377930.3390248

Association for Computing Machinery (ACM), New York

978-1-4503-7128-5

GECCO 2020: Proceedings of the 2020 Genetic and Evolutionary Computation Conference

2020 Genetic and Evolutionary Computation Conference, GECCO 2020, 2020-07-08 → 2020-07-12, Cancun, Mexico

Accepted author manuscript

Institutional Repository

conference paper

© 2020 Luca Manzoni, Domagoj Jakobovic, L. Mariot, S. Picek, Mauro Castelli