Deep Neural Networks and Optimization

Abstract

Machine learning has been a computer sciences buzzword for years. The technology has a lot of potential and a huge number of applications that spoke to people with and without knowledge of computer sciences. Image, text and speech recognition, social profiling, computergames, everything seemed possible. Machine learning is not as much in the spotlight now since the rise of blockchain technology, but the field is still relevant and used widely. The field concerns itself with ‘teaching’ computers to make choices or recognize data by repeating experiments or looking at data, and then making changes to itself to achieve better results. Now consider the field of optimization. It is not as well known outside of the mathematics community but incredibly important nonetheless. It has a miriad of uses: flight scheduling, ambulance placement and route planning are a minimal selection of all the problems that Optimization gets used in. Optimization is about solving problems as quickly and optimally as possible, looking at ways to speed up the process or finding the solution with the lowest cost. Both fields have an extensive range of methods and applications but have not been combined a lot before. This report takes a versatile topic within machine learning, namely deep learning and combines it with a classic problem from Optimization. This model was first proposed in 2017, meaning that is a very young line of research. This report will further explore this crossover and look into the performance and possible applications. There will first be an introduction to both fields to make the report understandable for students from either mathematics or computer sciences faculties. The model will then be established and explained in detail. There will then be time devoted to looking into applications of the model and seeing how they perform in experiments. Lastly the results of the experiments will be discussed and their implications.