Agent-based modelling and simulation (ABMS), whether simple toy models or complex data-driven ones, is regularly applied in various domains to study the system-level patterns arising from individual behaviour and interactions. However, ABMS still faces diverse challenges such as modelling more representative agents or improving computational efficiency. Research shows that machine learning (ML) techniques, when used in ABMS can address such challenges. Yet, the ABMS literature is still marginally leveraging the benefits of ML. One reason is the vastness of the ML domain, which makes it difficult to choose the appropriate ML technique to overcome a specific modelling challenge. This paper aims to bring ML more within reach of the ABMS community. We first conduct a structured literature review to investigate how the ABMS process uses ML techniques. We focus specifically on articles where ML is applied for the structural specifications of models such as agent decision-making and behaviour, rather than just for analysing output data. Given that modelling challenges are mainly linked to the purpose a model aims to serve (e.g., behavioural accuracy is required for predictive models), we frame our analysis within different modelling purposes. Our results show that Reinforcement Learning algorithms may increase the accuracy of behavioural modelling. Moreover, Decision Trees, and Bayesian Networks are common techniques for data pre-processing of agent behaviour. Based on the literature review results, we propose guidelines for purposefully integrating ML in ABMS. We conclude that ML techniques are specifically fit for currently underrepresented modelling purposes of social learning and illustration; they can be used in a transparent and interpretable manner.