Artificial Neural Networks (ANNs) are increasingly used for discrete choice analysis, being appreciated in particular for their strong predictive power. However, many choice modellers are critical – and rightfully so – about using ANNs, for the reason that they are hard to diagnose. That is, for analysts it is hard to see whether a trained (estimated) ANN has learned intuitively reasonable relationships, as opposed to spurious, inexplicable or otherwise undesirable ones. As a result, choice modellers often find it difficult to trust an ANN, even if its predictive performance is strong. Inspired by research from the field of computer vision, this paper pioneers a low-cost and easy-to-implement methodology to diagnose ANNs in the context of choice behaviour analysis. The method involves synthesising prototypical examples after having trained the ANN. These prototypical examples expose the fundamental relationships that the ANN has learned. These, in turn, can be evaluated by the analyst to see whether they make sense and are desirable, or not. In this paper we show how to use such prototypical examples in the context of choice data and we discuss practical considerations for successfully diagnosing ANNs. Furthermore, we cross-validate our findings using techniques from traditional discrete choice analysis. Our results suggest that the proposed method helps build trust in well-functioning ANNs, and is able to flag poorly trained ANNs. As such, it helps choice modellers use ANNs for choice behaviour analysis in a more reliable and effective way.@en

Despite having been known for a long time (e.g., McCulloch & Pitts, 1943; Rosenblatt, 1958), and despite having been occasionally used for the analysis of travel behavior since more than a decade ago (Hensher & Ton, 2000; Mohammadian & Miller, 2002), Artificial Neural Networks (ANNs) have only lately become – by a distance – the most prominent and promising Artificial Intelligence (AI) model for the analysis of travel behavior in the context of large, emerging data sources (e.g. Karlaftis & Vlahogianni, 2011; Chen et al., 2016; van Cranenburgh & Alwosheel, 2017). This sharp increase in the popularity of ANNs as a tool for travel behavior analysis has resulted from a range of improvements in ANNs’ capabilities, increases in computational power, and the rapidly increasing size and diversity of data which are at the disposal of choice modelers. This paper aims to help pave the way for further and effective deployment of ANNs for travel behavior analysis. It does so by highlighting and articulating an easily overlooked aspect of the ANN-methodology, which is of crucial importance for its successful use in a travel choice modeling context. More specifically, we study the relation between i) the assumed characteristics of the Data Generating Process (DGP; in this case the assumed model of travel choice behavior or decision rule), and ii) the size of the data that is required for meaningful, reliable travel choice analysis using ANNs. The core idea behind this relation is intuitive: if the DGP is relatively complex – e.g. highly non-linear – then a given ANN needs more data to be able to generate a reliable representation of the DGP, leading to accurate predictions. Despite or perhaps because of this straightforward intuition, choice modelers employing ANNs so far seem to have ignored important results from the AI literature which rigorously define this relation between the complexity of the DGP and resulting data-requirements, in the context of empirical analysis using ANNs. Such concepts as the Universal Approximation Theorem (Cybenko, 1989; Hornik et al., 1989), the notion of Probably Approximately Correct (Valiant, 1984) and the so-called V-C dimension (Vapnik & Chervonenkis, 1971) have helped AI-researchers in various fields of application determine the required size of their dataset as a function of the assumed characteristics of the DGP. This paper aims to introduce these theoretical concepts and notions from the AI-literature to the travel behavior research community, and moreover to translate them in a way that they can be readily used by travel choice modelers. By doing so, we aim to help travel behavior researchers who wish to use ANNs for discrete choice analysis, in the process of selecting data sets or collecting data. To focus our attention, we limit our discussion to the context of two particular travel choice models as DGPs: one is the well-known linear-additive MNL model based on utility maximization premises, which is the workhorse of discrete choice analysis and in many ways the least complex choice model available (Ben-Akiva & Lerman, 1985; Train, 2009). The other is the Random Regret Minimization model (in MNL form), which is one the most used behavioral alternatives to the canonical linear in parameters utility based MNL model (Chorus et al., 2008; van Cranenburgh et al., 2015). The regret function embedded in most RRM models is highly non-linear and includes attributes of all alternatives in the choice set. As such it is a considerably more ‘complex’ choice model than its utility based counterpart, something which for example shows in considerably higher runtimes (Guevara et al., 2016). As such, the comparison between these two models (i.e., DGPs) serves well to highlight how, in the context of discrete choice analysis based on ANNs, data-requirements follow from the characteristics – i.e., level of complexity – of the DGP. Our study thus consists of two parts. In Part 1 we will introduce all relevant concepts, notions and theorems that have been developed in the ANN literature to determine minimum sample sizes as a function of model complexity. We will make sure to present these ideas in a notation and framework that connects directly with conventional modeling practice in the travel behavior research community. In Part 2 we will use these ideas in a concrete example, for illustration purposes and to establish face validity. More specifically, we will show how Random Utility and Random Regret DGPs differ in terms of their data requirements, in the context of model estimation with ANNs. We conclude our study with the derivation and discussion of implications for researchers and practitioners in the field of travel behavior analysis. To get a flavor of the analyses which we performed in Part 2, we here present some first results. Our ‘empirical’ setting is a simple travel mode choice between three alternatives (car, bus, train) based on two attributes (travel time, travel cost). We generate two synthetic datasets containing mode choices: one dataset uses a Random Utility DGP (in MNL-form) and the other one uses a Random Regret DGP (also in MNL-form). Subsequently, we derive – using the introduced concepts from the ANN-literature – the theoretically expected minimum (training) sample size needed to achieve a reliable representation of the DGP by an appropriately specified ANN. We do this for the RUM and RRM DGPs, and show how – in line with expectations – the theoretically required minimum (training) sample size is larger for the latter. Finally, we verify this theoretical result by training ANNs, for each DGP, using increasingly large subsets of the synthetic data. As Figure 1 (RUM) and Figure 2 (RRM) show, the out of sample predictive ability – measured in terms of out of sample LogLikelihood – of the corresponding ANNs is found to increase sharply up to the theoretically identified minimum (training) sample size, after which marginal increments in model fit become notably smaller. This suggests that the theoretically established minimum sample size provides a reasonable indication of practical (training) sample size requirements for the two different DGPs.@en

Artificial Neural Networks (ANNs) are rapidly gaining popularity in transportation research in general and travel demand analysis in particular. While ANNs typically outperform conventional methods in terms of predictive performance, they suffer from limited explainability. That is, it is very difficult to assess whether or not particular predictions made by an ANN are based on intuitively reasonable relationships embedded in the model. As a result, it is difficult for analysts to gain trust in ANNs. In this paper, we show that often-used approaches using perturbation (sensitivity analysis) are ill-suited for gaining an understanding of the inner workings of ANNs. Subsequently, and this is the main contribution of this paper, we introduce to the domain of transportation an alternative method, inspired by recent progress in the field of computer vision. This method is based on a re-conceptualisation of the idea of ‘heat maps’ to explain the predictions of a trained ANN. To create a heat map, a prediction of an ANN is propagated backward in the ANN towards the input variables, using a technique called Layer-wise Relevance Propagation (LRP). The resulting heat map shows the contribution of each input value –for example the travel time of a certain mode– to a given travel mode choice prediction. By doing this, the LRP-based heat map reveals the rationale behind the prediction in a way that is understandable to human analysts. If the rationale makes sense to the analyst, the trust in the prediction, and, by extension, in the trained ANN as a whole, will increase. If the rationale does not make sense, the analyst may choose to adapt or re-train the ANN or decide not to use it at all. We show that by reconceptualising the LRP methodology towards the choice modelling and travel demand analysis contexts, it can be put to effective use in application domains well beyond the field of computer vision, for which it was originally developed.@en

This study develops a novel Artificial Neural Network (ANN) based approach to investigate decision rule heterogeneity amongst travellers. This complements earlier work on decision rule heterogeneity based on Latent Class discrete choice models. We train our ANN to recognise the choice patterns of four distinct decision rules: Random Utility Maximisation, Random Regret Minimisation, Lexicographic, and Random. Next, we apply our trained ANN to classify the respondents from a recent Value-of-Time Stated Choice experiment in terms of their most likely employed decision rule. We cross-validate our findings by comparing our results with those from: (1) single class discrete choice models estimated on subsets of the data, and (2) latent class discrete choice models. The cross-validations provide strong support for the notion that ANNs can be used to identify underlying decision rules in choice data. As such, we believe that ANNs provide a valuable addition to the toolbox of analysts who wish to investigate decision rule heterogeneity. The substantive contribution of this study is that we provide strong empirical evidence for the presence of decision rule heterogeneity amongst travellers.@en

For decades, Discrete Choice Models (DCMs) have been used to describe, understand and predict human choice behaviour in a wide variety of contexts including transportation, healthcare and marketing. The field of discrete choice modelling is firmly rooted in economic theory, and most DCMs are based on the assumption that decision-­makers, when asked to select an alternative among a set of presented alternatives, make deliberate trade-­offs by employing a stable function to assign utility to each alternative, and then select the alternative with the highest utility.@en

Artificial Neural Networks (ANNs) are increasingly used for discrete choice analysis. But, at present, it is unknown what sample size requirements are appropriate when using ANNs in this particular context. This paper fills this knowledge gap: we empirically establish a rule-of-thumb for ANN-based discrete choice analysis based on analyses of synthetic and real data. To investigate the effect of complexity of the data generating process on the minimum required sample size, we conduct extensive Monte Carlo analyses using a series of different model specifications with different levels of model complexity, including RUM and RRM models, with and without random taste parameters. Based on our analyses we advise to use a minimum sample size of fifty times the number of weights in the ANN; it should be noted, that the number of weights is generally much larger than the number of parameters in a discrete choice model. This rule-of-thumb is considerably more conservative than the rule-of-thumb that is most often used in the ANN community, which advises to use at least ten times the number of weights.@en

In the past few decades, Artificial Neural Networks (ANNs) have been used to identify and model choice behavior in wide variety of fields (e.g., Bishop, 1995). To give some examples from the field of travel behavior, ANNs have been applied to model commuter mode choice and car ownership (e.g., Hensher and Ton, 2000; Mohammadian & Miller, 2002). ANNs aim to efficiently recognize patterns in the data, without being explicitly programmed where to look. A key feature of ANNs lies in their capability to approximate any Data Generating Process (DGP), provided that sufficient processing units are available; this feature is known as the Universal Approximation Theorem (Hornik et al., 1989). However, despite the strong pragmatic appeal of ANNs, they have been criticized for being too much ‘data driven’ and ‘theory poor’, in effect presenting the analyst with a black box-model of the DGP. This limitation has hampered their use by discrete choice modelers and travel behavior researchers. In several ways, Discrete Choice Theory (DCT) – which is the dominant approach in the travel behavior research community to model choice behavior – is the mirror image of ANN. In contrast to ANN, DCT presupposes a particular decision rule (DGP) and estimates a model based on that rule on choice data. In addition to the classical linear-in-parameters utility maximization rule, several alternative decision rules have been proposed more recently; see Leong & Hensher (2012) and Chorus (2014) for overviews. Clear advantages of the DCT approach are that it allows for the extraction of deep behavioral insights from choice data (McFadden, 2001) and rigorous conclusions concerning welfare effects of policies (Small & Rosen, 1981). However, despite recent work which allows for a more flexible treatment of decision rules in discrete choice models (e.g., Hess et al., 2012; Van Cranenburgh et al., 2015), DCT can still be considered a relatively rigid approach to model choice data, compared to ANN. Our paper sets out to explore in more depth the advantages and disadvantages (relative to DCT) of using ANN as a framework to analyze choice data. We focus in particular on ANN’s ability to learn which decision rule best represents the DGP in a discrete choice context. To this end, we perform three types of analyses: We analytically explore to what extent ANN’s Universal Approximation Theorem applies to a discrete choice context, with particular emphasis on the role of the size of the training data set and the number of nodes of the ANN (Vapnik & Chervonenkis, 2015). Using synthetic datasets, we explore to what extent ANNs are able to explain choice behavior when the decision rule underlying the DGP is known to the analyst. We focus on standard Random Utility Maximization and Random Regret Minimization (Chorus, 2010) rules. Particular attention is paid to the role of error term variance and the size of the training data set. Using real datasets, we explore to what extent ANNs are able to explain choice behavior when the decision rule underlying the DGP is unknown to the analyst (as is usually the case). A range of comparisons is made between different types of ANNs and DCT-based choice models. With reference to the above mentioned types of analyses, our preliminary results can be summarized as follows: The Universal Approximation Theorem applies to a discrete choice context. Out-of-sample analysis shows that the performance of the ANN improves when the size of the ANN training data set increases, and approaches the performance of a discrete choice model whose decision rule matches the decision rule of the DGP when the data training data set is sufficiently large, see the Figure 1 & 2 at supplementary files. More specifically, the fitted power functions (of the form y = a.x^b + c ) reveal their asymptotic behavior, which is in line with theoretical expectations. Furthermore, the parameters of the fitted power functions show that ANNs converge faster in case the DGP is RUM, as compared to when the DGP is RRM. These conclusions pave the way for a more informed debate about the potential and limitations of using ANNs to explain and predict choice data; they also provide clues as to how ANN and DCT may be combined to capitalize on their respective strengths.@en