Children often struggle to retrieve age-appropriate information when seeking information online. One big reason for this is that their search queries are short, misspelled, or vague. As a solution to this problem, previous research investigated query reformulation, where the input query is manipulated in a way that the retrieved web results are more child-appropriate. This was measured by various metrics and scores, such as readability and content safety of the retrieved web search results. The problem with present query reformulation strategies, however, is that each tackles this problem from one perspective, missing out on the potential benefits of other perspectives. For instance, expanding the query with the “for kids” cue has shown to be a good way to target a specific audience and helps retrieve more child-appropriate content; however, on top of this considering substitutes for uncommon words with simpler synonyms might further enhance the child-appropriateness of the retrieved results as it tackles the reformulation from a different perspective than “for kids” audience cue expansion.

Motivated by this, we propose a multi-step query reformulation strategy that combines multiple reformulation strategies and applies them to the given input child query in a multi-step manner using a Large Language Model (LLM). We use LLM to apply the reformulation strategies to the input query in a chain-of-calls (where each call is prompted to apply a different reformulation strategy). This proposed method captures the perspective of multiple reformulation strategies, rather than a single one, unlike existing reformulation strategies. The results of our experiments, which include a baseline comparison (of the retrieved search results from the reformulated query against the original query) and an ablation study, provide insight into the performance of our strategy.

With this work, we aim to demonstrate the potential of combining multiple reformulation strategies and their impact on improving the readability and content safety of retrieved web search results when applied to children’s search queries. Our findings reveal a significant improvement in the readability of retrieved results after using the proposed reformulation method. Ultimately, this work contributes to the development of next-generation, child-centric search systems that deliver clearer, safer results for children. ... Read more

Large language models (LLMs) are increasingly used by children, yet their responses are often not tailored to young users’ reading levels or cognitive development. Previous attempts to improve content readability through prompt modifications such as adding "for kids" have shown limited success. This project explores an alternative strategy: persona-based prompting. Rather than directly specifying the target audience, we instruct the LLM to role-play a teacher as a familiar figure to children. Using real child authored queries, we evaluate whether this role-based approach leads to more readable and comprehensible responses across different LLMs. Readability and comprehension were measured using established metrics, including Flesch-Kincaid formulas and Age of Acquisition data. Our results show that for 4 out of the four evaluated models, persona based prompting consistently produces responses that are more readable and accessible across all readability metrics and some comprehensibility metrics compared to standard or intended-user prompting. This finding suggests that persona-based prompting is a promising strategy for improving the suitability of LLM outputs for young audiences. ... Read more

Children increasingly rely on web search engines to support their learning and exploration. However, conventional search systems are not optimised for their developmental stage, often returning information that is linguistically complex or educationally irrelevant. The retrieved results are often written at a higher reading grade level than children can easily comprehend, resulting in poor engagement and learning outcomes. This research investigates whether substituting simpler vocabulary into search queries can improve the educational relevance and readability of retrieved web content. We develop a reformulation pipeline consisting of: (1) a rule-based method that substitutes key query terms with synonyms ranked by Age of Acquisition (AoA) scores, and (2) a computational intelligence approach that uses a Large Language Model (LLM) to generate child-friendly rephrasings. The results retrieved from the queries are evaluated across two dimensions: readability and educational relevance. Our results show that rule-based reformulations improve readability, but retrieved results stayed consistent in terms of educational relevance. LLM-based reformulations enhance educational relevance; however, they don’t improve readability. This trade-off highlights the complementary strengths of both methods and underlines the potential of direct query reformulation to make web search more accessible and educationally effective for children. ... Read more

The curse of dimensionality poses a fundamental challenge in autonomous negotiations: as the number of issues and their interdependencies increase, exhaustive evaluation of the outcome space quickly becomes infeasible. This thesis addresses this problem by introducing a surrogate-based method that approximates uncertain hypercubic constraint-based utility functions with quadratic polynomials. An autonomous negotiation agent can then search for high-utility outcomes in this surrogate model. The research objective was to investigate how efficiently an autonomous negotiation agent can identify high-utility bids with this approach, and how this approach compares to linear approximations and established benchmark agents.

The main contributions of this thesis are threefold. First, it introduces a probabilistic complexity measure for these hypercubic functions, capturing how parameters such as dimensionality, constraint width, the number of constraints, and the number of issues interact to shape the function's complexity. Second, it develops a novel agent that leverages a regression model with quadratic basis functions to construct a surrogate model of a hypercubic constraint-based utility function. Third, it evaluates the agent through extensive experiments, demonstrating how performance scales with complexity. Following the steps outlined in this thesis, the performance of surrogate models can be directly compared.

The results demonstrate that the surrogate-based method is a promising approach, as the agent constructed in this thesis outperforms the agents from the 2014 Automated Negotiating Agent Competition which used similar scenarios as those considered in this thesis. These agents all have in common that they directly search the utility function as opposed to a surrogate model of it. Furthermore, the results indicate that simple basis functions, such as quadratic ones, enable the agent to reach the global maximum of its utility function in low-complexity hypercubic cases, with performance scaling reasonably well up to medium complexity. Beyond this point, however, performance deteriorates rapidly, clearly signaling the need for more expressive surrogate models. ... Read more

In the field of reinforcement learning (RL), effectively leveraging behavior-agnostic data to train and evaluate policies without explicit knowledge of the behavior policies that generated the data is a significant challenge. This research investigates the impact of state visitation mismatch methods on the performance of on-policy RL methods, an area crucial for improving policy performance in real-world applications where behavior policies are often unknown. Specifically, we compare the convergence speed and performance of Q-learning when initialized with Q-values learned through the Distribution Correction Estimation (DICE) method versus traditional random initialization. By generating datasets representing behavior and target policies, we employ the DICE estimator to initialize Q-values, and subsequently run Q-learning for both DICE-initialized and randomly-initialized scenarios. Our results demonstrate that initializing Q-learning with DICE Q-values enhances convergence speed, leading to faster attainment of near-optimal policies. This study provides valuable insights into the effectiveness of state visitation mismatch methods in improving the efficiency and performance of on-policy RL algorithms, contributing to the development of more robust RL applications in behavior-agnostic settings. ... Read more

In offline reinforcement learning, deriving a policy from a pre-collected set of experiences is challenging due to the limited sample size and the mismatched state-action distribution between the target policy and the behavioral policy that generated the data. Learning a dynamic model of the environment can improve the sample efficiency of the algorithm, but this mismatch can lead to the generation of suboptimal experiences. We propose SimuDICE, an algorithm that enhances the sampling of imaginary experiences using Dual stationary DIstribution Correction (DICE), and iteratively improves the DICE estimations with synthetically generated experiences. SimuDICE addresses the objective mismatch issue by iteratively updating both the world model and the DICE estimator, aligning the model's training objective (imitating the environment) with its usage objective (policy improvement). We show that SimuDICE requires less pre-collected data and fewer simulated experiences to achieve comparable results to other algorithms while having greater robustness to lower data quality. ... Read more

Behavior-agnostic reinforcement learning is a rapidly expanding research area focusing on developing algorithms capable of learning effective policies without explicit knowledge of the environment's dynamics or specific behavior policies. It proposes robust techniques to perform off-policy evaluation, namely Distribution Correction Estimation (DICE) methods, in the context of infinite horizon Markov Decision Processes (MDPs). This research paper investigates the impact of the initial start distribution mismatch on the accuracy of DICE estimators in behavior-agnostic reinforcement learning. To achieve this, seven systematic initial start distributions were created and utilized to calculate the initial start distribution mismatch via Kullback–Leibler (KL) divergence. Furthermore, off-policy evaluation performance was assessed using DICE estimators, with Mean Squared Error (MSE) comparisons against ground truth values. The study reveals that, based on the conducted experiments, the initial start distribution mismatch does not have a clear influence on the performance of the DICE estimators. Therefore, future research is required to increase the scope of the experiments and address some of the limitations of this study to accurately assess the impact of the initial start distribution mismatch on off-policy evaluation using DICE methods. This paper underscores the complexity of the initial start distribution choice in behavior-agnostic reinforcement learning, calling for further research to effectively evaluate its impact across diverse environments and measures. Additionally, exploring the relation between the initial start distribution and policies could provide deeper insights and further refine the understanding of their influence on DICE estimators. ... Read more

Off-policy evaluation has some key problems with one of them being the “curse of horizon”. With recent breakthroughs [1] [2], new estimators have emerged that utilise importance sampling of the individual state-action pairs and reward rather than over the whole trajectory. With the difference between behaviour and target policy, the state-visitation mismatch occurs. This paper is interested in answering the question how the degree of state-visitation mismatch affects the overall target policy performance. The approach is to calculate the state-visitation mismatch with the KL divergence, which consists of the state-visitation distribution of the behaviour policy and the distribution correction ratio of the DICE estimator. The state-visitation mismatch can be quantified in way. Furthermore, the effect on the target policy performance is quantified by the MSE between the estimated, empirical cumulative reward and the estimated reward by the DICE estimator. By analysing the KL divergence and MSE value, one may argue that the state-visitation mismatch does impact the performance of the target policy but further research needs to be conducted. ... Read more

This paper addresses the issue of double-dipping in off-policy evaluation (OPE) in behaviour-agnostic reinforcement learning, where the same dataset is used for both training and estimation, leading to overfitting and inflated performance metrics especially for variance. We introduce SplitDICE, which incorporates sample-splitting and cross-fitting techniques to mitigate double-dipping effects in the DICE family of estimators. Focusing specifically on 2-fold and 5-fold cross-fitting strategies, the original off-policy dataset is partitioned with random-split to get separate training and evaluation datasets. Experimental results demonstrate that SplitDICE, particularly with 5-fold cross-fitting, significantly reduces error, bias, and variance compared to naive DICE implementations, providing a more doubly-robust solution for behavior-agnostic OPE. ... Read more

In order to develop artificial agents that can understand social interactions at a near-human level, it is required that these agents develop an artificial Theory of Mind; the ability to infer the mental state of others. However, developing this artificial Theory of Mind is a highly difficult process. This is because Theory of Mind is an ambiguous and multifaceted concept, having several mechanisms associated with it, and being tested using many different tasks. In this thesis, we formalize what mechanisms constitute Theory of Mind, and establish how we can represent these mechanisms using artificial intelligence. Furthermore, we evaluate whether current artificial Theory of Mind models are able to reason effectively about these mechanisms. This is done by creating Theory of Mind tasks for artificial models, evaluating their effectiveness, and allowing us to provide recommendations for the development of future artificial Theory of Mind models. ... Read more

Spectral Monte-Carlo methods are powerful physically-based techniques for simulating wavelength-dependent phenomena such as dispersion. However, compared to tristimulus rendering, they involve sampling the spectral domain, which adds substantial overhead, requiring significantly more samples for noise-free, realistic-looking renders. Thereby, we propose a simple approach to efficiently sample emitters. We precompute a simple 2-dimensional data structure using spectral power distributions of scene emitters. We use it to model a probability distribution function to sample an emitter that yields high path throughput at every intersection when using Next Event Estimation. Our method handles various geometries and spectral distributions of scene emitters, improves convergence, and reduces noise with negligible overhead. ... Read more

Direct lighting calculation is an essential part of photorealistic rendering. Standard importance sampling techniques converge slowly in scenes where a light source is only visible through small openings as visibility is not considered. This problem is often addressed by manually placing light portals, marking the openings to the light. However, existing portal sampling techniques are not suitable for area lights since the portal is often times larger than the light itself. We present a novel portal sampling technique inspired by shadow volumes, which considers the light geometry to efficiently choose the optimal sampling strategy, depending on the location of the shading point. Our technique is unbiased, robust, applicable to many scenes, and easy to integrate into an existing renderer. ... Read more

Path tracing is a well-known light transport algorithm used to render photo-realistic images. However, it is an expensive algorithm with an active area of research for improving its efficiency. In our work, we present a method to measure and visualize the regions of high computational cost for unidirectional path tracers. We have defined metrics to estimate the cost per pixel that can be visualized as a two-dimensional(2D) image. To show its usefulness, we present how our method can visualize the effect of changing material properties, object placement and other scene variables on computational cost. With this insight, the user can make clever choices to improve computational time. ... Read more

Indirect illumination is an essential part of realistic computer-generated imagery. However, accurate calculation of indirect illumination comes at high compute costs. To this end, we replace lengthy indirect illumination paths by employing an ambient light cache based on photon mapping principles. By only performing cache queries after a certain path length has been reached, we show that significantly fewer photons than traditional photon mapping techniques are sufficient and that an inaccurate data structure can be used to store them. Despite these simplifications, our method generally outperforms unidirectional path tracing while adding little time overhead. ... Read more

Spectral Monte-Carlo rendering can simulate advanced light phenomena (e.g., dispersion, caustics, or iridescence), but require significantly more samples compared to trichromatic rendering to obtain noise-free images. Therefore, its progressive variant typically exhibits an extreme amount of chromatic noise in early renders. To that end, we propose a two-stage progressive approach. We initially restrict the original wavelength distribution, then slowly relax it. In the process of relaxing the range of wavelengths, all wavelengths that are outside of that restricted range will be propagated. Thereby, we lower variance and increase the perception of these early renders with little overhead. ... Read more

Collaboration in teams composed of both humans and automations has an interdependent nature, which demands calibrated trust among all the teammembers. For building suitable autonomous teammates, we need to study how trust and trustworthiness function in such teams. In particular, automations occasionally fail to do their job, which leads to a decrease in human’s trust. However, research has given contradictory statements about the effects of such a reduction of trust on the human’s trustworthiness, i.e. human’s characteristics that make them more or less reliable to the automation. As such, this study investigates how automation failure in a human-automation teamwork scenario affects the human’s trust in the automation and human’s trustworthiness towards the automation. We present a between-subjects controlled experiment in which the participants perform a simulated task in a 2D grid-world, collaborating with an automation in a “moving-out” scenario. During the experiment, we measure the participants’ trust and trustworthiness regarding the automation both subjectively and objectively. Our results show that automation failure negatively affects the human’s trustworthiness, as well as their trust in and liking of the automation. Learning the effects of automation failure in trust and trustworthiness can contribute to a better understanding of the nature and dynamics of trust in these teams, foreseeing undesirable consequences and improving human-automation teamwork. ... Read more

Adversarial attacks pose a risk to machine learning (ML)-based network intrusion detection systems (NIDS). In this manner, it is of great significance to explore to what degree these methods can be viably utilized by potential adversaries. The majority of adversarial techniques are designed for unconstrained domains such as the image recognition domain, where these methods apply alterations to the pixels in a picture. Therefore, the applicability of these techniques to the NIDS domain is very limited. Related work on adversarial techniques for NIDS generally considers feature-space techniques, which cannot be applied in a practical situation since only the extracted network traffic features are modified and not the actual network traffic. To solve these limitations, a traffic-space approach for creating adversarial examples for evading ML-based NIDS is proposed and assessed with several classification models. The proposed constrained adversarial crafting method is based on the Iterative Fast Gradient Sign Method (IFGSM) and is called the Constrained Iterative Fast Gradient Sign Method (CIFGSM). A constraint set is added as a penalty term to the loss function of the optimization to ensure that the adversarial values remain within the valid space. Additionally, an L2 regularization term is used to minimize the distance between the original and adversarial network traffic samples. The proposed method is evaluated and shown to be an effective way for generating realistic and practical adversarial evasion packets. To achieve this, network packet components and their characteristics are defined as a constraint set which can be used for the optimization task and a custom adversarial loss function is created that encapsulates the different elements of this optimization problem. Furthermore, multiple models are evaluated to test the transferability of this method. Conclusively, the proposed method is evaluated in a realistic scenario, where adversarial packet captures are crafted and examined. Where other state-of-the art works only modify the network traffic features in feature-space or on a connection level only and do not apply their method in a real world scenario, this work modifies the packet captures on a per-packet level which is subsequently used to evaluate flow based classification models. ... Read more

This research expands past research on implementing the TDNN-OPGRU network for Automatic Phoneme Recognition on Dutch speech by implementing and testing the TDNN-OPGRU network on Mandarin speech. The goal of this research is to investigate the performance of the TDNN-OPGRU architecture when decoding phonemes in Mandarin prepared and spontaneous speech. The difference in Phoneme Error Rate between prepared and spontaneous speech is being determined, and the effect that tones have on the PER is being investigated since Mandarin is a tonal language. The results are that a substantial amount of the PER comes from substitutions that are made where only the tone is incorrectly determined. However, tone does not appear to have an impact on the difference in PER between spontaneous and prepared speech since it is responsible for an similar amount of the substitutions in both types of speech. The inclusion of tone also causes the error rate of the TDNN-OPGRU architecture on base phonemes to increase. ... Read more

This research studies the Projected Bidirectional Long Short-Term Memory Time Delayed Neural Network (TDNN-BLSTM) model for English phoneme recognition. It contributes to the field of phoneme recognition by analyzing the performance of the TDNN-BLSTM model based on the TIMIT corpus and the Buckeye corpus, respectively containing read speech and spontaneous speech. The TIMIT corpus can be used as benchmark to make comparisons between architectures. The Buckeye corpus is used to better understand how the TDNN-BLSTM architecture would perform on recorded informal conversations.
Parameter values are taken from literature and are optimized.
Using the improved parameters, the results show Phoneme Error Rates (PER) for read speech to be 31.78% and for spontaneous speech to be 54.03%. Related work shows PER scores for read speech to be 14.9% and for spontaneous speech to be 23.4%.
This indicates that the TDNN-BLSTM architecture does not perform as well as other acoustic models for both spontaneous and read speech. ... Read more

A limitation of current ASR systems is the so-called out-of-vocabulary words. The solution to overcome this limitation is to use APR systems. Previous research on Dutch APR systems identified Time Delayed Bidirectional Long-Short Term Memory Neural Network (TDNN-BLSTM) as one of best performing state-of-the-art NN architecture for PR. The goal of this research is to evaluate the performance of the TDNN-BLSTM architecture for phoneme recognition on Mandarin read and spontaneous speech, analyze the differences in performance for the two speech styles as well as compare the results with previous research on Dutch PR.

To achieve this goal 4 different NN models of the TDNN-BLSTM architecture were built and trained on Mandarin read and spontaneous speech. The test results of the NN models were used to calculate the phoneme error rate (PER), decomposed PER, and the contribution of individual phonemes to the overall PER. Based on these findings, conclusions are formulated regarding the impact of different languages, speech styles, and the architectural changes on the performance of the TDNN-BLSTM architecture. ... Read more