Rank-Biased Overlap (RBO) is a widely used metric for comparing ranked lists, due to its ability to handle incomplete and non-conjoint rankings while emphasizing top-ranked items. However, traditional RBO only considers the identity of ranked items, ignoring any associated relevance values. In many real-world applications, different systems may retrieve non-overlapping documents with similar informational value. This paper proposes an extension of RBO that incorporates graded relevance scores, enabling the comparison of rankings based on the information they convey rather than shared items alone.

Two relevance-aware variants for redefining RBO are proposed using cumulative gain.
These variants are evaluated and analyzed using TREC ad hoc and simulated data, comparing them with each other and against standard RBO. The results demonstrate that the new RBO variants provide a more informative similarity measure when comparing rankings with differing identities but similar relevance patterns.

Europeana is a digital library of Europe's cultural heritage, housing a large corpus of data representing artworks, literature, historical locations and many culturally significant items. Europeana currently relies of traditional text-matching retrieval, such as BM25, to facilitate their search and discovery across millions of multilingual metadata-based records. However, these models are not capable of semantic understanding and require additional treatments to facilitate multilingual retrieval which costs Europeana resources, these treatments entail translating queries and data from other language into English and enriching content by adding entities from linked open data. Europeana's current methodology is ultimately limited in its ability to provide semantically relevant multilingual search results.

This thesis investigates the application of Neural Information Retrieval (NIR) to enhance Europeana's search capabilities. This investigation aims to assess the impact of NIR on multilingual retrieval and retrieval performance while also determining the value of existing translation and enrichment processes. To support this investigation, we contribute by developing a structured and preprocessed dataset specifically for NIR, as no such dataset previously existed for NIR. We conduct an extensive evaluation of NIR models, analyzing the impact of fine-tuning, query treatments, and document treatments on retrieval quality. Additionally, we assess the computational requirements, scalability, and practicality of deploying NIR, identifying trade-offs in retrieval efficiency and resource consumption, to provide an idea of an infrastructure Europeana would need to implement NIR.

This research required meticulous planning across all stages—from data collection and formatting to model training and evaluation—since applying NIR at this scale for metadata search is new for Europeana. Therefore, research not only provides insights into the viability of NIR as a replacement or enhancement to Europeana's existing search system but also lays the foundation for future advancements in multilingual retrieval for Europeana.

Through this thesis, we found that NIR models can offer promising improvements in multilingual retrieval and semantic search, reducing reliance on exact term matching. Our analysis suggests that not all of Europeana’s current preprocessing treatments are necessary for NIR models, as they inherently capture cross-lingual relationships more effectively than BM25, though the benefits vary depending on the model and configuration used. Overall, we recommend that a hybrid retrieval system that leverages both lexical and neural approaches may be the most practical solution for Europeana and warrants further exploration.

The integration of NIR presents several challenges, particularly in terms of infrastructure and evaluation. NIR models are sensitive to changes in document structure and content, requiring careful consideration of indexing and fine-training. Furthermore, while these models improve semantic search, they may struggle with entity-based queries, where BM25’s exact matching approach remains valuable.

A major limitation of this study was the absence of explicit relevance judgements in our dataset, which constrained our ability to make definitive conclusions about retrieval effectiveness. Future work should prioritize the development of a comprehensive evaluation framework, incorporating expert and user-based relevance assessments, to enable a more robust analysis of NIR’s impact.

Visual impairment affects over 2.2 billion individuals globally, emphasizing the critical need for effective assistive technologies. This work focuses on developing a video captioning model explicitly tailored for visually impaired users, leveraging advancements in deep learning techniques. Video captioning involves converting video frames into textual descriptions, effectively bridging the domains of computer vision (CV) and natural language processing (NLP). We surveyed young visually impaired individuals from the Visio organization, who provided key insights into the design of our model.
We enhance the existing S2VT model by modifying its temporal attention mechanism to improve the recognition of visual surroundings, addressing the unique challenges visually impaired individuals face.
This research explores critical questions surrounding the model's sensitivity to actions, the readability of generated captions, and methods for latency reduction. To evaluate the model's effectiveness, we implement readability metrics—an approach not previously utilized in video captioning assessments. Our findings contribute to enhancing accessibility and independence for visually impaired individuals through advanced video captioning solutions.

Rankings are more present in our daily lives than most people realize. Whether you are browsing Netflix and getting movies or shows based on your previous likes or dislikes, or you want to compare search engine results. To use rankings in the field of Computer Science a rank similarity is needed. Rank-Biased Overlap is one of those. It is top-weighted, can be used on uneven rankings, and when only a part of the ranking is known. A well-known problem in rank similarity measures is ties. There have been some ways of dealing with ties proposed since RBO was introduced. These ways have been shown to be promising but they only relate to the seen part. The unseen part of rankings is still a new concept with little research done about it. This paper aims to change that a bit. First, a full explanation is given of the three variations of dealing with ties. Then using these variants we show how the assumption that no ties exist in the unseen part affects these variants. Also, the current extrapolation method is researched as there is also a big influence of the above-mentioned assumption. We then use simulated data to give a clear data visualization to show how the theory relates to practice. We have tried to be clear and concise with our explanations and data visualizations so future researchers can use this paper to improve and progress RBO in the world of rank similarity measures.

Rank-biased Overlap (RBO) is a measure that is used to compare two rankings against each other mathematically using a hyperparameter for persistence, p, to define the importance of items higher up in the rankings. This is able to follow the properties of incompleteness, indefiniteness, and top-heaviness for its results, making it a flexible option for rank similarity. In traditional RBO, the intersection of the items in each of the rankings is weighted by the persistence to reach the final value for RBO as it tends towards infinity. RBO has several assumptions, such as on what happens when rankings are tied, having an infinitely long ranking, and a degree of conjointness between the rankings. In this paper, two new variations are derived on the aspects of having the rankings be fully conjoint, as well as the aspect of having a known finite domain for the rankings. These are described through the equations of RBOc, for fully conjoint rankings, and RBOf , for rankings within a known finite domain. While RBOc tends to be slightly larger and RBOf tends to be smaller when compared to traditional RBO, both can be more fitting depending on the greater context of their use cases.

Rankings play a crucial role in various contexts but often exhibit incompleteness, top-weightedness, and indefiniteness. Comparing rankings can reveal underlying similarities, yet traditional correlation coefficients like Kendall's tau do not adequately address these complexities. Rank-Biased Overlap (RBO) addresses these challenges by accommodating differences in rank length, appropriately weighting ranks, and minimizing data assumptions. This paper investigates the average Rank-Biased Overlap (RBO) between independent rankings, addressing the need for clearly indicated reference values similar to those of correlation coefficients. Our study explores how the expected RBO changes with varying p-parameters, prefix lengths, and degrees of conjointness between domains.
To facilitate this analysis, an algorithm is developed that performs extensive simulations across different values of p, list and domain sizes. By analyzing the simulation results, trends are provided in the average RBO between independent rankings based on these varying parameters and establish relevant reference values. This study focuses on scenarios where prefixes are of the same length and there are no ties in the rankings.

As a point estimate of the similarity score between two possibly indefinite rankings, extrapolated rank-biased overlap (RBOEXT) uses the assumption that the agreement observed at the last evaluation depth continues indefinitely across the unseen tails of the two lists. This assumption does not account for any patterns that occur in the visible prefixes, imposing a strict restriction on the extrapolation. In an effort to improve the accuracy of RBOEXT, three reformulations with a relaxed theoretical basis are proposed in this paper: one continually re-uses the agreement from the previous depth while the other two rely on regression to fit a function on the seen agreements. Using synthetic data, the performance of these new extrapolation methods is compared to the original's in terms of closeness to the true RBO score as well as the average distance between assumed and actual agreement in the rankings' unseen tails. Overall, an impactful difference is observed in the estimates of agreement generated by the four approaches: as the trends from the visible prefixes are barely captured by the simpler techniques or closely-reproduced by the more flexible ones, the trade-off between under- and overfitting becomes increasingly relevant. The results thus indicate a need for some middle-ground to be established such that it factors in the observed patterns while also generalizing well for the tails.

Reducing the similarity of two ranked lists to a single value proves to be useful in various fields of research and industry, such as Information Retrieval and Recommender Systems, leading to the introduction of several similarity measures. One such measure is Rank-Biased Overlap and its variants, possessing qualities such as the ability to handle incomplete rankings, non-conjoint ranking domains and the graceful evaluation of ranking pairs with tied items.
Comparing the performance of similarity measures, or comparing variants of a single measure, requires the presence of ranking data. In certain cases, generating synthetic ranking data may be a more viable option than using real data. However, a review of existing literature reveals a lack of parametrisable synthetic ranking algorithms. This paper introduces a novel method to generate a pair of rankings where one can tailor the conjointness of ranking domains, influence the ranking overlap as a function of depth and tune the presence of tie groups in a probabilistic manner. The paper demonstrates the output of the algorithm when varying the input parameters, verifying the methods performance empirically and statistically.

Learning to Rank is the application of Machine Learning in order to create and optimize ranking functions. Most Learning to Rank methods follow a listwise approach and optimize a listwise loss function which closely resembles the same metric used in the evaluation. Popular listwise loss functions such as nDCG, AP and nRBP do not have consistent bounds across topics and do not account for instance-difficulty. As a result, the loss score does not solely reflect the performance of the model but also depends on the instance properties. During training, each instance is assumed to be equally informative, while in reality, this informativeness might depend on the difficulty of the instance. In this thesis, we propose four bounding methods which utilize some notion of instance-difficulty to produce difficulty-aware losses. Experimental results showed that, in most cases, optimizing a bounded variant of nDCG, AP or nRBP results in a consistent but marginal increase in the overall performance. More interestingly, we found that optimizing a bounded variant of nRBP and AP may increase the nDCG@k score, increasing the recommendation utility. Overall, our results show promising results for user/query-wise metric bounding in Learning to Rank, especially when applied to nRBP.

In the field of Information Retrieval (IR), the reliable evaluation of systems is a key component in order to progress the state-of-the-art. Much of IR research focuses on optimizing the various aspects of evaluation. Stochastic simulation is one technique that can be used to assist this kind of research. It allows researchers to overcome certain limitations associated with IR data, such as limited size, and lack of control. Recently, there have been two parallel lines of work that use stochastic simulation to study the question of "which statistical significance test is optimal for IR evaluation data?". Surprisingly, the authors reach different conclusions, despite the fact that both use stochastic simulation. One line of work, lead by Urbano et al., simulates scores for a fixed set of systems on new random topics, and concluded that the t-test is optimal. Another line of work, lead by Parapar et al., simulates new random retrieval runs for a fixed set of topics, and concluded that the Wilcoxon test is optimal. Interestingly these two tests are the most popular in IR literature. In an attempt to shed some light on this disagreement between the two conclusions, we made a first attempt at providing some empirical evidence regarding the quality of the simulation approach that was used by Urbano et al. Our main findings is that the quality of the simulation is moderately good, and also discovered some opportunities to refine it. In addition, we proposed a new model selection criterion, that showed some promising results, and in many cases managed to select models more optimally than other, more established criteria, such as AIC.

Around the world millions of people get injured due to traffic accidents. Autonomous vehicles are expected to significantly reduce these numbers. To increase safety, autonomous vehicle communication can be used. Current vehicle communication networks called VANETs have security and privacy protection problems and the vehicle industry is reluctant to use them. Multi-Party Computation (MPC) is a cryptographic technique which allows a set of parties to compute the output of a function while not revealing the input. This paper investigates the potential and feasibility of multi-party computation to solve the present problems of autonomous vehicle communication. Two architectures are proposed and discussed on the basis of a literature study and interviews with experts in MPC and autonomous vehicle communication. The solutions seem to be feasible, but further experimentation is required to confirm this. The main obstacle for the implementation of MPC in autonomous vehicle communication is that at the moment, the autonomous vehicle industry is not concerned with improving autonomous vehicle communication.

Globally, citizens’ assemblies have been gaining momentum as a way to counter dissatisfaction in democracies. Central to the citizens’ assembly is sortition, the process of randomly selecting political representatives given certain demographic criteria. In order to have an assembly representative of the population, personal data is necessary to perform the sortition, making participant privacy a matter of concern when guaranteeing fairness of the process. Secure multi-party computation (MPC)makes it possible to perform calculations on encrypted data from multiple sources without revealing it to the other processing parties or data contributors. This paper outlines two sortition designs that use MPC to guarantee participant privacy during the sortition process. Design 1 allows participants to hide their personal details to the citizens’ assembly organisers. Design 2 makes it possible for involved local government to contribute citizen data to the sortition. Different domain experts were interviewed as a means to survey the needs of the sortition community and to provide feedback on the designs. We found that little research has yet been conducted on the effects of increased privacy on sign-up rates for deliberative events, creating a potential avenue for future research.

With recent advances in performance and complexity, multi-party computation, a privacy-preserving technology which allows for joint processing of hidden input data, has lately been found to be applicable in a number of use cases. Despite existing implementations for secure data aggregation, substantial adoptions of the technology remain limited in the industry, in particular within the domain of smart mobility. This paper addresses the current issue of the mobility data shortage by investigating the potential and feasibility of multi-party computation to share data with policy makers, and proposes a solution based on additive secret sharing. On the basis of a literature study and interviews with infrastructure management authorities, as well as micro-mobility service providers, the drivers of, and barriers to employing a secure data aggregation scheme were identified. The results suggest that the technical solution appears feasible given existing implementations, while trust, acceptance and willingness of participants emerged as obstacles to a realisation.

Reentrancy attacks target smart contracts of Decentralized Finance systems that contain coding errors caused by developers. This type of attacks caused, in the past 5 years, the loss of over 400 million USD. Several countermeasures were developed that use patterns to detect reentrancy attacks on smart contracts before deployment on the Ethereum blockchain. However, the smart contracts are by default public and immutable once deployed on the blockchain. That is why the research question is: How can we protect smart contracts of DeFi systems deployed on the Ethereum blockchain that are known to be vulnerable to reentrancy attacks? A solution that detects reentrancy attacks on smart contracts after their deployment is presented in this paper. It flags transactions when a difference is found between the users' funds on both the application and protocol layers before and after each transaction using special made smart wallets. A proof of concept shows that the proposed solution can detect reentrancy attempts and stop them during the execution phase of smart contracts.

Kyber is a Decentralized Finance (DeFi) system which runs on the Ethereum blockchain. DeFi aims to remove centralized intermediaries such as Market Makers. An Automated Market Maker (AMM), implemented in a smart contract, is a decentralized version of these. Kyber's Dynamic Market Maker (DMM) is a next-generation AMM which solves two issues: Capital Inefficiency (CI) and Impermanent Loss (IL). CI is decreased by an amplification factor which a Liquidity Provider sets upon creaton of a liquidity pool, whereas IL is decreased by dynamic fees. A DMM features two reserves: one real reserve that reflects the true amounts of the two tokens in the pool and one virtual reserve that reflects the amounts after the amplification factor is applied. The vulnerability to a sandwich attack exists because the virtual reserve ratio can be unbalanced by an attacker. This results in slippage for the victim when their transaction gets executed. Finally, the attacker can perform a swap using the incorrect ratio. The research question of this paper is: How can one mitigate sandwich attacks in Kyber DMM? Kyber's current mitigation features slippage protection to protect users from sandwich attacks. The slippage protection is implemented by adding two parameters to the function used when adding liquidity: one for specifying the lower bound for the virtual reserve ratio and one for specifying the upper bound. However, this mitigation is only present in the router. Therefore, users interacting with the pool contract directly remain vulnerable. To show that this is true, we modify Kyber's test case for sandwich attacks to encompass the mint function in the pool contract. The existing mitigation can be broadened by implementing a code correction in the mint function like the one present in the function used when adding liquidity.

Oracles are mechanisms that provide blockchain networks with data that only exists outside of the network, such as asset prices. Decentralized Finance (DeFi) protocols use this data, and therefore their usability depends on the reliability of oracles. One such oracle system, widely used by DeFi protocols for pricing feeds, is Chainlink. The Chainlink system mitigates the risk of oracle manipulation attacks that have occurred in various DeFi protocols with a decentralized data aggregation infrastructure. The participants of the Chainlink system are incentivized by a coordination game, which poses game theoretic risks. While some game theoretic analyses of blockchain based systems exist, no formal study has been done on the incentives securing the Chainlink system. In this paper, we present a formal incentive model of the participants in the Chainlink system. We show that users can not detect whether incentives are aligned such that honest node behaviour is a strictly dominant strategy, making it impossible for users to assess the security of the system. We propose a mitigation which enables users to assess the agent incentives of Chainlink nodes such that they can verify whether honest behaviour is a strictly dominant strategy for all participants.

Front-running is the illegal practice of obtaining information unavailable to the general public with regards to upcoming transactions and performing actions based on this knowledge as to gain profit. This type of attack has been an issue since the introduction of the first stock market and it is not a surprise it has spread into blockchain technologies. With the invention of Decentralized Exchange Systems, this type of exploit has been an even more common occurrence that puts both the user and system at a disadvantage. This paper aims to explore the circumstances that enable front-running in UniSwap, one of the most used DeFi Systems as of the writing of this paper. Factors such as lack of privacy, slippage, as well as a miner’s role in the exploit are analysed in a broad and UniSwap-centered context to provide more insight to the problem. Moreover, this paper analyses a potential adjustment of a commit-reveal scheme, a time lock scheme, and off-chain slippage limit as possible solutions and an overall analysis of the proposed solutions. Through comparison and adjustments of different solutions, this paper strives to expose where the root of the problem lies - the Ethereum blockchain itself.