Centralized platforms like TikTok are cause for significant concerns over information control, censorship, and bias. Decentralized systems offer a promising alternative, but their adoption is hindered by the lack of effective relevance ranking of search results. Existing decentralized approaches rely on heuristics that do not adapt to user behavior. This paper presents DART, the first decentralized ranking algorithm to leverage machine learning over users' search activities. DART adapts its ranking function using a Transformer-based learning-to-rank model trained on a real workload from a decentralized file-sharing application. We find that it improves over the best baseline by 19 % on our ranking metric (MRR).

This study introduces De-DSI, a novel framework that fuses large language models (LLMs) with genuine decentralization for information retrieval, particularly employing the differentiable search index (DSI) concept in a decentralized setting. Focused on efficiently connecting novel user queries with document identifiers without direct document access, De-DSI operates solely on query-docid pairs. To enhance scalability, an ensemble of DSI models is introduced, where the dataset is partitioned into smaller shards for individual model training. This approach not only maintains accuracy by reducing the number of data each model needs to handle but also facilitates scalability by aggregating outcomes from multiple models. This aggregation uses a beam search to identify top docids and applies a softmax function for score normalization, selecting documents with the highest scores for retrieval. The decentralized implementation demonstrates that retrieval success is comparable to centralized methods, with the added benefit of the possibility of distributing computational complexity across the network. This setup also allows for the retrieval of multimedia items through magnet links, eliminating the need for platforms or intermediaries.