Probing the Dark Web

Abstract

The inception of onion routing in the mid-1990s, evolving into Tor (The Onion Routing) and other anonymous networks, marked a pivotal moment in the quest for internet privacy. However, the emergence of the dark web, facilitated by these networks, has also increased cybercrime activities, necessitating a critical examination of its implications and challenges. Besides, the intricate security architecture of this hidden realm creates a persistent challenge in identifying and mitigating cyber threats, fostering a landscape that demands innovative methodologies for robust cybersecurity. This thesis addresses the research gap in the existing literature, predominantly focused on TOR v2, by investigating protocols and services operating within TOR v3 onion services. Moreover, it fills the void in the literature by proposing an optimized port-scanning methodology for comprehensive analysis. Unlike previous studies, which have only considered a small dataset of onions and a limited number of ports in their TOR v2 onion services analysis, this work proposes an optimized strategy for scanning all ports, thus providing a more thorough understanding of the network's dynamics. Our research uncovers several critical insights into the landscape of onion services. We identified many onion services operating on non-standard ports, escaping typical web crawlers and the Tor browser, which only display HTTPS/HTTP pages. Through a comprehensive port scan of 300,000 onion services, we discovered 196 unique ports, highlighting a broad spectrum of service configurations. We categorized these services into six main types: web, Bitcoin, remote access, chat, email, file transfer, and miscellaneous, with web services being the most prevalent. Additionally, we observed that a significant number of onion services discovered in 2019 remain active, suggesting durability within the dark web. Interestingly, some services exhibited extensive port usage, with up to 35 open ports reflecting diverse functionalities or potential security vulnerabilities. These findings provide a deeper understanding of the dark web's structure and the persistence of its services. The key findings of this research sheds light on the intricacies of the dark web's inner workings. By addressing key research questions and providing clear definitions and mechanisms, this study empowers stakeholders, such as security researchers, law enforcement, and cybersecurity professionals, to navigate the digital landscape with vigilance and develop robust defence mechanisms against emerging threats.