Internet of Thing (IoT) is one of the most influential technologies in the present time. People, processes, and things are connected with the Internet through IoT. With the increasing demands of user applications, the number of connections is also increasing exponentially. Therefore, security becomes a critical issue in IoTs. Confidentiality, Integrity, and Availability (CIA) services are important for IoT applications. IoTs must also ensure proper authentication mechanisms to ensure CIA in the second stage. Various researches in this direction address the authentication issues in IoTs. In this paper, we survey the authentication aspects in IoTs and their allied domains. We analyze the potentialities of the existing state-of-the art-approaches and also identify their limitations. We discuss the basics of authentication and its related attacks for the ease of interpretability of the readers. We show a taxonomical understanding of the approaches and try to connect the evolution of the solution strategies. These connections, to the best of our knowledge, are novel as compared to the existing authentication surveys. Besides, the multidimensional vision of this survey for IoT extensions is an add-on to the benefits. We also provide a discussion on the future direction of research in this domain. In a nutshell, this survey is a one-stop solution for academia and industry to understand the status quo of IoT authentication schemes/protocols.@en