This research paper aims to present how Theory of Mind (ToM) - the ability that allows humans to attribute mental states to others - can be used in the context of physically and virtually embodied computational agents. The focus is on using ToM for perspective-taking in environments with multiple computational agents interacting together. A Systematic Literature Review (SLR) was conducted providing a structured search process for collecting the literature. The findings from this review were obtained from 38 selected papers found in the literature since 2018. The review presented how computational ToM was inspired by the human ToM. Further, it summarised the current applications for perspective-taking for multi-agent settings, and it collected insights on high-level implementations of ToM agents. The findings highlight the complexity of ToM agents and the importance of ToM for agent interactions.