Computer and social networks can be effectively represented as complex temporal graphs where entities (nodes) keep interconnecting through various relationships (edges), forming evolving structures. Anomaly Detection (AD) in such networks consists of identifying patterns diverging from what is expected or normal. This task is fundamental for the detection of potential threats - e.g. suspicious connections (edge AD) or misbehaving entities (node AD), and challenging due to the lack of a common definition of anomaly. However, the literature is scarce about solutions to detect node anomalies on temporal graphs. This work addresses three challenges in AD as found in computer and social networks: fast-evolving graph structure, lack of ground truth, and simultaneous presence of anomalous nodes and edges. For this, we propose to use temporal Graph Neural Networks (tGNNs) coupled with specialised AD blocks trained in a self-supervised way. We also embed an attention mechanism providing interpretability to the decision process. We extensively validate the tGNNs on synthetic and real-world datasets showing that they successfully detect both node and edge anomalies simultaneously (≈0.9 of average AUC).