Self-healing concrete has created a lot of public interest in recent years. Several research groups worldwide are currently working on creating durable and sustainable self-healing concrete structures. HEALCON (the concrete which repairs itself) is a European Union funded project, which focuses on developing cementitious materials with different selfhealing mechanisms. The self-healing mechanisms can either repair the cracks and regain liquid-tightness, bridge the cracks and recover structural performance, or do both. One of the promising materials that have been studied within the project is the bacteria-based selfhealing mortar, which is able to regain liquid tightness after cracking and healing. Within HEALCON an experimental methodology, which comprises of tests for evaluating the ability of the cementitious material to regain liquid-tightness and mechanical properties, has been developed. This study focuses on evaluating the suggested experimental methodology through a round robin test (RRT) among five laboratories within the framework of RILEM/TC 253 MCI (Micro-organisms-Cementitious Materials Interactions), WG4 (Engineered bacteria-based protective systems for cementitious materials) and it concerns only the part that examines the sealing efficiency. The testing sequence includes: - tests for material characterization, - crack introduction on mortar prisms, - healing treatment and - water tightness examination. Specimens with and without bacteria-based self-healing agent were tested. After the completion of the tests the results of the different laboratories were gathered for purposes of comparison. The comparison revealed high scatter in the results of the suggested methodology. Therefore, the current paper gives some recommendations, for improving the tests procedures, which will later be adapted to the second RRT that will follow.