Automated tests have always been essential for changing a piece of software. They let developers detect and locate faults early on and provide confidence in the product’s quality. With the rise of Continuous Delivery (CD) in software development, changes are being deployed multiple times a day. Maintaining a high quality test suite has therefore never been more important. Based on the CD practices at ING and its delivery pipeline we have created Spectata, a tool that helps development teams assure the quality of their unit test code. Every time a change in the code is made Spectata calculates metrics that reflect the adequacy, fault finding effectiveness and maintainability of the test suite. It provides a quality verdict on those aspects, recommended refactorings and a detailed comparison with the quality of prior builds. To evaluate Spectata we perform four case studies in which we compare Spectata’s verdict and recommendations to ones given by the developers and ourselves. The results demonstrate that the testing quality verdict aligns with our own opinion and that the tool is able to help development teams assess, maintain and improve the quality of their test suites.