Planning Support Systems (PSS) are increasingly used to support collaborative planning workshops in urban adaptation practice. Research has focused on developing such tools and evaluating their use in workshops but has not measured tools' effects over time on real planning processes, on the participants involved, and on the final outcomes. The role that tools play in adaptation planning, therefore, remains unclear. A longitudinal case study was made to evaluate a PSS, the Adaptation Support Tool (AST), in a design workshop for sustainable urban water management, in Berlin, Germany. The case study also served to test the evaluation framework and generate insights regarding systematic evaluations of tools in planning processes. The case study was carried out over eighteen months, to capture both the details of the workshop and its longer-term effects on the project and participants. Our results show that the AST's most evident effects were (1) contributory and less tangible in nature (e.g., supporting learning), than directly causal and concrete (e.g., affecting planning decisions), and (2) a function of the process and context in which the workshop took place. This study demonstrates that making systematic, longitudinal evaluations are valuable for studying the role of PSS in urban adaptation planning.