The Spoofax Testing Language (SPT) is the existing solution for testing in the Spoofax language workbench. It allows developers of domain specific languages to write their test cases declaratively. As it aims to be implementation agnostic, developers don't need to concern themselves with the details of the artifacts generated by Spoofax, and can write their tests before implementing their language. However, the previous implementation has become slow and unusable for larger test suites and can not be executed programatically. This means it can't be used for continuous integration and automated regression testing. As Spoofax was redesigned to become more robust and platform independent, the previous SPT is no longer compatible. We took this opportunity to redesign SPT. In this thesis we will discuss the benefits of a testing approach like SPT, how far along it is on the path of testing any language, and what is required to make it usable by modern day developers. We will analyze the problems that SPT had to tackle, how it solved them, and which problems still remain. Finally, we present and evaluate our new design and implementation to solve some of these remaining problems. We created a platform independent, real-time performant, easily extendable architecture that allows SPT to be used for automated tasks such as continuous integration and the automated grading of students' domain specific languages.

Implementing the syntax and semantics of a new (domain-specific) programming language is a lot of work, which is worsened by the additional work needed to add support for the language to an editor such as Eclipse or VS Code. Lack of such support can impede language usability and adoption, as developers prefer different editors. However, supporting M editors for N languages requires M * N implementations to be built and maintained, which is known as the IDE portability problem. Portable editor services aim to reduce this to M + N implementations, which leads to the main question of this thesis: how can we make the editor services of languages portable across editors? Language definitions made in the Spoofax language workbench can automatically expose their editor services in any editor that Spoofax supports. Therefore, we evaluate the portability of Spoofax Core, the editor-agnostic core of Spoofax, through an implementation of the workbench in the IntelliJ editor. To get portability for editor services of languages in general, we first investigate how editor services can be added to the most popular editors, and explore their features, documentation, and API. From this, we derive a platform-agnostic model for portable editor services: AESI, the Adaptable Editor Services Interface. AESI describes the maximum set of common editor service features supported by the editors we investigated, while at the same time imposing minimal requirements upon any implementation of these editor services. We evaluate AESI by providing two language implementations, and adapting AESI to Eclipse, IntelliJ, and VS Code. Finally, we compare AESI with two other solutions to the IDE portability problem: LSP and Monto.

Build systems speed up builds by reusing build step outputs from previous builds when possible. This requires precise definitions of the dependencies for build steps. PIE is a build system with precise dependencies, but its task definitions in Java are verbose. The PIE DSL allows pipeline developers to write concise definitions of PIE tasks, but the PIE framework has evolved and the PIE DSL cannot express many tasks and projects. This thesis presents PIE DSL 2, which improves on PIE DSL 1 in three areas. It extends the language itself with a module system, generics and support for suppliers and injected values, which allows it to express more tasks within the DSL. There are four improvements for the code base. The first two are a specification of the static semantics in Statix and a new compiler backend that compiles to an AST instead of using string interpolation, both of which extend the features for the DSL that can be expressed. The second pair is constructors for semantic types and tests, which improve development speed of the DSL. The final area we improve is the user experience, which we improve by adding documentation for expressions and types in the PIE DSL. We compare PIE DSL 2 to Java in a case study. Only a single task can be expressed in the DSL, which means that the boilerplate is not reduced. Furthermore, the Java ecosystem has better error detection except for nullability. Finally, the PIE DSL is simpler than Java, but only when the full pipeline is supported by the DSL. We conclude that the DSL is not better than Java for full projects yet, but for tasks that it can express it is a slight improvement over Java. This leads to the hypothesis that it has potential to become better if it can express enough tasks. Due to time constraints, the case study did not use the latest version of the DSL. In theory the latest version of the DSL can express 11 of the 19 tasks, but this has not been verified experimentally. Overall, this thesis makes improvements to the PIE DSL and its environment, but that has not translated to the DSL being better than Java.

Context: Updating an old result by selective re-execution of the inconsistent parts of some computation is usually faster than recomputing everything. Incremental build systems and interactive development pipelines use this technique to speed up feedback. They consist of different tasks. These tasks form a graph by depending on the environment and the result of other tasks. To re-execute a build requires an incremental build algorithm to find and re-execute inconsistent tasks. This can be done by traversing the dependency graph top down. When the mutations to the input environment are known in advance a build algorithm can avoid graph traversal. Thereby scaling with the size of a change instead of the size of the graph. Another important distinction between incremental build systems is their ability to handle dynamic dependencies. That is, dependencies that are discovered during a build. PIE is a bottom-up build algorithm that supports dynamic task dependencies. It schedules and executes inconsistent tasks without traversing the entire dependency graph. The current PIE algorithm is capable of adding dynamic task dependencies. However, it does not process removing dynamic task dependencies. This preserves consistency, but limits scalability over time. Each detached task is still scheduled and executed by the bottom up algorithm. Inquiry: PIE is inefficient when it executes tasks that are no longer a transitive dependency. In this paper we introduce task observability to solve this. This problem is unique to bottom up scheduling build systems. However, we are able to re-use techniques from incremental build systems and garbage collections to implement our solution. Approach: We split the problem into three parts, determining task observability, scheduling, and re-observing. Knowledge With the new algorithm we are able to improve the efficiency of PIE and its scalability over time. Grounding: We verify and benchmark our changes with two artificial and one real world use case in the Spoofax Workbench. Importance: The PIE runtime is able to continue operating efficiently even when tasks are detached. In some situations the Spoofax PIE pipeline reduces the feedback time with 1800 ms when results are not observed by the user. Additionally, new design patterns are made possible. For Spoofax, toggling observability creates the opportunity to implement various quality of life features such as file and project renaming. As a secondary contribution we implement a garbage collector for detached tasks and create a visualization tool for displaying the dependency graphs stored in PIE.