The administration of transportation infrastructure entails addressing a multitude of obstacles arising from the intricate, fast changing and dynamic nature of the environment. This thesis focuses on improving infrastructure maintenance planning through the application of deep reinforcement learning. The research begins with a thorough examination of current approaches in transportation network simulation, revealing the omissions in current strategies for infrastructure upkeep. This is followed by the development of a simulation environment that simulates realworld conditions and integrates pavement and bridge condition models to evaluate various maintenance strategies.
The proposed framework is tested on a simulated transportation network in the United States, which incorporates a traffic model to account for dynamic changes. The DRL algorithms are then used to make maintenance policies for this environment. These policies are evaluated through extensive simulations, with a focus on reducing maintenance expenses and improving the overall condition of the infrastructure.

This master thesis rigorously explores the integration of a genetic algorithm-based workflow in the design and optimization of a passive fixed shading system utilizing ETFE cushion panels, with the objective of enhancing the thermal resilience of an existing building envelope. The case study focuses on a mid-rise office building located in the port of Athens, Greece. The central research question addressed is: "How can a genetic algorithm-based workflow be effectively employed in the multi-objective optimization of a shading system to improve the energy efficiency of an existing building envelope?"

To answer this question, the research investigates four interconnected domains: ETFE double skin structures, resilience quantification, multi-criteria decision-making approaches with genetic algorithms, and the analysis of the case study building facade system. The study centers on the preliminary design phase of the shading system, highlighting its potential as a retrofit solution for existing infrastructures challenged by rising temperatures.

The outcomes of this research include the development of a versatile workflow for evaluating the energy performance of existing buildings, facilitating interdisciplinary feedback within a design team, and applying multi-objective optimization to design problems. This work provides a comprehensive framework for integrating advanced computational methods in architectural design, thereby contributing to the improvement of building energy efficiency and thermal resilience.

This thesis investigates the integration of reinforcement learning (RL) techniques to enhance inspection and maintenance planning for timber structures, considering the increasing impact of climate change on their structural integrity. Timber, a critical material in historical construction, is vulnerable to environmental factors such as temperature, moisture, and biological degradation. These vulnerabilities are exacerbated by climate change, leading to significant alterations in mechanical properties and thereby challenging the longevity and safety of these structures.
Given the dynamic nature of these challenges, traditional inspection methods, which rely heavily on manual processes and individual expertise, are insufficient. This research employs machine learning to develop a predictive maintenance model that adapts to the evolving conditions affecting timber. The model aims to improve the accuracy and efficiency of inspections and maintenance planning, facilitating timely interventions to preserve the structural integrity of timber constructions.
This study addresses several key questions: identifying the principal factors influencing timber degradation, understanding the impact of climate change on these factors, evaluating current maintenance strategies, and determining the most suitable RL techniques for this application. Additionally, the research explores methods to optimize the RL model for reliability and accuracy, methods for validating and evaluating the model's planning capabilities, and strategies to enhance the interpretability of RL outputs for practical use in the field.
By leveraging advanced RL methodologies, this thesis contributes to the field of timber engineering and proposes a scalable and adaptable solution to enhance sustainable construction practices in response to evolving climate patterns.

This paper explores the scope of using a deep learning framework for shape optimization of Catalan vaults for medium seismic areas. Catalan vaults are thin tile vaults that optimize the material usage of a floor slab without form-work and thus, additional material and labour. These structures can be constructed from tiles made from locally sourced earth which can provide an alternate to steel, timber, and concrete for areas with poor access to such materials, bringing down transportation, material, and carbon costs, providing opportunity to accommodate the consequences of rapid population growth. Seismic optimization of these vaults usually requires topology optimization and shape optimization tools. However, conventionally, these are computationally expensive and time-consuming - making them unsuitable for initial design explorations where a vast array of designs need to be quickly explored. As an alternative, a deep learning framework is explored as a design generation and optimization tool. This uses a Variational Autoencoder (VAE) trained on a dataset of 10,000 samples to extract novel meshes whose seismic performance is then predicted with the help of fully-connected dense Neural Network (NN) surrogate models trained on the results of a Linear Dynamic analysis in Karamba (in Grasshopper). An optimization loop is set-up through Gradient Descent Optimization where the gradient of the predicted score is minimized with respect to the latent space of the VAE - for single and multi-objective optimization. Conditioning the latent space of the VAE is further explored (Conditional VAE) so that the user is able to extract samples from the latent space with particular desirable characteristics such as a desirable height of the vault. This opens up opportunities to gain better control of the latent space and generate meaningful new samples that are able to incorporate user specifications. The geometry of the Catalan vault is represented in terms of polyedge force-densities that allow a 99.91% reduction in dimensionality and thus, faster convergence, as compared to other data structuring techniques explored in the literature as half-adjacency matrices.

In response to the European Commission's 2020 Renovation Wave strategy aiming to double the annual rate of energy renovation for buildings by 2030, this thesis addresses the challenge of optimizing retrofitting planning using planning optimization algorithms. Despite the economic and practical hurdles identified in existing research, limited attention has been given to optimizing the timing and sequence of retrofitting actions. Recognizing the complexity introduced by uncertainty factors such as material degradation rates,this study draws parallels between retrofitting and predictive maintenance problems. This thesis is divided into two main sections: the first establishes the theoretical foundation, covering retrofitting principles, Markov Decision Processes, and planning algorithms like Q-learning and Value Iteration; the second applies these theories to practical implementation to develop a methodology for retrofitting planning optimization. Detailed methodologies are discussed, emphasizing building energy performance and degradation factors, with conclusions drawn to inform the simulation of building performance over time. This comprehensive approach aims to advance the current understanding and application of planning optimization in the building retrofitting domain.

The study of energy consumption across various building clusters offers a path to discerning intricate patterns and establishing energy efficiency metrics. However, these analyses have mostly been limited to small, controlled settings, leaving a vast potential for broader application in energy efficiency management and classification untapped. This research leverages machine learning models to determine gas consumption patterns and energy efficiency characteristics of buildings in real-life settings, based on a range of parameters including insulation properties and year of construction. The developed system was applied to a comprehensive dataset comprising eight distinct clusters of buildings, with a total of nearly 10,000 hourly gas consumption. To supplement the analysis, additional data was gathered concerning the building features. The findings indicate that the average gas consumption varies significantly across clusters, with dependencies shown for the age of the building, insulation characteristics, and building orientation The developed framework proved to be suitable for gaining insights into average gas consumption and usage patterns at a building level, non-intrusively and on a large scale. The additional data provided comparative insights between different building groups. The developed system can be easily expanded for other building characteristics and could be used to drive tailored feedback on energy efficiency improvements within buildings. This research paves the way for a more comprehensive approach to building energy efficiency, one that goes beyond the traditional parameters to include a broader set of variables such as building usage, occupant behavior,and heating system efficiency

Across the world, countries are facing housing shortages and the Netherlands is no different. The increasing demand for new housing exceeds the growth rate of the architecture, engineering, and construction industry. Current solutions remain small in scale and therefore unsustainable. Multi-family housing is the optimal typology to address the housing shortage but the industry cannot design and build these projects fast enough. Automation can help. In 2016, Kevin Kelly[2016] reframed the conversation about automation, by stating "our most important mechanical inventions are not machines that do what humans do better, but machines that can do things we can't do at all." Humans cannot address the housing crisis alone, but automation through the application of deep learning models can bring well-designed spaces to everyone.

Since the introduction of computers, architects have looked for ways to automate menial tasks. Some researchers even imagine a future where the machine becomes a partner to architects and designers, contributing to the design process. This ambition requires that the algorithms train themselves. Innovations in the field of deep learning have made this possible by allowing algorithms to train themselves through the use of artificial neural networks. When it comes to applying deep learning to generative design tasks, however, there is little research. The studies that have been done generate geometry that is small in size (64 x 64 x 64 voxels) and focuses on objects like chairs, not on buildings. 3D generative adversarial networks show promise for generating building geometry. By automating design, it is possible to apply expert knowledge on good design to all projects so everyone has access to well-designed buildings.

This research aims to develop the architecture for a generative adversarial network that produces feasible building geometry. An important first step was identifying and pre-processing a data set that could be used for this purpose. The data set is released with the publication of this thesis so it can be used for further research. Through this thesis, the Improved 3D Wasserstein Generative Adversarial Network architecture has also been developed and documented. The research found that using a combination of Wasserstein loss with gradient penalty, Leaky ReLU activation functions in the generator and the critic, and the RMS Prop optimizer results in an architecture with stable training and outputs that are similar in size, shape, and proportion to the training data with minimal noise in the output. Performance of 3D Wasserstein generative adversarial networks with these hyperparameters was improved even further when using ten layers and a larger number of channels. The experiments concluded that generative adversarial networks can be used to generate building geometry and can be an area of continued research to improve generative design tools and support the automation of architectural design.

Energy use, CO2 emissions, and waste production are all significant causes of environmental issues. The building sector is a major contributor to these problems, specifically the manufacturing of (structural) steel elements. Application of reuse and/or remanufacturing, as done in a circular economy, will reduce these effects. Therefore, these techniques must to be taken into account while designing buildings and structures. However, as actors in the construction industry recognize barriers, such as time delays in the early phases of the design process, these tactics are not yet commonly used. Nonetheless, reusing structural steel components is still favoured by structural engineers, particularly when the aforementioned obstacles can be removed. This is feasible with the use of computational tools.

In this thesis, an AI based deep generative design framework is developed to optimize a 3D spatial truss structure for structural performance, material use and similarity to a defined stock of reusable materials. This workflow consist of a labelled dataset of geometries and performance indicators, a variational autoencoder (VAE), a predictive surrogate model and optimization through gradient descent. The aim of this study is to gain insight into the extent to which such a workflow can be applied in early-stage architectural and structural design exploration, especially with regards to making circular design & reuse more feasible. The case study in this thesis is a spatial truss structure supporting a flat roof. It was found that a surrogate model can be successfully trained to predict the performance of a geometry. For this, various input data representations can be used, including adjacency matrices and edge-vertex matrices. Through training of a VAE, a latent space from which meshes could be generated was successfully constructed. The VAE was able to accurately reconstruct a significant part of the geometry dataset. Through gradient descent, novel meshes were generated. Predicted performance of these meshes was increased. After assessing performance with simulations and calculations, increases in performance often remained. The largest performance increase found was 74.2%. Minor editing of meshes based on user insight demonstrated further increase in mesh performance.

Operation and maintenance of the built environment have a major effect on socioeconomic stability and sustainability. A significant part of our built world approaches or has well exceeded its designated structural life. As engineers, we need to find efficient ways to extend this life while maintaining acceptable levels of safety and performance. In this direction, smart and adaptive life-cycle inspection and maintenance planning are of paramount importance to reduce costs, increase structural reliability, and minimize resource-intensive interventions. Deep reinforcement learning provides a novel approach to strategize these decisions for systems subject to uncertainties and deterioration.

The goal of this project is to determine optimal life-cycle inspection and maintenance policies for road networks. Road maintenance and inspection planning is a complex task, involving a multitude of different objectives, such as the minimization of lifecycle cost and CO2 emissions (both from the maintenance works and vehicle emissions). However, current literature has underestimated the importance of optimizing for multiple objectives, and doesn’t consider the environmental impact during planning. In the current project, we aim to optimize the maintenance and inspection schedule with respect to minimizing the maintenance costs, carbon emissions and vehicle owners costs at the same time. To achieve that, a multi-objective road network environment will be modelled and multiple multi-objective Deep Reinforcement learning approaches will be applied and compared to traditional life-cycle management policies.

In the last decades, climate change is causing our environment to change rapidly, unprecedented in recent history. Civil engineering structures are dependent on the deteriorating environment they are situated in. Changes can cause an increase in loading due to, for example, extreme weather events or alter the structure’s resistance by, for instance, accelerated corrosion or an increase in the number of frost days. However, planning for such events depends significantly on the state of climate change, which is not considered in the sequential decision-making optimization for inspecting and maintaining our infrastructures.

Sequential decision-making optimization refers to the process of finding an optimal sequence of actions to be performed to keep the structure safe. Optimality often refers to the plan with the lowest costs. Traditional inspection and maintenance plans seek a solution by applying heuristic-decision rules on, for instance, time- or condition constraints and fail to find an optimal solution. During the last decade, a new method called Deep Reinforcement Learning (DRL) has been applied and proven to find an optimal strategy to beat traditional approaches. Especially the ability of DRL to find an optimal solution for partially observable environments makes it a perfect candidate for the problem at hand.

This thesis has developed a framework that incorporates partial observability over possible climate scenarios in the decision-making of engineering structures’ inspection and maintenance planning. The framework explains the steps required to translate a physical system towards a Partially Observable Markov Decision Process (POMDP), as the mathematical framework needed to seek an optimal se- quence of inspection and maintenance actions for. Then, it is explained how the POMDP can be used to find an optimal policy with a DRL algorithm, which includes benchmarking and testing the policy. A belief state has been incorporated over the climate scenarios to simulate the partial observability of the climate. Updating over the scenarios is provided by Bayesian Inference, using a climate parameter, such as temperature.

The framework has been applied to a case study in the second part of the thesis. The case study con- figures a stochastic deterioration process for different climate scenarios. An optimal policy has been found by applying Proximal Policy Optimization (PPO) with a decentralized policy for the various com- ponents. Two well-established heuristic-based maintenance policies, time-based maintenance (TBM) and condition-based maintenance (CBM) have been configured as benchmarks for the framework. The framework is compared against the benchmarks using lifecycle costs and safety as metrics. The policy provided by the framework has outperformed both benchmarks in terms of costs while maintaining the same safety. The policy beat TBM with 5% and CBM with 1% in terms of lifecycle costs. Another important distinction is that the benchmarks are optimized for the climate scenarios, while the frame- work finds this distinction without prior knowledge. It is therefore concluded that the framework can find an optimal policy under the uncertainties related to climate change. The case study, however, does not fully capture the complexity of engineering structures that the framework can catch. It is therefore recommended to use the framework for a more complex structure in the future.

This thesis is part of a larger ongoing research project at the TU Delft regarding structural glass as a novel construction material. Here, the focus is directed towards the casting of glass to create complex free-form geometries. Thus far, only smaller components have been created, but a multitude of previous student work explored how the current limitations in the fabrication process could be overcome by using topology optimization to design larger cast glass components. This thesis continues this research by integrating the limitations and remarks concluded from previous work to further explore the potential of designing large cast glass components using topology optimization.

This thesis intends to contribute to the existing research by creating a tool for the design of 3D optimized cast glass components, which can take into consideration the structural properties of glass as well as manufacturing, annealing and specific design criteria. The tool is created in Matlab, using the SIMP optimization method with 8-noded hexahedral finite elements for the structural model.

From the literature review it was concluded that the SIMP methodology is best suited the requirements of this project. For the structural model, 8-noded hexahedral elements were selected. Two algorithms were developed: one with a compliance based objective and a second with a volume based objective.

The algorithm’s performance was tested on different domains. To facilitate comparison with existing literature, the algorithm was initially calibrated with a domain width of one element, effectively creating a two-dimensional optimization. Subsequently, its performance was assessed through the application of a small three-dimensional problem. Finally, the algorithm is used to design a case study involving the structural slab of a small pedestrian bridge located inside the British Museum.

The results validate the benefits of the 3D algorithm. The geometries optimized in three dimensions have a higher structural stiffness and offer more spatial insight. Moreover, the volume optimizations results in a larger material reduction than those achieved in two-dimensional optimizations. For this reason, the volume-optimized component was selected for post-processing and implementation in the final design.

An issue of utmost significance constitutes the maintenance of engineering systems exposed to corrosive environments, e.g. coastal and marine environments, highly acidic environments, etc. The most beneficial sequence of maintenance decisions, i.e. the one that corresponds to the minimum maintenance cost, can be sought as the solution to an optimization problem. Owing to the high complexity of this sequential decision optimization problem, traditional methods such as thresholdbased approaches, fail to arrive at an optimal strategy, while at the same time the commonly used offline knowledge about the environment can not capture efficiently the stochastic way in which an engineering system deteriorates. Over the last few years, Deep Reinforcement Learning (DRL) has been proven a promising tool to tackle such problems, being often limited though by the dimensionality curse and the implications caused by large state and action spaces, an issue which leads to simplifications like their discretization. Bayesian principles and model updating are the most widely used tools to model accurately systems with high uncertainty, by incorporating data acquired through monitoring devices and thus improving the knowledge about the stochastic system.

This research proposes an integrated framework that aims to determine an optimal sequence of maintenance decisions over the lifespan of deteriorating engineering systems, combining the aforementioned core concepts of Deep Reinforcement Learning (DRL) and Bayesian Model Updating (BMU). More specifically, it investigates different Deep Reinforcement Learning (DRL) algorithms, namely Double Deep Q-Network (DDQN), Advantage Actor Critic (A2C), and Proximal Policy Optimization (PPO), while the updating of the uncertain parameters is performed through sampling, i.e. No-U-Turn Sampler (NUTS). All these tools will be first applied to elementary problems for the sake of verification and validation, while the culmination of this research is the application of the framework on a more realistic and complicated, multi-component structure. The obtained results are compared with benchmark performances to properly showcase the efficiency and the weaknesses of the tool.