Diverse and realistic floor plan data are essential for the development of useful computer-aided methods in architectural design. Today’s large-scale floor plan datasets predominantly feature simple floor plan layouts, typically representing single-apartment dwellings only. To compensate for the mismatch between current datasets and the real world, we develop Modified Swiss Dwellings (MSD) – the first large-scale floor plan dataset that contains a significant share of layouts of multi-apartment dwellings. MSD features over 5.3K floor plans of medium- to large-scale building complexes, covering over 18.9K distinct apartments. We validate that existing approaches for floor plan generation, while effective in simpler scenarios, cannot yet seamlessly address the challenges posed by MSD. Our benchmark calls for new research in floor plan machine understanding. Code and data are open.

Real-world applications of Artificial Intelligence (AI) in architecture have been explored more recently at Technical University (TU) Delft by integrating AI in Design-to-Robotic-Production-Assembly and -Operation (D2RPA&O) methods. These embed robotics into building processes and buildings by linking computational design with robotic construction and/ or operation of building components and buildings. This paper presents two case studies in which AI-supported D2RA is implemented in a multidisciplinary approach that requires the integration of research domains such as architecture, robotics, computer and material science.

The development of Building Information Modelling (BIM) has enabled new opportunities, such as standard data storage and collaborative building design. Moreover, there exist many Life Cycle Assessment (LCA) tools and Building Energy Performance (BEP) simulators that use the Industry Foundation Classes (IFC) exports of BIM platforms as input for further operational analysis. While the extracted IFC files contain numerical and tabular data from the BIM model, the visual data including floor plans and section drawings is often obtained directly from the original 3D software such as REVIT. In this study, we introduce an open-source solution, BatchPlan, for batch processing IFC files of medium- and high-rise building projects, leading to floor plan extraction on a large scale. Furthermore, we have designed a user-friendly graphical interface that allows users to select floors manually. BatchPlan is based on open-source Python packages; thus users can easily edit and adapt it to their specific requirements. The presented solution enables a scalable data generation pipeline for downstream tasks that require extensive quantitative analysis, such as machine learning models to perform material detection, volume estimation, and environmental impact prediction.

This paper addresses the complexities inherent in constructing sustainable extraterrestrial habitats within lava tubes that are envisioned as promising locations for human habitation and scientific inquiry. These environments are characterized by various challenges, which are addressed in this case by integrating computer vision (CV) techniques and 3D printing in-situ. The CV component generates a detailed depth map from synthetic imagery to combine this depth map with an adaptive 3D printing process, which is proposed to ensure level surfaces at various depths, facilitating precise foundation and habitat placement within the demanding context of lava tubes. Significantly, synthetic imagery is employed due to the absence of real lava tube photos at this early stage of the current exploration. The focal point lies in utilizing advanced deep learning (DL) algorithms and convolutional neural networks (CNN) to generate depth maps for extra/-terrestrial environments. This research represents a platform for further knowledge development in the fields of CV and its application to 3D printing in-situ, hence opening new avenues for sustainable extraterrestrial habitats.

Recent advancements in machine learning (ML) in architectural design led to new developments in automated generation of floor plans. However, critical evaluation of ML-based generated floor plans has not progressed proportionally due to the subjectivity and complexity of the assessment, particularly for large and more complex floor plans. Accordingly, a hybrid (quantitative and qualitative) floor plan evaluation scheme is introduced in this study, focusing on multiple architectural aspects. To verify the effectiveness of the proposed framework, the evaluation scheme is applied on the generated floor plans resulting from two baseline computer vision models. The models have been trained on a newly introduced large-scale floor plan dataset called Modified Swiss Dwellings (MSD). The results showed that despite the progression of computer vision models for floor plan generation, they still have difficulty capturing the more complex architectural qualities. In addition, the prospect of floor plan generation and evaluation and possible future developments are discussed.

Human-Building Interaction (HBI) relies on sensor-actuator networks that are increasingly supported by Artificial Intelligence (AI). This paper presents a novel AI-supported Design-to-Robotic-Production-Assembly and -Operation (D2RPA&O) approach for reconfigurable furniture. It involves a multidisciplinary approach that relies on the integration of various domains such as architecture, robotics, computer, and material science. It contributes to the advancement of HBI by employing spatial reconfiguration relying on AI and lightweight material design, which is of relevance, particularly when the furniture consists of non-identical but similar components that are re−/ configured in a variety of possible combinations.

Ambient intelligence (AmI) relying on electronic devices employing information and communication technology (ICT) and artificial intelligence (AI) embedded in the network connecting these devices tends today to be insufficiently used. This deficiency implies that spaces are uncomfortable and considerable energy dissipates due to distribution losses, excessive or unnecessary climate control of little- and unoccupied spaces, etc. Building operations are responsible for ±27% of annual carbon dioxide (CO ₂) emissions, and infrastructure materials and construction are responsible for an additional ±13% annually; both need to be addressed integratively to meet sustainability goals. ^1,2 This paper addresses this in three AI-supported AmI test simulations of applications focusing on illumination and ventilation systems embedded in the built environment.

We propose a simple yet effective metric that measures structural similarity between visual instances of architectural floor plans, without the need for learning. Qualitatively, our experiments show that the retrieval results are similar to deeply learned methods. Effectively comparing instances of floor plan data is paramount to the success of machine understanding of floor plan data, including the assessment of floor plan generative models and floor plan recommendation systems. Comparing visual floor plan images goes beyond a sole pixel-wise visual examination and is crucially about similarities and differences in the shapes and relations between subdivisions that compose the layout. Currently, deep metric learning approaches are used to learn a pair-wise vector representation space that closely mimics the structural similarity, in which the models are trained on similarity labels that are obtained by Intersection-over-Union (IoU). To compensate for the lack of structural awareness in IoU, graph-based approaches such as Graph Matching Networks (GMNs) are used, which require pairwise inference for comparing data instances, making GMNs less practical for retrieval applications. In this paper, an effective evaluation metric for judging the structural similarity of floor plans, coined SSIG (Structural Similarity by IoU and GED), is proposed based on both image and graph distances. In addition, an efficient algorithm is developed that uses SSIG to rank a large-scale floor plan database. Code will be openly available.

Residential buildings are responsible for a considerable share of energy consumption and carbon emission. To decarbonize by 2050, as agreed in the Paris Climate Accord, immediate action for lowering the environmental impact of the building sector is needed. Environmental building design is a promising path, particularly during the early-stage design when design decisions are more impactful and long-lasting. One of the initial steps in the building design process is site assessment, during which the building context and environmental factors are to be evaluated. The surrounding environment plays a critical role in the building's energy performance and the thermal, visual, and acoustic comfort of its occupants. We choose quantitative approaches to study the complexity of the environmental design with respect to the building context by analyzing environmental cues embedded in architectural drawings that have been given less attention in previous studies. Nevertheless, disclosing site-specific geolocation data of buildings, more specifically residential type, is often challenging due to privacy issues. Therefore, there is a lack of context-related metadata in the current architectural datasets. Whereas simulation data are more available and provide a wealth of contextual information, however, it is less appealing for architects to interpret design patterns from extensive simulation figures. This research focuses on developing an interpretable visualization of the building’s micro-climate context from environmental simulation data without direct access to the geolocation of the site. The environmental context visualization is created from daylight, view, and noise from 3088 multifamily housing presented in the Swiss Buildings data set, merely based on available simulation data. The presented pipeline in this study facilitates the employment of existing simulation data in the built environment datasets while circumventing the concerns associated with geolocation data exposure. Further, the generated visualizations may be used to develop computer vision models for environmental assessments of building layout design.

Information on the history of architecture is embedded in our daily surroundings, in vernacular and heritage buildings and in physical objects, photographs and plans. Historians study these tangible and intangible artefacts and the communities that built and used them. Thus valuable insights are gained into the past and the present as they also provide a foundation for designing the future. Given that our understanding of the past is limited by the inadequate availability of data, the article demonstrates that advanced computer tools can help gain more and well-linked data from the past. Computer vision can make a decisive contribution to the identification of image content in historical photographs. This application is particularly interesting for architectural history, where visual sources play an essential role in understanding the built environment of the past, yet lack of reliable metadata often hinders the use of materials. The automated recognition contributes to making a variety of image sources usable for research.

While three-dimensional (3D) building models play an increasingly pivotal role in many real-world applications, obtaining a compact representation of buildings remains an open problem. In this paper, we present a novel framework for reconstructing compact, watertight, polygonal building models from point clouds. Our framework comprises three components: (a) a cell complex is generated via adaptive space partitioning that provides a polyhedral embedding as the candidate set; (b) an implicit field is learned by a deep neural network that facilitates building occupancy estimation; (c) a Markov random field is formulated to extract the outer surface of a building via combinatorial optimization. We evaluate and compare our method with state-of-the-art methods in generic reconstruction, model-based reconstruction, geometry simplification, and primitive assembly. Experiments on both synthetic and real-world point clouds have demonstrated that, with our neural-guided strategy, high-quality building models can be obtained with significant advantages in fidelity, compactness, and computational efficiency. Our method also shows robustness to noise and insufficient measurements, and it can directly generalize from synthetic scans to real-world measurements. The source code of this work is freely available at https://github.com/chenzhaiyu/points2poly.

We introduce AmsterTime: a challenging dataset to benchmark visual place recognition (VPR) in presence of a severe domain shift. AmsterTime offers a collection of 2,500 well-curated images matching the same scene from a street view matched to historical archival image data from Amsterdam city. The image pairs capture the same place with different cameras, viewpoints, and appearances. Unlike existing benchmark datasets, AmsterTime is directly crowdsourced in a GIS navigation platform (Mapillary). We evaluate various baselines, including non-learning, supervised and self-supervised methods, pre-trained on different relevant datasets, for both verification and retrieval tasks. Our result credits the best accuracy to the ResNet-101 model pre-trained on the Landmarks dataset for both verification and retrieval tasks by 84% and 24%, respectively. Additionally, a subset of Amsterdam landmarks is collected for feature evaluation in a classification task. Classification labels are further used to extract the visual explanations using Grad-CAM for inspection of the learned similar visuals in a deep metric learning models.

While half of all construction tasks can be fully automated the other half relies to a certain degree on human support. This paper presents a Computer Vision (CV) and Human–Robot Interaction/Collaboration (HRI/C) supported Design-to-Robotic-Assembly (D2RA) approach that links computational design with robotic assembly. This multidisciplinary approach has been tested on a case study focusing on urban furniture and involving experts from respective disciplines and students.

Historical photos of towns and villages contain a great deal of information about the built environment of the past. However, it is difficult to evaluate the information of images that are not labeled or incorrectly labeled or not organized in repositories or collections. In order to make the sheer volume of images that are not tagged with metadata found on the Internet or in institutional archives accessible for research, an automated recognition of the image content, in this case of buildings, is necessary. Computer vision can help to address this problem and enable the identification of historical image content. This article describes how artificial intelligence and crowdsourcing are used to identify buildings in nearly half a million historical images of the city of Amsterdam. It explains how computer science and humanities disciplines are linked together to accomplish this task.

This paper investigates sensitive minima in popular deep distance learning techniques such as Siamese and Triplet networks. We demonstrate that standard formulations may find solutions that are sensitive to small changes and thus do not generalize well. To alleviate sensitive minima we propose a new approach to regularize margin-based deep distance learning by introducing stochasticity in the loss that encourages robust solutions. Our experimental results on HPatches show promise compared to common regularization techniques including weight decay and dropout, especially for small sample sizes.

The ArchiMediaL project aims to bridge between data science and researches on contemporary and historical built environments by developing state of the art AI algorithms for the automatic linking of available meta-data and image repositories. As a case-study we use the 360,000+ historical images from the Amsterdam Beeldbank database.

Cross domain image matching between image collections from different source and target domains is challenging in times of deep learning due to i) limited variation of image conditions in a training set, ii) lack of paired-image labels during training, iii) the existing of outliers that makes image matching domains not fully overlap. To this end, we propose an end-to-end architecture that can match cross domain images without labels in the target domain and handle non-overlapping domains by outlier detection. We leverage domain adaptation and triplet constraints for training a network capable of learning domain invariant and identity distinguishable representations, and iteratively detecting the outliers with an entropy loss and our proposed weighted MK-MMD. Extensive experimental evidence on Office [17] dataset and our proposed datasets Shape, Pitts-CycleGAN shows that the proposed approach yields state-of-the-art cross domain image matching and outlier detection performance on different benchmarks. The code will be made publicly available.

Understanding how cities visually differ from each others is interesting for planners, residents, and historians. We investigate the interpretation of deep features learned by convolutional neural networks (CNNs) for city recognition. Given a trained city recognition network, we first generate weighted masks using the known Grad-CAM technique and to select the most discriminate regions in the image. Since the image classification label is the city name, it contains no information of objects that are class-discriminate, we investigate the interpretability of deep representations with two methods. (i) Unsupervised method is used to cluster the objects appearing in the visual explanations. (ii) A pretrained semantic segmentation model is used to label objects in pixel level, and then we introduce statistical measures to quantitatively evaluate the interpretability of discriminate objects. The influence of network architectures and random initializations in training, is studied on the interpretability of CNN features for city recognition. The results suggest that network architectures would affect the interpretability of learned visual representations greater than different initializations.

Built form dominates the urban space where most people live and work and provides a visual reflection of the local, regional and global esthetical, social, cultural, technological and economic factors and values. Street-view images and historical photo archives are therefore an invaluable source for sociological or historical study; however, they often lack metadata to start any comparative analysis. Date and location are two basic annotations often missing from historical images. Depending on the research question other annotations might be useful, that either could be visually derived (e.g. the number or age of cars, the fashion people wear, the amount of street decay) or extracted from other data sources (e.g. crime statistics for the neighborhood where the picture was taken). Recent advances in automatic visual analysis and the increasing amount of linked open data triggered the research described in this paper. We provide an overview of the current status of automated image analysis and linked data technology and present a case study and methodology to automatically enrich a large database of historical images of buildings in the city of Amsterdam.

We address the interpretability of convolutional neural networks (CNNs) for predicting a geo-location from an image. In a pilot experiment we classify images of Pittsburgh vs Tokyo and visualize the learned CNN filters. We found that varying the CNN architecture leads to variating in the visualized filters. This calls for further investigation of the effective parameters on the interpretability of CNNs.