RT

R. Taormina

info

Please Note

40 records found

Journal article (2026) - Giel W.A. Hagenbeek, Tim H.M. van Emmerik, Tianlong Jia, Pummarin Khamdahsag, Kittiphon Boonma, Riccardo Taormina, Thomas Mani, Marc Rußwurm
Rivers are major pathways for plastic pollution to oceans, with high emissions in tropical regions. Research in the Saigon River showed that invasive water hyacinths (WHs) can trap macroplastics and serve as proxies for detecting river plastic using remote sensing. We explore this phenomenon and its detection methods transferability to the Chao Phraya River. Along a 62.1 km river course, WHs trapped an average of 32% of floating plastics, reaching local maxima of 78%, comparable to 54%–82% in the Saigon. Plastic concentration in WHs was 59 times higher than in open water, increasing downstream. Object detection models transferred well for WHs and entangled plastics (Chao Phraya: mAP50 = 68% and 54%; Saigon River: mAP50 = 70% and 52%) but poorly for free-floating plastics (23% vs. 48%). Physical sampling found 14 times more plastics within WHs than imagery, highlighting WHs’ role in trapping plastics and their potential for monitoring and targeted clean-up efforts. ...
Automated sewer defect detection has advanced through deep learning, particularly supervised methods using CCTV images, but based on large annotated datasets. This paper proposes a semi-supervised learning (SSL) approach to reduce labeling demands. The method comprises self-supervised pre-training on unlabeled images using SwAV (Swapping Assignments between multiple Views) followed by fine-tuning for multi-label classification. Experiments on the Sewer-ML dataset demonstrate that the SSL approach, trained on only 35k labeled images, achieves an F1-score of 69.11%, and F2CIW of 54.22%, surpassing the fully supervised baseline trained from scratch on 1.04 million images. Increasing the unlabeled pre-training data further enhances performance, while ImageNet initialization consistently outperforms training from scratch. Self-supervised learning also helps mitigate the effects of mislabeled data, which is observed to be present even in the Sewer-ML ground truth. Overall, self-supervised learning provides an accurate, scalable, and cost-effective alternative to fully supervised approaches, particularly in data-scarce or imperfectly labeled scenarios. ...
Flood hazard maps are essential for protection and emergency plans, yet their probabilistic application is constrained by the computational cost of numerical models. Deep learning surrogates can provide orders of magnitude faster predictions, but their use for uncertainty quantification in realistic settings and their ability to incorporate hydraulic structures remain largely unexplored. Studying deep learning surrogates for probabilistic flood maps is non-trivial because of the lack of reference ground-truth data that might lead to misleading confidence in predictions. Moreover, hydraulic structures are challenging to include due to their generally unidimensional nature. In this work, we investigate the use of deep learning surrogates for realistic, large-scale flood simulations in case studies with hydraulic structures under diverse boundary conditions. To this end, we employ the multi-scale shallow-water-equations graph neural network (mSWE-GNN) that enjoys transferability to different boundary conditions and locations and whose graph-based architecture allows to represent structures such as canals, underpasses, and elevated elements as inputs. To address the lack of reference ground-truth data, we further introduce the average relative mass error (ARME), a mass-conservation-based criterion that helps identify physically plausible simulations. We applied the model on dike ring 41 in the Netherlands, generating probabilistic flood maps that account for uncertainties in breach location and breach outflow hydrographs. The model was trained on 30 simulations, generated with Delft3D, and evaluated against unseen benchmark simulations from the Dutch national flood catalogue, achieving a critical success index (CSI) of 73.6 % while running 10 000 times faster than the numerical simulator. The proposed ARME is negatively correlated with the CSI, with a Spearman correlation coefficient of -0.7, making it a useful indicator of simulation plausibility when evaluating unseen case studies. We obtained probabilistic flood maps by running 10 000 different flooding scenarios on a computational mesh of 180 000 cells in approximately 10 h, with about half of the simulations classified as plausible based on the mass-conservation check. This framework offers a practical tool for rapid probabilistic flood hazard assessment and a way to prioritize detailed physical simulations, supporting more efficient and robust flood risk management. ...
Simulating urban drainage hydraulics is computationally demanding, limiting its application in tasks that require real-time or repeated simulations. Graph Neural Networks (GNNs) are promising metamodels, but the effect of their internal components and transferability potential remain underexplored. This study addresses these gaps through two main contributions: (1) a systematic evaluation of key architectural components, including graph layer type, processor depth, and prediction window with links to physical transport dynamics; and (2) transferability experiments across domains (across two distinct drainage networks) and tasks (from head to flow prediction). As case studies, we selected two combined sewer networks in The Netherlands that differ in their hydraulic dynamics. We find that metamodels with moderate depth and a ten-step prediction window achieve high accuracy (RMSE of 2–5 cm for hydraulic heads and 0.02 m3/s for flowrates). They also reach speed-ups of up to four orders of magnitude higher compared to the physics-based model, SWMM, when executing parallel simulations in GPU. Based on our two case studies, we find that pre-trained metamodels with full fine-tuning effectively adapt to a new task within the same domain, whereas cross-domain transfer requires appropriate normalization and fine-tuning. Furthermore, joint training on both case studies enables the metamodel to capture representations of both systems, suggesting potential for more general applicability. These findings demonstrate that metamodel architecture can reflect physical system behavior and offer practical guidance for building fast, accurate, and generalizable GNN-based metamodels—establishing a foundation for their use in applications such as uncertainty analysis, design optimization, and nowcasting. ...
Deep-learning-based surrogate models represent a powerful alternative to numerical models for speeding up flood mapping while preserving accuracy. In particular, solutions based on hydraulic-based graph neural networks (SWE-GNNs) enable transferability to domains not used for training and allow the inclusion of physical constraints. However, these models are limited due to four main aspects. First, they cannot model rapid differences in flow propagation speeds; secondly, they can face instabilities during training when using a large number of layers, needed for effective modelling; third, they cannot accommodate time-varying boundary conditions; and fourth, they require initial conditions from a numerical solver. To address these issues, we propose a multi-scale hydraulic-based graph neural network (mSWE-GNN) that models the flood at different resolutions and propagation speeds. We include time-varying boundary conditions via ghost cells, which enforce the solution at the domain’s boundary and drop the need for a numerical solver for the initial conditions. To improve generalization over unseen meshes and reduce the data demand, we use invariance principles and make the inputs independent from coordinates' rotations. Numerical results applied to dike-breach floods show that the model predicts the full spatio-temporal simulation of the flood over unseen irregular meshes, topographies, and time-varying boundary conditions, with mean absolute errors in time of 0.05 m for water depths and 0.003 m2 s−1 for unit discharges. We further corroborate the mSWE-GNN in a realistic case study in the Netherlands and show generalization capabilities with only one fine-tuning sample, with mean absolute errors of 0.12 m for water depth, a critical success index for a water depth threshold of 0.05 m of 87.68 %, and speed-ups of over 700 times. Overall, the approach opens up several avenues for probabilistic analyses of realistic configurations and flood scenarios. ...
The steady state of a water distribution system abides by the laws of mass and energy conservation. Hydraulic solvers, such as the one used by EPANET approach the simulation for a given topology with a Newton-Raphson algorithm. However, iterative approximation involves a matrix inversion which acts as a computational bottleneck and may significantly slow down the process. In this work, we propose to rethink the current approach for steady state estimation to leverage the recent advancements in Graphics Processing Unit (GPU) hardware. Modern GPUs enhance matrix multiplication and enable memory-efficient sparse matrix operations, allowing for massive parallelization. Such features are particularly beneficial for state estimation in infrastructure networks, which are characterized by sparse connectivity between system elements. To realize this approach and tap into the potential of GPU-enhanced parallelization, we reformulate the problem as a diffusion process on the edges of a graph. Edge-based diffusion is inherently related to conservation laws governing a water distribution system. Using a numerical approximation scheme, the diffusion leads to a state of the system that satisfies mass and energy conservation principles. Using existing benchmark water distribution systems, we show that the proposed method allows parallelizing thousands of hydraulic simulations simultaneously with very high accuracy. ...
Journal article (2025) - Tianlong Jia, Riccardo Taormina, Rinze de Vries, Zoran Kapelan, Tim H.M. van Emmerik, Paul Vriend, Imke Okkerman
Supervised deep learning methods have been widely employed to detect floating macroplastic litter (>5 mm) in (fresh)water bodies. However, few studies used them to quantify floating litter fluxes in rivers with wide cross-sections, that is important for pollution assessment. Additionally, commonly used supervised learning (SL) models rely on extensive labeled data, that is time-consuming and expensive to obtain. Moreover, regardless of the model type, current deep learning models for litter detection usually fail to correctly identify small litter items. To overcome these issues, we propose a semi-supervised learning (SSL)-based framework combined with Slicing Aided Hyper Inference (SAHI) for quantifying cross-sectional floating litter fluxes in rivers. The framework includes four steps: (a) collecting camera images of river surfaces from multiple locations across the river, (b) developing a robust litter detection model using SSL, (c) applying this model with SAHI to detect litter items in images, and (d) post-processing the detection results to quantify fluxes. The SSL method involves: (i) self-supervised pre-training of a ResNet50 on a large amount of unlabeled data, and (ii) supervised fine-tuning of a Faster R-CNN with the ResNet50 backbone on a limited amount of labeled data. We evaluated the in-domain detection performance of SSL models with varying pre-training epochs and pre-training dataset sizes, using images from waterways of The Netherlands, Indonesia and Vietnam, that were used for model pre-training and fine-tuning. Additionally, we assessed the zero-shot out-of-domain detection performance of SSL models and litter flux quantification performance of the proposed framework on a Vietnam case study, that was not used for model development. We benchmarked our results against the SL methods and human visual counting. The results show that SSL models benefit from longer pre-training time and larger pre-training dataset, achieving an in-domain F1-score increase of 0.2 and a zero-shot out-of-domain increase of up to 0.14, over baseline SL benchmarks. Furthermore, the SAHI method correctly identifies 45 additional small litter items (areas < 1,000 cm2), improving the F1-score by up to 0.19, compared to the results obtained without SAHI. The flux measurement results indicate that the SSL-based framework substantially underestimates fluxes by a factor of 3–4 compared to human measurements, due to missed detections of transparent litter items and items entrapped in water hyacinths. However, it estimates nearly twice the fluxes of the baseline SL-based framework, aligning more closely with human measurements. These findings highlight the potential of SSL-based framework to enhance litter flux measurement. Scaling it with broader datasets could significantly advance global-scale litter monitoring systems. ...

A conditional Generative Adversarial Network for geotechnical subsurface schematisation

Journal article (2025) - F. A. Campos Montero, B. Zuada Coelho, E. Smyrniou, R. Taormina, P. J. Vardon
Subsurface schematisations are a crucial geotechnical problem which generally consists of filling substantial gaps in subsurface information from the limited site investigation data available and relying heavily on the engineer’s experience and occasionally geostatistical tools. To address this, schemaGAN, a conditional Generative Adversarial Network (GAN) to generate geotechnical subsurface schematisations from site investigation data is introduced. This novel method can learn complex underlying rules that govern the subsurface geometries and anisotropy from a big database of training cross-sections, and can produce subsurface schematisations from Cone Penetration Tests (CPT) in an insignificant timeframe. To test and demonstrate the performance of schemaGAN, a database of 24,000 synthetic geotechnical cross-sections with their corresponding CPT data was created, including spatial variability and gradually spatially varying layers. After training, the effectiveness of schemaGAN was compared against several interpolation methods, and it is seen that schemaGAN outperforms all other methods, with results characterised by clear layer boundaries and an accurate representation of anisotropy within the layers. SchemaGAN’s superior performance was confirmed through a blind survey, and in two real case studies in the Netherlands, where the model demonstrates better predictive accuracy for known CPT data. ...
Abstract (2025) - Antonio Magherini, Erik Mosselman, Victor Chavarrias Borras, Riccardo Taormina
Braided rivers are the most dynamic type of rivers, with a rapid and intricate morphological evolution. A limited understanding and inadequate algorithm implementation of specific morphological processes limits the prediction capabilities of physics-based models. The design of structures, infrastructure, and other interventions is consequently hampered. In recent years artificial intelligence (AI) techniques rapidly gained popularity across different contexts. Additionally, the availability of satellite images increased. This research sets a novel attempt to predict the planform evolution of braided rivers by means of deep learning and satellite images. The Brahmaputra-Jamuna River, in India and Bangladesh, was selected as case study. A convolutional neural network (CNN) with U-Net architecture was developed. The model was trained with the Global Surface Water Dataset (GSWD). The goal of the model was to classify each pixel as either "Non-water" or "Water". Four images, representative of the same month over four consecutive years, were used as input. The fifth-year image represented the target. The model demonstrated good skills in predicting the planform development. Processes like the migration of meanders, the abandonment of channels, and the evolution of confluences and bifurcations were often well captured. However, a lack of temporal patterns was noticed. More complex phenomena, like the formation and shifting of channels, were never predicted. The total areas of erosion and deposition were constantly underpredicted. Metrics such as precision, recall, F1-score, and critical success index (CSI) were tracked. Overall, our model achieved a 5-6% total improvement of these metrics compared to the benchmark method for which no morphological change is assumed to occur. Our model could be useful as a preliminary tool for water management authorities in India and Bangladesh. It can support the prioritisation of bank protection measures in areas subject to erosion or land reclamation projects in areas subject to deposition and assist inland navigation. Given the inherent tendency of the model to underpredict erosion, caution is always advised. More research is required to improve the current model. Despite this, deep-learning modelling could become a potentially valuable field of research. Testing alternative model architectures, increasing the datasets size, and incorporating additional data, such as water levels or river discharge, are some of the proposed strategies to improve the model performance. ...
Conference paper (2025) - Antonio Magherini, Erik Mosselman, Víctor Chavarrías, Riccardo Taormina
Braided rivers are the most dynamic type of rivers, with a rapid and intricate morphological evolution (Stecca et al., 2019). Being able to predict where and how rivers evolve is crucial for supporting spatial-related decisionmaking processes in the vicinity of these rivers. However, a limited understanding and inadequate algorithm implementation of specific morphological processes limits the prediction capabilities of physics-based models (Jagers, 2003; Siviglia and Crosato, 2016). The design of structures, infrastructure, and other interventions is consequently hampered at the expenses of the popoulation safety. In recent years artificial intelligence techniques rapidly gained popularity across different contexts (Blake et al., 2021) and the availability of satellite images increased. This research sets a novel attempt to predict the planform evolution of braided rivers by means of a deeplearning algorithm and using satellite images. The Brahmaputra-Jamuna River, in India and Bangladesh, was selected as case study (Best et al., 2022). ...
Journal article (2024) - J. J. Grosfeld, M. M. Schoor, R. Taormina, W. M.J. Luxemburg, F. P.L. Collas
Current research on riverine macrolitter does not yet provide a theoretic framework on the dynamics behind its accumulation and distribution along riverbanks. In an attempt to better understand these dynamics a detailed field survey of three months was conducted in which location of macrolitter items within a single groyne field along the Waal riverbanks was tracked. The data provided insight into the daily changing patterns of spatial item distribution with respect to the waterline. Furthermore, the rates of item uptake and deposition were monitored and related to hydrologic fluctuations. Uptake was initiated by rising water levels and was generally higher when the water level increased faster. Deposition occurred continuously, despite hydrologic fluctuations. This caused the riverbank macrolitter budget to be positive during stable or dropping water levels and negative during rising water levels. Although the results show clear patterns an extended monitoring duration is required to fully understand the fate of plastic objects. ...
Journal article (2024) - Tianlong Jia, Rinze de Vries, Zoran Kapelan, Tim H.M. van Emmerik, Riccardo Taormina
Researchers and practitioners have extensively utilized supervised Deep Learning methods to quantify floating litter in rivers and canals. These methods require the availability of large amount of labeled data for training. The labeling work is expensive and laborious, resulting in small open datasets available in the field compared to the comprehensive datasets for computer vision, e.g., ImageNet. Fine-tuning models pre-trained on these larger datasets helps improve litter detection performances and reduces data requirements. Yet, the effectiveness of using features learned from generic datasets is limited in large-scale monitoring, where automated detection must adapt across different locations, environmental conditions, and sensor settings. To address this issue, we propose a two-stage semi-supervised learning method to detect floating litter based on the Swapping Assignments between multiple Views of the same image (SwAV). SwAV is a self-supervised learning approach that learns the underlying feature representation from unlabeled data. In the first stage, we used SwAV to pre-train a ResNet50 backbone architecture on about 100k unlabeled images. In the second stage, we added new layers to the pre-trained ResNet50 to create a Faster R-CNN architecture, and fine-tuned it with a limited number of labeled images (≈1.8k images with 2.6k annotated litter items). We developed and validated our semi-supervised floating litter detection methodology for images collected in canals and waterways of Delft (the Netherlands) and Jakarta (Indonesia). We tested for out-of-domain generalization performances in a zero-shot fashion using additional data from Ho Chi Minh City (Vietnam), Amsterdam and Groningen (the Netherlands). We benchmarked our results against the same Faster R-CNN architecture trained via supervised learning alone by fine-tuning ImageNet pre-trained weights. The findings indicate that the semi-supervised learning method matches or surpasses the supervised learning benchmark when tested on new images from the same training locations. We measured better performances when little data (≈200 images with about 300 annotated litter items) is available for fine-tuning and with respect to reducing false positive predictions. More importantly, the proposed approach demonstrates clear superiority for generalization on the unseen locations, with improvements in average precision of up to 12.7%. We attribute this superior performance to the more effective high-level feature extraction from SwAV pre-training from relevant unlabeled images. Our findings highlight a promising direction to leverage semi-supervised learning for developing foundational models, which have revolutionized artificial intelligence applications in most fields. By scaling our proposed approach with more data and compute, we can make significant strides in monitoring to address the global challenge of litter pollution in water bodies. ...
Journal article (2024) - Bulat Kerimov, Riccardo Taormina, Franz Tscheikner-Gratl
Data-driven metamodels reproduce the input-output mapping of physics-based models while significantly reducing simulation times. Such techniques are widely used in the design, control, and optimization of water distribution systems. Recent research highlights the potential of metamodels based on Graph Neural Networks as they efficiently leverage graph-structured characteristics of water distribution systems. Furthermore, these metamodels possess inductive biases that facilitate generalization to unseen topologies. Transferable metamodels are particularly advantageous for problems that require an efficient evaluation of many alternative layouts or when training data is scarce. However, the transferability of metamodels based on GNNs remains limited, due to the lack of representation of physical processes that occur on edge level, i.e. pipes. To address this limitation, our work introduces Edge-Based Graph Neural Networks, which extend the set of inductive biases and represent link-level processes in more detail than traditional Graph Neural Networks. Such an architecture is theoretically related to the constraints of mass conservation at the junctions. To verify our approach, we test the suitability of the edge-based network to estimate pipe flowrates and nodal pressures emulating steady-state EPANET simulations. We first compare the effectiveness of the metamodels on several benchmark water distribution systems against Graph Neural Networks. Then, we explore transferability by evaluating the performance on unseen systems. For each configuration, we calculate model performance metrics, such as coefficient of determination and speed-up with respect to the original numerical model. Our results show that the proposed method captures the pipe-level physical processes more accurately than node-based models. When tested on unseen water networks with a similar distribution of demands, our model retains a good generalization performance with a coefficient of determination of up to 0.98 for flowrates and up to 0.95 for predicted heads. Further developments could include simultaneous derivation of pressures and flowrates. ...
Storm water systems (SWSs) are essential infrastructure providing multiple services including environmental protection and flood prevention. Typically, utility companies rely on computer simulators to properly design, operate, and manage SWSs. However, multiple applications in SWSs are highly time-consuming. Researchers have resorted to cheaper-to-run models, i.e. metamodels, as alternatives of computationally expensive models. With the recent surge in artificial intelligence applications, machine learning has become a key approach for metamodelling urban water networks. Specifically, deep learning methods, such as feed-forward neural networks, have gained importance in this context. However, these methods require generating a sufficiently large database of examples and training their internal parameters. Both processes defeat the purpose of using a metamodel, i.e., saving time. To overcome this issue, this research focuses on the application of inductive biases and transfer learning for creating SWS metamodels which require less data and retain high performance when used elsewhere. In particular, this study proposes an auto-regressive graph neural network metamodel of the Storm Water Management Model (SWMM) from the Environmental Protection Agency (EPA) for estimating hydraulic heads. The results indicate that the proposed metamodel requires a smaller number of examples to reach high accuracy and speed-up, in comparison to fully connected neural networks. Furthermore, the metamodel shows transferability as it can be used to predict hydraulic heads with high accuracy on unseen parts of the network. This work presents a novel approach that benefits both urban drainage practitioners and water network modeling researchers. The proposed metamodel can help practitioners on the planning, operation, and maintenance of their systems by offering an efficient metamodel of SWMM for computationally intensive tasks like optimization and Monte Carlo analyses. Researchers can leverage the current metamodel’s structure for developing new surrogate model architectures tailored to their specific needs or start paving the way for more general foundation metamodels of urban drainage systems. ...

Implications for root zone storage and streamflow predictions

Journal article (2024) - Nienke Tempel, Laurène Bouaziz, Riccardo Taormina, Ellis van Noppen, Jasper Stam, Eric Sprokkereef, Markus Hrachowitz
This paper investigates the influence of multi-decadal climatic variability on the temporal evolution of root zone storage capacities (Sr,max) and its implications for streamflow predictions in the Meuse basin. Through a comprehensive analysis of 286 catchments across Europe and the US that are hydro-climatically comparable to the Meuse basin, we construct inter-decadal distributions of past deviations in evaporative ratios (IE) from expected values based on catchment aridity (IA). These distributions of ΔIE were then used to estimate inter-decadal changes in Sr,max and to quantify the associated consequences for streamflow predictions in the Meuse basin. Our findings reveal that, while catchments do not strictly adhere to their specific parametric Budyko curves over time, the deviations in IE are generally very minor, with an average ΔIE=0.01 and an interquartile range (IQR) of −0.01 to 0.03. Consequently, these minor deviations lead to limited inter-decadal changes in Sr,max, mostly ranging between −10 and +21 mm (−5 % to +10 %). When these changes (ΔSr,max) are accounted for in hydrological models, the impact on streamflow predictions in the Meuse basin is found to be marginal, with the most significant shifts in monthly evaporation and streamflow not exceeding 4 % and 12 %, respectively. Our study underscores the utility of parametric Budyko-style equations for first-order estimates of future Sr,max in hydrological models, even in the face of climate change and variability. This research contributes to a more nuanced understanding of hydrological responses to changing climatic conditions and offers valuable insights for future climate impact studies in hydrology. ...
Conference paper (2024) - Alessandro Erba, Andres F. Murillo, Riccardo Taormina, Stefano Galelli, Nils Ole Tippenhauer
In recent years, a number of evasion attacks for Industrial Control Systems have been proposed. During an evasion attack, the attacker attempts to hide ongoing process anomalies to avoid anomaly detection. Examples of such attacks range from replay attacks to adversarial machine learning techniques. Those attacks generally are applied to existing datasets with normal and anomalous data, to which the evasion attacks are added post-hoc. This represents a very strong attacker, who is effectively able to observe and manipulate data from anywhere in the system, in real-time, with zero processing delay, and no computational constraints. Prior work has shown that such strong attackers are theoretically difficult to detect by most existing countermeasures. So far, it is unclear if such an attack could be practically realized, and if there are challenges that would impair the attacker. In this work, we systematically discuss options for an attacker to mount evasion attacks in real-world ICS, and show the constraints that result from those options. To validate our findings, we design and implement a framework that allows the realization of evasion attacks and anomaly detection for ICS emulation. We demonstrate practical constraints that arise from different settings, and their effect on attack performance. For example, we found that network packet replay might trigger network errors, which will result in unexpected spoofing patterns. ...
Journal article (2024) - Bulat Kerimov, Vincent Pons, Spyros Pritsis, Riccardo Taormina, Franz Tscheikner-Gratl
The operation of water distribution systems is based on reliable knowledge about the steady state of the system. This involves sensors to measure flow, facilitating a comprehensive overview of the system’s performance. Given the costs associated with sensor installation and operation, it is important to be strategic with sensor allocation. Recently developed Gaussian Processes with topological kernels can efficiently model mass and energy conservative flows and provide uncertainty bounds. Our work proposes a novel method of state estimation and a greedy search algorithm for water flow meter placement based on the uncertainty bounds provided by a Gaussian Process. ...

A Data-Efficient GNN Metamodel for SWMM Flowrates

Computational models for water resources often experience slow execution times, limiting their application. Metamodels, especially those based on machine learning, offer a promising alternative. Our research extends a prior Graph Neural Network (GNN) metamodel for the Storm Water Management Model (SWMM), which efficiently learns with less data and generalizes to new UDS sections via transfer learning. We extend the metamodel’s functioning by adding flowrate prediction, crucial for assessing water quality and flooding risks. Using an Encoder–Processor–Decoder architecture, the metamodel displays high accuracy on the simulated time series. Future work is aimed at incorporating more physical principles and testing further transferability. ...
Large Multimodal Models are emerging general AI models capable of processing and analyzing diverse data streams, including text, imagery, and sequential data. This paper explores the possibility of exploiting multimodality to develop more interpretable AI-based predictive tools for the water sector, with a first application for sewer defect detection from CCTV imagery. To this aim, we test the zero-shot generalization performance of three generalist large language-vision models for binary sewer defect detection on a subset of the SewerML dataset. We compared the LMMs against a state-of-the-art unimodal Deep Learning approach which has been trained and validated on >1 million SewerML images. Unsurprisingly, the chosen benchmark showcases the best performances, with an overall F1 Score of 0.80. Nonetheless, OpenAI GPT4-V demonstrates relatively good performances with an overall F1 Score of 0.61, displaying equal or better results than the benchmark for some defect classes. Furthermore, GPT4-V often provides text descriptions aligned with the provided prediction, accurately describing the rationale behind a certain decision. Similarly, GPT4-V displays interesting emerging behaviors for trustworthiness, such as refusing to classify images that are too blurred or unclear. Despite the significantly lower performance from the open-source models CogVLM and LLaVA, some preliminary successes suggest good potential for enhancement through fine-tuning, agentic workflows, or retrieval-augmented generation. ...
Journal article (2024) - Marie-Philine Gross, Riccardo Taormina, Andrea Cominola
Recent research highlights the potential of consumption-based feedback for water conservation, emphasizing the need for Non Intrusive Water Monitoring (NIWM). However, existing NIWM studies often rely on small datasets, a pre-selected class of models, and inaccessible software. Here, we introduce PyNIWM, a machine learning-based open-source Python framework for NIWM. PyNIWM enables water end-use classification via (i) data characterization and feature engineering, (ii) water end-use event classification with four machine learning classifiers, and (iii) performance assessment. We demonstrate PyNIWM on a real-world dataset containing around 800,000 labeled end-use events from 762 homes across the USA and Canada. The four PyNIWM classifiers achieve F1 scores above 0.85, indicating high suitability for water end-use classification. However, a tradeoff between accuracy and computational cost exists. Finally, data balancing through oversampling enhances classification of low-represented end-use classes, but does not improve overall classification. We release PyNIWM as an open-source software, aiming for collaborative and reproducible research. ...