SW
Stephen Wang
info
Please Note
<p>This page displays the records of the person named above and is not linked to a unique person identifier. This record may need to be merged to a profile.</p>
3 records found
1
From thought to visual composition
A brain-driven visual blends technique for visual blending tasks
Visual blends is a design technique that combines elements from multiple images into harmonious compositions and has been increasingly explored as a means to support early-stage ideation in engineering design. However, existing blending workflows rely heavily on manual image selection and composition, making the process difficult, time-consuming, and skill-intensive for designers. In this work, we present a proof-of-concept brain-guided visual blends technique that integrates an EEG-to-image model to simplify the image acquisition process and a local image editing model to enable automated and controllable image composition. Our EEG-to-image model employs a two-stage training strategy, combining pretraining on large-scale unlabelled EEG data with fine-tuning in an EEG-conditioned diffusion model, achieving state-of-the-art performance in reconstructing visual stimuli. To support visual blending tasks, we incorporate a local editing model (Paint-by-Example) that generates coherent blends using user-provided masks, reference images, and backgrounds. A user study with 15 participants demonstrated that the model effectively supported the creation of visual blends that aligned with users' design vision, even without artistic skills. The results suggest that brain-guided blending can serve as a early-stage ideation interface in engineering design, helping designers iterate on mental concepts before formal modelling and evaluation.
...
Visual blends is a design technique that combines elements from multiple images into harmonious compositions and has been increasingly explored as a means to support early-stage ideation in engineering design. However, existing blending workflows rely heavily on manual image selection and composition, making the process difficult, time-consuming, and skill-intensive for designers. In this work, we present a proof-of-concept brain-guided visual blends technique that integrates an EEG-to-image model to simplify the image acquisition process and a local image editing model to enable automated and controllable image composition. Our EEG-to-image model employs a two-stage training strategy, combining pretraining on large-scale unlabelled EEG data with fine-tuning in an EEG-conditioned diffusion model, achieving state-of-the-art performance in reconstructing visual stimuli. To support visual blending tasks, we incorporate a local editing model (Paint-by-Example) that generates coherent blends using user-provided masks, reference images, and backgrounds. A user study with 15 participants demonstrated that the model effectively supported the creation of visual blends that aligned with users' design vision, even without artistic skills. The results suggest that brain-guided blending can serve as a early-stage ideation interface in engineering design, helping designers iterate on mental concepts before formal modelling and evaluation.
Journal article
(2025)
-
Pan Wang, Xun Zhang, Liyan Wei, Peter Childs, Stephen Jia Wang, Yike Guo, Maaike Kleinsmann
Ideation is a critical step in the engineering design process, enabling designers to develop creative and innovative concepts and prototypes. Currently, the ideation workflow requires designers to generate new designs based on product requirements, heavily relying on their personal expertise and experience. To advance human-AI collaboration design and assist designers in the idea-generation process, this paper proposes an Object Combination Generative Adversarial Network (OC-GAN) for combinational creativity. The proposed method includes an image encoder module and a cross-domain object combination generator module. The image encoder module captures and encodes image structure information into latent space, while the cross-domain object combination generator module leverages GANs to combine object images based on user preferences, producing new design images. A design case study is used to evaluate the new ideation approach and reveal not only strong cross-domain concept combination capabilities but also improvement in designers' workflow and provision of novelty to the design case. Highlights An AI approach to improve the efficiency of idea generation in the design process. A case study evaluates its support for idea generation and design creativity. The OC-GAN is used for multi-domain object image combining tasks. Exemplifies the feasibility of human-AI collaboration design for enhancing creativity.
...
Ideation is a critical step in the engineering design process, enabling designers to develop creative and innovative concepts and prototypes. Currently, the ideation workflow requires designers to generate new designs based on product requirements, heavily relying on their personal expertise and experience. To advance human-AI collaboration design and assist designers in the idea-generation process, this paper proposes an Object Combination Generative Adversarial Network (OC-GAN) for combinational creativity. The proposed method includes an image encoder module and a cross-domain object combination generator module. The image encoder module captures and encodes image structure information into latent space, while the cross-domain object combination generator module leverages GANs to combine object images based on user preferences, producing new design images. A design case study is used to evaluate the new ideation approach and reveal not only strong cross-domain concept combination capabilities but also improvement in designers' workflow and provision of novelty to the design case. Highlights An AI approach to improve the efficiency of idea generation in the design process. A case study evaluates its support for idea generation and design creativity. The OC-GAN is used for multi-domain object image combining tasks. Exemplifies the feasibility of human-AI collaboration design for enhancing creativity.
Conference paper
(2025)
-
Pan Wang, Xun Zhang, Zhibin Zhou, Peter Childs, Kunpyo Lee, Maaike Kleinsmann, Stephen Jia Wang
Typeface design plays a vital role in graphic and communication design. Different typefaces are suitable for different contexts and can convey different emotions and messages. Typeface design still relies on skilled designers to create unique styles for specific needs. Recently, generative adversarial networks (GANs) have been applied to typeface generation, but these methods face challenges due to the high annotation requirements of typeface generation datasets, which are difficult to obtain. Furthermore, machine-generated typefaces often fail to meet designers’ specific requirements, as dataset annotations limit the diversity of the generated typefaces. In response to these limitations in current typeface generation models, we propose an alternative approach to the task. Instead of relying on dataset-provided annotations to define the typeface style vector, we introduce a transformer-based language model to learn the mapping between a typeface style description and the corresponding style vector. We evaluated the proposed model using both existing and newly created style descriptions. Results indicate that the model can generate high-quality, patent-free typefaces based on the input style descriptions provided by designers.
...
Typeface design plays a vital role in graphic and communication design. Different typefaces are suitable for different contexts and can convey different emotions and messages. Typeface design still relies on skilled designers to create unique styles for specific needs. Recently, generative adversarial networks (GANs) have been applied to typeface generation, but these methods face challenges due to the high annotation requirements of typeface generation datasets, which are difficult to obtain. Furthermore, machine-generated typefaces often fail to meet designers’ specific requirements, as dataset annotations limit the diversity of the generated typefaces. In response to these limitations in current typeface generation models, we propose an alternative approach to the task. Instead of relying on dataset-provided annotations to define the typeface style vector, we introduce a transformer-based language model to learn the mapping between a typeface style description and the corresponding style vector. We evaluated the proposed model using both existing and newly created style descriptions. Results indicate that the model can generate high-quality, patent-free typefaces based on the input style descriptions provided by designers.