Feedforward 3D Shape Generation: Aligning Semantic Latents with Geometric Spaces

Master Thesis (2026)
Author(s)

X. Han (TU Delft - Electrical Engineering, Mathematics and Computer Science)

Contributor(s)

P. Kellnhofer – Mentor (TU Delft - Electrical Engineering, Mathematics and Computer Science)

X. Zhang – Graduation committee member (TU Delft - Electrical Engineering, Mathematics and Computer Science)

Faculty
Electrical Engineering, Mathematics and Computer Science
More Info
expand_more
Publication Year
2026
Language
English
Graduation Date
27-08-2026
Awarding Institution
Delft University of Technology
Programme
Computer Science, Data Science and Artificial Intelligence Technology
Faculty
Electrical Engineering, Mathematics and Computer Science
Page Views
74
Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Abstract

Current text-to-3D generation frameworks are bottlenecked by extreme computational demands, relying heavily on expensive test-time optimization or massive datasets to learn multi-modal alignments. This research investigates exploiting the highly structured semantic latent space of pre-trained Large Language Models (LLMs) to directly predict continuous 3D shapes, bypassing the need to build intricate alignment networks from scratch. To bridge the topological mismatch between language and 3D space, we propose a decoupled, two-stage architecture utilizing continuous Signed Distance Functions (SDFs). First, an auto-decoder optimization framework explicitly aligns the semantic text space with the geometric shape space to ensure high structural fidelity. Second, a mapping network learns the non-linear projection between the frozen LLM text latents and the optimized geometric latents. Extensive evaluations demonstrate that this decoupled paradigm prevents the geometric collapse observed in fully joint, end-to-end training, and confirms that pre-trained LLM embeddings serve as superior geometric priors compared to random initializations. Ultimately, this methodology establishes a stable, optimization-free inference pipeline, enabling feedforward synthesis of 3D geometries directly from raw textual prompts.

Files

Thesis_final_xingyu.pdf
(pdf | 57.4 Mb)
License info not available