JR
J. Romeu Huidobro
info
Please Note
<p>This page displays the records of the person named above and is not linked to a unique person identifier. This record may need to be merged to a profile.</p>
2 records found
1
Recent advances in generative AI have enabled high-quality video generation from text prompts. However, the majority of existing approaches rely exclusively on prompts, making it difficult for an artist to control the generated scene layout and motion. In this thesis, we propose a novel method for geometry-guided Text to Video generation. Our method takes as input an animated mesh sequence and a text prompt and generates a video following both the text prompt and input geometry. Our pipeline consists of two main stages: Firstly, we use an existing text-driven texture generation method to create an initial rough texture for the geometry. Next, a depth-conditioned T2I model is used to generate video frames following the guidance animation, using the generated texture to enforce temporal consistency across frames. By generating video frames rather than directly using the result of the texture generation, our method supports generating deformations from the guidance geometry and variable lighting and by using the texture for feature alignment, we acheive significantly stronger robustness to occlusions and camera motion than existing controllable video-generation approaches. We begin by identifying the failure modes of existing methods through a set of initial experiments, we then use these findings to propose our method and finally evaluate it through a series of comparisons and ablations.
...
Recent advances in generative AI have enabled high-quality video generation from text prompts. However, the majority of existing approaches rely exclusively on prompts, making it difficult for an artist to control the generated scene layout and motion. In this thesis, we propose a novel method for geometry-guided Text to Video generation. Our method takes as input an animated mesh sequence and a text prompt and generates a video following both the text prompt and input geometry. Our pipeline consists of two main stages: Firstly, we use an existing text-driven texture generation method to create an initial rough texture for the geometry. Next, a depth-conditioned T2I model is used to generate video frames following the guidance animation, using the generated texture to enforce temporal consistency across frames. By generating video frames rather than directly using the result of the texture generation, our method supports generating deformations from the guidance geometry and variable lighting and by using the texture for feature alignment, we acheive significantly stronger robustness to occlusions and camera motion than existing controllable video-generation approaches. We begin by identifying the failure modes of existing methods through a set of initial experiments, we then use these findings to propose our method and finally evaluate it through a series of comparisons and ablations.
Direct lighting calculation is an essential part of photorealistic rendering. Standard importance sampling techniques converge slowly in scenes where a light source is only visible through small openings as visibility is not considered. This problem is often addressed by manually placing light portals, marking the openings to the light. However, existing portal sampling techniques are not suitable for area lights since the portal is often times larger than the light itself. We present a novel portal sampling technique inspired by shadow volumes, which considers the light geometry to efficiently choose the optimal sampling strategy, depending on the location of the shading point. Our technique is unbiased, robust, applicable to many scenes, and easy to integrate into an existing renderer.
...
Direct lighting calculation is an essential part of photorealistic rendering. Standard importance sampling techniques converge slowly in scenes where a light source is only visible through small openings as visibility is not considered. This problem is often addressed by manually placing light portals, marking the openings to the light. However, existing portal sampling techniques are not suitable for area lights since the portal is often times larger than the light itself. We present a novel portal sampling technique inspired by shadow volumes, which considers the light geometry to efficiently choose the optimal sampling strategy, depending on the location of the shading point. Our technique is unbiased, robust, applicable to many scenes, and easy to integrate into an existing renderer.