Stability AI, a leader in generative AI models, has recently released Stable Video 3D (SV3D), a new tool for rendering 3D video. Building upon Stability AI’s previous Stable Video Diffusion model, SV3D allows users to generate and transform multi-view 3D meshes from a single input image. This new technology is set to revolutionize the gaming sector and enhance the e-commerce shopping experience.
Stable Video Diffusion, developed by Stability AI, enables users to generate short videos from an image or text prompt. With SV3D, Stability AI takes this capability further by incorporating novel view synthesis and 3D generation. By adapting the Stable Video Diffusion model and adding camera path conditioning, SV3D can create multi-view videos of an object, providing a more immersive and interactive experience.
One of the key applications of SV3D is within the gaming sector. Varun Jampani, lead researcher at Stability AI, highlights the value of SV3D in generating 3D assets for game creation. The ability to generate 360-degree orbital videos also enhances the e-commerce industry, offering a more immersive shopping experience for consumers.
Stability AI is known for its Stable Diffusion text-to-image generative AI models, such as SDXL and Stable Diffusion 3.0. The release of Stable Zero123 in December 2023 marked the beginning of a series of 3D models from Stability AI. SV3D is a successor to Stable Zero123, taking a different approach to 3D generation.
Unlike Stable Zero123, which outputs one image at a time, SV3D generates multiple novel views simultaneously. This results in higher quality novel views and better 3D meshes from a single input image. Stability AI researchers detail the techniques used in SV3D to enable 3D generation from a single image using latent video diffusion. The ability to generate consistent novel multi-view images of an object sets SV3D apart from previous approaches, which often struggle with limited perspectives and inconsistencies.
SV3D also focuses on optimizing 3D meshes. By leveraging its multi-view consistency, SV3D can generate high-quality 3D meshes directly from the novel views it produces. This optimization improves the overall quality of the 3D meshes and enhances the final output.
SV3D comes in two variants: SV3D_u and SV3D_p. SV3D_u generates orbital videos based on single image inputs without the need for camera conditioning. On the other hand, SV3D_p accommodates both single images and orbital views, allowing users to create 3D videos along specified camera paths.
Stability AI has made SV3D available for commercial use with a Stability AI Professional Membership. Creators and developers with less than $1 million in annual revenue can access SV3D for $20 per month. For non-commercial purposes, users can download the model weights from Hugging Face.
In conclusion, Stability AI’s release of Stable Video 3D (SV3D) brings new possibilities to the world of video stability and 3D generation. With its ability to create and transform multi-view 3D meshes from a single input image, SV3D is set to revolutionize the gaming sector and enhance the e-commerce shopping experience. The advancements in novel view synthesis and optimization of 3D meshes make SV3D a valuable tool for creators and developers. As Stability AI continues to innovate in the field of generative AI models, SV3D stands out as a powerful addition to their portfolio.
