Summary
This article delves into the latest update from Stability AI, the Stable Diffusion, which promises to provide a significant leap in the field of generative AI by being capable of creating short-form videos. The article further explores this technology’s capabilities, limitations, and potential future developments.
Stability AI, the pioneering developer behind Stable Diffusion, unveils a new generative AI that has the potential to revolutionize short-form video creation. Named Stable Video Diffusion, this AI system comprises two models, SVD and SVD-XT, and flaunts the ability to generate clips at a resolution of 576×1024 pixels.
Stable Video Diffusion allows the user to customize the frame rate speed, ranging from three to 30 FPS. The video length depends on the model chosen. SVD produces content playing for 14 frames, while the SVD-XT model extends the length to 25 frames. However, as confirmed on the official Hugging Face listing, these durations do not significantly affect the rendered clips, which play for approximately four seconds before ending.
“Every great advance in science has issued from a new audacity of imagination.” – John Dewey.
Capabilities and Demonstrations
Stability AI recently showcased the capabilities of Stable Video Diffusion on its YouTube channel. The quality of the generated content, particularly the Ice Dragon demo, impressed viewers with its detailed dragon scales and picturesque mountainous backdrop. However, the animations were relatively limited, presenting either a slow panning shot or a stiff walking cycle.
Limitations
Despite its impressive capabilities, Stable Video Diffusion has its limitations. It reportedly struggles to achieve perfect photorealism, generate legible text, and render faces. However, one demonstration from Stability AI shows the model successfully causing a man’s face without any noticeable flaws, suggesting that successful face rendering may be case-dependent.
Early Stages and Future
As an early-stage project, Stable Video Diffusion is not yet ready for an extensive release. Stability AI clarifies that the model is not intended for real-world or commercial applications but is designed for research purposes. This cautious approach from Stability AI is understandable, especially in light of an incident last year where their model leaked online, leading to misuse by malicious actors to create deep fake images.
Availability and Preview
For those interested in experiencing Stable Video Diffusion, Stability AI offers a waitlist via a form on their website. The preview will include a Text-To-Video interface. While the admission timeline is not specified, interested parties can utilize this waiting period to delve into the AI’s white paper and gain an in-depth understanding of the project.
Training Material and Legalities
The white paper interestingly mentions using “publicly accessible video datasets” as part of the training material, which is unsurprising considering that Stability AI faced a lawsuit from Getty Images for data scraping allegations earlier this year. It appears that the team is making concerted efforts to be more meticulous to avoid any future legal issues.
Share the Article by the Short Url:
