Vidu 2.0 - First Reactions
I am in the Vidu Artist's program, so I've had a chance to play with version 2.0 before the official launch on the 15th. What I'm working with is a pre-launch build, and has improved day-to-day, so this may not reflect the final release.
I haven't yet had a chance to give it the full paces-run-through it deserves, but here's some early samples, and early thoughts. (Converted to GIF because you can only upload one video per post.)
The short version is that everything has been incrementally improved: Better coherence, better prompt responsiveness, better motion, and way, way better speed. Without doing exact time-tests it's say it's at least 25% the time to generate a video of the same dimensions.
While there's still some of the "smudge-blurring" that you got with 1-1.5, it happens less frequently, and is more mitigated with an image/animation that match.
Motion varies gen-by-gen, but impressive results seem to be the norm.
While his sticks are somewhat flexible at full framerate, the cat drummer's cymbal hit struck me as particularly nice.
Control and Coherence
While the roar may not seem particularly impressive, roars, howls, and other emotional outbursts didn't work well in previous versions. Aunt Acid's fumes and drips are are particularly fun, and while it still has problems with her tail, PteroDarla's crest and wings are actually working the way they should (after a number of attempts).
For a long time, I've wanted the last shot of the TMax opener to be Max starting with a zoom-in on the eye going out to a roar and pose. While this isn't quite where I want it, 2.0 is the first time I've gotten him to go through the whole sequence. Which is promising.
Weird Stuff Works
What remains impressive about Vidu is how well it handles concepts and characters that are off-the-beaten-path. Hailuo just released a character consistency feature that only works with humans, but here...
Here's my friend Cole's OC, the Waffler (Intergalactic Bounty-Hunter.) He's one unbalanced breakfast. He's also rather resistant gen AI replication because he's an SD space man with a waffle for head, a very specific waffle for a head turned at a 45 degree angle. Vidu 1.0 wasn't able to work with him, almost always giving him a mouth or rotating his waffle, if not completely glitching out. 2.0 is much better to handle it.
The numerous dino-anthros above are all in the "Tricky for AI" box. If I was into doing what could be gened easily, however, I'd just be pumping out an endless parade of pillowy waifus.
In my defense, I classify SexBomb as more of a 'strifu'. This particular one was an attempt to see if a toony image prompt could be rendered live-action with text prompting. Long story short it can't, but it can produce some interesting effects like the faux-posterized background.
I've had AI gen close to her costume before, but it never adds the fuse or does the boob-window right, and here we are.
One of my old bits of Transformers fanart of the Pretender Monster Icepick served as the character model for the one on the right.
Fantastical Creatures in general are a lot easier to execute in this version as well.
And... Action!
Motion is a lot more natural this time around.
Weapons fire (though sometimes a bit literal) tends to come out of the barrel semi-consistently now, characters can fight the waves without melting themselves and...
A giant rubber monster can eat your protagonist (if you're lucky.)
Quirks and Flaws
Nothing is perfect, and all AI you see is curated. So lets talk areas to be improved.
A lot of stuff presently generates with multilingual gibberish captions sometimes, which I expect is an early model bug. Versions 2-3 of Midjourney would have similar artifacts, and that sort of thing isn't hard to correct for.
There are still issues with blurring/smudging, especially with things like tail-tips, hands, and any motion the robot doesn't quite get.
Sometimes stuff shapechanges or appears that ought not to, like the knight's floppy second blade.
Or speed gets off requiring being fixed in post.
And stuff just goes dumb sometimes, which one should expect (and in my estimation, desire) from any generative system, artificial or analogue. Should the water go on the fire rather than the firemen? Yes. Do I regret this gen? No.
One quirk of the system is how it resolves incongruous multi-prompts. I've been accustomed to Midjourney, which, when generating an image must blend everything requested. You can put two completely different backgrounds in as image prompts and it will blend them into something new and wacky.
Vidu resolves problems like having two background images at once by taking advantage of the 4th dimension. Confuse the robot too much and it will just cut/fade from one idea to the next.
And then there's stuff that just happens, like, a shot being perfect except a painted (and thus ought to be static) background object animating beautifully (going retro is a path wrought with irony) and the robot deciding it'd rather do CG-style than 2d.
And while it doesn't show up great in the gif of Max at the construction site there, 2.0 is more vulnerable to interpreting bad transparency-clipping as part of the character design, so be careful if you're using transparent PNGs.
Also, if you slap a character and a background together without elaborating on the setting with the text prompt, it will often slap the background back there as a static backdrop and produce a very "greenscreen-y" effect.
Rather than laden this post down with more animated GIFs, I'll be setting up a batch of them as posts for the upcoming days. At least, that's the plan.