#viduq2onstage

[deepdreamnights]

Disclosure: I am in the Vidu artists program, pretty sure this is technically sponsored content. This is a fixed version of the original post.

Of course, it's another Fauxstalgia Fake-Em-Up from yours truly, this time demonstrating the beta-access potential of the new Vidu Q2 reference-to-video mode (get 100 free credits if you click here to try it out for free).

So, lets talk about how it was done (an overview)

GIFs are unedited wherever possible to show raw results.

The big problem with using AI for anything actually productive is inconsistency. While there's always wobble and bad gens, Vidu's reference-to-video option is the best solution I've seen thus far.

Essentially, you build a little profile with up to three reference pics and a basic prompt for your characters, props, etc. From then on out you can use the reference in your prompts to generate new scenes. It decreases confusion about what character is what, and it lets you establish styles of movement and other effects characters should always have going without having to use prompt-space in every scene.

As you can see with Tilly Tepesh's jacket and the ghost of her uncle Drac's translucency, and with the fact that I can have four different kinds of near-identical blokroid in the same scene without them morphing into each other or making weird hybrids.

Images can also be used with ref-to-video without making a full reference profile. For the various Drac-and-Tilly playing scenes the robot kept making filly play the back of the keyboard and spaced them to far apart, so I composited a demonstration image together to help guide the process.

Which helped a lot.

I used a variation on my typical asset-creation process on this (tutorials), utilizing a lot of early gens and ref-to-images in Vidu's system (tutorial) to bulk up the process, along with Midjourney, Sora, and Civit.

For the audio, I recorded the lines myself, then loaded them into Suno to modify my voice into something more announcer-like and bring the music in under it.

Visit Original Link

All the editing, compositing, logos, etc were done the old fashioned way.

Quality Improvements (also an overview)

Basics:

Up to 8 second gens.

4:3 and 3:4 aspect ratios (in addition to 1:1, 9:16 and 16:9)

General improvements to quality, coherence, and prompt understanding.

Extend clips multiple times up to 5 mins (launched after the video above was finished)

While I'll get into the details of the improvements to quality as well as a number of other new features (up to 8 second generations, clip extending, and sound (very early on that one)) in other posts, those who have followed TyrannoMax may remember the issue with boulders.

Well, after a lot of coaching and getting Max a new trailer, he's finally stopped screwing around on set.

In short, a lot of things that just weren't doable before now work, and the general quality of gens is higher and sharper than ever before.

What I was not expecting was for it to emulate Gerry Anderson-style puppetry well. Ironically, the kind of jank you get from pre-digital media is hard for AI to duplicate. If you want something to look polished, smooth, and modern, that's easy.

It did take a lot of prompting for non-moving faces/immobile dolls, and editing is always needed, but the differences between Q1 and Q2 is apparent when you compare the same prompt in both:

The deadly starmantis gingerly setting the Queen Seltza prop down is adorable, but not what the director called for.

All in all, it took me about a week to go from concept to video, mostly on the back of my needing to make base character, prop, and set assets for everything that wasn't TyrannoMax more or less from scratch. If you want to give it a try yourself, here's that link again, and I'm posting a bunch of tutorials, old and (hopefully) new this week.

And as always, like, comment and subscribe.

[melinoelabs]

KDMU After-Cool Adventure Block Promo (8-20-87)

A friend of mine at Buzby-Spurlock sent me a rare surviving KDMU bumper that was in an old box of materials sent in for approvals. I remember afternoons watching the Omnistation before mom and dad got home and made me change the channel because the monsters on ROM gave me nightmares.

Pretty sure I've still got my ScardeyHoot stuffed animal in mom's attic, but the tail got too loose to hold him up after a couple of years of me dragging him around by it.

Let me know if this brings back memories for any of you guys.

Humbert Outreach Liaison

[deepdreamnights]

Using Ref-to-Video With Vidu

Disclosure: I'm in the Vidu Artist's Program

This is a long one, have a listen while you read:

Visit Original Link

One of the ongoing limitations of genAI is consistency. Image prompting and img-2-vid start frames are helpful, but there's always a part of the character not shown that the AI had to interpolate.

Vidu addresses this problem through the use of ref-to-video/ref-to-image.

Either a single-use image or a profile of up to three images and a descriptive prompt take the place of a noun in your video prompt. It's that thing I've wanted since 2023! Neat!

This method was used to make the vast majority of my Radio Free Ultramerica videos.

I've covered how to use the basic functions of ref-to-vid in other posts. So this will be more about how to make a good character reference.

Quick Tips:

Keep your character prompts backed up in a separate document. Whenever you change the images in the Create Reference menu, the system auto-generates an updated prompt, so backing up your edited ones is essential.

Avoid transparent PNGs. If there's any white haloing around the clipping, it can cause artifacts.

More reference is good reference. A single reference image can work depending on the project, but ideally, you want at least a front view, a side or back view, and a face closeup

Fortunately, if we've got one piece of reference, we can use it to make others. So, lets build us a monster. A robot monster! (no, not that one)

This is my Thanorite, a water heater with legs I doodled for a pulp RPG campaign that never quite materialized. Sadly I never scanned his raw lines and that black is a pain to remove manually, so it's off to (shudder) ChatGPT to get an animation cell style interpretation. As loathe as I am to admit it, ChatGPT/Sora is currently the best at 'give me this in this style' without massive reinterpretations.

It also, however, sees through an amber-colored lens. Everything ChatGPT generates is more yellowed and darker than it ought to be, so it's off to photoshop. And while I'm at it, lets get a more interesting colors scheme and restore the plasma bubbles, and remove his gun, since asymmetry of limbs requires a lot more trial-and-error.

Next, I bring the Thanorite into Vidu, and set up a ref-2-vid instructing the character to turn around, and advising what their back should look like. I'm doing this as a video rather than an image because vidu's ref-2-img strongly biases the input image, and is reluctant to turn characters around in its current version.

That part about 'full body, head and feet fully visible'? Very important. Otherwise it will tend toward a mid-closeup.

Vidu's got a built-in frame-cap for just this purpose, so I yank some frames, clean them up, and we're good to go:

Tutorial continues under fold:

Next we create a new reference with those pics. Here you assign a name, one of several or no voices (currently a very early-stages features), and a tag for sorting. Then a second screen where you have style and descriptions. I tend to re-write these, and you'll want to make sure the auto-caption didn't specify pose or background elements.

I went with-

Animation, traditional hand-drawn cel animation, sakuga

-for the style and -

A purple robot with a cylendrical body. Its faceplate shows a bubbling orange and yellow liquid. it has segmented tube arms and legs, with purple bands at the wrists and knees. The robots hands are green c-clamps. On its chest, there is a green panel with a vertical gauge and two circular dials. On its back, it has two purple tanks connected by a green pipe and a yellow circular valve. It moves with the plodding gait of a heavy machine. It stands 8 feet tall, and is an imposing, emotionless figure.

-for the description.

From there it's just a matter of using the character in a scene, which can be prompted like any other. If you need to refer to the character more than once in a prompt, you'll need to copy/paste their highlighted name.

You can combine multiple refrences, but performance gets iffy after 7 images. For a sizable project you'd likely want multiple references per character, with specific ones for dialog scenes, group shots, etc.

Give it a shot here, with an extra 100 credits to play around with.

[deepdreamnights]

Happy Halloween everyone.

[deepdreamnights]

E'GG!

Made with Vidu + Suno.

Process and tutorial under the fold.

This is a fun one.

I started by taking a chunk of Suno tune I hadn't finished, and laying over it various sounds seals calling for "egg" from a couple of vids and some thunder SFX, then continuing and modifying it in Suno into the final version. This was under 4.5, which just today went live for free users, so, hey, give it a shot.

From there, I used Vidu's IMG2Vid q2 model with the extension feature that would have been really helpful while making the Omnistation video, but it came out literally the day after I finished the video gens. :/

From there, I used Vidu's ref2img to make a batch of the same seal in just slightly different poses:

Now, theoretically I could have used one starting image and just varied what happened with text prompts, but to provide more control and to keep the seal relatively stable, I used this method. A similar process could be used to anchor concepts in any number of extended-shot videos, or to serve as a form of storyboard keyframing, as you can vary how much you extend from 1sec up to 7.

By choosing the end frame and the next starting frame, I was able to edit largely as I went, in the generator.

By this method I wound up with a 3 minute long video, but once I laid everything down I had to make some fixes to sections to line up timing with the song.

To do this, I'd take start and end frames from the full video, extract them, and then generate new clips that were either longer or shorter than the original to fill the appropriate gaps in places where adjusting the speed of the footage isn't viable.

This process is going to see a lot of use, I expect, in video production in the future. Flubs, too-short scenes, issues with timing, those can all be smoothed out in a couple of hours rather than with days of reshoots or CGI-team make-work.

For the laid-over space explosions, I made a series of mockup frames using movie screencaps and stock images of eggs and fish, photoshopped together.

Loading those into Vidu's ref-to-img system, I got a few explosions that I could then 'screen' over the footage to produce the backdrop for the thunderclaps.

In future lengthy-shot projects, for ease of editing, I'm probably going to manually stitch together 20-30 second clips for more direct control and timing flexibility, but being able to extend clips to multi-minute lengths definitely comes in handy.

Give vidu a shot yourself and get 100 bonus creds with Code: THEROBOTMONSTER