Has science gone too far? Inspired by Janelle Shane (aka @lewisandquark), I’ve taught a neural network how to draw my cat Mulder. The results are ... delightfully off-kilter.
Here’s what the process started with:
Here’s the ALARMING midpoint:
And here, after 3,000 iterations, is the rather impressive result!
The process is actually easy enough a non-computer-geek like me could figure it out — there’s no coding involved, nor do you need a ton of computing power. All you need is a video of your pet (or whatever you want to see AI try to recreate), and two pieces of free software: VLC Media Player and RunwayML.
DETAILED TUTORIAL BELOW THE CUT... If you use it, please reblog this post with your results -- I’d love to see ‘em! Note: I’m a Windows user; the process for Mac might be slightly different at some steps.
Step One: Gather your dataset
In order to train an AI, you’ll need to feed it a sampling of 500-5,000 relevant images. Rather than take all those pictures individually, we’re going to use VLC Media Player to extract the frames from a video you take.
1. Record a roughly two-minute video of your subject (eg., your own cat, a mushroom, your mug collection, your own face if you’re really brave, whatever). The subject should be centered in the frame. For the best (most coherent) results, you’ll want good lighting and contrast, for your subject to remain relatively stationary.
2. Download the video onto your computer from Google Photos or whatever cloud service your phone backs up onto.
3. Download and install VLC Media Player. If you already have VLC Media Player, ensure you’re running the most recent version by clicking “Help” and then “Check for updates.”
4. Run VLC Media Player as administrator. (Right-click the shortcut on your desktop or in the start menu and select “Run as administrator.”)
5. Hit ctrl+P in VLC to open your preferences menu. In the bottom lefthand corner of the menu, you’ll see the words “Show Settings” and the options “Simple” and “All.” Select the “all” option. You should see this:
6. In the menu on the left, scroll down to the bottom and click on “Filters” (not the “>” sign next to Filters, but the word itself.) Find “Scene video filter” and tick the box, as shown:
7. On your computer, prepare the file where you want those frames to go once VLC extracts them. I just created a new file on my desktop named “Mulder frames.” Double-click the file to open it, then in the navigation bar, click the dropdown to show the whole file path. It’ll be something like C:\Users\elven\Desktop\Mulder frames. Ctrl+C it.
8. Back in VLC, in the menu on the left, expand the Video>Filters submenu by clicking the > button next to Filters. Scroll down til you see “Scene filter” and click that to open its settings. In the “image format” field, input “jpg” or “png”.
9. In the “directory path prefix” field, paste in the file path from above.
10. Then, choose your “recording ratio.” If you put in 1, VLC will extract every single frame from the video. If you input 3, it’ll be 1 in 3, and so on. If your video’s on the short side, you’ll probably want to grab every frame. I went with 1 in 3.
11. DON’T CHANGE ANYTHING ELSE in this menu. You’ll especially want to avoid ticking the “always write to the same file” box -- that’ll just overwrite the same frame over and over again. (Yeah, I did that to myself.) Your settings should end up looking like this:
12. HIT SAVE.
13. Hit Ctrl+O and select your video. VLC will extract the frames automatically as it plays. Open the destination file you set up earlier -- it should be filled with hundreds of pictures. (Once that’s done, if you intend to continue using VLC, make sure to open settings again and UNTICK the “Scene video filter” box you ticked in #6. Otherwise, VLC will continue extracting frames from every single video you watch.)
Step 2: Set up your experiment
1. Download and install RunwayML.
2. Launch RunwayML. Create your account, then dismiss whatever “welcome to the program”-type popup it gives you.
3. In the lefthand column, click the button that looks like a lil wiggle. That should open this page:
4. Click “Train a new image model,” name your experiment, and click “Create.”
5. It’ll prompt you to select a dataset. Click the first box, with the + symbol, and navigate to the file of frames you made earlier. Select it and wait for it to upload. (If your file is larger than 5GB, you’ll have to delete some of the images.)
6. Click next. That’ll take you to “Setup.” This is where you choose a model someone else has trained as a starting point for your own model -- it’s faster to teach a Neural Network how to turn, say, human faces into cats than it is to teach it to make cats from scratch. I suggest just leaving the settings as-is, though if you want you can click “change” and browse other models.
7. Click “start training.”
Step 3: Enjoy what thou hath wrought
1. Be patient while RunwayML runs the experiment. It took about 3 hours for mine to wrap up. While you wait, you can watch the progress in Runway. Don’t worry about missing something -- you’ll be able to go back to this experiment and review the whole training process whenever you want. The FID score on the right should slowly count down -- the closer it gets to 0, the closer the AI’s generated results are to the dataset you gave it.
2. Once your experiment is complete, you can use the slider to move back and forth through the training steps. Click the “save sample image” button to save steps you find particularly entertaining, or hit the “create progress video” button and Runway will generate a video of the whole process.
3. Click “Next.” That’ll take you to this screen, with a gif of the final product and a few other options.
4. Click “save video” so you can look back at your freaky AI-rendered pet whenever you want. (That’s mine at the top of the page.) From here, you can also generate more sample images using the model you’ve just trained.
Congrats! You did it! Now try not to see the results in your nightmares!
