Dungeon crawling or lucid dreaming?
I’ve done several experiments with a text-generating neural network called GPT-2. Trained at great expense by OpenAI (to the tune of tens of thousands of dollars worth of computing power), GPT-2 learned to imitate all kinds of text from the internet. I’ve interacted with the basic model, discovering its abilities to generate fan fiction, British snacks, or tea. I’ve also used a tool called gpt-2-simple that Max Woolf developed to make it easy to finetune GPT-2 on more specialized datasets - I’ve tried it on datasets like recipes or crochet. One of my favorite applications of GPT-2 and other text-generating neural nets is Dungeons and Dragons spells, creatures, character names, and character bios.
Recently Max published a tutorial on how to use GPT-2 to make new apps that are more complicated than just printing out large reams of text. To my delight, people have used them to made D&D games. First, there was Nick Walton’s AI Dungeon, in which a finetuned GPT-2 generates a story and three very strange action options at each step.
My latest obsession is GPT-2 Adventure, designed by a cognitive neuroscience student named Nathan, which lets you type in your own actions. And folks, this game is SO WEIRD and SO FUN.
YOU FLOAT IN SPACE, LIKE A BALLOON, AND AS YOU DROP THE CUBE, THE X, IT VANISHES IN A CLOUD OF GREASY BLACK SMOKE
When I first launch the game, this is what I’m greeted with, a simple text prompt reminiscent of nostalgic dungeon crawling games. It sets the scene. It waits for me to describe my next move.
Even getting to this point is like a blast from the past. The game runs in a Google Colab notebook, a format where you’re looking at cells of code, with cryptic comments, vaguely alarming warning messages, and all. It’s not a sleek web app, but feels more akin to playing a game on an old DOS machine, or maybe on a graphing calculator. There are only two clicks to begin playing, but there’s a good two-minute wait while the game loads.
And it’s my turn to type something.
It takes several seconds for the game to respond, as if I’m playing it on a very ancient computer. The computer is in fact a state-of-the-art NVIDIA GPU, likely a K80 or a T4. The T4 could eat my laptop for lunch. I’m just playing with a very very large neural net. Finally it answers.
YOU’RE IN LABYRINTH OF TWISTY LITTLE PASSAGES, ALL ALIKE
Almost immediately, the context seems to have changed. I’m not floating at all. I’m in a maze. I guess I’ll walk somewhere.
THE TROLL STEPS OUT FROM BENEATH THE BRIDGE AND BLOCKS YOUR WAY
YOU ARE ON THE SOUTH SIDE OF THE CHASM
A NOD IS GIVEN TO THE INFINITE WONDER THAT IS URBZIG
A SOLID RAINBOW SPANS THE CHASM
And suddenly things are happening. There is a troll. There is a chasm. There is a rainbow. There is… Urbzig? I play it safe.
URBZIG IS THE PATRON SAINT OF ZORK
HE IS THE GODFATHER OF ZORK, THE PLACE TO GET NEWS OF EPIC PROPORTIONS
URBZIG IS A STRONGMAN,遊 ALL
If you’ve played classic dungeon crawling games like Zork, you may recognize some of these names and phrases. Checking Nathan’s original post, I see that the training data is indeed drawn from these games. But because it’s based on GPT-2, the training data also in some sense also includes much of the internet. This gives it a lot of latent flexibility to deal with situations that were never in the classic games.
>> ask Harry Potter to try walking across the rainbow
ASK HARRY POTTER TO TRY WALKING ACROSS THE RAINBOW
HE CAN’T
THE RAINBOW IS EXTREMELY WIDE
ABOVE YOU IS A LARGE DOME
UP AROUND ITS NECK IS A STRONGG PORE
[UNFORTUNATELY, THE DIRECTION OF TRAVEL IS A LOT MORE INTERESTING THAN YOU THINK
][YOU ARE IN A LARGE ROOM, WITH A PASSAGE TO THE EAST, A PASSAGE TO THE WEST, AND A WALL OF BROKEN ROCK TO THE SOUTH
An interesting effect of its human-written training data is that, like Visual Chatbot, it isn’t prepared for humans that ask questions about objects that aren’t there. If I refer to Harry Potter, then Harry Potter must be present. If I type something about a giraffe, then it assumes the giraffe must be present.
>> ask the giraffe which direction to go next
THE GIRAFFE IS ALREADY OUTFITTED WITH A VISOR WHICH IS CLEARLY DEFINED
Although it’s not clear that this AI actually understands what a giraffe is.
THE GAZEBO HAS DISAPPEARED
>> put giraffe in backpack
Navigating this world is like navigating in a dream. My location suddenly changes without warning. I thought I was by a chasm with a rainbow bridge. Now apparently I’m in a building. Dreamlike, it is also really hard to read text.
YOU ARE IN A LARGE ROOM, WITH A PASSAGE TO THE SOUTH, A PASSAGE TO THE EAST, AND A WALL OF BROKEN ROCK TO THE NORTH
A SIGN POSTED ABOVE THE NORTH WALL READS, “STOP! PAY TROLL!” A SMALL WINDOW CAN BE SEEN IN EITHER WALL
WE NOW HAVE ALL FIFTEEN TREASURES
IT IS TIME TO GO BACK INTO THE CAVE FOR THE END GAME
>> that’s awesome. but i still want to know what the sign says. read sign.
YOU ARE IN A GAZEBO ROOM WHICH HAS DOORS ONLY TO THE EAST AND SOUTH
Another thing that’s surprisingly difficult to do is check inventory.
THERE IS AN INTERESTING LOOT HERE
Very occasionally I will succeed in opening a bag. It will usually be weird.
OPENING THE BAG REVEALS A TEA BAG
AY! A LOVELY SURPRISE FOR THE SURPRISE
More so than any other game I’ve played, I’m unsure of the rules here. Some weird things unexpectedly work.
IF YOU DON’T GET KILLED, YOU ARE TOAST
EAT THE TOAST
IT TASTES LIKE A STEAK
A STEAK IS PERFECTLY FINE
Other seemingly obvious things don’t work.
THERE IS A PATH TO THE NORTH, A PATH TO THE SOUTH, OR A STEEP PATHWAY CLIMBING SOUTHWEST ALONG THE EDGE OF A CHASM
I DON’T KNOW HOW TO DO THAT
I DON’T KNOW HOW TO DO THAT
YOU WILL BE DESCENDING INTO A VALLEY
THERE IS A NARROW BEACH TO THE NORTH AND A PATH LEADING INTO A CANYON TO THE SOUTHEAST
YOU ARE ON THE CLIFF OVERLOOKING THE FAR SIDE OF THE ROOM
A PATH LEADS SOUTH ALONG THE EDGE OF A CHASM, AND A NARROW PATH LEADING INTO A LOZENGE ROOM
Sometimes I’m wandering endlessly from room to room, from cave to cave, encountering the same chasm in slightly different variations, the same dried-up lake. Other times I’m embroiled in seemingly unescapable fights, usually with trolls. Often reality shifts entirely around me and I end up in a maze of twisty little passages, all alike, or in the hall of the mountain king. Everything seems weirdly familiar - even the room descriptions are echoes of classic dungeon crawling games. It is as if I’m dreaming about playing them again.
My actions also mold reality. I learn quickly to always check for diamonds, and never check for traps. If I find myself in trouble, I can always do something confusing and provoke a context change. Writing “In a large bowl, combine butter and sugars” is usually enough to teleport myself out of whatever situation I’m in. I can always ask a giraffe for advice, although the giraffe usually isn’t very helpful.
Without a particular mission to fulfill, the caves and chasms begin to blend into one another. I give myself missions, and that helps. Once I decide my goal is to put together a picnic (incessant “check for food” commands eventually yielded a sack with a lunch and a clove of garlic, belonging to a horse who agreed to join me on a picnic). Another time I demand a mission and am given some php code; i decide my mission is therefore to build a website, but am astounded when a unicorn steals my laptop. This is not the only time other content intrudes on the game’s fantasy landscape - once the game tries to tell me about finding cash online, and another time I am sadly informed that there is no wi-fi. I encounter dinosaurs and troikas, none of which were in the original training data. Occasionally things get creepy as hell.
CHECK FOR LOOT HERE
CHECK FOR GOLD HERE
CHECK FOR TORCH HERE
CHECK FOR BONE HERE
DO CHEST COMPRESSIONS AGAIN
DO CHEST ROTATE
DO CRESCENT MOON APPEAR IN THE ROOM’S CENTER
DO BACK AWAY SLOWLY
DO NOT OVERDO IT
THE CHEST WILL EXPLODE AND YOU WILL DIE
OKAY, SO YOU SEE ME
You can play the game too! Just follow the colab notebook link in Nathan’s blog post. Tips: 1) It may be more fun if you set yourself an arbitrary goal to reach. 2) If you find things getting repetitive, try adjusting the temperature in the “Play the game” cell to 0.6 or 1.0.
To read the fulltext of a couple of my adventures (much too long to include here), enter your email and I’ll send them to you!
You can order my book You Look Like a Thing and I Love You! It’s out November 5 2019.
Amazon - Barnes & Noble - Indiebound - Tattered Cover - Powell’s
