Fandom Userscript Cookbook: Five Projects to Get Your Feet Wet
Target audience: This post is dedicated, with love, to all novice, aspiring, occasional, or thwarted coders in fandom. If you did a code bootcamp once and don’t know where to start applying your new skillz, this is for you. If you're pretty good with HTML and CSS but the W3Schools Javascript tutorials have you feeling out of your depth, this is for you. If you can do neat things in Python but don’t know a good entry point for web programming, this is for you. Seasoned programmers looking for small, fun, low-investment hobby projects with useful end results are also welcome to raid this post for ideas.
You will need:
The Tampermonkey browser extension to run and edit userscripts
A handful of example userscripts from greasyfork.org. Just pick a few that look nifty and install them. AO3 Savior is a solid starting point for fandom tinkering.
Your browser dev tools. Hit F12 or right click > Inspect Element to find the stuff on the page you want to tweak and experiment with it. Move over to the Console tab once you’ve got code to test out and debug.
Javascript references and tutorials. W3Schools has loads of both. Mozilla’s JS documentation is top-notch, and I often just keep their reference lists of built-in String and Array functions open in tabs as I code. StackOverflow is useful for questions, but don’t assume the code snippets you find there are always reliable or copypastable.
That’s it. No development environment. No installing node.js or Ruby or Java or two different versions of Python. No build tools, no dependency management, no fucking Docker containers. No command line, even. Just a browser extension, the browser’s built-in dev tools, and reference material. Let’s go.
You might also want:
jQuery and its documentation. If you’re wrestling with a mess of generic spans and divs and sparse, unhelpful use of classes, jQuery selectors are your best bet for finding the element you want before you snap and go on a murderous rampage. jQuery also happens to be the most ubiquitous JS library out there, the essential Swiss army knife for working with Javascript’s... quirks, so experience with it is useful. It gets a bad rap because trying to build a whole house with a Swiss army knife is a fool’s errand, but it’s excellent for the stuff we're about to do.
Git or other source control, if you’ve already got it set up. By all means share your work on Github. Greasy Fork can publish a userscript from a Github repo. It can also publish a userscript from an uploaded text file or some code you pasted into the upload form, so don’t stress about it if you’re using a more informal process.
A text editor. Yes, seriously, this is optional. It’s a question of whether you’d rather code everything right there in Tampermonkey’s live editor, or keep a separate copy to paste into Tampermonkey’s live editor for testing. Are you feeling lucky, punk?
Project #1: Hack on an existing userscript
Install some nifty-looking scripts for websites you visit regularly. Use them. Ponder small additions that would make them even niftier. Take a look at their code in the Tampermonkey editor. (Dashboard > click on the script name.) Try to figure out what each bit is doing.
Then change something, hit save, and refresh the page.
Break it. Make it select the wrong element on the page to modify. Make it blow up with a huge pile of console errors. Add a console.log("I’m a teapot"); in the middle of a loop so it prints fifty times. Savor your power to make the background wizardry of the internet do incredibly dumb shit.
Then try a small improvement. It will probably break again. That's why you've got the live editor and the console, baby--poke it, prod it, and make it log everything it's doing until you've made it work.
Suggested bells and whistles to make the already-excellent AO3 Savior script even fancier:
Enable wildcards on a field that currently requires an exact match. Surely there’s at least one song lyric or Richard Siken quote you never want to see in any part of a fic title ever again, right?
Add some text to the placeholder message. Give it a pretty background color. Change the amount of space it takes up on the page.
Blacklist any work with more than 10 fandoms listed. Then add a line to the AO3 Savior Config script to make the number customizable.
Add a global blacklist of terms that will get a work hidden no matter what field they're in.
Add a list of blacklisted tag combinations. Like "I'm okay with some coffee shop AUs, but the ones that are also tagged as fluff don't interest me, please hide them." Or "Character A/Character B is cute but I don't want to read PWP about them."
Anything else you think of!
Project #2: Good Artists Borrow, Great Artists Fork (DIY blacklisting)
Looking at existing scripts as a model for the boilerplate you'll need, create a script that runs on a site you use regularly that doesn't already have a blacklisting/filtering feature. If you can't think of one, Dreamwidth comments make a good guinea pig. (There's a blacklist script for them out there, but reinventing wheels for fun is how you learn, right? ...right?) Create a simple blacklisting script of your own for that site.
Start small for the site-specific HTML wrangling. Take an array of blacklisted keywords and log any chunk of post/comment text that contains one of them.
Then try to make the post/comment it belongs to disappear.
Then add a placeholder.
Then get fancy with whitelists and matching metadata like usernames/titles/tags as well.
Crib from existing blacklist scripts like AO3 Savior as shamelessly as you feel the need to. If you publish the resulting userscript for others to install (which you should, if it fills an unmet need!), please comment up any substantial chunks of copypasted or closely-reproduced code with credit/a link to the original. If your script basically is the original with some key changes, like our extra-fancy AO3 Savior above, see if there’s a public Git repo you can fork.
Project #3: Make the dread Tumblr beast do a thing
Create a small script that runs on the Tumblr dashboard. Make it find all the posts on the page and log their IDs. Then log whether they're originals or reblogs. Then add a fancy border to the originals. Then add a different fancy border to your own posts. All of this data should be right there in the post HTML, so no need to derive it by looking for "x reblogged y" or source links or whatever--just make liberal use of Inspect Element and the post's data- attributes.
Extra credit: Explore the wildly variable messes that Tumblr's API spews out, and try to recreate XKit's timestamps feature with jQuery AJAX calls. (Post timestamps are one of the few reliable API data points.) Get a zillion bright ideas about what else you could do with the API data. Go through more actual post data to catalogue all the inconsistencies you’d have to catch. Cry as Tumblr kills the dream you dreamed.
Project #4: Make the dread Tumblr beast FIX a thing
Create a script that runs on individual Tumblr blogs (subdomains of tumblr.com). Browse some blogs with various themes until you've found a post with the upside-down reblog-chain bug and a post with reblogs displaying normally. Note the HTML differences between them. Make the script detect and highlight upside-down stacks of blockquotes. Then see if you can make it extract the blockquotes and reassemble them in the correct order. At this point you may be mobbed by friends and acquaintainces who want a fix for this fucking bug, which you can take as an opportunity to bury any lingering doubts about the usefulness of your scripting adventures.
(Note: Upside-down reblogs are the bug du jour as of September 2019. If you stumble upon this post later, please substitute whatever the latest Tumblr fuckery is that you'd like to fix.)
Project #5: Regular expressions are a hard limit
I mentioned up above that Dreamwidth comments are good guinea pigs for user scripting? You know what that means. Kinkmemes. Anon memes too, but kinkmemes (appropriately enough) offer so many opportunities for coding masochism. So here's a little exercise in sadism on my part, for anyone who wants to have fun (or "fun") with regular expressions:
Write a userscript that highlights all the prompts on any given page of a kinkmeme that have been filled.
Specifically, scan all the comment subject lines on the page for anything that looks like the title of a kinkmeme fill, and if you find one, highlight the prompt at the top of its thread. The nice ones will start with "FILL:" or end with "part 1/?" or "3/3 COMPLETE." The less nice ones will be more like "(former) minifill [37a / 50(?)] still haven't thought of a name for this thing" or "title that's just the subject line of the original prompt, Chapter 3." Your job is to catch as many of the weird ones as you can using regular expressions, while keeping false positives to a minimum.
Test it out on a real live kinkmeme, especially one without strict subject-line-formatting policies. I guarantee you, you will be delighted at some of the arcane shit your script manages to catch. And probably astonished at some of the arcane shit you never thought to look for because who the hell would even format a kinkmeme fill like that? Truly, freeform user input is a wonderful and terrible thing.
If that's not enough masochism for you, you could always try to make the script work on LiveJournal kinkmemes too!
It’s almost certain that any code you write will have a bug in it. There’s a lot things that can help with finding them and one of those things is logging to the console. Most programming languages have a function called something like “print” or “log” to output data and messages while the program is running.
The way logging works on StoryDevs is a little more complicated. For brevity I’m just going to describe how logging works in the browser and skip how it’s done on the server.
Simple Logging
In JavaScript we use the console object which has a few different ways to log things. For example if you have a script with a line of code that reads:
console.log(“Hi there!”);
If you open up Chrome’s dev tools (shortcut is F12) and go to the “Console” tab you’ll see this:
Note how it says “script.js:2342” on the far right. This tells us where the line of code that logged this information is. It can be useful when you’re trying to figure out where an error is coming from, especially if you have thousands of lines of code. The “script.js” part refers to the file name and “2342” is a line in that file.
Default Logging Is Nice
The default methods for console are nice because you can log things other than just text. You can essentially log anything and it’ll look pretty good in the console. For example let’s say we have the following array:
let a = [“Some string”, 52, true];
And then we log it:
console.log(a);
The console will output this:
Any time you log something like an array or an object you’ll be able to expand it to see what it contains. Again, useful for debugging.
I Log A Lot
So much, in fact, that I find typing “console.log()” kind of annoying. So I made a function called “log” which works exactly the same and just has a different name:
There’s actually another way to do this which is:
let log = console.log;
Either way you can now write:
log(“My message…”);
And it’ll work. The second way actually avoids a problem I’m about to show you with the first approach but we’re doing it the first way to illustrate a point.
Remember how console.log tells you the file and the line you logged from? Well the problem with creating your own function named “log” and then calling console.log inside it is that it’ll always list the same line: the one that console.log is on.
So let’s say we have the following code:
This is what the result will be:
Notice how on the right it says the array was logged from line 3, not line 7 where we actually did our logging. In fact, if you defined the log function in another file it’d even list that file instead of the one you logged from. This isn’t what we want! So why did I do it this way and how do I get around it?
Why I’m Doing It The Hard Way: Part 1
To explain this I need to show you how I work. When I write JavaScript for StoryDevs I try to sort the code into a few broad categories and then make each category its own file. So here’s my JavaScript folder (there’s three other folders inside it but just ignore those):
Each file does stuff related to its name. For example, columns.js contains all the code related to StoryDevs’ columns. When the site is public I don’t want all the code split up like this.
Firstly, it makes understanding the code a lot easier because the code is logically organised. StoryDevs will skew left (politically speaking) and I don’t want it to be easy to make a “without-the-politics” (i.e., right-leaning in this climate) knock-off of StoryDevs. I want them to have to cobble together disparate code or write it themselves :)
The second reason is that one big file compresses better than several smaller files. The basics of compression is that it finds a repeating pattern in the text and then replaces it with a smaller code. To allow for uncompressing a little dictionary is kept which tells us what pattern each code refers to.
Let’s say we have this file:
If we zip this it’s smaller than if you have two zipped files with the functions split between them. The larger the files get the more true this is. Combining all my JavaScript files into one big file and then compressing it results in a 10% smaller zip. That’s a decent saving for a file that will be requested a lot.
The third reason is it’s just nicer to only have to link to one script file! I don’t have to remember to include a new JavaScript file if I happen to make a new one.
What Does This Have To Do With Logging!
Remember earlier when we logged something and it showed us the line number and the file name? Well if you combine all your files into one big file before it’s sent that means the logging info will be incorrect. It will log the big file’s name (I simply call it “script.js”) and the line number won’t be the original line number.
So if we have a file, “little_1.js”:
And we write a helper program that joins it with “little_2.js” (this is basically what I do on the server):
We’ll get this:
What was previously line 1 in “little_2.js” is now line 10 in our new automatically generated script. The script itself doesn’t know which parts of it originally belonged to which file. Both these things make debugging harder. So to get around this I add a little extra info at the bottom of my big script file:
As I build the single script file from the smaller script files I save each file’s name and the number of lines it contains to an object. Then I write this object at the very end of the file. Using this, a bit of math allows us to figure out the appropriate name and number to use when logging.
Why I’m Doing It The Hard Way: Part 2
Okay now I’m actually going explain this! Remember how there’s two ways to create a function called “log”? There was this way:
And this way:
let log = console.log;
And the first one has a problem in that it will also think the logging is occurring on the line where console.log is rather than from where you’re calling log from. Despite this, I chose to do it that way because I need to do that math to work out the file name and line number anyway before logging.
If you look at the second approach to creating a log function, all it’s doing is saying “‘log’ now means ‘console.log’” — it doesn’t allow us to jam more code into it. Whereas in the first approach you can write as much code as you want before the logging takes place. And boy do I do that:
Let’s break this down. First I start out by creating an error object. I’m doing this because an error object contains the stack property. Now, in programming there’s something called the “call stack” and this is what stack refers to. It’s a list of all the functions that were called to get to this point in your code.
Here’s an example of what it looks like if we log err.stack:
Keep in mind it starts from the most recent function call. If you were to say this in plain English you might say, “There was an error in the ‘log’ function which was called by ‘init_actionables’ which itself was called by ‘dynamic_content_init’ [. . .]” and so on.
The err.stack property of an error is a string, which means you can call string methods on it. For example, we could do err.stack.toLowerCase() which would make all the text it logs lower case. In my case I call split on it:
First, what’s “\n” mean? That’s an invisible character computers use to mark where a new line begins. It’s inserted every time you hit your enter key. See how when we logged err.stack above it’s on multiple lines? That means the string contains “\n” characters.
When I call the string’s split method and pass in “\n” it means “split this string where the new line characters are”. It results in you getting back an array of strings. So if we log err.stack.split(“\n”) we get this:
Instead of one long string we now have six strings. Because we divided the original string by its lines, we now have the name of one function per line as well as the file name and line number it was called from. (The exception is the first string that just says “Error.”)
So, err.stack.split(“\n”) results in an array of strings. Because this error was created inside the log function we know the first function will always be log. The one after that (the third element) will always contain a string describing the function that called log.
So we get the third index of the array (which is 2 because it starts from 0) and then call trim on it. The trim function just gets rid of whitespace padding on the left or right of a string. As you can see above there’s some leading whitespace that we don’t want.
Finally, we assign the result of all these operations to line. So right now line contains the string:
What we want from this string is the function name and the line number. So with the next bit of code we start to extract the line number:
We’re take the string we assigned to line, call the split method on it (passing it a space character), and then store the result in parts. After this the contents of parts is an array containing separate strings that were previously joined by spaces:
(Incidentally, this is why I called trim beforehand — there would be four empty strings before “at” because of the four leading spaces. So I got rid of them to make the array easier to think about.)
Okay, let’s speed things up a bit. The next few lines are doing something similar to the previous ones:
First we grab the last element in the parts array and assign it to source. To get the last element of an array you have to get the arrays length (parts.length) and then subtract one from it. The last index of an array that is 3 elements long is “2” as most programming languages have arrays that count from zero.
That means source now contains the string:
“(http://localhost:8080/js/script.js:2532:5)”
Next, we break source into parts, dividing it anywhere a “:” appears. Note that I name it “source_parts” instead of overwriting parts because I still need parts for something I’ll do later. So source_parts now looks like this:
Then we grab the fourth element (index 3) and assign it to abs_line (the name stands for “absolute line”). This is the line number of where log was called in the big script we built earlier. Now abs_line contains “2532” as a string value. The next part looks complicated but it’s not too bad.
First we define some variables source_file, rel_line, and total. I give total a default value of “0” because I’m gonna do math with it. The reason I define these outside the loop is that any variable created inside the loop only exists for that iteration of the loop. I need total (and the other variables) to persist after that.
Next we start the loop. The object script_meta was the thing we defined at the very end of the big script that listed all the names of the JavaScript files and how many lines they contain. It looked like this:
Each time we loop, the fn variable (which stands for “filename”) is assigned a key from the script_meta object, starting from “column.js” through to “tools.js.” Next we assign the result of subtracting total from abs_line to rel_line (“rel_line” stands for “relative line”). Since abs_line is 2532 and total is 0 we get 2532.
UTTERLY POINTLESS YOU FOOL. Well hey hold on. Next we grab the total line count associated with the current file by indexing script_meta with fn. If this were the first iteration of the loop fn would be “column.js” so script_meta[fn] would result in 526. Then we add this to total, making it 526 as well. Then…
...we have this little conditional. It says “if the total so far is higher than the absolute line (the line we logged from in the big file) then do this.” Then we say source_file is the current file name and use break, a keyword for ending loops early and continuing with the rest of the code after the loop.
In essence what we’re doing is going through script_meta and adding each file’s line count to total. Then we check if total has become higher than the line number we logged from in the big script file. If it has, we know that it must have originally been in this file because all the scripts were added together in alphabetical order.
The abs_line we logged from was 2532. Let’s visualise what the loop does:
Here we can see that once we get to “main.js” our total line count jumps to 4121, which is above abs_line’s 2532. That means the line was originally in the file “main.js” before we joined it together with the others. So our conditional tests true and kicks us out of the loop with break.
Now the two variables defined outside the loop (source_file and rel_line) contain the original source file name and the correct line number relative to the original file. That thing we were doing at the beginning of the loop:
rel_line = abs_line - total;
Was to figure out what the original line number was. If you subtract the total line count of the files so far from the line you logged from in the big file you’ll wind up with the original line. It’s telling us what the line number would be if you were logging from that file.
This results in source_file being “main.js” and rel_line being 192:
2532 - (526 + 1690 + 124) = 192
Christ, Can You Break This Up With Another Header Or Something
Yeah, sorry. I honestly didn’t think this wouldn’t take so long. I regret this post and my life as wel , ,
A Nicer Section
Next, a simpler part of the log function:
Again, we’re defining a variable before using it in the conditionals below so that it still exists outside those blocks. Then we ask if parts is longer than 2. What the hell is parts?! This is the thing we were saving for later. It’s the array we got by pulling out a line from the error’s call stack:
We’re checking its length because it can either be 2 or 3. In this case the function on this line has a name “init_actionables” but there’s a thing in JavaScript (and other languages) called an “anonymous function.” Basically they’re functions without names. If the function is anonymous parts will have a length of 2.
So if parts has a length of 2 we make the caller’s name “anonymous” otherwise we grab the second item and make that the caller’s name. Either way we now have a name!
Lastly, we have this:
The console object has methods other than just log. The group and groupEnd methods mark the start and end of a group respectively when logging to the console. Anything you log to the console after calling group will be nested under the heading you give it.
And that means the result of calling log like this:
Shows up in the console like this:
Woohoo! We have the correct file name, line number, and even the function name that log was called within. Over to the right you’ll see it says 5234 which is where console.group and console.log are logging from in the script file (which we don’t care about!)
As with the original console.log we can still log complex objects and have them display nicely:
Summary
That’s a lot so let’s recap:
Logging is useful for debugging.
StoryDevs has multiple JavaScript files. Before sending them to the user they’re combined into one big file because it compresses better, among other reasons.
A little bit of metadata that preserves the file names and their line counts is saved at the end of the big file.
When you use console.log the console will show you which file logged the message and from what line.
That file name and line number will be incorrect because we combined all the files into one big file.
Therefore, we need to do a little bit of parsing and math to work out the original file name and line number, using the metadata we saved earlier.
Conclusion
Well that’s it. It should be noted that I wrote this for myself in one afternoon so it’s not intended to be super robust. It’s only been tested in Chrome and I don’t care if it doesn’t work perfectly elsewhere at the moment.
Anyway, I hope someone finds this interesting/useful because my wrists hurt!
StoryDevs is a place for developers of story-focused games and visual novels to find each other and collaborate. The site is under development but handle reservations are open: www.storydevs.com/reserve
One of the little used and known properties in JavaScript is the limit property in split. Limit returns the number of splits specified instead of all that can be found in a string. This is simpler than using regular expressions for simple operations like getting the first 2 lines of a string:
Limit can also help with efficiency when parsing larger strings. For example, when parsing a file, it would be more efficient to extract the first 3 columns each time if a split with a limit of 3 was used:
In this article, we are going to see the Extracting Numbers from a String in JavaScript. As we know, extracting a number from the given string will be easier when we apply some methods like parseInt() and parseFloat() methods, right? But it also has some limitation which we are going to see in here, and also we will see some different ways to extract numbers from a string.
Extracting Numbers…
