rina, 29 years old, full time fangirl, shipper, lazy ass, obsessed with too many shows and groups loves merch, cartoons, tv shows, fanfictions, books, k-pop and lipsticks
Utility for downloading fanfiction in bulk from the Archive of Our Own - nianeyna/ao3downloader
Well folks I've been sitting on this little script for ages and finally decided to just go ahead and publish it. What does it do?
you can enter any ao3 link - for example, to your bookmarks or an author's works page - and automatically download all the works and series that are linked from that page in the format of your choice. if there are multiple pages, the script will page through results for you until there are no more fics to download
you can check your fanfic-savin' folder for unfinished fics and automatically update them if there are new chapters. you can also check for new fics in series!
if you're a dinosaur who uses Pinboard, you can back up all the Pinboard bookmarks you have that link to ao3
don't worry about crashing ao3 with this! this baby takes forever to run, guaranteed. anyway ao3 won't let me download stuff too fast even if I wanted to so it's quite safe
I've been working on this for about two years and it's finally in a state where it does everything I want and isn't breaking every two seconds, so I thought it was time to share! I hope y'all get some use out of it.
note: this is a standalone desktop app that DOES NOT DO ANYTHING aside from automate clicking on buttons on the ao3 website. Everything this script does, can be done by hand using ao3's regular features. It is just a utility to facilitate personal backups for offline reading - there's no website or server, I have no access to or indeed interest in the fics other people download using this. No plagiarism is happening here, please don't come after me.
---
2024 update
This is now a fairly mature project that I've been maintaining for quite a while (3 years?? omg), so let's do an FAQ
Question 1: Wait, a python script? That's scary! I don't know anything about running python scripts!
Answer: You don't need to! When I published this program I knew that there would be people who would want to use it that did not know anything about running python scripts. To that end I have provided extremely detailed instructions (accessible at the link above), which have been fine-tuned over the last three years to make them as absolutely easy to follow as possible. Not only do you not need to know anything about python scripts to use this program, you don't even need to know anything about computers at all. Try it and see! ;)
Question 2: I already use Calibre and FanFicFare to download fanfics - isn't this the same thing?
Answer: Nope! In fact, one of the reasons I created ao3downloader is that I tried FanFicFare and found it unsuited to my needs.
- FanFicFare requires a lot of configuration to work well. I wanted something that would get out of my way and just do what I needed it to do with as little prompting as possible. ao3downloader has sensible default behavior that will do what most people want it to do without too much pre-work.
- FanFicFare doesn't handle multi-page downloads very well - you need to enter the link to each page of results individually. What if I want to download dozens, or even hundreds of pages of fanfic? (Honesly, even two pages is too tedious for me...) I want to download all of the results I'm interested in, not just one page at a time. ao3downloader can download up to five thousand pages of fanfiction in one go if you're willing to let it run for that long... and that limit is only because 5,000 is the maximum amount of pages that ao3 itself will show you in a search result.
- FanFicFare doesn't handle ao3's rate limit very well - you need to babysit it and fine-tune the amount of time it pauses between downloads. I want the downloading process to handle itself, so I can completely ignore it until it's finished. ao3downloader will automatically pause when it hits the rate limit, and automatically resume when the rate limit expires. You can also quit ao3downloader entirely, and it will do its best to resume at the same point you left off when you turn it on again.
- FanFicFare uses its own download format, which doesn't match the files I download from ao3 directly - I want them to be consistent. ao3downloader downloads fics using the native download buttons from ao3, so it's exactly the same file as if you downloaded them manually.
- FanFicFare has an option to check incomplete fics for updates, but it relies on you telling it which fics in your library are incomplete. I don't want to have to keep track of that. ao3downloader can scan your existing files for incomplete fics and identify them for you, then check ao3 for updates and only re-download the ones that have new chapters. (It can also scan for incomplete series, and do the same thing!)
Question 2a: That's all great, but I need Calibre's organizational features.
Answer: Me too! That's why I import all the fics I download using ao3downloader into my Calibre library after I'm done downloading them.
Question 2b: That's all great, but I prefer FanFicFare's download format and I don't want the native ao3 one.
Answer: ao3downloader has a special option for this which instead of downloading actual files, will simply harvest a list of links to give to FanFicFare. You won't get ao3downloader's rate limit handling or automatic update functionality, but you will get to avoid entering individual page links! If you choose, you can also include detailed fic metadata in the links download (such as title, author, summary, tags, etc) which will be saved in spreadsheet format.
Question 3: Do I need to enter my ao3 login information? What happens when I enter my login information into this program?
Answer: You only need to enter your login information if you want to use ao3downloader to download archive-locked fics, or to download a page that is only accessible when you are logged in (such as your history or subscriptions). Hopefully the reason behind this requirement is obvious. If you don't need those things, you can simply skip the login prompt and the program will work fine! That said, if you do log in, here's what will happen. 1) ao3downloader will save your login information to a file so you don't have to enter it every time. (If you don't want your login information to be saved, you can turn this behavior off using the included .ini file - just open up the file named settings.ini and read the instructions inside to configure it.) 2) then, ao3downloader will use your login information to log itself in to ao3... aaaand that's it. It's not stored or sent anywhere other than your own computer and archiveofourown.org. In fact, there is absolutely no tracking or spyware in ao3downloader of any kind - I don't even know how many people have downloaded it, let alone what you type into it. And that's the way I like it!
Question 4: I thought of a feature that doesn't exist yet that would be really cool! Can you add it?
Answer: Very possibly! Feel free to contact me via one of the options listed in the readme (that's the page linked above) if you have a feature request.
Question 5: Do I have to pay you?
Answer: Never.
Question 6: But I really want to pay you... do you have a ko-fi?
Answer: Yes. It's on the github page in the sidebar.
Question 7: Does it work on mobile?
Answer: No. :( but watch this space... there may be certain things on the horizon. we'll see!
BDSM AU In desperate need of money, Gihun renters the BDSM community to take on one time customers. What he doesn’t expect is to sign a contract to be Inhos long term sub and to be shown how healthy and healing BDSM can be when you have a partner that cares about you.
Curiosity (satisfaction)
Cannon Divergence While searching for the games, Gihun fails to realize that his new handsome and possessive lover is actually the enemy he’s been looking for
Oh Wha A Tangled Web We Weave
Mafia/Omegaverse Gihun goes to work at an infamous casino that hires pretty omegas to work the floor. He’s also met a super nice man that’s definitely an omega and not the alpha running the whole casino empire. (I’d never read omegaverse before this and was always sus of it. BUT THIS FIC HAS GOT ME ALL IN)
Four Plum Blossoms
Cannon Divergent When Gayeong and her family get taken hostage by the Frontman, Gihun will do anything to ensure her safety.
In the Devils Teeth
Serial Killer AU As the only surviving victim of the brutal serial killer The Frontman, Gihun is determined to bring him to justice.
White Mask
The ONLY corruption/co-frontman au I read! Gihun agrees to become Lieutenant to the Frontman and take down the games from the inside.
Strange Attractors
Fix-it fic Gihun survives the fall. This story follows his journey of growth and healing with Inho and Junhees baby by his side
Fine Print
Inho comes across pre-game Gihun begging for spare change. He agrees to let Gihun live with him out of boredom. This is fun and gets darker as the chapters go.
Epiphany
What if Junhee giving birth in the games actually made sense and everyone was written beautifully??? This is less inhun centric but does have a dash of it
I’m Not in a Position to Help Anyone at the Moment
Cannon divergent Gihun chooses to not go back to the games after Red Light Green Light. Turing over a new leaf, he decides to put in the work to save his mom and be the parent Gayeong needs.
Short/one shots I can’t leave out:
In No Position -Whump
Love Looks Best When It’s Carved Into You -unfinished serial killer
Sloppy toppy all of the authors!!!
If anyone has any fics they’d recommend let me know!!!! My only squeaks are hard core rape (dubcon is ok) and mpreg :)
it’s unfortunate that (some) people in fandom spaces are starting to get too comfortable complaining and being rude to writers and artists who create contents they personally don’t like (“why are you making this character a top when he’s obviously a bottom? omg do you even understand his character?” “I’m so sick of seeing fan art and fanfic where this character is portrayed as a sadist when he’s actually misunderstood in my opinions, therefore anyone who disagrees with me is wrong and should be shamed” just to name a few I’ve seen) instead of curating their own fandom experiences by engaging only with contents they do like.
you want more fics where (x) is written in this specific way? either write one yourself or politely expressing your opinion about how you hope there will be more fics where (x) is written in this specific way instead of making fandom a toxic place by being rude to writers and artists who dare make contents that are not to your Personal Liking.
if the universe does not revolve around you, strangers and fandom spaces don’t have to cater exclusively to your personal preferences either.
to all my beloved writers and artists, write whatever you want, draw whatever you want. portray that character in whichever way you want to portray. I hope you have fun doing what you love. don’t let anybody tell you what you can or can’t do with the blorbo. go wild. I will always support you
Dae-ho fans thinking they were gonna get an exploration of ptsd flashbacks only for Dae-ho to have a complete personality transplant and admit he lied about being in the marines
Hyun-ju fans thinking they were gonna get a bunch more badass scenes with her only for her to get killed off in episode 2 by the worst character ever
Jun-ho fans thinking he was actually going to be allowed to shine and impact the plot in a tangible way this time only for the writers to pull yet another "you could take him out of the show and the plot wouldn't really change"
Hwang brothers fans thinking they were going to get the reunion they were all waiting for only for Jun-ho to shout a few words at In-ho from a considerable distance away and then nothing more
Inhun / 457 fans anticipating the big reveal of who In-ho really is to Gi-hun only for it to last two minutes, with Gi-hun never finding out In-ho's true identity as Jun-ho's brother, no exploration into why In-ho became the front man, etc
Gi-hun fans thinking he would survive because surely after all he's been through he'll be allowed to live only for... yeah
a fandom doesn’t die because the source material ended btw. it only dies if fans stop talking about it. so it will always stay alive even if just one person still talks and is passionate about it.
support artists. comment nice things on their fics, leave kudos on their fics, like and reblog their art, engage with posts about your blorbo if you don’t want your fandom to die.
bbc merlin, house md, nbc hannibal have all ended for more than a decade and their fandoms are still as alive and active as ever. they literally trend here every other month. we can be like them <3
