#restricted works

[destinationtoast]

An Archive of Our Own, a project of the Organization for Transformative Works

This is such a great list/set of analyses! Thanks to Randomist1031 for putting in the work to collect and share this data.

I'll be doing my annual top fandoms/ships/characters based on public works sometime in January (ish). But it's REALLY good to have the private works listed also, and to see how that varies by ship!

[thai-drama-ao3-stats]

Thai Drama Stats Special Edition:

The Great Archive Lockdown 🔒

Hi folks! In case you weren't aware, there are various scraping bots that trawl through AO3 and use the data for AI training, content mill sites, or other vaguely nefarious purposes. One site, "Fanfic Books", is essentially creating an unauthorized mirror of AO3. Here are some posts about it.

To combat this, many users have recently chosen to "Archive Lock" their fics.

What is Archive Locking?

An archive locked work, or "restricted" work, is only visible to users who are logged into AO3. This prevents anonymous users (and bots that aren't using login credentials) from reading your fic or finding your fic in searches. This doesn't block all scraping bots, but it should keep most of them out.

What does this have to do with fandom stats?

The AO3 scraping I do doesn't use login credentials, so I can't count archive locked fics. That's totally fine! I am in no way telling you to stop archive locking! Lock or unlock to your hearts content!

It does, however, mean that the data I pulled from my Thai Drama AO3 Trends Dashboard this week (July 1 - July 7, 2024) are looking especially strange.

Holy moly! We actually have negative growth. More fics were locked than posted, which is why the Net New is negative. I'd estimate that about 1% of all previously public Thai Drama fics were archive locked this week.

This matches trends on all of AO3. This week, the total number of publicly available fics actually decreased by 0.7% -- and that's including all the new fics being posted!

When did this happen?

The timing for both Thai Drama fandom and all of AO3 is pretty consistent.

For Thai Drama fandom, most of the locking happened on Friday, July 5, but there was also some locking on Sunday.

When we look at all of AO3, it seems like most of the mass-locking happened on July 5th as well, with additional locking happening all throughout the weekend.

Which Thai Drama fandoms were most affected?

When we look at sheer numbers, KinnPorsche, of course, has a lot of newly-locked fics. 3 Will Be Free, My Engineer, and Dark Blue Kiss were locked down a lot as well.

When we look at the top fandoms by negative growth, The Player saw almost all of its fics vanish overnight. 3 Will Be Free was cut neatly in half.

This data is cool I guess, but... so what?

If these numbers are accurate, it represents another sudden and massive shift towards archive locking on AO3.

According to @star-grazing's stats about archive locking in December 2022, the total number of archive locked works on AO3 increased by 70% in just a couple weeks after a reddit post went viral about AI bots scraping AO3 for machine learning material.

Those stats show that in December 2022, 5.79% of AO3 fics were archive locked. When I checked the numbers again today, 9.37% of all works were archive locked.

Using rough estimates, from the last few days of AO3 data, I'd say that the total number of archive locked works increased by 8% since last Thursday (7/4). And trends seem to indicate that the great lockdown is still going!

Anyway...

Thanks for sticking with me! This is a really fun time to be collecting AO3 stats :) If you have more questions, feel free to reach out. I also put some more details under the cut! Thanks y'all!

Are we sure it was archive locking, and not some other data issue?

Er, good question. It's my best guess, and I've tried to rule out other potentially culprits. The AO3 Fandom Trend Analysis Dashboard, which has data about all fandoms on AO3, doesn't seem to show anything amiss. Their data uses login credentials, meaning they can count archive locked fics.

I also went through several tags manually while logged in and logged out to compare numbers from this week to previous weeks. It doesn't seem like there was a mass deletion or retag that I could see.

I also used the "restricted:true" search operator to search for archive locked fics while logged in. A lot of those missing fics pop back up!

I absolutely welcome other theories though, if you think of one!

Is this still happening?

Seemingly yes, for Thai Dramas at least! When I checked the "All Thai Dramas" AO3 search this morning, the total number of Thai Drama fics had dropped below the 40K mark - lower than when I first started keeping track a month ago!

We probably have a lot more archive locking in our future!

How do I archive lock my own fics?

There's a really good tutorial over here.

Help! I don't have an AO3 account, so I can't read all these archive locked fics anymore.

Please message me! I have some spare invites.

Which fandoms are the most "locked down"?

I'm not sure, but there is a Fanlore article about Hockey RPF and the Fourth Wall which provides some comparison stats. Hockey fandom has traditionally been one of the most locked down fandoms; less than half of hockey rpf fics are publicly available.

You can also peruse this AO3 search to see all archive locked fics.

[sitp-recs]

Hi! I have an ao3 account, but I only ever log in to upload stuff and get out, or download stuff and get out. I know this means there’s probably a whole bunch of really amazing drarry fics I’ve missed because they’re restricted works, and was wondering if there were any you could recommend in particular as a place to start?

Hi anon! I know a few authors keep their works restricted so I’d probably start by checking their catalogues. I linked their AO3 and one personal rec below. For additional recs I had to consult my bookmarks because I never noticed which fics are restricted! I think these would be a great starting point:

Authors

Aideomai | My rec

Carpemermaid | My rec

Korlaena | My rec

Maesterchill | My rec

Phdmama | My rec

Tryslora | My rec

Artist

Dustmouth | My rec

Fics

Sleight of Hand by thestarryknight

Still Life, Home County, HP and the Future He Doesn't Really Want and Wild (all orphaned)

Teeth by Amelior8or

[destinationtoast]

Hi!

I've been tracking the number of restricted works on AO3 since about 24 hours after the recent Reddit post on ao3 probably being scraped for use in commercial writing AIs. I think it could be interesting to make some meta with that.

Do you, or does one of your followers happen to have data on the number of restricted works on ao3 prior to December 2nd? Or know where I could find it?

Oh, that sounds really interesting! I don't have any recent data about restricted works on AO3 prior to that date (it's vaguely possible I looked at this number at some point in the past, but if so, it was years ago). Any of my followers have relevant data?

[randomist1031]

2023 Ao3 Most Written Ships (with both Public + Locked Fiction)

Completed the list and posted it on Ao3 at the beginning of the month but finally decided to put it on Tumblr as well.

Shy poster here, mercy please.

[star-grazing]

Restricted Works on AO3

In December 2022, I made a post with statistics on the number and percentage of restricted works on ao3, because quite a few people wanted to archive-lock their work due to the reddit post calling attention to the fact that ao3 has been scraped to train AI text bots.

Since then I have been tracking the percentage of restricted works on the archive as a whole once per week.

On May 13th, the OTW made an announcement on 'AI and Data Scraping', which renewed interest in the matter. So, here are a couple of stats looking at the change since last year!

First, a chart that shows the percentage of restricted works on the whole archive, including some individual historic values and weekly values since the beginning of December.

There were 3.38% of restricted works a day after the reddit post (Dec 2nd) and 5.79% at the time of my last post (Dec 16th). The percentage of restricted works dropped slightly and steadily since since then until Mid-May.

The decrease is mostly due to an increase in overall works and the number of restricted works staying relatively constant at around 580,000 works.

The week before the OTW announcement, there were 5.27% of restricted works (May 7th) and since then the percentage has risen to 8.25% of restricted works (June 25th).

Other things (like comment bots) may influence the number of restricted works as well.

Do specific categories drive this increase?

I looked at three ways of categorizing works that I had already made stats for last December and compared the changes since then: Rating, Language and Year of Last Update.

All three charts show the percentage of works restricted in Dec 2022, the percentage of works restricted in June 2023 (exact value on June 25th in chart), the rate of increase in percent between these two values and finally the rate of increase in percent for the total number of works in the category. The category names include the number of total works the categories have in June 2023.

For example, for works with No Rating: There were 1,014,291 works with No Rating on the archive in Dec, of which 3.6% were restricted. Yesterday (June 25th, 2023), there were 1,129,002 works with No Rating on the archive, of which 4.89% were restricted. The percentage of restricted works increased by 35.85% and the number of total works by 11.31%.

Rating:

The percentage of restricted works increased most for Teen and Gen (~50%) and least for Explicit (~30%).

In the last post it was brought up how creators of Explicit works may be more hesitant to make their works archive-locked, because people may only want to interact anonymously.

Language:

The percentage of restricted works increased most for Indonesian and Filipino (~80%) and least for Chinese (decrease by ~10%). Indonesian also has the highest percentage of restricted works at 11.35%. The number of restricted Chinese works has actually increased by 600, but since the total number of works increased by about 100,000, the percentage of restricted works fell.

I suspect the drastic variation between languages could be because the reddit post and ao3 announcement spread differently in these language communities. And obviously many other things might influence language communities differently, like political events in the countries the language is spoken in. We also speculated a bit on the reasons for differences between languages in the notes of the old post.

Year of Last Update:

The number of total works decreases in the later years, this will be mostly because people update their works and that pulls them into being updated in 2023. Interestingly, though, we can probably see the archiving efforts in the increase of total works in the years before 2010.

We can also see that even though older years had and have a higher percentage of restricted works, the more recent years experienced a larger increase of restricted works - likely because those users are more active on ao3 still.