#optimization}

[askagamedev]

Could you explain "deduplicating data" and why more AAA games don't do this? Helldivers 2 was reduced from 150 GB to 23 GB. With how large AAA games are nowadays, it seems odd that this isn't more of a priority.

In the world of computers, it takes varying amounts of time to read data from a particular source. Certain data sources are much faster to read from than others. Here's the general hierarchy for read times:

Cache hit (1-20 nanoseconds)

Not in cache, read from memory (15-20 nanoseconds)

Not in memory, read from SSD (25-100 microseconds, i.e. 25,000-100,000 nanoseconds)

Not in memory, read from hard disk (5-10 milliseconds, i.e. 5,000,000-10,000,000 nanoseconds)

Not in memory, read from blu-ray disc (100+ milliseconds, i.e. >100,000,000 nanoseconds)

As you can see, cache and memory reads are really fast but reading from either SSD or HDD are extremely slow in comparison. HDD reads are orders of magnitude slower than reading from SSDs, which are also orders of magnitude slower than reading from memory. Reading from physical media like a blu-ray disc is even slower than that. This time cost is incurred whenever the game needs to read data that isn't in memory, e.g. when loading a new area, a new enemy, a new character, effect, etc. from the disk.

One of the tricks for optimizing load times is to minimize the amount of "seek time" it takes for the slower devices like HDDs and blu-ray discs by having redundant copies of the same data on the disk. This is like having multiple kitchens set up around the office instead of just in the kitchen - workers don't have to walk as far to get a snack. It isn't important if the office is small, but having only one kitchen when your office houses a thousand people across ten floors means a pretty massive bottleneck at mealtimes. Multiple redundant kitchens will speed things up because workers just need a kitchen and they will generally prefer the closest available one. Multiple redundant chunks of data will make seek times faster because they're all the same and the reader doesn't have to go as far to get what it's looking for.

Having redundant options comes at the cost of disk space. Having multiple drink stations or kitchens means that space can't be used for a lounge or game stations or storage or more offices or anything else. Not every game does this because they don't always have that option - if the game features a lot of super high res textures and high quality audio, those assets will eat up a ton of disk space on their own even without duplication for access speed. This is a choice the Helldivers dev team made to help speed up access time in lower spec machines. It seems like they decided that the speed savings wasn't worth the disk space tradeoff instead.

[Join us on Discord] and/or [Support us on Patreon]

Got a burning question you want answered?

Short questions: Ask a Game Dev on Twitter

Short questions: Ask a Game Dev on BlueSky

Long questions: Ask a Game Dev on Tumblr

Frequent Questions: The FAQ

[buildroot-official]

I've been thinking about trying out the x32 ABI.

Not to be confused with x86-32 and x86-64, x32 is a Linux-specific ABI designed to combine the computational efficiency of x86-64 with the memory efficiency of x86-32.

x32 uses the x86-64 instruction set, allowing for more registers and faster instructions, while using 32-bit pointers and 32-bit integers, allowing for a smaller memory footprint.

Smaller memory footprint also means that the CPU cache is more effective and less memory bandwidth is required.

Various benchmarks I've seen 'round the 'net show that x32 provides up to 40% less memory consumption and 10% faster performance (especially for pointer-heavy applications like Java) compared to x86-64

However, x32 has a couple disadvantages:

Applications are limited to up to 3GiB of virtual memory space each, like x86-32.

Most applications that have assembly optimizations wont work, or at least those parts wouldn't be optimized, unless the developers specifically add x32 support.

It is quite esoteric, and therefore more prone to bugs, due to the lack of users for testing.

There's a chance it'll be removed in the kernel some time in the near future. (There were talks of removing it back in 2018, but it didn't go anywhere)

As far as I'm aware, only glibc has proper support for x32. Musl has preliminary experimental support for it, but I doubt it'll ever get finish due to the obscurity of it.

[thirdity]

We optimize ourselves to death. Relentless self-exploitation leads to mental collapse. Brutal competition ends in destruction. It produced an emotional coldness and indifference towards others as well as towards one's own self.

Byung-chul Han, Capitalism and the Death Drive

[suliqyre]

We measure our successes and failures in units of practical value that we call wealth or status or fame. We sometimes even see everything in terms of its practical value, to the point that we see value itself as having a practical foundation.

To instead seek out what is impractical might seem almost quaint, as though it were for someone else — a different person living in a very different world. Sometimes we sense that there might be more to our lives than the practical, but this idea gets pushed out by the chorus of voices telling us to focus always on what's practical.

A consequence of this is that it can sound like madness to insist there are moments when we must completely ignore the practical. Yet it is precisely by going beyond the realm of the practical that we become capable of discovering another kind of value, a value that does not depend on this or that, a value that cannot be quantified because it is immeasurable.

Read more...

[thefloralmenace]

This applies to activism but also to the corporate world and regular life: Never underestimate the value of sitting down, analyzing your activities and then optimizing them. Personally, I think the best place to start is with stuff you do on a regularly timed basis, like events that take place monthly, yearly, etc.

I'll give you an example: I was part of an organization in college that did a semesterly event where we did science experiment demos for children. When I initially joined the organization, the days leading up to the event were a little chaotic: The person in charge of outreach would call the school to schedule, research experiments for us to do or try to remember the ones we usually do, make a shopping list, try to recruit people to run the experiments, go all over town getting the supplies together, harangue people into looking up the experiments and the relevant scientific background to make sure we could explain them to the kids, then get everything in the car on the day of the event.

After a couple of rounds of doing this, I got to thinking about how inefficiently we were handling this event, given that we know it's going to happen every semester. So over spring break, I went to the dollar store and got six large baskets and index cards. On each index card, I wrote the name of one of the experiments we commonly do (elephant toothpaste, magic dry erase art, Alkaseltzer lava lamps, etc.). I then wrote the list of ingredients/supplies needed for the experiment on the back of the card so that if you flipped the card from the bottom up, the words would appear right side up. I "laminated" the cards with packing tape and taped one to each basket. They were then labeled on the front, and if you flipped the card up, it told you everything that was supposed to be in the basket. I then looked up the scientific background of each experiment, compiled information with sources, and printed background sheets for each, which I put in sheet protectors. These went in the baskets as well.

When we came back from spring break, our outreach manager gave me the budget, and I went out and did the shopping for each experiment. In past, we had shared common reagents across multiple experiments to be cost efficient, but this time I chose to get duplicate items where necessary (since it was possible with the budget). I put all the supplies needed for each experiment within that experiment's basket, and because I duplicated shared materials, that made the baskets modular - you could pick up one, take it, and be guaranteed to have everything you needed for that experiment in the basket without having to remember to take other baskets (decreasing the mental load).

Suddenly I had streamlined this event from something that stressed out our outreach manager and required a mad dash to look up experiments, make a shopping list, get the supplies, and get people to run the experiments into something that was basically grab-and-go the day of the event. We could just pick up three or four baskets (so that we had experiments to rotate in and out), put them in the car, and go. Even if the people who volunteered to run the experiments didn't have time to research them, they could just read the background sheets in the car.

I also advocated shifting the time for the shopping. I asked if we could find money from elsewhere or double the budget for supplies just this one time so that we could stock for the upcoming event, and then immediately restock afterwards. Why? Because if the time for shopping is shifted so that the responsibility is "buy to replenish" rather than "buy for the event," the shopping becomes lower stress, especially if we can't find something we need. If we shop before the event and can't find Alkaseltzer tablets, we may have to rearrange our entire plan for the demonstrations, but if we shop after the event and can't find them immediately, we have a whole semester to find a source for them. This was especially useful for the elephant toothpaste experiment because we had to order the iodine tablets for it from a water treatment company, and they take two weeks to ship. If we forget to order in time for the event, it's a crisis, but if we order more immediately after the event, they have all the time in the world to get there, and then we're set to just grab-and-go next semester again.

Now this might not seem super important or impactful, but here are the benefits: Our outreach manager was less stressed. Lower stress for the people carrying out the event is enough benefit alone, but additionally, because everyone running the event was less stressed, we were more likely to put on a better quality demonstration for the kids (people are more likely to show up knowing the background, and we're less likely to forget materials). And finally, reducing the stress and effort associated with events saves time - time our organization could use to do additional outreach, increasing our impact.

This kind of analysis isn't glamorous, but as I said, it can maximize the impact of your organization and what it does. This includes activist organizations. Just shifting the time at which the shopping was done for this event completely eliminated supply chain issues. So think about the stuff you do. Think about the timing, what stays the same year to year, what processes can really be done once instead of every time (ex: There was no reason to research the experiments every year - do it once, and reuse the sheets). Also think about creating structure. My baskets were a physical imposition of structure that meant we no longer had to scramble around the building picking up bowls and spoons and common household ingredients. Structure can also be imposed administratively, like delegating responsibilities or digitally, like creating a Google Drive that is organized into folders.

It's this boring stuff that feels like business or administrative busy work that can really help things run smoothly, and if you do this kind of analysis with all your regular events, the time-saving, quality-improving, and stress-reducing benefits start to stack in a big way.

*And of course I acknowledge that there are people out there who struggle with organizational skills in one way or another, so if you can't personally apply this, that's okay. Not everything I suggest works for everybody.

[sangeteria]

thought I proved smth last week. read the proof 50 times. got it looked over by 3 different friends, who saw no issue. looked at it this week. MAJOR flaw, very difficult to correct around. fml

[x11-official]

The constant struggle with web bloat: The Arch Linux Forums

I recently reblogged a post about how many websites today are borderline unusable if you have lower-end, even recent lower-end hardware—which is true of the majority of online users in the Global South, and even some of us in North America and Europe. Twitter was one of the worst offenders, with the website taking nearly 3 seconds to load anything usable even on a high-end laptop with internet better than what most people in the U.S. have.

Unfortunately, the Arch Linux forums, if current plans come to fruition, will be yet another victim of people who have absolutely no idea what average hardware actually looks like.

A caveat: this is old news. But I think it isn't widely known, and if you've ever looked for help on the Arch forums, this will eventually affect you.

---

One specific example the site brings up is Discourse. Discourse is a modern and increasingly popular software package for internet forums. The Fedora forums are just one example (see also screenshot below), but most tech support forums seem to use it now.

A notable exception of course is Arch. If you've ever been on the Arch Linux forums, they definitely don't resemble 2025 web design—they still use fluxBB, an older package that has its roots in the desire to make a more minimalist version of the better-known phpBB, which is already very minimal and fast by today's standards.

Well, the Arch forums, as announced about two years ago, are moving to Discourse, against the advice of basically everyone involved in their day-to-day administration. It may not be set in stone, and progress has been going at a glacial pace—but that is the current line. You can read the open Gitlab issue on the topic at the link here.

I will say that the reasons for a migration away from the current fluxBB are valid. The software is not maintained. Concerns were also brought up about compliance with certain laws (mainly the EU's "right to be forgotten"). Arch should indeed be working on a move. But Discourse is not a good choice.

Now I don't think Discourse looks bad, and it's certainly more "modern" in appearance. However it has terrible performance, especially on low-end hardware, and above all, it has a team who is utterly, rudely uninterested in attempting to optimize it whatsoever.

---

You can click on the first link in this post for more, as Discourse's attitude towards its poor performance is a repeated theme. I will include just a few data points from there:

On a Tecno Spark 8C phone—a $100 phone from 3 years ago, very typical among what people in less wealthy countries are currently using—Xenforo and phpBB sites load in under 2 seconds. MyBB loads in less than 1. vBulletin is slow, over 4 seconds. Discourse takes FIFTEEN SECONDS.

Even on an M3 Max—a high-end laptop by any standard—Discourse takes more than a second to load, compared to a ½ second or less for all the other named forum sites.

Discourse co-founder Jeff Atwood: "we've been trending towards infinite CPU speed for decades now... optimize for the things that matter."

Also from Atwood: Q: "Are 100% of users on iOS?" A: "The influential users who spend money tend to be, I’ll tell you that … Pointless to worry about cpu."

Again, click the link above for more on this, it is utterly damning.

Of course, the Arch forum staff said as much during the linked discussion, and it was like talking to a brick wall. There's also the fact that Discourse doesn't work very well with Javascript disabled, as many Arch users choose to do in their browsers, and that it has many "anti-features" like infinite scroll, voting, and so on.

I will also add a personal anecdote. The article I reblogged focuses on CPU constraints, but network access is also important.

---

I lived in Madagascar for nearly two years of my life. In Madagascar, there is no such thing as unlimited data. Rather, you buy scratch-off cards with money, and then use those to buy temporary "data plans". The cheapest plan, the one you buy in an emergency when your bus gets stuck in the mountains and you have to let your boss know somehow, would get you 20 MB of data usable for just one day.

Some of the websites studied will burn through a quarter of that just loading their home pages. Threads, which I admittedly have never used, burns through nearly a half. I wound up never, ever using certain apps while I lived there because they would eat my entire data allowance in no time. By refusing to optimize, web developers are locking out a huge part of the world.

This is somewhat understandable, though infuriating, for commercial websites that want to attract wealthy users. But it's nonsense when you are a Free and Open Source Software project that delights in being able to bring old hardware back to life.