#16naughts

[16naughts]

Dev Log Apr 18 - Pro Tip: This ain't Hollywood. Stick at it.

This is a bit of a rant separate from my normal topics about development here that you'll likely never see again, but I really feel like I can save some people a little bit of heartache with this. My forte is (obviously) the technical side, but this is currently a one-man show, so I also have to draw and animate all of the graphics, compose all of the music, record all of the sound effects, design the stages, and do all of the marketing and business stuff. I can afford to do it because of some extreme planning spanning over a decade and a couple of lucky breaks. But, if you want to go fully indie, you will be doing most of this too at some point. So at this time, since it's still relatively quiet around here and nobody can claim survivorship bias, I really, really want to impress on anybody also considering taking the plunge: Your game is not going to blow up overnight. Ever. Period. The concept of the game dev has been extremely romanticized as this David vs Goliath struggle against the AAA industry, and being tied so closely with the flash-in-a-pan culture that is the internet, it's kind of bled over into mainstream culture as well. The cold reality is that if your game makes it to release, you're in the minority. And then if you can crack 5 sales on your first title within the first year, I'd consider that an overwhelming success. When Crescent Roll released on Steam, I counted 75 other titles released the same day. The fact that we sold any copies at all was a miracle in my eyes, but I'm not getting the same response from anybody else. It's always "what a shame, it looks so nice". And this isn't the schoolyard bully pointing and laughing - it's your friends and family trying to be consoling, but in doing so are inadvertently labeling your efforts a failure. They don't know that it hurts. A lot of people in art communities get it, but a lot of the people in more tech-oriented stuff (like game development) don't seem to experience it quite as much: gaining traction takes time. Pizza Tower? 5 years of development. Marketing for it started before development actually began, as McPig was using Peppino in comics and other games. Terraria? Redigit found his team and an audience working on the fan game Super Mario Bros X. Balatro? LocalThunk had 48 wishlists at the end of the beta. Go read his website. Among Us sat for almost 2 years before blowing up, and Henry Stickman was already popular. Nobody ever bursts onto the scene and just immediately grabbed the world's attention from nowhere. If it looks like they did, you're missing at least half the picture. it will be soul-crushing when nobody wants to play that thing that you spent a year of your life on. But making any kind of art is always a slow burn. The key is going to be persistence. And I don't mean just throwing as much stuff at the wall as you can - pick a direction, put out your best work, and just keep pushing it forward. It doesn't have to be free updates: make more good games, post art, whatever. Keep your presence known. Obviously, this is all talking to the dreamers who have been sketching level concepts in their notebooks since grade school. If you need money, you're really looking in the wrong place. And you should really, really make sure you can take care of yourself financially before venturing out into no-man's land here, 'cause you'll be in the hole for a while. So this isn't so much an inspirational thing, as more of a warning for others considering going full indie - it's tough. You'll need discipline to get there, and then more discipline to keep it up. I thought I was over-prepared for our jumping-off, and we still ended up having to defer features for after release, and still don't have a real trailer yet. So, if your plans require overnight success, you might want to stick with your day job for a bit longer until it doesn't have to. Which it won't. Trust me.

[16naughts]

Dev Log Mar 7 2025 - All Hands on Deck

Crescent Roll v1.0 is now live on Steam, but with it being something of a soft launch, there's quite a bit of work still to be done. And our first order of business is getting together a Steam Deck version so the more players can take advantage of those all-too-essential motion controls.

The Steam Deck is a Linux machine with a custom OS (aptly called SteamOS) that has a compatibility layer called Proton that lets you run most Windows games straight out of the box. However, Crescent Roll is not most Windows games. We use .Net8 and WebView2 on Windows, neither of which Proton is able to efficiently emulate straight out of the Steam library (you can do some trickery on Desktop mode to get it running, but the point of the Deck is to make it easier for non-tech people, so that ain't gonna fly) Fortunately, we've already thought on this for quite a bit. The .Net platform can be embedded in the executable itself with no need to install anything, which solves problem 1. The replacement for WebView2 is a little more tricky - we have to embed essentially an entire web browser into the game. Steam uses something called the Chromium Embedded Framework, or CEF, to show the popover that you see in games, as well as their storefront and a bunch of other little widgets. It's essentially a self-bundled customized Chrome instance that you can stick in pretty much anything. That would be perfect, but unfortunately for us, after searching for days, it doesn't seem to be exposed anywhere in the Steam runtime, so we can't piggyback off of that. Unfortunate. Embedding it ourselves is also going to add a whopping 1GB to our 100MB install, so uh, perhaps not. So, alternatively, there's WebKit. It's to Safari what CEF is to Chrome. Just the bare minimum of a web browser that you can slap into anything. SteamOS doesn't have it pre-bundled, but it's around 100MB-ish, so you know - better than CEF. So, there we go. Except, CEF expected you to just drag-and-drop libcef.so into your project, and WebKit wants to be installed separately. Which you can't really do on the Steam Deck. So, what I've been doing for the past week is chopping up and patching pieces of the WebKit2GTK project to get it running on the Deck under our sub-directory. Which has not been remotely fun, and will probably get its own write-up here to help out anybody else doing the same thing. But...

It works. Well, as of writing this, it's mostly works. The audio isn't playing, and we're locked at 20FPS display even though the game runs at full speed and is only using 10% of the CPU, but it's actually entirely playable. We were hoping to start releasing a weekly update every Monday starting next week. Fingers crossed we'll be able to iron those last two issues out for it. Although this is big enough that if we're close, we might just wait for Tuesday to ship out both at once. Fortunately, nobody seems to be knocking down our door for this, so we've got a little bit of leeway.

[16naughts]

Dev Log December 1

Crescent Roll - The Thanksgiving Update - Steam News

15 new levels, extra cosmetic info, and a significant tech upgrade.

Happy (late) Thanksgiving! v1.6.0 of Crescent Roll is out, so now I get to do a bit of a status update.

Initially, v1.6.0 was going to be all about the Tiny Shade Garden, but once again, we had to adjust our plans a little bit. We mentioned previously that Android and IPhone versions were in the works, however they've gone a bit dormant due to us wanting to get to v2.0 before releasing them, as pushing updates to 4 different SKUs as fast as we do would be rather annoying. Because of this, Google gave us a bit of a scare at the beginning of the month, threatening to deactivate our developer account if we didn't do anything with it. Getting that account set up took a solid 2 months and lot of hair-pulling, and so it was war-room time to figure out if we needed to rush production, or if there was anything else we could do to keep our position. We got the Android build up and publishing again, created a beta store page, and now have a way to go through Google's special signature stuff. Fortunately, that was enough to satisfy them as being "active". Although it ended up requiring quite a bit of technical changes behind the scenes, which is why this update is just a little light on the cool new big stuff compared to the previous ones.

We're now looking at v1.7.0 scheduled for sometime around Christmas/New Years Day, and it's probably going to be a significant number of changes to the way the modding system is structured, potentially with it coming out of beta and Steam Workshop support, if all goes right. I really, really want the level editor to be included as part of that, but before that happens, the UI toolkit needs a few more bells and whistles, so it may or may not make it. As for the TSG, it plays a significantly larger part in the next title, so some of the previously designed enhancements may be deferred until that is a little more solid and closer to being out.

So, the current update schedule looks something like this: v1.7.0 : Late Dec 2025 - Mod and Workshop Update v1.8.0 : Feb-Mar 2026 - Campaign Update v1.9.0 : Apr-May 2026 - VS Update v2.0.0 : Jun-Jul 2026 - Clean Up and Android+IPhone Release

And that's pretty much going to be it on the really big new CR features. We set up stages and hats and stuff to be added in perpetuity, but that's content that could technically be done by anybody thanks to the modding system. Granted, this is still subject to change, as we're a very small team, and any time the wind blows we get pushed around and have to re-orient. So dunno - that's at least our current intent. We'll see how it pans out.

[16naughts]

Dev Log November 1

Crescent Roll - The Halloween Update - Steam News

Tiny Shade Garden, 5 new Cookie Jar stages, and a lot of other little treats.

Happy Halloween! v1.5.0 of Crescent Roll just came out, which includes the first version of the new Tiny Shade Garden. Though this is the dev blog and not an ad, so you'll have to read the Steam post if you want the finer details. Admittedly, it's also a little lightweight on stuff to do with the little beasts just yet, but we might just have something good here by Thanksgiving.

Going forward, I'm only going to be posting here for sprint postmortems and planned stuff after new versions rather than the preview kind of thing we were doing weekly before. A lot of the "getting off the ground" stuff is over at this point, and turnaround on features is either so quick that there's nothing to really talk about, or in a lot of cases is something that we want to keep a surprise and can't talk about anyway so a lot of the previous updates just turned out to be fluff.

So, we've already done a few posts about the development of the TSG, and there's not a whole lot more that we could cover without getting into the next update. The TSG in the code is considered completely separate from the rest of Crescent Roll, so for anyone looking to modding it, it's not quite supported at the moment. We had to take additional care for how modded items affect the stat growth of the Shades, as well as how transfer of those Shades to other games would work if they have modded appearances, so it will likely be another update or two before that is all opened up.

To recap, the next update is probably going to be somewhere around Thanksgiving, if all goes well. It's mostly just some things for the TSG that we weren't able to completely polish in time. Probably a few more stages and hopefully another set or two of Challenge Mode levels - the typical Holiday style updates we were doing in the Spring again. Christmas might have something bigger, but it's too early to tell at this point, so no promises.

We have hit another issue with the Linux WebKit stuff becoming significantly harder to deal with - our testing apparatus on Windows to emulate a Steam Deck setup had a massive performance hit to the point where it's completely unusable. Ironically, the Steam Deck itself now runs faster than ever, so it's something Microsoft changed with WSL2 interfacing. Fortunately, checking everything in CR is pretty quick, but we did have to scramble a little bit to fix a coding discrepancy where our version of WebKit was missing some Iterator functions somewhat last-minute. So, at some point in the near future, we're going to need to take the time to set up some real testing infrastructure, especially once the other platforms come into the picture.

So, that's it for now. Since we're going to be doing these updates more sparingly, it's probably better to email us rather than posting here if you want to get in contact. We're looking for feedback about what else needs to be included in the modding framework, so if you have anything that you want at all, definitely reach out.

[16naughts]

Dev Log September 26

We're moving right along with the Shade Garden addition. It's a lot of little tweaks and additions, and not a whole lot of fun stuff to talk about. Which seems to be the general theme with a lot of these updates, but you know - that's life in general.

One thing that we've been kind of keeping our eyes on in the background is the continued growth of Linux in the gaming sphere. Since Valve made the Steam Deck, a lot of other hardware companies are starting to jump on the bandwagon. Which, as long as the Steam Deck remains the minimum specs, Crescent Roll should run on any Linux box that supports Steam.

...Up to 1280x800 60Hz, that is...

Call me paranoid, but I still haven't gotten over the pain of that WebKit saga. We've done everything we can to get it running at full speed, but having the final draw to screen being bottlenecked to the CPU by WebKit itself dashes all hopes we could have of getting to 4K on Linux, even though our Windows build can handle it effortlessly. And now since even the Switch 2 does 4K, that's not a good look.

If Valve releases the Steam Runtime 4.0 based on Debian 12, we can upgrade WebKit, and then the new version uses a GPU-based compositor and everything is golden. But there hasn't been any kind of official announcement, so we're still picking through alternatives to WebKit.

Ladybird is a completely new Web Browser, and is looking really promising, but right now it has the same problem that all of these kinds of independent purist projects have - you have to compile the dang thing yourself, and it has a laundry load of prerequisites and takes _hours_ to do so. So please, if you're a developer, and you have a public project with a CI build, also make that build publicly available. I can spin up a VM with any OS in a couple of minutes to test the thing out and maybe help with some bug reporting or reproduction investigation, but we have full time jobs, so it's not worth wasting a week screwing with custom build scripts.

Another, slightly older alternative is Servo, which is based on the Firefox stack. It was mostly dormant for a bit, but this past few years did some major catching up. It's got some weird little quirks about different JS modules that don't exactly work how they do in other browsers, but for our purposes, it's good enough. I think. We've only really messed with the Windows version, and would need some significant development for our engine to be able to bootstrap it for Linux, but from what we've tested so far, yeah - that's a definitive maybe.

And uh, actually, that's pretty much all of the current alternatives. CEF is just flat-out no good, as the Linux version is a whopping 1GB, and the smaller 200MB Windows version never could run in Proton. If we get truly desperate, we could pull a S&ndbox and bootstrap into Godot or Unreal with one of the JS plugins and some custom polyfilling. But that's a lot of work, so fingers crossed that Valve's got our back on this one.

[16naughts]

Dev Log September 19

Our house is the last house on the block to stick up Halloween decorations at this point. It's the middle of September. I'm feeling peer pressured.

The Shade Garden work is progressing smoothly, albeit a bit more slowly than I was aiming for. Code is great and easy. Music is considerably less so, and art is just a slog if you can't catch a wave of inspiration. The little creatures are all running around at this point, but the menu designs are just not coming together. We still have a month or so though before my self-imposed deadline, so nothing to do but keep throwing stuff at it until something sticks, I suppose.

Something kind of exciting for me is that I was finally able to knock down that loading time a little. To summarize, we found that on average for our machines, it takes 1-3 sec for all of the game data to load (and I mean all BGM, stages, animations, the whole thing), and then an additional 10-15 sec for the shaders to compile. Which, we can't pre-compile, and we can't cache due to limitations in the WebGL specification, and we can't switch to WebGPU because Steam Deck can't support it.

So, I took a couple of days to mess around with different parts of the shaders to see what was actually causing it. I changed packing order, tried hard-coding attribute locations to avoid lookups, manually inlined stuff. Just all kinds of weird tricks to poke and prod what the compiler was having such a hard time with. Then we found it: by dummying out the texture lookup code, the shader compilation time dropped to a few seconds. So bingo - we found our root cause.

Crescent Roll uses a batch rendering system for its sprites, where every texture slot on the GPU is filled up (16 for the GL standard), and then it sends the draw call to the GPU. Talking to the GPU is expensive, so this causes a pretty big speedup for the game. This complicates the shaders a little bit, as there's now conditional logic for picking exactly which texture slot to use (as you can't index arrays with non-constants like in most other languages). Conditionals are famously slow for GPU programming, but we've already done profiling for this that, and switched from a 16 case conditional to a 4-level binary tree check for the texture selection to get that running super fast on the GPU. However, what we did not initially notice is that this nested branching seems to be the root cause of the majority of our compilation time woes.

So, what to do? Obviously, we have to reduce branching. How do you do that? OpenGL has the function "mix", which does linear interpolation between two values based on a third value which is 0.0-1.0, and then "step", which is a cheaper non-branching version of "return a > b ? 1 : 0". We can fake a conditional by using a mix with a step as the third parameter.

Observe, the 16-texture switch-off:

I am pleased to report that this strategy drops compilation time from 10 sec down to merely 4 sec on my 1070Ti, while only increasing GPU usage from about 5.5% up to 6.0% on the Main Menu (currently our most demanding scene).

The only unfortunate part is that any work done for the two value arguments for mix are always calculated no matter what, unlike a real conditional which can just skip that. However, in the case of modern GPUs, the next instruction of code is already being executed before the current one finishes, and in the case of a conditional, it essentially has to guess which one to use before it knows for certain. And if it gets it wrong, it has to back up and re-do the other side of the branch. This is the expensive part, as the execution pipeline might be 10+ stages long, and if you have to throw out 10 cycles worth of calculations, that's a lot of energy spent on just generating heat. In our case specifically, all 16 textures are used in rotation, but it's always one incremented after the other, so the same branch path will never be taken twice in a row, and it will always miss (minus when the program would get divided up into multiple cores and the same one would run for the same texture slot every time, but we're getting so far into the weeds at this point as it is). Yes, it's doing the additional sampling(s) on the texture, but two samplings in a row are cheaper than sampling the wrong one and having to backtrack to do the other.

I did try making everything into a mix, but GPU usage jumped all the way up to 10%, and compilation only dropped to about 3 sec. One more mix was about 7.5% and was still 4 sec. Just one mix had no change, somewhat confirming my suspicion of the double work being cheaper than the backtrack. Granted, 10% GPU usage on a 10-year old GPU is small peanuts, and this had zero effect on any of the mobile platforms. But the goal is always fun and small and fast and lean. And it's my project, I get to pick the arbitrary thresholds. So there.

[16naughts]

Dev Log September 12

I'm typing this up as the Nintendo Direct wraps up, and I'm always amazed at the sheer range of games announced and showcased. You've got Overcooked, Stardew Valley, and Suika Game, then here comes Resident Evil and Mortal Combat, then over to Mario and Pikachu and Lego. Every size of developer all sort of hanging out in this mishmash of genres for every age range in essentially a commercial news broadcast. If you think about it too long, it really shouldn't work on paper, but these are now one of The Gaming Events of each year. But everyone has that One Series they really care about, and when it gets its screen time, the hype is unreal. Also Virtual Boy. There's balls coming out of left field, and then there's your third baseman getting thrown a basketball.

Just yesterday, we pushed out patch v1.4.6 for Crescent Roll. I was hoping to hold off on any additional updates until v1.5 here, but, unfortunately, our hand was forced due to Microsoft rolling out an update that caused some flickering on the Main Menu. The scary part was that it retroactively applied to _every_ single version of Crescent Roll, and there's nothing we can do to fix those.

Crescent Roll is an HTML5 game written in Ecmascript (no calling it JS until Deno wins the patent nullification) played in a special desktop harness application that uses Microsoft WebView2 under the hood to essentially use the Edge web browser core for interpreting the code. The practice of baking in a Web Browser with the OS is pretty common now - only Linux doesn't want to do it (which caused a saga with the Steam Deck, but we've already gone over that). So it's write once, run everywhere. For the most part.

At first, I thought it was just my graphics card dying, as I've had similar problems in the past on a laptop well past its prime. Once one of our associates noticed an issue though (on the Steam release that's been there for almost a month), that's when panic set in. We figured out that the GPU was suddenly drawing all of the GL commands too early pretty quickly, but on seemingly random frames. Nobody else was complaining about this online, so it was definitely something somewhat unique to our setup.

The first thing that we did was attempt to switch to using a fixed version of WebView2 instead of the Evergreen one. It was surprisingly easy, (and we've left that option in as a secret in the runtime if we ever need to tell people to use it to fix other issues in the future), and we were horrified to find out that the issue was present even in those older fixed versions - so we now knew that the change was something deep down closer to the display driver, which we just can't hack around. Flickering can cause seizures, so we scrambled to check the CEF runtime that we had on backup, and fortunately, it seemed to be fine, so we at least had an out if we couldn't fix the root issue. But we knew what was happening already, so I decided to go for the fix.

After some "throwing things at the wall to see what sticks" attempts, we stumbled out of the GL calls and into the setup of the canvas itself. And within the HTMLCanvasElement object, the getContext call. We've had to tweak this a couple of times to try and squeeze the best performance out of the system, and there was always a little confusion about what exactly every single option meant. So we went one-by-one, and presto: disabling "desynchronized" fixed the premature draw. Originally, I understood the documentation to mean "don't implicitly push at the end of a frame and instead wait for an explicit gl.Flush()", but apparently, it actually means "just push whenever idgaf". Which, okay, sure. I don't know how that's actually useful, or why Edge is the only one doing it right now. But we found our issue. Problem solved, I guess.

I _suppose_ this might be how they want you to go about doing double-buffering by first rendering everything to a texture, and then just drawing that full screen with some post-processing. Which could technically save some performance if that implicitly flushed when it detected that render to texture. But, I'm just spitballing and that seems kind of risky that you have stuff where you're using that texture render in the scene itself. So, honestly, I have no idea. And I have no idea if this is a Chromium or Windows bug, if it's something with Edge or actually a Display Driver thing, or if this was _actually_ the expected behavior since the beginning and the previous behavior was the bug.

So yeah - even though I wrote our engine from scratch, it's still not really "from scratch", as there's a million pieces below it. And as much as I hate it, that means we'll never get away from other people's changes affecting what we make. But, oh well. At least we don't open ourselves up to stuff like the recent NPM supply chain attack fiasco.

[16naughts]

Dev Log September 5

And we're back. Boston was nice - reminds me a bit of Pittsburgh, except every single building is super-sized. Like, seriously. I think every 3 Pittsburgh street blocks is 1 downtown Boston block.

Silk Song has also released, which uh, might slow down some things. Just a little. The storefronts being unable to handle the demand and crashing is absolutely hilarious in the best way possible. It brings me a small malicious spark of joy imagining there's some high-level execs out there gnashing their teeth in jealousy that this title made by a essentially a team of 3 is going to do gangbusters compared to their multi-billion dollar cooperate franchise, but I digress. Here's to hoping that all independent creators can find even a fraction of this kind of success moving forward. Anyways.

As far as actual progress, there's not a ton to report on this week. The standard structure for releases is the One Big Version Update that takes about a month or so to make, and then after that, just a series of minor updates for fixes, additional content, and minor tweaks. And since we're at the start of v1.5.0, most of the stuff is just internal changes to boring engine stuff.

One thing I will bring up though, is our investigation into pairing down the load time at the beginning of the game. Even though it takes only about 10 seconds at most and is hidden by the intro cinematic if you let it play out, it's been a major sore spot for us since the beginning. The entire game is under 100MB, and since we've been very diligent about background loading stuff like the BGM, the game really only needs about 10MB of that stuff to get into the Main Menu, which loads in less than a second on most modern PCs even if you're using an HDD. Then, we do the rest of the stuff passively, so technically, if you're stupid quick with a stupid slow computer, you could hit another loading screen when entering any game mode, but I've personally never been able to. The thing that is eating up the entire time, is the shader compilation. Literally, it's 95% of the startup time.

It was sort of a major breakthrough to be able to play the intro animation without stuttering back in v1.4, but it actually didn't do anything to improve the full time taken for the startup - the moon just rotates without hitching. Even though we're using Ubershaders, the full program is less than 100 lines, so it's not the size. There's no loops for unrolling, or any of the other 'gotchas' that we've read about so far that could be contributing to the time, unless the branching is somehow blowing it up, but no conditionals are nested. There isn't any support for saving or loading the bytecode, so caching after the first compilation for subsequent runs is out (although when running in dev mode, we have hot reload, so it's literally only the first time you start the exe that it's slow). The extra frustrating part is that compilation is super quick on mobile, but it's only on Desktop that it takes forever, especially with certain Intel chips.

There has to be something that we're missing that's causing the load to be so abysmal (well, abysmal relatively speaking). Theoretically, we could jump ship over to WebGPU, as that's reportedly 3x faster in most cases, and allows bytecode storage in some cases, but the WebKit version we have available for the Steam Deck doesn't support it, so it would need to be side-by-side. So, I'm going to continue to investigate sort of in the background. Adding the Garden is far more important than a one-time performance penalty at startup. But man, those 10 seconds bother me.