#gradient descent

[many-blogs]

Watching a YouTube course about how to make a neural network in Python because we’re insane, and the guy said in the intro that you don’t have to know anything about linear algebra or calculus to understand neural networks.

At the start of episode 2 he’s answering some questions, including why there’s only 1 bias for the formula.

Well.

If you knew anything about linear algebra, you’d know why there’s only 1 bias 😒

(It’s because the formula is just y = mx + b, where b = the constant = the bias, and the weight = the slope = m. So the formula/coding a neuron in a neural network is y = mx + b, and there’s an “mx” for every previous neuron. So in his example, it’s y = mx + mx + mx + b. And each individual neuron performs Gradient Descent to determine if it’s active or not, ie the lowest point of all the slopes put together).

[kneadingmaths]

Just an illustration of the standard analogy of gradient descent as mountain unclimbing.

(Also, sorry for yesterday's accidental reblog…)

[dotlilith]

I'll pluck the buds out one by one

[agentnoun]

emotionally devastating myself while planning a Mothership campaign by listening to too much Radiohead

[vintagerpg]

So, if Dead Planet is an adventure and A Pound of Flesh a campaign hub, then Gradient Descent is a stab at a Mothership Megadungeon. That is as horrible and punishing as you think!

Like, regular D&D megadungeons are awful places, you know Mothership is going to make it even more so. The idea here is that there is a massive, abandoned android factory at the edge of known space. The facility is under blockade — the corporations don’t want anyone going in. But people do — Divers — looking for valuable salvage (of which there is plenty). Plenty of danger, too, in the form of blockade enforcement teams that regularly kill everything they encounter, a rogue artificial intelligence with diabolical plans and the Bends, a psychosis in which humans begin to think they are androids. Diving into the the Deep is going to almost always result in death, madness and maybe, on occasion, a tidy profit.

It’s absolutely every bit as punishing and disturbing as you likely expect. A lot of this is artist Nick Tofani’s fault. I hate (by which I mean, love) his horrible drawings. They have a squishy sense of corruption that reminds me a lot of Scary Stories to Tell in the Dark illustrator Stephen Gammell.

I will say that Gradient Descent is a bit punishing to read as well. The text is small, the layout…aggressive. Gradient Descent probably represents the saturation point for how much information you can cram into a 64-page zine. I wouldn’t mind getting this reissued in a roomier version, even if the claustrophobia of the layout lends something to the atmosphere of the adventure.

[onion-souls]

Gradient Descent

64 pages. Full color interior.Written by Luke GearingIllustrated by Nick TofaniEdited by Jarrett CraderLayout by Sean McCoy  Gradient Descen

[datascience-from-the-trenches]

Deep Double Descent

The performance of unregularized CNNs, ResNets, and transformers is non-monotonic: improves, then gets worse, then improves again with increasing model size, data size, or training time.

This effect is often avoided through careful regularization. While this behavior appears to be fairly universal, we don’t yet fully understand why it happens.

Source: https://openai.com/blog/deep-double-descent/

Deep Double Descent

We show that the double descent phenomenon occurs in CNNs, ResNets, and transformers: performance first improves, then gets worse, and then

[eldritch-muck]

The Next Module

sonofabch. i promise i’m not just posting kickstarters, but Mothership.

Gradient Descent

megadungeon. sci fi Mothership megadungeon. yess