lossfunctions

They are a window to your model's heart. Contribute loss functions to @karpathy. It doesn't matter if your loss functions are flat, converge, diverge, step or oscillate (or any combination of the above). All loss functions are computed beautiful in their own way and are sought after with equal tenacity.

[lossfunctions]

Tired: loss minimalists. Wired: loss maximalists.

by @sharifshameem :)

[lossfunctions]

Datasets with a data loader without a shuffle after each epoch? Generously contributed by @richardgalvez.

[lossfunctions]

A highly amusing specimen from @_karfly . Truly baffles the mind.

[lossfunctions]

“The Snek”, a gracious contribution from @TheReibel and Vicki :)

[lossfunctions]

Evades diagnosis. Graciously contributed by @bleyddyn.

[lossfunctions]

Another loss function contributed by Ray Zhang. Diagnosis impossible.

[lossfunctions]

A heart rate or a loss function? :)

This one of a custom implementation of an RNN, graciously contributed by Ray Zhang.

[lossfunctions]

Blue: baseline. Red: attempt to create a new architecture :D

Contributed by Hyun Jae Kim.

[lossfunctions]

An educational post! We’re looking at the validation accuracy of a model as a function of dropout we train with. This trend is consistent with my overall experience: models with dropout train faster, but models with higher dropout win eventually. The dropout of one model is quite extreme (0.85), but it is gaining on the others! What’s going to happen as we train longer? #soexciting

[lossfunctions]

Spatial Transformer Network identifying right whales, L2 reg and loss plot.

 Contributed by ‏@robibok

[lossfunctions]

"the slow start", contributed by Tom White.

[lossfunctions]

This RNN smoothly forgets everything it has learned. God knows what happened. Contributed by Jeremy, as seen on his blog post https://jblkacademic.wordpress.com/2015/09/02/find-your-dream-job/

[lossfunctions]

Taming Spatial Transformer Networks, contributed by Diogo. For the record, it’s not supposed to look like that.

[lossfunctions]

A nasty-looking plateau. Sometimes. Contributed by @Luke_Metz .

[lossfunctions]

“One survivor” contributed by Taco. Beautiful overfitting curves exhibiting exotic non-U shapes

[lossfunctions]

Ah, the Sharp Corner Loss (SCL). Bad initialization a prime suspect.

[lossfunctions]

A beautiful rainbow of learning! This code is definitely bug free. Learning rate decay might be slightly too high.