I love the idea of mute casters and think sign language is the best way to do it. I'd rule that they can still cast spells while in the area of Silence, but if they're grappled they can't move around enough to cast. This sign casting is obviously available to able-bodied casters too, but you can only learn one kind.
IIRC Silent Spell is a thing as a 3.5ed metamagic feat, and I guess you can house rule that a deaf/mute caster can substitute sign language somatics for spoken components at no extra mental tax.
In other words, under such a convention an appropriately trained deaf/mute caster can treat all verbal components as somatic ones.
I'm a bit curious whenever it work the same with Bards, by the way.
Subtle Spell metamagic is a thing in 5e but it would get a bit taxing on your sorcery points.
Bards can be any kind of performers, so dancers or acrobats could totally be mute/deaf
