(different anon, on the topic of gender and genitalia/bodies in omegaverse)
(cw: i do not use the terms "afab anatomy" or "amab anatomy" because i think it muddies the point i'm trying to make. i use the anatomically correct terms for the specific body parts in question).
this is something i have also thought about a lot. the central issue is that there are 6 sex (2 primary, 3 secondary) but only 2 reproductive roles in human biology: the normalized male role (providing sperm) and the normalized female role (providing eggs, carrying pregnancy, giving birth, providing milk). worldbuilding for omegaverse requires figuring out how those roles should be divided between 6 sexes. at least, if you're trying to create a world that holds up to scrutiny, which i know not everyone cares about.
most people have clear ideas about what divides alphas from omega. they have less clear ideas about divides betas from either because most societies (especially not western ones) don't have a normalized third gender to drawn inspiration from, which is why betas as a concept as often neglected or relegated to being "normal". seriously, people seem really averse to the idea of betas having scents or instincts and i think it stems from beta "normalness" or "neutrality" being their defining trait distinct from alphas and omegas.
but accepting this creates the problem that there's no gender or reproductive trait actually shared between beta men (who are just regular men) and beta women (who are just regular women), so why are they grouped together? under the common framework, i often wonder why beta men aren't just considered lackluster alphas and beta women considered lackluster omegas. they're often treated like that ("oh no, omegas are rare so i guess we have to settle for [shudders] beta women"), but not categorized like that.
and least clear is what divides men from women in omegaverse. which partly seems intentional since omegaverse is often used a metaphor for irl gender issues or to put distance between the author/readers and irl gender issues. the concepts of maleness and femaleness still exist in omegaverse, but its often not explained what they mean or why they matter in that society. like concept of betas, the concepts of omegaverse men and omegaverse women are just the elephant in the room that the majority of people are incapable or unwilling to address. as i alluded to earlier, i think its because people aren't sure to divide 2 irl reproductive/gender roles 3-6 different ways.
alphas are assigned the normalized male role, which feel intuitive for male alphas but contradictory for female alphas. omegas are assigned the normalized female role, which feels intuitive for female omegas but contradictory for male omegas. to me (to me!) the dual-sex female alphas and dual-sex male omegas feel like a way to reconcile this contradiction. female alphas have penises because they are alphas and vaginas because they are women. male omegas have vaginas because they are omegas and penises because they are men.
this is a theory, not a judgement. i do not think most omegaverse writers believe that all women must have vaginas and that all men must have penises (as many of them are trans or trans allies). i make a clear distinction in my head between irl men and omegaverse men / irl women and omegaverse women, and i believe most people do as well. i do think that when faced with the question "what distinguishes people who are assigned female and people who are assigned male in this fictional setting?" people tend to default to "genitals" as the answer regardless of the other traits involved.
i think there are alternatives, like distinguishing omegaverse assigned-men and omegaverse assigned-women with different body parts. in my omegaverse setting, assigned-female people develop breasts during puberty which remain the same size once they are finished growing and respond to the prolonged presence of any child under the age of two by lactating. if there is a baby around, they can feed it, regardless of whether they gave birth to that baby. assigned-male people develop small breasts only after giving birth which shrink back to being flat once the child has been weaned.
the social consequence of this is that woman are expected to be pseudo-parents to any random kid in their vicinity and held to the same standard as an actual parent, even if that kid is a stranger. men do not face the same expectation and its socially acceptable for them to only care about their biological offspring (which makes non-biological family dynamics particularly thorny). plus there are the health consequences of persistent lactation (mastitis, clogged ducts, nutritional concerns) that women have to deal with. i once read a fic where a male omega has stretch marks on his chest from feeding his child and thought "that's definitely going in the pile".
as for genitals, i use two different frameworks. 1) alphas have penises and are expected to be tops, omegas have vaginas and are expected to be bottoms, and betas have both and are expected to switch depending on their partner. 2) alphas and betas have penises, with alphas providing the sperm and betas providing the eggs, while omegas have vaginas/wombs and give birth. i've spent ages constructing alternative gender roles for each framework based on the material realities caused by these reproductive roles.
i think that's what it really comes down: how do sex, reproduction, pregnancy, childbirth, and child-rearing function in the setting? what types of normative body structures would lead to those outcomes? before modern science, people had weird ideas about all that stuff, so maybe look to history as inspiration. my second framework was inspired by the concept of paritable paternity, which is the belief that children could have more than one biological father (any man the mother slept with during or shortly before her pregnancy).
another thing: a very popular (if not defining) trope in omegaverse is that character don't know their secondary sex until they present during or after puberty. how does that work? in my setting, for both frameworks, people are born with undifferentiated genitals which then become differentiated during presentation. an alternative i came up with but didn't flesh out: everyone is born dual-sex and secondary sex is determined by which reproductive system becomes active (alpha -> impregnate others, omega -> become pregnant, beta -> both).
in setting where (only) female alphas and male omegas are dual-sex, i often wonder why they aren't grouped together. especially if both are capable of filling both reproductive role? and if genitals are known from birth, how do you tell if a child is a female alpha or a male omega? a male alpha or a male beta? a female omega or a female beta? i read a fic where a couple was told their unborn child was female via ultrasound and though "based on what?"
then there's the other popular trope which is that after presentation characters wake up with a different set of genitals than they were born with, or in additional to their original genitals which are now defunct. so what was the point of their original genitals? why would their body waste resources like that? it makes sense in bitching/studding fics, but not when its their presentation.
actually, as i'm writing this, i just came up with a solution. if someone doesn't have kids by say, age 25-35, their body switches things up because "obviously you suck at reproducing using this method so let's try something else". oh, imagine the ramifications this would have on a society where people are delaying having kids en mass for whatever reason (economic depression, resource scarcity, social/political conflict, changing values).
let's not forget that for most people, omegaverse is kink trope. i think most people write based on vibes or their personal preferences and if they care to, work backwards to explain why things are like that. another thing is that omegaverse is often used to let m/m or f/f couples occupy the same plots and tropes that are typically reserved for m/f couples, so they're not trying to construct alternative gender roles, just play around with existing ones (again, observation not judgement).
if one really wants to think deeply about what makes someone an (assigned) alpha or beta or omega, or a (assigned) man or woman, then there has to be a reason why that distinction is acknowledged, why people care about it, and how it effects their experiences in society. not necessarily a good, justified, or important reason, but a reason.
Anon! this is the best message I've ever received. thank you so much for giving me so much food for thought that I literally had to compose my thoughts in a google doc. I wish you weren't anonymous lol, let's be buds.
Lot of material here, let's start from the top. TLDR: I basically just "yes and" your thoughts. PSA: this response is only about omegaverse, not about the experiences of our friends in the misce community.
Omegaverse dynamics are both too strict and too fluid. Too strict because the overwhelming power of traditional views of masculinity/femininity driving the boat. Too fluid because the dynamics lack has solid meaning or distinction, and therefore are meaningless. Let the dynamics have true purpose, not social purpose but each a unique effective/efficient reason that they helped contribute to the continuation of the species.
First of all, omegaverse as a genre was created with a western frame of mind - a mind that is completely infatuated with binary systems while constantly attempting to disavow them at every possible turn. That created the very pleasant big picture omegaverse, in which there are three dynamics, however really none of that makes a lick of sense - not because it's fantasy, but because to your point, there are so few differences between some of the six 'genders' that it becomes very clear that six is just a fancy way of saying 'two'. I agree that to solve the case it all goes back to reproduction (which encompasses copulation, pregnancy, and childbirth). Child-rearing not so much because that can be more socially situated apart from breast-feeding.
There are two ways I think the genre could address this muddiness and become a lot more interesting - tripartite reproduction (in which you need three people to make a baby), or concretizing what is it that makes the dynamics different in a way that cannot be changed. Both of these will never happen because they're horribly unpopular with the people who love the omegaverse the most.
Tripartite reproduction is a real turn of to those that love the omegaverse for it's heightened portrayals of masculinity and femininity, thus the kink of it all. This sort of system would destroy that, because it would require the invention of a true third thing that is not both of the other options, which would likely require the two other options to be completely reworked as well. tbh this actually makes the most sense, as it addresses what the point of having three options is anyway and why one didn't get bred out (especially if alphas/omegas are hypersexual, therefore more likely to pass on their genes, compared to betas).
The second thing, the needing to really separate the dynamics, by boiling each one down to an immovable core, is likely the more damning issue. Let me paint you a picture. Let's say alphas have orange skin, betas have purple skin, and omegas have green skin. There are male and female varieties of each, however the only thing that suggests what your dynamic is in this case is skin color. This then wrecks what many writers appreciate the genre for - the ability to examine, understand, and "play" (in quotes because it's not really play, rather good investigation grounded in the freedom of the pen) with gender, because if you were born orange you can't become green.
Sure, of course this image does nothing to speak of the socially constructed elements of what gender is and does, instead it's only reproductive sex type - eliminating questions such as how does an green act in a way that's different than a purple? It's a bio-reductionist frame of mind (if bio-reductionist is even a phrase) which completely goes against what the genre is for a majority of it's audience. It also then begs the question that if a dynamic is something that you could not alter about yourself, can it even be mentioned in the same breath as the word "gender" at all? In an age where, as I type out these words I can look at my body and think, "there is technology available which would allow me to change nearly 100% of my outward physical appearance", what then! What if this was taken to the next extreme, not only green/orange/purple skin but the bones too. What then?! There could be no fluidity [pending skeleton removal technology].
I've just done a really messy job with a terrible illustration to say that if you live in the west today, you are likely familiar with a more wholistic post-modern understanding of both gender and sexuality that embraces grayness and fluidity. I'm gripped by your question "what distinguishes people who are assigned female and male in this fictional setting" and I raise you "what distinguishes people who are assigned alpha, beta, or omega?!".
Heaven help me something really subversive comes along - a wave of popularity for male omega x male omega, or male beta x female alpha, or maybe everybody says alphas are actually infertile, give me something really interesting to chew on! Get out of here with that double male alpha romance, or the double female omega romance, or the male alpha x female omega, female alpha x male omega.
The majority of omegaverse lovers are casual participants in it for the romance. The second largest majority are in it for the 'play' and investigation of real life circumstances, and finally there are those of us [a tiny huddle] here for the worldbuilding - the complaining and the fixing and the pointing out that it doesn't make sense. But I'm so glad we've found one another.