Okay, lets play a little thought game. Imagine for a second that we're in an alternate universe where everything is the same, except there's a mental illness that (among other things) obliterates your gender identity, turning you agender. Like it's got a lot of symptoms, but that one's a mandatory associated feature. Nonbinary people still exist in this alternate universe, but the mental illness that makes you nonbinary (MITMYNB) is broadly thought of to be the cause of nonbinary identities, and nonbinary genders are not yet recognized or understood.
Naturally, everyone on earth sees these two groups as being identical. The broader public assumes all nonbinary people to have MITMYNB, it isn't understood how one could be nonbinary without suffering greatly from all of the illness's other symptoms. Some nonbinary people even claim to be happy about being nonbinary, when MITMYNB is known to cause crushing dysphoria and depression over one's loss of gender identity. They're appropriating the struggles of the mentally ill for attention, if they were really nonbinary they would be miserable, they'd be lining up for conversion therapy, (which works sometimes in this universe because it is occasionally an associated feature of a mental illness but causes incalculable pain to those who are nonbinary but don't have it)
Of course, the only thing that nonbinary-gendered people can do is create tightly knit communities and enforce dogmatic validity as a defense against the impossible weight of invalidating wrought on them by people who are, for all they know, in the same group as them. Remember, we know in our universe that "nonbinary" is a gender, but they still don't. Mantras form. You don't need to be mentally ill to be nonbinary, you don't need a diagnosis, there is nothing to be treated, you are valid just the way you are. Which, while it is probably exactly what needed to be said to some people, it ends up further confusing and invalidating people who are suffering from the mental illness. "What do you mean, I am seeing spots and forgetting my last name, I don't want to be valid, I need help, I need community." Screamed helplessly to people who can't fully understand, who believe they're being told that their earnestly held gender identity, their selfhood, is a mental illness. No one wins. This is how I feel about plurality.