Both phalloplasty and vaginoplasty involve replacing one genital with another, they're fundamentally both a net zero for the amount of genitals someone's body has. However, when cis people talk about vaginoplasty they talk about it as removing a penis, and when they talk about phalloplasty they talk about adding a penis. Like, I've heard so many transphobes talk about trans women's penises being cut off, I've never heard them talk about trans men as losing their vaginas or clits when they're fearmongered about. And most fearmongering about phalloplasty is about being disgusted by neopenises as a constructed organ, something you kind of see with neovaginas but the loss of a penis is still the overwhelming narrative there.
It really shows how misogyny views female genitals as fundamentally lesser then male genitals. Vaginas aren't given equal weight as a part of a human body, they're essentially treated as an absence of penis first and foremost. A body with a vagina instead of a penis is seen as lesser, while a body with a penis but no vagina is never thought of as lacking anything. A penis is viewed as the god given symbol of the patriarch granting him is power, a vagina is simply viewed as one of many holes that he can use. So trans men are viewed as stealing this divine power like Satan emulating God's throne, while trans women are merely seen as casting off this divine power, like an angel actively rejecting heaven. (I realize that fallen angel imagery works with both of these, and that's because the fascistic strong but weak paradox applies to how Christians view their own mythology about demons, even though its a mythology they invented.)
Also, somewhat tangential: if you want bottom surgery that doesn't give you net zero genitals that's a thing you can do. There are surgeries for being smooth like a doll, and there are surgeries for having both a penis and a vagina.