Saying that any activist movement must be "for" everyone is an inherent contradiction (and also notably vague, if someone is referring to actual activist work and direct action then they need to specify that, because feminism can also refer to theory and discourse; I am going to assume you are referring to people talking about the former). The whole point is that the activist work is focused on helping a specific set of marginalized people and addressing issues that largely affect those people.
Feminism is simply one activist movement and it's focused on the liberation of women who are oppressed on the basis of their sex. Unlike other activist movements, it may be considered unique in that it necessitates recognizing that males exist as an oppressor class, and that males benefit from male supremacy, even if they are oppressed on another axis of oppression. Could some men oppressed on those other axes of oppression potentially benefit from a world that eradicates misogyny? Sure, and there's nothing wrong with that. But again, activist work focused on one group cannot by default be "for" everyone because then that work would no longer be considered specialized for that group. Even if specific feminist activist groups choose to work on helping one subset of women who are impacted by other axes of oppression, the work being done would still be considered feminist as long as the primary goal is to help women and girls, i.e. female people.
If you believe that statement is wrong simply because you insist that female people don't really exist as most people who ask the question you did tend to believe, then you've been brainwashed. The fact is that we do exist. We are half the population. It's not a coincidence that nearly all of us who were born with a vagina also happen to have the same reproductive organs and functions, and are at risk for the same potential medical issues at different points of our lives. Believe it or not, fighting for female people means that anyone who is born with and develops a female body naturally, even with a DSD, can obviously benefit from feminist work without erasing the language needed for class solidarity. A woman who has had a hysterectomy will still benefit from a feminist organization fighting against medical misogyny without needing other women to be called "uterus havers." Ask yourself, who benefits from denying women solidarity and class consciousness?
And wouldn't you agree that it's extremely inappropriate and rude to ask other marginalized groups to fight for anyone who doesn't belong to that group and particularly their oppressor class? Do you ask groups based on activism for the disabled to fight for non-disabled people? Do you ask civil rights groups focused on black Americans to fight for white people? Wouldn't you also agree that forcing other marginalized groups to be crammed in under one movement would result in ineffective action for everyone?
I hope I've answered your question, but I also suggest you think seriously about the moral and ethical nature of the wording in your original question, "CAN ONLY EVER BE FOR..." Not a single oppressed class should ever be subjected to condemnation for not helping its oppressor class. No group is obligated to divert money, time, and resources to helping its oppressor class. If you don't agree with those statements, then there's no point in engaging in further discussion.