The idea trans men can "pass easier" comes from a place of privilege, survivorship bias, and a lack of understanding of trans masc oppression.
Society sees trans men as delusional girls, as people breaking out of its strict concept of femininity, and treat then accordingly.
This means that most trans men will be placed into situations that prevent any form of freedom, like child marriage, arranged marriages, being forced to be a caretaker of a house, ect.
This means that society will degrade trans men and make them "know their place" through rape, sexual assault and pregnancy. (A very common occurrence as we have the highest sexual assault statistic of any minoroty group with 1 in 2 trans men facing it)
This means society will treat trans men like shit until they kill themselves, which works as 50% of us from ages 11 to 19 attempt suicide.
And this means trans men must detransition themselves for their safety and to get help.
This blocks a significant amount of trans man from transitioning. So, let's take a look at who we have left.
Those who are in the middle of transitioning are most likely to be seen as lesbians by others, not as transitioning trans men.
And those who have easier access to transitioning, meaning they have white privileges, parental support, and/or the money to transition.
Of course, people who have white privileges, parental support, and/or the money to transition will have a much easier time transition. They have less pushback from their communities, can buy packers and binders, have the money to get clothing that hides curves, etc. They also have the support to be able to be way more vocal about transitioning, too.
So, to summarize, most trans men cannot transition due to outside circumstances or die before they can, causing a small portion of us to be able to transition. Those of us who can are usually seen as lesbians by others if we don't pass and those who have an easier time transition are people with support/money/white privilege to do so, which also allows them to be more vocal about transitioning.
As such, when people look at trans men who are transitioning, they only ever see the minority who are vocal about it, and assume that all of us have it easy, when in reality that isn't the case.