The incredibly obvious answer to that very annoying question (With regards to my country, the USA) is that the economic and geographical landscape of America was built by racism. Wealth moves from generation to generation and there are a number of concrete, discrete, historically identifiable moral atrocities which have played into the current dynamics of racial wealth.
For fuck's sake, my dad is old enough to remember de Jure segregation!
Which means that, if you're black and have parents about my age, the first few years of their life may well have been spent in a segregated town.
The assertion is that this will have had some kind of effect on their earning potential and ability to lay down solid foundations for their children to build on.
Imagine that, as a child, you murder my father, steal his money, and burn down my house. I don't go to college because my mom can't afford it and I end up making minimum wage.
I think it's absurd to sit their and ask, "Gosh I don't get it how is your poverty different from anybody else's?"
Racial wealth gaps were deliberately constructed as a matter of explicit policy for the first three quarters of US history, that's why "we" focus on them.
For christ's sake, you said it yourself!
"Hurting someone's ability to gain money or maintain wealth is probably the best single-metric measure of how much something hurts them, since money is used to buy almost everything except social relationships, and even buys the tools to help with that, too."
So if, for example, it was the overt policy of a country to specifically hurt the ability of certain races to gain or maintain wealth well into living memory, then the psychological focus of that country's people would be on...