i think one of the reason why mammon is so intolerable to fon is because of the environment they're used to being in, the blurred lines between conceitedness and confidence for them and the way fon presents himself to them.
we know that in canon, mammon does not enjoy being told what to do. in fact i believe their obsession with money isn't a matter of money itself despite varia being the sins bla bla is the fact that they are obsessed with control. they needed to hoard money because they need to be in control of rainy days.
i mean they quite literally just, blew their money off just for the rings for the arcobaleno battle, which kinda shows how much being in control meant to them, because the curse was out of their control, they and the other arcobalenos were outsmarted into it. so i think that their obsession doesn't stop at money but at control.
and fon's nagginess is just a soft form of control that they could not stand, sure, they could just not exercise, not learn martial arts, not stop their obsession with money because fon is quite literally powerless against them and these choices. but the naggings have to hit nerves more often than not. plus fon, in my opinion, is not the most tactful with choice of words. he is definitely imo, genuinely nice and morally good, but lacks sensitivity in his word choices, which is obvious when he was shocked that mammon was bothered by most of the things he told them.
then comes the actual battle. it's important to note that at it's peak, the only contenders were fon, hibari, mammon and xanxus. and with the use of mirage r, it is quite unsurprising that it doesn't affect hibari and xanxus that much. both of them are super competent, super confident about their abilities and actually look down on people, and do not make it a point to pretend they don't.
while i don't actually think fon looks down on anybody's abilities and whatnot, i think the issue lies with the fact that viper's environment makes them feel like confidence is cockiness, which is a dynamic seen in varia and some of the arcos alike. the former just having that kind of personality while the latter are genuinely the best in their fields, dubbed the "strongest seven" on arrival. that with the added fact that fon seems to, mostly jab illusions, intended or not, just leaves a bad taste in their mouth. he is not seen telling colonnello hey sniper guns isn't as competent as using your fist, or telling verde that being zen in your soul beats science. so to mammon, it may feel like he simply thinks he is better than them and to them, that is cockiness.
so when mirage r fails to hit him, they couldn't tell if it was confidence or cockiness, because even irl, these emotions are hard to tell, maybe the best way to distinguish them is competency. yet, in the khr environment where almost everyone is the cream of their crop and is subjected to being outplayed solely because someone's best is better than your best, the lines are thoroughly blurred. mammon, for probably more reasons than one, already doesn't have an A+ impression of fon, so it is natural they assume the negative. and that piles up with the fact that they just don't buy his good person act.