I could offer some headcanons.
Maxwell has been through a lot and it still takes its toll on him. He tends to shun cheery things . Some evidence:
"Its power is spent. Now it's just garishly cheery."
Holiday Cheer- "I prefer my fuel more nightmarish."
Cawnival Sapling- "It's so... ugh... whimsical."
Egg Scramble (playing)- "Such pointless frivolity..."
And then there's this one
Eating Holiday Cheer- "It tastes bitter.
My interpretation is that Maxwell feels a measure of guilt for what he's done, especially failing Charlie, and doesn't allow himself to enjoy things because it feels wrong. Wortox, meanwhile, is all about being silly, frivolous and cheerful. Wortox likely even goes out of his way to try and cheer Maxwell up (though admittedly a big part of that might be by pranking him), which Maxwell would likely find particularly grating- at least Wes doesn't speak.
Also, part of how Maxwell stays relatively sane is preserving his dignity (including being dapper). Wortox undermines that with pranks and mockery.
On a slightly deeper level, I could even see Wortox coming across as monstrous. Now, I'll admit this in dependent on Maxwell knowing Wortox's backstory (I think he probably does and might have even been Possessions' narrator, but I'm not certain). Where Maxwell withdraws out of guilt, Wortox acts as hedonistic as ever. He'll complain about having to eat souls, but also make jokes about it. I could see that coming across as insincere. He might just be unrepentantly using the Survivors as playthings: a most unwanted reminder of the things Maxwell himself has done.
So yeah, there's a major disconnect there. I'd say Wortox probably thinks Maxwell secretly enjoys his company.
Terrarium (no boss setting)- "It seems nobody here likes pranks." - Wortox
That line implies that he actually would hold off on pranks if he thought no one else would enjoy them. Though you could easily argue that while Maxwell doesn't enjoy that sort of thing, other survivors might really like seeing Maxwell harmlessly humbled, I suppose. I find it an interesting dynamic where both are somewhat in the wrong. Their ways of coping with the world are just so opposed to each other.
Quick aside, I have a headcanon about Wortox and the Terrarium. He saw an object that could potentially be used to transport beings between universes and stole it for the other survivors. Going to Terraria wouldn't be the same as going back home, but it might be a crucial step in that, or at least give them more comfort/choices.
Of course, we know it didn't work out. The terrarium broke. Why didn't he tell the survivors what he'd intended?
Sealed Portal (switch off)- "I won't tell the mortals it's here, it'll just get their poor hopes up."
He didn't want to upset them. Better to let them think it was just a souvenir.
He probably had to fight the Eye of Chtulhu for it and assumed the eye was the rightful owner- the eye was actually a more malevolent demon (or rather a small piece of a huge one) trying to steal it too. (his part is based on Terraria's lore.
But admittedly, there's no direct evidence that this is what happened. I just think it fits in with his sweetness and being more apologetic about stealing than he was in the short. It could be that he just wanted a neat magical object, or to see what kinds of pranks he could pull with it.