Orange was a kid that required a lot of love, and took time and patience to love. As he grew up, he decided to project the love that was given to him, outwards. His "hell tantrums", are brought up as a joke among the Others. It's something they joke freely about, because they know it's all jokes.
They know none of them mean anything by it, even Virgil that may not like them knows that Orange was just a bratty kid. Even Virgil knows to not pick on that, and even if he says it, makes sure that it's clear (at least to the Others) that he means no serious harm. Whenever Patton brings it up however, it's malicious and meant to hurt.
That's the clear distinction between the two. Some of the words may be the same, but the difference is the intent. And Orange knows that. That's why he doesn't take offense to Virgil's words.
Roman may pick on it, but he knows not to go too deep. And Orange knows it's either said without thinking, or from a place of ignorance, so he glances over Roman's words. Logan knows not to even bring it up.
Patton tries to say Orange is just being discriminatory, because he takes offense to his words and not anyone else's. Orange kindly and patiently (in the most passive-aggressively malicious way he can without being called out) explains the difference. Patton knows Orange is looking down on him when he explains it, and tries to say so, but everything else either didn't notice or ignored it.
Oh, you know Patton brings up the tantrums that are off limits, even as a joke. The ones that were started because of something really awful happening, the ones where Orange found himself blacking out for days on end, the ones where his eyes were practically bleeding from how much he was crying.
Orange would call out for Janus relentlessly during these, endlessly asking when he was going to get home. Sometimes he felt the tantrums coming on in the middle of the night and desperately called for him while he helplessly trudged through the dark.
Patton pokes at those moments with the most cheery, sickly sweet demeanor he can muster. Making fun of the ghost like wails Orange made while saying "Dee" over and over. Commenting on how the house was such a mess for weeks and how it was so noisy he could wake the dead- oh, see what he did there? Because he actually has before!~
Patton really loves making him feel uncomfortable and guilty, for daring to be born and as a Dark Side, no less. He feeds off of Orange's insecurities of feeling like he was a burden to Janus and the others. That Janus was annoyed by always having to come and save him, that Remus had to act silly to cheer him up too many times, that Virgil had worsened anxiety because of his outbursts. Patton thrives if he can get Orange to shrink, even a bit.
If Orange starts crying? Janus pops up. His big brother senses have tingled. Patton is sweating bullets. He has seconds- seconds to either await his fate or run.
