The thing about Thor and Loki's relationship that is unfortunately very overlooked is that it isn't equal at all.
And what I mean by that is not even Odin playing favorites and Loki being jealous of how Thor is treated, that's all external issues.
Within the relationship itself, they aren't equal. Loki doesn't get a say in when they are going on adventures. He gets no say in where they are gonna go and what they are gonna do. No say in anything, to be frank. And no right to complain, no right to have opinions or feelings, because the moment he expresses a thought or a feeling that isn't overwhelmingly positive, he gets shushed, told to know his place, etc.
The relationship isn't equal because Thor holds all the power in it, and takes advantage of it frequently.
Thor surrounded himself with people who bully Loki, and if Loki wants to maintain a relationship with him, he has to suck it up and tolerate them. Thor loves dangerous activities and Loki, if he wants to spend time with Thor at all, has to participate in them to. Etc.
Technically speaking, Thor doesn't force Loki to do anything. Except he withdraws from Loki if Loki doesn't act like he wants him to. Except he straight up sides with Loki's bullies if Loki doesn't do as he say or approve of Thor's decisions in their entirety. Except if Loki wants to spend any time with his brother, he has to do it on Thor's terms.
We don't see Loki making any decisions or having any input when it comes to it: Thor decided to go to Jotunheim, then Loki has to too. Loki is unhappy? Dramatic and cowardish of him. And so on and so forth.
Thor misses his relationship with Loki because Loki was truly his shadow with no objections and no feelings of his own. (Because being unhappy if you are Loki is punishable in that family). Loki doesn't miss it nearly as much because it was all him pushing his own discomfort and wishes aside in order to keep the relationship going.
No, seriously, name me one time in the MCU in which the brodinsons don't spend time together the way Thor decides (demands) to. When they don't do what Thor wants to do. One time when Loki had an emotion or thought of his own and Thor didn't immediately react to it in a mean or borderline violent way.
Have Thor ever spent time at the library with Loki? Have Thor ever asked Loki if Loki needs help with anything? (Whenever Thor has a problem, he demands Loki to help fix it, but has Thor ever helped Loki with his troubles? No, he doesn't even consider that Loki might need help with anything).
Of course Thor doesn't do anything when he suspects that Loki might be controlled in NYC. Of course he calls Loki's reasons to be unhappy "imagined slights".
Because their relationship is truly is one of a man and a shadow. A shadow will do as you do, it has no choice but to do so, and when was the last time you even thought of your shadow?