I feel the need to defend second person perspective, the most misunderstood and unfairly maligned perspective. I blame the prevalence of xReader for the spread of this perception, but the truth is: SECOND PERSON NARRATORS AREN'T INHERENTLY TALKING TO YOU AND YOU'RE MAKING A VERY SIMILAR COGNITIVE ERROR TO THE PEOPLE IN THE OG SCREENCAPS WHEN YOU ASSUME THAT THEY ARE.
Reader-insert fic (I remember calling it "you-fic" Back In The Day when it was still uncommon) is a very specific subset of (typically) 2nd person narration where the writer inherently intends the reader to insert into the narrated role.
Most 2nd person literature does not have this intention. Even interactive fiction like Choose-Your-Own-Adventures is less rigidly referring to You, The Reader than reader-insert is. CYOA narratives are generally more in line with video games or tabletop RPGs. You can just as easily engage with CYOA from a "player" perspective (the narrator is the GM, addressing the character you have taken on the role of) or a "master" perspective (you are the GM/narrator addressing the character and describing their actions).
If you see two people having a conversation and somebody uses the term "you", do you assume they're referring to you any more than you assume that somebody using the term "I" is referring to you? If you pick up a letter addressed to someone else, that talks about things "you" did with the letter-writer, do you think "Well, I didn't do any of those things!" or do you think "Wow, the person this was addressed to sure got up to some shit"? Do you assume "I Know What You Did Last Summer" is accusing you, the audience, of something?
1st Person Narration assumes the involvement of one (1) party. That is, it is a firsthand narrative of events. The firsthand narrator could be addressing an audience (which may or may not be the reader), but it can also be framed as diary/journal entries, or as bodycam footage, or as if you, the reader, are an invisible entity observing someone's perspective from inside their head. There are many ways to style a 1st person narrative, but in all of them involve describing events from the perspective of the person experiencing them.
2nd Person Narration assumes the involvement of two (2) parties: a person who experiences events firsthand and a person providing secondhand narration of those events. Since there are only two assumed parties, the narrator can only address their narration towards the person experiencing the events. Generally, the first party (narrated character) should NOT be assumed to be the reader unless the writer explicitly states otherwise, and the second party (narrator) should not be assumed to be talking to the reader.
There are also many ways to style a 2nd person narrative: it could be a character A explaining to character B events that character B experienced (maybe B has amnesia, maybe character A wants to compare B to a summer's day); the narrator could be an invisible entity observing someone's perspective from inside their head and providing running narration back to them, and YOU, the reader, are an invisible entity inside the narrator's head.
And for completeness: 3rd Person Narration assumes the involvement of three (3) parties: a person who experiences events firsthand, a person providing secondhand narration of those events, and third party to whom the secondhand narration is being told. The third party can be the reader, but doesn't have to be. The 1st party HAS to be a diegetic character in a narrative, otherwise there's no narrative. The 2nd (narrator) & 3rd (audience) parties may or may not be characters that exist within the narrative.
2nd and 3rd person narration are BOTH secondhand narratives! The only difference is if the narrator is addressing the 1st party or a 3rd party.
SO YEAH. You don't have to like 2nd person narratives. You can feel like 2nd person narratives are trying to direct you, even when they're not intended to be read that way. But maybe don't judge people for not being able to separate themselves from the character in 1st person perspective when you can't separate yourself from the character in 2nd person perspective.
"I'M NOT DOING THAT-" YEAH YOU AREN'T, HARROW THE NINTH IS DOING THAT, THE NARRATOR IS TALKING TO AND ABOUT HER