I mean, let's be honest about this. You don't have to be particularly familiar with the details of Joss Whedon's biography -- literally five minutes on Wikipedia will suffice -- to realize that there is more textual evidence to support any of the other core Scoobies being "Whedon self-inserts" than there is evidence to link this charge to Xander Harris.
Joss Whedon's parents divorced when he was young. As a child he had a traumatic experience involving a friend drowning. As a teenager he had to move to a new school in a new town and make completely new friends. When he was in his twenties his mother died suddenly and unexpectedly. Who, exactly, does that remind us of? (Did you really say 'Xander Harris'?)
Joss Whedon's family were comfortably middle-class. His mother was a teacher and a feminist activist. Perhaps unsurprisingly given his background, Whedon did very well academically. He graduated from one of the most highly regarded private liberal arts colleges in the country. However, he struggled to make friends at school, has talked in the past about being bullied by his fellow students, and also complained that his parents could often be aloof or dismissive if he didn't meet the academic targets they expected of him. Who, exactly, does that remind us of? (Did you say 'Xander' this time?)
As a child, Whedon had a well-documented obsession with British media and culture, particularly shows like The Avengers [not the Marvel comics one: the one with Emma Peel and John Steed]. He spent three years in a private, all-boys boarding school in England. By becoming a TV writer, something he was initially reluctant to do, Whedon was following the footsteps of his father and his grandfather before him. Whedon very publicly presented himself as a mentor figure to younger women. Is there some other character this reminds us of? (Except for Xander, of course?)
But, of course, nobody suggests that Buffy or Willow or Giles are "Whedon self-inserts". People like Buffy, and Willow, and Giles. People (very understandably, very correctly) do not like Joss Whedon. (To be completely clear: I too do not like and I am not interested in defending Joss Whedon.)
Many people do not want to give Whedon credit for any involvement in the parts of the show they like. More than that, many people seem to genuinely not want to think about the idea that somebody who is, objectively, a pretty terrible human being might actually have been responsible for a creative work that means something significant to them. I have read multiple posts where people very clearly struggle with the idea that they might enjoy something Whedon made, knowing now the sort of person Whedon is and what was going on behind the scenes of the show. (This last part is not something I can really relate to personally: the fantasy that only good people can create good art is just obviously false and the sooner you abandon it forever the better.)
Meanwhile ... yes, Xander is often awful -- sexist, entitled, rude, creepy, emotionally manipulative, self-absorbed -- and often in ways that the show doesn't seem to really acknowledge. It is very easy to dislike Xander a lot of the time, especially in the earlier seasons of the show. (Although I would point out that many of Xander's worst moments happen in episodes Whedon himself didn't write, that the show is more criticial of Xander than many people like to admit and that insisting that Xander is nothing but the sum of his worst elements means refusing to engage with the show on the level it wants to be watched. I would also point out that Xander's flaws are hardly unique to him: a lot of the show is written in a way that undermines its public commitment to feminism.)
And yes, you can find interviews in which Whedon will talk about how Xander is "based on" Whedon himself. (Or, at least, that some fantasy version of Xander is based on some imagined fantasy version of himself: the particular quote I'm thinking of has Whedon describing Xander as somebody that "all the girls wanted to be friends with in high school" which is both a bizarre take on Xander Harris as actually written and a pretty weird thing for somebody who went to an all-boys boarding school in England from the age of fifteen to say about themselves).
Xander is "based on" Whedon to the extent he gets to voice things that one suspects Whedon wishes he'd gotten to say when he was a teenager. But Xander is not a "Whedon self insert" in the way anybody in 2025 uses the term.
Joss Whedon was not, in fact, some random creep who won a contest to get a character "based on himself" inserted into an otherwise perfect show with flawless feminist credentials against everybody else's wishes. Xander is (somewhat) based on Whedon in the same way that any character any writer creates will be, in some sense, based on themselves and their own lived experience. The same way that (aspects of) Giles and Willow and Buffy are clearly based on parts of Whedon's own life as well. The same way many other characters, both major and minor -- characters like Jesse and Marcie Ross and Andrew Wells and ... - are based on Whedon or the other show's other writers and their own experiences.
Xander and Whedon are not very similar in a number of fundamental ways. They do not share similar class backgrounds or domestic circumstances or academic careers or personal lives. Even if they did, Xander is clearly not living out some kind of fantasy wish-fulfilment. He is very, very often the butt of the joke, or rejected romantically, or made to look weak or flawed or stupid or wrong. He makes jokes that his friends don't laugh at, and fails repeatedly at things they find easy. He has a terrible home life, with drunk and abusive parents, and this situation is never resolved in a way that might give him closure. While all his friends leave for college, he works a string of low-status, low-paid jobs that he does not enjoy doing and does not do well. He becomes increasingly less prominent on the show as time goes on, outshone by characters with super-strength and magical powers he never acquires.
And Whedon very pointedly did not have him end up with the hero of the show (who turns him down as soon as he asks her out, in an episode Whedon wrote and directed, and never changes her mind or shows the slightest regret about doing so) or with the girl who had a crush on him in high school (who dates a different, much cooler and more popular boy before coming out as a lesbian), or the first girl he dates (who he cheats on, and who -- unlike Willow's boyfriend, who quickly forgives her for cheating -- never forgives him for doing so), or the first girl to sleep with him (who holds him in nothing but open contempt, tries to kill him shortly afterwards and never speaks about him again). Can we not agree that these are all very strange writing choices for a character who is (allegedly) the creator's "self-insert"?
Obviously you don't have to (and, indeed, you should not) like or defend Joss Whedon. You don't have to like Xander Harris either (although Xander has the distinct advantage of being fictional). I don't hate Xander myself, but I will concede he is often a very frustrating character to watch and is the focus of some of my least favorite parts of the show (Teacher's Pet, Bewitched, Bothered & Bewildered, that awful speech he gives Buffy in Into The Woods). It is fair to say that many of Xander's flaws reflect the parts of the show that have aged particularly poorly (or that were just bad to begin with).
But the constant insistence that Xander Harris is bad because he is (somehow, uniquely) a "Joss Whedon self-insert" is just very, very tiresome to read. It is a transparently ridiculous take on the show, entirely without intellectual merit or textual foundation, and motivated only by a stubborn refusal to engage seriously with the fact that Whedon was the primary creative force on Buffy. Can we please stop parroting this nonsense?