Let’s Talk Review Scores
Metacritic. Rotten Tomatoes. IGN. As I cruise the regularly visited sites, there is, almost inevitably at the bottom of any review, a number. What’s this number? How’s it helpful? Let’s talk about this.
Let’s start with the basics - I get why we use numbers in reviews. It’s an easy system that lets me quickly see, on a pre-specified scale, how “good” something is. I can get behind the idea of a numerical score; it’s fast, I don’t have to read all those words and letters (because my brain), and (in theory) it’s objective. 7 is better than 5, right? If I’ve gotta choose between a 7 and a 5, it’s 7 every time. Makes sense.
Here’s where things go batshit. Let’s start with media powerhouse IGN. In a recent review for an Arkham Knight comic (oh, don’t worry Arkham Knight video game - your day is coming), the reviewer noted the art is “cartoonish” but the writer generally gets by and sometimes tells a good story. He gave the comic a 7.2 out of 10. For an X-Men comic, the same author writes “Fortunately, there's plenty of to like with the new series and creative team, even if this issue isn't quite the rousing start the book needed.“ He gives this a 7.1 out of 10. Of all the comics reviewed this week, only 4 dip below 6/10 and of those, only 2 dip below 5.
This might be a specific-to-IGN issues, but why have a 0 to 10 scale if you’re just going to hang out around 7 or 8. It’s not uncommon to read a review that basically says “This sucked real bad. The writing was terrible, the production was awful, etc - 7.3 out of 10.” A guideline for how these numbers are generated would be helpful. The qualitative review doesn’t always match the quantitative part; if I ignore the number, it sounds a lot worse than what amounts to a 75%.
Another point, which again might be specific to IGN but I’m sure other folks do this too - they use decimals, meaning they’re basically reviewing shit on a 100-point scale. Is this necessary? Do the nuances of the written review necessitate a spectrum of 100 points to choose from? More than that, do these numbers add anything meaningful? How is a 7.5 significantly different in terms of quality from a 7.6? Again, my brain says “7.6 is better than 7.5,” but is it really? In the above example, the X-Men review basically said it was ho-hum, but it was only 1/10th of a point less than the other comic. I’d argue this scale is way too big to be useful in any real way.
Does anybody remember X-Play? It was a show on G4 that reviewed video games in brief segments and discussed gaming stuff. Their scale was PERFECT. 5 points. 1 meant garbage, 2 meant pretty bad. 3 was a big meh (playable, but not great), 4 was good with some minor problems, 5 was great (maybe some problems, but overall a good experience). When they said a game was a 4 or a 5, you knew it was worth checking out. If it was a 3, you could sense it was iffy. Below that, you knew you were in for a rough time. It was intuitive and simple. I’d also like to point out Kotaku’s Yes/No system for “Should you play this?” as a good example of how to do this. Even Angry Joe, who uses a 10-point scale, generally agrees that 5 means “it’s ok” - it’s in the freaking middle of the scale; could be better, could be worse.
Obviously, the elephant in the room is aggregate review sites, like Metacritic and Rotten Tomatoes. I get it - people just want to be able to look at a quick thing instead of sifting through review after review, and get a sense of whether something’s good or not. But what the hell do these numbers represent? I’m going to show my hand here and say that, in my for real life, I conduct some research, which may have instigated some of these arguments. As part of that background knowledge, I know that it often causes problems when you compare and combine things that are on different scales. They need to be converted so that the numbers are on an even playing field. A 5/5 is not the same as 5/100, even though they’re both 5s. So when RT says that a movie is 78% fresh, I’m not entirely sure what that means. Does that mean 78% of reviewers like it? Is the movie 78% good? How did you get that from a “7.4″, “Go see it”, “3 stars”, and a thumbs up? It’s needlessly complicated and (allow me to be a cynic here), I’m going to wager that the reason text and number don’t always match is because there’s some kind of kickback for boosting the RT score. Why else would a “this is garbage” review still get somewhere between 7.3 and 8.6? The only thing that number seems to be influencing is the aggregate score.
Consumers aren’t helping. You don’t need to look any further than the comments posted at the bottom of a review to see the savage hounds drooling over the numbers. “YOU GAVE THIS A 7.8 AND COD A 7.6?!?!?! COD IS THE BEST!! KILL YOURSELF!!!ONEONEONE!!!” or “The reviewer obviously sucks. They didn’t like this and gave it a 6.8. This deserves at least a 7.3″ Here’s one actually from the X-Men review: “Disagree with the review of extraordinary X-men issue 3, I think the series is moving along nicely, lots of action so far, full of a nice colourful art style and its building to the return of an old X-men classic villain. Really do wonder if some of these reviewers even read these comics they review.” First, it’s an opinion - just because the reviewer didn’t agree with you doesn’t mean they’re wrong. Second, it still got a 7.1, which, out of 10, is still pretty good. Obviously, every fanbase has its vocal minority, but is the number that important to the typical consumer? Probably not. People just like seeing the things they love get high numbers because it validates their enjoyment of that thing. It didn’t influence the sale, since those rabid folks were going to consume that product anyway, but wars are started over this kind of thing. Digital wars.
(I also want to take a quick second to say that people online often confuse the number for some kind of scientific measure of how good something is, like it’s a pure science, so when someone doesn’t like something and give it a bad review, they complain that they should’ve gotten someone who likes it to review it. Newsflash, all reviews are subjective. The number isn’t changing the product.)
I’m trying this thing where I don’t criticize something unless I have a suggestion for how to make it better, so here goes. Reviewers: keep it simple. Either have a rubric for how your number was generated so that I see there’s a consistent scale for everything you review (even if it’s just yes/no) or simplify your scoring system; otherwise, it just looks like a random, inflated number thrown in there for the sake of padding. Readers: find a person you like. Find someone who generally likes the games you like and someone you trust to tell you whether something’s worth your time. And actually read what they have to say; don’t just skim to the bottom and see how many points it earned.
Fucking numbers, amirite?