Numerical ratings on games are worthless

psiufoxx2

psiufoxx2

CAGiversary!
Feedback
6 (100%)
I've come to a conclusion lately.. Rating games by numbers is a fruitless endeavor.

Okay.. they're not "worthless" as the title suggests, but flawed.

Now.. don't get me wrong. I understand the desire to form a "concrete" system of measurement, and as humans we perceive numbers as a very tangible unit of measure. But what exactly does an 8.0 out of 10.0 mean? What does it mean to you? to me? to him/her? It could be interpreted a number of ways by a number of different people.

In fact there ARE a number of different interpretations published every day, every week, and every month. Each online or print publication follows their own rating system and with it their own scale. Some may say 8.0 = A must buy. Others may use 8.0 = Above average. Unless you are informed as to the system you may incorrectly interpret the review.

Not to mention, if you look closer at where these numbers are derived from, another inherent flaw comes to light. We tend to break games down to their core categories such as sound, graphics, story, gameplay, control, etc. Now I'm not a pretentious prick, but would you break down the Mona Lisa as such and evaluate the individual brush strokes, colors, and choice of materials? Of course that comparison is not flawless as games are certainly not (at least not often) works of art, but the idea still remains: Can you break a game down to its base components and then average them to measure how "good" it is? I'm not sure.

We've also reached a point in gaming culture where a 10 point scale is flawed. In order to be fair to games and their designers, most reviewers award truly abysmal games with at least a 4. And subsequently the average and mediocre games are given at least a 7. We're then left with a 6 point scale, not a true 10 point scale. Why use 1-10 at all?

I guess I've only come to this conclusion because I'm jaded - and because of my own personal opinion. I certainly respect that we each have opinions to provide, but I just simply cannot stand when I love a game that gets 5.0s and 6.0s across the board.

And yeah, I guess gamerankings.com does a good job of giving an overall picture of the different rating systems and the general opinion of a game, but it's still includes that flawed assumption that a game is merely the sum of its parts.

Though I guess ratings are better today then they were in the days dominated by Nintendo Power. ie, "The Legend of Zelda, 5 out of 5 stars, BUY IT NOW! GIVE US ALL YOUR MONEY!" Not the most objective reviews, were they?

I dunno.

Reply if you hate me.
 
Thats why I dont take ratings seriously. If I really do want to look at scores I would go to gamerankings because it has a basic overview of many reviews and scores. Most of the time I just look at the reviews like my test scores, 9-10 = A, 8-8.9 = B, etc etc. I tend to ignore games below a B.... unless they have bewbies and is $9.99 ;)
 
[quote name='psiufoxx2']Reply if you hate me.[/QUOTE]
I am replying to your topic not because I disagree (which I do), but rather because I hate you.

Your welcome.
 
That's why I just read the reviews.

If you get into the habit of reading the review first, you'll often find that the numbers/stars assigned don't necessarily make sense. A good example was Mario Kart:DD, which had some of the weirdest disconnects between written reviews and at-a-glance ratings.
 
[quote name='dothog2']That's why I just read the reviews.

If you get into the habit of reading the review first, you'll often find that the numbers/stars assigned don't necessarily make sense. A good example was Mario Kart:DD, which had some of the weirdest disconnects between written reviews and at-a-glance ratings.[/QUOTE]


The infamous 7.9
 
[quote name='SOSTrooper']The infamous 7.9[/QUOTE]
More what I had in mind was how you'd read the occasional review that went on about how DD was just "more of the same," how disappointing it was in terms of tracks/gameplay, and how the additional karts/characters were kind of a cop-out, and then you'd see 5 out of 5 stars in every category rated or a 9.4/10 at the end of the review.

I dunno, I think ratings make idiots out of the authors as much as they do the reades.
 
[quote name='javeryh']That's why God invented gamerankings.com.[/QUOTE]
And it's the exact reason that GameRankings isn't as reliable as people make it out to be.

If a 9.4 out of 10 is attached to a mostly disappointed review, what's a 94% at GameRankings really tell you about the game? Pretty much nothing.

It's better to just find a few reviewers who share your tastes, hope they come across something you're interested in, and read their review/ignore the scores when it becomes available.
 
I agree, it's flawed because they it's based on our own method of assigning grades in school-- which is formed by averaging hard numbers and review scores are not. It's a valiant effort, but the numbers only serve to reinforce whatever experience I've had after I've played a game. they mean nothing before hand.

so if they did away with the grading part of reviewing altogether, I would not miss it a bit. It would help people to see the whole rather than little chunks of info, anchored by a number. and it would filter out the dumbasses who only look at the grade, taboot.

Nowadays I just read reviews to see what a professional thinks of a game but it doesn't serve any reak purpose, since I try stuff out before I buy it.
 
All reviews are just different people's opinions, and each reviewer brings with them their own unique preconceived notions, biases, etc. Reviews can be useful tools in making an informed purchasing decision, but they're just tools.

Use a review point score simply as a starting place, and recognize when a pattern starts to emerge (a number of reviews seem to agree, etc.). In the end, you the consumer are still responsible for your purchasing decisions, so you can't afford to put all of your faith into someone else's opinion.
 
I generally rely on about 3 or more reviews. If it has 8+ across the board and I enjoyed other games in its genre I will consider picking it up (once it reaches $14.99 after GGC). I don't have enough time to play everything so I go with the best rated games. I trust EGM's reviews usually. I know these guys (and others) review games for a living so I generally trust their opinion.
Thats what it comes down to. A lot of the time I will go to a movie based soley on what one of my friends says is a great movie if I know I share their tastes.
 
My favorite review pertaining to this subject is Meteos for the DS.

Gamespot calls it "the best DS game yet!!!" and praises it in the whole review, yet when it comes time to give a score they only give it an 8.5?

I'm not sure how "the best game on the DS" is only an 8.5, unless they really hate all the other available DS games.
 
[quote name='Purkeynator'] A lot of the time I will go to a movie based soley on what one of my friends says is a great movie if I know I share their tastes.[/QUOTE]
Yeah, this is exactly the philosophy I apply to games, as well.

I know he's not terribly popular for the scores he gives out, but I think Greg Kasavin's reviews (the text, not the number ratings at the end) are pretty critical of the things I look for in games. So if he's done a game I'm interested in, I'll typically read his review before I consider getting the game.

It's not that I agree with Kasavin on everything, but I like that he critiques the things I'm most interested in about a game, the things I look for when I decide, "Do I want to buy this or not?" To go back to the easy example that is Double Dash, one of the things that pissed me off in many reviews were people who were criticizing MKDD for not containing features of games that don't even belong to the karting genre. It was eye-crossingly retarded.
 
I see where the OP is coming from. I may use the numbers simply as a quick guide for something that I may have seen a preview on and was only mildy interested in. I have bought several games that were rated "lowly" in mags that I enjoy. Critical Depth being a good example.
 
[quote name='Roufuss']Yet when he goes to give it a score, he gives it a 70%?[/QUOTE]
This is again a shortcoming of GameRankings. The GameSpy dude gave it 3.5/5 stars. Now, yes, we can all do the math and see that this is 70% of the stars possible.

But this comes back to a point that was made above: some people are ranking these games like they're grading elementary school kids' essays, and in those cases 70% of stars is average or slightly bad.

In other cases, however, some critics rate things like they're movies and the old movie-star convention, wherein (for example) 5 is classic, 4 is outstanding, 3 is good, 2 is okay, and 1 is a waste of film.

So the reviewer in the link seems as though he's using the movie star system and GameRankings is representing his review/score as a grade, which the reviewer obviously didn't intend.

The mash up that GameRankings does only leads to more confusion. Your example is a really good reason why people should read the reviews first and either ignore the scores altogether or look at them only in passing.
 
[quote name='dothog2']This is again a shortcoming of GameRankings. The GameSpy dude gave it 3.5/5 stars. Now, yes, we can all do the math and see that this is 70% of the stars possible.

But this comes back to a point that was made above: some people are ranking these games like they're grading elementary school kids' essays, and in those cases 70% of stars is average or slightly bad.

In other cases, however, some critics rate things like they're movies and the old movie-star convention, wherein (for example) 5 is classic, 4 is outstanding, 3 is good, 2 is okay, and 1 is a waste of film.

So the reviewer in the link seems as though he's using the movie star system and GameRankings is representing his review/score as a grade, which the reviewer obviously didn't intend.

The mash up that GameRankings does only leads to more confusion. Your example is a really good reason why people should read the reviews first and either ignore the scores altogether or look at them only in passing.[/QUOTE]


However with the amount of praise he leveraged on the game, you'd expect it to at least be 4 stars. If someone just skipped the review and looked at the score, they saw he merely thought it was an average game and had some flaws in it.

When, in actuality, he loved the game, but for some reason scored it low for a game that seemingly dosen't have any glaring flaws.

I love Gamespy's reviews, but hate the way they do scores. It almost seems like he was "influenced" to score it below the average, because his review is most certainly not what you'd expect out of the score.

The whole time I read the review, I was thinking "So what's the flaw keeping it from receiving a higher score?" when there really wasn't one.
 
[quote name='Roufuss']However with the amount of praise he leveraged on the game, you'd expect it to at least be 4 stars. If someone just skipped the review and looked at the score, they saw he merely thought it was an average game and had some flaws in it.
[/QUOTE]
I should've qualified my other post. Your example is an excellent example in that it illustrates both the inadequacy of GameRankings and comparisons of stars/grades (as I noted) and it shows the disconnect between reviews and the scores accompanying those reviews (as you note).

As for the score, I guess he puts a high premium on originality, especially as he didn't fault the game for anything other than originality. I agree it's a low score given the nature of the review.
 
Personally, I won't touch a game that hasn't scored at least an eleventy-seven at IGN.

[quote name='dothog2']Yeah, this is exactly the philosophy I apply to games, as well.

I know he's not terribly popular for the scores he gives out, but I think Greg Kasavin's reviews (the text, not the number ratings at the end) are pretty critical of the things I look for in games. So if he's done a game I'm interested in, I'll typically read his review before I consider getting the game.

It's not that I agree with Kasavin on everything, but I like that he critiques the things I'm most interested in about a game, the things I look for when I decide, "Do I want to buy this or not?"[/QUOTE]

But do you have his rookie card?

20020121l.gif


Seriously though, I agree. Greg (Gamespot in general, but him more so) seems to be the reviewer that covers all the things I want to know in a review. Plus if he likes a game he'll talk some more about it in his journal on the site. For instance, Gungrave OD wasn't an exceptionally well rated game (69 at Gamerankings.com) but he gave it a thumbs up. I take numerical scores as more of a starting point for reviews. The content matters more.
 
Most review sites these days state that the overall score is NOT an average of the component scores.

I like the numeric score. If I only have a moment to check a game I can see what it scored at IGN and it's over all ranking at gamerankings.com If I have some more time I can read the text of a review or two and get a better picture. GR.com is a great snapshot of what the critics think. Anything over 90% is generally really good. Anything under 60% is generally pretty bad. In the 60s is iffy, 70s can go either way, and 80s usually means a good game. There are obviously exceptions and personal faves or hates based on taste, but I still say it's a very useful metric.

Your point that each site uses its own scale is a very good one. But you just have to get familiar or look at what they've scored past games you know of, and then you'll know the deal with their scoring.

"I should've qualified my other post. Your example is an excellent example in that it illustrates both the inadequacy of GameRankings and comparisons of stars/grades (as I noted) and it shows the disconnect between reviews and the scores accompanying those reviews (as you note)."

Yes but assuming the same sites review most all of the games on GR.com these same shortcomings will be applied to all games uniformly and therefore it will even out. In other words if a 3.5 star rating at a site indicates the same level of quality to every game it is applied (which it should, a site should be consistent with it's scoring across games or it isn't worth much.) I don't see the big problem.
 
[quote name='wubb']
Yes but assuming the same sites review most all of the games on GR.com these same shortcomings will be applied to all games uniformly and therefore it will even out. In other words if a 3.5 star rating at a site indicates the same level of quality to every game it is applied (which it should, a site should be consistent with it's scoring across games or it isn't worth much.) I don't see the big problem.[/QUOTE]
Well, kinda. This will "even out" for the games on either end of the spectrum, the ones getting either universally praised or panned. But for the in-between games, like stuff between 60-85%, the disparity between the two systems (stars v. grades) makes the "average" GameRankings score for the game even more nebulous.

However, I agree with you that GameRankings is definitely NOT a big problem. It's great to have a site that pulls all the reviews together and gives you a general feel for the climate around the game. Though I don't put much credence in their avg. scores, I use GR as my starting point when I want to read about a new game. It's a good site to have.

It isn't that out-of-ten scores or so-many-star ratings are bad things in general, they're just abused too often by careless reviewers. Instead, it pays to read the review surrounding the score and take the numbers/stars with a grain of salt, and the same is true for using the avg. scores at GR as an overall guide of gaming goodness.
 
Different sites rate things differently, so you should just compare the scores on one site and get to know what they mean by them. Instead of comparing the scores of one game across many sites or magazines as they all mean different things. But the scores of one site or magazine allow you to quickly compare how they like different things.

Though I agree with not liking how games are broken down into categories, Gamespot rates by an average of those categories and IGN is even worse by weighting by the graphics category. Games' ratings should be based mostly on how much fun it is not sound or graphics.

EDIT: My above reasons are why I never use Game Rankings, I find a few sites that I generally agree with and only look at those reviews. But I don't base everything on the reviews anyway.
 
A numerical score is nothing more than the reviewers overall opinion of the game. Some sites and mags try to break down their scores or put a lot of technical jargon behind them, but at the end of the day it's still just the reviewers overall opinion of the game. So a score can't be right or wrong. It's the reviewers opinion and should be taken as such.

The only facts you will (hopefully), find in a review are in the actual review where you should come across game details like length, bugs and other gameplay elements.

Word of advice. Don't waste your time trying to figure out why site X rates a certain way or why mag Y didn't like a game. It's all just opinion and once you realize that, the world of gaming media becomes a lot easier to manage. Especially when you realize that "pro" reviewers can be fanboys and haters just like everyone else, but that is another lesson.
 
What I think is dumb is how people have become such sheep to reviews...like at EB or gamestop I will often hear the clerk telling people, this game has a 90% blah blah on gamerankings, it's awesome. Or, some of my friends won't touch a game unless it's 85% or better no matter what. How many people thought FFX-2 sucked, and what score does it have? Certain games will never get bad scores either due to hype, or their background, like Halo, FF, etc.
 
I never have or will use rating to keep me from buying a game. If they are low rating I go and rent the game If I like then I go buy. If it sucks like the ratings says then I stay away. I have a lot of games in my collection that were never rated higher then a 5. However there is some games that no matter the ratings I am buying. Shenmue, Way of the samurai, Tenchu.
 
Just play the game yourself and see if you like it. Read a few reviews first, I mean some games are like movies. Some will love others that most hate and vice versa.
 
You must log in or register to reply here.
bread's done
Back
Top