Dragon Age II Metacritic Scandal

R

randall82

CAG Veteran
Here's the article, in case you haven't read it, and a link to the thread on reddit.

I've seen people try to argue both sides of this. Some say it's shady, and some sort of conspiracy to manipulate scores. Honestly, I think that's bullshit. But on the other hand, people are saying that it was just a guy who was proud of the work he did, so he went and wrote a review. In fact, that's exactly how Kotaku explains it away:

"Is it unethical? Is it dirty? Personally I feel that it's one thing if a marketing team spends time and resources seeding the usual places with shining reviews. It's another thing if someone that worked hard on a product for several years goes onto a public forum and lets his or her pride over what they've accomplished shine through." -Mike Fahey, Kotaku

I'm not a fan of Kotaku, it's just not my vibe, and I'm not the type to say someone's opinion is wrong, but it seems like they're grasping at straws to try and justify something that is clearly unethical. I can understand that someone is proud of their work, and wants to go into a public forum and gush a little. I don't have a problem with that. But it crosses ethical guidelines when said person doesn't clearly identify themselves as affiliated with the product they're promoting.

What bothers me more than the review though, which can be easily explained as enthusiasm from a team member, is the response from EA/Bioware. You can read the quote from their Public Relations Manager in the Kotaku article linked above, but they basically say that this is how it works, and it's equivalent to an actor voting for themselves at the Oscars. This is completely disingenuous, because it's not simply a vote. It's someone misrepresenting themselves and an unbiased 3rd party, which they clearly aren't.

How easy would it be for EA to just say, "We've talked to our employee, and while we appreciate his enthusiasm for our product, we've made it policy for our employees to disclose their affiliation when making statements in public forums". The fact that they didn't say this, seems to be and indication that they plan on doing this more in the future, and will continue trying to pass off PR as genuine consumer opinion.

The only upside I can see is that this is another crack in the crumbling foundation that is Metacrtic and the review "score". Over that last few years, with the rise of Giant Bomb, Joystiq, and our good friend Shipwreck, I've begun paying more and more attention the the small, italicized addendum at the bottom of a review that tells you who the author is. Many of them don't try and pretend to be unbiased, and when you hear their opinion you know exactly where it's coming from. So to EA, I say keep it up. The quicker we can trivialize Metacritic the better.

tl;dr Shipwreck > Metacritic
 
Kotaku has never had a good opinion on anything. It's shallow to rate your own work, let alone pretend like you created a masterpiece.

The guy should be ashamed, and I hope the other people at Bioware aren't as arrogant.
 
Totally agree with your point about small individual reviewers being > Metacritic. I think Metacritic is only doing harm to the industry.

And yeah, it is unethical for a guy to review his own game without disclosing his identity, but pragmatically it isn't going to have an effect on the overall score. So I don't see this kind of thing leading to the downfall of review aggregate sites. However, all the dishonest reviews and platform wars that go on within Metacritic do trivialize the user reviews averages. (which is noticeable once again with DA2)
 
This ain't the first time someone from a company left a glowing review for a product they worked on on a website...and it sure won't be the last. The real news here is the guy doing it was sloppy enough to get caught. Doesn't mean I think it's right...you're trying to manipulate systems designed to give consumers a voice...I just don't see how it can be stopped.

As for reviews...they could use some reform on an industry-wide scale...the entire "7-10" scale thing for starters.
 
I don't really think this is a big deal at all. Some dude loves a game he worked on. Maybe he should have said he worked for bioware, but seriously people are turning something small into something huge.
 
[quote name='whoknows']I don't really think this is a big deal at all. Some dude loves a game he worked on. Maybe he should have said he worked for bioware, but seriously people are turning something small into something huge.[/QUOTE]

This. There have been far bigger stories that have gone unnoticed, so I don't understand why this one is getting so much attention.
 
This is the first, and probably final, time that people discuss the "ethics" of user reviews. You're kidding yourself if you think there ever was integrity in user reviews since the majority either give a game the highest score or the lowest score possible because that's their best chance of affecting the average user score.

This being a scandal depends on whether the idea was initiated by Bioware or their PR team, which seems unlikely due to only one guy doing this. PR wouldn't bother having one guy do it if they were behind it and it doesn't affect the score that would affect their pay anyway. He's dumb for using the same username he uses everywhere else like nobody could google search it to see who he is, so disclosure would've been the best idea. Since his review wouldn't affect the Metacritic score, it only affects something that's already meaningless in the first place.

Metacritic is partly to blame for this since they don't offer any simple means of discussion for each game like they do on their sister sites: GameSpot and GameFAQs. 1up had the same issue where people that wanted to comment on reviews wrote user reviews to get their say in and gave it a random score since they had to score it.

If the people that gave it zeroes to make a point wanted to a better chance for it to be heard, they should've just organized a letter-writing campaign to make more of an impact than random zero reviews on a website.
 
I don't think anyone takes meta scores or reviews in general very seriously. I just don't like what the guy is doing.

It's not a big deal given the numbers, but you don't want your employees representing your game like this. It's bad.
 
[quote name='panzerfaust']I don't think anyone takes meta scores or reviews in general very seriously.
[/QUOTE]

I soooooo wish this were the case.
 
[quote name='ihadFG']This. There have been far bigger stories that have gone unnoticed, so I don't understand why this one is getting so much attention.[/QUOTE]

It really isn't that he wrote the review that bothers me. While unethical, that can be explained away. It's the fact that this is viewed as "business as usual", and this one just happened to get caught. We might not look at reviews as something to take seriously, but sadly, the industry does. People's livelihoods are tied to a number. It effects whether studios get work or not, what studios get shutdown, and more importantly, it effects the bonuses and job security of the PR department of the publisher.

When score is king, nothing else matters. That's why embargoes are allowed to be broken early if a certain score is met. It's why certain publications get review copies while others don't. Objectivity is lost when PR Reps are under so much pressure to control the discussion that they resort to unethical behaviors to change public perception. If we want to hold up game journalism as something to be taken seriously, we have to realize that such unethical behavior is an attack to its cornerstone, objectivity.
 
@Frisky - That's why I think the biggest take away from the story isn't that one dude wrote a user review in an unethical manner, it's that the studios are completely okay with this lack of ethics. I completely understand your point and agree when it comes to the user review that started this discussion. I wouldn't even care, had it not been for EA's response.
 
[quote name='randall82']It really isn't that he wrote the review that bothers me. While unethical, that can be explained away. It's the fact that this is viewed as "business as usual", and this one just happened to get caught. We might not look at reviews as something to take seriously, but sadly, the industry does. People's livelihoods are tied to a number. It effects whether studios get work or not, what studios get shutdown, and more importantly, it effects the bonuses and job security of the PR department of the publisher.

When score is king, nothing else matters. That's why embargoes are allowed to be broken early if a certain score is met. It's why certain publications get review copies while others don't. Objectivity is lost when PR Reps are under so much pressure to control the discussion that they resort to unethical behaviors to change public perception. If we want to hold up game journalism as something to be taken seriously, we have to realize that such unethical behavior is an attack to its cornerstone, objectivity.[/QUOTE]

Agreed. I just don't get why this story is getting attention when stories about actual major review shenanigans don't (such as a lot of the policies you mentioned). Additionally, user review averages are hardly as important as critic reviews to most people.
 
I don't know why anyone would care about this. It was a user review, right? Does anyone honestly care about those? Really? If this review was published in a magazine or on a website, I would see the problem. Though I really don't think we need to be crying about "ethics" when it comes to a video game magazine, I at least agree that the reviews should be somewhat "unbiased," especially if that's the product you're peddling. But user reviews are an open forum. Should the guy have done this? No, but that's like saying you shouldn't get into a flame war on a message board. Who really cares?

What I find really odd about the Kotaku article is that the guy has less of a problem with marketing departments doing this sort of thing. I mean, if we are going to talk about the ethics of this stuff, that's a lot more unethical than a proud developer wanting to hype his own product. Again, was it unprofessional for him not to identify himself? Sure, but I don't see the harm in it (besides making himself look silly), and it certainly wouldn't cause as much harm as a marketing department planting 50 or 100 fake reviews to pump up a score.
 
[quote name='randall82']How easy would it be for EA to just say, "We've talked to our employee, and while we appreciate his enthusiasm for our product, we've made it policy for our employees to disclose their affiliation when making statements in public forums". [/QUOTE]
This. I'm fully prepared to believe that this was just a single BioWare employee who is proud of his company's work and made an error in judgment, nothing more. If they would just acknowledge this, the "scandal" would go away. But between this and the shenanigans on their forums with banning and segregating users, they make themselves look guilty-- the coverup is worse than the crime.

I am a huge fan of BioWare's games (including Dragon Age II, which is fantastic) but they are the kings of shooting themselves in the foot with boneheaded PR moves.
 
Publishers offer their Developers cash bonuses based on aggregate Metacritic scores. Meaning if the average Metacritic score is over X, then the developer receives more cash.

Is it any surprise then that this happens?

I bet this practice is far more widespread than anyone imagined.

If I could make more money at my job by boosting the rating on a website, I would have a hundred aliases posting a hundred glowing reviews of whatever I worked on.

The bigger issue here is the Industries blind reliance on Metacritic reviews.
 
[quote name='Puffa469']Publishers offer their Developers cash bonuses based on aggregate Metacritic scores. Meaning if the average Metacritic score is over X, then the developer receives more cash.

Is it any surprise then that this happens?

I bet this practice is far more widespread than anyone imagined.

If I could make more money at my job by boosting the rating on a website, I would have a hundred aliases posting a hundred glowing reviews of whatever I worked on.

The bigger issue here is the Industries blind reliance on Metacritic reviews.[/QUOTE]

User Review scores are not taken into account for these kinds of bonuses, so your point holds no water. On another note does that mean if I review the CAGcast in iTunes and give it a 5 out of 5 because I am proud of my show, that means I have no ethics?
 
@Wombat - If you identified yourself, not at all. I don't believe all reviews should be unbiased, but any bias should be known so the review can be weighted appropriately. If you posted a review of your own product and intentionally hid your identity in an attempt to seem impartial, to me that seems unethical. I wouldn't think you were completely unethical, but that would be a little shady.

Once again though, I have to stress, I have much less of a problem with the review than I do with EA's reaction to it.
 
This guy probably only got caught because his name was first on the list. How he got first on the list is another matter altogether.

Also, is Dragon Age II really that bad? I've heard it's getting A TON of hate.
 
[quote name='FriskyTanuki']This is the first, and probably final, time that people discuss the "ethics" of user reviews. You're kidding yourself if you think there ever was integrity in user reviews since the majority either give a game the highest score or the lowest score possible because that's their best chance of affecting the average user score. [/QUOTE]
It's hilarious because the same is true of Yelp, which is increasingly being used as the standard to select restaurants by the public. I doubt that the Churro cart at Disneyland is equivalent to the Michelin 2-star restaurant in Beverly Hills.

There ARE actually standards for reporting. Bloggers are required by the FTC to disclose when the product they are reviewing is free. (http://www.msnbc.msn.com/id/33177160/ns/technology_and_science-tech_and_gadgets/)

Even on sites like Amazon where all user reviews are stored, even one that says "Amazon didn't deliver my book so this book sucks," it still does have an impact, at least an initial one, to those who don't read through completely.

I guess we're lucky right now that we don't have a central videogame website that deals with reviews that everybody goes to, like Yelp. Even though Metacritic user reviews are meaningless now, I'm sure this will become an issue down the line.
 
[quote name='KingBroly']Also, is Dragon Age II really that bad? I've heard it's getting A TON of hate.[/QUOTE]
Yes and no. The story and setting are incredibly weak this time around, but the combat is smoother and class trees refined a bit. The lack of story-driven goals and each act seemingly tacked on to each other is what people are hating on most. That and you're restricted to Kirkwall and the city outskirts the entire game. There's no epic tale of becoming a Grey Warden and fighting the evil darkspawn invasion RPLoL! The characters are also pretty forgettable or annoying other than Varric. That said, if you were drawn into the combat system of the first game you will like this one just as much.

Just don't go in with expectations of the first and you'll have fun.
 
[quote name='kainzero']It's hilarious because the same is true of Yelp, which is increasingly being used as the standard to select restaurants by the public. I doubt that the Churro cart at Disneyland is equivalent to the Michelin 2-star restaurant in Beverly Hills.

There ARE actually standards for reporting. Bloggers are required by the FTC to disclose when the product they are reviewing is free. (http://www.msnbc.msn.com/id/33177160/ns/technology_and_science-tech_and_gadgets/)

Even on sites like Amazon where all user reviews are stored, even one that says "Amazon didn't deliver my book so this book sucks," it still does have an impact, at least an initial one, to those who don't read through completely.

I guess we're lucky right now that we don't have a central videogame website that deals with reviews that everybody goes to, like Yelp. Even though Metacritic user reviews are meaningless now, I'm sure this will become an issue down the line.[/QUOTE]
I'm not saying that people aren't blind to how most user reviews work and base their purchasing decisions on them anyway, but that's their fault if they take them at their score.

I don't think the FTC regulates user reviews nor do they seem to have much presence in video game blogs since that's the only thing I've seen about the ruling.

[quote name='Wombat']User Review scores are not taken into account for these kinds of bonuses, so your point holds no water. On another note does that mean if I review the CAGcast in iTunes and give it a 5 out of 5 because I am proud of my show, that means I have no ethics?[/QUOTE]
Sure, it's like Cheap Ass Gamer being up for a Cheapy award on CheapAssGamer.com for Best Trading Website isn't a huge conflict of interest.
 
[quote name='Wombat']User Review scores are not taken into account for these kinds of bonuses, so your point holds no water. On another note does that mean if I review the CAGcast in iTunes and give it a 5 out of 5 because I am proud of my show, that means I have no ethics?[/QUOTE]
Sure, it's like Cheap Ass Gamer being up for a Cheapy award on CheapAssGamer.com for Best Trading Website isn't a huge conflict of interest.
 
[quote name='Jodou']Just don't go in with expectations of the first and you'll have fun.[/QUOTE]

Considering I've played about 15 minutes of the original (10 of which was the intro) I think it'll be a long time before I play it.
 
[quote name='Jodou']At first I thought the title read Dragon Age II Metacritic Sandal and all I could think was: ENCHANTMENT?![/QUOTE]


ENCHANTMENT!!!!!:applause:
 
If you get mad at this, you should get mad at all those random reviews for games where it's generally considered a good game my the masses, and you get the one angry guy that gives it a 1/10 because of some random reason that doesn't have anything to do with the game.

It's not like his one review made the entire score invalid. It's not meta critic because it only pulls from 1-2 sources. It's the average of just about everything it can find. =P
 
[quote name='ExplodingRat']It's not like his one review made the entire score invalid.[/QUOTE]
That's kind of the whole reason why it wasn't cool that the person didn't disclose their affiliation. Because if BioWare or EA had deliberately planted this review as part of their marketing strategy, how many others could they have also planted? 10? 100? Not that a user review score is the end-all be-all, but I for one would like to believe that it's not completely invalid and does represent the views of actual consumers at least to some extent.
 
[quote name='Ryuukishi']That's kind of the whole reason why it wasn't cool that the person didn't disclose their affiliation. Because if BioWare or EA had deliberately planted this review as part of their marketing strategy, how many others could they have also planted? 10? 100? Not that a user review score is the end-all be-all, but I for one would like to believe that it's not completely invalid and does represent the views of actual consumers at least to some extent.[/QUOTE]

But we are talking about user reviews, the Bioware employee is still a user and has just as much right to give a review as you do. He does not have to disclose anything.
 
[quote name='Wombat']But we are talking about user reviews, the Bioware employee is still a user and has just as much right to give a review as you do. He does not have to disclose anything.[/QUOTE]
It's not about what someone has the right to do. It's about the usefulness of the user review average. Can I be somewhat confident that it represents the honest views of gamers like me? Not if I have no idea how many of those scores may have been planted by the company that wants me to buy the game.
 
[quote name='Ryuukishi']It's not about what someone has the right to do. It's about the usefulness of the user review average. Can I be somewhat confident that it represents the honest views of gamers like me? Not if I have no idea how many of those scores may have been planted by the company that wants me to buy the game.[/QUOTE]
What about all of the reviews by people that gave it a 0 to protest the changes they made to the way the original played?
 
It's now very clear to me that many people have no idea what ethics are. This is a non-story.

Dude posts a user review wherein he hypes his own product. And? Did he swindle anyone out of money? Did he somehow singlehandedly blow the curve and sway the overall review score? Did he force anyone to buy the product who would not have purchased it without his influence?

So all other review scores by random assholes who you know absolutely nothing about and who may have ties to companies or personal axes grind are okay, but his review is an unholy abomination because you now know that he has ties to the product? Anyone see the problem here?

ALL review scores are suspect. They are people's opinions and should hold absolutely no value above that. That dude's involvement in the project may have swayed his opinion but he's still entitled to express his opinion and is not obligated at all to tell you how he came to that opinion. Metacritic isn't some legal document. It's a fucking internet forum. Anyone spends any legitamate energy on this "scandal" truly has far too much time on their hands.
 
[quote name='Ryuukishi']It's not about what someone has the right to do. It's about the usefulness of the user review average. Can I be somewhat confident that it represents the honest views of gamers like me? Not if I have no idea how many of those scores may have been planted by the company that wants me to buy the game.[/QUOTE]
Reviews are only useful in a sense that they set expectations. The idea of a finite number defining a game is pure fantasy.
 
[quote name='FriskyTanuki']What about all of the reviews by people that gave it a 0 to protest the changes they made to the way the original played?[/QUOTE]
Their opinions may be idiotic, but at least they are who they're purporting to be. :lol:

I mean really, this is like a faith healer who calls up a planted audience member. The plant may genuinely think that the guy is all that, but does that make the whole thing any less suspect?
 
It's lame to shill your own work, but it happens all the time on review sites, so this is nothing new.

You have to take reviews on sites like those with a grain of salt. I just go there to look at the individual scores from major review sources.

User reviews I couldn't give a crap less about as there's too much bias there with fanboys trashing everything from rival companies etc.
 
As others have already said, user reviews are extremely biased and useless to begin with. DA2 has a bunch of 0's already. 0, folks. How does a game even get a 0? I would have to think the only way to deserve a 0 is if I put the game into the console, and it explodes. User reviews and scores are a joke, so I really don't see the problem with this guy posting a review. It certainly isn't unethical, and it's definitely not a scandal.
 
[quote name='Ryuukishi']Their opinions may be idiotic, but at least they are who they're purporting to be. :lol:[/QUOTE]
Except that it's messing with the usefulness of the average review score since that you were talking about. They're just hijacking the system to make a point in a way that hurts the average reader.
 
[quote name='FriskyTanuki']Except that it's messing with the usefulness of the average review score since that you were talking about. They're just hijacking the system to make a point in a way that hurts the average reader.[/QUOTE]
I don't disagree with that. Just different ways of looking at it I guess-- maybe different "levels" of being invalid? :) I would rather see an exaggerated negative review that is at least based on something that an actual consumer genuinely hated about the game, than a fawning review that may or may not be genuine at all.
 
[quote name='Wombat']But we are talking about user reviews, the Bioware employee is still a user and has just as much right to give a review as you do. He does not have to disclose anything.[/QUOTE]

He might not have to, but I think it's unethical for him not to disclose that he's obviously biased. No one is saying you can't comment on your own work or be proud of your accomplishments, but not disclosing that you had a part in making it, to me, is shady.
 
[quote name='bigdaddybruce44'] How does a game even get a 0? I would have to think the only way to deserve a 0 is if I put the game into the console, and it explodes. [/QUOTE]

:lol: I love that.


[quote name='Ryuukishi']I don't disagree with that. Just different ways of looking at it I guess-- maybe different "levels" of being invalid? :) I would rather see an exaggerated negative review that is at least based on something that an actual consumer genuinely hated about the game, than a fawning review that may or may not be genuine at all.[/QUOTE]

I actually don't doubt that the Bioware employee was genuine in thinking the game is great. The review was based on his opinion of the game. He is also biased though, so that is where the issue comes in. But this is also a user review section, which is full of trolls and fanboys and just general idiots in the first place. I think a slightly biased review from a dev is still more honest and useful than a review from a random dude on the internet who didn't even play through the game and gave it a 0 because everyone is saying it is different from DA1.
 
[quote name='ihadFG'] I think a slightly biased review from a dev is still more honest and useful than a review from a random dude on the internet who didn't even play through the game and gave it a 0 because everyone is saying it is different from DA1.[/QUOTE]

Exactly. Opinions are fairly useless in general, but there's no more useless opinion than those posted by users on the internet.

Fanboy bias, people being more prone to bitch when they dislike something or have problems than to take the time to praise something they enjoy etc.

So I just don't see the big deal. At the end of the day the only opinion that matters is your own, and if you want to use reviews it's best to look at the average among professional reviewers to get a sense of whether a game/movie/etc. is worth checking out on your own than the stupid user ratings that are full of bias. Professional reviews can be as well, but they're (when there are several of the anyway) at least a bit more believable to be at least somewhat objective than the user reviews that have no accountable mechanisms at all. At least professional reviewers can be fired if they're found being too biased.
 
[quote name='ihadFG']I actually don't doubt that the Bioware employee was genuine in thinking the game is great. The review was based on his opinion of the game. He is also biased though, so that is where the issue comes in. But this is also a user review section, which is full of trolls and fanboys and just general idiots in the first place. I think a slightly biased review from a dev is still more honest and useful than a review from a random dude on the internet who didn't even play through the game and gave it a 0 because everyone is saying it is different from DA1.[/QUOTE]
This is still a step up from the user review system that they had before where you didn't even need an account to write up a review, which lead to every system exclusive game getting really low scores by trolling fanboys.
 
[quote name='FriskyTanuki']This is still a step up from the user review system that they had before where you didn't even need an account to write up a review, which lead to every system exclusive game getting really low scores by trolling fanboys.[/QUOTE]

A step up sure. But it's still not anywhere near relevant. Dragon Age 2 has a user average of 3-4 across all three platforms. Apparently people are still willing to make an account so they can give the game a 0.
 
[quote name='FriskyTanuki']I don't think the FTC regulates user reviews nor do they seem to have much presence in video game blogs since that's the only thing I've seen about the ruling.[/QUOTE]
It's probably because a company or consumer won't get much if it goes after a video game blogger on a review. =)

Restaurants and other businesses that rely on repeat business, getting a bad review on a blog could be killer. And plus you have Yelp acting Mafia-like spotlighting negative reviews if that certain business doesn't want to pay ad money...

And anyway, professional video game reviews are a joke anyway, haha.
 
[quote name='randall82']It really isn't that he wrote the review that bothers me. While unethical, that can be explained away. It's the fact that this is viewed as "business as usual", and this one just happened to get caught. We might not look at reviews as something to take seriously, but sadly, the industry does. People's livelihoods are tied to a number. It effects whether studios get work or not, what studios get shutdown, and more importantly, it effects the bonuses and job security of the PR department of the publisher.

When score is king, nothing else matters. That's why embargoes are allowed to be broken early if a certain score is met. It's why certain publications get review copies while others don't. Objectivity is lost when PR Reps are under so much pressure to control the discussion that they resort to unethical behaviors to change public perception. If we want to hold up game journalism as something to be taken seriously, we have to realize that such unethical behavior is an attack to its cornerstone, objectivity.[/QUOTE]

Except game reviews are inherently subjective.

[quote name='randall82']He might not have to, but I think it's unethical for him not to disclose that he's obviously biased. No one is saying you can't comment on your own work or be proud of your accomplishments, but not disclosing that you had a part in making it, to me, is shady.[/QUOTE]

Every reviewer comes in with a certain set of biases coming into a review. We are just talking about different levels of bias.
 
[quote name='msdmoney']Except game reviews are inherently subjective.[/QUOTE]

I think he just meant lack of bias and favoritism though (in regards to websites and companies). Or at least that's how I took it. Either way, you could replace "objectivity" with professionalism, and his points are still valid.
 
[quote name='ihadFG']A step up sure. But it's still not anywhere near relevant. Dragon Age 2 has a user average of 3-4 across all three platforms. Apparently people are still willing to make an account so they can give the game a 0.[/QUOTE]
I never said it was relevant, just that they acknowledged the issues and have moved to something that at least makes it harder to just submit reviews anonymously to mess with the scores.

I don't know what they could do to legitimize the user reviews, as you'd probably have to require all user reviews to be submitted in writing to cut out the random 10/0 scores and require some sort of approval process where the community would have to approve the reviews to a certain degree that would allow it to show up to the public and count in the average user score.
 
You must log in or register to reply here.
bread's done
Back
Top