"A game like Eschalon: Book 1 should be reviewed based upon how well it appeals to "old school RPG fans.""

Seems to me a game should be reviewed according to the audience that the site/paper as a whole targets, not the audience that each specific game caters to.
 

Ultimately, yes. And that's the approach I try to take. However, what defines the audience of the site? Now that you've got things like GameRankings providing aggregate scores, that's a pretty broad audience. And as DanC points out --- the representation on the aggregate sites is self-selectively biased in one direction.

A case in point - as I understand it, Strategy Informer used to be focused on strategy games. But it's broadened its audience quite a bit since then. So --- does that invalidate its earlier reviews? (Not that anybody cares about reviews of older games anyway...)

So yeah, your answer is obviously the "correct" one, but it's annoyingly complicated in implementation.
 

Eschalon: Book 1 may appeal to RPG fans, but definitely not to "old school" RPG fans. It's incredibly easy, and absolutely stuffed with magic items. While I had it (briefly) I called it the "twinkie game" or "twink adventure". Not disparagingly - it's just designed for 8-year-olds.
 

Having read Danc's screed about Super Mario Galaxy... I'm not impressed. He grabbed a game of the kind he doesn't like, and didn't like it. Whatever. I could maybe write an article about all the ways I don't like knitting and needlepoint, but why?

His main difficulty seemed to be that he simply didn't know what to expect from a Mario game. Maybe Gamerankings.com could make things easier by categorizing it, maybe not. Who knows if Dan would even have recognized what "platformer" meant? Being unfamiliar with the territory is a difficulty that everyone faces, and there's only so much hand-holding you can do.

Dan doesn't seem to accept that the game, as published, appeals to a large audience even though he doesn't like it. He should stop describing all the ways Mario could please him and find games that do.
 

A game in any genre should only be reviewed by people who like that genre.

A game in any genre should be reviewed by people who like most all genres because most gamers aren't niche gamers.

GameTunnel has dealt with balancing the two statements above over the years as well, and I don't think there is an easy answer, and aggregate review websites make things more difficult.

My old baseball/football example is that I love baseball games to the point that I ONLY want to read a review from someone who has played DOZENS of them. However, for a football game I want someone who isn't a football nut, b/c I can't relate or understand why one little aspect is better than another over the years and likely won't notice the difference. I just want a game that is good.

So a solution would be to have reviews that have a section for fans of the game type and another section for general game fans. However, that would absolutely require 2 reviewers.

In the end what most of us do (well the ones who read lots of reviews anyway) is find someone whose views are similar to our own. I read IGN over Gamespot b/c Matt Casamassina is a Nintendo Fanboy and increasingly so am I.

I use Metacritic to read review snipits and determine which reviews to read, and after having read so many over the years I can quickly come to grips on whether or not I look at games the same way the writer does. Reviews are just a personal bias, so determining the writer's bias is essentially scoring the game for yourself.

Jay I think your blog is great for your bias, however, there are some games I probably wouldn't listen to your bias on as I look at the world differently.

The sad part is that for the unthinking, who just look at an aggregate score, many games are likely to be missed.

oh...and my personal opinion is that Mario Galaxy is one of the best games I've ever played...and I've played a few ;). It solves the nightmare of the last 2/3s of Sunshine and best the beauty of Mario 64. Great Fun!
 

Dan doesn't seem to accept that the game, as published, appeals to a large audience even though he doesn't like it.

But that's exactly the point. Many reviewers don't seem to accept that a game as published appeals to a large audience because it doesn't appeal to them personally. Adventure game sites are constantly rolling their eyes at mainstream gaming sites that give out horrible review scores to such games for being adventure games. (That is, for having the typical characteristics of the niche, such as lots of puzzles which aren't always 'realistic' and, usually, a linear plot without much replay value.)

What's the point of giving it a low score? What useful information does this provide either to people who like adventures or to people who don't?
 

now admittedly adventure gamers are not that much of a LARGE audience (although they do exist) but with casual games, the point is there. If you don't understand the game's appeal at all, you really can't score it.

You can certainly talk about your experiences with it even if you don't get it, and if you use less value-laden language, this will still help people clearly see various features (and misfeatures) of the game.

Of course, people like value-laden language, because constant swearing is funny and drives people to read reviews. :)
 

@Russell - I think you stated the quandary more concisely than I did. Thank you!

And I think that's why my beef is more with review scores than reviews. I mean, I love Scorpia's reviews --- I have since her tenure in CGW. The lady does have a pretty fierce bias, however - and not one that matches my own tastes. But since I understand her bias, I incorporate that knowledge into what I read from her reviews and commentary - and I end up informed. It helps that she still writes very detailed reviews.

And yeah - I enjoy a broad spectrum of games, but I also have a pretty fierce bias of my own. I'll cut indie games some slack that I might not give a triple-A title, for example. I can totally get into a turn-based strategy game and lose track of hours at a time... but the very idea of turn-based strategy is like dragging fingernails across a blackboard for many players.

But with MetaCritic, GameRankings, etc., the personality and bias of individual reviewers get lost. So, as Whiner says, you end up with adventure games getting crappy scores by virtue only of them being an adventure game. Ditto for kids games.

So --- is there something that could be done to improve things? If embracing the bias is the way to go, it seems that we'd need tools that would do a better job of introducing readers to individual reviewers (not just sites), rather than de-personalizing things with aggregate, faceless numbers.
 

...and I think your thought makes a lot of sense and is what I believe is the future of reviewing. Making the reviewers more key in the review.

Notably that seems to step right against the Web 2.0 trend of the power of everyone being more valuable than the thoughts of anyONE. However, I think the internet should lead to more people finding reviewers who they agree with (or very much disagree with). The old Siskel and Ebert thing of polarizing reviewers who each have large following I think still has a lot of validity, but it isn't found in the games space. N'Gai and Totillo just aren't representative enough or distinct enough.

The other approach is for continued gaming niches with websites for each type of gamer. The problem I run into, as I think you do, is that I cross all the segments. I don't have time to search down all the sites to hit all my tastes.
 

@Josh: Can't say I agree with your Eschalon assessment, but it may be a combination of luck and trying to play a rogue from the get-go. My deaths kept mounting....

But that's where differing viewpoints come in so very handy.

@Whiner: You know, that's where having dedicated adventure-game / RPG fan reviewers (like Desslock, Scorpia, etc) can come in so useful. You know, so many of these mainstream games get reviewed by fanboys who try oh-so-hard just to suppress their fanboy instincts and provide some pretense of a rational, unbiased opinion of the latest Halo sequel. And come to think of it, if they'd put away that silly pretense and simply provided a well-thought-out gushing over the latest installment of their favorite game series, I'd be cool with that. I might enjoy the review, and with an understanding of that bias I may find it very informative.

I'd sure like more niche specialists adding their voice. And I'd like some filters to help me find those opinions, and get to know the people behind them.
 

About Eschalon - difficulty aside, any game with piles of gold lying about in random urns is definitely aimed at at a less-serious demographic. It looked like they wanted a dumbed-down Baldur's Gate, but without the complexity of implementing a full party. But somebody just LOVED the idea of a familiar. And when the familiar runs out of hit points, it "runs away". A far cry from Baldur's gate II where it would die PLUS you lost a point of con, permanently.

You asked what can be done about biased reviews. I'd like reviewers to briefly summarize their experience in the genre. Probably only a few will. My solution is to read the right blogs and get to know the writer a bit :) I doubt that any solution will be regularly implemented or institutionalized.

For example, Q. Random Blogger is very good at analyzing balance and other strategic issues. If he blasts Oblivion for its "self-balancing system", I take it very seriously and wouldn't put that game very high on my to-try list. But he simply can't deal with arcade-game elements, so whether he likes or hates the latest incarnation of Doom, I don't pay a lot of attention. (It would help if he wouldn't flatly pronounce that it's too hard, but hey, reading's free.)
 

@Josh

I don't think it's aimed at a less-serious demographic, but perhaps a more casual one, if you get my meaning. Personally, I thought Eschalon was a pretty good balance. When I want something to really challenge me, I go play Angband or Dwarf Fortress. When I pick up something like Eschalon, I'm hoping to be able to explore a bit, unearth some of the story, and not get (too) frustrated.

The piles of gold thing isn't quite as bad as all that; they at least made sure you only find it in places where it makes a tiny bit of sense to find gold (caskets, hiding holes, etc) and they had the good humor to set up the one scene in the lizard pit.

I suspect that the reason for the gold lying around is to help reduce the inventory-management problem that you get in a lot of these games. Gold is just a counter, after all, whereas saleable items require some tending, some trips home, and frequently some hard choices.