And Yet More IGF Problems

Posted by Rampant Coyote on March 1, 2012

I don’t want to disparage the IGF right before the awards, ‘cuz I’m sure they will be awesome, and the games will be too . If they may not necessarily the best indie games of 2011, they will at least be quite worthy of attention.

But this post confirms some deep-rooted suspicions about the competition, which I hope will be improved upon in the future:

What’s Wrong With the IGF?

Quick Summary: Because the IGF chose TestFlight for their iOS distribution, which gave canny developers access to information on the judges’ activities with the game. In short, of eight total judges assigned to the game, one never installed the game, two others installed it but never played it even once, and of the other five, only three played it for more than ten minutes. Only one played it for nearly an hour, giving it what the author considered a good, honest shake.

Fair? Maybe, maybe not. While the $95 entry fee is pretty low, I’d still expect the judges to install and try my game for at least a few minutes. I understand the judges are busy folks too, and they may have their own methods of “weeding out” games that are clearly not going to make the cut for the year’s finalists. With all the entries this year, it’s understandable, and in a sense the first round of judges are the “slush pile editors” for the competition.

But it really makes me wonder, then, as to the value of the IGF. As many people have noted, the only games guaranteed to get a good, hard look are the ones that have already made a name for themselves. Games that are not already popular have the deck stacked against them, particularly if they are unable to stand out to the judges who are supposed to be giving them a serious look. Isn’t the IGF supposed to be about shining the spotlight on these otherwise “unknown” games? Or is it just a popularity contest?

I don’t know if there’s a good solution. Maybe the IGF needs to raise its entry fee and pay the judges a bit more and guarantee a certain amount of play-time on a title. Maybe there needs to be a better structure for how the nominations “ripple up” to become finalists in each category. Whatever the case, while a hundred bucks isn’t very steep, it’s enough that someone shouldn’t be concerned that their entry will be completely ignored.

Filed Under: Indie Evangelism - Comments: 16 Comments to Read

Noumenon said,

I can see judging a game in ten minutes, but not playing it at all is beyond the pale. Get new judges.
Rampant Coyote said,

There are a lot of games I’ve dismissed in under ten minutes. But I’ve never been asked to be a judge for them, either.

But I think this once again reinforces the “first five minutes rule.” If you can’t even get the judge for a competition to play it for more than five minutes, how will you get a customer to do so?

Another point: The IGF solicits unfinished (but late-in-development) games. Before they’ve had a chance to get polished and get promotional materials (videos, etc) finalized. How is one supposed to compare a finalized, polished game with a lot of “indie cred” and a real (but indie) budget to an early-beta title by some talented but inexperienced (and unknown) developers?
Lars Doucet said,

I think a lot of the IGF’s problems could be solved if they would just move to only allowing finished, released games.
Rampant Coyote said,

Yeah, but becoming an IGF finalist *really* boosts your chances of getting onto Steam & getting some media attention when you release. It’s extremely valuable to have pre-release.

But then we have Monaco, which won the IGF grand prize two years ago, which has still not yet been released… I like Andy a lot, but man… release your game already, dude!
Yodhe said,

If you have to pay to enter the competition, then all it really is just another marketing/promotional “scam”. And you know what they say about a fool and his money.
Rampant Coyote said,

“… are soon partying?” 🙂
WhineAboutGames said,

“pay the judges a bit more”

You mean, more than zero? 🙂

I’ve already had my say on this subject a few times, but to reiterate, I don’t think the dinoland people got a bad shake. They didn’t get what they _wanted_, but they weren’t treated unfairly or ignored.
Yem said,

IGF Solicits “unfinished” games. It is ultimately the developer’s call as to whether their game is in an acceptable state or not. If the game is barely playable and is submitted, whining that you didn’t win is a bit silly.

The number of games I have dismissed after playing for very little time is fairly high. It doesn’t take that many issues for someone who plays a lot of games (especially if required to do so) to walk.

Further, how many judges were, quite simply, turned off by a contestant nagging them to play their game already? That would probably put me off a touch.

Really, one way or another, someone not getting the result they feel entitled to out of a marketing endeavor whining about it isn’t exactly credible.
Rampant Coyote said,

@WhineAboutGames – would you mind linking to some of your comments on this? You have been on both sides of it, haven’t you?

Ultimately, my biggest concern is that there were a *staggering* number of entries this year, and it’s only going to get worse.
spicey said,

Require a review of the game from the judge, longer than “it sucks”. Thus, the developers would be guaranteed to get a couple of good reviews from Important People they so much crave for, and that could make them feel their IGF money wasn’t spent in vain.

The player is more than welcome to toss out a game he dislikes after a minute of playing. But for a judge? Doesn’t they really have anything to say? Well, maybe the graphics tore their heart. What about the level design? Overall world? Sounds? Music? Anything?

If a judge is unable to find anything meaningful to say, then good riddance to them, there is an abundance of Simons Cowells already.
WhineAboutGames said,

Eh, I’ve mostly been ranting in forum and comment threads that are hard to link to.

I’ve been on both sides because I thought the most useful thing to do after being frustrated with the contest as a competitor was to get involved as a judge. 🙂 The contest remains imperfect in a vast number of ways, but I see them _trying_ hard to meet their goals, and that compels me to defend them.

But yeah, theres a ridiculous number of games (and personally I don’t approve of lumping the phone games into the main category) and there are guaranteed to be good games that don’t make the final cuts for any number of reasons.

IMO we need more competitions with more specific remits in order to provide different avenues for showcasing.

(Of all the things I’ve made, LLTQ is probably the most likely to have a slim chance of being recognised at the IGF. It plays faster and the key design concept behind it is easier to see, and can actually be argued as a ‘design’ rather than pure storytelling, which is something the IGF still lacks a category for.)
Joe Larson said,

Only one judge actually played your game for an hour? You should demand a refund. I’m serious. They took your money and didn’t even give you a chance. Get a refund.
WhineAboutGames said,

joe: I really hope you’re kidding.

You are not paying for hours of playtesting time with that entry fee. If no one played your game AT ALL you deserve a refund. If they played it sufficiently to determine that it was not a contender for any of the award categories and then moved on, they *did their jobs*.

The first-line judges exist to sort the no-hopers from the possible contenders and pass those along to the second-line judges who play things in more detail.

Playing every IGF entry for an hour each is utterly and completely a waste of the judges’ time. A depressing number of them do not even launch, even with repeated followups to the developer, and a few more launch but are so unplayably buggy that they crash every few minutes.

But let’s say you’ve got a game that does indeed work, but it’s made of mismatched art, no sound, and derivative gameplay that seems poorly balanced. You scour the game’s description looking for hints of irony or hidden depths and you find nothing. You play through the first level, which takes a couple of minutes, and see no sign of the game suddenly changing for the better. You search the web for more information on the game in case you’re missing the joke, you find nothing.

You’re an unpaid volunteer who took this job because you love games. How long are you going to keep playing a game that you actively don’t enjoy and are not going to nominate for anything?

Conversely, some games you don’t have to play for more than a few minutes before you can obviously tell that they ARE contenders (generally in Art or Sound).

The games that demand the most playtime are those which are vaguely intriguing but unclear, the ones that you’re not sure about and need more time to evaluate.
pajamas said,

“…utterly and completely a waste of the judges’ time”
Oh, boohoo judges and their precious time. They signed up for IGF, least they could do is actually judge.
delve said,

@pajamas Belittling the judges isn’t a good way to accomplish anything useful.
*Everyone’s* time is the most important, non-refundable asset they possess. Let’s keep that in perspective while we re-read WhineAboutGames’ comment, shall we?
WhineAboutGames said,

@pajamas

‘Judge’ and ‘Playtest’ are not the same word. 🙂

There’s no point in playing a game for a mandated stretch of time if you already HAVE judged it.