Tales of the Rampant Coyote

Adventures in Indie Gaming!

Game Journalism and the Challenge of Early Releases

Posted by Rampant Coyote on February 6, 2014

When I complained about “Early Access” games a month ago – games that are released (and sold) to the public in an unfinished state, I neglected one of the critical aspects of gaming that it impacts.

What are game reviewers supposed to do about rating early access games?

The only thing I can figure, like the reviewer, is updating the review to match the development of the game. But who has time for this?  Especially if it wasn’t fun on initial release. There are gazillions of new games released every week, and they can all use reviews. I don’t want reviewers stuck re-reviewing the same game every two months.

In theory, I think early releases are a great thing. Customers may have to deal with software being less stable and mature, but in return get to help drive development, and (hopefully) get it at a discounted price. Win / win right?  And the software gets treated – at least for a while – as “live” development, constantly being improved. I’m used to regular updates of Windows, Unity, Blender, and other software.  That’s a decent way for software to work, right?

To be fair, I think that’s how many (if not most) game developers honestly think of it, too.

But in practice, too often this system gets abused. It’s an opportunity to crank up the old “release it broken, and patch it later” cycle to eleven. But now – a studio without much of a reputation to risk can collect half the revenue for a game up-front with only 25% of the work. All you need is some clever marketing, pretty pictures, and something a little cooler than a tech demo. (Oh, did I just describe the majority of Kickstarter game projects as well?) Maybe they even go into it with good intentions, but as the pre-order revenue dries up and they realize they aren’t going to make the kind of money they’d expected for this game, it becomes easier just to quietly taper off development, rename the studio, and move on to bigger and better things.

I think if we could solve the review problem, it would help. It would behoove studios to make sure that what they release is worthy of a good review. It would probably encourage studios to release a game only when it was far closer to completion, and has received some major polish efforts.

Some guidelines I’d like to suggest to reviewers:

1. Having “Early Access” reviews specially marked as “Work in Progress” (WIP) reviews might help – an obvious, impossible-to-miss visual indicator. This would at least clue in readers that there’s a difference – and risk – when comparing a commercially available work-in-progress to an actual “released” game. Obviously, both may undergo changes, but part of the rating / recommendation for the WIP would be based on its potential and promise, as opposed to what’s actually there. The former is a guess – the latter is far harder to deliver (and easier to criticize) and should count for more, if only by differentiation.

But really – an “early access” WIP games request a different set of standards than a finished game – always adding the caveat that it’s still in full development and may improve substantially over time. I think reviews need to acknowledge that, and make it very clear that they are operating under much softer standards than a full release.

2. While doing constant updates to a review for an early access / commercially available work-in-progress, minor updates – a “delta” to show how things are going – could be in order. If things seem to be really improving, maybe a “+” or “++” could be appended to the score. If things do not seem to be improving over time, a “-” could be appended – again, with potential comments. In my view, a game that hasn’t been updated (or only received a “token” update) in three months should earn an automatic “–” suffix. Exceptions may be granted , but it seems like a game that is still “in development” and ready enough to be sold to people should be able to have updates once a quarter.

3. Possibly provide two scores or recommendations for WIP games. One would be for the game as it stands at the time of the review, and the other would be the guess as to likely deliverable value. The first one would be more important, as it is what the player might be stuck with should the developer stop development and never produce another update (always a possibility).

My biggest concern overall is that we’re going to end up with a barren indie wasteland of incomplete games, and customers suspicious of indies who never finish their jobs.

On the flip side, I think developers who continually update and improve their products after their official “release” deserve plenty of praise. It’s easy to come down on them and say, “You should have fixed this before release,” but I think most of us are tech-savvy enough to understand that there’s no such thing as perfect software, and every release has its share of surprises – sometimes bad. But the ones that keep fixing and improving their software – for free – deserve praise.

Ultimately, I guess what I am calling for is sticky reputations for developers. A dev with a good rep deserves the benefit of the doubt with early releases, crowd-funding efforts, and so forth. A developer without any track record should be viewed with suspicion. And any developer who has screwed up in the past and not made a good faith effort to make things right by their customers should have a lot of repentance to do in order to escape their reputation. And really – game journalism is where these kinds of reputations (or lack thereof) can be applied.

Filed Under: Biz - Comments: 8 Comments to Read

  • Bob (probably not my real name...) said,

    I couldn’t agree more with your thoughts on Early Access/Kickstarter. It also crowds the field of what games are actually released. I go on steam and sometimes have a hard time figuring out whats a “new release” and what’s early access. Sometimes I think it can hurt them too because sometimes (I suspect) they get attention in early access and then reviewers have already moved on to the next early access and maybe all dev gets is “btw, X game is now final”.

    Then of course you have the kickstarters that are (suspicion) underfunded even when they meet their ‘goal’ of $20k.

    On the flipside, I agree the early access/kickstarter approach has some real positives. I think it offers some game designers/devs the ability to consider/stretch to new IPs. Jeff Vogel (Avernum, Indie Dev, Spiderweb Software)frequently re-mines his IP and has stated before it comes down to sort of a ‘bird in the hand’ approach vs. risking development of a new IP without interest. If he kickstarted a post-apocalyptic/sci-fi setting (for example) or other kind of game, he could judge in advance, or at least get a sense of the interest in going in that direction.

    The other plus is that kickstarter/EA can allow you to shorten the development/risk timeline by engaging artists or programmers or whatever additional support you need by giving you the cash reserves to contract with surety you’ll be able to pay them.

    Positives and negatives, but generally, I bookmark the games I like and then occasionally poke back in once every month or so and see how things are going.

    For what it’s worth, I’d greenlight/kickstart Frayed Knights 2. You’ve demonstrated you can deliver a product I like, and I support IPs and Devs I like with my cash when I can. I bought expansions to Warhammer Quest (IOS) and the deluxe version of Might and Magic X (PC) because I wanted to support them and the kind of games I like, more than any real desire or enticement for the benefits of the IAP/deluxe version.

    So.. Frayed Knights 2 is in my bookmarks.. See you all in month or so! (newsletter sign up?)
    –Bob (Just in case the NSA is watching these gaming boards…)

  • morteng said,

    I don’t understand the hand wringing about this. Early Access is for people who are already invested in a game (like people who used to create guilds, or engage in roleplaying on forums, for MMOs years before they were done). Insofar as journalists should acknowledge their existence at all, it should be treated like any other preview (note the p). Consumers need to engage with Early Access like the ‘devel’ branches of open source projects. That is to say, they should consider engaging with an Early Access title as their contribution to the project (testing/feedback), not something they do purely for their own enjoyment.

  • Rampant Coyote said,

    @Bob – Thanks. I will keep that in mind. I haven’t ruled out crowdfunding something in the future, but I’m not in any hurry. Sounds like my usual problem. I’m glad you are excited about FK2. I will have some cool things to show in a few weeks. But yeah – a lot of kickstarters really need more money than they ask for, but they are afraid of asking for too much and NOT being funded. So they ask for less and hope they overfund.

    @Morteng – The problem is – in my view, at least – early access games are starting to be sold just like any other games. They go on sale, they get reviews, they have forums, they are included in bundles, they get voted on for Game of the Year (depending upon the award-giver)… etc. etc. etc. While places like Steam and Desura do not hide the fact that they are early access (this would be counterproductive for everybody), they are just another game descriptor now. It’s funky.

    If it’s being sold like any other game (more or less), it’d be useful if there was someone on the consumer advocate side saying whether it’s something really cool, or not.

  • Anon said,

    I concur with most things being said (especially with the suggestion to label reviews), but I can’t agree to this one here – at least not in the general way it seems to be meant:

    “but I think most of us are tech-savvy enough to understand that there’s no such thing as perfect software”

    While it’s true that buggy software & patches are now apparently accepted, there is a fundamental difference between a game and an operating system or another productivity tool: A game is usually meant for entertaining purposes and most often works on the principle of immersion.

    Everything that breaks this immersion reduces the fun/entertainment (unless you specifically play something because it’s buggy or because it’s your job, as a reviewer). This is why I consider bugs in “not really important software” like games often more critical than a bug in a compiler that limits the functionality of an experimental optimizer, for example.
    If a bug renders a productivity tool basically useless, then it’s naturally “game-breaking”, too.

    And of course: Operating systems have become so vast that they can’t be error-free (and nobody in their right minds believe that) – but they are hardly comparable to game software. A big uproar only happens if a basic function like starting a program isn’t working as expected, it seems. But this is a pretty low hanging fruit, is it?

    And this all sums up why I don’t pay for public betas or newer (and expensive) mainstream games in version “1.0” – they all get patched up to full functionality later, which is when they are also much cheaper…

  • alanm said,

    I don’t really think early-access releases should be reviewed. My time is precious and I’m not interested in playing a half-formed game, let alone reading someone’s detailed review about a half-formed game. Then when it’s released, what? You expect me to play through all the same content again? Read more reviews of material that has already been partly covered? No thanks.

    I’m in software, I understand that you need to iterate and gather user feedback. So enroll people in your open beta program, cultivate your fan forums and so on. But don’t force it on the general public. Release it when it’s ready. I’m patient, but you only have one chance to impress me.

  • Ryuken13 said,

    At a minimum, these Early Access games should be completely separate lists and not on the same marketing pages.. My point is if you want to look at an Early Access game it should be on a page that you are fully prepared to look at alpha and beta stage games.. The games need a fair and realistic overview of the game build for prospective buyers and returns need to be given when the developer betrays the backers, such as the Realms of Arkania remake..

  • SniperHF said,

    I don’t think people should be formally reviewing early access titles. Any articles that even match the depth of a true review should be labeled previews and have no scores.

    Some sites obviously aren’t going to do this but I think it’s the fairest way to everyone. A reviewer wouldn’t normally post a 3-5 page review with a score and recommendation based on a demo he played at a trade show. Treat Early access the same way from a review standpoint. Talk about it and discuss what it is and how it plays but don’t score it.

  • ShadowTiger said,

    Early Access games should be treated like Alpha and Beta… and while MMO’s soft launch into Open Beta all the time, I think only preview articles should be written about Early Access games.

    When the game is declared finished is the only fair time to give a review.

    Journalists should give evolutionary reviews from Early Access to Release to Post-Release when DLC and big patches are released. Like all things in life, games are a trend over time and not a single data point.

    So I hope that journalists have an extremely difficult time with Early Access games and that forces them to reevaluate how they do things.

    In the mean time, I prefer “Wot I think” and “WTF Is” type reviews anyways… metacritic and its informants only serve as a rough indicators of public opinion for me.