Behind the Curtain: How Game Reviews Work

The Timeline

When review copies are forthcoming — which they often are, but not always — the lead time on a review can vary drastically. Sometimes, games are available for weeks ahead of release, such as with Saints Row IV. Other times, a copy will show up the Friday before release, or even later, which can lead to a sprint through the game in interest of meeting embargo.

In Game Front’s case, we tend to try to let reviews take as long as they need in order to ensure quality, but this is often easier said than done. As mentioned, the maximum value for readers and for the site is to post a review at its embargo date and time, and letting a review languish too much diminishes its worth for everyone.

That can mean some concerted efforts and sometimes even all-nighters pulled in attempts to get reviews done on time. Other, much rarer cases result in writers going to “review events.” In fact, Devin Connors attended a review event for Battlefield 4, and Alex Rubens attended one for Call of Duty: Ghosts. The ins and outs of such events are sometimes complex, but they mostly amount to writers going to a specific place and spending several days with the game — and nothing else.

Review events have their uses, such as in testing multiplayer, where there’s a captive audience to fill servers; it’s often tough to review multiplayer before release because there just aren’t enough people playing. Those events also have their own idiosyncrasies, though, because playing with a developer close at hand is much different from playing alone in your living room, where there’s no ready assistance.

Such events are rare, however (probably because they’re in no way practical for any but the largest publishers from a cost standpoint). Much more often, it’s a Steam code, a review build, or just a retail copy of the game when it’s available.

The Aftermath

Every review published potentially has repercussions attached to it. There have been scenarios in which certain publishers and developers seemed to be a bit icier to Game Front after receiving what was perceived as a low score on a particular game (even if that game’s score really wasn’t that low). Mostly, though, it’s business as usual, because every company is on to its next project, just as Game Front is on to its next review after each is finished.

Usually, PR employees and journalists have solid working relationships, and understand one another, but without interfering with one another beyond setting up coverage opportunities (you’ll almost always set up an appointment through PR for an interview, for example), scheduling review copies, providing information or press releases, and setting embargoes. Personally, I’ve been doing this for better than four years now at Game Front, and longer at other publications, and I’ve never been hassled by anyone, in any industry, over my coverage.

What doesn’t happen, almost exclusively, is any kind of definitive meddling with or pressure about a review score, either positive or negative. That’s not to say there aren’t subtle psychological probes toward the positive side of the scale — sometimes, though not often for writers at Game Front, a review copy will come with extra “swag,” like t-shirts. That stuff either gets given away or winds up in a closet somewhere at Defy Media.

And while some readers refuse to believe it, money never changes hands for positive reviews, for the same reason that overt negative pressure is rarely, if ever, used — there’s no way to keep it a secret, for one, and it’s completely impractical, for two. A positive review at Game Front just isn’t worth that much on its own to any given publisher; the same is likely true for any outlet you could name, especially in the age of Metacritic. What’s more, bullying any outlet with threats for a negative score is just begging to be outed to the audience, and the same is true for paying for reviews. Whistleblowers exist, and if these things were happening as often as some people think, you’d hear more about it from journalists. Apart from the ethics of the situation, there’s a lot to be gained in breaking the huge story of how EA or Activision paid for positive reviews, and the watchdog capacity of the press would come into play.

Any questions? If you’d like to know more about what we do here at Game Front and how it all works, let us know in the comments.


Read more of Phil Hornshaw’s work here, and follow him and Game Front on Twitter: @philhornshaw and @gamefrontcom.

Join the Conversation   

* required field

By submitting a comment here you grant GameFront a perpetual license to reproduce your words and name/web site in attribution. Inappropriate or irrelevant comments will be removed at an admin's discretion.

14 Comments on Behind the Curtain: How Game Reviews Work

Sandeep

On November 11, 2013 at 1:38 pm

This was a really great read, I learned a few things about what exactly you guys do and how difficult it sometimes is. I had no idea that you have to pull all-nighters. This just makes me appreciate all the hard work you guys put in. Thanks

Phil Hornshaw

On November 11, 2013 at 1:50 pm

@Sandeep.

Thank you for reading! All-nighters don’t happen often, but I’ve pulled a few. I finished Mass Effect 3 at like 5 a.m. with the embargo the next morning (if I recall correctly). That was a hell of a day.

Wayhawk

On November 11, 2013 at 2:30 pm

Thanks for a really good article. I think getting paid for good critizism, does’nt work in the long run. If a person or company recommended games i did’nt like. I would very soon turn to someone who gave reviews that coresponded to my taste. So it’s seems in the long run to do honest reviews.
Once again thanks for a great article.

Wayhawk

On November 11, 2013 at 2:37 pm

Thanks for a really good article. I think getting paid for good critizism, does’nt work in the long run. If a person or company recommended games i did’nt like. I would very soon turn to someone who gave reviews that coresponded to my taste. So it’s seems in the long run to do honest reviews pays out.
Once again thanks for a great article.

Crap

On November 11, 2013 at 3:46 pm

The only people who think that there’s a literal ‘payment’ for good reviews are either fanboys or people who deliberately miss the point of the argument. The reality is different but has pretty much the same goal – developers and publishers will offer better access and more lucrative info to outlets that they consider more likely to be on board with their business practices than others. I don’t know how anyone can even deny this to be true when the overwhelming bulk of evidence proves it, and GameFront made the point of challenging the consensus during the rest of the games media’s pathetic elitist backslapping last year regarding ME3.

Phil Hornshaw

On November 11, 2013 at 4:34 pm

@Crap

There is an issue of access being given to some outlets over others, but there’s not much we can do there. We have to leave it up to you guys to decide what sort of coverage you’ll seek out. It’s not an every-time thing — and there’s also the matter of how well a site “rates” among other big outlets. Game Front remains relatively new on the scene, so there are times when we don’t get the same kind of access as bigger outlets simply for that fact.

Other times, as I mentioned above, you’ll see things like “exclusive reviews,” which is a publisher giving specific access and often using embargoes to do it. The only thing that will change that situation, unfortunately, is fewer people reading those exclusives and stopping that strategy from being financially viable.

quicktooth

On November 11, 2013 at 9:18 pm

Gamefront, please read all the way to the end. This sh!t matters and so do you. It’s not short.

Guys. How many “review events” have been written up and talked about. The strippers. The booze. The shooting rocket launchers in the desert WITH strippers (Rockstar games did this, I believe). Not all events like that? How about the publisher paying for flights and accommodation? You lot have so many corrupting influences and pressures with these events that you would all, literally, have to be saints not to be influenced. And humanity is just so replete with hordes of saints (just ask the US government, the NSA, the CIA, Britain’s GCHQ, Google, Facebook, their apologists, etc!).

And a free copy of a game IS in fact a perk (guess who doesn’t get one- everyone else). But putting aside that for a moment, guys. Almost every game, almost every where, has a review of 7-10 out of 10. From EVERY outlet for reviews. All of them- except the few who fight back, and don’t put scores (or some other effort). That’s good. But to assert the whole industry is somehow squeaky clean when it’s patently clear you lot generally are bought and paid for (at least to bump up low scores if nothing else) insults our intelligence. Don’t do it.

Gamefront is different from everyone else BECAUSE it seems to have it’s readers interests at heart. True, you’ve suddenly experimented in blatant manipulation of your audience through their genitals (sexy cosplay galleries), which is what killed Kotaku for me. I’ve read at least one expose, from Penny Arcade’s game news section, explaining explicitly that you all make so little money from page views that you somehow “HAVE” to attack your readers through their genitals. Never mind it’s your d@mn JOB to figure out how to make money, in a place where you HAVE to respect the folks who pay for your livelihoods (you don’t have to, but look at the Sim City launch. Are YOU the person who wants to be the ex-CEO?). I hope you never do that again. I’ll leave if you do. There’s other places, like Polygon, that don’t seem to do it. I like Gamefront’s articles more, but if you try to manipulate me again I’ll just go. This is horribly serious; if your friends soullessly seduced you into things, if your PARENTS tried to seduce you into anything, would you just go along with it? No? Welcome to the Thinking Ape Species. Wish we had more members.

And don’t try to say that somehow I’m off my rocker, that all I’m saying here is made up. It is common knowledge. There have been too many people writing frankly about this industry for too many years. I have no idea where you get the idea there aren’t whistleblowers- there are many. For the best, try Insomnia’s “The Video Games News Racket”.

And if the Escapist can run a comic worriedly oberving that (to slightly paraphrase) ‘gamers just don’t TRUST games journalists. That much is clear from the Xbone launch’, then your industry has a problem. You created this. You accept conflicts of interest as part of doing buisness. You put useless reviews up for games that are actually glorified press releases. You BOAST about your corrput practices in your reviews. You blatantly attempt to suck money out of our penises. Guys. Don’t piss on our shoulders and tell us it’s raining. It’s beneath both you, and us.

Gamefront is mostly different from everyone else. It still mostly champions it’s readers. It has mostly put up really useful reviews (and don’t think they all have to be polished masterpieces, either; just honest texts). When the games industry attacks it’s customers in some new way (DRM, always-online, SOPA, PIPA, on-disk-DLC, etc) Gamefront justly calls that industry out at warns everyone about what’s happening. Gamefront is still clearly (mostly) not misogynistic (the current “cosplay gallery” excepted). But you still do a bit of everything. The only thing, of all, that simply reading your site isn’t enough to show you’re up to, is being *directly* bought for cash.

And we all know. All of us. Every games journalism outlet makes it clear with quite how open you all are about this (unless called out directly), and your utterly unrepentant stance on it. There are blind, enslaved people. They will in fact (no matter the cost) ALWAYS prop up their false beliefs about games companies (or whoever!) when they’re proved wrong, with empty and loud rants. The smart ones use words that honest people use but few others (eg. “you’re all so childish”), but they’ve done it so often and so publicly that it looks like people aren’t fooled anymore. They are not readers who speak the truth, just sad obessive folks who need help. They’re the only ones who stand up (so far as I know) for this sh!t. I love games. I love games journalism (for some reason). But it’s so obvious I can no longer trust most any review of games that I don’t. Ever. I always watch a Let’s Play to guage how much I might like a game. Then I get all the ones that look good.

As a final point, you know and I know that an RSS feed aggregator (etc) is enough to put you all out of buisness (we can just go to a company website for info on their stuff). If reviews and news were what it’s all about. It opinions didn’t matter. But they DO matter. YOU matter. What you SAY matters. Know why? Because you can spend more time on this thing we all do than anyone else. You are honest (mostly). You are insightful (mostly). And that’s why Gamefront is better than any other games journalism outlet, and why I still come here. Kotaku is dead to me (even though SOME of their people seem to still talk and act like people). I can’t bring myself to go to the Escapist anymore, though bless Jimquisition’s soul. It’s been a long long time since I was fooled by IGN or Gamespot. You lot are the last man standing, because you don’t exhibit the blindness you seem to in the article this is a comment on. Because you don’t (almost ever!) seem to AGREE with what you’ve written, above. Please stay good. We’ll stay readers. Ohhey- ever notice how Gamefront’s comments aren’t riddled with psychopaths and flaming @ssholes? There’s probably a reason… probably something something birds of a feather something…

TLDR- “We love you lots, but don’t piss on our shoulders and tell us it’s raining”.

Phil Hornshaw

On November 11, 2013 at 10:11 pm

@quicktooth

First, in defense of BlizzCon galleries: We’ve done these before at BlizzCon, because BlizzCon has, perhaps, the coolest set of cosplayers out there. They’re all video game-related and they all go all-out. The costume contest really is THE marquee event of the weekend for BlizzCon attendees. Second, I took a great many of the cosplay pictures and floor photos at BlizzCon this year (not the screen grabs from the actual contest, however), and I did my best to make it A Cosplay Gallery and not so much A Sexy Cosplay Gallery. People like photos of cool costumes and I tried to take photos of cool costumes. That sometimes includes sexy ones (of which there are an inordinate number at the event, in fact). It also sometimes includes not-so-sexy ones. I made an conscious effort there, but of course it’s up to you to decide if I succeeded on that front.

I’ll admit that in the past, Game Front was guilty of sexy cosplay galleries. We have no excuse, and I can only explain by saying that we were learning. I should hope that if you page through the gallery I made from the floor at BlizzCon, it would be apparent. There WERE sexy cosplayers at BlizzCon, as much as there were lots of other cosplayers. They’re excited to dress up for their favorite games, and people like galleries. I was banking on the fact that people would like cosplay galleries regardless of whether “Sexy” needed to be attached to the front. Again, you guys have to judge (and please, keep the feedback coming).

Anyway, on to the point at hand: I definitely, one-hundred-percent appreciate where you’re coming from. There are A LOT of perks in this industry for journalists — as there are in any sector that reports on entertainment or commodities. There’s also a lot of perks for movie critics or tech writers. I don’t do it so much anymore, but I spent some time covering tech for another website, and it’s much the same in games — people want you to like their products.

So what I was hoping to do here is two-fold: 1. Make an effort to pull back some of the shroud of secrecy on this stuff (which, judging by posts on Reddit and elsewhere, seems to persist) precisely for the reason that there are a lot of potentially corrupting influences, and 2. Provide a little of our perspective on some of those influences, which are really not as corrupting as you might think. Review copies, for example: yes, I have plenty of games for which I have not paid. But it’s important to remember that for each of those games I have not purchased, I have to put forth a concerted effort to play them to completion, whether I like them or not, for hours on end, and to turn around deep, often multi-thousand-word reviews full of critical thinking and analysis that are, hopefully, useful to readers. Any given review probably represents, on average, 12-15 hours of work. That’s for a fairly short game, say, Dead Space 3. It does not take into account any time spent editing, reviewing edits, or doing any of the ins and outs of publishing the review. Yes, that’s hours of playing games, but it’s just as often hours of playing games at the expense of having free time, hanging with friends or family, working on side projects, or even playing a game you’d like to play for fun rather than for review. My ultimate point is: writing a game review isn’t skipping through a meadow full of rainbows, even if the copy is free.

On to the rest of it: what I was driving at here, and I’m starting to think I didn’t get there, is that the day-to-day of game reviews is a rather simplistic thing. Game Front really, really rarely goes to review events, for example — it was actually something of a coup that we were allowed to go to the ones for Battlefield 4 and CoD:Ghosts, which I’m told were rather uneventful. We as a site (and I personally) have never attended a stripper-laden helicopter ride with rocket launchers, nor anything even remotely close. I only know about that stuff from reading about it on the Internet.

So I don’t “believe” really anything about this stuff, necessarily — I’m just telling you what it’s like for us. Ron Whitaker and Mark Burnham spend a lot of time playing email tag with PR people. We often get Steam codes for games we need to review anywhere from weeks ahead to the day of launch (see Bioshock Infinite’s Burial at Sea DLC, for example — review coming tomorrow because we don’t have anything to review yet). And like any publication, we try to take steps to divorce ourselves from the potential of being “bought” or coming off as “bought.” We care about doing a good job. For a journalist, as well as a publication, reputation is everything — it’s a helluva lot more valuable than some free games that may or may not even be good.

Anyway, based on the feedback to this article, I have some inkling of writing more of these, with, again, the intent of peeling back how things work here. I for one feel the “exposés” like the one you mentioned are often too cynical and too bitter to be useful: they paint games journalists as twisted monsters or unthinking automatons, sometimes both, and having worked at this stuff for several years now, it’s just not accurate in the way it’s presented (though I definitely don’t mean to say it’s wholly wrong — although the article you pointed me at from Insomnia is, in fact, wrong; if you want to talk further on it, hit me at phil at gamefront.com). Yeah, you could do a lot of what some games journos do with an RSS feed — which is why Game Front has greatly reduced our daily news content. We’ve maintained SOME because there’s a part of our audience that wants some. So when you say that it’s our jobs to figure out a way to make money, well, yeah, we’re trying to figure it out right now, in front of you.

I guess what I’m saying is this: Yes, continue to hold the people you read accountable, because it’s the only way any of us will get good games coverage. But also, too, remember that the men and women on the opposite sides of the keyboard are human, and in general are out to do a good job because they like that job and think it is, to some degree, important. Absolutely everyone needs to be kept honest. But it’s also not as bad as a lot of people think (like Alex Kierkegaard up there), at least not at our site, and much of how it all works is a system that is overstated by an outside viewpoint. I think trying to explain that system is beneficial. I’ll do my best to keep it as simple and honest as possible.

Ron Whitaker

On November 12, 2013 at 5:59 am

Quick aside on the Blizzcon cosplay gallery:

I look at the cosplay galleries as very different beasts from things like booth babe galleries. Booth babes are being paid to do what they do. Cosplayers are there because they love the game they’re cosplaying. They’re everywhere at shows like Blizzcon and PAX, and we love the passion that those gamers display. It’s not about making a “sexy” cosplay gallery, although there are always folks who could fit that definition. It’s about showing off the costumes and the creativity that gamers put together. It’s about letting our readers see what is going on at these cons outside the panels and the trailers. Heck, it’s about celebrating gamers being gamers.

My favorite cosplayer this year was the South Park WoW guy (great pic, Phil). Not sure how “sexy” that is, but it was a great costume.

Phil Hornshaw

On November 12, 2013 at 6:37 am

@quicktooth @Ron

There is a group of blonde booth babes at BlizzCon every year who are dressed as elves specifically for photo ops. If you look through the gallery — they’re not in it.

But anyway.

psycros

On November 14, 2013 at 7:10 pm

“I have no idea where you get the idea there aren’t whistleblowers- there are many.”

Where did YOU get the idea that the article claims this, when it says precisely the opposite?

Aedelric

On November 15, 2013 at 5:40 am

Reviewers are -all- so very trustworthy, *cough* IGN/Eurogamer/Gamespot *cough*.

In reality you have bad people and good, not every reviewer or site is the same. To a degree I trust Gamefront and perhaps some of it’s reviewers but as with everything, taste in games is a personal preference.

I rather tend to ignore reviews and reviewers, people by nature are not impartial and the more they insist they are, the less likely they tend to be. When it comes to games and determining if one is worthy of buying, I will try a demo, watch a few gameplay videos and read up on the game’s concept and features and news. I would not ask a stranger to go pick out my lunch if they did not know what I like, why would games be different?

Damien027

On November 18, 2013 at 6:52 pm

So how would you describe the perks you received from EA after being one of the sole voices of reason in a sea of shills & sell outs with the ME3 ending debacle? Do you even receive free review copies anymore?

Phil Hornshaw

On November 18, 2013 at 7:05 pm

@Damien

It’s actually not that bad, kind of as I mentioned above. A good, recent examples: Battlefield 4, which our Devin Connors played at a review event. It’s really a case-by-case basis. But like I said, the business of games is a whole lot bigger than any one title. Then again, we believe we had to purchase (at least some) Mass Effect 3 DLC to review it. It’s definitely pretty subjective.