Behind the Curtain: How Game Reviews Work

Playing the Embargo Game

Often, review copies come with caveats, like embargoes and review guidelines. Publishers add these “rules” as part of the agreement when they send a review copy — they agree to give us their product early and let us judge it how we see fit, and we agree to hold back that judgment to a certain date and time. Review guidelines generally have to do with what can be posted as video and what elements of story or character should or should not be discussed to avoid spoilers. (I’ve never seen a publisher or developer dictate anything more stringent than spoilers.)

Embargoes usually have a pretty good reason for existing: They put all coverage on equal footing. Embargoes get used in previews and other capacities as well, sometimes attached to non-disclosure agreements to make them even more foreboding, but in general they’re a good thing. If Game Front attends a preview event for a game in June, for example, but another event is happening for European journalists in July, the embargo lets everyone coordinate their coverage and release it at the same time — the European journos don’t get screwed over just because they’re European and their event happened later. When it comes to reviews, embargoes mean that it’s not a sprint through the game to write a quick review and get it out first for the hits; instead, all the reviews drop at the same time, and everyone gets an equal shot. Mostly.

So mostly, embargoes aren’t only reasonable, they’re helpful. They give us time to do our jobs well and allow the focus of writers to be on writing good articles, and less on out-competing each other with speed.

They’re not always for the best, though. There are times when NDAs and embargoes are wielded by PR agencies to give all outlets the chance at coverage, while giving a few outlets exclusive access to posting their reviews and coverage first. And when that happens, there’s little to do for journalists who are left waiting to publish but sit and wait, because burning an embargo can often mean burning access to a publisher and its developers. And that can limit our ability to do good work.

It’s worth mentioning also the flip-side of the coin, occupied by outlets or journalists who burn embargoes — and bridges — in order to post work first, for a short-term benefit that’s gained at the expense both of the other outlets that agreed to the embargo, and the PR firms and employees who created those embargoes.

Dealing with such situations is a constant battle, because, as readers might see it, a journalist’s job should be of an adversarial nature with those they cover. And it absolutely is — we still try to put tough questions to developers and publishers and hold them (and one another) accountable for their actions. There’s also a degree of cooperation and picking battles, however. Publishing a review earlier than its embargo isn’t really worth straining a relationship with the people that made that game, and it’s often looked down upon by other critics as being underhanded. But some stories are worth that strain, and it’s our job to recognize the difference and to weigh the consequences of such actions.

It’s impractical to avoid dealing with embargoes and rules altogether by buying every game we review, as some readers and Reddit posters have suggested. It strains our ability to do the job well, to get reviews done on time, and to budget Game Front as an effective business. There are plenty of times when we’ll pick up a game and review that purchased copy, but it’s not a sustainable practice for every review, and it doesn’t help readers.

Join the Conversation   

* required field

By submitting a comment here you grant GameFront a perpetual license to reproduce your words and name/web site in attribution. Inappropriate or irrelevant comments will be removed at an admin's discretion.

14 Comments on Behind the Curtain: How Game Reviews Work

Sandeep

On November 11, 2013 at 1:38 pm

This was a really great read, I learned a few things about what exactly you guys do and how difficult it sometimes is. I had no idea that you have to pull all-nighters. This just makes me appreciate all the hard work you guys put in. Thanks

Phil Hornshaw

On November 11, 2013 at 1:50 pm

@Sandeep.

Thank you for reading! All-nighters don’t happen often, but I’ve pulled a few. I finished Mass Effect 3 at like 5 a.m. with the embargo the next morning (if I recall correctly). That was a hell of a day.

Wayhawk

On November 11, 2013 at 2:30 pm

Thanks for a really good article. I think getting paid for good critizism, does’nt work in the long run. If a person or company recommended games i did’nt like. I would very soon turn to someone who gave reviews that coresponded to my taste. So it’s seems in the long run to do honest reviews.
Once again thanks for a great article.

Wayhawk

On November 11, 2013 at 2:37 pm

Thanks for a really good article. I think getting paid for good critizism, does’nt work in the long run. If a person or company recommended games i did’nt like. I would very soon turn to someone who gave reviews that coresponded to my taste. So it’s seems in the long run to do honest reviews pays out.
Once again thanks for a great article.

Crap

On November 11, 2013 at 3:46 pm

The only people who think that there’s a literal ‘payment’ for good reviews are either fanboys or people who deliberately miss the point of the argument. The reality is different but has pretty much the same goal – developers and publishers will offer better access and more lucrative info to outlets that they consider more likely to be on board with their business practices than others. I don’t know how anyone can even deny this to be true when the overwhelming bulk of evidence proves it, and GameFront made the point of challenging the consensus during the rest of the games media’s pathetic elitist backslapping last year regarding ME3.

Phil Hornshaw

On November 11, 2013 at 4:34 pm

@Crap

There is an issue of access being given to some outlets over others, but there’s not much we can do there. We have to leave it up to you guys to decide what sort of coverage you’ll seek out. It’s not an every-time thing — and there’s also the matter of how well a site “rates” among other big outlets. Game Front remains relatively new on the scene, so there are times when we don’t get the same kind of access as bigger outlets simply for that fact.

Other times, as I mentioned above, you’ll see things like “exclusive reviews,” which is a publisher giving specific access and often using embargoes to do it. The only thing that will change that situation, unfortunately, is fewer people reading those exclusives and stopping that strategy from being financially viable.

quicktooth

On November 11, 2013 at 9:18 pm

Gamefront, please read all the way to the end. This sh!t matters and so do you. It’s not short.

Guys. How many “review events” have been written up and talked about. The strippers. The booze. The shooting rocket launchers in the desert WITH strippers (Rockstar games did this, I believe). Not all events like that? How about the publisher paying for flights and accommodation? You lot have so many corrupting influences and pressures with these events that you would all, literally, have to be saints not to be influenced. And humanity is just so replete with hordes of saints (just ask the US government, the NSA, the CIA, Britain’s GCHQ, Google, Facebook, their apologists, etc!).

And a free copy of a game IS in fact a perk (guess who doesn’t get one- everyone else). But putting aside that for a moment, guys. Almost every game, almost every where, has a review of 7-10 out of 10. From EVERY outlet for reviews. All of them- except the few who fight back, and don’t put scores (or some other effort). That’s good. But to assert the whole industry is somehow squeaky clean when it’s patently clear you lot generally are bought and paid for (at least to bump up low scores if nothing else) insults our intelligence. Don’t do it.

Gamefront is different from everyone else BECAUSE it seems to have it’s readers interests at heart. True, you’ve suddenly experimented in blatant manipulation of your audience through their genitals (sexy cosplay galleries), which is what killed Kotaku for me. I’ve read at least one expose, from Penny Arcade’s game news section, explaining explicitly that you all make so little money from page views that you somehow “HAVE” to attack your readers through their genitals. Never mind it’s your d@mn JOB to figure out how to make money, in a place where you HAVE to respect the folks who pay for your livelihoods (you don’t have to, but look at the Sim City launch. Are YOU the person who wants to be the ex-CEO?). I hope you never do that again. I’ll leave if you do. There’s other places, like Polygon, that don’t seem to do it. I like Gamefront’s articles more, but if you try to manipulate me again I’ll just go. This is horribly serious; if your friends soullessly seduced you into things, if your PARENTS tried to seduce you into anything, would you just go along with it? No? Welcome to the Thinking Ape Species. Wish we had more members.

And don’t try to say that somehow I’m off my rocker, that all I’m saying here is made up. It is common knowledge. There have been too many people writing frankly about this industry for too many years. I have no idea where you get the idea there aren’t whistleblowers- there are many. For the best, try Insomnia’s “The Video Games News Racket”.

And if the Escapist can run a comic worriedly oberving that (to slightly paraphrase) ‘gamers just don’t TRUST games journalists. That much is clear from the Xbone launch’, then your industry has a problem. You created this. You accept conflicts of interest as part of doing buisness. You put useless reviews up for games that are actually glorified press releases. You BOAST about your corrput practices in your reviews. You blatantly attempt to suck money out of our penises. Guys. Don’t piss on our shoulders and tell us it’s raining. It’s beneath both you, and us.

Gamefront is mostly different from everyone else. It still mostly champions it’s readers. It has mostly put up really useful reviews (and don’t think they all have to be polished masterpieces, either; just honest texts). When the games industry attacks it’s customers in some new way (DRM, always-online, SOPA, PIPA, on-disk-DLC, etc) Gamefront justly calls that industry out at warns everyone about what’s happening. Gamefront is still clearly (mostly) not misogynistic (the current “cosplay gallery” excepted). But you still do a bit of everything. The only thing, of all, that simply reading your site isn’t enough to show you’re up to, is being *directly* bought for cash.

And we all know. All of us. Every games journalism outlet makes it clear with quite how open you all are about this (unless called out directly), and your utterly unrepentant stance on it. There are blind, enslaved people. They will in fact (no matter the cost) ALWAYS prop up their false beliefs about games companies (or whoever!) when they’re proved wrong, with empty and loud rants. The smart ones use words that honest people use but few others (eg. “you’re all so childish”), but they’ve done it so often and so publicly that it looks like people aren’t fooled anymore. They are not readers who speak the truth, just sad obessive folks who need help. They’re the only ones who stand up (so far as I know) for this sh!t. I love games. I love games journalism (for some reason). But it’s so obvious I can no longer trust most any review of games that I don’t. Ever. I always watch a Let’s Play to guage how much I might like a game. Then I get all the ones that look good.

As a final point, you know and I know that an RSS feed aggregator (etc) is enough to put you all out of buisness (we can just go to a company website for info on their stuff). If reviews and news were what it’s all about. It opinions didn’t matter. But they DO matter. YOU matter. What you SAY matters. Know why? Because you can spend more time on this thing we all do than anyone else. You are honest (mostly). You are insightful (mostly). And that’s why Gamefront is better than any other games journalism outlet, and why I still come here. Kotaku is dead to me (even though SOME of their people seem to still talk and act like people). I can’t bring myself to go to the Escapist anymore, though bless Jimquisition’s soul. It’s been a long long time since I was fooled by IGN or Gamespot. You lot are the last man standing, because you don’t exhibit the blindness you seem to in the article this is a comment on. Because you don’t (almost ever!) seem to AGREE with what you’ve written, above. Please stay good. We’ll stay readers. Ohhey- ever notice how Gamefront’s comments aren’t riddled with psychopaths and flaming @ssholes? There’s probably a reason… probably something something birds of a feather something…

TLDR- “We love you lots, but don’t piss on our shoulders and tell us it’s raining”.

Phil Hornshaw

On November 11, 2013 at 10:11 pm

@quicktooth

First, in defense of BlizzCon galleries: We’ve done these before at BlizzCon, because BlizzCon has, perhaps, the coolest set of cosplayers out there. They’re all video game-related and they all go all-out. The costume contest really is THE marquee event of the weekend for BlizzCon attendees. Second, I took a great many of the cosplay pictures and floor photos at BlizzCon this year (not the screen grabs from the actual contest, however), and I did my best to make it A Cosplay Gallery and not so much A Sexy Cosplay Gallery. People like photos of cool costumes and I tried to take photos of cool costumes. That sometimes includes sexy ones (of which there are an inordinate number at the event, in fact). It also sometimes includes not-so-sexy ones. I made an conscious effort there, but of course it’s up to you to decide if I succeeded on that front.

I’ll admit that in the past, Game Front was guilty of sexy cosplay galleries. We have no excuse, and I can only explain by saying that we were learning. I should hope that if you page through the gallery I made from the floor at BlizzCon, it would be apparent. There WERE sexy cosplayers at BlizzCon, as much as there were lots of other cosplayers. They’re excited to dress up for their favorite games, and people like galleries. I was banking on the fact that people would like cosplay galleries regardless of whether “Sexy” needed to be attached to the front. Again, you guys have to judge (and please, keep the feedback coming).

Anyway, on to the point at hand: I definitely, one-hundred-percent appreciate where you’re coming from. There are A LOT of perks in this industry for journalists — as there are in any sector that reports on entertainment or commodities. There’s also a lot of perks for movie critics or tech writers. I don’t do it so much anymore, but I spent some time covering tech for another website, and it’s much the same in games — people want you to like their products.

So what I was hoping to do here is two-fold: 1. Make an effort to pull back some of the shroud of secrecy on this stuff (which, judging by posts on Reddit and elsewhere, seems to persist) precisely for the reason that there are a lot of potentially corrupting influences, and 2. Provide a little of our perspective on some of those influences, which are really not as corrupting as you might think. Review copies, for example: yes, I have plenty of games for which I have not paid. But it’s important to remember that for each of those games I have not purchased, I have to put forth a concerted effort to play them to completion, whether I like them or not, for hours on end, and to turn around deep, often multi-thousand-word reviews full of critical thinking and analysis that are, hopefully, useful to readers. Any given review probably represents, on average, 12-15 hours of work. That’s for a fairly short game, say, Dead Space 3. It does not take into account any time spent editing, reviewing edits, or doing any of the ins and outs of publishing the review. Yes, that’s hours of playing games, but it’s just as often hours of playing games at the expense of having free time, hanging with friends or family, working on side projects, or even playing a game you’d like to play for fun rather than for review. My ultimate point is: writing a game review isn’t skipping through a meadow full of rainbows, even if the copy is free.

On to the rest of it: what I was driving at here, and I’m starting to think I didn’t get there, is that the day-to-day of game reviews is a rather simplistic thing. Game Front really, really rarely goes to review events, for example — it was actually something of a coup that we were allowed to go to the ones for Battlefield 4 and CoD:Ghosts, which I’m told were rather uneventful. We as a site (and I personally) have never attended a stripper-laden helicopter ride with rocket launchers, nor anything even remotely close. I only know about that stuff from reading about it on the Internet.

So I don’t “believe” really anything about this stuff, necessarily — I’m just telling you what it’s like for us. Ron Whitaker and Mark Burnham spend a lot of time playing email tag with PR people. We often get Steam codes for games we need to review anywhere from weeks ahead to the day of launch (see Bioshock Infinite’s Burial at Sea DLC, for example — review coming tomorrow because we don’t have anything to review yet). And like any publication, we try to take steps to divorce ourselves from the potential of being “bought” or coming off as “bought.” We care about doing a good job. For a journalist, as well as a publication, reputation is everything — it’s a helluva lot more valuable than some free games that may or may not even be good.

Anyway, based on the feedback to this article, I have some inkling of writing more of these, with, again, the intent of peeling back how things work here. I for one feel the “exposés” like the one you mentioned are often too cynical and too bitter to be useful: they paint games journalists as twisted monsters or unthinking automatons, sometimes both, and having worked at this stuff for several years now, it’s just not accurate in the way it’s presented (though I definitely don’t mean to say it’s wholly wrong — although the article you pointed me at from Insomnia is, in fact, wrong; if you want to talk further on it, hit me at phil at gamefront.com). Yeah, you could do a lot of what some games journos do with an RSS feed — which is why Game Front has greatly reduced our daily news content. We’ve maintained SOME because there’s a part of our audience that wants some. So when you say that it’s our jobs to figure out a way to make money, well, yeah, we’re trying to figure it out right now, in front of you.

I guess what I’m saying is this: Yes, continue to hold the people you read accountable, because it’s the only way any of us will get good games coverage. But also, too, remember that the men and women on the opposite sides of the keyboard are human, and in general are out to do a good job because they like that job and think it is, to some degree, important. Absolutely everyone needs to be kept honest. But it’s also not as bad as a lot of people think (like Alex Kierkegaard up there), at least not at our site, and much of how it all works is a system that is overstated by an outside viewpoint. I think trying to explain that system is beneficial. I’ll do my best to keep it as simple and honest as possible.

Ron Whitaker

On November 12, 2013 at 5:59 am

Quick aside on the Blizzcon cosplay gallery:

I look at the cosplay galleries as very different beasts from things like booth babe galleries. Booth babes are being paid to do what they do. Cosplayers are there because they love the game they’re cosplaying. They’re everywhere at shows like Blizzcon and PAX, and we love the passion that those gamers display. It’s not about making a “sexy” cosplay gallery, although there are always folks who could fit that definition. It’s about showing off the costumes and the creativity that gamers put together. It’s about letting our readers see what is going on at these cons outside the panels and the trailers. Heck, it’s about celebrating gamers being gamers.

My favorite cosplayer this year was the South Park WoW guy (great pic, Phil). Not sure how “sexy” that is, but it was a great costume.

Phil Hornshaw

On November 12, 2013 at 6:37 am

@quicktooth @Ron

There is a group of blonde booth babes at BlizzCon every year who are dressed as elves specifically for photo ops. If you look through the gallery — they’re not in it.

But anyway.

psycros

On November 14, 2013 at 7:10 pm

“I have no idea where you get the idea there aren’t whistleblowers- there are many.”

Where did YOU get the idea that the article claims this, when it says precisely the opposite?

Aedelric

On November 15, 2013 at 5:40 am

Reviewers are -all- so very trustworthy, *cough* IGN/Eurogamer/Gamespot *cough*.

In reality you have bad people and good, not every reviewer or site is the same. To a degree I trust Gamefront and perhaps some of it’s reviewers but as with everything, taste in games is a personal preference.

I rather tend to ignore reviews and reviewers, people by nature are not impartial and the more they insist they are, the less likely they tend to be. When it comes to games and determining if one is worthy of buying, I will try a demo, watch a few gameplay videos and read up on the game’s concept and features and news. I would not ask a stranger to go pick out my lunch if they did not know what I like, why would games be different?

Damien027

On November 18, 2013 at 6:52 pm

So how would you describe the perks you received from EA after being one of the sole voices of reason in a sea of shills & sell outs with the ME3 ending debacle? Do you even receive free review copies anymore?

Phil Hornshaw

On November 18, 2013 at 7:05 pm

@Damien

It’s actually not that bad, kind of as I mentioned above. A good, recent examples: Battlefield 4, which our Devin Connors played at a review event. It’s really a case-by-case basis. But like I said, the business of games is a whole lot bigger than any one title. Then again, we believe we had to purchase (at least some) Mass Effect 3 DLC to review it. It’s definitely pretty subjective.