catbarf on 24/11/2008 at 01:54
There's a lot of complaining on the forums about focus testing. I know that I've certainly put blame on focus testing for some of the simplification in Bioshock, where elements were altered or cut to make the game appeal to the average gamer. I was playing Left 4 Dead the other day and decided to listen to the developer commentary, where they explained that many of their choices were due to feedback from the focus groups, and the changes implemented were things that really helped the game.
What was different between the two? Where is it that focus testing goes wrong? Was it in Bioshock's case the attempt to take a complex game and give it to some random guy to test, making the difference one of genre? Do companies handle it very differently?
In other words, where does focus testing go from a positive aspect that enhances the game to a negative impact that damages it?
Chade on 24/11/2008 at 02:35
Complaining about focus testing has always been a bit dodgy ... it's not as if anyone here actually knows what the games were like before they were focus tested, after all. There's just a lot of speculation - some of it with a solid grounding in dev interviews, and some of it probably being wildly optimistic.
Focus testing is a pretty obviously good idea ... acquiring solid information about how people actually play your game is always going to be a good thing. But just having that information doesn't dictate how you make use of it.
My (speculative) impressions of the BS dev interviews are that the devs were pretty surprised at the results of the focus tests, and got a bit scared. They probably ended up playing it more safe then was strictly necesary. It's not really a problem with focus testing.
The other side of the coin is who you focus test on. And again, if you are bitter about games being focus tested on "random guys", then your problem is really about the game being targetted at those "random guys". It's still not really a problem with the focus testing itself.
Angel Dust on 24/11/2008 at 02:36
Yeah, it's a tricky one but I think it really comes down to knowing your audience and therefore your focus group. I would say most games could benefit doing a little bit of focus testing with complete noobs as far as usability is concerned but hold off testing and changing game mechanics until you are actually working with your audience focus group.
As for Bioshock, Irrational obviously wanted to make a more mainstream version of SS2, and there is nothing wrong with that, but I don't think they really had a clear vision of exactly what that was and consequentially the focus testing pushed it too far in mainstream direction. Meanwhile Valve South, from what I've heard of the 'Left 4 Dead' commentary, had a clear vision of what they wanted to achieve and used the focus testing to refine and enhance that vision.
Scots Taffer on 24/11/2008 at 02:47
Yeah, I wrote a post then shitcanned it because I don't want to rant on BioShock anymore but essentially if you're employing a focus group it is to test your intended audience's reaction to your intended gameplay experience. I don't know if they selected the focus group correctly for BioShock or if the gameplay was that terrible that it deserved scrapping, but ultimately they totally flipped on what their original intention was for the game whereas I think focus testing is supposed to generally return areas for finesse or improvement, not scrap it and try again.
Melan on 24/11/2008 at 08:50
Quote Posted by Chade
Complaining about focus testing has always been a bit dodgy ... it's not as if anyone here actually knows what the games were like before they were focus tested, after all. There's just a lot of speculation - some of it with a solid grounding in dev interviews, and some of it probably being wildly optimistic.
Well, on the other hand, I certainly enjoyed games more before developers started to mention they were doing "focus testing". They obviously had testers before, but they didn't go into all the hullaballoo about placating focus groups; it was a background thing, not the big deal it seems to be now. So I think there is a correlation between this trend and games moving in a direction I don't like so much. But of course it is still speculation.
Gaph on 24/11/2008 at 09:00
I guess it's playtesting's absolute focus on the gameplay, as if nothing else existed, that rubs me the wrong way. That's a great way to design games like Tetris or Bejeweled but a long time ago I realized that gaming can be much more than just a fun way to pass the time. The other side, what I think gaming is really all about, is roleplaying or immersive sim or whatever you want to call those games that TTLG is particularly interested in.
VTM: Bloodlines would've been a disaster at a focus testing but for what it was worth, let you convincingly play the role of a vampire set loose around California. Letting you play a role and embellishing it with enough meaninful choices and it becomes much more than just fun. It's fulfilling a fantasy you could never have otherwise experienced and the more embellished, the more convicing and the better it becomes... but I feel like I'm getting way too indepth.
gunsmoke on 24/11/2008 at 15:52
I think that it's changed from testers just trying to find bugs and whatnot, to full blown demographical focus testing. The job has changed significantly, anyway.
dvrabel on 24/11/2008 at 18:50
Focus testing (if properly applied) might have saved Bloodlines from some of the disasters in the later part of the game.
Bloodlines has the dubious pleasure of being one of two games I've cheated in. In Bloodlines' case it was the awful, never-ending fight with that tentacled thing. The other game is Mafia, where (before Illusion Softworks got a clue and patched the game) I cheated past the race.
mothra on 24/11/2008 at 18:53
i always thought playtesting has nothing to do with focustesting.
i'm a playtester for a mod and we give suggestions when something is inconsistent or illogical but mainly our purpose is to find, document, regress and confirm bugs, bugs, bugs, bugs,......and play the game to provide data for balancing. focus testing would be if the devs invite some "outside" guys to look at new maps and features and see if they "get" them, at what part of the map they get stuck or what feature they never use.
I bring an example for dystopia. the legboosting implant has been revised and tweaked but only in usage, not in energy drain/cost. so you only have to hit a different key-combination to activate it. playtesting showed that it worked as intended. focus testing showed that the new config scheme is too complicated to pick up and clumsy in a few situations, especially for "new" players. so it was changed. it was a valid input, us old testers just didn't have the feeling for it to be complicated.
the dreaded focustesting I always speak about makes the dev team cut the feature totally, change the speed of the player avatar to compensate and in that process change the overall feel of combat fundamentally which in turn trickles down the whole development chain and volá: biohysteria is born.
to avoid those extremes (and save time in developing an functioning system of meaningful branches the player can choose from ...think..implant groups/perks/skill sets) you go for the "more of the same" route: volá: the bioshock plasmid system of no significance is born.
The_Raven on 24/11/2008 at 19:48
The only other "downside" I can see from focus testing is the fact that it sometimes completely removes the game's character. Given the place that this is, I think most of us will agree that the little flaws in the old Looking Glass games helped give them a feeling that would have been completely butchered with today's type of focus testing. Sure, major things like the gameplay and interface should be tested on a wide range of people in order to make sure the whole thing is intuitive as possible to people who haven't been staring at the thing for months, etc... Smaller things, however, like minor encounters might be just better served by trusting your instincts.
Since I'm having trouble articulating this abstractly, I'll provide an example. Take the difference between Half-Life 1 and 2. While it is a common fact that both games were focus tested a lot, it is pretty obvious in the final product that the second one was more so than the other. Personally, I liked the original game better than its follow up. The original game had all sorts of highs, lows, and little touches that made the whole thing come alive from my perspective as a player. Half-Life 2 had these huge set pieces that were exciting the first time around, but completely typical and boring on repeated playthroughs of the game. I just felt that Half-Life 2 didn't have the soul or character that the first game possessed because everything was buffed and polished to a mirror shine, and so became very boring. It guess the best non-gaming example I can think of is the difference between an old-fashioned, hand-built wooden desk and one of those curvy plastic and shiny aluminum ones. The later just comes across as more flashy and hollow, no matter how solidly the individual components were constructed.
