Stitch on 25/11/2008 at 15:48
Focus testing is an incredibly valuable tool as long as the feedback is kept in perspective. For Bioshock, Irrational focus tested for a broad audience and implemented the results, which was the right course of action given their stated goal of broad appeal.
Focus testing doesn't make a dev team do anything, it simply provides feedback.
Qooper on 25/11/2008 at 16:05
The key to understanding the role and importance of bogus testing is throwing it out the window in the first pla..
Hmm. :weird:
/me reads the thread title again, carefully this time
Oh. :sweat:
BEAR on 25/11/2008 at 16:12
What I can't figure out is this (and I've said this plenty): why focus test with a group other than the one you are marketing to. I know the game was intended for a fairly broad appeal, but look what came from the developers prior to the game coming out. The fact that the game seemed nothing like what had been advertised makes me wonder what the point of advertising it was, do they think they will sell more games if the people who buy it don't realize what the fuck they are buying?
As for focus groups: as stitch said, they are just a feedback tool, and they are useful, but why do they pick random stiffs from the bus station to test their game? Did I not read that one person couldn't find their way out of the very first room in the game, when there is 1 direction to go? Throw that one out, seriously.
Don't advertise an "intelligent" game and then test it on people who half the time cant find the front door to their own house. This seems to be a problem for a lot of people. Take tribes:vengence for example. A lot of the betatesters were not tribes players, which is the only demographic you are guarenteed to get (if you make a half decent game). You hope to pick up people from elsewhere, but the people who already play the game are the ones who made it profitable enough to make a fucking sequel, so why for the love of god test the game on people who don't know a thing about what they are playing. The relativly few tribes people who got to betatest tried to explain how shitty it was, but the vast majority probably thought it was great. Game gets put on sale and guess what? Nobody buys it. Every tribes person who played the open beta (their first look at the game) said fuck this and didn't but it. You fucked the only people you really needed to buy the game, and for that you fail. T :DS had the same problem.
They need to learn how to put focus groups in perspective, yeah why did I bother writing all this.
june gloom on 25/11/2008 at 19:53
I think they're often trying to see if the game appeals to non-gamers, so they go down to the unemployment office and offer some fuckos a couple hundred bux to play something they'd otherwise never ever would have played to begin with. What you get is a game that appeals to nobody- non-gamers don't care, and the gamers hate it. The only people who you should be trying to appeal to are the pre-existing gamers. Non-gamers are typically either going to only play Halo and Wii Sports, or they're not going to play games at all, no matter how awesome your game is. That's because they're non-gamers.
Zygoptera on 25/11/2008 at 20:37
They (
http://www.ttlg.com/forums/showthread.php?t=111910) asked here for testers (hmm, wonder if that gives the timeframe for when they started gutting all the interesting game mechanics?), which is not quite the unemployment office. Don't know if anyone took them up on it though.
I do have to agree that focus testing on non or casual gamers is largely pointless, if they're going to try something more serious they will try Halo or CoD or something else with lots of brand and which they're likely to get recommendations for from Gamespot or IGN. For all its soft and fluffy you're in a safe place don't worry mummy will help you gameplay Bioshock still has features that baffle those who try it- eg how do I swim, and lots of people simply do not get the story at all.
Fafhrd on 25/11/2008 at 21:49
The thing is, focus testing is meant to be focused. "Do you prefer Control Scheme A or Control Scheme B? How intuitive did you find the interface?" and the like. When you broaden it out to encompass the entire game world, and complex AI systems, you're not going to get results with any meaning. It can take HOURS to really get how some games all work. When you say "60% of our group didn't understand the relationships between AIs after 20 minutes to an hour of play" that doesn't mean that there's something fundamentally wrong with the AI from a game design perspective, it just means that some people are going to need longer to figure it out (and would likely play on a lower difficulty until they do).
