Beta Testing: Marketing the Modern-Day MMO

As the industry expands, more publishers are relying on beta testing phases for promotion and pre-orders. Meanwhile, painfully-obvious bugs are slipping through the cracks to launch.

Last month I came across an editorial by MMORPG columnist Garrett Fuller, titled "Over-Marketing and The Beta Effect;" a blog entry examining the over-hyped campaigns launched by MMO developers and publishers these days, some of which begin years before the game's actual release. Fuller explains the "Beta Effect" as a result of all this  publicity, driving too many modern MMO gamers to seek out beta access for all the wrong reasons (i.e., all those but actually testing).

While I agree with most of Fuller's points—which are more detailed than previously mentioned—there's another problem with modern MMO beta testing, and it's a trend that's gained momentum throughout the last few years. Most publishers have adopted the traditional beta testing model as a promotional and marketing tool, widening access by tenfold and offering unique incentives to join up, like head-starts, price discounts and exclusive in-game items. Using beta tests as a promotional tool encourages just about everything except the actual testing, which might be part of the reason why some MMOs face so many unforeseen problems once the floodgates open on launch day.

The concept of alpha/beta testing computer games and applications is pretty much as old as the software itself, and it's not a concept that's even unique to our generation. Every new product or innovation, from the discovery of gunpowder to the invention of the automobile, requires a great deal of real-world testing and trial runs before it's ready to go public.

Generally, MMOs require a lot more vigorous testing than other PC and console games, because there are so many more variables involved in their gameplay. The interaction of hundreds of thousands of players in persistent, online worlds leaves tens of thousands of bugs and gameplay issues that need to be resolved. Consider the 180,000 bugs Blizzard developers are currently tracking in World of Warcraft, which has almost five years under its belt. But before they can be fixed, bugs and gameplay issues need to be found.

Originally, alpha and beta testing phases weren't something that most gamers were privy to, let alone interested in taking part in, unless they were the "hardest of the hardcore." Game developers traditionally required time-consuming commitments from participants, stress-testing various game elements and tediously repeating many of the same actions and scenarios over and over again, doing everything they could to go looking for bugs.

Beta testers not affiliated with the developers (fans and community members) were expected to do a lot more than today's majority of participants, like replicating specific actions on request and submitting detailed bug reports on a regular basis. That's not to say that beta testing video games was ever as annoying as a job for most players, but it used to attract a much smaller crowd than it does today. Many gamers who sought out beta access knew it was a tit-for-tat relationship; in exchange for exclusive access to the game before everyone else, they understood a certain amount of debugging and constructive feedback was expected of them.

But in most of today's MMO beta tests, it's not uncommon to hear players complaining in-game about the very bugs they're meant to be looking for, without contributing to the resolution of those bugs and issues. Instead of taking an active role in an MMO's post-production refinement, many players are only interested in using the beta as demo, or getting a head-start in advance of everyone else. And to some extent, it's the publishers' own fault.

MMO publishers invite a vastly larger number of indiscriminate players in the newly-popular "open beta" tests, instead of the moderately-sized groups of dedicated testers they worked with in the past. In many cases, beta participation used to require a mandatory minimum amount of feedback, or specific, task-oriented testing and debugging. Although it's true that today's bigger and more-ambitious MMOs need more players to sniff out all those bugs and balance issues, the ratio of "dedicated testers" to "self-interested slackers" starts favoring the latter when publishers take advantage of their beta tests, using them as marketing assets.

As publishers increase the hype surrounding an MMO's launch by marketing its beta testing phase as a promotional tool, they're doing a disservice to both themselves and their gamers. Publishers have exploited betas in almost every conceivable way; offering beta keys with the purchase of separate MMO, hosting in-game live events with rewards that carry over to the official release, introducing head-start deals and discounts for participants and more.

As we've seen with recent MMOs like Aion, Fallen Earth and Champions Online, the practice of using beta phases as a marketing tool can provide an initial leg-up, boosting pre-order sales and community interest. But it's a short-sighted move that often results in a post-launch blowback, when hundreds of thousands of players are met with unforeseen problems like server instability, gameplay bugs and balance issues. Just like anything else, MMOs only get one chance to make their first impression, and those stellar pre-order numbers don't mean a thing for the game's future if players drop their subscriptions a few weeks after launch.

As Fuller mentions in his Beta Effect editorial, part of the problem might be that players and the media are spoon-fed too much hype, too early. "Setting expectations is great, but then you have to manage them," he wrote. "How many MMO companies have died in the last few years because of mistakes in managing the expectations on their games? How many more will follow?"

But I'm not sure if the over-hyping of premature MMOs is any more to blame than the way in which many publishers are conducting their beta and pre-release development phases. We all know that bugs and MMOs are a package deal, no matter what—especially within the first few months after a game's launch. However, there's no excuse when it comes to dismissing quality for publicity's sake, as some testers accuse publishers of doing.

Ultimately, the question is whether or not an MMO's launch could have been smoother if its beta phase was devoted more to actually testing the game, rather than supplementing a marketing campaign. What if EA and Mythic were able to delay Warhammer Online's release just a bit longer, focusing on stricter beta testing and taking feedback more seriously? Why did Aion suffer from some of the most painfully obvious problems and gameplay obstacles after all that so-called "beta testing?" So far, Aion has been able to ride out the initial blowback and overcome its launch problems respectably well, but will future MMOs be so lucky?

Gamers shouldn't have to deal with beta-quality gameplay after an MMO's official release, especially if it could have been avoided by forward-thinking development. After all, customer retention is paramount in this industry, and most players aren't interested in sticking around to wait for a prematurely-released MMO to stabilize. Not while they're paying a monthly subscription, anyway.

Tags: Editorial

Comments

Post Comment
­
# Nov 11 2009 at 2:32 PM Rating: Good
I don't understand why people WOULDN'T be reporting bugs if they were part of a beta test. I know they aren't getting paid for it, but come on... If you're testing out a game, you want to report bugs so they will be fixed. You'll have a much greater chance of being heard if you're a tester than if you are a player reporting a bug AFTER the game is released.
Post Comment

Free account required to post

You must log in or create an account to post messages.