[Testing] The "Quality Bar(s)"

One term that gets bandied about on a regular basis is the "quality bar." Technically, it means the level of quality that you are shooting for, but quite often the real question becomes what exactly the quality bar is. The plain truth of the matter is that the quality bar is not just one bar, but a collection of various metrics.

Let's go over some of the various quality bars that people may use to measure their project, and some examples of where each bar would lie.

"Stability Bar": Does the game function in such a way that the vast majority of people will never encounter a crash or data loss? Some games, like "Diablo," are at the high end of this bar. In fact, I've never had "Diablo" crash. Other games, like the original boxed release of "BattleCruiser 3000 A.D.," would fall on the low-end of this scale.

"Usability Bar": Does the game function in such a way that the average user is able to figure out what to do at a particular point in time? Does the game act predictably when the user makes his command? Just so I can compare two products from the same company, I'd put "Curse of Monkey Island" on the high-end of the usability scale (except for the ship combat portions), while I'd put "Grim Fandango" on the low-end.

"Appearance Bar": How does the game measure up to the visual concepts and promises made by the team? How does the game compare to other projects not only on the market, but coming out? Have you promised "3D Studio MAX"-render-quality and are delivering "Joe Bob's Stick Figure Theater?" ("Each one of those stick figures contains over 10,000 polygons and a full skeletal animation set..." "BUT THEY'RE STILL STICK FIGURES!!!")

"Playability Bar": Does the game provide the player with a set of interesting choices? Are there any times when the player is confused, frustrated or bored with the game? Is the game only completable if the player can hold "tea" and "no tea" at the same time? Does the game reset the progress on everything else you've done every time you complete an objective? Is your framerate acceptable?

"Consistency Bar": Does the world and the game feel like a continuous whole? If not, is there a compelling reason for it? "Psychonauts" pulled off discontinuity quite nicely, so there are times when it is acceptable to break this bar if it serves gameplay, but even in the midst of the discontinuity, there were still consistent elements to bridge the game together: the power and health icons, the base form of the censors, the luggage tags and luggage shapes, etc.

"Audio Bar": Does the word provide a compelling soundscape? Most of "Diablo II" had a very compelling soundscape...until you had a horde of those annoying little voodoo tiki dolls running around.

This is not an all-inclusive list. In fact, your project may have its own set of bars above and beyond this. If there's ever a question as to whether or not something meets the quality bar, sometimes it's helpful to step back and figure out which quality bar you're trying to meet first.

There may be times when an issue will face dueling bars. For example, you may find sound too low-fidelity in a section, but if you raise the audio fidelity, it causes framerates to drop to unacceptable levels.

Other times, you may find a bar dueling with itself. For example, you may find yourself torn between making a straight-forward solution to a puzzle so people won't get frustrated, or multiple less-obvious solutions so people will have a set of interesting choices.

So, what bars did I miss, besides the open bar? (Token joke for the marketing man who's been spamming the comments...)