November 15, 2004

Classifications of Testers

I've been in Quality Assurance for a long time, and after a few years, you start seeing several different classifications of testers. While several of the classifications that I'm going to share with you may seem tongue-in-cheek, there is quite a bit of truth to the stereotypes. As a heads-up, a good test department usually has representatives from most of these classes. In addition, most testers are combinations of two or more of these.

KEYBOARD MONKEY. One of the most derogatory testing terms is actually a good fit for our first classification. The Keyboard Monkey takes a list of test cases and goes through them one by one. The upside is that since the test cases are pre-created, development usually has a nice set of repro steps. The downside is that a Keyboard Monkey will never look past their defined test case suite.

SADIST. This is what a Keyboard Monkey turns into the moment they look past their test cases. You can tell that you're dealing with a sadist when you hear, "I wonder what would happen if I did this...BOOM! Ahahahahah!" No test department is complete without a few sadists. While most sadists are great at getting bugs to trigger, the repro steps are usually a little lacking.

FUGU CHEF. Fugu is the name of a poisonous blowfish that, if not prepared for cooking correctly, can kill you. Fugu chef testers take their feature area and gradually dissect them until they find the "poison." While extremely thorough in their area, fugu chef testers are usually oblivious to how their feature interconnects with other features.

CARPET BOMBER. These guys will indiscriminately bug EVERYTHING as a seperate bug, even if all of the bugs they enter are the same. If none of your menu items have keyboard shortcuts, they will bug each menu item seperately. If none of your images in your help file appear, they will bug each missing image seperately. Carpet bombers will inadvertently inflate the find, fix and resolve rates for entire projects by not realizing that the bugs are related. Example: On "Links 2003," we had a tester file over 60 bugs against images not appearing in the help file. A single character fix resolved all the bugs.

CSI. When the CSI tester finds a bug, all other work for that tester stops. The CSI examines the bug from all angles, and will not only report the bug, but why it happened, explicit repro steps, which check-in(s) led to the bug, and a psychological analysis of the developer who checked in the bug. While this type of tester is usually the least productive from a numbers standpoint, they are invaluable assets to a team. Also called the "Grissom."

L33T HAXX0R. The L33t Haxx0r tester wants it to be known that he can cause your program to crash whether you want it to or not. He'll modify your save files using a hex editor to bring out bugs in your load routines. He'll man-in-the-middle attack your network streams to bring out bugs in your network code. While these guys are valuable, they're also dangerous if you decide not to fix their bugs. Chances are that their exploit code will leak after release because if you don't find their input valid, they'll find someone who does.

EDITOR. These guys will bug any typo on any screen that they can find. They'll be so focused on entering a bug that you have four periods in an ellipsis instead of the standard three that they won't notice the major crash that occurs after you click any button on that screen.

OKIE. The Okie tester is so in-tune with the users that he finds all of the bugs that the average user would normally find. You can tell an Okie bug because it's a bug that no tester in his/her right mind would ever think of testing for, but as soon as you read it, you realize that every single user will encounter this bug. You don't find many Okie testers, so do whatever it takes to hold on to the Okie testers you find.

SAVANT. The savant tester doesn't test. He simply sits down at a testing station and bugs manifest themselves in places you never would have expected. The savant will find bugs, but unfortunately, that's all they're ever good at. Never let a savant interact with a person outside of the test department.

WINNER. The "winner" tester is a tester type exclusive to games testing. The winner doesn't care about the quality of the product. He only cares about winning. A brief anecdote: Microsoft Game Studios has a team of dedicated gamers that they hire called the Recon team. The Recon team had been helping out with playbalancing "Age of Empires II." However, for over a month, one of the testers had cheats enabled. As a result, the month's worth of playbalance data generated from the Recon team had to be thrown out.

If you can think of other classifications of testers, please post them in a reply. I may do a second set if I get enough follow-ups.

No comments: