Game QA Bill of Rights

Tomorrow, I step back into "the real world" for a time to rebuild my confidence in the games industry, restore some sanity, and refill my savings. However, I cannot leave my people hanging. I've dedicated the majority of my adult life to quality assurance. I've worked with teams that I would consider among the best in the industry. I've worked with others that I wouldn't trust to test if sponges could pick up water. But regardless of the department, there were many things in play that reduced not only the effectiveness of test, but the way that test interacted with the remainder of the department.

If quality is to improve in the products that the games industry creates, the way that we treat those responsible for maintaining the quality bar must improve as well. With that in mind, I present for your consideration "The Games Quality Assurance Bill of Rights."

1. You have the right to work no more than ten hours per day.
As a tester, your primary tools of the trade are your eyes and ears. Your powers of observation are crucial to your success as a tester. Studies have shown that after six hours, a person's ability to be observant begins to decline. In practice, any tester starts being more of a hindrance than a help after about ten hours of testing. Experience has shown that each hour spent by a tester after ten hours adds an additional 2.5 hours to the overall testing load to account for double-checking questionable and bad bugs and having to retest the area that was tested.

2. You have the right to at least one full day off per week.
Everyone needs at least one day off per week, be it to rest, attend religious services, do your laundry, pay bills, etc. Just because you're testing doesn't mean that your life outside of work stops. Your family misses you, your friends miss you...you need "you" time. Take it. It will help you recharge so you'll be more effective when you come back in.

3. You have the right to advance notice of expected overtime.
Testers need to have a life outside of work. You work to live, not live to work. However, there are times when extra hours are needed, be it for milestone testing or the final push to get a product out the door. I always tried to provide at least 48 hours notice of any extra hours coming up, and if I wasn't able to give that notice and the hours were required, I excused the extra hours for testers who had plans already.

4. You have the right to information about the game you are testing.
If we don't have information about what we're testing, if we're expected to test based on "feel" or "gut," we may as well just be monkeys with controllers. Testing isn't just throwing random input at a game until something breaks...it's a directed effort into the innards of the game trying to find the flaws that our customers are going to encounter so that they can be fixed. In order to have that directed effort, we need information. We need maps, behavior descriptions, system descriptions for systems that affect the user experience but are not exposed via UI elements, interaction charts...we need to know how the game is supposed to work so that when we are in the game, we'll know if something isn't working right.

5. You have the right to access your development team.
In this industry, we have a horrible habit of segregating test from the rest of the company. Builds aren't delivered, they're "thrown over the wall." There are times when text is not the ideal way to describe a bug. There are times when a tester should be able to go to a developer and say, "I've got this odd behavior...does this seem like a bug to you?" There are times when a tester should be able to go to a developer and ask, "I'm not sure how this feature is supposed to work...can you describe it for me?" Some people think that interaction like this is a waste of the programmer's time, but ask yourself this...would you rather spend two minutes talking to the person testing your feature, or a week handling "bad" bugs? Would you rather spend an hour talking to the tester who is going to be working on breaking your system and work out most of the major kinks before the system is even written, or four months rewriting your system after a major flaw is discovered by test?

6. You have the right to be evaluated on more than just your bug count.
A lot of QA managers evaluate their testers on bug count alone, but that doesn't paint an accurate picture of your performance as a tester, and can actually lead to an increase in counter-productive activities like bug padding, bug repetition, and skipping verifying repro steps in an effort to get the bug in faster. Activities such as test casing, bug triages, meetings with developers, and buddy testing reduce the amount of time that testers can use to find bugs, but still positively affect the quality of the project. In addition, not all bugs are created equal. Is one crash bug the same as a missing period in a dialog box? Also, not all areas will be as buggy as others. I used bug counts as an indicator of a potential problem, but only as an indicator that I need to dig deeper to see if there really is a problem.

7. You have the right to know why it was decided that your bugs would not be fixed.
In most companies, there is a triage process during the endgame where bugs "live or die." There may be times when a bug is just deemed too small to fix. Other times, fixing a bug may be seen as too risky because of the likelihood of other bugs being introduced by the fix. Other times, bugs may be offered up as trade in an effort to get other bugs fixed. Regardless of the decision, testers deserve the right to know why their bugs were waived. It's demoralizing to have a bug come back "won't fix," but having an understanding as to the process that led to that decision often helps.

8. You have the right to appeal a decision made about your bugs.
There have been many times when a bug was waived because it wasn't seen to meet a certain level of severity, even though the actual bug itself was significantly worse than people thought. Either the subject line of the bug didn't make it seem harsh enough, or the description came across as soft, or just the bug wasn't properly understood. There have been many times when a tester's appeal has caused a bug to be seen in a new light, and led to the bug being fixed, resulting in a higher-quality product.

9. You have the right to have your department decide how a feature should be tested.
Many people have a low opinion of testers. They believe that people with limited to no training and/or extremely low wages in comparison to themselves couldn't possibly understand the complications and/or ramifications of a system. What happens on occasion is that the means of how something should be tested will be passed down to the test manager or test lead as an "edict from on high." The main problem with this scenario is that the person deciding how the testing should be done has no experience with testing. It could be a coder who wants "the correct codepath" tested. It could be a middle-manager who wants to feel important. It could be someone who is trying to reduce the cost of testing a feature. Regardless of who, it is the right of the test department to decide how the feature should be tested. There is nothing wrong with taking the suggestion under advisement and use it as a guide towards one way of testing a feature, but if you are going to be held responsible for successfully testing a feature, you should at least be able to determine your own plan.

10. You have the right to be treated like a human being, both inside and outside your department.
Testers have to deal with the worst of all worlds. Their jobs are often temporary, low-paying and offer no benefits. As part of their jobs, they are forced to be the bearer of bad news. Their department is often the first to be blamed for any defect with any product, even if the issue was discovered and reported during testing. Starting from that, the job is already on a bit of a down note. It gets worse when you receive condescending comments from other departments, like "You're just a bunch of keyboard monkeys," or, "This system is so complex, I don't think you'd be able to understand it." There have been times when meals were provided for all employees working twelve or more hours except for the temp testers. (All of these examples are unfortunately real.) It can be just as bad between full-time testers and temporary testers. You are not your career and your worth is not defined by your profession. You are a human being, and you should be treated like one, whether you're full-time, contract on location, or outsourced in Shanghai.

Remember, we are all test, as bugs affect us all.