The One Way To Save By Testing Late

Most of you know my views on when you should be testing. You might say I've experienced a bit of grief over my views, but they're fairly well founded and backed by documented evidence. However, there is an exception to the rule...there is in fact one case where a company can save money by not testing from day one. For most companies it won't be a significant savings, and the savings are beginning to vanish, but it does exist.

The one circumstance where you save money by testing late is if you cancel it during the prototype phase. If your company is smart enough to spend some time and money by letting a small team prototype out their game or application prior to doing any real development, and if your company does the right thing by not allowing any code or assets from the prototype to make it into the production tree, then you can save a lot of money not just on testing, but on an entire team. Once the prototype is "green lit," that's when you enter pre-production and should add at least the test lead.

Sadly, many companies have scaled back on prototyping. They're going straight from licensing to full production with little to no time spent making sure that the game is fun first, and as a result we're getting nothing but carbon copies of franchises built on the buggy codebases of yesteryear.

In the past, many game publishers would cancel nine out of every ten titles they worked on before the first whiff of them ever got out to the public, and that's the way it should have been. It's cheaper to spend $100,000 and cancel it once you know it's not fun than it is to spend $2,000,000 on the game and another $13,000,000 on production and marketing just to find out the same thing.

Now some of you may have noticed the caveat of not allowing anything from the prototype into the production tree. That was intentional. A prototype is essentially a house of cards...a facade showing the potential of the final product. Most are built as such. Memory bounds checking is never to be found, no consideration is made as far as making it run on multiple machines or platforms, the restrictions of the platform aren't taken into consideration, etc. It's not a very stable foundation to build a winning product on top of. The point of prototyping is to find what works and take the lessons learned while making the prototype to heart while making the final product, not to short circuit the preproduction process.

So the question you need to ask yourself now is this: Does your company cancel enough products to make it cost-effective not to test from the beginning?