February 11, 2006

Marketing & Testing :: Oil & Water? Yes and No...

Most people in testing have an inherent distrust of people in marketing. The job of testing is to find problems and report them to improve the product. The job of marketing is to hide problems and show the product in the best possible light.

Most testers dislike being pulled off of their testing duties to handle spurious marketing duties like creating screenshots and demonstration videos, and marketing tends to dislike having their feet held to the fire when QA shows that one or more of their promises will go unfulfilled.

In short, marketing and testing are destined to hate each other. However, if approached correctly, a testing department can pull a Scoble and really enhance the public's appreciation of a product. How is this possible? Simple. The two speak to two different markets. The marketing guys talk to the guys who believe marketing. The testers speak to the guys who don't.

It's like why you send a technician and a salesman on a sales call to meet with a manager and a tech. The salesman is there to close the sale. The technician is there to convince the other tech that it's a good idea.

So, how can you use QA to help with marketing? Follow these simple steps to success.

1. Make sure you have a winner. If your QA staff doesn't believe in the product, don't even bother using them for marketing. Signs that your QA staff may think they're working on a stinker: significantly higher than average sick days taken, repeated requests to transfer to different projects, claw marks on the sides of cubicles, multiple test leads in the span of a year quitting rather than continue to work on the project, etc.

2. Wait until alpha/code complete. This does two things. First, it gives your QA staff a chance to find out what's "cool" in the product. Second, it helps reduce the likelihood that a feature a QA guy is going to rave about is going to get cut.

3. Let them post about things that have been released publicly. You may not realize it, but your QA staff is most likely already regular visitors to message boards/forums/newsgroups/blogs that have spoken about your product. QA tends to use these resources to find out user concerns, get motivation for use cases and test cases, and for a bit of an ego boost to know that people are interested in what they are working on. Just ask them to limit their comments to positive comments about features that they like. Most testers browse these resources while they wait for builds to prop or for automated tests to run, so the time is already there. Why not harness it?

For an example, while I'm really excited about the dynamic difficulty system built into "SiN: Episodes," I'm more excited that the environments are believable. When I'm walking around the world, everything just seems to belong, and it's been a long time since I've played a first-person shooter like that.

4. Don't panic and don't overreact. On occasion, a bit too much information may get let out. Maybe it's about a feature that is related to a feature made public. Maybe it's something that hasn't been announced yet. In cases like that, don't panic. However, do speak with the tester who released the information and remind them that it isn't public yet.

For example, I recently posted about a feature that I thought was public. I saw one of my testers grab footage of the feature, saw the footage in the CES video, so I thought it was public knowledge. Turned out that I missed that the video cut away just prior to the footage that showed the feature I posted about. It was caught by our marketing man, pointed out to me, and I won't repeat that mistake again.

However, unless the leak is something extremely major, don't overreact. Overreacting will have two consequences. First, it's going to potentially cost you the services of a great tester. Second, it's going to reduce the likelihood that your test staff will continue to participate in a program like this.

5. Don't ask your testers to lie, "stretch the truth," etc. As I've said before, the most important characteristic of a tester is integrity. If you ask a tester to lie for you, one of two things will happen.

a) The tester will decline.
b) The tester will agree, but get caught.

The moment that one of your QA guys get caught in a lie in public, the game is over. Everything else that your QA guys have posted will be considered suspect.

The key for marketing is to "stay on message." The key for QA is to "tell the truth." When the two coincide, magic can happen.

No comments: