Okay, our department got off to a shaky start, but Kevin H. made a really good decision. He decided that several of us should go to Redmond to train. Then, we would come back and train the rest of the department.
Michael B. needed to stick around and take care of patch duties on LS 2000, so Kevin picked two test leads and one tester to send up. Myself and Nick would represent the test leads, and Seth B. would represent the tester contingent. Kevin also joined us in Redmond.
We were sent up for six weeks. We were able to come back every other weekend, but for the most part, we were Redmond-bound for the six weeks.
When we got up there, we were introduced to the test team, and then we were each assigned to our products for training. Nick was assigned to "Crimson Skies." I don't remember what Kevin was assigned to. Seth went to learn about multiplayer testing, and was assigned to the multiplayer lab. I was assigned to "Mechwarrior 3: Pirate's Moon."
"Wait a second," some of you may cry. "'Mechwarrior 3: Pirate's Moon' was a Microprose product. You worked for Microsoft. Why would you be testing a Microprose product? You're full of shit."
About a month before we found out that Access had been purchased, Microsoft bought FASA Interactive, and along with it, the rights to interactive products based off of the FASA properties. Hasbro and Microprose had licensed the MechWarrior license prior to the acquisition, though, so this was a contractually necessary product.
I was assigned to the product as a regular tester, working under James M., an experienced test lead. The product was supposed to be released to manufacturing two weeks after I got there. The only way we could stop the product on our end is if the product, as released, would damage the value of the MechWarrior license.
Now to explain how our communication system with Microprose worked. The product manager at Microprose would send a copy of their bug database (stored in an Excel spreadsheet) to us. We would test, and enter our bugs into RAID, Microsoft's internal bug database software. Each night, James M. would send our bugs (full text and everything) to the product manager at Microprose. Then, the product manager would take the bugs and forward them to Zipper Interactive, the studio developing the software. Zipper was just a couple of miles away from Redwest-E, where we were stationed. So, while inefficient, especially since Crimson Skies was also in development at Zipper, it was what was requested by Microprose, so we capitulated.
Over the two weeks, we received a new build every other night. It was getting slowly better, but most of the major showstopping bugs were not being fixed. At the end of the two weeks, we received a gold master candidate. To say I was underwhelmed was an understatement. We were not able to complete a single multiplayer game, and could only connect less than 10% of the time. We could not complete a single level of the single player game. And most of the sound effects were placeholder sounds...literally, someone saying, "Placeholder A-23," or something similar.
I shared my results with James, and he agreed with me that the game should not ship as is, but unlike most Microsoft products where test can stop the product from shipping, this was not a Microsoft product. James took it to his manager, who took it up higher and higher, until it was decided that the product should not ship as it was.
While that discussion was going on, I was sharing an office with Nick, who was testing "Crimson Skies." He also had placeholder sound effects, and he was intensely amused by them. His were not ID codes. His were placeholder dialog recorded by the developer. His favorite line occured during a mission where he had to fly his plane next to a train to pick someone up. The little man would climb up into the plane, and he'd hear this disco-Stu'ish voice state, "I'm in, let's boog-ay!" For the rest of my time at Microsoft, I would hear Nick say that at the most inappropriate times. I loved it. So did he.
James came and told me that the decision was made to halt the product. Suddenly, the tone from Microprose changed. They wanted us to work with Zipper to get the product shipped. Turns out that Zipper hadn't been getting all of our bug reports. So, we had Seth working on the product in the multiplayer lab, me focusing primarily on the single player, and James handling coordination duties, communication and testing. "M3:PM" was supposed to be on store shelves by Thanksgiving weekend, or else one of the major Christmas sales windows would be missed.
It took and additional three weeks of twelve and sixteen hour days to stabilize the product and get the product to the stability bar we had set for it. As a result, the game shipped to stores one week after Thanksgiving 1999, arriving between November 30 and December 3.
As an indirect result of the slipped ship date, Microprose closed the office that we were working with that Christmas. Every once in awhile, I still feel guilty about it, knowing that I had a part in people losing their jobs in a competitive industry like this. But then I remember that the delay helped turn "M3:PM" from something that was rivalling "Extreme Paintbrawl" for lameness to something that got an average 78% rating according to GameRankings. The customer won in this case. It was the proper decision.
We all came away from Redmond with a new appreciation for what video game test departments go through. We also all felt invigorated by what we felt was the power that Microsoft gives testers. While our department got off to a shaky start, this trip and the peer-training we did when we got back helped solidify our department's foundation, and give our department an excellent second chance.
However, while we went to Redmond for training, none of the developers did. Empowered testers vs. a culture where testing didn't matter? It was the groundwork for war during a critical period at MGS SLC, but that's a story for another time.
No comments:
Post a Comment