How To Ship Your Product On-Time

Over the last several years, I have worked on over two dozen products, and I've learned a lot of different lessons about them. That being said, there is one lesson from my gaming days that rings true across everything.

If you want to ship on-time, use a structured text file (like XML) for everything that makes sense until the last minute, and don't mix binary and text values in the same file. For games, that means save games, option files, map data, everything except for your image data. For utility software, anything you save out aside from images. I'm going to give you a few different examples of how following this advice would have saved us time and effort back at MGS, and one area where the change did help quite significantly.

First off, how many times during the development of your product has the file format changed? For "Links 2001," our binary file format changed over a dozen times. Each time, it was to either reorder the data or add an additional field. Each time, parts of the team were essentially down for anywhere from two to five days while we waited for the conversion to be handled.

All of the main data files for the "Links" series are zlib archives with encrypted directories. The archives for the courses contain textures unique to the course as well as the full geometry data for the course. All that was changing was the format for the geometry data. Had the geometry data been stored in XML format, it would have been excessively easy to make the conversion happen either at load time in the course designer or during the next save. When adding a new field, simply add a sensible default on load and save out the current value on save. When reordering, just change the order that you save the items. As long as your loading code can understand XML, your loading code will not change aside from adding support for the new fields.

The save game files for the "Links" series were all binary reads directly into a structure. However, this led to some cheats. For example, in "Links 2001" you could customize your club distances to match your real game. However, some people figured out that you could hex edit those values in the save game so they would be outside of the legitimate range, then take that player online. Had we used XML as a base (even if we encrypted it prior to saving), we could have imposed a range on the input values and refused to load a save game that did not validate. The same thing was happening with our clubs. You are limited to a maximum of 13 clubs in your bag, but I was able to give a golfer all of the clubs using the same technique. Same fix as well. Our lead developer spent two days adding the validation code for "Links 2003."

The "Amped" series used a highly customized version of the scripting language Lua internally. In fact, all of the save game files are Lua scripts. All Xbox save games are required to be digitally signed. This was a bit of a problem for the first "Amped." Text files are not a certain size, and memory was at a premium. The digital signatures were not the same size, and neither were the files, so how could you tell which was which?

For "Amped 2," the development team had several different Lua files. Their solution was to store the digital signatures in a seperate file. Each Lua script would be signed, and each signature would go to a dedicated space in the signature file. If the signature file was changed, the scripts wouldn't verify, and if the scripts were changed, the signature file wouldn't verify. All was good with the world. The signature code was added quite late in the cycle, but the save game code didn't change as a result of it. Because of that, we only lost a couple of hours due to a single bug in the validation code.

Now I can hear two objections to this: the first being load times will increase, the second that disk space requirements will increase. Well, given the number of security holes that I've seen in games in this industry, I see the tradeoff as acceptable. However, there are times where load time is of the essence.

The biggest speedup will come from compressing the file. While you may think that decompression is going to be slower, I've learned that disk access is slower by several orders of magnitudes, especially from CD or DVD. Plus, XML compresses quite nicely. So, squish the file before writing it to disk and decompress it on read.