Preprocessing / Optimization / Re compression For Better Compression of Games

[Joe Forster/STA]

Imagine that you're the developer of game X. You've already released version 1.0 (CD/DVD, Steam etc.) but found some bugs since (wow!). You fixed (some of) them, calling it version 1.1, but you can't re-release the whole game; you rather collect the changes into a patch. This patch can be a package of the full files that have been changed between version 1.0 and 1.1 but that can be really big if it contains many files and/or some of the files are big.

A smarter idea is collecting the actual differences between the files. (Obviously, if those files exist in both versions.) Say, you have a big data file that has an internal structure and it contains many little files. (E.g. as simple as a ZIP archive.) If you add one internal file then only that part of the big data file changes, the rest stays the same. If you fix some code and recompile the program file (executable) then, again, most of the file stays the same, only some parts of it (added code, added data, modified jump offsets, modified data pointers/references) will change.

Xdelta and .RTPatch are programs that can do a binary comparison between a bunch of "old" files (those of version 1.0) and "new" files (version 1.1) and create a patch. If your customer downloads the patch and has the "old" files, xdelta can use ("apply") the patch to convert the "old" files into the "new" ones.

It doesn't matter if the "old" and "new" files are big, if the actual differences are small then the patch will be small, too. You can create a single patch file out of the differences of multiple files, including creating a new file (that didn't exist in version 1.0 yet), deleting a file (that doesn't exist in version 1.1 anymore) or modifying a file. I'm not sure about xdelta but with .RTPatch you can also create multi-version patches (version 1.0 or 1.1 or 1.2 -> 1.3) that can modify many versions of "old" files into the "new" ones.

I've seen xdelta used in the game called "The Thing" (from 2002 or something). Most developers use .RTPatch because it probably has more features but it's extremely expensive (thousands of dollars) and closed source, while xdelta is free and open source.