Best Compression Methods for 'Specific' Games. Q&A

[JustFun]

Hitman: Contracts

Selected ARC/DS method for Data1a-01.bin was: 
xtool:c1792mb:mbik+xtool:mpreflate+msc_tak+srep_new:490g+lolz
------------------------------------------------------------------------------------------
Overall input size: 859,45 MB
Compressed game size: 580,22 MB (Ratio 67.51%)

[JustFun]

Hitman 2: Silent Assassin

Selected ARC/DS method for Data1a-01.bin was: 
xtool:c1792mb:mbik+xtool:mpreflate+msc_tak+srep_new:490g+lolz
------------------------------------------------------------------------------------------
Overall input size: 492,81 MB
Compressed game size: 182,00 MB (Ratio 36.93%)

[JustFun]

The Saboteur - (GOG Version, English Only)

Selected ARC/DS method for Data1a-01.bin was: 
xtool:c1792mb:mbik+xtool:mpreflate+srep_new:490g+lolz
------------------------------------------------------------------------------------------
Overall input size: 5,86 GB
Compressed game size: 2,32 GB (Ratio 39.56%)

[trillestdwane]

I am having a bit of trouble trying to compress this game. When I use the Cry Engine plugin (specifically for Crysis 3) i end up getting a "runtime error 216" message box popping up at 10.2%, then

ERROR: general (de)compression error in xtool:c32mb:mc 10.2%

I still get the same error when I only do it on the games .pak files.

[wareck]

I use DiskspanGUI, and I done this :
-go in ADDON diskspangu folder, unzip the special oodle library oo2core_7_win64.dll and oo2core_8_win64.dll in the oodle library folder root

-in the game folder, removed language I don"t use (only big .pak files)

Data1:
Name Data3.bin
xtool:mkraken:7:mmermaid:8:mreflate+srep_new:490g+ LOLZ_ULTRA => NOTICE
(only one file, it's just to let diskspangui to copy dll in xtool folder and use them for the next step)

Data2
Name Data1.bin
xtool:mkraken,t400:mreflate+srep_new:490g+LOLZ_NOR MAL =>*.pak

Data3
Name Data2.bin
xtool:mkraken:mreflate+srep_new:490g+LOLZ_NORMAL => all other files

Results is around 23GB

If you wants to keep languages for future update:
Data4
xtool:mkraken,t400:mzlib:mreflate:mpng:d1+srep_new :490g+LOLZ_NORMAL and add only files (ar.pak , de.pak ...) and do not forget to disable them in Data1.bin
You also can add task and several data to make installer with multiple choice
for language
Language pack is around 15.6GB compressed

[JustFun]

Call of Juarez - (Steam Version)

Selected ARC/DS method for Data1a-01.bin was: 
xtool:mpreflate+oggre+srep_new:490g+lolz
------------------------------------------------------------------------------------------
Overall input size: 2,28 GB
Compressed game size: 1,09 GB (Ratio 47.92%)

[L33THAK0R]

Scene Release - CPY

Original Size: 2.82 GiB
Final Size: 1.80 GiB (Incl. installer) (1,940,176,780 bytes)
Ripped Files: None.
Tools used:

Xtool v0.8.9 (Codecs: reflate, zlib)
Srep
Lolz
Oggre

Additional notes:

"GzsTool.exe" & "sdp_tool.exe", and WEM processing tools are attached below.

"data_01.g0s" & "data_02.g0s" contain majority of assets, "data_00.g0s" just contains a FMV cut-scene. Archives are XOR-ed, in layman's terms a simple bit-shifting operation is used to encrypt the archive contents.

My solution isn't ideal (data for each relevant archive is written to the disk twice), however I couldn't find a better approach. It should also be noted this isn't a CRC-perfect packing process but it is deterministic from my testing, ensure no additional files are within the target directory for packing, nor is it missing any original files. "GzsTool.exe" is used to unpack the archives, and then repack them post-install. Within these archives are a variety of file-types, notably ".wem" and ".sdp". The former file-type is self explanatory, and can be processed (for those who are unsure) with the following chain

ww2ogg.exe "XXX.wem" --pcb "packed_codebooks_aoTuV_603.bin" -o "XXX.tmp"
oggre_enc.exe "XXX.tmp" "XXX.packed"
oggre_dec.exe "XXX.packed" "XXX.unpacked"
hdiffz -c-lzma2-9 -C-no "XXX.unpacked" "XXX.wem" "XXX.patch"

To eliminate ".wem" files that are inflated by packing, I run a basic test/filter for file diff. that I wrote in python, as I've found the <100KiB rule that I've been using for a good while doesn't always hold true.

".sdp" files are just ".bnk" files with an additional header, they can be extracted quickly using "sbp_tool.exe", there you can isolate the wem files, process them and make a patch. Processing ".sdp" archives only nets around ~30MiB (rough estimate, actual figure is under ~100MiB but I've forgotten the exact number).

I've been trying to make any additional gains to at least match the best publicly available size (CorePack's release coming in at 1.73GiB), but I'm unable to determine where further gains can be made. I do have their "Phantom Pain" release and can confirm for that title they use a similar unpacking/repacking method, however I'm unable to source their "Ground Zeroes" release for further analysis. I feel like I'm missing something glaringly obvious but crawling through community documentation has netted nothing that jumps out at me. 2nd place is a nice consolation prize but it would be nice to at least know what they did for a 70MiB reduction.

I'm currently finalisng my "Metal Gear Solid V: The Phantom Pain" approach, and will publish the results soon.

[JustFun]

Dracula: Origin (Steam Version)

Selected ARC/DS method for Data1a-01.bin was: 
xtool:c1792mb:mbik+xtool:mpreflate+oggre+srep_new:490g+lolz
------------------------------------------------------------------------------------------
Overall input size: 1,90 GB
Overall output size: 1,36 GB (Ratio 71.38%)

[JustFun]

NARC (2005) (Retail)

Selected ARC/DS method for Data1a-01.bin was: 
xtool:c1792mb:mbik+xtool:mpreflate+srep_new:490g+lolz
------------------------------------------------------------------------------------------
Overall input size: 1,74 GB
Overall output size: 1,17 GB (Ratio 67.41%)

[JustFun]

Manhunt

Selected ARC/DS method for Data1a-01.bin was: 
xtool:c1792mb:mbik+xtool:mpreflate+srep_new:490g+lolz
------------------------------------------------------------------------------------------
Overall input size: 1,89 GB
Overall output size: 1,21 GB (Ratio 64.17%)

[KaktoR]

Originally Posted by L33THAK0R

Scene Release - CPY

Original Size: 2.82 GiB
Final Size: 1.80 GiB (Incl. installer) (1,940,176,780 bytes)
Ripped Files: None.
Tools used:

Xtool v0.8.9 (Codecs: reflate, zlib)
Srep
Lolz
Oggre

Additional notes:

"GzsTool.exe" & "sdp_tool.exe", and WEM processing tools are attached below.

"data_01.g0s" & "data_02.g0s" contain majority of assets, "data_00.g0s" just contains a FMV cut-scene. Archives are XOR-ed, in layman's terms a simple bit-shifting operation is used to encrypt the archive contents.

My solution isn't ideal (data for each relevant archive is written to the disk twice), however I couldn't find a better approach. It should also be noted this isn't a CRC-perfect packing process but it is deterministic from my testing, ensure no additional files are within the target directory for packing, nor is it missing any original files. "GzsTool.exe" is used to unpack the archives, and then repack them post-install. Within these archives are a variety of file-types, notably ".wem" and ".sdp". The former file-type is self explanatory, and can be processed (for those who are unsure) with the following chain

ww2ogg.exe "XXX.wem" --pcb "packed_codebooks_aoTuV_603.bin" -o "XXX.tmp"
oggre_enc.exe "XXX.tmp" "XXX.packed"
oggre_dec.exe "XXX.packed" "XXX.unpacked"
hdiffz -c-lzma2-9 -C-no "XXX.unpacked" "XXX.wem" "XXX.patch"

To eliminate ".wem" files that are inflated by packing, I run a basic test/filter for file diff. that I wrote in python, as I've found the <100KiB rule that I've been using for a good while doesn't always hold true.

".sdp" files are just ".bnk" files with an additional header, they can be extracted quickly using "sbp_tool.exe", there you can isolate the wem files, process them and make a patch. Processing ".sdp" archives only nets around ~30MiB (rough estimate, actual figure is under ~100MiB but I've forgotten the exact number).

I've been trying to make any additional gains to at least match the best publicly available size (CorePack's release coming in at 1.73GiB), but I'm unable to determine where further gains can be made. I do have their "Phantom Pain" release and can confirm for that title they use a similar unpacking/repacking method, however I'm unable to source their "Ground Zeroes" release for further analysis. I feel like I'm missing something glaringly obvious but crawling through community documentation has netted nothing that jumps out at me. 2nd place is a nice consolation prize but it would be nice to at least know what they did for a 70MiB reduction.

I'm currently finalisng my "Metal Gear Solid V: The Phantom Pain" approach, and will publish the results soon.

Looks like you cannot use the sbp tool in a for loop with /r option sadly.

[st0rm]

Game Version : v2024.06.21

Ripped - Re-encoded Files : Nothing

Original Size : 2.68 GB

Compressed Size : 476 MB

Install Time : 2 Mins on HDD (Depends your system)

Method : srep:m3f+lzma2

[L33THAK0R]

Originally Posted by KaktoR

Looks like you cannot use the sbp tool in a for loop with /r option sadly.

Yeah it's a touch annoying, I just generated a checksum for all the loose files and then isolated the .sbp entries and ran as a batch script. Thankfully it's only needed to extract the initial WEMs, and a hdiffz patch takes care of the rest post-install.

[L33THAK0R]

Scene Release - CODEX

Original Size: 22.9 GiB
Final Size: 10.8 GiB (Incl. installer)
Ripped Files: Online MP Assets
Tools used:

Xtool v0.8.9 (Codecs: reflate, zlib)
Srep
Lolz
Oggre

Additional notes:

Used very similar processing methods to my "Ground Zeroes" results, see here for more details

Used a different fork of "GzsTool", this same version is attached at the end of this post.

There's around ~100MiB that can be gained, however, like with "Ground Zeroes" I'm unable to determine where these gains can be found.

[JustFun]

Sherlock Holmes: The Mystery of the Persian Carpet (Steam Version)

Selected ARC/DS method for Data1a-01.bin was: 
xtool:mpreflate+oggre+srep_new:490g+lolz
------------------------------------------------------------------------------------------
Overall input size: 93,70 MB
Overall output size: 81,07 MB (Ratio 86.53%)