FileForums - Best Compression Methods for 'Specific' Games. Q&A

FileForums (https://fileforums.com/index.php)

- Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)

- - Best Compression Methods for 'Specific' Games. Q&A (https://fileforums.com/showthread.php?t=99554)

Quote:

36.3 GB -> 103 GB -> 91 GB -> 32.26 GB

Considering the almost three-hour installation - such an occupation

Yes, it's not worth it at all tbh.

SIFU
v1.12.4.477 (31.08.2022)

Code:

 09:39:20 - Selected ARC/DS method for Data1a-01.bin was: xtool:c512mb:mue4,k0x40A266F41FDBCE91312FBB86060D2E9425B7D922C0CF0031F634CAD9AECB49DA+srep_new:1+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000

-------------------------------------------------------------------------------------------------------------------------------------------

 10:14:56 - Overall input size: 21.08 GB

 10:14:56 - Overall output size: 8.27 GB (Ratio 39.26%)

 10:14:56 - Overall conversion time: 00:35:29

Code:

Extracted 366 files, 8,884,582,619 => 22,632,539,812 bytes. Ratio 39.26%

Extraction time: cpu 824.38 sec/real 109.09 sec = 756%. Speed 207.46 mB/s

All OK

Middle-Earth: Shadow of Mordor
GOG version

Code:

 16:59:22 - Selected ARC/DS method for Data1a-01.bin was: bpk+srep_new:1+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000

 16:59:22 - Selected ARC/DS method for Data1b-01.bin was: xtool:mzlib+srep_new:1+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000

-------------------------------------------------------------------------------------------------------------------------------------------

 18:38:03 - Overall input size: 53.70 GB

 18:38:03 - Overall output size: 23.01 GB (Ratio 42.85%)

 18:38:03 - Overall conversion time: 01:38:39

Code:

-------------------------------------------------------------------------------------------------------------------------------------------

 Extracting Data1a-01.bin file, Please Wait...

-------------------------------------------------------------------------------------------------------------------------------------------

FreeArc 0.67 (March 15 2014) extracting archive: Conversion_Output\CONVERSION\Data1a-01.bin

Extracted 338 files, 10,656,657,345 => 12,229,724,967 bytes. Ratio 87.14%

Extraction time: cpu 908.11 sec/real 531.97 sec = 171%. Speed 22.99 mB/s

All OK



-------------------------------------------------------------------------------------------------------------------------------------------

 Extracting Data1b-01.bin file, Please Wait...

-------------------------------------------------------------------------------------------------------------------------------------------

FreeArc 0.67 (March 15 2014) extracting archive: Conversion_Output\CONVERSION\Data1b-01.bin

Extracting 66 files, 45,435,263,268 bytes. Processed  99.9%

Extracted 66 files, 14,051,170,767 => 45,435,263,268 bytes. Ratio 30.93%

Extraction time: cpu 815.78 sec/real 646.88 sec = 126%. Speed 70.24 mB/s

All OK

Halo Infinite
Build 9347748, incl. Campaign, no HD pack

Code:

 20:53:02 - Selected ARC/DS method for Data1a-01.bin was: oo2reck:8+srep_new:1+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000

 20:53:02 - Selected ARC/DS method for Data1b-01.bin was: xtool:mzlib+srep_new:1+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000

 20:53:02 - Selected ARC/DS method for Data1c-01.bin was: srep_new:1+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000

-------------------------------------------------------------------------------------------------------------------------------------------

 23:32:27 - Overall input size: 42.36 GB

 23:32:27 - Overall output size: 29.04 GB (Ratio 68.57%)

 23:32:27 - Overall conversion time: 02:39:18



Data1a = *.module*

Data1b = package\*

Data1c = rest of the files

Code:

-------------------------------------------------------------------------------------------------------------------------------------------

 Extracting Data1a-01.bin file, Please Wait...

-------------------------------------------------------------------------------------------------------------------------------------------

FreeArc 0.67 (March 15 2014) extracting archive: Conversion_Output\CONVERSION\Data1a-01.bin

Extracting 117 files, 28,222,165,341 bytes. Processed  99.8%

Extracted 117 files, 20,616,611,986 => 28,222,165,341 bytes. Ratio 73.05%

Extraction time: cpu 1139.09 sec/real 500.03 sec = 228%. Speed 56.44 mB/s

All OK



-------------------------------------------------------------------------------------------------------------------------------------------

 Extracting Data1b-01.bin file, Please Wait...

-------------------------------------------------------------------------------------------------------------------------------------------

FreeArc 0.67 (March 15 2014) extracting archive: Conversion_Output\CONVERSION\Data1b-01.bin

Extracting 88,043 files, 4,710,860,569 bytes. Processed  99.8%

Extracted 88,043 files, 851,140,347 => 4,710,860,569 bytes. Ratio 18.07%

Extraction time: cpu 61.89 sec/real 149.54 sec = 41%. Speed 31.50 mB/s

All OK



-------------------------------------------------------------------------------------------------------------------------------------------

 Extracting Data1c-01.bin file, Please Wait...

-------------------------------------------------------------------------------------------------------------------------------------------

FreeArc 0.67 (March 15 2014) extracting archive: Conversion_Output\CONVERSION\Data1c-01.bin

Extracting 1,883 files, 12,549,104,127 bytes. Processed  99.9%

Extracted 1,883 files, 9,714,711,673 => 12,549,104,127 bytes. Ratio 77.41%

Extraction time: cpu 334.20 sec/real 220.25 sec = 152%. Speed 56.98 mB/s

All OK

Halo The Master Chief Collection
Build 9367265, no multiplayer, no extended language packs

Code:

 20:09:49 - Selected ARC/DS method for Data1a-01.bin was: bpk

 20:09:49 - Selected ARC/DS method for Data1b-01.bin was: xt_zlib+srep_new:1+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000

 20:09:49 - Selected ARC/DS method for Data1c-01.bin was: srep_new:1+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000

-----------------------------------------------------------------------------------------

 00:52:22 - Overall input size: 109.60 GB

 00:52:22 - Overall output size: 66.74 GB (Ratio 60.90%)

 00:52:22 - Overall conversion time: 04:42:31



Data1b contents:

halo1\prebuild\shaders_2_cache\prebuild\*.sdc

halo2\prebuild\shaders_2_cache\prebuild\*.sdc

halo2\preload\paks\*.pck

halo3\maps\*.map

halo3odst\maps\*.map

halo4\maps\*.map

haloreach\maps\*.map

mcc\Content\Paks\*.pak

data\ui\Screens\*.gfx

Quote:

Originally Posted by Masquerade (Post 497780)

You've answered your own question. FitGirl's repack is smaller because she uses additional tools outside of the FreeArc process to properly compress the game files.

For example:

- Game uses BINK videos that BPK will produce CRC imperfect files, so you must HDiffPatch them (definitely sure this will work)

So there you go, how to shrink your repack.

Hello, how can I use HdiffPatch on Cyberpunk 2077 bink files? I've searched into website, but no concrete example, no concrete instruction. Can anyone help me? Thank you!

Quote:

Originally Posted by Dragonis40 (Post 498183)

Hello, how can I use HdiffPatch on Cyberpunk 2077 bink files? I've searched into website, but no concrete example, no concrete instruction. Can anyone help me? Thank you!

https://www.fileforums.com/showpost....postcount=3101

Assassin's Creed Origins
v1.60, 1 language, all DLC

Code:

 13:10:48 - Selected ARC/DS method for Data1a-01.bin was: xtool:manvil:4+srep_new:1+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000

 13:10:48 - Selected ARC/DS method for Data1b-01.bin was: bpk+srep_new:1+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000

-------------------------------------------------------------------------------------------------------------------------------------------

 17:08:59 - Overall input size: 69.70 GB

 17:08:59 - Overall output size: 32.56 GB (Ratio 46.71%)

 17:08:59 - Overall conversion time: 03:57:17



Decompression time: 1 hour (6c 12t)

I still had a backup with oo2recm. This time I used xtool anvil plugin to compare the full size. The size is almost the same (oo2recm is ~50mb smaller), but xtool anvil plugin is much faster (~7hrs vs 4hrs). I leave it here just for information on full sizes instead of just samples.

PS: I will do the same for Odyssey and Valhalla soon.

Quote:

Originally Posted by Masquerade (Post 498184)

https://www.fileforums.com/showpost....postcount=3101

Thank you for your reply! I've used hdiffz.exe and, thank to your instructions, it works! What I don't understand is the pro of all of this. Let me better explain: I use hdiffz.exe to avoid CRC error when I decompress .bik and .bk2 files, but I have no advantages in terms of compression, since I have to use temp file and diff file to create the original .bik and .bk2 file.
I thought that I had to use compressed file (bpk file) to create the original one, but as far as I can see, this file is not used at all.
Is there anything i don't understand?

Of course you have to use compressed bink file + diff file to restore the original file.

bink_unpack.exe will fail on decompressing the compressed bink file, but file remains.

Then you use the diff file to patch the decompressed bink file back to original.

In the example above, you have to include BPK2 files and Bink2DIFF files.

Jojo's bizarre adventure: All-star battle r

Recently the title in question got an official PC-port, as it was originally a PS3 exclusive. The compression is, in theory fairly easy to decipher, as it's just Crilayla, however all the assets in their relevant ".cpk" archives are encrypted, the algorithm seems to be the same from a glance however some archives aren't original, so its not clear if a different or multiple private keys were used/replaced.

Attached is the only semi-working (QuickBMS) script I can find (source) that can sort of process some data but falls short very quickly. I'm not very adept at QuickBMS scripting but my guess is the encryption key is incorrect.

I have also attached a sample for anyone interested. For those curious current gains are only ~200MB with the encrypted archives.

Assassin's Creed Odyssey
v1.5.6, 1 language, all DLC

Code:

 20:41:34 - Selected ARC/DS method for Data1a-01.bin was: xtool:manvil:4+srep_new:1+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000

 20:41:34 - Selected ARC/DS method for Data1b-01.bin was: bpk+srep_new:1+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000

-----------------------------------------------------------------------------------------

 00:44:06 - Overall input size: 99.23 GB

 00:44:06 - Overall output size: 42.15 GB (Ratio 42.47%)

 00:44:06 - Overall conversion time: 04:02:25

oo2recm

Quote:

Originally Posted by L33THAK0R (Post 498204)

Attached is the only semi-working (QuickBMS) script I can find (source) that can sort of process some data but falls short very quickly. I'm not very adept at QuickBMS scripting but my guess is the encryption key is incorrect.

You can extract the encrypted data using CriPack Browser v6.7. As for decryption, it's clearly not an XOR implementation like what was seen in Super Robot Wars 30 as all the files are different.

Assassin's Creed Valhalla
v1.6.0, 1 language, all DLC

Code:

 11:02:57 - Selected ARC/DS method for Data1a-01.bin was: xtool:manvil:8+srep_new:1+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000

 11:02:57 - Selected ARC/DS method for Data1b-01.bin was: srep_new:1+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000

-------------------------------------------------------------------------------------------------------------------------------------------

 18:06:25 - Overall input size: 136.98 GB

 18:06:25 - Overall output size: 49.58 GB (Ratio 36.19%)

 18:06:25 - Overall conversion time: 06:50:10