Best Compression Methods for 'Specific' Games. Q&A

[Dunnowho69]

What is the best compression method for .unity3d files ? Does it vary from game to game ?

[KaktoR]

Mostly same as bundle files.

The file from that MOUSE game use lz4hc,l12 I guess

[wareck]

yes i try but recover file have not the same size... and give crc error...
i also try -w -r -r and hdiff , but too big and to slow to recover

[shazzla]

Use unity plugin with the correct lz4 dll.
It must br good...

[Dunnowho69]

Is there a easier way to find the correct version of zstd,lz4,oodle etc or do you just sit and test'em all one by one?

[KaktoR]

You can just anticipate the range of libraries. This comes simply with experience over the years.
Examples: UE4/5 games will most likely never use 4.x libraries of oodle (the lowest I've seen is 2.7x in UE4, 2.8x in UE5).
Unity engine will use most likely 1.9+ of lz4 libraries (if the files are compressed).
I suggest using xtoolui for this purpose.

[hwang4]

best compression method ?

[wareck]

Originally Posted by hwang4

best compression method ?

For this file (maybe not realy representative for the full game)

it's: xtool:mreflate+srep+lolz_normal
No ecryption, no oodle, zlib like compression

[KaktoR]

WWE 2K24

 22:11:15 - Selected ARC/DS method for Data1a-01.bin was: xtool:mhydra:core_2.9.14:lib_2:dd3:l10,d128mb
 22:11:15 - Selected ARC/DS method for Data1c-01.bin was: xtool:dd3:l10,d128mb
-----------------------------------------------------------------------------------------------------------------------------------------
 04:48:25 - Overall input size: 105,99 GB
 04:48:25 - Overall output size: 84,96 GB (Ratio 80.16%)
 04:48:25 - Overall conversion time: 06:37:08

Decompression took about 1,5-2 hours
The game use oodle hydra compression (same as 2K25 and 2K26) so without any plugin this is the best approach to use it
Extracted and processed Wwise audio


WWE 2K25

 00:56:15 - Selected ARC/DS method for Data1a-01.bin was: xtool:mhydra:core_2.9.12:lib_2:dd3:l10,d128mb
 00:56:15 - Selected ARC/DS method for Data1c-01.bin was: xtool:dd3:l10,d128mb
-----------------------------------------------------------------------------------------------------------------------------------------
 08:56:28 - Overall input size: 114,75 GB
 08:56:28 - Overall output size: 88,34 GB (Ratio 76.99%)
 08:56:28 - Overall conversion time: 08:00:11

Decompression took about 1,5-2 hours
The game use oodle hydra compression so without any plugin this is the best approach to use it
Extracted and processed Wwise audio


WWE 2K26

 23:23:47 - Selected ARC/DS method for Data1a-01.bin was: xtool:mhydra:core_2.9.14:lib_2:dd3:l10,d128mb
 23:23:47 - Selected ARC/DS method for Data1b-01.bin was: xtool:lm:c1792mb:mbik:mbk2:df20p
 23:23:47 - Selected ARC/DS method for Data1c-01.bin was: xtool:dd3:l10,d128mb
-----------------------------------------------------------------------------------------------------------------------------------------
 11:18:45 - Overall input size: 160,84 GB
 11:18:45 - Overall output size: 107,42 GB (Ratio 66.78%)
 11:18:45 - Overall conversion time: 11:54:56

Decompression took about 1,5-2 hours
The game use oodle hydra compression so without any plugin this is the best approach to use it
Extracted and processed Wwise audio and bink2 videos

[JustFun]

Rayman Legends

Selected ARC/DS method for Data1a-01.bin was: 
mpreflate+srep_new:490g+razor_mtx
------------------------------------------------------------------------------------------
Overall input size: 5,46 GB
Overall output size: 2,64 GB (Ratio 48.31%)

[Dunnowho69]

Originally Posted by wareck

work in progress ...

Data1.bin (*.pak excepted pakchunk10-Windows.pak pakchunk16-Windows.pak)


and maybe better with aes :


For data2 and data3:
pakchunk10-Windows.pak have a lot of small wem files. not very efficient with wemtool.
better to compress with razor than lolz.


pakchunk16-Windows.pak is several bink2 videos who are not compressible with binkpack.
easy way is to again use razor


I'm working on extract bk2 form this .pak to makes a process with lolz and bk2 with other compressor.
but due to AES i cant' use xtool/ringi/mesa to extract/recover files.
I'm working with a bms script in attachment

Can you share the final size and installation time ?

[wareck]

Originally Posted by Dunnowho69

Can you share the final size and installation time ?

Hello
I haven't finish yet.
I'm working on it, it's take time, it's around 17 hours to do a full compress.
My last test is around 98gb compressed with all stuff (bonus + full game) and take 2 hour to uncompress

For now i'm not satisfy about results

[wrathma]

whats the error here ? my machine has 6c12t + 16gb ram. tried on another similar machine, same results.

FreeArc 0.67 (March 15 2014) creating archive: data1.bin
Compressing 202 files, 20,707,514,776 bytes. Processed  10.0%
Compressing 20,707,514,776 bytes with "SREP\srep.exe" -hash- -m3f -c256 -l256 -a0/0 InFile OutFile
 10.0%SREP 3.93a beta (October 11, 2014): input size 19748 mb, memory used 2380 mb, -m3f -l256 -c256 -a0/0 -hash- -b8mb
100%: 20,707,514,776 -> 9,475,155,108: 45.76%.  Cpu 99 mb/s (200.203 sec), real 90 mb/s (219.252 sec) = 91%.  Remains 00:00
Sorting matc 10.0%  Second pass: 18.0%   Remains 00 10.0%
Errorlevel=-1073741819
 10.0%
ERROR: general (de)compression error in srep

[wareck]

Hello
there is something wrong during the pass2 (recheck)
it's more or less like a CRC error in memory during rebuild dedup
You can try another version of srep (3.2 o the 4)
Or you can try to change config:

packcmd = "srep.exe" -hash- {options} InFile OutFile

to

packcmd = "srep.exe" -hash=vmac {options} InFile OutFile

it will add some crc check in the srep file.
In some case it work with a slighty lower ratio.

[wrathma]

thanks for the help wareck, using 3.92b fixed it.