Best Compression Methods for 'Specific' Games. Q&A

[L33THAK0R]

Been digging through the files myself, zstd+deflate streams

[IRAQIGHOST]

s.t.a.l.k.e.r. 2: heart of chornobyl

best method?

[KaktoR]

Send sample man. Everytime the same.

Edit: This is again a shit UE5 game. Try the usual things.

[Toximan69]

Originally Posted by KaktoR

Send sample man. Everytime the same.

Edit: This is again a shit UE5 game. Try the usual things.

sample from Stalker 2
https://drive.google.com/file/d/1AZe...usp=drive_link

[dixen]

Originally Posted by IRAQIGHOST

s.t.a.l.k.e.r. 2: heart of chornobyl

best method?

mue5:k0x33A604DF49A07FFD4A4C919962161F5C35A134D37EFA98DB37A34F6450D7D386:n128+kraken:l8+mermaid -lm -d3

NOTE. Decompression is very slow
oo2core_9_win64.dll 2.9.6 or 2.9.9

[KaktoR]

Do not use 2.9.6.

2.9.5 is the correct one.

[Danziel123]

Originally Posted by KaktoR

Do not use 2.9.6.

2.9.5 is the correct one.

Sorry to join the discussion, so what should i write in dsg to compress stalker 2?

[dyren]

Originally Posted by Danziel123

Sorry to join the discussion, so what should i write in dsg to compress stalker 2?

 xtool:c32mb:d1:mue5,k0x33A604DF49A07FFD4A4C919962161F5C35A134D37EFA98DB37A34F6450D7D386:mkraken:core_2.9.5+srep_new:393b+lolz

for *.pak files

[KaktoR]

Alan Wake 2
v1.2.4, nothing ripped

 20:16:46 - Selected ARC/DS method for Data1a-01.bin was: xtool:maw2:dd3:l10,d128mb
 20:16:46 - Selected ARC/DS method for Data1b-01.bin was: xtool:dd3:l10,d128mb
 20:16:46 - Selected ARC/DS method for Data1c-01.bin was: xtool:lm:c1500mb:mbik:mbk2:df20p
-----------------------------------------------------------------------------------------------------------------------------------------
 01:36:40 - Overall input size: 146.06 GB
 01:36:40 - Overall output size: 82.67 GB (Ratio 56.60%)
 01:36:40 - Overall conversion time: 05:19:53

Bink2 videos were ripped out of game files first and then compressed afterwards for reasons, hence the input size. Original size is 111GB.

[ZAZA4EVER]

Indiana Jones and the Great Circle (Test)

Files Test (50.7MB)

"base\gameresources_50.streamdb"
"base\gameresources_patch1_pc.resources"
"base\gameresources_10_patch1.streamdb"
"base\gameresources_20_hq_patch1.streamdb"
"base\gameresources_45_patch1.streamdb"

Method Kraken (oodle file attached)

-moo2reck

Result

FreeArc 0.67 (March 15 2014) creating archive: ZAZA.Test.V5.bin
Compressed 5 files, 53,250,548 => 122,929,176 bytes. Ratio 230.85%
Compression time: cpu 0.05 sec/real 18.41 sec = 0%. Speed 2.89 mB/s
All OK

You can use oodle lib version 2.9.12 (attached with game files)
or oodle lib version 2.9.10 all both is the same result
thanks to Kaktor for oodle lib versions Here

[ZAZA4EVER]

Red Dead Redemption (Test)

Files Test (381MB)

"mapres.rpf"
"blackwater.rpf"

Method ZSTD (ZSTD file attached)

-mzstd,l19

Result

FreeArc 0.67 (March 15 2014) creating archive: ZAZA.Test.bin
Compressed 2 files, 399,803,872 => 1,213,731,228 bytes. Ratio 303.58%
Compression time: cpu 0.61 sec/real 44.09 sec = 1%. Speed 9.07 mB/s
All OK

You can use Zlib+Zstd and get some streams

-mzlib+zstd,l19

FreeArc 0.67 (March 15 2014) creating archive: ZAZA.Test.bin
Compressed 2 files, 399,803,872 => 1,224,533,926 bytes. Ratio 306.28%
Compression time: cpu 0.61 sec/real 46.82 sec = 1%. Speed 8.54 mB/s
All OK

You can use Xtool Ver 0.4.3 to Ver 0.6.0 all works fine with ZSTD

Special thanks to Masquerade , Fitgirl and Kaktor

[IRAQIGHOST]

Indiana Jones and the Great Circle

.streamdb and .resources files

oo2reck:7+srep_new:490g+LOLZ_NORMAL

Sound files

msc_tak+srep_new:490g+LOLZ_NORMAL

rest of files

srep_new:490g+4x4:b96mb:lzma:ultra:bt4:fb273:lc8:m c1000000:96mb

Game size: 112GB
Repack Size: 86GB

use the file attached
thanks to Kaktor for oodle lib

[L33THAK0R]

Based on latest CSF

xtool (marginal zlib gains) + srep + lolz

Original Size: 22.9 GiB
Without Ripped Videos: 12.9 GiB
With Ripped Videos: 14.4 GiB

Non-EN loc., 894 MiB of "optional videos" (start-up cinematics and a 853 MiB promotional video that was used for the "Dawn of the Rise of the Fallen Machines" DLC (same video as seen in this YouTube video), as well as the credits cinematic. Should be noted for ripped videos a dummy Bink video needs to be written from the start offset of a given ripped video, otherwise the title softlocks.

Bink videos are also unable to be packed despite being KB2j variants, either with the xtool plugin or with bpk, bpk refuses outright, xtool plugin at least detects the streams, decoding with the RAD Video Tools shows no visual or audio artefacts strangely.

Attached are some modding scripts that are useful if anyone wants to dig around the files further, I know the original XBOX release had assets compressed with LZ4 (From watching GDC talks where Insomniac developers directly reference this) however I was unable to find any LZ4 streams in either the asset archives or in the decompressed assets, zlib streams are largely in the toc/dag files from testing.

Further gains could likely be made by compressing the audio, encoded with the *.wem structure. Spent a long time trying different approaches, very disappointed with how this one turned out but I've got quite the backlog of projects and alas I have to move on.

[KaktoR]

@L33THAK0R

I was able to pack the bink videos. There are 3, one of them is BK2, the others BIK.

FreeArc 0.67 (March 15 2014) creating archive: Conversion_Output\Data1a-01.bin
Compressed 3 files, 1,580,482,272 => 1,292,223,837 bytes. Ratio 81.76%
Compression time: cpu 1.08 sec/real 154.58 sec = 1%. Speed 10.22 mB/s
All OK

As for the WEM files. I tested a bit with the language files and you will gain just a small amount of ratio even if you group+squash the files into one single file.

g00s001 WEM files 733MB -> 672MB

Just make sure to exclude WEM files which are less than 100kb in size if you process them. There is always a magic line for the WEM files for which - if they less than size X - processing them will yield in zero ratio or sometimes even in negative ratio.

[JustFun]

Dreamfall: The Longest Journey - Steam Version

Selected ARC/DS method for Data1a-01.bin was: 
xtool:c1792mb:mbik+xtool:mpreflate+srep_new:393b+lolz
------------------------------------------------------------------------------------------
Overall input size: 5.92 GB
Overall output size: 1.74 GB (Ratio 29.30%)