Best Compression Methods for 'Specific' Games. Q&A

[IRAQIGHOST]

Indiana Jones and the Great Circle

.streamdb and .resources files

oo2reck:7+srep_new:490g+LOLZ_NORMAL

Sound files

msc_tak+srep_new:490g+LOLZ_NORMAL

rest of files

srep_new:490g+4x4:b96mb:lzma:ultra:bt4:fb273:lc8:m c1000000:96mb

Game size: 112GB
Repack Size: 86GB

use the file attached
thanks to Kaktor for oodle lib

[L33THAK0R]

Based on latest CSF

xtool (marginal zlib gains) + srep + lolz

Original Size: 22.9 GiB
Without Ripped Videos: 12.9 GiB
With Ripped Videos: 14.4 GiB

Non-EN loc., 894 MiB of "optional videos" (start-up cinematics and a 853 MiB promotional video that was used for the "Dawn of the Rise of the Fallen Machines" DLC (same video as seen in this YouTube video), as well as the credits cinematic. Should be noted for ripped videos a dummy Bink video needs to be written from the start offset of a given ripped video, otherwise the title softlocks.

Bink videos are also unable to be packed despite being KB2j variants, either with the xtool plugin or with bpk, bpk refuses outright, xtool plugin at least detects the streams, decoding with the RAD Video Tools shows no visual or audio artefacts strangely.

Attached are some modding scripts that are useful if anyone wants to dig around the files further, I know the original XBOX release had assets compressed with LZ4 (From watching GDC talks where Insomniac developers directly reference this) however I was unable to find any LZ4 streams in either the asset archives or in the decompressed assets, zlib streams are largely in the toc/dag files from testing.

Further gains could likely be made by compressing the audio, encoded with the *.wem structure. Spent a long time trying different approaches, very disappointed with how this one turned out but I've got quite the backlog of projects and alas I have to move on.

[KaktoR]

@L33THAK0R

I was able to pack the bink videos. There are 3, one of them is BK2, the others BIK.

FreeArc 0.67 (March 15 2014) creating archive: Conversion_Output\Data1a-01.bin
Compressed 3 files, 1,580,482,272 => 1,292,223,837 bytes. Ratio 81.76%
Compression time: cpu 1.08 sec/real 154.58 sec = 1%. Speed 10.22 mB/s
All OK

As for the WEM files. I tested a bit with the language files and you will gain just a small amount of ratio even if you group+squash the files into one single file.

g00s001 WEM files 733MB -> 672MB

Just make sure to exclude WEM files which are less than 100kb in size if you process them. There is always a magic line for the WEM files for which - if they less than size X - processing them will yield in zero ratio or sometimes even in negative ratio.

[L33THAK0R]

Originally Posted by KaktoR

@L33THAK0R

I was able to pack the bink videos. There are 3, one of them is BK2, the others BIK.

FreeArc 0.67 (March 15 2014) creating archive: Conversion_Output\Data1a-01.bin
Compressed 3 files, 1,580,482,272 => 1,292,223,837 bytes. Ratio 81.76%
Compression time: cpu 1.08 sec/real 154.58 sec = 1%. Speed 10.22 mB/s
All OK

As for the WEM files. I tested a bit with the language files and you will gain just a small amount of ratio even if you group+squash the files into one single file.

g00s001 WEM files 733MB -> 672MB

Just make sure to exclude WEM files which are less than 100kb in size if you process them. There is always a magic line for the WEM files for which - if they less than size X - processing them will yield in zero ratio or sometimes even in negative ratio.

I feel very silly now! It just so happens out of the 3 videos I sampled all had the KB2j signature, talk about rough luck. I need to start packing WEM files, never had massive luck with them due to the annoying packing process, appreciate the advice!

[JustFun]

Dreamfall: The Longest Journey - Steam Version

Selected ARC/DS method for Data1a-01.bin was: 
xtool:c1792mb:mbik+xtool:mpreflate+srep_new:393b+lolz
------------------------------------------------------------------------------------------
Overall input size: 5.92 GB
Overall output size: 1.74 GB (Ratio 29.30%)

[KaktoR]

Don't hassle with the language files. Imo they aren't worth to process with wemtool (just ~15mb per language if you skip all files below 100kb).

You can extract the WEM files with this tool I guess (I use another tool without 2GB limitation)
https://www.fileforums.com/showpost....6&postcount=82

A bit messy but if you get any errors with that 2GB file limitation, just search for the first riff wavefmt stream somewhere in mid-file offset, then split this file into two halfs and use the extractor if you don't have other tools by hand. Like I said this is a bit messy but should work as a workaround. After split and extracting WEM files, erase WEM files with xtool erase command. Afterwards just merge the two files back into one file, then decode back WEM files afterwards with xtool. The rest is up to you.

If I remember correctly the total save with processed WEM files is about 200-250MB in total without language files (input ~1.27 GB).

[JustFun]

Crash Time 2 - Steam Version (Only 1 Language)

Selected ARC/DS method for Data1a-01.bin was: 
xtool:mpreflate+msc_tak+srep_new:393b+lolz
------------------------------------------------------------------------------------------
Overall input size: 1.95 GB
Overall output size: 890.81 MB (Ratio 44.58%)

[dyren]

The Crust

xtool:mreflate,l9+srep+lolz

Game size: 5.16GB
Compressed size: 3.60GB

Paks are not encrypted, uses ZLib compression method.
Compression took ~10 hours on 4c/8t CPU.

[kj911]

Any tools/idea from precompressing old TRS game series? Aka TRS2004 to 2012 games used *.ja archives. Newer TRS2012 SP1 and newer 20xx versions use (probably uncompressed) only *.tzarc game files.

Very old TRS2004 related *.ja extractor added and *.ja archive sample from the TRS2012 game.

In last stage, the overall best methods from *.ja files:

RAZOR only (stdio version) with 1023mb dictionary size, the files shrunked to 4.22GB.

Any rest files use in: xZLib/xtool+msc+delta+srep+lolz or xZLib/xtool+RAZOR

Output size differences abysmal, less than ~0.5MB's compressed size. (RZ used 256MB dict size. Larger probably get in 0.5-2MB extra size reduction.) The game not easily fit to onto ONE DVD5 sizes.

[dyren]

Finnish Cottage Simulator

For paks:

xtool:mkraken:ue5:k0x22269AD77003B4D9225C8FE775B448B8BEACA9D76D3AB881E5BD99D1584E4C76+srep+lolz

Rest:

srep+lolz

Game size: 4.66GB
Compressed size: 2.93GB

Paks are encrypted, uses Oodle, kraken compression method.
Compression took ~3.5 hours on 4c/8t CPU.
Oodle version: 2.9.10, [Unreal Engine 5] oo2core_9_win64.dll (from Masquerade's xtool archive)

[wareck]

*.pak xtool:c32mb:mue4:mzlib+xtool:dd3+lolz:d62:mtt1:mt1 6:mc1023
*.bk2 xtool:dd3+bcm (not compatible with bkp bk2 compression)
other files : xtool:mzlib:mreflate:dd3+razor_mtx

14:58:28 - Overall input size: 21,09 GB
14:58:28 - Overall output size: 9,85 GB (Ratio 46.70%)
14:58:28 - Overall conversion time: 01:00:27
14:58:28 - Overall conversion media: 1 custom media

[IRAQIGHOST]

Hi
What is the best method for Bodycam?
simple
https://www.swiftuploads.com/QN9mE6N736dW/file

[KaktoR]

Maybe use key from Bodycam-Win64-Shipping.exe to find out.

[IRAQIGHOST]

Originally Posted by KaktoR

Maybe use key from Bodycam-Win64-Shipping.exe to find out.

Thanks I found it

[Danziel123]

Hi everyone, may I ask. Maybe someone knows the right compression method for the Ace Attorney Investigations Collection game? This game uses the unity engine and this is a sample file:
https://drive.google.com/file/d/168I...ew?usp=sharing
Thank you