Best Compression Methods for 'Specific' Games. Q&A

[kj911]

Try its probably used old Xtool v0.12 with old lz4 libraries. (or used mixed/replaced in game related lz4 library dll.) Check it...

[KaktoR]

If I remeber correct, 1.8.0+ should do it.

Edit:

Streams: 1790 / 1815
Time: 00:00:01 (CPU 00:00:00)

Size: 22.8 MB >> 60.2 MB

lz4 library 1.8.3. Try with others aswell

[Masquerade]

Mary Skelter Finale

xtool_zlib_dd5+lolz/$mp4=srep/$bra=oggre

12.8GB ---> 8.7GB

[Masquerade]

Golf With Your Friends [Build 12062185]

xtool_unity_dd5+lolz

5.1GB ---> 2.5GB

[Masquerade]

Wild question, but would anyone happen to have a backup of the tool linked here?
https://forum.xentax.com/viewtopic.php?t=11106

The blog link is offline. The page is on the Wayback Machine, but the actual file download is blocked behind a xentax login so the file isn't on the wayback machine annoyingly.

[kj911]

Originally Posted by Masquerade

Wild question, but would anyone happen to have a backup of the tool linked here?
https://forum.xentax.com/viewtopic.php?t=11106

The blog link is offline. The page is on the Wayback Machine, but the actual file download is blocked behind a xentax login so the file isn't on the wayback machine annoyingly.

Would that be it? -> https://www.moddb.com/downloads/dpk4util

[IRAQIGHOST]

Forza Horizon 5 V 1.604.481

xtool:c256mb:mpreflate:mzlib+xtool:dd3+LOLZ_ULTRA

Repack Size: 104 Gb
Final Size: 164 Gb

[IRAQIGHOST]

PAYDAY 3

xtool:dd3+LOLZ_ULTRA

Repack Size: 18.7 Gb
Final Size: 46 Gb

[L33THAK0R]

I've been investigating a 2017 title "Lone Echo", which seems to use a previously undocumented Oodle Library, v0.1.0, and whose CRCs don't match any other libraries, as posted by KaktoR.

    
File:   oodle_11_win64.dll
CRC-32: 15a0f169

Attempting to discern which member of the oodle family is used has proven to be a tad tricky, with each sample spitting back a result of [0] (Invalid/Unknown) as reported by Razor's Oodle Stream Scanner (oo2scan_7_win64.exe) utility. Attached is both a sample and the library in question. Since these streams are coming up as Invalid/Unknown is it not possible to re-compress them?

[Razor12911]

Originally Posted by L33THAK0R

I've been investigating a 2017 title "Lone Echo", which seems to use a previously undocumented Oodle Library, v0.1.0, and whose CRCs don't match any other libraries, as posted by KaktoR.

    
File:   oodle_11_win64.dll
CRC-32: 15a0f169

Attempting to discern which member of the oodle family is used has proven to be a tad tricky, with each sample spitting back a result of [0] (Invalid/Unknown) as reported by Razor's Oodle Stream Scanner (oo2scan_7_win64.exe) utility. Attached is both a sample and the library in question. Since these streams are coming up as Invalid/Unknown is it not possible to re-compress them?

This oodle version is so old and might not be worth looking into, xtool supports v2.1.5+, anything older than that is considered ancient. An update in xtool might add support for this game but I don't plan on releasing anymore updates this year.

Information I gathered from the file:

Signature - 8 byte (0x53534552504D4F43)
Unpacked size - 8 bytes (70921828)
Packed size - 8 bytes (33767326)

repeated 256kb compressed oodle blocks

unpacked block size - 8 bytes
packed block size - 8 bytes
compressed block

Advanced configuration can be made from this information to collect the streams and that's needed is library that uses oodle_11_win64.dll for precompression.

[KaktoR]

I wouldn't say it is undocumented. The problem is that the game in question is exclusively for the oculus rift vr thing. Libraries can differ on different platforms.

And if I look at all the code in shared library, it differs much from the pc equivalent.

[IRAQIGHOST]

Witchfire

xtool:mzlib:dd3+LOLZ_ULTRA

Repack Size: 16.3 GB
Final Size: 23.5 Gb

[KaktoR]

Cyberpunk 2077

I did a bit of digging in last few days. Here are some observations. Maybe you might find it usefull.

audio*.archive
lang_*.archive

• Contains WEM's, BNK's and BIN's.
• For the majority of WEM's and BIN's you can use WEMTool from Masquerade (rename *.bin to *.wem first).
• For some of the files you have to use MSC, look out for this. Honestly I did not have checked how much files are affected by this, and I don't even know if there are tools out there which can check for specific blocks or decoded text to separate the files to be used by WEMTool or MSC.

basegame_5_video.archive
ep1_2_gamedata.archive

These two files contains bink2 files, for which you can use the xtool bink plugin. However the file "ep1_2_gamedata.archive" also contains other files alongside with bink2's, so I guess it should be wise to process the file together with all the other archive files if you don't want to extract the file entirely. However the binks are stored as plain text, so you can patch them out of the file and compress the extracted binks together with the "basegame_5_video.archive" file. Patch them back afterwards (not tested). Bink2 files have *.bin extension.

---

You can unpack the archives with WolvenKit.
I used the console version and used command like this

WolvenKit.CLI.exe <command> <InputFile> <option><OutputFolder>

WolvenKit.CLI.exe extract "G:\SteamLibrary\steamapps\common\Cyberpunk 2077\archive\pc\content\audio_1_general.archive" -o"E:\Test\audio_1_general"

[Hexagon123]

Originally Posted by KaktoR

Cyberpunk 2077

I did a bit of digging in last few days. Here are some observations. Maybe you might find it usefull.

audio*.archive
lang_*.archive

• Contains WEM's, BNK's and BIN's.
• For the majority of WEM's and BIN's you can use WEMTool from Masquerade (rename *.bin to *.wem first).
• For some of the files you have to use MSC, look out for this. Honestly I did not have checked how much files are affected by this, and I don't even know if there are tools out there which can check for specific blocks or decoded text to separate the files to be used by WEMTool or MSC.

basegame_5_video.archive
ep1_2_gamedata.archive

These two files contains bink2 files, for which you can use the xtool bink plugin. However the file "ep1_2_gamedata.archive" also contains other files alongside with bink2's, so I guess it should be wise to process the file together with all the other archive files if you don't want to extract the file entirely. However the binks are stored as plain text, so you can patch them out of the file and compress the extracted binks together with the "basegame_5_video.archive" file. Patch them back afterwards (not tested). Bink2 files have *.bin extension.

---

You can unpack the archives with WolvenKit.
I used the console version and used command like this

WolvenKit.CLI.exe <command> <InputFile> <option><OutputFolder>

WolvenKit.CLI.exe extract "G:\SteamLibrary\steamapps\common\Cyberpunk 2077\archive\pc\content\audio_1_general.archive" -o"E:\Test\audio_1_general"

Use VGMStream to check audio info.

[KaktoR]

Cyberpunk 2077
v2.0, incl DLC, 2 languages, incl bonus content

 08:53:27 - Selected ARC/DS method for Data1a-01.bin was: xtool:mcp2077+xtool:lm:c1024mb:mbk2:df20p+xtool:dd3+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000
 08:53:27 - Selected ARC/DS method for Data1b-01.bin was: xtool:mcp2077+xtool:dd3+msc_tak+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000
 08:53:27 - Selected ARC/DS method for Data1c-01.bin was: xtool:dd3+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000
-----------------------------------------------------------------------------------------
 12:55:21 - Overall input size: 83.05 GB
 12:55:21 - Overall output size: 62.31 GB (Ratio 75.02%)
 12:55:21 - Overall conversion time: 04:01:50

Same as above but packed all archive files together

Overall output size: 62.60 GB (Ratio 75.38%)