Best Compression Methods for 'Specific' Games. Q&A

[L33THAK0R]

Has anyone else experienced some issues with Oggre when processing a large amount of streams? I'm currently working on "Nioh 2", whose audio can be extracted into a readable Vorbis Ogg format. I individually packed every file (~9000) and compared the original file checksums to determine which files were incompatible and removed them accordingly. In the past this process has never failed me and has always without fail resulted in a successful decompression of those particular assets, however this time around it seems I'm left with a bit of a confusing result. Attempting to pack all the streams together results in the installer, presumably due to the CLS Oggre plugin, crashing.

Investigating the issue by creating a large stored archive of all files in their packable format shows a much clearer image:

Original Input, Stored Archive (7.79 GB) -> Processed with Oggre (5.29 GB) -> Unpacked with Oggre (4.67 GB)

My current solution I'll be employing for my conversion is simply post-install, unpacking the stored archive with the "temp" packable files, then using a patch made between the temp files and a stored archive of the original files.

I'll be then putting these 2 files alongside each other into one large archive, whilst praying I can squeeze some small gains but I'm under no illusion it'll almost certainly be a satisfactory result. Definitely not happy about the amount of additional R/W operations that'll be performed on the end-users drive, but its the best solution I can come up with.

I should mention I was able to process the audio from the original "Nioh" without any trouble, which follows the same structure as "Nioh 2", with a pretty decent final result of 20.3 GB for v1.21.04 of the Complete Edition release for "Nioh".

UPDATE: I ran an additional test with the original unmodified audio archives packed with a generic srep + lolz configuration, when alls said and done, srep actually works well with the duplicated data, resulting in an inflated final result of only 390MB. Given the negligible gains compared to the amount of data that would would need to be written (~16gb) for the original patching approach to work, I think for the time being, and in the interests of the end-user, I'll sacrifice this 390MB.

[Masquerade]

Yeah, oggre is a mess.

[L33THAK0R]

Can anyone recommend which library and/or version of XTool to use for processing "Mirror's Edge: Catalyst"'s LZ4 streams? I recently tried running my collection of LZ4 libraries against a 100MB sample, using XTool 0.7.8 however seemingly none of them were of any use (see the following pastebin output). I've had success in the past with another LZ4 Frostbite 3 title, "Battlefield: Hardline", however attempting to use the same library for Catalyst results in 0 streams being detected/processed.

I can only find one user actually succeeding to pre-compress Catalyst, who indicates they used "xtlz4(v12)" but I've no idea if this refers to the library or tool.

Any advice would be greatly appreciated, I'm properly stumped with this one.

[kj911]

Try its probably used old Xtool v0.12 with old lz4 libraries. (or used mixed/replaced in game related lz4 library dll.) Check it...

[KaktoR]

If I remeber correct, 1.8.0+ should do it.

Edit:

Streams: 1790 / 1815
Time: 00:00:01 (CPU 00:00:00)

Size: 22.8 MB >> 60.2 MB

lz4 library 1.8.3. Try with others aswell

[Masquerade]

Mary Skelter Finale

xtool_zlib_dd5+lolz/$mp4=srep/$bra=oggre

12.8GB ---> 8.7GB

[Masquerade]

Golf With Your Friends [Build 12062185]

xtool_unity_dd5+lolz

5.1GB ---> 2.5GB

[Masquerade]

Wild question, but would anyone happen to have a backup of the tool linked here?
https://forum.xentax.com/viewtopic.php?t=11106

The blog link is offline. The page is on the Wayback Machine, but the actual file download is blocked behind a xentax login so the file isn't on the wayback machine annoyingly.

[Razor12911]

Originally Posted by L33THAK0R

Can anyone recommend which library and/or version of XTool to use for processing "Mirror's Edge: Catalyst"'s LZ4 streams? I recently tried running my collection of LZ4 libraries against a 100MB sample, using XTool 0.7.8 however seemingly none of them were of any use (see the following pastebin output). I've had success in the past with another LZ4 Frostbite 3 title, "Battlefield: Hardline", however attempting to use the same library for Catalyst results in 0 streams being detected/processed.

I can only find one user actually succeeding to pre-compress Catalyst, who indicates they used "xtlz4(v12)" but I've no idea if this refers to the library or tool.

Any advice would be greatly appreciated, I'm properly stumped with this one.

https://fileforums.com/showpost.php?...82&postcount=2
Use the frostbite 3 plugin for this game -mfrostbite3:lz4

[L33THAK0R]

Originally Posted by Razor12911

https://fileforums.com/showpost.php?...82&postcount=2
Use the frostbite 3 plugin for this game -mfrostbite3:lz4

I should have specified I was having difficulty with finding streams using the frostbite 3 plugin, turns out though I was just testing against a sample that didnt have any streams

Once I tried against a larger sample the plugin worked perfectly, many thanks for your hard-work on it!

[kj911]

Originally Posted by Masquerade

Wild question, but would anyone happen to have a backup of the tool linked here?
https://forum.xentax.com/viewtopic.php?t=11106

The blog link is offline. The page is on the Wayback Machine, but the actual file download is blocked behind a xentax login so the file isn't on the wayback machine annoyingly.

Would that be it? -> https://www.moddb.com/downloads/dpk4util

[Masquerade]

Thanks for the link but that is for DPK4, I was looking for DPK5 version. It's alright now though, the game in question only used zlib so I've done it and moved on.

[IRAQIGHOST]

Forza Horizon 5 V 1.604.481

xtool:c256mb:mpreflate:mzlib+xtool:dd3+LOLZ_ULTRA

Repack Size: 104 Gb
Final Size: 164 Gb

[IRAQIGHOST]

PAYDAY 3

xtool:dd3+LOLZ_ULTRA

Repack Size: 18.7 Gb
Final Size: 46 Gb

[L33THAK0R]

I've been investigating a 2017 title "Lone Echo", which seems to use a previously undocumented Oodle Library, v0.1.0, and whose CRCs don't match any other libraries, as posted by KaktoR.

    
File:   oodle_11_win64.dll
CRC-32: 15a0f169

Attempting to discern which member of the oodle family is used has proven to be a tad tricky, with each sample spitting back a result of [0] (Invalid/Unknown) as reported by Razor's Oodle Stream Scanner (oo2scan_7_win64.exe) utility. Attached is both a sample and the library in question. Since these streams are coming up as Invalid/Unknown is it not possible to re-compress them?