Best Compression Methods for 'Specific' Games. Q&A

[L33THAK0R]

Hi all,

Does anyone know if there are any tools/methods to ID Oodle streams? The only means I know of, is identifying the 0x8c byte in a given sample, but at least in the sample "*.lodpack", "*.texpack" and "*.wad" files that I tested against, I had no luck in finding any streams, when running the sample separately through oo2reck/m/s, and against 3 different "oo2core_7_win64" libraries.

The reason for my testing is this off-hand note in the readme for one of the more popular modding/unpacking suites for the title, which I was using to determine if any additional assets could be ripped and packed separately for further gains.

[Razor12911]

https://fileforums.com/showthread.php?t=102453

This comes with oodle scanner, just drag and drop file.

[Masquerade]

Isn't the 2018 God Of War a PlayStation exclusive? That explains why its using Oodle.

I know the PC re-release in 2022 uses no compression.

[L33THAK0R]

Originally Posted by Masquerade

Isn't the 2018 God Of War a PlayStation exclusive? That explains why its using Oodle.

I know the PC re-release in 2022 uses no compression.

I feel like a right rube again, I contacted the developer and got a message back, just before reading the responses here, the developer also confirmed there isn't any compression used, if I'd taken the time to read through the github release notes I would've seen they officially noted this.

It seems odd the developers wouldn't use Oodle on the PC release given how many folk still have older and slower hardware akin to a PS4's specs.

[Masquerade]

Originally Posted by L33THAK0R

It seems odd the developers wouldn't use Oodle on the PC release given how many folk still have older and slower hardware akin to a PS4's specs.

Perhaps I am missing the point, but wouldn't it make more sense for the developers NOT to use Oodle for the PC release? Older hardware would offer poor decompression speed compared to being able to just read game data from disk straight into memory and have it ready to use.

[L33THAK0R]

Originally Posted by Masquerade

Perhaps I am missing the point, but wouldn't it make more sense for the developers NOT to use Oodle for the PC release? Older hardware would offer poor decompression speed compared to being able to just read game data from disk straight into memory and have it ready to use.

I was thinking more so about read operations from the disk being less intensive if the data was compressed but yes I'll admit I wasn't thinking about the fact it'd be more intensive on a user's memory and processor!

[L33THAK0R]

xtool 0.3.21 + srep + lolz

(Non-Eng. Loc. Ripped)

4.97 GiB >>> 2.58 GiB

Note: Perhaps others will have better luck than me but every bank dumping utility I tried to run this title's "*.banks" through failed on every account.

[Masquerade]

Originally Posted by L33THAK0R

Note: Perhaps others will have better luck than me but every bank dumping utility I tried to run this title's "*.banks" through failed on every account.

Use the FSB5 quickbms script.

+ the devs included a full copy of the movies folder inside the largest .FUK file so you can XTool erase it.

[L33THAK0R]

Originally Posted by Masquerade

Use the FSB5 quickbms script.

+ the devs included a full copy of the movies folder inside the largest .FUK file so you can XTool erase it.

Ah cheers I'll check out the quickbms script, I noted the duplicate movies but I'm fairly sure srep picked them up. I might be missing something but is there a point in blanking out the duplicates, since it'd still be the same amount of data written to disk.

Now that I think about it I didn't actually test to see if deleting the loose files would cause the packed duplicates to be used in their place, or if the loose files were used at all, I'll definitely have to have another quick crack at this one.

[Gehrman]

Game Size: 17.7 GB

Method #1

xtool:d1:c256mb:mgwtokyo+oo2reck+srep:m3f:l512+4x4:lzma:a1:mfbt4:32m:fb273:mc128:lc8

CT: 3H
DT: 30M

Repck Size:14.3 GB

__________________________________________________ ______

Method #2

xtool:c256mb:d1:mgwtokyo:mkraken:dd3+4x4:lzma:a1:mfbt4:32m:fb273:mc10000:lc8

CT: 2H
DT: much longer + high memory usage

Repck Size:14.3 GB

[Gehrman]

Game Size: 17.06 GB

Rip:
MID\Content\Movies\MID_Interview.Webm
MID\Content\Movies\MID_Interview.mov

oo2reck+srep:m3f:l512+4x4:lzma:a1:mfbt4:32m:fb273:mc128:lc8

CT: 2H
DT 40M

Repack Size: 13.7 GB

[Masquerade]

Handyman Corporation

Generate a database for Oodle Kraken using "unreal_tournament_4_0.4.27c_paks_only.bms".

Don't use oo2reck - it seems to have a memory leak during decompression (eating up all your ram) and is insanely slow.

For the pak file, use your database with XTool dedup (requires srep.exe) and add lolz.

For the rest use rz.

2GB --> 1.3GB

[L33THAK0R]

Tools used:

oggre + msc + srep + lolz

Note applicable for all titles: MSC does work on a majority of the .wav/.mp3 loose files as well as the container files used in some titles, such as the "LEGO Harry Potter" titles as well as the "LEGO Middle-Earth" titles, however it's recommend to do a bulk test on all applicable files as there are a fair few instances of CRC-mismatches and there isn't an easy means to identify this without testing. The same can be said for all applicable Vorbis audio. Newer waveform audio in the "*.ADP" container format cannot be processed currently. I haven't tested to see if the engine will accept re-coded audio in the standard and compressible PCM format, given the fact the engine used for all TT titles (bar maybe the Skywalker Saga, given its a fairly considerable engine revision) is building upon the foundations of the previous iteration, it might (?) be possible to do so. Same can probably be done for the various incompatible "*.ogg" files given the format is the same, and some chunks just don't play well with Oggre.

Files ripped: Non-English Loc., unused left-over files from the console releases & left-over demo-exclusive files.

Final notes:

- All tests were based off the latest clean GOG/Steam RIPs.

- Some decompressed sizes may be slightly larger than what others may experience as I tend to include any and all QoL fixes/mods as included offerings, and only include larger less critical mods like the soundtrack replacement mods for the various licensed titles by TT, as optional component offerings.

///////////////////////////////

LEGO City Undercover
15.20 GiB >>> 5.07 GiB


LEGO Harry Potter: Years 1-4
8.54 GiB >>> 1.77 GiB


LEGO Harry Potter: Years 5-7
12.20 GiB >>> 2.73 GiB


LEGO Indiana Jones: The Original Adventures
5.77 GiB >>> 1.41 GiB


LEGO Indiana Jones 2: The Adventure Continues
7.15 GiB >>> 1.50 GiB


LEGO Jurassic World
20.70 GiB >>> 4.52 GiB


LEGO MARVEL Super-Heroes
9.38 GiB >>> 2.30 GiB


LEGO MARVEL Avengers
22.30 GiB >>> 5.30 GiB


LEGO MARVEL Super-Heroes 2
26.73 GiB >>> 7.43 GiB


LEGO Middle-Earth: The Lord of the Rings
8.07 GiB >>> 2.24 GiB


LEGO Middle-Earth: The Hobbit
8.75 GiB >>> 2.69 GiB


LEGO Movie Videogame, The
9.79 GiB >>> 3.89 GiB


LEGO Movie Videogame, The 2
15.00 GiB >>> 3.65 GiB


LEGO Ninjago Movie Videogame, The
29.60 GiB >>> 6.09 GiB


LEGO Pirates of the Caribbean: The Videogame
11.70 GiB >>> 2.17 GiB


LEGO Star Wars - The Complete Saga
9.39 GiB >>> 1.61 GiB


LEGO Star Wars III - The Clone Wars
9.78 GiB >>> 2.10 GiB


LEGO Star Wars: The Force Awakens
28.10 GiB >>> 6.61 GiB


LEGO The Incredibles
22.70 GiB >>> 4.61 GiB


LEGO Worlds
4.36 GiB >>> 1.24 GiB


LEGO DC Batman: The Videogame
8.02 GiB >>> 1.32 GiB


LEGO DC Batman 2: DC Super Heroes
5.62 GiB >>> 1.26 GiB


LEGO DC Batman 3: Beyond Gotham
12.10 GiB >>> 3.33 GiB


LEGO DC Super-Villains
26.14 GiB >>> 6.62 GiB

[L33THAK0R]

Quick question for anyone who's given "Days Gone" a go, has anyone been able to process both the oodle and zlib streams present in the title's "*.pak" archives? I've been using oo2reck (since xtool seems to only be able to use one oodle library and both a "oo2core_7*" & "oo2core_8*" are required to grab all (or at least the majority) of oodle streams. I've tried using both "oo2reck + xtool_0.3.21 (zlib+reflate) + srep + lolz" as well as "xtool_0.3.21 (zlib+reflate) + oo2reck + srep + lolz" but to no avail, interestingly the latter combination results in ~15mins of memory usage with srep and disk write operations, but unfortunately also results in a -11 (corrupt archive) FreeArc decompression error code.

If anyone has had any luck or knows how to get xtool to use both libraries I'd be very much interested in any tips anyone would be willing to offer. Attached are the libraries I've found to be the most effective.

[KaktoR]

Originally Posted by L33THAK0R

Quick question for anyone who's given "Days Gone" a go, has anyone been able to process both the oodle and zlib streams present in the title's "*.pak" archives? I've been using oo2reck (since xtool seems to only be able to use one oodle library and both a "oo2core_7*" & "oo2core_8*" are required to grab all (or at least the majority) of oodle streams. I've tried using both "oo2reck + xtool_0.3.21 (zlib+reflate) + srep + lolz" as well as "xtool_0.3.21 (zlib+reflate) + oo2reck + srep + lolz" but to no avail, interestingly the latter combination results in ~15mins of memory usage with srep and disk write operations, but unfortunately also results in a -11 (corrupt archive) FreeArc decompression error code.

If anyone has had any luck or knows how to get xtool to use both libraries I'd be very much interested in any tips anyone would be willing to offer. Attached are the libraries I've found to be the most effective.

Use two xtool instances with --oodle option in arc.ini like this:

[External compressor:xtool1]
header = 0
packcmd   = xtool.exe precomp -mkraken -c64mb -t100p --oodle=oo2core_7_win64.dll - - <stdin> <stdout>
unpackcmd = xtool.exe decode -t75p - - <stdin> <stdout>

[External compressor:xtool2]
header = 0
packcmd   = xtool.exe precomp -mkraken -c64mb -t100p --oodle=oo2core_8_win64.dll - - <stdin> <stdout>
unpackcmd = xtool.exe decode -t75p - - <stdin> <stdout>

Then use xtool1+xtool2+xtool:zlib