Best Compression Methods for 'Specific' Games. Q&A

[:( Sad8669]

Masquerade

Input size: 43.63 GB
Output size: 30.95 GB
Method : xtool_zlib+srep_new+lzma:a1:mfbt4:256m:fb273:mc1000000000:lc8

Compressing 5,611 files, 46,846,025,126 bytes. Processed  11.1%
Compressing 86,819,705,627 bytes with "OTHERS\SREP\Win64\srep.exe" -m3f -a0/0 InFile OutFile
 11.1%SREP 3.93a beta (October 11, 2014): input size 82797 mb, memory used 4922 mb, -m3f -l512 -c512 -a0/0 -hash=vmac -b8mb
100%: 86,819,705,627 -> 68,740,152,915: 79.18%.  Cpu 18 mb/s (4527.188 sec), real 17 mb/s (4785.019 sec) = 95%.  Remains 00 11.1%    %
Decompression memory is 1748 mb.  11,685,296 matches = 186,964,736 bytes = 0.27% o 11.1%
 11.1%
Errorlevel=0
Compressed 5,611 files, 46,846,025,126 => 33,235,367,739 bytes. Ratio 70.95%
Compression time: cpu 240.02 sec/real 47292.86 sec = 1%. Speed 0.99 mB/s
All OK

Side Notes : Please don't post this in the INDEX, i will shortly update it in another post.

[:( Sad8669]

Method : xtool_zlib+srep_new+lzma:a1:mfbt4:256m:fb273:mc1000000000:lc8
Compression Time : 13 hours 8 minutes 19 seconds.
Initial Size : 43.63 GB
Final Size : 30.95 GB

[L33THAK0R]

Hi all,

I recently finally figured out how to properly encode and package .usm files, and I was wondering if anyone who has some experience here might be able to help me out with a little issue pertaining to implementing the new, encoded files. In the title "Batman: Arkham City" every encoded .usm, bar the initial file loaded "0_CW_Intro.usm", soft-locks the title at the conclusion of the video file, the only solution to this is manual user input to skip the cut-scene, which the title enables users to do for every pre-rendered cut-scene (all pre-rendered cut-scenes are contained within .usm files). This isn't a massive issue, as it can be solved by adding additional footage informing the user when to skip, however, I want to minimise this additional user-inputs if it is at all possible. I have multiple other titles I want to test out encoding .usm files on ("The Witcher 3: Wild Hunt", "Batman: Arkham Origins" & "Batman: Arkham Knight", to name a few), and as this is the first title I have attempted to encode .usm files on, I cannot say whether this is a title-specific issue or not.

My current plan is to stitch on a 10 second clip at the end of each effected video notifying the user their input is required, as I don't think there is any other solution.

[:( Sad8669]

Days Gone - Precompression

XTool - Kraken is 50% faster than OodleRec - Kraken, while the difference is between their ratio is 5% - 6%

OodleRec - Kraken

Compressed 34 files, 35,821,723 => 75,484,606 bytes. Ratio 210.72%
Compression time: cpu 0.06 sec/real 32.63 sec = 0%. Speed 1.10 mB/s
All OK

XTool - Kraken

Compressed 34 files, 35,821,723 => 73,598,522 bytes. Ratio 205.46%
Compression time: cpu 0.06 sec/real 17.51 sec = 0%. Speed 2.05 mB/s
All OK

Side Notes : It's basically waiting for twice the XTool - Kraken and getting only 10% ~ 15 % ratio.

Final Compressors :

XTool:Kraken + SREP + LOLZ :

Compressed 34 files, 35,821,723 => 16,565,774 bytes. Ratio 46.25%
Compression time: cpu 0.06 sec/real 86.91 sec = 0%. Speed 0.41 mB/s
All OK

OodleRec:Kraken + SREP + LOLZ :

Compressed 34 files, 35,821,723 => 16,007,052 bytes. Ratio 44.69%
Compression time: cpu 0.17 sec/real 98.59 sec = 0%. Speed 0.36 mB/s
All OK

Side Notes : Squadrons Decompression time updated.

[L33THAK0R]

Hi all,

Does anyone know if there are any public methods/tools to rip and restore, (however just knowing if it's possible to rip files would be appreciated) files from "id Tech Engine" data files? I'm particularly interested in the 5th and 6th iterations of the "id Tech" engine.

[:( Sad8669]

Days Gone - Full Game Precompression

XTool : v0.3.9 Used.
Codec : Kraken.
Libraries : Provided by Razor12911 for Days Gone.

Results :

Compressed 3,092 files, 64,172,576,777 => 103,896,774,882 bytes. Ratio 161.90%
Compression time: cpu 108.59 sec/real 12011.73 sec = 1%. Speed 5.34 mB/s
All OK

Initial Size : 59.7 GB
Final Size : 96.7 GB

[:( Sad8669]

Method :xtool_kraken+srep:m3f:l512+lzma:a1:mfbt4:256m:fb273:mc1000000000:lc8
Compression Time : 10 Hours.
Decompression Time : 1 Hour 16 Minutes. (4T)
Initial Size : 59.7 GB.
Final Size : 36.9 GB.
Libraries : Both Oodle7 and Oodle8 Provided by Razor12911.
Possibilities : You can rip foreign languages.

[doofoo24]

unravel 1 final size 2.04gb
3.17gb to 4.68gb using munravel1 -c64mb without --dedup
then srep+lolz from 4.68gb to 2.04gb nothing rip include trial exe
srep:m3f:d1g:a2+lolz:mt4:mtt1:d128:fba4096:mc1023: pc4:gm20:dtb1:x2

unravel 2 final size 3.5gb
6.76gb to 10.1gb using munravel2 -c32mb without --dedup
then srep+lolz from 10.1gb to 3.5gb...
srep:m3f:d1g:a2+lolz:mt4:mtt1:d128:fba4096:mc1023: pc4:gm20:dtb1:x2

using razor unravel_R1.7z with xtool 0.3.18

*note you can improve ratio by using mt0 and increase d but super slow...

[doofoo24]

Mass Effect 1 8.7gb
Mass Effect 2 15.3gb
Mass Effect 3 19.7gb
only english
using razor unreal_R5...

method
mue3:m2+srep:m3f:d1g:a2:lolz:mt4:mtt1:d128:fba4096 :mc1023:pc4:dtb1

edit : compression with cls-bpk pass but give CRC error during decompression,
maybe because of multichannel audio ?
if you edit bik file it may work with cls-bik i remember that i tested it in the past when profargo released on number of games and some give the same error duing compression , need to tested...
ME MISTAKE FOR NOT TESTING CLS-BPK UNPACK, SORRY...

[Masquerade]

Guys, you can't use bpk for the videos in Mass Effect - crash on decompression...

[KaktoR]

Yes, you will get crc errors on some files.

[L33THAK0R]

Has anyone given "Sacred 3" a shot? I can't find much information about the engine used online, and no modding community exists (that I know of) to unpack the title's archives. level 9 zlib streams are present according to the "Game File Scanner" tool, however reflate/prelate have little effect (only able to find 10 streams). I'll try applying pzlib to see if that'll have any difference, I'll update in the coming days with results. If anyone would like a sample I'd be happy to upload it.

[:( Sad8669]

Yakuza 0 - Some Tests

.usm files when extracted, have no audio tracks to rip.
.cpk files when extracted, useless because it spawns 20,000 files of 20-50 KB's.
.Contains no streams at all, so you don't need to use a precompressor.

Overall, you can't rip anything out with the modding tools.

Most of you guys might have known about this but i still wanted to share it.

Edit : Upon testing, here are the results. GFS didn't detected any streams though.

precomp_mtx :
Compressed 4 files, 75,380,736 => 84,305,014 bytes. Ratio 111.84%
Compression time: cpu 0.11 sec/real 5.13 sec = 2%. Speed 14.69 mB/s
All OK

ZTool : Compressed 4 files, 75,380,736 => 84,394,635 bytes. Ratio 111.96%
Compression time: cpu 0.17 sec/real 14.04 sec = 1%. Speed 5.37 mB/s
All OK

Reflate : Compressed 4 files, 75,380,736 => 84,426,712 bytes. Ratio 112.00%
Compression time: cpu 0.11 sec/real 3.77 sec = 3%. Speed 19.98 mB/s
All OK

Preflate : Compressed 4 files, 75,380,736 => 84,480,615 bytes. Ratio 112.07%
Compression time: cpu 0.11 sec/real 4.35 sec = 3%. Speed 17.34 mB/s
All OK

Edit 2 : Nevermind, don't use a precompressor, i.e because i think only the one or two of above files contained streams and other files are either .usm or pck and some other extensions.

[Masquerade]

CPK
Use crilayla Sad8669

[:( Sad8669]

Masquerade

No luck man .

Thanks for the help, if you come up with some ideas be sure to remind me.