Best Compression Methods for 'Specific' Games. Q&A - Page 206

Gehrman · #1 15-12-2022, 12:01

Quote:

Originally Posted by KaktoR

Grand Theft Auto V
Build 10107212 (13 December 2022)

Code:

 17:17:06 - Selected ARC/DS method for Data1a-01.bin was: bpk+xtool:mreflate:db:-dedup=3+4x4:lzma
 17:17:06 - Selected ARC/DS method for Data1b-01.bin was: xtool:db:-dedup=3+4x4:lzma
-----------------------------------------------------------------------------------------
 20:20:08 - Overall input size: 108.55 GB
 20:20:08 - Overall output size: 65.61 GB (Ratio 60.45%)
 20:20:08 - Overall conversion time: 03:02:59

Data1a
common.rpf
x64*.rpf
update\*.rpf

Edit: Use srep instead of xtool dedup. At least for me xtool just crashes in the middle, even with low settings. With srep cls it is working fine.

It is more optimal with --mem=10p

Code:

[External compressor: xtool]
header = 0
unpackcmd = "{compressor}.exe" decode -t100p --mem=10p - - <stdin> <stdout>

Also, use the 32-bit version of SREP.exe.

KaktoR · #2 16-12-2022, 09:57

Elden Ring
v1.08.1

Code:

 17:21:11 - Selected ARC/DS method for Data1a-01.bin was: xtool1+xtool2+xtool:db:-dedup=3+4x4:lzma
 17:21:11 - Selected ARC/DS method for Data1b-01.bin was: xtool:db:-dedup=3+4x4:lzma
-------------------------------------------------------------------------------------------------------------------------------------------
 18:57:27 - Overall input size: 48.07 GB
 18:57:27 - Overall output size: 38.88 GB (Ratio 80.88%)
 18:57:27 - Overall conversion time: 01:36:15

Gehrman · #3 17-12-2022, 11:24

Code:

(EN) Without 4KMovie,SignLanguage

Code:

xtool:c256mb:mpreflate:mzlib+srep:m3f:l512+4x4:lzma:a1:mfbt4:32m:fb273:mc10000:lc8

Code:

109.98 GB → 180 GB → 120 GB → 67.6 GB

Code:

CT: 7H 35M
DT: 52M

KaktoR · #4 25-12-2022, 07:17

Crisis Core - Final Fantasy VII - Reunion
v1.01

Code:

 13:45:17 - Selected ARC/DS method for Data1a-01.bin was: xtool:c1gb:d1:lm:mccff7r+xtool:mkraken,l7:core_2.9.5+xtool:mpreflate:db:dd3+4x4:lzma
 13:45:17 - Selected ARC/DS method for Data1b-01.bin was: xtool:db:dd3+4x4:lzma
-------------------------------------------------------------------------------------------------------------------------------------------
 14:42:31 - Overall input size: 18.17 GB
 14:42:31 - Overall output size: 14.94 GB (Ratio 82.21%)
 14:42:31 - Overall conversion time: 00:56:26
 14:42:31 - Overall conversion media: 1 custom media
-------------------------------------------------------------------------------------------------------------------------------------------
 14:59:20 - Overall decompression time: 00:16:32
 14:59:20 - All files have been successfully extracted!

dixen · #5 25-12-2022, 08:16

Some tests MK11

DISMEMBERMENTSHARED_ScriptAssets.psf

Old version (oodle lzna). Need unreal.dll & oo2core_5_win64.dll from game folder

Quote:

Compressed 1 file, 959,141,047 => 2,044,941,730 bytes. Ratio 213.21%
Compression time: cpu 0.17 sec/real 101.97 sec = 0%. Speed 9.41 mB/s
All OK

Quote:

Tested 1 file, 2,044,941,730 => 959,141,047 bytes. Ratio 213.21%
Testing time: cpu 0.50 sec/real 88.30 sec = 1%. Speed 10.86 mB/s
All OK

New version (oodle kraken, last patch from 15.12.2022). Need oo2core_5_win64.dll from game folder

Quote:

Compressed 1 file, 1,052,382,310 => 2,005,105,169 bytes. Ratio 190.53%
Compression time: cpu 0.16 sec/real 36.26 sec = 0%. Speed 29.02 mB/s
All OK

Quote:

Tested 1 file, 2,005,105,169 => 1,052,382,310 bytes. Ratio 190.53%
Testing time: cpu 0.50 sec/real 7.24 sec = 7%. Speed 145.43 mB/s
All OK

KaktoR · #6 25-12-2022, 11:50

Immortals - Fenyx Rising
v1.3.4

Code:

 16:18:24 - Selected ARC/DS method for Data1a-01.bin was: oo2recs:core_2.9.5+srep+4x4:lzma
 16:18:24 - Selected ARC/DS method for Data1b-01.bin was: srep+4x4:lzma
-----------------------------------------------------------------------------------------
 19:09:55 - Overall input size: 50.66 GB
 19:09:55 - Overall output size: 23.68 GB (Ratio 46.73%)
 19:09:55 - Overall conversion time: 02:51:28
-----------------------------------------------------------------------------------------
 20:47:21 - Overall decompression time: 00:21:56
 20:47:21 - All files have been successfully extracted!

Masquerade · #7 28-12-2022, 11:10

Melty Blood: Type Lumina [Build 10190280]

Code:

xtool (-mreflate --dedup=5) + lolz

20.7GB ---> 4.9GB

Masquerade · #8 30-12-2022, 10:18

Just an FYI, don't use XTool -mzlib+reflate on SnowRunner as it gives CRC fail upon decompression.

KaktoR · #9 31-12-2022, 06:09

Hardspace Shipbreaker
v1.3.0

Code:

 14:56:59 - Selected ARC/DS method for Data1a-01.bin was: xtool:munity,lz4hc,l12:lz4_193+xtool:db:dd3+4x4:lzma
 14:56:59 - Selected ARC/DS method for Data1b-01.bin was: xtool:db:dd3+4x4:lzma
-------------------------------------------------------------------------------------------------------------------------------------------
 15:06:54 - Overall input size: 5.03 GB
 15:06:54 - Overall output size: 2.33 GB (Ratio 46.38%)
 15:06:54 - Overall conversion time: 00:09:49
-------------------------------------------------------------------------------------------------------------------------------------------
 15:09:38 - Overall decompression time: 00:02:12
 15:09:38 - All files have been successfully extracted!

BKR-TN · #10 01-01-2023, 01:00

Europa Universalis IV Incl. All DLCs (GOG) v1.34.5.6

Code:

Compressed 25,670 files, 7,864,063,100 => 4,243,642,157 bytes. Ratio 53.96%
Compression time: cpu 36.48 sec/real 7572.58 sec = 0%. Speed 1.04 mB/s
All OK

Tested 25,670 files, 4,243,642,157 => 7,864,063,100 bytes. Ratio 53.96%    
Testing time: cpu 13.08 sec/real 102.04 sec = 13%. Speed 77.07 mB/s
All OK

Initial size: 7.32 GB
Final size: 3.95 GB
2 cores, 4 threads

Code:

*.mp3, *.ogg and *.zip => xtool:db:dd3
*.wav and *.bmp => msc_tak+xtool:db:dd3
rest of the files => xtool:db:dd3+lolz:dtb1:d128:mtt1:mt4:mc1023

Quote:

Note: There is not that much of gain with compressing mp3 files by mpz (saving ~100 MB), ogg files by oggre (~90 MB) and preflating zip files by xtool (~110 MB) considering how much slower the installer will be. Also, the "eu4.pdb" file will be ~36 MB after compressing.

kj911 · #11 02-01-2023, 03:25

Good compression scheme, methods packing from old XIII game *.hsc files? (And the complete game.) These files encrypted or missed any header data from detectable? MSC not detect reasonable compressibble raw PCM/WAV or MP3 audio data the files. OGGRE it says, no effect. Size reduction its abysmal 5-6-7% no less higher. (Tested in: XIII GOG Update 32942 version.)

*.hxc files finely compressibble in MSC(tak)+ZSTD methods packed to in ~210MB size. (26% ratio, or use any hard final compressor, ex. LOLZ, RZ, more.)

UPD: SREP+DLZ its 2* better than MSC+ZSTD methods. Thanks hints the any repackers from net.

Code:

Compressed 59 files, 849,601,186 => 103,557,504 bytes. Ratio 12.19%
Compression time: cpu 2.20 sec/real 497.64 sec = 0%. Speed 1.71 mB/s
All OK

Testing now MSC+SREP+LOLZ combo.

Masquerade · #12 08-01-2023, 03:25

Watch_Dogs

Full game will ALL languages and ALL DLC content.

Redists and UPlay installer ripped - they are completely unneeded if NoCD is applied.

Requires latest XTool with Watch_Dogs Database and XCompress Plugin thanks to Razor12911.

Requires QuickBMS thanks to Luigi Auriemma and Dunia Engine script.

Thanks to Nicolas Comerci for precomp stdio fork.

Do not use Oggre on the vorbis audio. Encoding is fine, but the decoder will crash.

Conversion Structure:

Archive 1: *.DAT (all except sound.dat, language dats and videos.dat)
Archive 2: All remaining files (binaries and misc documents, *.FAT except language fats)
Archive 3: Bink Videos listed below (pre-packed with BPK)
Archive 4: Videos.dat with BPK compatible videos removed
Archive 5: Sound.dat + English FATs/DATs
Remaining Language Archives: FATs/DATs for extra Non-English languages.

Compress with the following:

Archive 1:
Code:
```
xtool_mwatchdogs_dedup5+lolz
```
Archive 2:
Code:
```
precomps+srep+lolz
```

Archive 3:

Extract the videos.dat with QuickBMS. Use XTool erase command to erase the following videos from the dat:

Code:

0000000000000000.kb2
0000000000000001.kb2
0000000000000003.kb2
0000000000000005.kb2
0000000000000007.kb2
000000000000000a.kb2
000000000000000b.kb2
000000000000000d.kb2
000000000000000f.kb2
0000000000000012.kb2
0000000000000018.kb2
000000000000001a.kb2
000000000000001c.kb2
000000000000001d.kb2
000000000000001f.kb2
0000000000000021.kb2
0000000000000022.kb2
0000000000000026.kb2
0000000000000028.kb2
0000000000000029.kb2
000000000000002b.kb2
000000000000002c.kb2
000000000000002e.kb2
0000000000000033.kb2
0000000000000035.kb2
0000000000000038.kb2
000000000000003b.kb2
000000000000003e.kb2
000000000000003f.kb2
0000000000000040.kb2
0000000000000041.kb2
0000000000000042.kb2
0000000000000045.kb2
0000000000000048.kb2
0000000000000049.kb2
000000000000004a.kb2
000000000000004b.kb2
000000000000004c.kb2
000000000000004d.kb2
000000000000004e.kb2
0000000000000051.kb2
0000000000000052.kb2
0000000000000053.kb2
0000000000000054.kb2
0000000000000055.kb2
0000000000000057.kb2
000000000000005e.kb2
000000000000005f.kb2
0000000000000060.kb2
0000000000000061.kb2
000000000000006b.kb2
000000000000006c.kb2
0000000000000072.kb2
0000000000000073.kb2
0000000000000075.kb2
0000000000000076.kb2
0000000000000078.kb2
000000000000007b.kb2
000000000000007e.kb2
0000000000000080.kb2
0000000000000081.kb2
0000000000000084.kb2
0000000000000086.kb2
0000000000000087.kb2
0000000000000089.kb2
000000000000008b.kb2
000000000000008e.kb2
0000000000000094.kb2
0000000000000095.kb2
0000000000000096.kb2
000000000000009a.kb2
000000000000009b.kb2
00000000000000a2.kb2
00000000000000a5.kb2
00000000000000a8.kb2
00000000000000aa.kb2
00000000000000ad.kb2
00000000000000ae.kb2
00000000000000b0.kb2
00000000000000b6.kb2
00000000000000b7.kb2
00000000000000b8.kb2
00000000000000ba.kb2
00000000000000bb.kb2
00000000000000bd.kb2
00000000000000c0.kb2
00000000000000c2.kb2
00000000000000c3.kb2
00000000000000ca.kb2
00000000000000cc.kb2
00000000000000cf.kb2
00000000000000d3.kb2
00000000000000d5.kb2
00000000000000d7.kb2
00000000000000d8.kb2
00000000000000da.kb2
00000000000000db.kb2
00000000000000dc.kb2
00000000000000e5.kb2
00000000000000e9.kb2
00000000000000ea.kb2
00000000000000eb.kb2
00000000000000ec.kb2
00000000000000f1.kb2
00000000000000f2.kb2
00000000000000f3.kb2
00000000000000f6.kb2
00000000000000fa.kb2
00000000000000fd.kb2
00000000000000fe.kb2
0000000000000101.kb2
0000000000000102.kb2
0000000000000104.kb2
000000000000010c.kb2
000000000000010d.kb2

Pack those videos with BPK (use CLS if you want, I used BPK executable version since I could MT it with mparallel). Those above videos don't require patching. The other videos from the dat get motion_data warning upon packing so packing those is impossible.
Pack Archive 3 with srep.

Archive 4:
Code:
```
srep+LZMA2
```
Archive 5:
Code:
```
rz
```

Remaining Language Archives:

Code:

xtool_mwatchdogs+rz

(don't use xtool dedup since rz has built in dedup)

Final Size

Code:

-------------------------
| Original Size: 28.9GB |
-------------------------
| Game + EN   | 12.12GB |
-------------------------
|      + DE   | +1.21GB |
|                       |
|      + ES   | +1.21GB |
|                       |
|      + FR   | +1.25GB |
|                       |
|      + IT   | +1.20GB |
|                       |
|      + JP   | +1.32GB |
|                       |
|      + PTBR | +1.20GB |
|                       |
|      + RU   | +1.21GB |
| ----------------------|
| TOTAL       | 20.72GB |
-------------------------

KaktoR · #13 14-01-2023, 07:53

One Piece Odyssey

Code:

 15:46:00 - Selected ARC/DS method for Data1a-01.bin was: xtool:c128mb:mue4,k0x6005F03B0D2DABCD75892C80958FFB00CBA5437AD37EB4BFCD5960FFE0C4E12D+xtool:mpreflate+xtool:db:dd3+4x4:lzma
-------------------------------------------------------------------------------------------------------------------------------------------
 16:37:14 - Overall input size: 29.31 GB
 16:37:14 - Overall output size: 24.72 GB (Ratio 84.34%)
 16:37:14 - Overall conversion time: 00:51:12

Masquerade · #14 14-01-2023, 08:50

One Piece Odyssey

Used CLS-UE4D library by Shegorat, but you can also use XTool by Razor12911 (method here by KaktoR).

The file "pakchunk10-WindowsNoEditor.pak" contains only CriMovie USM files so you can just Store this file or decrypt and dedup.

Remaining files:

Code:

ue4d:key=6005F03B0D2DABCD75892C80958FFB00CBA5437AD37EB4BFCD5960FFE0C4E12D+xtool_zlib_dedup5+lolz

Start Size: 29.2GB
Final Size: 22.7GB

KaktoR · #15 15-01-2023, 05:08

Aquatico

Code:

 13:26:49 - Selected ARC/DS method for Data1a-01.bin was: srep+lolz:d64:mtt1:mt12:mc1023
 13:26:49 - Selected ARC/DS method for Data1b-01.bin was: msc_tak+srep+4x4:lzma
 13:26:49 - Selected ARC/DS method for Data1c-01.bin was: srep+4x4:lzma
-------------------------------------------------------------------------------------------------------------------------------------------
 14:04:23 - Overall input size: 8.35 GB
 14:04:23 - Overall output size: 2.00 GB (Ratio 23.99%)
 14:04:23 - Overall conversion time: 00:37:32
-------------------------------------------------------------------------------------------------------------------------------------------
 14:07:58 - Overall decompression time: 00:03:15
 14:07:58 - All files have been successfully extracted!

Data1a = *.mdl|*.dds|*.stl
Data1b = *.bmp|*.wav|*.mp3
Data1c = rest of the files

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Games Are For Kids	JoyBoy	Chit Chat	83	16-11-2021 14:28
CorePack Compression methods ? Help	ramazan19833	Conversion Tutorials	34	09-08-2018 06:51
i got the email in the mail about copying ps2 games	EVOEvo	PS2 Games	7	21-11-2003 07:56
Self-booting games on an older Japanese DC?	Named	DC Games	1	26-09-2003 17:48
Having Problems Burning Numerous Recent Games....Help is Appreciated	Protosstic	PSX Games	2	15-01-2002 21:27

The Following User Says Thank You to Gehrman For This Useful Post:
KaktoR (16-12-2022)

The Following 2 Users Say Thank You to KaktoR For This Useful Post:
Gehrman (17-12-2022), JustFun (12-03-2023)

The Following 3 Users Say Thank You to Gehrman For This Useful Post:
Danziel123 (28-04-2023), JustFun (12-03-2023), Razor12911 (26-12-2022)

The Following 3 Users Say Thank You to KaktoR For This Useful Post:
Gehrman (25-12-2022), JustFun (12-03-2023), Razor12911 (01-01-2023)

The Following 2 Users Say Thank You to dixen For This Useful Post:
Gehrman (25-12-2022), Razor12911 (01-01-2023)

The Following 4 Users Say Thank You to KaktoR For This Useful Post:
Gehrman (25-12-2022), JustFun (12-03-2023), pcblizzard (25-12-2022), Prettyboy099 (25-12-2022)

The Following 5 Users Say Thank You to Masquerade For This Useful Post:
Gehrman (28-12-2022), JustFun (12-03-2023), KaktoR (31-12-2022), Razor12911 (01-01-2023), ScOOt3r (30-12-2022)

The Following User Says Thank You to Masquerade For This Useful Post:
shazzla (30-12-2022)

The Following 2 Users Say Thank You to KaktoR For This Useful Post:
JustFun (12-03-2023), Razor12911 (01-01-2023)

The Following User Says Thank You to kj911 For This Useful Post:
Razor12911 (08-01-2023)

The Following 9 Users Say Thank You to Masquerade For This Useful Post:
BKR-TN (08-01-2023), fabrieunko (08-01-2023), Gehrman (08-01-2023), JustFun (12-03-2023), kj911 (08-01-2023), Razor12911 (08-01-2023), ScOOt3r (08-01-2023), wrathma (31-01-2026), ZAZA4EVER (29-01-2023)

The Following User Says Thank You to KaktoR For This Useful Post:
JustFun (12-03-2023)

The Following 2 Users Say Thank You to Masquerade For This Useful Post:
JustFun (12-03-2023), wrathma (31-01-2026)