FileForums

FileForums (https://fileforums.com/index.php)
-   Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)
-   -   Best Compression Methods for 'Specific' Games. Q&A (https://fileforums.com/showthread.php?t=99554)

dixen 06-02-2022 10:14

SIFU

*.pak not used zlib or kraken but

Total encrypted UE records 35470/35497

AES key for ue4d
Code:

0x40A266F41FDBCE91312FBB86060D2E9425B7D922C0CF0031F634CAD9AECB49DA

Masquerade 06-02-2022 10:34

Steel Division 2 v67500 + All DLCs + Bonus Content

Code:

xtool:reflate+srep+lolz
66.2GB ===> 30.6GB

Notes:
  • DO NOT use ZLIB codec for XTool, crash on decompression.
  • Not much point in separating bonus content, just keep it in main archive.
  • 20 hours max needed for this one, especially if you lolz which needs about 17 hours on its own.

Masquerade 06-02-2022 10:48

There is some WWise Audio in SIFU (especially Chinese voiceovers) but the PAK is encrypted and this is not something I am 100% certain you can tamper with. I know QuickBMS reimporting does not work here.

fabrieunko 07-02-2022 02:33

hello, how did you get 9.8Gb?

dixen 07-02-2022 04:45

Quote:

Originally Posted by fabrieunko (Post 495732)
hello, how did you get 9.8Gb?

Who? SIFU?
ue4d+srep+lolz

KaktoR 07-02-2022 07:57

Dying Light 2
1 language

Code:

19:28:03 - Selected ARC/DS method for Data1a-01.bin was: bpk+xtool:mreflate+srep_new+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000
 19:28:03 - Selected ARC/DS method for Data1b-01.bin was: srep_new+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000
-------------------------------------------------------------------------------------------------------------------------------------------
 21:13:24 - Overall input size: 43.01 GB
 21:13:24 - Overall output size: 25.57 GB (Ratio 59.45%)
 21:13:24 - Overall conversion time: 01:45:15


Masquerade 08-02-2022 10:45

OlliOlli World [Build 7770373]

Step 1: Decompress all .bundle files using UBT. Compress the decompressed files to tmp files and make HDiffZ patches with all tmp files to original bundles. Discard tmp files and original bundles leaving HDiffZ patches and decompressed bundles.

You should end up with about 60mb of HDiffZ patches - not bad for 1100 bundles :D

Step 2: Compress with srep+lolz

2.9GB ====> 4.4GB (after UBT) ====> 720MB

Masquerade 12-02-2022 07:21

The Alien Cube [Build 8187558]

Step 1: Decrypt PAK files using tutorial here.

Step 2: Compress
Code:

xtool+srep+lolz
18.4GB ===> 8.5GB

Masquerade 12-02-2022 07:28

Titanic: Fall of a Legend
Code:

srep+lolz
5.9GB ====> 650MB

pslove 12-02-2022 09:17

Hello everyone, Does anybody have ue4dt decryption key for Sherlock Holmes Chapter One?

Masquerade 12-02-2022 13:23

Quote:

Originally Posted by pslove (Post 495807)
Hello everyone, Does anybody have ue4dt decryption key for Sherlock Holmes Chapter One?

This game is not encrypted.

pslove 12-02-2022 22:03

Quote:

Originally Posted by Masquerade (Post 495809)
This game is not encrypted.

Thank you I thought all unreal engine games are encrypted. What's a good compression method for this game?

Masquerade 12-02-2022 23:24

Quote:

Originally Posted by pslove (Post 495813)
Thank you I thought all unreal engine games are encrypted. What's a good compression method for this game?

XTool with zlib+reflate codec + srep + lolz (or lzma - faster but larger)

kj911 13-02-2022 03:54

MSTS 21GB's version final version results
 
MSTS again, its final results my repacked archives. (This biggest ~21.41GB's version)

Binaries: *.exe/dll and more files. (70 files)
DATA1: *.ace (57832 files)
DATA2a/b/c: see below
SND: *.wav (5316 files)

Note: old DATA2: 133400 files, except wav, exe/dll, ace
DATA2a: 95287 file, more extensions splitted to:

DATA2a: 39188 files, *.raw
DATA2b: 38113 files, *.w, *.ws, *.s, *.pdf
DATA2c: 56099 files, more extensions

Best compresion switches:
Code:

DATA1: xZLib+srep:m3f:a0+lolz:dt:dtb1:dtw1:dtm1:dto1:dtd1:mtt1:mt4:d160m:tt32:oh14:gm01:fba4096:mc1023
DATA2a (OLD): delta+lzma:176mb:normal:bt4:273:mc1000000:lc8
DATA2a (NEW): delta+lolz:mtt1:mt4:d128m:fba4096:mc1023
DATA2b: xZLib+srep:m3f:a0+lolz:dt:dtb1:dtw1:dtm1:dto1:dtd1:mtt1:mt4:d160m:tt16:gm20:fba4096:mc1023
DATA2c: srep:m3f:a0+lolz:dt:dtb1:dtw1:dtm1:dto1:dtd1:mtt1:mt4:d160m:tt16:fba4096:mc1023
SND: msc+rep:256mb+4x4:tor:3:1mb+lzp:8mb:3:h22:d64mb
SND (NEW): msc+srep:m3f:a0+lolz:dt:dtb1:dtw1:dtm1:dto1:dtd1:mtt1:mt4:d160m:tt8:fba4096:mc1023+lzp:8mb:3:h22:d64mb
Binaries: rep:17mb+dispack070+delta+lzma:17mb:normal:bt4:273:mc1023:lc8

Binaries: 70 files, 17,373,096 bytes, 3,669,347 compressed
DATA1: 57,832 files, 12,412,720,905 bytes, 3,137,451,367 compressed
DATA2a (OLD): 39,188 files, 5,908,783,084 bytes, 652,271,468 compressed
DATA2a (NEW): 39,188 files, 5,908,783,084 bytes, 605,077,496 compressed
DATA2a (NEW2): 39,188 files, 5,908,783,084 bytes, 602,084,428 compressed
DATA2b: 38,113 files, 1,165,304,556 bytes, 98,504,355 compressed
DATA2c: 56,099 files, 1,086,843,438 bytes, 89,146,961 compressed
SND: 5,316 files, 2,397,690,229 bytes, 688,262,158 compressed
SND (NEW): 5,316 files, 2,397,690,229 bytes, 685,834,402 compressed
Total (best results): 196 618 files, 22 988 715 308 byte, 4,616,690,860 compressed

Compare, LOLZ vs LZMA with *.raw files only:
LOLZ variant (OLD): srep:m3f:a0+lolz:dt:dtb1:dtw1:dtm1:dto1:dtd1:mtt1: mt4:d160m:tt16:gm20:fba4096:mc1023
39,188 files, 5,908,783,084 bytes, 725,963,280 compressed
LZMA variant1: rep:512mb+exe+delta+lzma:176mb:normal:bt4:128
39,188 files, 5,908,783,084 bytes, 663,495,144 compressed
LZMA variant2 (best): delta+lzma:176mb:normal:bt4:273:mc1000000:lc8
39,188 files, 5,908,783,084 bytes, 652,271,468 compressed
LOLZ variant (NEW): delta+lolz:mtt1:mt4:d128m:fba4096:mc1023
39,188 files, 5,908,783,084 bytes, 605,077,496 compressed
LOLZ variant (NEW2): delta+lolz:mtt1:mt4:d128m:tt16:fba4096:mc1023
39,188 files, 5,908,783,084 bytes, 602,084.428 compressed

Old DATA2a vs. splitted a+c versions:
Old: 762 844 107 byte
a+c total: 741 418 429 byte
Old DATA2 vs. splitted a+b+c versions:
Old: 879 942 307 byte
a+b+c total (OLD): 839 922 784 byte
a+b+c total (NEW): 793 609 681 bytes
a+b+c total (NEW2): 790 616 138 byte

21.41GB --> 4.35GB (OLD) --> ~4.301GB (FINAL)
Huuuhh. :D

UPDATE: Data2a results updated!!! And wait next days in newest "NEW2" only results.

UPDATE2: New SND results added. Its probably totally finalized my their repacks.

KaktoR 13-02-2022 09:34

You can compress raw files with msc too

Edison007 13-02-2022 10:02

kj911
please upload RAW-files

kj911 13-02-2022 10:19

Kaktor: The MSC not properly handled files, according in my memory, its will compression, used Win7 SP1 x64 OS with, not my laptop. (BMF compressor issues.)

Edison007: The full Data2a.arc file?? Its huge uploading from, my very slooowly net connection. (2/0.19Mbit/s) Wait in few days and uploadig the files from, use my friend faster internet connections.

Waiting from private messages, from file uploading finished.

KaktoR 13-02-2022 11:05

I don't have the game so I can't say for sure :)

And Edison007 just want some sample files, not the whole archive :D

kj911 13-02-2022 13:31

Edison007

sample1: https://mixdrop.co/f/o71r3rkwcjxngl

Contains "-01*.raw" files from the smallest ~17.8GB's MSTS packages.

Uploading now sample2, contains "-11e1*.raw" files.

sample2: https://mixdrop.co/f/l6lr3d1lsqg9886

KaktoR

Tested MSC compression, totally ignored files, don't compress. Any game files, not compress properly this BMF compressor. (any *.dds and much more)

:( Sad8669 15-02-2022 19:30

Sifu - version 4.26.2.0
 
Code:

Method : ue4dt+srep:m3f:l512+lolz:dtb1:d128:mtt0:mt4:mc1023
Compression Time: 10.1 Hours
Initial Size : 18.93 GB
Final Size: 9.73 GB
Notes : Game is encrypted, you will need a key.


Masquerade 16-02-2022 15:05

Nebuchadnezzar [v1.3.0]

Code:

precomp:d1:brute:cn:intense+srep+lolz/$ogg=oggre
970MB ===> 360MB

Razor12911 16-02-2022 22:57

3 Attachment(s)
Watch Dogs Complete Edition

Method: xtool:wd1+srep:m3f+lzturbo:49:b128
Compression time: 1 hrs, 52 mins
Install time: 20 mins, 44 secs

Final size 12.5 GB + 1.19 GB (English) = 13.7 GB

Things you'll need
xtool 0.4+
xmemcompress
bms2xtl (only if you want to generate your own database using the attached script)

Notes
bpk crashes on video.dat
I used lzturbo because I want speed, you can use lzma or lolz for better results
You can generate your own database if you have issues with this database

its_gnar_again 17-02-2022 11:59

The king of fighters xv
 
THE KING OF FIGHTERS XV

Code:

srep+lolz
Input: 31.5 GB (.paks only)
Output: 28.3 GB

Not much can be done I guess, I haven't looked at the .paks themselves though. You can replace lolz with LZMA and have faster compression time outputing a similar size.

Masquerade 17-02-2022 12:45

Quote:

Originally Posted by its_gnar_again (Post 495871)
Not much can be done I guess, I haven't looked at the .paks themselves though.

Very bold statement considering you didn't even look at the game files :rolleyes: :mad: I got the following result in about 2 minutes :rolleyes:

Game is encrypted.

Code:

0x22BB765FC95A096C0668A534CD8B3370EF3D47208F4E5361A81D2684228FF685
Code:

Compressing 1 file, 333,594,310 bytes
  Compressing pakchunk0optional-WindowsNoEditor.pak                      0%
Init encryption key: 22BB765FC95A096C0668A534CD8B3370EF3D47208F4E5361A81D2684228FF685
Total encrypted UE records 422/422

Compressed 1 file, 333,594,310 => 865,991,185 bytes. Ratio 259.59%
Compression time: cpu 0.94 sec/real 21.95 sec = 4%. Speed 15.20 mB/s
All OK


its_gnar_again 17-02-2022 13:26

Quote:

Originally Posted by Masquerade (Post 495873)
Very bold statement considering you didn't even look at the game files :rolleyes: :mad: I got the following result in about 2 minutes :rolleyes:

Yep! You're absolutely right, because I literally just said right there on the post I didn't look at the actual .paks. I woke up early and started it without out any inspection of them just to see. I just got back to my PC and posted the result of it. Thanks for the info though!

It's been a year already and you're still passive aggressive. Let it go.

:( Sad8669 17-02-2022 23:38

Code:

Compressed 12 files, 2,228,196,760 => 4,685,624,638 bytes. Ratio 210.29%
Compression time: cpu 4.42 sec/real 117.65 sec = 4%. Speed 18.94 mB/s
All OK

Use xt_CriLayla on .cpk files for Yakuza : LAD
Don't use it on movies file with extension "cpk".

:( Sad8669 18-02-2022 04:03

Hedon Bloodrite - v2.2.0
 
Code:

Method :xt_zlib+reflate+srep:m3f:l512+lolz(normal)
Initial Size : 566 MB
Final Size : 492 MB
Ripped Files : Bonus OST


Unjk223 18-02-2022 04:21

Quote:

Originally Posted by :( Sad8669 (Post 495898)
Code:

Method :xt_zlib+reflate+srep:m3f:l512+lolz(normal)
Initial Size : 566 MB
Final Size : 492 MB
Ripped Files : Bonus OST


Pretty close good than my single file Hedon.ipk3 (compressed 494 MB).

So after encoding, how can I decode back from compressed file to original files like subfolders (folder directory) back?

- Unjk223

:( Sad8669 18-02-2022 06:32

It's not just a file, it's the whole game.

Put the archive near Freearc and use the unpack command to decode it, make sure you set the arc.ini for the compressors and precompressors.

st0rm 19-02-2022 12:38

Resident Evil 4 [2007]
 
Code:

Game Version : 1.1.0

Ripped Files : Nothing Ripped or Encoded (Multi-5)

Original Size : 4.61 GB

Compressed Size : 2.25 GB

Install Time : 5-6 Mins on HDD (Depends your system)

Method : srep:m3f+lzma2


KaktoR 19-02-2022 13:28

Cyberpunk 2077 v1.5 bink movies

Exclude the following files from bpk
Code:

base\movies\fullscreen\q001\q001_dex_briefing.bk2
base\movies\fullscreen\q004\q004_bd_cctv.bk2
base\movies\fullscreen\q101\q101_07_ripperdoc_critical_state.bk2
base\movies\fullscreen\q104\q104_bg.bk2
base\movies\fullscreen\q110\agent_cinema\q110_agents_burning.bk2
base\movies\misc\loading_screens\loading_screen_12_after_q114_and_q116\shot_01\fire_smoke_loop.bk2
base\movies\misc\q110\2024_countdown.bk2
base\movies\misc\q110\agent_cinema\booting_systems.bk2
base\movies\misc\q113\world_middle_v2_outro_glitch.bk2
base\movies\misc\q115_lockdown\qr_code.bk2

You can use wolvenkit.cli

Or just use bink patch from Razor12911 without doing anything and just compress (not tested yet).

Edit: Test done using bpk patch from Razor12911
Code:

Compressed 1 file, 8,894,373,888 => 8,210,544,161 bytes. Ratio 92.31%
Compression time: cpu 447.69 sec/real 2163.81 sec = 21%. Speed 4.11 mB/s
All OK


:( Sad8669 20-02-2022 00:37

Attack on Titan - Precompression (LINKDATA_EU_A.BIN)

Code:

precomp_mtx :

11:41:33 PM - Overall input size: 3.95 GB
11:41:33 PM - Overall output size: 6.41 GB
11:41:33 PM - Overall conversion time: 01:03:44

ZTool :

11:53:01 PM - Overall input size: 3.95 GB
11:53:01 PM - Overall output size: 6.41 GB
11:53:01 PM - Overall conversion time: 00:07:50

xt_zlib (2020) :

11:58:16 PM - Overall input size: 3.95 GB
11:58:16 PM - Overall output size: 6.40 GB
11:58:16 PM - Overall conversion time: 00:04:35

xt_zlib+reflate :

12:06:12 AM - Overall input size: 3.95 GB
12:06:12 AM - Overall output size: 6.41 GB
12:06:12 AM - Overall conversion time: 00:07:20

Reflate :
12:13:40 AM - Overall input size: 3.95 GB
12:13:40 AM - Overall output size: 6.47 GB
12:13:40 AM - Overall conversion time: 00:06:01

Preflate :

12:26:29 AM - Overall input size: 3.95 GB
12:26:29 AM - Overall output size: 6.41 GB
12:26:29 AM - Overall conversion time: 00:07:05


Razor12911 20-02-2022 02:13

Quote:

Originally Posted by :( Sad8669 (Post 495920)
Attack on Titan - Precompression (LINKDATA_EU_A.BIN)

When it comes to zlib codecs (mostly reflate or preflate), greater output doesn't mean the best especially if every single one of them is close to each other in terms of result. Apply strong compression and see for yourself. You may find out that reflate producing larger output is the worst.
Just a minor tip... That's why in the last update, I added reflate level control which can be access by -mreflate:l9, the best level is the one that gives the least output. (Default is 6)

:( Sad8669 20-02-2022 02:16

Thanks for the info!

KaktoR 20-02-2022 08:43

1 Attachment(s)
Final Fantasy VII Remake Intergrade
Database test

Code:

E:\_Compress\#FA_example>arc.exe a -ep1 -r -ed -s; -w.\temp -mxtool:kraken data.arc "pack\*"
FreeArc 0.67 (March 15 2014) creating archive: data.arc
Compressed 1 file, 68,497,536 => 154,750,687 bytes. Ratio 225.92%
Compression time: cpu 0.03 sec/real 13.99 sec = 0%. Speed 4.90 mB/s
All OK

---

E:\_Compress\#FA_example>arc.exe a -ep1 -r -ed -s; -w.\temp -mxtool:ff7 data.arc "pack\*"
FreeArc 0.67 (March 15 2014) creating archive: data.arc
Compressed 1 file, 68,497,536 => 177,710,239 bytes. Ratio 259.44%
Compression time: cpu 0.08 sec/real 11.44 sec = 1%. Speed 5.99 mB/s
All OK

*Database was created from latest game version

Usage:
Code:

Place xtl, key file and oodle library next to xtool.exe

Use -mxtool:ff7 along with -d1 parameter

xtl, key file can be removed for decoding

Edit: Attachment updated

KaktoR 20-02-2022 08:51

1 Attachment(s)
Assassin's Creed Origins
Database test

Code:

E:\_Compress\#FA_example>arc.exe a -ep1 -r -ed -s; -w.\temp -mxtool:mermaid data.arc "pack\*"
FreeArc 0.67 (March 15 2014) creating archive: data.arc
Compressed 1 file, 78,315,520 => 180,755,153 bytes. Ratio 230.80%
Compression time: cpu 0.03 sec/real 8.34 sec = 0%. Speed 9.39 mB/s
All OK

---

E:\_Compress\#FA_example>arc.exe a -ep1 -r -ed -s; -w.\temp -mxtool:acorigins data.arc "pack\*"
FreeArc 0.67 (March 15 2014) creating archive: data.arc
Compressed 1 file, 78,315,520 => 191,434,607 bytes. Ratio 244.44%
Compression time: cpu 0.08 sec/real 6.28 sec = 1%. Speed 12.46 mB/s
All OK

*Database was created from latest game version

Usage:
Code:

Place xtl and oodle library next to xtool.exe

Use -mxtool:acorigins

xtl file can be removed for decoding

Edit: Attachment updated
Edit 2: Fixed database, sorry

KaktoR 20-02-2022 10:46

1 Attachment(s)
Assassin's Creed Odyssey
Database test

Code:

E:\_Compress\#FA_example>arc.exe a -ep1 -r -ed -s; -w.\temp -mxtool:mermaid data.arc "pack\*"
FreeArc 0.67 (March 15 2014) creating archive: data.arc
Compressed 1 file, 82,968,576 => 265,248,323 bytes. Ratio 319.70%
Compression time: cpu 0.09 sec/real 14.60 sec = 1%. Speed 5.68 mB/s
All OK

---

E:\_Compress\#FA_example>arc.exe a -ep1 -r -ed -s; -w.\temp -mxtool:acodyssey data.arc "pack\*"
FreeArc 0.67 (March 15 2014) creating archive: data.arc
Compressed 1 file, 82,968,576 => 274,301,319 bytes. Ratio 330.61%
Compression time: cpu 0.09 sec/real 10.35 sec = 1%. Speed 8.02 mB/s
All OK

*Database was created from latest game version

Usage:
Code:

Place xtl and oodle library next to xtool.exe

Use -mxtool:acodyssey

xtl file can be removed for decoding

Edit: Attachment updated
Edit 2: Fixed database, sorry

:( Sad8669 20-02-2022 19:24

Attack on Titan + All DLCs - From the Attack on Titan Series 1/2
 
Code:

Method : xtool_zlib+srep:m3f:l512+lolz:dtb1:d128:mtt0:mt4:mc1023
Compression Time: 7 Hours 48 Minutes
Decompression Time : 11 Minutes
Initial Size : 17.15 GB
Final Size: 5.17 GB
Ripped Files : France and German Videos.


KaktoR 21-02-2022 14:29

1 Attachment(s)
Middle-Earth Shadow of War
Database test

Code:

E:\_Compress\#FA_example>arc.exe a -ep1 -r -ed -s; -w.\temp -mxtool:kraken data.arc "pack\*"
FreeArc 0.67 (March 15 2014) creating archive: data.arc
Compressed 1 file, 477,373,800 => 671,976,339 bytes. Ratio 140.77%
Compression time: cpu 0.34 sec/real 64.02 sec = 1%. Speed 7.46 mB/s
All OK

---

E:\_Compress\#FA_example>arc.exe a -ep1 -r -ed -s; -w.\temp -mxtool:mesow data.arc "pack\*"
FreeArc 0.67 (March 15 2014) creating archive: data.arc
Compressed 1 file, 477,373,800 => 677,463,660 bytes. Ratio 141.91%
Compression time: cpu 0.45 sec/real 21.50 sec = 2%. Speed 22.20 mB/s
All OK

*Database was created from latest game version

Usage:
Code:

Place xtl and oodle library next to xtool.exe

Use -mxtool:mesow

xtl file can be removed for decoding


KaktoR 21-02-2022 14:31

1 Attachment(s)
Star Wars Jedi Fallen Order
Database test

Code:

E:\_Compress\#FA_example>arc.exe a -ep1 -r -ed -s; -w.\temp -mxtool:mermaid+xtool:zlib data.arc "pack\*"
FreeArc 0.67 (March 15 2014) creating archive: data.arc
Compressed 1 file, 572,494,714 => 1,290,271,600 bytes. Ratio 225.38%
Compression time: cpu 0.70 sec/real 59.56 sec = 1%. Speed 9.61 mB/s
All OK

---

E:\_Compress\#FA_example>arc.exe a -ep1 -r -ed -s; -w.\temp -mxtool:swjfo+xtool:zlib data.arc "pack\*"
FreeArc 0.67 (March 15 2014) creating archive: data.arc
Compressed 1 file, 572,494,714 => 1,363,100,580 bytes. Ratio 238.10%
Compression time: cpu 0.53 sec/real 60.23 sec = 1%. Speed 9.50 mB/s
All OK

*Database was created from latest game version

Usage:
Code:

Place xtl and oodle library next to xtool.exe

Use -mxtool:swjfo

xtl file can be removed for decoding



All times are GMT -7. The time now is 08:39.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2026, vBulletin Solutions Inc.
FileForums @ https://fileforums.com