Best Compression Methods for 'Specific' Games. Q&A [Archive] - Page 14

KaktoR

02-11-2022, 14:13

Divinity Original Sin 2 Definitive Edition
v27.09.2022, Nothing ripped, bonus contents are included (Artbook, Lorebook, Concept Art, Soundtrack)

18:42:24 - Selected ARC/DS method for Data1a-01.bin was: xtool:mdos2de:mpreflate+srep_new:3+lzma_sdk
18:42:24 - Selected ARC/SPLIT method for Data1b-01.bin was: srep_new:3+lzma_sdk/$mp3pack=mpz_pmt/$jpgpack=xtool:mbrunsli/$deflatepack=xtool:mreflate+srep_new:3+lzma_sdk
18:42:24 - Selected ARC/DS method for Data1c-01.bin was: bpk
18:42:24 - Selected ARC/DS method for Data1d-01.bin was: 1
-------------------------------------------------------------------------------------------------------------------------------------------
21:28:17 - Overall input size: 60.50 GB
21:28:17 - Overall output size: 34.01 GB (Ratio 56.21%)
21:28:17 - Overall conversion time: 02:45:51
-------------------------------------------------------------------------------------------------------------------------------------------
22:03:03 - All files have been successfully extracted!

Paks are using lz4hc with level 9 and some reflate streams.
However not all lz4 streams can be processed.
To get a good amount of streams set --diff=20p in arc.ini (the sweatspot is somewhere between 15p and 20p, I used 20p. Above this you will get negative ratios on some files. The higher #p, the more negative ratio) and use xdelta3_dll.dll near xtool.exe

Due to false detection by xtool use preflate, not reflate. If you use reflate you will get crc errors on extraction.
For xtl creation use lspk.bms and in bms2xtl.ini set lz4=lz4hc:l9
Use liblz4 180

bpk information

Data1c
Data\Localization\Video.pak
Data\Localization\Video_1.pak
Data\Localization\Video_2.pak
Data\Localization\Video_3.pak
Data\Localization\Video_4.pak
DefEd\Data\Localization\Video.pak

Data1d (crc errors on extraction, don't use bpk on them, or extract the archives first and use binkpatch)
DefEd\Data\Localization\Video_1.pak
DefEd\Data\Localization\Video_2.pak
DefEd\Data\Localization\Video_3.pak
DefEd\Data\Localization\Video_4.pak

Thanks to Razor12911 and Masquerade for some informations :D

dixen

03-11-2022, 01:53

Divinity Original Sin 2 Definitive Edition
v27.09.2022, Nothing ripped, bonus contents are included (Artbook, Lorebook, Concept Art, Soundtrack)
*.pak (excluded video, voice and sound)

srep+lolz = 6.6 gb
Xtool(LZ4 + xtl db)+srep+lolz = 6.4 gb
:D

KaktoR

03-11-2022, 08:45

WRC Generations - The FIA WRC Official Game

lz4f,l10

CHUNK_1.PKG
Compressed 1 file, 230,225,845 => 356,636,013 bytes. Ratio 154.91%
Compression time: cpu 0.25 sec/real 5.90 sec = 4%. Speed 39.00 mB/s
All OK

KaktoR

03-11-2022, 10:07

The Entropy Centre

zlib. There is a key but however it seems you don't need it at all.

KaktoR

03-11-2022, 12:20

Are you sure?:confused:

xtool:reflate:d1:ue4,k0x72463A7B70FA819E286295CF74 C280AC7C928D0BCB349CF811CAF1AE27780A41
Compressed 1 file, 11,896,018,323 => 22,955,933,549 bytes. Ratio 192.97%
Compression time: cpu 6.00 sec/real 496.69 sec = 1%. Speed 23.95 mB/s
All OK

xtool:reflate
Compressed 1 file, 11,896,018,323 => 23,008,806,166 bytes. Ratio 193.42%
Compression time: cpu 5.69 sec/real 566.02 sec = 1%. Speed 21.02 mB/s
All OK

Masquerade

03-11-2022, 23:42

FModel gave me this output for most files :confused:

https://i.ibb.co/Bq8LLH7/Screenshot-1.png

KaktoR

04-11-2022, 00:15

Maybe it's the xtool plugin. I will test with ue4d

Wanterlude

04-11-2022, 04:06

You will need the key. Most files are compressed + encrypted.
The Entropy Centre (v.1.0.7) (GOG)

xtool(zlib)+srep+lolz - 7.83 GB
ue4d+xtool(zlib)+srep+lolz - 7.92 GB

zlib
Compressed 1 file, 11,896,018,323 => 22,940,077,806 bytes. Ratio 192.84%
Compression time: cpu 9.69 sec/real 157.83 sec = 6%. Speed 75.37 mB/s

ue4d+zlib
Compressed 1 file, 11,896,018,323 => 22,517,092,816 bytes. Ratio 189.28%
Compression time: cpu 166.56 sec/real 266.32 sec = 63%. Speed 44.67 mB/s

Masquerade

04-11-2022, 10:52

Sorry for that mix up and thanks for the corrections KaktoR and Waterlude :p

kuyhaa

05-11-2022, 13:53

HARVESTELLA

kraken+zlib+ue4

key

0xAA07B9EF2AD5A0F2C1299355B0760B2808F11038753D5959 D5CA0923A6EE44A5

Harvestella-WindowsNoEditor.ucas

Streams: 190379/194399194420
Time: 00:09:15 (02:01:43)
Memory: 520 MB (520 MB)

100.0%
Errorlevel=0

Compressed 1 file, 10,680,406,976 => 26,584,688,310 bytes. Ratio 248.91%
Compression time: cpu 195.23 sec/real 857.13 sec = 23%. Speed 12.46 mB/s
All OK

dixen

06-11-2022, 12:31

Sackboy: A Big Adventure (2nd repack)

*.pak - XTool v0.65 + unreal.dll + SREP + LOLZ

Compressed 20 files, 56,772,715,077 => 19,317,821,479 bytes. Ratio 34.03%
Compression time: cpu 159.44 sec/real 18183.77 sec = 1%. Speed 3.12 mB/s
All OK

Test

Tested 20 files, 19,317,821,479 => 56,772,715,077 bytes. Ratio 34.03%
Testing time: cpu 6.55 sec/real 263.93 sec = 2%. Speed 215.11 mB/s
All OK

No used precompress function, decrypt .pak only

Razor12911

06-11-2022, 23:51

Burnout Paradise Remastered

-mxzlib+srep+fl2
Compressed 8,468 files, 7,752,954,658 => 3,415,874,982 bytes. Ratio 44.06%
Compression time: cpu 5.81 sec/real 237.31 sec = 2%. Speed 32.67 mB/s

7,21 GB >> 3.18 GB

Need For Speed Hot Pursuit Remastered

-m0
UI\MOVIES
UI\MOVIES_4K
Use store compression, bpk fails on a lot of videos

-mxzlib+srep+fl2
Compressed 20,498 files, 15,721,041,999 => 3,315,144,330 bytes. Ratio 21.09%
Compression time: cpu 12.61 sec/real 277.57 sec = 5%. Speed 56.64 mB/s
on the rest of the files

34.8 GB >> 23.2 GB w/ 4K videos
22,2 GB >> 10.7 GB w/o 4K videos

KaktoR

08-11-2022, 15:30

DOOM Eternal

I made some tests but couldn't find the correct oodle library. Maybe someone find this usefull.

gameresources_0_2.streamdb
oo2core_4_win64_v3 712/1283 Ratio 126.48%
oo2core_6_win64 312/1283 Ratio 101.85%
oo2core_7_win64_v1 650/1283 Ratio 148.50%
oo2core_7_win64_v2 710/1283 Ratio 126.35%
oo2core_7_win64_v3 650/1283 Ratio 148.50%
oo2core_7_win64_v4 759/1283 Ratio 168.52%
oo2core_7_win64_v5 712/1283 Ratio 126.48%
oo2core_7_win64_v6 650/1283 Ratio 148.50%
oo2core_8_win64_v1 759/1283 Ratio 168.52%
oo2core_8_win64_v2 710/1283 Ratio 126.35%
oo2core_8_win64_v3 712/1283 Ratio 126.48%
oo2core_8_win64_v4 759/1283 Ratio 168.52%
oo2core_8_win64 759/1283 Ratio 168.52% (Days Gone (Special Libraries))
oo2core_9_win64_v1 710/1283 Ratio 126.35%
oo2core_9_win64_v2 45/1283 Ratio 100.08%

Side note: The suspicion is obvious that "oo2core_4_win64_v3" is in fact at least "oo2core_7_win64". Stream count, Ratio and file size speaks for itself I think. But I have only tested on this single sample and this should not be used as an evidence at all. Libraries from this post (https://fileforums.com/showpost.php?p=498959&postcount=97).

Razor12911

08-11-2022, 22:56

Doom Eternal uses two libraries on this file however I'm not sure if this applies to the whole game.

oo2core_7_win64.dll -> 283 MB > 421 MB (691/1332)
oo2core_8_win64.dll -> 283 MB > 479 MB (790/1332)

For some reasons, streams that can be processed 7 cannot be processed by 8 and visa versa.

However if you precompress the game using both libraries one after the other you get

283 MB > 421 MB > 614 MB

The second pass using oo2core_8_win64.dll processed the remaining streams (640/640).

This is part of the reason I added --oodle= parameter in xtool, to allow you to specify exactly what library you want to use.

[External compressor:doom7]
header = 0
packcmd = xtool.exe precomp -mkraken,l6,n256 -c32mb -t50p --oodle=oo2core_7_win64.dll - - <stdin> <stdout>
unpackcmd = xtool.exe decode -t100p --oodle=oo2core_7_win64.dll - - <stdin> <stdout>

[External compressor:doom8]
header = 0
packcmd = xtool.exe precomp -mkraken,l6,n256 -c32mb -t50p --oodle=oo2core_8_win64.dll - - <stdin> <stdout>
unpackcmd = xtool.exe decode -t100p --oodle=oo2core_8_win64.dll - - <stdin> <stdout>

Your arc.ini then looks like this then you run the method

-mdoom7+doom8

KaktoR

09-11-2022, 01:49

HARVESTELLA

09:53:02 - Selected ARC/DS method for Data1a-01.bin was: xtool:mue4,k0xAA07B9EF2AD5A0F2C1299355B0760B2808F1 1038753D5959D5CA0923A6EE44A5:mkraken,l5:9+srep+4x4 :lzma
09:53:02 - Selected ARC/DS method for Data1b-01.bin was: srep+4x4:lzma
-------------------------------------------------------------------------------------------------------------------------------------------
10:39:44 - Overall input size: 10.65 GB
10:39:44 - Overall output size: 8.45 GB (Ratio 79.31%)
10:39:44 - Overall conversion time: 00:46:40
-------------------------------------------------------------------------------------------------------------------------------------------
10:49:06 - Overall decompression time: 00:08:08
10:49:06 - All files have been successfully extracted!

Use oo2core_9_win64 in attachment to get all streams.

PS: You can use reflate on pak file to get additional ~20mb inflation.

panker1992

09-11-2022, 06:57

Elden Ring is also one more game that uses 2 different oodle libraries to have 100% coverage of streams.

its best to create a massive batch to process combinations of libraries

Masquerade

09-11-2022, 08:06

Elden Ring is also one more game that uses 2 different oodle libraries to have 100% coverage of streams.

Which libraries are best for this game?

KaktoR

09-11-2022, 08:11

Which libraries are best for this game?

v2.8.14
1267/1413

v2.5.5 / v2.5.2 / v2.5.0
132/134
_____________
1399/1413

I can't get more. I have tested the first 200mb from Data0.bdt

200 -> 355 -> 366

Edit: Here is another test for Data3.bdt

v2.8.14
15077/18312

v2.5.5 / v2.5.2 / v2.5.0
2566/2641
_____________
17643/18312
_____________
669 streams left

---

2.47 GB -> 4.35 GB -> 4.51 GB

Maybe you get more out of it using n# option

dixen

09-11-2022, 12:19

F.I.S.T.: Forged In Shadow Torch

Tests

XZlib+Reflate (-d3 -c32mb)
21.2 gb > 39 gb

unreal.dll+Xzlib+Reflate (-d3 c128mb)
21.2 gb > 65 gb

AES Key
0xEABBDBD5A27C73D8CDAE3179A36C9EB793C3C61188A5763F BC35ACD155D0C604

KaktoR

09-11-2022, 13:54

Halo Infinite
Winter Update (8th November 2022)

14:02:46 - Selected ARC/DS method for Data1a-01.bin was: xtool:mkraken,l4,n64:8+srep+4x4:lzma
14:02:46 - Selected ARC/DS method for Data1b-01.bin was: xtool:mreflate+srep+4x4:lzma
14:02:46 - Selected ARC/DS method for Data1c-01.bin was: srep+4x4:lzma
-----------------------------------------------------------------------------------------
16:25:26 - Overall input size: 44.02 GB
16:25:26 - Overall output size: 31.19 GB (Ratio 70.86%)
16:25:26 - Overall conversion time: 02:22:21

KaktoR

10-11-2022, 03:07

ELDEN RING
v1.07

09:30:58 - Selected ARC/DS method for Data1a-01.bin was: xtool1+xtool2+srep+4x4:lzma
09:30:58 - Selected ARC/DS method for Data1b-01.bin was: srep+lzma_sdk
-----------------------------------------------------------------------------------------
11:55:14 - Overall input size: 45.07 GB
11:55:14 - Overall output size: 36.75 GB (Ratio 81.54%)
11:55:14 - Overall conversion time: 02:21:24

Oodle libraries used: v2.5.5 (xtool1) + v2.8.14 (xtool2)

xtool options: l6,n64

L33THAK0R

10-11-2022, 08:46

Does anyone know how BMP detection under MSC works? Got a few titles I'm working on that mainly use BMP images for texture assets, however MSC seems to skip over the majority of files. I had a brief trawl through some posts here but searching with shorter search terms is a massive pain in the arse and didnt yield a lot of information.

Masquerade

10-11-2022, 09:08

Maybe the thread on Krinkels.org may help you?

KaktoR

11-11-2022, 14:26

Saints Row (2022)
v1.1.6.4392638

21:38:42 - Selected ARC/DS method for Data1a-01.bin was: xtool:mlz4f,l9:lz4_192+srep+4x4:lzma
21:38:42 - Selected ARC/DS method for Data1b-01.bin was: srep+4x4:lzma
-------------------------------------------------------------------------------------------------------------------------------------------
23:25:52 - Overall input size: 44.08 GB
23:25:52 - Overall output size: 25.51 GB (Ratio 57.88%)
23:25:52 - Overall conversion time: 01:47:08

KaktoR

13-11-2022, 13:03

Wolfenstein II The New Colossus

15:56:42 - Selected ARC/DS method for Data1a-01.bin was: xtool:mkraken:5+srep+4x4:lzma
15:56:42 - Selected ARC/DS method for Data1b-01.bin was: bpk
15:56:42 - Selected ARC/DS method for Data1c-01.bin was: srep+4x4:lzma
-------------------------------------------------------------------------------------------------------------------------------------------
21:50:55 - Overall input size: 58.80 GB
21:50:55 - Overall output size: 43.14 GB (Ratio 73.37%)
21:50:55 - Overall conversion time: 05:53:20

Notes:
- Majority of *.resources files using level 4, majority of *.texdb and *.cache using level 5. I was too lazy to sort them out properly so I didn't use the l# option for xtool.
- On many files I didn't got all kraken streams, I checked with every library I have and even used different libraries with --oodle= option like with Elden Ring but to no avail.
- *.pages files using lz4hc but most likely weak compression, but couldn't get all streams (overall about half of streams), so I decided to just compress them with srep+lzma.

dixen

15-11-2022, 09:40

Somerville (Steam version)

*.wem, *.bnk - srep+razorMT
6.3 gb> 1.6 gb

Rest files - srep+lolz
3.7 gb > 430 mb

Install time - 1 min.

Masquerade

17-11-2022, 09:08

Unlike it's original variant, paks in Crysis 3 Remastered are not encrypted. They are just standard zip files and can be precompressed with xtool:

Compressed Animations.pak, 144,174,121 => 229,174,364 bytes. Ratio 158.96%

KaktoR

17-11-2022, 11:34

Is anyone here working with xtool's --dedup option?

I made many tests since yesterday and I think this will replace srep command entirely, at least for me.

xtool+srep+4x4:lzma
Compressed 27 files, 4,594,892,800 => 2,346,970,567 bytes. Ratio 51.08%
Compression time: cpu 3.69 sec/real 921.49 sec = 0%. Speed 4.99 mB/s
All OK

15min 23sec

xtool+4x4:lzma
Compressed 27 files, 4,594,892,800 => 2,346,861,976 bytes. Ratio 51.08%
Compression time: cpu 2.69 sec/real 660.86 sec = 0%. Speed 6.95 mB/s
All OK

11min 02sec

--dedup=(1..5)
srep.exe must be present next to xtool.exe for compression and decompression (no cls-srep required)

KaktoR

18-11-2022, 04:40

Here is another test. You can use xtool without any precompression too.

xtool+4x4:lzma
Compressed 7,383 files, 31,661,584,708 => 8,601,282,118 bytes. Ratio 27.17%
Compression time: cpu 16.39 sec/real 1506.53 sec = 1%. Speed 21.02 mB/s
All OK

28min

srep+4x4:lzma
Compressed 7,383 files, 31,661,584,708 => 8,601,527,510 bytes. Ratio 27.17%
Compression time: cpu 24.27 sec/real 2312.41 sec = 1%. Speed 13.69 mB/s
All OK

38min

[External compressor:xtool]
header = 0
default = -c64mb -t100p
packcmd = "{compressor}.exe" precomp { -moption} --dbase --dedup=3 - - <stdin> <stdout>
unpackcmd = "{compressor}.exe" decode -t100p - - <stdin> <stdout>

KaktoR

18-11-2022, 07:54

Crysis 2 Remastered

14:52:34 - Selected ARC/DS method for Data1a-01.bin was: oggre
14:52:34 - Selected ARC/DS method for Data1b-01.bin was: xtool+4x4:lzma
14:52:34 - Selected ARC/DS method for Data1c-01.bin was: xtool:mcrysis2remastered+4x4:lzma
14:52:34 - Selected ARC/DS method for Data1d-01.bin was: xtool+4x4:lzma
-------------------------------------------------------------------------------------------------------------------------------------------
16:53:20 - Overall input size: 53.76 GB
16:53:20 - Overall output size: 34.79 GB (Ratio 64.72%)
16:53:20 - Overall conversion time: 02:00:44

Data1a
GameCrysis2\music.pak
GameCrysis2\music_chk0.pak

Data1b
GameCrysis2\Video*.pak

Data1c
*.pak (excl 1a+1b)

Data1d
rest of the files

Thanks to Masquerade for database file

Masquerade

18-11-2022, 13:57

For Spiderman Miles Morales, use XTool Spidey plugin with liblz4 v1.9.4 to get all streams.

KaktoR

18-11-2022, 14:43

Edit: Have mixed up with another title :)

fabrieunko

18-11-2022, 23:57

For Spiderman Miles Morales, use XTool Spidey plugin with liblz4 v1.9.4 to get all streams.

thank you very much but where to find?

Masquerade

19-11-2022, 00:20

thank you very much but where to find?

https://github.com/lz4/lz4/releases/tag/v1.9.4

C'mon man, just use Google

fabrieunko

19-11-2022, 09:39

^^ I had used google and found the same link but I was not sure and I did not understand how to replace the dll...

KaktoR

19-11-2022, 10:39

"msys-lz4-1.dll" is the file you need. Rename it to "liblz4.dll".

KaktoR

20-11-2022, 04:54

Anyone knows how to deal with OGV video files?

L33THAK0R

20-11-2022, 05:37

Anyone knows how to deal with OGV video files?

I've had very hit or miss results with Theora (ogv) using Oggre, some, even from the same title and encoded with the same library will fail to decompress but others are able to decompress just fine, if I were a betting man I'd wager it might have something to do with the size of the Vorbis streams within a given OGV container but thats just a guess.

Masquerade

20-11-2022, 05:39

Oggre will always fail with ogv in every time I've tested it. I usually just use razor or lzma on them.

shazzla

20-11-2022, 07:42

@Kaktor : maybe ffmpeg can help you. Or not. Just an idea...

kj911

21-11-2022, 06:54

@Kaktor: Try it use Masqueraqde posted compressors or combine with Theora rehuffing+any compressors if the fact that the rehuffed OGVs are not a problem, we get them in a non-lossless way at the output of the program. Nothing else is out of the question.

KaktoR

21-11-2022, 07:30

Tbh I like it lossless without any recoding or something else like this. So I will stick with lzma2.

Snake288

21-11-2022, 07:36

Hello

i'm getting an error in the institution, why is it caused?

https://i.hizliresim.com/6sm46ue.jpg (https://www.hizliresim.com/6sm46ue)

-----------------------------------------------------------------------------------------
-------------------------------- CONVERSION DATE 20.11.2022 --------------------------------
-----------------------------------------------------------------------------------------
21:08:34 - You have chosen the DVD5 type for the media size.
21:08:34 - ISO images will not be created when conversion is complete.
21:08:34 - The computer will remain on when the conversion is complete.
21:08:34 - Selected ARC/DS method for Data1a-01.bin was: xtool:c32mb:mspidey+srep_new:1+lolz
-----------------------------------------------------------------------------------------
21:08:35 - Compression initialized in 20.11.2022 at 21:08:35.
21:08:35 - Starting Data1a-01.bin archive creation.
17:28:42 - Moving Data1a-01.bin » Data1a-07.bin files to media folder.
17:28:44 - Compression finished in 21.11.2022 at 17:28:44.
-----------------------------------------------------------------------------------------
17:28:44 - Getting size of the game: Marvel’s Spider-Man Miles Morales
17:28:44 - » Installation folder size: 47,41 GB
17:28:44 - » Compressed game size: 27,49 GB (Ratio 57.99%)
-----------------------------------------------------------------------------------------
17:28:44 - Overall input size: 47,41 GB
17:28:44 - Overall output size: 27,49 GB (Ratio 57.99%)
17:28:44 - Overall conversion time: 20:20:08
17:28:44 - Overall conversion discs: 7 DVD5
-----------------------------------------------------------------------------------------

KaktoR

24-11-2022, 08:24

Evil West

For bink2 files use bpk patch method.

15.9 GB -> 14.3 GB
(diff size = 228 MB is included)

Edit:

17:28:19 - Selected ARC/DS method for Data1a-01.bin was: xtool:mreflate+xtool:mpreflate+xtool:db:-dedup=3+4x4:lzma
17:28:19 - Selected ARC/DS method for Data1b-01.bin was: xtool:db:-dedup=3+4x4:lzma
-------------------------------------------------------------------------------------------------------------------------------------------
18:27:20 - Overall input size: 31.64 GB
18:27:20 - Overall output size: 27.34 GB (Ratio 86.38%)
18:27:20 - Overall conversion time: 00:58:59

Masquerade

24-11-2022, 23:43

Automobilista 2 [v1.4.3.3]

*.bff, *.dds
xtool_automobilista2_dedup5_d1+lolz
Everything else:
razor

89.1GB -----> 17.7GB

shazzla

25-11-2022, 03:23

XTool "dedup level 5" ?
Where ,how ? I cant find any information about that !
Is it another not published setting ,like -dd ?!

KaktoR

25-11-2022, 03:44

XTool "dedup level 5" ?
Where ,how ? I cant find any information about that !
Is it another not published setting ,like -dd ?!

--dedup=[1..5]
srep.exe must be present next to xtool.exe for compression and decompression (no cls-srep required)

--dedup=5 is like you use srep -m5f setting

Example: https://fileforums.com/showpost.php?p=499141&postcount=3282

Especially on HDD the speed improvements are very good.

Maybe this is interesting aswell for some users

--compress=[l#,][hi0/1,][t#[p]]
default is l6
requires fast-lzma2.dll next to xtool.exe

KaktoR

29-11-2022, 12:21

Stellaris
v3.6.0, incl all DLC, Soundtrack

15:22:36 - Selected ARC/DS method for Data1a-01.bin was: xtool:mreflate+msc_tak+oggre
15:22:36 - Selected ARC/DS method for Data1b-01.bin was: msc_tak
15:22:36 - Selected ARC/DS method for Data1c-01.bin was: oggre
15:22:36 - Selected ARC/DS method for Data1d-01.bin was: xtool:db:-dedup=3+lolz:d128m:mtt1:mt12:mc1023
15:22:36 - Selected ARC/SPLIT method for Data1e-01.bin was: xtool:db:-dedup=3+lolz:d128m:mtt1:mt12:mc1023/$mp3pack=mpzmt/$deflatepack=xtool:mpreflate:mpng:d1:db:-dedup=3+lolz:d128m:mtt1:mt12:mc1023/$textpack=xtool:db:-dedup=3+lzma:a1:mfbt4:256m:fb273:mc1000000000:lc8
-----------------------------------------------------------------------------------------
16:48:10 - Overall input size: 16.69 GB
16:48:10 - Overall output size: 6.46 GB (Ratio 38.71%)
16:48:10 - Overall conversion time: 01:25:32

Data1a = *.zip
Data1b = *.wav
Data1c = *.ogg
Data1d = *.dds
Data1e = rest of the files

KaktoR

29-11-2022, 14:46

Mount & Blade II Bannerlord

Seems like tpac files use lz4hc level 12. On this level you will get low streams but output is much bigger then with other levels.

Compressed 1 file, 7,120,255 => 14,068,315 bytes. Ratio 197.58%
Compression time: cpu 0.00 sec/real 30.88 sec = 0%. Speed 0.23 mB/s
All OK

However it is slow af

KaktoR

30-11-2022, 07:33

Assassin's Creed Valhalla
v1.6.2, 1 language, all DLC

10:59:07 - Selected ARC/DS method for Data1a-01.bin was: xtool:manvil:8+xtool:db:-dedup=3+4x4:lzma
10:59:07 - Selected ARC/DS method for Data1b-01.bin was: xtool:db:-dedup=3+4x4:lzma
-------------------------------------------------------------------------------------------------------------------------------------------
16:29:11 - Overall input size: 143.49 GB
16:29:11 - Overall output size: 50.14 GB (Ratio 34.94%)
16:29:11 - Overall conversion time: 05:29:24

kj911

02-12-2022, 08:53

Sorry from bumping the old post.

Here is: dlz_v0.2.2b.7z (https://fileforums.com/attachment.php?attachmentid=22205&d=1530380111)

The "dlz_unpack.exe" file its works?? Required special commands or any from decompression natively packed any data in used dlz_x86/64 executables without FreeArc? (aka ex: Bink_(un)pack or oggre_enc/dec...) Tested from Win XP x32 and Win7 SP1 x64 and crashing now. The CLS filter its works with FreeArc/Unarc usage.

KaktoR

03-12-2022, 03:07

The Callisto Protocol

Compressed 2 files, 6,558,206,036 => 13,302,561,746 bytes. Ratio 202.84%
Compression time: cpu 2.78 sec/real 243.09 sec = 1%. Speed 26.98 mB/s
All OK

reflate
0x1958456F8B54A02D25DF32327C57C9D20DFCF7E948967D08 CFB48E8EBC65BEEB

Edit: Seems like you don't need the key. Without the key the output is the same.

L33THAK0R

06-12-2022, 16:23

Is there a way to control memory usage during decompression in SREP.exe?
LAST MINUTE EDIT: Just realized you're talking about using xtool's srep functionality, so not sure if this is useful at all :o

I use the following settings in my "CLS.ini" file, I do use the CLS version of Srep for decompression so not sure if that would have any impact if you're using a non-CLS package.

[Srep]
Bufsize=24m
transfer_ReadBufSize=512k
transfer_WriteBufSize=512k
Memory=60%-2048m
TempPath={app}

[Srep_new]
Bufsize=24m
transfer_ReadBufSize=512k
transfer_WriteBufSize=512k
Memory=60%-2048m
TempPath={app}

abror

06-12-2022, 19:04

please tell me the best compress method for gta 5 game. I am using Cesar82 Diskpan Gui

Cesar82

06-12-2022, 19:12

please tell me the best compress method for gta 5 game. I am using Cesar82 Diskpan Gui
DiskSpan GUI has compression preset in DATABASE for this game, just load the preset.

KaktoR

10-12-2022, 11:18

Anno 1800
v16.0, incl all DLC, all languages, artbook, ost

21:12:27 - Selected ARC/DS method for Data1a-01.bin was: xtool:mflac+xtool:mpreflate+xtool:db:-dedup=3+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1 000000
21:12:27 - Selected ARC/SPLIT method for Data1b-01.bin was: xtool:db:dedup=3+razor_mtx/$mp3pack=mpzmt
-----------------------------------------------------------------------------------------
23:12:09 - Overall input size: 86.73 GB
23:12:09 - Overall output size: 31.42 GB (Ratio 36.23%)
23:12:09 - Overall conversion time: 01:59:41

>> Unpacking finished successfully!

Edit: Method changed from "reflate" to "preflate" because of crc errors on file data13.rda

Masquerade

10-12-2022, 11:21

@KaktoR
You can patch out the bink videos with XTool erase and squeeze the size down with bpk too

KaktoR

10-12-2022, 11:34

@Masquerade
I know. bk2 files are spread all over inside many files and I for myself are just to lazy to do it :D

abror

11-12-2022, 07:33

DiskSpan GUI has compression preset in DATABASE for this game, just load the preset.
thank you, do you know how to make the game install process not take long.
I have compressed the gta 5 game using the method that is in the database but during install (unpack) it takes very long around 5-7 hours

KaktoR

11-12-2022, 11:00

Halo The Master Chief Collection
v1.3065.0.0 (7th December 2022), ripped multiplayer

17:02:27 - Selected ARC/DS method for Data1a-01.bin was: bpk+xtool:db:-dedup=3
17:02:27 - Selected ARC/DS method for Data1b-01.bin was: xtool:c128mb:mpreflate:db:-dedup=3+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1 000000
17:02:27 - Selected ARC/DS method for Data1c-01.bin was: xtool:db:-dedup=3+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1 000000
-----------------------------------------------------------------------------------------
19:06:36 - Overall input size: 103.82 GB
19:06:36 - Overall output size: 64.34 GB (Ratio 61.97%)
19:06:36 - Overall conversion time: 02:04:07

>> Unpacking finished successfully!

---

Data1b:
halo1\prebuild\shaders_2_cache\prebuild\*.sdc
halo2\prebuild\shaders_2_cache\prebuild\*.sdc
halo2\preload\paks\*.pck
halo2\h2_maps_win64_dx11\*.map
halo2\h2_maps_win64_dx11\textures.dat
halo3\maps\*.map
halo3odst\maps\*.map
halo4\maps\*.map
haloreach\maps\*.map
mcc\Content\Paks\*.pak
data\ui\Screens\*.gfx

With reflate you will get a crc error on "MCC-WindowsNoEditor.pak" file. Seems there are lz4 streams too but couldn't process any of them properly (also look in "Data\ui\texturepacks" folder). With old xtool the output is same as reflate option (~1gb smaller) but I prefer speed :p

128mb chunk size is the sweet spot here. However you will get some additional mb on 256mb.

bpk output is 22.5 GB, with srep 21.5 GB (many video stream duplicates).

KaktoR

13-12-2022, 12:00

Crisis Core - Final Fantasy VII - Reunion

pak/ucas -> kraken
Use oo2core v2.9.6

Edit: Ok seems like zlib/reflate is in there too. What a weird mess I made, sorry :D

pakchunk0-WindowsNoEditor.pak

156mb -> 175mb (kraken) -> 199mb (reflate)

Edit 2:
pakchunk1-WindowsNoEditor.pak has no kraken streams, but only reflate.

Edit 3:
pakchunk5-WindowsNoEditor.pak, pakchunk6-WindowsNoEditor.pak -> seems like only usm video
pakchunk4-WindowsNoEditor.pak -> sounds

Masquerade

13-12-2022, 12:33

Crisis Core - Final Fantasy VII - Reunion UCAS contains encrypted Kraken streams but the UnrealDB creator does not work for this game and requires an update.

Razor12911

13-12-2022, 18:19

Crisis Core - Final Fantasy VII - Reunion

Used unreal_db
key 0x277041CBB2888087611DB5CBFEA119462920E1F67092C87E 87825725E38B320F
Oodle v2.9.5 dll

-mccff7r+kraken:l7 -d1 -c1gb -lm
pakchunk7-WindowsNoEditor.ucas
1.53 GB > 2.79 GB

idk, unreal db works for me, but there was a catch. The AES blocks can be larger than 700MB so just set a high chunk size value but add -lm parameter so that xtool doesn't tank your PC.

Edit: I recommend that you use two separate xtool processes (one for decryption and the other for processing kraken streams), since these aes blocks are big, xtool will use 1 thread in d1 mode for both encoding and decoding in an attempt to use the lowest possible memory, which will slow down installation tremendously if you plan to make a repack. This also eliminates the need for -d1, you can opt for oo2reck if you want, up to you.

KaktoR

14-12-2022, 00:54

Thanks, this seem to work :)

-mff7rcc+xtool:kraken,l7

Compressed 2 files, 1,644,918,858 => 3,004,384,018 bytes. Ratio 182.65%
Compression time: cpu 1.22 sec/real 207.25 sec = 1%. Speed 7.94 mB/s
All OK

Extracted 2 files, 3,004,384,018 => 1,644,918,858 bytes. Ratio 182.65%
Extraction time: cpu 1.28 sec/real 191.11 sec = 1%. Speed 8.61 mB/s
All OK

[External compressor:xtool]
header = 0
packcmd = xtool.exe precomp { -moption} -c64mb -t100p - - <stdin> <stdout>
unpackcmd = xtool.exe decode -t75p - - <stdin> <stdout>

[External compressor:ff7rcc]
header = 0
packcmd = xtool.exe precomp -mccff7r -c1gb -d1 -lm -t100p - - <stdin> <stdout>
unpackcmd = xtool.exe decode -t75p - - <stdin> <stdout>

KaktoR

14-12-2022, 07:59

Witcher 3 v4.0 Next Gen Update

Still the same algo from before
https://fileforums.com/showthread.php?p=498295&highlight=witcher#post498295

-mxtool:witcher3+xtool:zlib+xtool:lz4hc,l9

Compressed 1 file, 286,247,872 => 1,249,481,045 bytes. Ratio 436.50%
Compression time: cpu 0.41 sec/real 51.41 sec = 1%. Speed 5.57 mB/s
All OK

Extracted 1 file, 1,249,481,045 => 286,247,872 bytes. Ratio 436.50%
Extraction time: cpu 0.59 sec/real 37.01 sec = 2%. Speed 7.73 mB/s
All OK

KaktoR

15-12-2022, 02:01

Grand Theft Auto V
Build 10107212 (13 December 2022)

17:17:06 - Selected ARC/DS method for Data1a-01.bin was: bpk+xtool:mreflate:db:-dedup=3+4x4:lzma
17:17:06 - Selected ARC/DS method for Data1b-01.bin was: xtool:db:-dedup=3+4x4:lzma
-----------------------------------------------------------------------------------------
20:20:08 - Overall input size: 108.55 GB
20:20:08 - Overall output size: 65.61 GB (Ratio 60.45%)
20:20:08 - Overall conversion time: 03:02:59

Data1a
common.rpf
x64*.rpf
update\*.rpf

Edit: Use srep instead of xtool dedup. At least for me xtool just crashes in the middle, even with low settings. With srep cls it is working fine.

Gehrman

15-12-2022, 12:01

KaktoR

16-12-2022, 09:57

Elden Ring
v1.08.1

17:21:11 - Selected ARC/DS method for Data1a-01.bin was: xtool1+xtool2+xtool:db:-dedup=3+4x4:lzma
17:21:11 - Selected ARC/DS method for Data1b-01.bin was: xtool:db:-dedup=3+4x4:lzma
-------------------------------------------------------------------------------------------------------------------------------------------
18:57:27 - Overall input size: 48.07 GB
18:57:27 - Overall output size: 38.88 GB (Ratio 80.88%)
18:57:27 - Overall conversion time: 01:36:15

Gehrman

17-12-2022, 11:24

(EN) Without 4KMovie,SignLanguage
xtool:c256mb:mpreflate:mzlib+srep:m3f:l512+4x4:lzm a:a1:mfbt4:32m:fb273:mc10000:lc8

109.98 GB → 180 GB → 120 GB → 67.6 GB

CT: 7H 35M
DT: 52M

KaktoR

25-12-2022, 07:17

Crisis Core - Final Fantasy VII - Reunion
v1.01

13:45:17 - Selected ARC/DS method for Data1a-01.bin was: xtool:c1gb:d1:lm:mccff7r+xtool:mkraken,l7:core_2.9 .5+xtool:mpreflate:db:dd3+4x4:lzma
13:45:17 - Selected ARC/DS method for Data1b-01.bin was: xtool:db:dd3+4x4:lzma
-------------------------------------------------------------------------------------------------------------------------------------------
14:42:31 - Overall input size: 18.17 GB
14:42:31 - Overall output size: 14.94 GB (Ratio 82.21%)
14:42:31 - Overall conversion time: 00:56:26
14:42:31 - Overall conversion media: 1 custom media
-------------------------------------------------------------------------------------------------------------------------------------------
14:59:20 - Overall decompression time: 00:16:32
14:59:20 - All files have been successfully extracted!

dixen

25-12-2022, 08:16

Some tests MK11

DISMEMBERMENTSHARED_ScriptAssets.psf

Old version (oodle lzna). Need unreal.dll & oo2core_5_win64.dll from game folder

Compressed 1 file, 959,141,047 => 2,044,941,730 bytes. Ratio 213.21%
Compression time: cpu 0.17 sec/real 101.97 sec = 0%. Speed 9.41 mB/s
All OK

Tested 1 file, 2,044,941,730 => 959,141,047 bytes. Ratio 213.21%
Testing time: cpu 0.50 sec/real 88.30 sec = 1%. Speed 10.86 mB/s
All OK

New version (oodle kraken, last patch from 15.12.2022). Need oo2core_5_win64.dll from game folder

Compressed 1 file, 1,052,382,310 => 2,005,105,169 bytes. Ratio 190.53%
Compression time: cpu 0.16 sec/real 36.26 sec = 0%. Speed 29.02 mB/s
All OK

Tested 1 file, 2,005,105,169 => 1,052,382,310 bytes. Ratio 190.53%
Testing time: cpu 0.50 sec/real 7.24 sec = 7%. Speed 145.43 mB/s
All OK

KaktoR

25-12-2022, 11:50

Immortals - Fenyx Rising
v1.3.4

16:18:24 - Selected ARC/DS method for Data1a-01.bin was: oo2recs:core_2.9.5+srep+4x4:lzma
16:18:24 - Selected ARC/DS method for Data1b-01.bin was: srep+4x4:lzma
-----------------------------------------------------------------------------------------
19:09:55 - Overall input size: 50.66 GB
19:09:55 - Overall output size: 23.68 GB (Ratio 46.73%)
19:09:55 - Overall conversion time: 02:51:28
-----------------------------------------------------------------------------------------
20:47:21 - Overall decompression time: 00:21:56
20:47:21 - All files have been successfully extracted!

Masquerade

28-12-2022, 11:10

Melty Blood: Type Lumina [Build 10190280]

xtool (-mreflate --dedup=5) + lolz

20.7GB ---> 4.9GB

Masquerade

30-12-2022, 10:18

Just an FYI, don't use XTool -mzlib+reflate on SnowRunner as it gives CRC fail upon decompression.

KaktoR

31-12-2022, 06:09

Hardspace Shipbreaker
v1.3.0

14:56:59 - Selected ARC/DS method for Data1a-01.bin was: xtool:munity,lz4hc,l12:lz4_193+xtool:db:dd3+4x4:lz ma
14:56:59 - Selected ARC/DS method for Data1b-01.bin was: xtool:db:dd3+4x4:lzma
-------------------------------------------------------------------------------------------------------------------------------------------
15:06:54 - Overall input size: 5.03 GB
15:06:54 - Overall output size: 2.33 GB (Ratio 46.38%)
15:06:54 - Overall conversion time: 00:09:49
-------------------------------------------------------------------------------------------------------------------------------------------
15:09:38 - Overall decompression time: 00:02:12
15:09:38 - All files have been successfully extracted!

BKR-TN

01-01-2023, 01:00

Europa Universalis IV Incl. All DLCs (GOG) v1.34.5.6
Compressed 25,670 files, 7,864,063,100 => 4,243,642,157 bytes. Ratio 53.96%
Compression time: cpu 36.48 sec/real 7572.58 sec = 0%. Speed 1.04 mB/s
All OK

Tested 25,670 files, 4,243,642,157 => 7,864,063,100 bytes. Ratio 53.96%
Testing time: cpu 13.08 sec/real 102.04 sec = 13%. Speed 77.07 mB/s
All OK

Initial size: 7.32 GB
Final size: 3.95 GB
2 cores, 4 threads
*.mp3, *.ogg and *.zip => xtool:db:dd3
*.wav and *.bmp => msc_tak+xtool:db:dd3
rest of the files => xtool:db:dd3+lolz:dtb1:d128:mtt1:mt4:mc1023

Note: There is not that much of gain with compressing mp3 files by mpz (saving ~100 MB), ogg files by oggre (~90 MB) and preflating zip files by xtool (~110 MB) considering how much slower the installer will be. Also, the "eu4.pdb" file will be ~36 MB after compressing.

kj911

02-01-2023, 03:25

Good compression scheme, methods packing from old XIII game *.hsc files? (And the complete game.) These files encrypted or missed any header data from detectable? MSC not detect reasonable compressibble raw PCM/WAV or MP3 audio data the files. OGGRE it says, no effect. Size reduction its abysmal 5-6-7% no less higher. (Tested in: XIII GOG Update 32942 version.)

*.hxc files finely compressibble in MSC(tak)+ZSTD methods packed to in ~210MB size. (26% ratio, or use any hard final compressor, ex. LOLZ, RZ, more.)

UPD: SREP+DLZ its 2* better than MSC+ZSTD methods. Thanks hints the any repackers from net. :D

Compressed 59 files, 849,601,186 => 103,557,504 bytes. Ratio 12.19%
Compression time: cpu 2.20 sec/real 497.64 sec = 0%. Speed 1.71 mB/s
All OK

Testing now MSC+SREP+LOLZ combo.

Masquerade

08-01-2023, 03:25

Watch_Dogs

Full game will ALL languages and ALL DLC content.

Redists and UPlay installer ripped - they are completely unneeded if NoCD is applied.

Requires latest XTool with Watch_Dogs Database (https://www.fileforums.com/showpost.php?p=499631&postcount=337) and XCompress Plugin (https://fileforums.com/showpost.php?p=495860&postcount=237) thanks to Razor12911.

Requires QuickBMS (https://aluigi.altervista.org/quickbms.htm) thanks to Luigi Auriemma and Dunia Engine script (http://aluigi.altervista.org/bms/fat2_fat3.bms).

Thanks to Nicolas Comerci for precomp stdio fork (https://fileforums.com/showthread.php?t=105410).

Do not use Oggre on the vorbis audio. Encoding is fine, but the decoder will crash.

Conversion Structure:

Archive 1: *.DAT (all except sound.dat, language dats and videos.dat)
Archive 2: All remaining files (binaries and misc documents, *.FAT except language fats)
Archive 3: Bink Videos listed below (pre-packed with BPK)
Archive 4: Videos.dat with BPK compatible videos removed
Archive 5: Sound.dat + English FATs/DATs
Remaining Language Archives: FATs/DATs for extra Non-English languages.

Compress with the following:

Archive 1:
xtool_mwatchdogs_dedup5+lolz
Archive 2:
precomps (https://fileforums.com/showthread.php?t=105410)+srep+lolz
Archive 3:

Extract the videos.dat with QuickBMS. Use XTool erase command to erase the following videos from the dat:
0000000000000000.kb2
0000000000000001.kb2
0000000000000003.kb2
0000000000000005.kb2
0000000000000007.kb2
000000000000000a.kb2
000000000000000b.kb2
000000000000000d.kb2
000000000000000f.kb2
0000000000000012.kb2
0000000000000018.kb2
000000000000001a.kb2
000000000000001c.kb2
000000000000001d.kb2
000000000000001f.kb2
0000000000000021.kb2
0000000000000022.kb2
0000000000000026.kb2
0000000000000028.kb2
0000000000000029.kb2
000000000000002b.kb2
000000000000002c.kb2
000000000000002e.kb2
0000000000000033.kb2
0000000000000035.kb2
0000000000000038.kb2
000000000000003b.kb2
000000000000003e.kb2
000000000000003f.kb2
0000000000000040.kb2
0000000000000041.kb2
0000000000000042.kb2
0000000000000045.kb2
0000000000000048.kb2
0000000000000049.kb2
000000000000004a.kb2
000000000000004b.kb2
000000000000004c.kb2
000000000000004d.kb2
000000000000004e.kb2
0000000000000051.kb2
0000000000000052.kb2
0000000000000053.kb2
0000000000000054.kb2
0000000000000055.kb2
0000000000000057.kb2
000000000000005e.kb2
000000000000005f.kb2
0000000000000060.kb2
0000000000000061.kb2
000000000000006b.kb2
000000000000006c.kb2
0000000000000072.kb2
0000000000000073.kb2
0000000000000075.kb2
0000000000000076.kb2
0000000000000078.kb2
000000000000007b.kb2
000000000000007e.kb2
0000000000000080.kb2
0000000000000081.kb2
0000000000000084.kb2
0000000000000086.kb2
0000000000000087.kb2
0000000000000089.kb2
000000000000008b.kb2
000000000000008e.kb2
0000000000000094.kb2
0000000000000095.kb2
0000000000000096.kb2
000000000000009a.kb2
000000000000009b.kb2
00000000000000a2.kb2
00000000000000a5.kb2
00000000000000a8.kb2
00000000000000aa.kb2
00000000000000ad.kb2
00000000000000ae.kb2
00000000000000b0.kb2
00000000000000b6.kb2
00000000000000b7.kb2
00000000000000b8.kb2
00000000000000ba.kb2
00000000000000bb.kb2
00000000000000bd.kb2
00000000000000c0.kb2
00000000000000c2.kb2
00000000000000c3.kb2
00000000000000ca.kb2
00000000000000cc.kb2
00000000000000cf.kb2
00000000000000d3.kb2
00000000000000d5.kb2
00000000000000d7.kb2
00000000000000d8.kb2
00000000000000da.kb2
00000000000000db.kb2
00000000000000dc.kb2
00000000000000e5.kb2
00000000000000e9.kb2
00000000000000ea.kb2
00000000000000eb.kb2
00000000000000ec.kb2
00000000000000f1.kb2
00000000000000f2.kb2
00000000000000f3.kb2
00000000000000f6.kb2
00000000000000fa.kb2
00000000000000fd.kb2
00000000000000fe.kb2
0000000000000101.kb2
0000000000000102.kb2
0000000000000104.kb2
000000000000010c.kb2
000000000000010d.kb2

Pack those videos with BPK (use CLS if you want, I used BPK executable version since I could MT it with mparallel (https://github.com/lordmulder/MParallel/releases/tag/1.0.4)). Those above videos don't require patching. The other videos from the dat get motion_data warning upon packing so packing those is impossible.

Pack Archive 3 with srep.

Archive 4:
srep+LZMA2

Archive 5:
rz

Remaining Language Archives:
xtool_mwatchdogs+rz

(don't use xtool dedup since rz has built in dedup)

Final Size

-------------------------
| Original Size: 28.9GB |
-------------------------
| Game + EN | 12.12GB |
-------------------------
| + DE | +1.21GB |
| |
| + ES | +1.21GB |
| |
| + FR | +1.25GB |
| |
| + IT | +1.20GB |
| |
| + JP | +1.32GB |
| |
| + PTBR | +1.20GB |
| |
| + RU | +1.21GB |
| ----------------------|
| TOTAL | 20.72GB |
-------------------------

KaktoR

14-01-2023, 07:53

One Piece Odyssey

15:46:00 - Selected ARC/DS method for Data1a-01.bin was: xtool:c128mb:mue4,k0x6005F03B0D2DABCD75892C80958FF B00CBA5437AD37EB4BFCD5960FFE0C4E12D+xtool:mpreflat e+xtool:db:dd3+4x4:lzma
-------------------------------------------------------------------------------------------------------------------------------------------
16:37:14 - Overall input size: 29.31 GB
16:37:14 - Overall output size: 24.72 GB (Ratio 84.34%)
16:37:14 - Overall conversion time: 00:51:12

Masquerade

14-01-2023, 08:50

One Piece Odyssey

Used CLS-UE4D library by Shegorat, but you can also use XTool by Razor12911 (method here (https://www.fileforums.com/showpost.php?p=499782&postcount=3332) by KaktoR).

The file "pakchunk10-WindowsNoEditor.pak" contains only CriMovie USM files so you can just Store this file or decrypt and dedup.

Remaining files:

ue4d:key=6005F03B0D2DABCD75892C80958FFB00CBA5437AD 37EB4BFCD5960FFE0C4E12D+xtool_zlib_dedup5+lolz

Start Size: 29.2GB
Final Size: 22.7GB

KaktoR

15-01-2023, 05:08

Aquatico

13:26:49 - Selected ARC/DS method for Data1a-01.bin was: srep+lolz:d64:mtt1:mt12:mc1023
13:26:49 - Selected ARC/DS method for Data1b-01.bin was: msc_tak+srep+4x4:lzma
13:26:49 - Selected ARC/DS method for Data1c-01.bin was: srep+4x4:lzma
-------------------------------------------------------------------------------------------------------------------------------------------
14:04:23 - Overall input size: 8.35 GB
14:04:23 - Overall output size: 2.00 GB (Ratio 23.99%)
14:04:23 - Overall conversion time: 00:37:32
-------------------------------------------------------------------------------------------------------------------------------------------
14:07:58 - Overall decompression time: 00:03:15
14:07:58 - All files have been successfully extracted!

Data1a = *.mdl|*.dds|*.stl
Data1b = *.bmp|*.wav|*.mp3
Data1c = rest of the files

L33THAK0R

15-01-2023, 16:51

Had a quick look so maybe I'm missing something obvious but are there any lossless compressors that support WaveForm "MS ADPCM" formatted audio? MSC only supports PCM formatted samples, and I got a feeling some substantial gains could be had if this format is able to be properly processed across a range of titles.

Masquerade

16-01-2023, 00:48

No afaik. ADPCM is a lossy format. You could try looking into this: https://monkeysaudio.com/

But it works in a similar way to Toms Lossless Audio Kompressor so then again it may do nothing.

L33THAK0R

18-01-2023, 02:22

All results include the installer's weight (usually between 4 - 9mb)

Avatar: The Game, James Cameron’s

3.31 GiB >>> 1.98 GiB

xtool 0.6.5 + srep + lolz

Batman: Arkham Knight

51.75 GiB >>> 26.2 GiB (Incl. Optional Videos + Lossless Videos) | 25.3 (Incl. Lossy Videos)

Note: In order to properly create lossy videos you have to extract the cue points from the appropriate USMs'. This process isn't tricky and only takes around a minute in an ideal setup however calculating the offsets and translating them into usable integers can take a hot minute. As I'm sure those who have done a similar job are aware this process can't be automated due to the lack of consistency that cue timings are stored within their appropriate headers, this was definitely more so a proof of concept considering the fact there are ~880 videos that require cue point decoding.

xtool (I forgot to document the version I used, since I did this one back in mid 2020) + uelr + srep + lolz

Far Cry 2 (Fortunes Edition)

3.27 GiB >>> 1.62 GiB

delzorec + srep + lolz

Just Cause 2

4.08 GiB >>> 1.00 GiB

Note: Final size could've been smaller (~950MiB) but I included a couple dozen QoL fixes that added a bit of weight in additional texture data.

xtool 0.3.21 + msc + srep + lolz

Mad Max

32.88 GiB >>> 3.55 GiB (Incl. Lossless Videos) | 3.50 GiB (Incl. Lossy Videos)

Note: Lossy videos were definitley just an experiement, 1080p Bink2 videos rescaled to 720p Bink1, even at 99% the target filesize look like dog-shit. Honestly just wanted to see how small I could get things. There is potential room for improvement if someone can pack the ~900 MiB of dialogue thats packed into one of the Banks, but it's encoded in a format that I can't seem to process, or even losslessly transform to and from a packable format.

xtool 0.3.21 + oggre + srep + lolz

The Surge

21.78 GiB >>> 3.86 GiB (720p Lossless Videos) | 3.72 GiB (720p Lossy Videos)

Note: 1080p + 720p videos both exist in the original game files, I just chose to present 720p for the results as its the smallest, valid, lossless size. Use a LZ4 library to recompress, I got pretty similar results accross a good dozen or so libraries, but some streams were still dropped, I'd encourage people to investigate further but I honestly spent too much time on this series for me to justify spending any more time testing is tricky.

xtool 0.3.21 + oggre + srep + lolz

-----------------------------------------------

I would post more, but these are just the most interesting results I've gotten. I've managed to beat a fair few records posted both here and by other repackers on their own websites but only by using more modern methods and by managing to process troublesome blocks of data by patching out the offending data.

Masquerade

18-01-2023, 08:00

@L33THAK0R
Would love it if you could make a post detailing more about the Cue points in the USM videos. I believe it would truly be a valuable resource.

Carldric Clement

20-01-2023, 05:02

Original Size: 6.82 GB
Compressed Size: 1.68 GB

Option:
* JPG File = packjpg
* OGG File = oggre:s3
* Wav File = tta
* Rest files = xtool:zlib:reflate+srep:m3f:a2+exe2+7z:d128m:mt2

KaktoR

20-01-2023, 12:11

Sid Meier's Civilization VI
v1.0.12.31 (19 January 2023), all DLC, OST ripped

19:27:51 - Selected ARC/DS method for Data1a-01.bin was: bpk
19:27:51 - Selected ARC/DS method for Data1b-01.bin was: xtool:db:dd3+lolz:d32:mtt1:mt12:fba4096:mc2
19:27:51 - Selected ARC/SPLIT method for Data1c-01.bin was: xtool:db:dd3+lolz:d32:mtt1:mt12:fba4096:mc2/$deflatepack=xtool:mpng:d1+xtool:mpreflate+xtool:d b:dd3+lolz:d32:mtt1:mt12:fba4096:mc2/$oggrepack=oggre
-----------------------------------------------------------------------------------------
21:02:04 - Overall input size: 21.36 GB
21:02:04 - Overall output size: 8.13 GB (Ratio 38.08%)
21:02:04 - Overall conversion time: 01:34:11

Data1a = *.bk2 (exclude "DLC\GranColombia_Maya\Platforms\Windows\Movies\Gra nColombia_Maya_DefeatClimate.bk2" or use bink patch method for this single file)
Data1b = *.blp
Data1c = rest of the files

PS: I used wemtool by Masquerade for the wem's

kuyhaa

22-01-2023, 07:23

hi

i try precomp Steel Division 2

use xtool reflate but still got CRC error when decomp

any info for solve ?

thanks

shazzla

22-01-2023, 07:33

Exclude those files from precomp process. I did it,too.

Gehrman

22-01-2023, 22:31

hi

i try precomp Steel Division 2

use xtool reflate but still got CRC error when decomp

any info for solve ?

thanks

Try with compatibility with Windows 7
ECHO OFF
Reg.exe add "HKCU\Software\Microsoft\Windows NT\CurrentVersion\AppCompatFlags\Layers" /v "%~dp0Setup.exe" /t REG_SZ /d "~ DPIUNAWARE WIN7RTM" /f
START Setup.exe
Reg Delete "HKCU\Software\Microsoft\Windows NT\CurrentVersion\AppCompatFlags\Layers" /v "%~dp0Setup.exe" /f
Exit

Or use XTool 0.2. (It works well with ZLib's strange streams.):D

Gehrman

22-01-2023, 22:56

30.36 GB (EN)
Sound*
msc:raw=1+srep:m3f:l512+4x4:lzma:a1:mfbt4:32m:fb27 3:mc10000:lc8

*.DAT
xtool (https://fileforums.com/showpost.php?p=481382&postcount=1):c256mb:mfc4 (https://fileforums.com/showpost.php?p=498324&postcount=48):dd3+lolz2:d32:dm00:fba0:gm20:mt8:mtt 1:mc128

Other Files
Srep+LZMA

Final Size: 12.2 GB

L33THAK0R

22-01-2023, 23:15

27.30 GiB >>> 10.9 GiB (Incl. Lossless Videos + Ripped Startup Videos, weighing in at ~40 MiB)

DELZORec + srep + lolz

Note: BPK can't be used as these are the "g" revision of the Bink2 codec. Ripped files include any included redists and non-English loc., as well as the startup video.

It's interesting to see that @Gehrman managed to get MSC working with the by just using the RAW detection, I'll give it a shot myself to see if any further gains can be had.

Gehrman

23-01-2023, 00:22

L33THAK0R

23-01-2023, 01:29

The minimum game size is 30 GB
Most likely your version is without multiplayer files:rolleyes:

Ah thanks for that, with multiplayer files I got a final size of 11.5 GiB (the MP files clocked in at ~600 MiB). It's possible and I'd wager likely a smaller size can be achieved, should both the main + multiplayer files be packed together, but I had the multiplayer files as an optional component.

For anyone interested, with MSC I only saw a ~9 MiB reduction. Upon testing I wasn't able to successfully decompress these files. It's hard to say whether this is within variance of running the data through LOLZ, or pure gains due to MSC, however since I wasn't able to get a valid result, I don't think I can comment much on the matter.

L33THAK0R

23-01-2023, 20:34

Hi all,

Does anyone know if there are any tools/methods to ID Oodle streams? The only means I know of, is identifying the 0x8c byte in a given sample, but at least in the sample "*.lodpack", "*.texpack" and "*.wad" files that I tested against, I had no luck in finding any streams, when running the sample separately through oo2reck/m/s, and against 3 different "oo2core_7_win64" libraries.

The reason for my testing is this off-hand note in the readme (https://github.com/HitmanHimself/GOWTool) for one of the more popular modding/unpacking suites for the title, which I was using to determine if any additional assets could be ripped and packed separately for further gains.

Razor12911

23-01-2023, 21:24

https://fileforums.com/showthread.php?t=102453

This comes with oodle scanner, just drag and drop file.

Masquerade

23-01-2023, 23:46

Isn't the 2018 God Of War a PlayStation exclusive? That explains why its using Oodle.

I know the PC re-release in 2022 uses no compression.

L33THAK0R

24-01-2023, 00:02

Isn't the 2018 God Of War a PlayStation exclusive? That explains why its using Oodle.

I know the PC re-release in 2022 uses no compression.

I feel like a right rube again, I contacted the developer and got a message back, just before reading the responses here, the developer also confirmed there isn't any compression used, if I'd taken the time to read through the github release notes I would've seen they officially noted this.

It seems odd the developers wouldn't use Oodle on the PC release given how many folk still have older and slower hardware akin to a PS4's specs.

Masquerade

24-01-2023, 00:24

It seems odd the developers wouldn't use Oodle on the PC release given how many folk still have older and slower hardware akin to a PS4's specs.
Perhaps I am missing the point, but wouldn't it make more sense for the developers NOT to use Oodle for the PC release? Older hardware would offer poor decompression speed compared to being able to just read game data from disk straight into memory and have it ready to use.

L33THAK0R

24-01-2023, 01:44

Perhaps I am missing the point, but wouldn't it make more sense for the developers NOT to use Oodle for the PC release? Older hardware would offer poor decompression speed compared to being able to just read game data from disk straight into memory and have it ready to use.

I was thinking more so about read operations from the disk being less intensive if the data was compressed but yes I'll admit I wasn't thinking about the fact it'd be more intensive on a user's memory and processor!

kuyhaa

24-01-2023, 04:24

Or use XTool 0.2. (It works well with ZLib's strange streams.):D

i try still got error bro, stuck in decomp, now i excluded of error files

Gehrman

24-01-2023, 15:40

i try still got error bro, stuck in decomp, now i excluded of error files

Sorry, the version is wrong
You should use XTOOL 1908 R3 (https://mega.nz/folder/03BkjKQA#WpTxyngxucC-ToBRK9xbkw/file/cvJwwKCb).

The "Xatab Repack (http://rutor.info/torrent/704988/steel-division-2-total-conflict-edition-v-46442-dlcs-2019-pc-repack-ot-xatab)" also has used the XTOOL 1908 R3 to pre-compress the game.

L33THAK0R

25-01-2023, 02:29

xtool 0.3.21 + srep + lolz

(Non-Eng. Loc. Ripped)

4.97 GiB >>> 2.58 GiB

Note: Perhaps others will have better luck than me but every bank dumping utility I tried to run this title's "*.banks" through failed on every account.

Masquerade

25-01-2023, 05:20

Note: Perhaps others will have better luck than me but every bank dumping utility I tried to run this title's "*.banks" through failed on every account.

Use the FSB5 quickbms script.

+ the devs included a full copy of the movies folder inside the largest .FUK file so you can XTool erase it.

L33THAK0R

25-01-2023, 05:47

Use the FSB5 quickbms script.

+ the devs included a full copy of the movies folder inside the largest .FUK file so you can XTool erase it.

Ah cheers I'll check out the quickbms script, I noted the duplicate movies but I'm fairly sure srep picked them up. I might be missing something but is there a point in blanking out the duplicates, since it'd still be the same amount of data written to disk.

Now that I think about it I didn't actually test to see if deleting the loose files would cause the packed duplicates to be used in their place, or if the loose files were used at all, I'll definitely have to have another quick crack at this one.

Gehrman

26-01-2023, 00:49

Game Size: 17.7 GB

Method #1
xtool:d1:c256mb:mgwtokyo+oo2reck+srep:m3f:l512+4x4 :lzma:a1:mfbt4:32m:fb273:mc128:lc8
CT: 3H
DT: 30M

Repck Size:14.3 GB

__________________________________________________ ______

Method #2

xtool:c256mb:d1:mgwtokyo:mkraken:dd3+4x4:lzma:a1:m fbt4:32m:fb273:mc10000:lc8

CT: 2H
DT: much longer + high memory usage
Repck Size:14.3 GB

Gehrman

26-01-2023, 18:24

Game Size: 17.06 GB

Rip:
MID\Content\Movies\MID_Interview.Webm
MID\Content\Movies\MID_Interview.mov

oo2reck+srep:m3f:l512+4x4:lzma:a1:mfbt4:32m:fb273: mc128:lc8

CT: 2H
DT 40M

Repack Size: 13.7 GB

Masquerade

27-01-2023, 07:27

Handyman Corporation

Generate a database for Oodle Kraken using "unreal_tournament_4_0.4.27c_paks_only.bms".

Don't use oo2reck - it seems to have a memory leak during decompression (eating up all your ram) and is insanely slow.

For the pak file, use your database with XTool dedup (requires srep.exe) and add lolz.

For the rest use rz.

2GB --> 1.3GB

L33THAK0R

28-01-2023, 07:57

Tools used:

oggre + msc + srep + lolz

Note applicable for all titles: MSC does work on a majority of the .wav/.mp3 loose files as well as the container files used in some titles, such as the "LEGO Harry Potter" titles as well as the "LEGO Middle-Earth" titles, however it's recommend to do a bulk test on all applicable files as there are a fair few instances of CRC-mismatches and there isn't an easy means to identify this without testing. The same can be said for all applicable Vorbis audio. Newer waveform audio in the "*.ADP" container format cannot be processed currently. I haven't tested to see if the engine will accept re-coded audio in the standard and compressible PCM format, given the fact the engine used for all TT titles (bar maybe the Skywalker Saga, given its a fairly considerable engine revision) is building upon the foundations of the previous iteration, it might (?) be possible to do so. Same can probably be done for the various incompatible "*.ogg" files given the format is the same, and some chunks just don't play well with Oggre.

Files ripped: Non-English Loc., unused left-over files from the console releases & left-over demo-exclusive files.

Final notes:

- All tests were based off the latest clean GOG/Steam RIPs.

- Some decompressed sizes may be slightly larger than what others may experience as I tend to include any and all QoL fixes/mods as included offerings, and only include larger less critical mods like the soundtrack replacement mods for the various licensed titles by TT, as optional component offerings.

///////////////////////////////

LEGO City Undercover
15.20 GiB >>> 5.07 GiB

LEGO Harry Potter: Years 1-4
8.54 GiB >>> 1.77 GiB

LEGO Harry Potter: Years 5-7
12.20 GiB >>> 2.73 GiB

LEGO Indiana Jones: The Original Adventures
5.77 GiB >>> 1.41 GiB

LEGO Indiana Jones 2: The Adventure Continues
7.15 GiB >>> 1.50 GiB

LEGO Jurassic World
20.70 GiB >>> 4.52 GiB

LEGO MARVEL Super-Heroes
9.38 GiB >>> 2.30 GiB

LEGO MARVEL Avengers
22.30 GiB >>> 5.30 GiB

LEGO MARVEL Super-Heroes 2
26.73 GiB >>> 7.43 GiB

LEGO Middle-Earth: The Lord of the Rings
8.07 GiB >>> 2.24 GiB

LEGO Middle-Earth: The Hobbit
8.75 GiB >>> 2.69 GiB

LEGO Movie Videogame, The
9.79 GiB >>> 3.89 GiB

LEGO Movie Videogame, The 2
15.00 GiB >>> 3.65 GiB

LEGO Ninjago Movie Videogame, The
29.60 GiB >>> 6.09 GiB

LEGO Pirates of the Caribbean: The Videogame
11.70 GiB >>> 2.17 GiB

LEGO Star Wars - The Complete Saga
9.39 GiB >>> 1.61 GiB

LEGO Star Wars III - The Clone Wars
9.78 GiB >>> 2.10 GiB

LEGO Star Wars: The Force Awakens
28.10 GiB >>> 6.61 GiB

LEGO The Incredibles
22.70 GiB >>> 4.61 GiB

LEGO Worlds
4.36 GiB >>> 1.24 GiB

LEGO DC Batman: The Videogame
8.02 GiB >>> 1.32 GiB

LEGO DC Batman 2: DC Super Heroes
5.62 GiB >>> 1.26 GiB

LEGO DC Batman 3: Beyond Gotham
12.10 GiB >>> 3.33 GiB

LEGO DC Super-Villains
26.14 GiB >>> 6.62 GiB

L33THAK0R

28-01-2023, 08:04

KaktoR

28-01-2023, 08:17

Quick question for anyone who's given "Days Gone" a go, has anyone been able to process both the oodle and zlib streams present in the title's "*.pak" archives? I've been using oo2reck (since xtool seems to only be able to use one oodle library and both a "oo2core_7*" & "oo2core_8*" are required to grab all (or at least the majority) of oodle streams. I've tried using both "oo2reck + xtool_0.3.21 (zlib+reflate) + srep + lolz" as well as "xtool_0.3.21 (zlib+reflate) + oo2reck + srep + lolz" but to no avail, interestingly the latter combination results in ~15mins of memory usage with srep and disk write operations, but unfortunately also results in a -11 (corrupt archive) FreeArc decompression error code.

If anyone has had any luck or knows how to get xtool to use both libraries I'd be very much interested in any tips anyone would be willing to offer. Attached are the libraries I've found to be the most effective.

Use two xtool instances with --oodle option in arc.ini like this:

[External compressor:xtool1]
header = 0
packcmd = xtool.exe precomp -mkraken -c64mb -t100p --oodle=oo2core_7_win64.dll - - <stdin> <stdout>
unpackcmd = xtool.exe decode -t75p - - <stdin> <stdout>

[External compressor:xtool2]
header = 0
packcmd = xtool.exe precomp -mkraken -c64mb -t100p --oodle=oo2core_8_win64.dll - - <stdin> <stdout>
unpackcmd = xtool.exe decode -t75p - - <stdin> <stdout>

Then use xtool1+xtool2+xtool:zlib

Gehrman

28-01-2023, 17:03

Game Size: 47/4 GB

Method#1
xtool:c256mb:mzlib:mpreflate:dd3+4x4:lzma:a1:mfbt4 :32m:fb273:mc128:lc8
30.5 GB

Method#2

xtool:mreflate:dd3+4x4:lzma:a1:mfbt4:32m:fb273:mc1 024:lc8
32.3 GB

KaktoR

29-01-2023, 03:18

A Plague Tale: Requiem
v1.4.0.0, ripped languages from soundbanks

10:58:07 - Selected ARC/DS method for Data1a-01.bin was: xtool:mreflate+xtool:db:dd3+4x4:lzma
10:58:07 - Selected ARC/DS method for Data1b-01.bin was: xtool:db:dd3+4x4:lzma
-------------------------------------------------------------------------------------------------------------------------------------------
12:16:24 - Overall input size: 47.48 GB
12:16:24 - Overall output size: 29.86 GB (Ratio 62.89%)
12:16:24 - Overall conversion time: 01:18:16

KaktoR

29-01-2023, 05:07

Dead Space Remake

Library loaded: zstd_145\libzstd.dll

Streams: 7656/7657
Time: 00:00:16 (00:01:33)
Memory: 177 MB (177 MB)

Results: 230 MB >> 478 MB

Edit:

20:14:55 - Selected ARC/DS method for Data1a-01.bin was: xtool:mzstd:zstd_145+xtool:db:dd3+4x4:lzma
20:14:55 - Selected ARC/DS method for Data1b-01.bin was: xtool:db:dd3+4x4:lzma
-------------------------------------------------------------------------------------------------------------------------------------------
21:32:58 - Overall input size: 36.32 GB
21:32:58 - Overall output size: 26.31 GB (Ratio 72.45%)
21:32:58 - Overall conversion time: 01:18:01
-------------------------------------------------------------------------------------------------------------------------------------------
21:57:32 - Overall decompression time: 00:23:12
21:57:32 - All files have been successfully extracted!

L33THAK0R

30-01-2023, 16:50

Seems like no matter how I go about attempting to pre-compress this title, I'm unable to pick up all the oodle streams, as a last resort I'm going to try unpacking the core 2 "*.pak" archives as a dirty solution and see if any gains can be had with this approach. Attached image displays my last two attempts to process all the Oodle streams with XTool (0.6.9).https://i.imgur.com/IPQmy0M.png

Razor12911

30-01-2023, 20:30

https://fileforums.com/showpost.php?p=492058&postcount=101

Masquerade

01-02-2023, 00:27

PowerWash Simulator [v1.1]

Use SREP v3.2 due to crash on newer versions.
Don't use XTool built in deduplication.

xtool_unity+srep+lolz

6.6GB ---> 3.3GB

L33THAK0R

01-02-2023, 08:41

A while back I took a shot at a title, "Trek to Yomi", which contains ~260mb of H.264 video, encoded at an exceedingly silly bit-rate. Linked here is a sample of my re-encoding (https://mega.nz/folder/yS4RiCpD#-d9RdFFPUvzi5_YmsUVBFg).

The below figure should speak for itself, but after tuning ffmpeg a fair bit, I managed to get quite an extraordinary outcome, coupled with srep & lolz, a result of a 328x improvement with a near imperceivable loss in fidelity.

https://i.imgur.com/BgwlKUG.png

Has anyone else had any similar success stories? It's got me wondering how and why these algorithms were able to process these lower bit-rate frames in such an efficient manner compared to the original lossless files.

(And yes before anyone asks I did verify the archives to be valid)

shazzla

01-02-2023, 10:59

For the records:

I recommend use CRF (lets say ,its constant quality) instead of targeting bitrate.

L33THAK0R

01-02-2023, 17:09

For the records:

I recommend use CRF (lets say ,its constant quality) instead of targeting bitrate.

Constant Rate Factor (CRF) encoding is pretty much mandatory I'd argue for producing any quality lossy videos. You can still use the available switches cap the maximum bit-rate as well manipulating the VBV buffer. I'd also recommend testing the various available tuning profiles (not to be confused with H.264 profiles or ffmpeg encoding presets) within ffmpeg especially if you're dealing with 2D video, as you can get some noticeable ratio improvements for a slightly longer encoding time.

shazzla

02-02-2023, 00:17

Then give a shot for

Selur's Hybrid. Not ffmpeg.

I used it for encode TBs of x265 movies..

L33THAK0R

02-02-2023, 07:54

xtool (0.3.21) + srep + lolz

42.26 GiB >>> 19.4 GiB

Ripped: Non-Eng. Loc., ~10 MiB of startup videos

Note: the entire "sfpaks" folder can be deleted, I'm surprised others that have tackled this title don't seem to know it just contains CRC-perfect duplicates of data contained within the 2 primary "*.pak" archives. In my conversion I unpacked the archives to remove the need for oodle pre-compression, honestly there are only ~200 MiB of zlib/reflate streams (See Figure 1) to be processed which only results in a ~60 MiB improvement (See Figure 2) over just sticking with srep + lolz. In this form there are only ~6 MiB of Oodle streams to be processed.

Interestingly enough I found on my system, when testing for ~1hr to make sure everything worked properly, I encountered fewer stutters during game-play when using the unpacked files.

Figure 1
https://i.imgur.com/Knsc4jq.png

Figure 2
https://i.imgur.com/Hp0zUv6.png

dixen

08-02-2023, 07:29

Hogwarts Legacy

Game used oodle kraken

Compressing pakchunk11-WindowsNoEditor.ucas 98%
Compressing 676,197,072 bytes with xtool.exe precomp -mkraken -c128mb -t100p $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
XTool is created by Razor12911

Streams: 19271/19280
Time: 00:00:58 (00:08:35)
Memory: 514 MB (514 MB)

100%
Errorlevel=0

Compressed 1 file, 676,197,072 => 2,228,651,670 bytes. Ratio 329.59%
Compression time: cpu 0.08 sec/real 72.49 sec = 0%. Speed 9.33 mB/s
All OK

Needed oo2core_8_win64.dll v2.8.14

KaktoR

08-02-2023, 09:22

@dixen
Also bink2 videos are kb2n, so bpk is useless here.

Damn, I guess next year at the least bpk will be useless entirely for new titles :(

Masquerade

08-02-2023, 09:28

Epic Games owns Unreal Engine and Rad Tools so its a free for all when it comes to Unreal Engine games using Bink Video and Oodle compression. The latest Bink version is KB2n so it's this year that BPK will be obsolete for newer titles.

-tara

12-02-2023, 04:15

Bright Memory: Infinite
*.pak
xtool(zlib)+srep+lolz
(58.60% Ratio)

*.png
precompmt+srep+lolz
(79.57% Ratio)

*.mp4
srep
(99.97% Ratio)

rest
razor
(16.73% Ratio)

Size Before - 10.8 GB After - 6.36 GB (58.88% Ratio)
CT: 6 hours and 20 minutes DT: 9 Minutes (On i5-4570s 4T/4C + 8GB RAM)
Fallout New Vegas Ultimate Edition
*.bsa
xtool(zlib)+srep+lolz
(54.5% Ratio)

FNVIntro.bik
bpk
(77.8% Ratio)

*.mp3
mpz (Multithread this with PMT/MTX)
(81.4% Ratio)

MailTitle.wav
msc (You can skip this however, its just 6 MB and can be passed to rz for simular result)

rest
razor
(60.1% Ratio)

Size Before - 9.4 GB After - 5.36 GB (57.02% Ratio)
CT: Don't have anymore but 5+ Hours DT: 8 Minutes (On i5-4570s 4T/4C + 8GB RAM)

Hike Isle
*.assets
*.resS
*.resource
(no extension)
srep+lolz
(12.17% Ratio)

rest
razor
(25.07% Ratio)

Size Before - 2.5 GB After - 321 MB (12.8% Ratio)
CT: 1 Hour and 45 Minutes DT: 1 Minute (On i5-4570s 4T/4C + 8GB RAM)

Inscryption
*.assets
*.resS
*.resource
(no extension)
srep+lolz
(49.13% Ratio)

rest
razor
(23.82% Ratio)

Size Before - 3.3 GB After - 1.6 GB (48.48% Ratio)
CT: 1 Hour and 38 Minutes DT: 2 Minutes (On i5-4570s 4T/4C + 8GB RAM)

Mirror's Edge Catalyst
Ripped - Trial executable

*.cas
xtlz4(v12)+srep+lolz
(72.3% Ratio)

rest
razor
(31.5% Ratio)

Size Before - 23 GB After - 16.5 GB (71.73% Ratio)
CT: 8 Hours and 40 Minutes DT: 12 Minutes (On i5-4570s 4T/4C + 8GB RAM)

Need for Speed: Payback – Deluxe Edition
This is compression of the full version with all DLCs and depots included

*.cas
xtlz4(v12)+srep+lolz
(~62% Ratio)

rest
razor
(20.58% Ratio)

Size Before - 27 GB After - 15.9 GB (58.88% Ratio)
CT: 10 Hours and 30 Minutes DT: 15 Minutes (On i5-4570s 4T/4C + 8GB RAM)

Papers, Please
*
razor

Size Before - 58 MB After - 24.4 (42.06% Ratio)
CT: 2 Minutes DT: 2 Seconds (On i5-4570s 4T/4C + 8GB RAM)
(This was done outside of Arc, the decompression is done with a simple bat)

PowerWash Simulator v1.1
*.bundle
xtool(unity plugin)+srep+lolz
(50.6% Ratio)

rest
razor
(37.1% Ratio)

Size Before - 6.6 GB After - 3.3 GB (50% Ratio)
CT: Lost data, sorry DT: 12 Minutes (On i5-4570s 4T/4C + 8GB RAM)

Sapper - Defuse The Bomb Simulator v1.0.18
*.assets
*.resS
*.resource
(no extension)
srep+lolz
(29.7%% Ratio)

rest
razor
(25.3% Ratio)

Size Before - 7.1 GB After - 2.1 GB (29.57% Ratio)
CT: Lost data, sorry DT: 3 Minutes (On i5-4570s 4T/4C + 8GB RAM)

Stray v1.4#227
Hk_project-WindowsNoEditor.pak
oo2reck+xtool(zlib)+srep+lolz
(70.2% Ratio)

rest
razor
(To not waste time with razor for bk2, pass them to srep+lz4/anything fast)

Size Before - 6.8 GB After - 4.7 GB (69.11% Ratio)
CT: Lost data, sorry DT: 26 Minutes (On i5-4570s 4T/4C + 8GB RAM)

Subnautica
Ripped - Soundtrack

*.assets
*.resS
*.resource
(no extension)
srep+lolz
(31.5% Ratio)

rest
razor
(79.9% Ratio)

Size Before - 7.4 GB After - 2.7 GB (36.45% Ratio)
CT: Lost data, sorry DT: 6 Minutes (On i5-4570s 4T/4C + 8GB RAM)

Super Meat Boy
rz/$gd=srep+lolz

Arc.groups:
$gd
*.dat

Size Before - 486 MB After - 315 MB (64.1% Ratio)
CT: Lost data, sorry DT: 10 Seconds (On i5-4570s 4T/4C + 8GB RAM)

Super Meat Boy Forever
rz/$gd=srep+lolz/$bik=bpk
(I used PMT on Razor here)

Arc.groups:
$gd
*.swfc
*.smbl
*.tga
*.rply

$bik
*.bik

Size Before - 5.2 GB After - 1.7 GB (32.4% Ratio)
CT: Lost data, sorry DT: 2 Minutes (On i5-4570s 4T/4C + 8GB RAM)

The Stanley Parable Ultra Deluxe
*.assets
*.resS
*.resource
(no extension)
srep+lolz
(33.83% Ratio)

*.mp4
lzma (~50mb smaller output compared to lz4/zstd)
(87.32% Ratio)

voice.deafult
srep
(99.87% Ratio)

rest
razor
(20.53% Ratio)

Size Before - 7.2 GB After - 2.7 GB (37.5% Ratio)
CT: 2 Hours and 40 Minutes DT: 4 Minutes (On i5-4570s 4T/4C + 8GB RAM)

PS: If this type of method sharing is not useful in the thread anymore, delete this reply. And if the format in which they're shared is a bit bulky and mods want reformat, I'm happy to oblige. This is just a few games I've done for the month :)

FitGirl

15-02-2023, 09:43

Returnal has Leviathan inside. Unpacking on 24-threads CPU takes 3 hours and saves only 4 GBs compared to pure srep+lolz, don't bother.

Butza

16-02-2023, 09:15

I am newbie. this is what it says to me when I use it Skipped VO_MEDIA.PC.PCK (Larger than 2147483647)

Make batch files for this, so you don't have to type all manually in cmd.

unpack.bat for quickbms
for %%f in (*.pck) do quickbms_4gb_files.exe -d -o -Y wwise.bms "%%f"
pause

xtool_erase.bat for xtool
xtool erase "VO_MEDIA.PC.PCK_extract\german" "VO_MEDIA.PC.PCK" test.xtl

This will delete all german voice files from the file.

PS: Make a backup of the file first ;)

Razor12911

16-02-2023, 16:39

Returnal has Leviathan inside. Unpacking on 24-threads CPU takes 3 hours and saves only 4 GBs compared to pure srep+lolz, don't bother.

If you are stubborn and still want to compress the game regardless for archival purposes then I suggest using the unreal engine plugin with the method -mue4:leviathan:l9 include -s parameter as well to skip verification this way you can skip precompression part of the process because its the slowest part of compressing this game.

Note: Skip works provided you have used the correct oodle library.

Compressed 1 file, 462,045,227 => 1,175,858,081 bytes. Ratio 254.49%
Compression time: cpu 0.44 sec/real 4.67 sec = 9%. Speed 99.03 mB/s

I am newbie. this is what it says to me when I use it Skipped VO_MEDIA.PC.PCK (Larger than 2147483647)

the original file must not be part of the extracted streams.

Butza

16-02-2023, 23:31

Did now the test.xtl file appeared, but the file itself did not change the size. what to do next with this file? file size does not decrease after erasing

Make batch files for this, so you don't have to type all manually in cmd.

unpack.bat for quickbms
for %%f in (*.pck) do quickbms_4gb_files.exe -d -o -Y wwise.bms "%%f"
pause

xtool_erase.bat for xtool
xtool erase "VO_MEDIA.PC.PCK_extract\german" "VO_MEDIA.PC.PCK" test.xtl

This will delete all german voice files from the file.

PS: Make a backup of the file first ;)

KaktoR

17-02-2023, 05:52

The file will not change the size, but it is better compressable because the erased files are just overwritten with zero bytes (00).

Butza

17-02-2023, 06:17

Thanks, got it)
The file will not change the size, but it is better compressable because the erased files are just overwritten with zero bytes (00).

KaktoR

17-02-2023, 13:59

If you are stubborn and still want to compress the game regardless for archival purposes then I suggest using the unreal engine plugin with the method -mue4:leviathan:l9 include -s parameter as well to skip verification this way you can skip precompression part of the process because its the slowest part of compressing this game.

Note: Skip works provided you have used the correct oodle library.

Compressed 1 file, 462,045,227 => 1,175,858,081 bytes. Ratio 254.49%
Compression time: cpu 0.44 sec/real 4.67 sec = 9%. Speed 99.03 mB/s

Big welp

Extracted 1 file, 1,175,857,894 => 462,045,227 bytes. Ratio 254.49%
Extraction time: cpu 0.16 sec/real 260.22 sec = 0%. Speed 1.78 mB/s
All OK

I guess extraction time for the whole thing should take a half day :D

Razor12911

17-02-2023, 14:20

Big welp

Extracted 1 file, 1,175,857,894 => 462,045,227 bytes. Ratio 254.49%
Extraction time: cpu 0.16 sec/real 260.22 sec = 0%. Speed 1.78 mB/s
All OK

I guess extraction time for the whole thing should take a half day :D

aRrChIvAl pUrPoSeS

:rolleyes:

KaktoR

18-02-2023, 05:18

Don't worry, I've read it :D

It was just meant as another information :p

Masquerade

19-02-2023, 11:42

Football Manager 2023

*.fmf, *.dat, *.dbc:
xtool_zstd_dedup5+lolz
Everything else:
razor

5GB ---> 2.4GB

libzstd v1.4.7 used.

dixen

19-02-2023, 12:28

Atomic Heart (DEV BUILD)

*.pak
XTool (zlib)+sdrep+lzma2
78 gb > 84 gb > 59 gb > 28.7 gb

Rest Files
srep+lzma2

Repack's size = 29.7 gb
Install time = ~7 min.

KaktoR

20-02-2023, 02:56

Can anyone tell me something about the new settlers game? Seems to be zstd mostly and some zlibs aswell but couldn't get much out of it. Seems like 1.5.0 library is the correct one.

Streams: 2887/11649
Time: 00:02:05 (00:12:23)
Memory: 204 MB (204 MB)

Results: 256 MB >> 331 MB

Sample
https://pixeldrain.com/u/jzeJThv3

Razor12911

20-02-2023, 04:39

For zstd streams

33904

33905

The version is 1.5.1 or 1.5.2, not 1.5.0

For deflate streams

33906

so... png+reflate and -d1

-mreflate+png+zstd:l16 -d1

Streams: 11977/11984
Time: 00:00:16 (00:01:32)
Memory: 215 MB (215 MB)

Results: 256 MB >> 686 MB

Make sure there is no xdelta library, xtool freezes because of what I assume would be false positive streams or very small streams.

KaktoR

20-02-2023, 05:34

Thanks, seems my collection was a bit out of date..

Sebazz

24-02-2023, 08:08

Hogwarts Legacy

Game used oodle kraken

Needed oo2core_8_win64.dll v2.8.14

You can upload this oodle dll file please ?

KaktoR

24-02-2023, 08:18

You can upload this oodle dll file please ?

You can find them here
https://fileforums.com/showpost.php?p=499062&postcount=97

KaktoR

26-02-2023, 09:03

Assassin's Creed Valhalla
v1.7.0, all DLC, 1 language

12:34:56 - Selected ARC/DS method for Data1a-01.bin was: xtool:manvil:core_2.8.4+xtool:db:dd3+4x4:lzma
12:34:56 - Selected ARC/DS method for Data1b-01.bin was: xtool:db:dd3+4x4:lzma
-------------------------------------------------------------------------------------------------------------------------------------------
18:02:03 - Overall input size: 143.64 GB
18:02:03 - Overall output size: 50.18 GB (Ratio 34.93%)
18:02:03 - Overall conversion time: 05:27:04

L33THAK0R

26-02-2023, 15:46

Has anyone found a reliable way to rip localized audio from "Quantic Dreams" PC ports (i.e. Heavy Rain, Beyond Two Souls, Detroit Become Human)? I've only found a semi-reliable unpacking utility for "Detroit Become Human", but even then recent builds of the titular title cause the unpacker to crash. All titles only use deflate/zlib for compression, but annoyingly it seems everything, including the audio and pre-rendered Bink videos are also compressed, making ripping the streams a tad tricky.

KaladinDMP

28-02-2023, 21:41

Hello im having trouble with the game Raft. Its the newest version using clean steam files. I am using DSG newest version with ASIS installer also newest version. I have used them both without issues in the past on many games this is not a new setup. The compression goes fine but always the same error message on multiple computers at the end of installation. https://ibb.co/TTx1T2t

the compression i have tried is Srep + Lolz (number 4 in predefined on DSG) and Xtool 2020 (zlib) srep + lolz (number 12 in predefined) .... i dont know much about what to use for what types of games just getting into that part so please go easy on me if this is obvious. Usually i use number 12 and then if errors switch to number 4 and i rarely have issues even though i know this isnt "the best" it usually at least works.

Any help is appreciated or pointers. And sorry if my error has nothing to do with compression type thats part of what im trying to narrow down. TIA

Gehrman

28-02-2023, 21:54

33/1 GB

*.BK2
xbink (https://fileforums.com/showpost.php?p=500233&postcount=341):c256mb:mbk2+4x4

Other Files
xtool:c128mb:mzlib:mpreflate:db:dd3+lolz:mtt1:mt8: mc256

25.9 GB

Masquerade

01-03-2023, 11:21

Resident Evil 5 [Build 10544787]

xtool_zlib_reflate_dedup5+lolz
*.sngw
oggre
*.wmw
srep

8.5GB --> 2.7GB

KaktoR

01-03-2023, 11:27

Scars Above

Here are some tests, not digged deeper as I have no time nor interest in this

zlib
Chunk size: 32.0 MB, Threads: 6, Depth: 0

Streams: 336707/337175
Time: 00:02:32 (00:11:07)
Memory: 457 MB (457 MB)

Results: 12.5 GB >> 21.7 GB

reflate
Chunk size: 32.0 MB, Threads: 6, Depth: 0

Streams: 337175/337175
Time: 00:03:27 (00:16:07)
Memory: 457 MB (457 MB)

Results: 12.5 GB >> 21.7 GB

preflate
Chunk size: 32.0 MB, Threads: 6, Depth: 0

Streams: 335018/337175
Time: 00:05:14 (00:26:43)
Memory: 458 MB (458 MB)

Results: 12.5 GB >> 21.8 GB

Masquerade

01-03-2023, 11:48

^^
Pak contains a shit ton of WWise Audio too so you can patch out the larger files using FModel / MESA and XTool's erase function and use WemTool. I'm skipping this one too :p

Gehrman

03-03-2023, 05:48

61/9 GB

*.Bin
dst+xtool:c32mb:mkraken:dd3+lolz2:mtt1:mt6:mc256

Movie.mpk
4x4

Other Files
srep:m3f+4x4:lzma:a1:mfbt4:32m:fb273:mc1024:lc8

51 GB

CT: 11 H
DT: 20 M

KaktoR

05-03-2023, 12:55

Wo Long Fallen Dynasty

EventMaker3.rdb.bin
zlib
Chunk size: 32.0 MB, Threads: 6, Depth: 0

Streams: 685469/685469
Time: 00:00:44 (00:03:12)
Memory: 317 MB (317 MB)

Results: 4.09 GB >> 10.2 GB

Files in archive folder are webm videos.

KaktoR

06-03-2023, 09:23

Hogwarts Legacy
v18th Februrary 2023, 2 languages

14:17:08 - Selected ARC/DS method for Data1a-01.bin was: xtool:c128mb:mkraken,l5,n256:core_2.8.14+xtool:db: dd3+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc10000 00
14:17:08 - Selected ARC/DS method for Data1b-01.bin was: xtool:db:dd3+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc 8:mc1000000
-------------------------------------------------------------------------------------------------------------------------------------------
18:20:05 - Overall input size: 75.40 GB
18:20:05 - Overall output size: 64.43 GB (Ratio 85.45%)
18:20:05 - Overall conversion time: 04:02:52

Masquerade

06-03-2023, 10:59

The DioField Chronicle

DFC-WindowsNoEditor.pak
ue4d:key=15E3110F40A348CFCC989F0A7A5506B6DE13942DB 2F2F8C07F0D640968F9AF26+xtool_kraken_dedup5+lolz
Everything else:
razor

Thanks to Shegorat for UE4D library and Razor12911 for XTool library.

Oodle Core v2.9.5 used (v2.9.6 has outrageous memory issues).

22.8GB ---> 18.8GB

Compression Time: 9 hours.

Not worth it with lolz due to very little amount of space saved, best just use LZMA :p

KaktoR

09-03-2023, 10:47

The Outer Worlds Spacer's Choice

kraken

pakchunk15-WindowsNoEditor.pak
Library loaded: v2.8.14\oo2core_8_win64.dll

Streams: 64668/64668
Time: 00:09:47 (00:57:18)
Memory: 479 MB (479 MB)

Results: 7.47 GB >> 15.0 GB

It seems that you can also use v2.9.2/2.9.5/2.9.6 libraries.

Masquerade

09-03-2023, 10:59

Don't use library v2.9.6, there's a memory bug.

Sebazz

10-03-2023, 11:33

Hello, I tried to make a repack of Halo Infinite game, an its stuck at 60% ... i don´t know why. It's the only game that this happens to me.

Here is my arc.ini settings, i use srep+lzma2

[External compressor:srep]
header = 0
packcmd = srep -m3f -a2 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = srep -d - - <stdin> <stdout>

[External compressor:lzma2]
header = 0
packcmd = 7z a -txz -an -mcrc=0 -m1=lzma2:d25:fb=273:mf=bt4:mc=1000000:lc=4:lp=0 -mx9 -mmt=on -si -so <stdin> <stdout>
unpackcmd = 7z x -txz -an -y -si -so <stdin> <stdout>

Gehrman

10-03-2023, 11:36

KaktoR

10-03-2023, 14:12

Halo Infinite
Season 3 Echoes Within (7th March 2023)

21:14:05 - Selected ARC/DS method for Data1a-01.bin was: xtool:mkraken,l4,n64:core_2.8.9+xtool:db:dd3+4x4:b 64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000
21:14:05 - Selected ARC/DS method for Data1b-01.bin was: xtool:mreflate+xtool:db:dd3+4x4:b64mb:lzma:ultra:6 4m:bt4:fb273:lc8:mc1000000
21:14:05 - Selected ARC/DS method for Data1c-01.bin was: xtool:db:dd3+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc 8:mc1000000
-----------------------------------------------------------------------------------------
23:00:58 - Overall input size: 50.64 GB
23:00:58 - Overall output size: 34.11 GB (Ratio 67.36%)
23:00:58 - Overall conversion time: 01:46:49

kuyhaa

15-03-2023, 20:56

Masquerade

17-03-2023, 09:42

The Sims 4 [v1.96.365.1030 + All DLCs]

xtool_zlib_dedup5+LZMA2

56.2GB ---> 35.1GB

Not used LOLZ because it takes too long.

KaktoR

20-03-2023, 10:26

Diablo IV (Closed Beta)

zlib

Streams: 19842 / 19842
Time: 00:02:59 (CPU 00:02:58)
Duplicates: 0 (0.00 MB / 0.00 MB [0.00 MB])

Size: 1.00 GB >> 1.53 GB

I guess this will still the case for the release version.

Masquerade

23-03-2023, 11:46

Don't use MSC on The Walking Dead Saints and Sinners Chapter 2: Retribution. It will cause archive corruption. :(

L33THAK0R

23-03-2023, 16:13

Could anyone give me a bit of advice on how to properly use the UE4 XTool plugin? I'm currently using Unreal R8 with XTool 0.6.9 for a title called "The Entropy Center", but initial tests, at least to me are indicating I've somehow not managed to get XTool to decrypt the relevant ".PAK", ".UCAS" & ".UTOC" archives, I've attached an image below, if anyone could show me what I'm doing wrong I'd very much appreciate it.

https://i.imgur.com/spqCCGz.png

Razor12911

24-03-2023, 00:14

https://fileforums.com/showpost.php?p=493012&postcount=14

dixen

24-03-2023, 01:21

https://fileforums.com/showpost.php?p=493012&postcount=14

This version does not work on UCAS from Entropy Centre((

Razor12911

24-03-2023, 02:18

This version does not work on UCAS from Entropy Centre((

XTool is created by Razor12911

Streams: 353168 / 353168
Time: 00:09:03 (CPU 00:06:38)

Size: 10.9 GB >> 21.0 GB

Doesn't work you say, I suggest using two separate processes of xtool for this.

dixen

24-03-2023, 04:56

XTool is created by Razor12911

Streams: 353168 / 353168
Time: 00:09:03 (CPU 00:06:38)

Size: 10.9 GB >> 21.0 GB

Doesn't work you say, I suggest using two separate processes of xtool for this.

Sorry, I forgot change AES key in db.bat))

panker1992

24-03-2023, 07:20

Resident Evil 4 Remake

use zstd and zlib and reflate from xtool 6.9

they don't use oodle anymore :D

rest is srep and lzma

Masquerade

24-03-2023, 11:29

Cities Skylines [v1.16.1-f2]

srep+lolz/$raw=msc+xtool_dedup5_fl2/$ogg=oggre

16.4GB ---> 6GB

KaktoR

26-03-2023, 10:48

Elden Ring
v1.09

17:30:13 - Selected ARC/DS method for Data1a-01.bin was: xtool:mkraken:odoo2core_5_win64.dll:core_2.5.5+xto ol:mkraken:odoo2core_8_win64.dll:core_2.8.14+xtool :db:dd3:c32mb:dd:cpl8,t100p
17:30:13 - Selected ARC/DS method for Data1b-01.bin was: xtool:db:dd3:c32mb:dd:cpl8,t100p
-------------------------------------------------------------------------------------------------------------------------------------------
19:11:18 - Overall input size: 49.92 GB
19:11:18 - Overall output size: 40.30 GB (Ratio 80.72%)
19:11:18 - Overall conversion time: 01:41:04
-------------------------------------------------------------------------------------------------------------------------------------------
19:11:23 - Starting Data1a-01.bin archive decompression.
19:45:48 - Starting Data1b-01.bin archive decompression.
19:47:32 - Overall decompression time: 00:36:09
19:47:32 - All files have been successfully extracted!

IRAQIGHOST

28-03-2023, 04:45

Hi
New way to compress Call of Duty Black Ops 3
SP+MP+ZM (with bots)

Repack Size: 76 Gb
Final Size: 124 Gb

*.xpak
srep_new:3+razor_stdio

*.ff + *.fd
xt_zlib+srep_new:3+LOLZ_NORMAL

*.sabl + *.sabs
srep_new:3+razor_stdio

rest of files
srep_new:3+LZMA-MT*

dixen

28-03-2023, 09:51

Last Of Us Part 1

Game used oodle kraken (need oo2core_9_win64.dll renamed to oo2core_8_win64.dll from game folder)
UPD.
Some files used zlib+kraken

Masquerade

29-03-2023, 00:07

Last Of Us Part 1

Game used oodle kraken (need oo2core_9_win64.dll renamed to oo2core_8_win64.dll from game folder)

DO NOT use the v2.9.6 Oodle core, it has terrible memory problems!!

KaktoR

29-03-2023, 01:19

You can use core 2.8.14 up to core 2.9.5

common.psarc
kraken:l4:n32

Streams: 14951 / 14974
Time: 00:00:35 (CPU 00:00:44)
Duplicates: 0 (0.00 MB / 0.00 MB [0.00 MB])

Size: 1.10 GB >> 1.66 GB

You will get some more streams if you increase scan iterations.

kraken:l4:n128

Streams: 16320 / 16331
Time: 00:00:16 (CPU 00:01:05)
Duplicates: 0 (0.00 MB / 0.00 MB [0.00 MB])

Size: 1.10 GB >> 1.68 GB

kraken:l4:n256

Streams: 16679 / 16686
Time: 00:00:38 (CPU 00:01:25)
Duplicates: 0 (0.00 MB / 0.00 MB [0.00 MB])

Size: 1.10 GB >> 1.69 GB

-----

core.psarc

kraken,l4,n256
Streams: 30184 / 30197
Time: 00:02:13 (CPU 00:01:40)
Duplicates: 0 (0.00 MB / 0.00 MB [0.00 MB])

Size: 2.12 GB >> 3.39 GB

zlib
Streams: 1243 / 1243
Time: 00:07:06 (CPU 00:06:51)
Duplicates: 0 (0.00 MB / 0.00 MB [0.00 MB])

Size: 3.39 GB >> 5.75 GB

Razor12911

29-03-2023, 04:47

Forza Horizon 5 Premium Edition [1.573.834.0]
xtool:c16mb:mzlib:t50p:db:dd3:cpl6,t50p

Streams: 1364716 / 1373346
Time: 02:20:20 (CPU 16:23:30)
Duplicates: 408493 (8.52 GB) [12.9 GB >> 21.9 GB]
Srep decompression memory: 5.20 GB [19.7 GB*]

Size: 160 GB >> 241 GB >> 219 GB >> 170 GB >> 101 GB

dixen

29-03-2023, 06:06

Duplicates: 252533 (7.83 GB) [10.9 GB >> 18.9 GB]How do it?) XTool without SREP?

KaktoR

29-03-2023, 06:08

https://i.imgur.com/PV0dqwK.png

hdneo

29-03-2023, 07:57

https://i.imgur.com/PV0dqwK.png

what is version of this xtool?

KaktoR

29-03-2023, 08:11

It is xtool.exe + xtoolui.dll.

I guess Razor12911 shared it in latest version. If not, then well :D

KaktoR

29-03-2023, 08:27

The Last of Us Part 1

14:38:25 - Selected ARC/DS method for Data1a-01.bin was: xtool:mkraken,l4,n256:core_2.9.5+xtool:mpreflate+x tool:db:dd3+4x4:lzma
14:38:25 - Selected ARC/DS method for Data1b-01.bin was: xtool:db:dd3+4x4:lzma
-------------------------------------------------------------------------------------------------------------------------------------------
16:55:48 - Overall input size: 69.72 GB
16:55:48 - Overall output size: 50.55 GB (Ratio 72.51%)
16:55:48 - Overall conversion time: 02:15:20
16:55:48 - Overall conversion media: 1 custom media
-------------------------------------------------------------------------------------------------------------------------------------------
17:07:14 - Starting Data1a-01.bin archive decompression.
17:24:41 - Starting Data1b-01.bin archive decompression.
17:25:31 - Overall decompression time: 00:18:17
17:25:31 - All files have been successfully extracted!

dixen

30-03-2023, 05:29

Last of Us. Part 1

Little test core.psarc

xtool 0.67 (kraken+zlib) - 2.1 gb > 4.3 gb
oo2reck+xtool (zlib) - 2.1 gb > 5.6 gb

IRAQIGHOST

30-03-2023, 05:43

The Last of Us Part 1

Only English

*.psarc
oo2reck:9+srep_new:3+LOLZ_NORMAL
oo2core (2.9.5)

Other Files
srep_new:3+LOLZ_NORMAL

65 Gb --->> 43 Gb

KaktoR

31-03-2023, 13:35

The Outer Worlds Spacer's Choice
v1.5931.19079

19:29:41 - Selected ARC/DS method for Data1a-01.bin was: xtool:mkraken,l7:core_2.9.5+xtool:mreflate+xtool:o db_odd3+xtool:odb:odd3:ocpt100p,l6
19:29:41 - Selected ARC/DS method for Data1b-01.bin was: xtool:odb:odd3:ocpt100p,l6
-------------------------------------------------------------------------------------------------------------------------------------------
22:34:25 - Overall input size: 51.84 GB
22:34:25 - Overall output size: 45.05 GB (Ratio 86.90%)
22:34:25 - Overall conversion time: 03:04:41

Razor12911

01-04-2023, 12:23

Wo Long: Fallen Dynasty [1.0.6]

xtool:c32mb:mzlib:t100p:db:dd3:cpl10,t100p

Compression time: 32 minutes, 29 seconds
Final Size: 39.7 GB > 21.2 GB

Wanterlude

02-04-2023, 07:16

Wo Long: Fallen Dynasty [1.0.6]

xtool:c32mb:mzlib:t100p:db:dd3:cpl10,t100p

Compression time: 32 minutes, 29 seconds
Final Size: 39.7 GB > 21.2 GB
Wo long v.1.05 (last update)

system.rdb.bin CRC error

I try:
[xtool:zlib or preflate] [db or -] [dd5 or dd3] [lolz or lzma]

I want to know if this error is from my side or not.
I would be very grateful for your help!

Download a file [2.61 gb]:
Fileditch (https://b3.thefileditch.ch/CBkwHFsFyLsRsovqexZC.bin) / Pixeldrain (https://pixeldrain.com/u/SvFRg4WG)

Razor12911

02-04-2023, 23:03

Wo long v.1.05 (last update)

system.rdb.bin CRC error

I try:
[xtool:zlib or preflate] [db or -] [dd5 or dd3] [lolz or lzma]

I want to know if this error is from my side or not.
I would be very grateful for your help!

Download a file [2.61 gb]:
Fileditch (https://b3.thefileditch.ch/CBkwHFsFyLsRsovqexZC.bin) / Pixeldrain (https://pixeldrain.com/u/SvFRg4WG)

I forgot I had fixed these issues on the WIP version of xtool as I've noticed this bug a few days after I released 0.6.9. Essentially for this input, do not use dd nor dd# because for one, the deduplication feature for this input is bugged. There are so many 16KB stream in this game and a lot of duplicates that the hash function used by xtool ended up detecting duplicates so do not use deduplication on 0.6.9 and older. I used 4x4:lzma equivalent which you can use to achieve the same result or lolz for better.

rbn1120

06-04-2023, 00:21

Ready or Not | 92,00GB -> 33,90GB

.pak
srep:m3f:a2:l256+oodleMT:m0:l3:c128:t6

.wav
msc

rest
srep:m3f:a2:l256+lolz:dt:dtb1:dtw1:dtm1:dto1:dtd1: mtt1:mt4:d128m:fba4096

rbn1120

06-04-2023, 04:02

Satisfactory | 19,90GB -> 6,08GB

.pak
srep:m3f:a2:l256+lolz:dt:dtb1:dtw1:dtm1:dto1:dtd1: mtt1:mt4:d128m:fba4096

rest
srep:m3f:a2:l256+oodleMT:m0:l3:c128:t6

KaktoR

07-04-2023, 08:27

EVERSPACE 2

zlib
Streams: 927573 / 927957
Time: 00:08:21 (CPU 00:36:08)
Duplicates: 42779 (352 MB) [663 MB >> 2.60 GB]
Srep decompression memory: 619 MB [1.15 GB*]

Size: 28.0 GB >> 57.2 GB >> 54.6 GB >> 46.6 GB

Edit ---

21:04:04 - Selected ARC/DS method for Data1a-01.bin was: xtool:mzlib:odb:odd3+4x4:lzma
21:04:04 - Selected ARC/DS method for Data1b-01.bin was: msc_tak
21:04:04 - Selected ARC/DS method for Data1c-01.bin was: xtool:odb:odd3+4x4:lzma
-------------------------------------------------------------------------------------------------------------------------------------------
22:20:13 - Overall input size: 34.27 GB
22:20:13 - Overall output size: 24.85 GB (Ratio 72.51%)
22:20:13 - Overall conversion time: 01:16:08
-------------------------------------------------------------------------------------------------------------------------------------------
22:20:19 - Starting Data1a-01.bin archive decompression.
22:28:59 - Starting Data1b-01.bin archive decompression.
22:29:14 - Starting Data1c-01.bin archive decompression.
22:31:15 - Overall decompression time: 00:10:55
22:31:15 - All files have been successfully extracted!

KaktoR

08-04-2023, 12:36

Hearts of Iron IV
v1.12.10

20:54:11 - Selected ARC/DS method for Data1a-01.bin was: srep_new:1+lolz:d33:mtt1:mt12:mc1023
20:54:11 - Selected ARC/DS method for Data1b-01.bin was: msc_tak
20:54:11 - Selected ARC/DS method for Data1c-01.bin was: oggre
20:54:11 - Selected ARC/DS method for Data1d-01.bin was: 1
20:54:11 - Selected ARC/SPLIT method for Data1e-01.bin was: srep_new:1+lolz:d33:mtt1:mt12:mc1023/$jpgpack=xtool:mbrunsli/$mp3pack=7zdll-pm:m0=mp3det:m1=packmp3c/$deflatepack=xtool:c128mb:mpreflate:mpng:d1+srep_n ew:1+lolz:d33:mtt1:mt12:mc1023/$storepack=1/$textpack=srep_new:1+lzma:a1:mfbt4:256m:fb273:mc10 00000000:lc8
-------------------------------------------------------------------------------------------------------------------------------------------
21:18:08 - Overall input size: 6.38 GB
21:18:08 - Overall output size: 3.03 GB (Ratio 47.49%)
21:18:08 - Overall conversion time: 00:23:56
-------------------------------------------------------------------------------------------------------------------------------------------
21:34:34 - Overall decompression time: 00:06:05
21:34:34 - All files have been successfully extracted!

---

Data1a = *.dds
Data1b = *.wav, *.bmp
Data1c = *.ogg (exclude "dlc\dlc035_no_step_back_preorder_bonus\music\Katyu sha.ogg")
Data1d = "dlc\dlc035_no_step_back_preorder_bonus\music\Katyu sha.ogg"
Data1e = rest of the files - masked

L33THAK0R

10-04-2023, 23:19

Bit of a basic question but how exactly does ZSTD re-compression work? Within XTool, is it a library-based like LZ4 or more universal like reflate? Additionally is ripping a block, say for example the first 200mb of a larger archive a suitable method for creating a sample to run tests on? I'm currently giving most of the RE Engine titles a crack and have been having some difficulties with testing different ZSTD libraries, as I've had no success with the stock XTool v0.5.3 ZSTD codec library, and its a bit of a waste to write so much data to the disk just for it to be for nothing.

dixen

12-04-2023, 02:03

What about *.vpk from Half-Life Alyx? LZ4 algo?

Ra7eN

12-04-2023, 14:26

definitely wrong area. Please delete this.

KaktoR

13-04-2023, 13:24

X4 Foundations v6.0

Maybe someone might find this usefull

zlib+msc (oggre will crash)
01.dat
05.dat
06.dat
07.dat
extensions\ego_dlc_boron\ext_01.dat
extensions\ego_dlc_boron\ext_02.dat
extensions\ego_dlc_pirate\ext_01.dat
extensions\ego_dlc_pirate\ext_02.dat
extensions\ego_dlc_split\ext_01.dat
extensions\ego_dlc_terran\ext_01.dat
extensions\ego_dlc_terran\ext_02.dat

msc
02.dat

oggre will crash
03.dat
04.dat

L0v3craft

15-04-2023, 02:13

What about *.vpk from Half-Life Alyx? LZ4 algo?

Seems to have lz4 compression but for some reason xtool is not able to process the streams:

https://t85.pixhost.to/thumbs/94/347471141_immagine.png (https://pixhost.to/show/94/347471141_immagine.png)

Masquerade

15-04-2023, 06:04

Please may someone upload a sample, such as
game/hlvr/maps/a1_intro_world/a1_intro_world_skybox.vpk?

Thanks!

L0v3craft

15-04-2023, 06:14

Please may someone upload a sample, such as
game/hlvr/maps/a1_intro_world/a1_intro_world_skybox.vpk?

Thanks!

here

https://gofile.io/d/EafcDT

Zyzz

17-04-2023, 09:07

I am using Diskspan with xtool+srep+lzam(1 and 2) and i am getting this error

WARNING: can't open file "E:\Kerbal Space Program 2\KSP2_x64_Data\StreamingAssets\aa\StandaloneWindo ws64\parts-base_assets_generator_0v_thermoelectric_radioisoto pe_icon.png_62877585fabce190dba4ae03f3ade21f.b 9.4%
WARNING: can't open file "E:\Kerbal Space Program 2\KSP2_x64_Data\StreamingAssets\aa\StandaloneWindo ws64\parts-base_assets_generator_3v_thermoelectric_radioisoto pe_icon.png_d09dd8a19b5eddfc5024b06104f62283.b 10.8%

i was thinking of using lolz but it takes too long. i tried the xtool unity plugin but that didnt seem to help.

Masquerade

17-04-2023, 09:49

Your compression tools aren't incorrect here, I think this issue is because you are hitting the filepath limit in Windows.

This (https://www.howtogeek.com/266621/how-to-make-windows-10-accept-file-paths-over-260-characters/) quick registry hack should fix it.

Also, do use the Unity plugin because afaik this game uses lz4 compressed bundles.

KaktoR

17-04-2023, 10:04

^
Just the small ones. All of the big files are not compressed it seems.

Compressed 1,762 files, 32,373,757,878 => 37,763,790,489 bytes. Ratio 116.65%
Compression time: cpu 14.59 sec/real 877.50 sec = 2%. Speed 36.89 mB/s
All OK

Razor12911

21-04-2023, 15:46

Star Ocean: The Divine Force

Compressed 1 file, 349,845,408 => 1,256,090,567 bytes. Ratio 359.04%
Compression time: cpu 0.30 sec/real 21.82 sec = 1%. Speed 16.03 mB/s

more info here (https://fileforums.com/showpost.php?p=500900&postcount=350)

KaktoR

22-04-2023, 07:11

Could someone send me a small sample, like "chunk_0\prop.cpk", from "Star Ocean The Divine Force"?

Would be appreciated :)

Masquerade

22-04-2023, 07:42

Could someone send me a small sample, like "chunk_0\prop.cpk", from "Star Ocean The Divine Force"?

Would be appreciated :)

https://hexupload.net/un3syuvgoidh

kj911

24-04-2023, 02:54

This piggy and fbz compressor tell someone something?

fbz.exe => Blizzard v0.24b! Available from encode.su
piggy.dll -> https://github.com/kaby76/Piggy ??

Does anyone know what "cls-sndpack.dll" could be used for in some repacks? I came across it once, 3kB in size.

Masquerade

25-04-2023, 12:30

Strayed Lights

Ripped *.PDB

xtool_zlib+msc_tak+srep+lolz

*.bk2 = xtool_dedup5_fl2

7.5GB ---> 3GB

KaktoR

29-04-2023, 06:35

Star Wars Jedi Survivor

kraken
pakchunk15optional-WindowsNoEditor.ucas

Streams: 9944 / 9950
Time: 00:08:15 (CPU 00:46:45)

Size: 3.56 GB >> 9.73 GB

KaktoR

30-04-2023, 00:20

The Last of Us Part 1
v1.0.4.1

14:05:49 - Selected ARC/DS method for Data1a-01.bin was: xtool:mkraken,l4,n256:core_2.9.9+xtool:mpreflate+x tool:odd3+4x4:lzma
14:05:49 - Selected ARC/DS method for Data1b-01.bin was: xtool:odd3+4x4:lzma
-----------------------------------------------------------------------------------------
16:36:26 - Overall input size: 69.43 GB
16:36:26 - Overall output size: 50.22 GB (Ratio 72.32%)
16:36:26 - Overall conversion time: 02:18:58

With xtool plugin (https://fileforums.com/showpost.php?p=500982&postcount=352)

22:45:38 - Selected ARC/DS method for Data1a-01.bin was: xtool:mpsarc+xtool:mpreflate:odd3+4x4:b64mb:lzma:u ltra:64m:bt4:fb273:lc8:mc1000000
22:45:38 - Selected ARC/DS method for Data1b-01.bin was: xtool:odd3+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8: mc1000000
-----------------------------------------------------------------------------------------
00:43:06 - Overall input size: 69.43 GB
00:43:06 - Overall output size: 48.51 GB (Ratio 69.87%)
00:43:06 - Overall conversion time: 01:57:27

dixen

30-04-2023, 04:23

KaktoR
Did you tested *.pak RE 4 Remake? 80% ratio for me(
zstd+zlib

KaktoR

30-04-2023, 04:58

@dixen
No I don't have this game.

Dragonis40

04-05-2023, 06:33

Good afternoon! I've been trying to get some info about best .wad files compression method in Batman Arkham Knight, but i've found very little info on web and nothing on fileforums site.
Is there any way to massively losslessy compress (and decompress) these files? I've tried almost every tool in Diskspan, but the only good compression mask seems to be to usual srep + lolz.
Tham you in advance!

KaktoR

04-05-2023, 07:42

Send some samples here, so we can tell you something usefull :)

Dragonis40

04-05-2023, 09:46

I'd like to compress all .wad files, not specific files. So I don't know which file I can send you.

KaktoR

04-05-2023, 09:50

Looking on steamdb (https://steamdb.info/depot/208651/), it seems that these files are actually sound files. But I don't have the game so I cannot say for sure, just looking at the file names it seem to be sound files actually.

Maybe wwise in some unreal engine format or whatever. Cannot tell you anything about it.

Masquerade

04-05-2023, 11:02

Not to mention this is just pure awkwardness - if you want help to compress all the files, upload a small one so we can at least see what type of file it is rather than giving some dumb excuse. File Extensions mean nothing, what you really need is a Hex Editor to see what the file actually is.

Something like

BmGame/CookedPCConsole/SFX/SFX_4.wad

Might be good.

Dragonis40

05-05-2023, 05:37

https://we.tl/t-e6tlDpZfJ1

FitGirl

05-05-2023, 16:36

https://we.tl/t-e6tlDpZfJ1
WWise Ogg, can be processed manually by Oggre, check Masquerade's tutorial.

Average gain ~10% of initial size.

KaktoR

06-05-2023, 10:05

Lego Star Wars The Skywalker Saga
Update 04.05.2023

17:08:51 - Selected ARC/DS method for Data1a-01.bin was: xtool:mlegoswtws+xtool:dd3+4x4:lzma
-------------------------------------------------------------------------------------------------------------------------------------------
18:32:13 - Overall input size: 30.48 GB
18:32:13 - Overall output size: 22.88 GB (Ratio 75.05%)
18:32:13 - Overall conversion time: 01:23:20
-------------------------------------------------------------------------------------------------------------------------------------------
18:57:23 - Overall decompression time: 00:25:04
18:57:23 - All files have been successfully extracted!

Dragonis40

07-05-2023, 01:19

WWise Ogg, can be processed manually by Oggre, check Masquerade's tutorial.

Average gain ~10% of initial size.

are you sure? How can I use wWWise ogg procedure on .wad files? I've tried to use wemtool, but of course the exe file found nothing.

L33THAK0R

07-05-2023, 09:22

Currently I'm taking a swing at the various Quantic Dreams titles that were given a proper PC release, however I've run into a bit of a roadblock. The archives for these titles are only compressed with a basic zlib/deflate algorithm, with XTool being able to process the vast majority of streams.

Unfortunately unlike every other title I've worked on, even the localised Vorbis audio is compressed within these archives, leaving me stumped on how to rip it to make the various localised packs into selective offerings.

Currently the only method I could think to exploit was decompressing the archives as a singular block with XTool, blanking out the offending data and then try and restore the block, however, as I feared I was only met with decompression errors and was unsuccessful.

I should mention I the QuickBMS scripts for Quantic Dream titles doesn't work at all with any of the PC ports that I tested, making for scanning compressed blocks with QuickBMS currently not possible.

Has anyone had a similar problem, and if so would such an individual be willing to share their solution, I'd greatly appreciate it, these blasted titles are doing nothing but keeping me awake at night!

Gehrman

09-05-2023, 21:03

*.forge 134 GB > 44.7 GB

xtool:manvil:db:dd3:l10

Other Files 10 GB > 7.22 GB

xrep:db:dd3:l10

Full
144 GB > 51.9 GB

CT: 04:05:19
DT: 02:26:47

XTool 2020 (https://fileforums.com/showpost.php?p=481382&postcount=1)
Anvil Engine (XTool 2020 Plugins) (https://fileforums.com/showpost.php?p=498193&postcount=286)
oodle library 2.8.4
(https://www.fileforums.com/showpost.php?p=499062&postcount=97)

[External compressor:xtool,xrep]
header = 0
unpackcmd = "XTool.exe" decode -t100p -dm10p -sm10p - - <stdin> <stdout>

adrianskiloses

10-05-2023, 02:15

Offworld Trading Company please.
Thank you.

Masquerade

10-05-2023, 03:45

Offworld Trading Company please.
Thank you.

srep+lolz

adrianskiloses

10-05-2023, 05:21

srep+lolz

Thank you very much.
Can you also suggest compression methods for FLAC files (for DLCs)?

Masquerade

10-05-2023, 08:02

FLACs cannot be losslessly compressed, but you can use ffmpeg to recode them into compressed audio formats like MP3

adrianskiloses

10-05-2023, 08:24

FLACs cannot be losslessly compressed, but you can use ffmpeg to recode them into compressed audio formats like MP3

There is an OST DLC that has both mp3 and FLAC versions. So should I ignore compressing the FLAC entirely in that case?

Also is there any guides for compressing Unity games in general?
Thanks

Hexagon123

10-05-2023, 11:41

There is an OST DLC that has both mp3 and FLAC versions. So should I ignore compressing the FLAC entirely in that case?

Also is there any guides for compressing Unity games in general?
Thanks

Depends on the game,

Some use LZ4, LZMA or none

SREP+LOLZ

IRAQIGHOST

11-05-2023, 06:19

*.pak
xtool:mzlib:mzstd+srep_new:3+LOLZ_NORMAL

other files
srep_new:3+LZMA-MT*

Repack Size: 42 Gb
Final Size: 57 Gb (Deluxe Edition)

adrianskiloses

11-05-2023, 06:49

Depends on the game,

Some use LZ4, LZMA or none

SREP+LOLZ

How do you find out?
Also what I'm looking for is some sort of tool that you can take Unity assets off of and precompress, and during install they're decompressed and put back on the Unity files. Is it possible?

Masquerade

11-05-2023, 10:30

Also what I'm looking for is some sort of tool that you can take Unity assets off of and precompress, and during install they're decompressed and put back on the Unity files. Is it possible?

You can use the XTool unity plugin to precompress lz4 compressed Unity Asset Bundles.

Junior53

15-05-2023, 00:31

I extract the DataPC.forge file using quickBMS script and after that i try to put them back into where it was using raw injector but it doesn't work out because the all extracted *.dat files doesn't have the same hexadecimals address so my question is how do i solve this & why blocks are isn't the same?

Trying to find the 87.dat block Start Offset and End Offset in DataPC.forge using HxD
https://drive.google.com/file/d/15Og1Ba8sx5OQdr1TvoU-CB5atcpOML7D/view

But it doesn't work out?
https://drive.google.com/file/d/1ME-WC8UUjICmNUDJEeZ5UO3GABq2mgRJ/view

These are the unpack files
https://drive.google.com/file/d/1ova6u6lbshqVQCJv5rJ4Zh1U1YYmZxn-/view

KaktoR

18-05-2023, 03:47

Can someone look at this kind of files and what I can do with them? They have "RIFF8-��WAVEfmt".

Sample as attachment

Masquerade

18-05-2023, 04:55

Not sure what it is. VGMStream cannot detect what it is and wwise2ogg will only give a 17kb output.

Masquerade

18-05-2023, 13:12

Golf With Your Friends (Build 11145867)

xtool_unity_dd5+lolz

4.6GB ---> 2.3GB

shazzla

21-05-2023, 00:56

Hi all !

Frostpunk V1.6.1 - GOTY

Latest XTool : Zero inflation. Any kind of variations of ZLib,Preflate,Reflate -cXXmb ,no success.

But this version inflates the data well !

XTool version 0.9

Created by Razor12911

Commands:
e - encode
d - decode

Operations and codecs available:
precomp - data precompression
zlib : loaded
crilayla : not loaded
lz4 : loaded
zstd : loaded
lzo : loaded
oodle : loaded

Options:
precomp - data precompression
c# : chunk size (default 16mb)
t# : number of threads, use p for percentage
dm#: decoding memory [1(low) .. 3(max)]
dt#: diff tolerance (default 0.1)

Other game's ZLib files are OK.
What am i do wrong ?

KaktoR

21-05-2023, 11:50

^
I can confirm.

Sample should be around 106%

kuyhaa

21-05-2023, 20:41

Hi all !

Frostpunk V1.6.1 - GOTY
XTool version 0.9

try xtool 0.12

shazzla

21-05-2023, 20:56

@kuyhaa : i would like to use the newest version,btw thank you !:)

Sample from GOTY V1.6.1 :

XTool v0.9 :

arc a -mxprecomp32 e:\animations32.arc e:\animations.dat
FreeArc 0.67 (March 15 2014) creating archive: e:\animations.arc
Compressed 1 file, 29,260,989 => 50,177,682 bytes. Ratio 171.48%
Compression time: cpu 0.02 sec/real 12.71 sec = 0%. Speed 2.30 mB/s
All OK

arc t e:\animations32.arc
FreeArc 0.67 (March 15 2014) testing archive: e:\animations32.arc
Tested 1 file, 50,177,682 => 29,260,989 bytes. Ratio 171.48%
Testing time: cpu 0.02 sec/real 0.85 sec = 2%. Speed 34.25 mB/s
All OK

Latest XTool :
Nothing.

Edit:

XTool V0.7.1 : Works fine ! Thank you Razor12911 !

Masquerade

22-05-2023, 14:34

Marvels Spiderman Remastered [Build 10619930]

Archive 1 content: All files from asset_archive folder except dag, toc and languages.
Archive 2 content: All remanining files except languages.

Archive 1:
xtool_spidey_dd5+lolz
Archive 2:
xtool_reflate+razor-stdio
Languages:
razor-stdio

Repack

Original Size | 79GB
--------------------------------
Game | 37.5GB
--------------------------------
+ Arabic | +1.1GB
+ English | +1.1GB
+ French | +1.1GB
+ German | +1.1GB
+ Italian | +1.1GB
+ Japanese | +1.1GB
+ Polish | +1.1GB
+ Portguese | +1.1GB
+ Portuguese-Brazil | +1.1GB
+ Russian | +1.1GB
+ Spanish-Latin-America | +1.1GB
+ Spanish-Spain | +1.1GB
--------------------------------
TOTAL | 50.9GB

Thanks to Razor12911 for XTool, ProFrager for lolz, Bulat Ziganshin for FreeArc and srep.

Wanterlude

23-05-2023, 12:04

Warhammer 40,000: Boltgun [1.17.38829.471]

kraken+preflate+dedup+lolz

4.38 gb => 2.96 gb

Masquerade

23-05-2023, 12:44

[B][U]Warhammer 40,000: Boltgun

You can also use this database I made using bms2xtl:
https://bayfiles.com/D6bdh7taze/Boltgun_xtl

hwang4

26-05-2023, 05:00

You can also use this database I made using bms2xtl:
https://bayfiles.com/D6bdh7taze/Boltgun_xtl

Which Version of oo2core.dll , should i use ?

deepshit

29-05-2023, 05:26

Can someone look at this kind of files and what I can do with them? They have "RIFF8-��WAVEfmt".

Sample as attachment

This is a custom wwise bnk archive.
It contains multiple wem files.
I converted one of them.

L33THAK0R

29-05-2023, 10:02

Has anyone had any luck with any LZ4 library for "State of Decay 2"? Threads around the net indicate it uses some sort of "custom" LZ4 solution, however QuickBMS is able to parse it with it's built in LZ4 comtype. Currently exhausted a good collection of ~20 libraries through trials and I'm yet to pick up or detect a single stream.

Attached is a sample ".pak" archive should this compression quandary tickle anyone else's fancy.

Masquerade

31-05-2023, 09:58

Shame Legacy

*.pak:
ue4d:key=07E35C91100A9B400B1A87B15337BEEAC3545EEE3 0786F21CB2EE7283AF51743+xtool_zlib_dd5+lolz
Everything else:
razor-stdio

9.9GB ---> 6.9GB

Used CLS-UE4D library by Shegorat.
Thanks to Razor12911 and ProFrager for XTool and Lolz.
Thanks to Bulat Ziganshin for FreeArc and srep.

KaktoR

01-06-2023, 09:09

Anyone can help me with srep options to use? I'm getting stuck here a bit.

ERROR! Can't allocate memory: 9705 mb required

Edit: -l# option seems to be the trick.

KaktoR

03-06-2023, 16:39

Hitman 3 test using rpkg tool (because no plugin is available atm)

chunk0.rpkg (13.5GB)

srep+lolz_fast
Overall input size: 14.03 GB
Overall output size: 12.10 GB (Ratio 86.29%)
Overall conversion time: 00:53:55

srep+lolz_fast with rpkg tool
Overall input size: 15.50 GB
Overall output size: 11.41 GB (Ratio 73.61%)
Overall conversion time: 02:01:43

Input is bigger because chunk0.rpkg was extracted (lz4 variant is used). However output is smaller. But that time to process is insane and I recommend not to do this at all.

The time above was achived on a m2 ssd for a single test. Here again it is not recommended to do this kind of stuff on a ssd, even it is much faster.

Extracting all rpkg's on a HDD took barely 4 hours.
Next step is to make TAR files out of extracted files (this will take another bunch of hours), because somehow freearc seems to have problems with nearly 2 million extracted files.

*.WWEM
*.WWES

Rename them to *.WEM so you can use WemTool on them. After decoding rename them back to *.WWEM and *.WWES.
You can save approximately 10% of original size.

*.WWEV
They have "RIFF8-��WAVEfmt" but cannot do anything with them.
More info: https://fileforums.com/showpost.php?p=501212&postcount=3495

This is just a info test out of curiosity and I wanted to share some infos here.