Best Compression Methods for 'Specific' Games. Q&A [Archive] - Page 15

KaktoR

05-06-2023, 00:20

XTool 0.6.9 + xdelta3_dll.dll + lz4_193 + doboz.dll

bms2xtl config
lz4=lz4hc:l9

dixen

05-06-2023, 09:29

XTool 0.6.9 + xdelta3_dll.dll + lz4_193 + doboz.dll

bms2xtl config
lz4=lz4hc:l9
It works but..VERY slow on lz4hc streams.

Masquerade

05-06-2023, 09:39

Repella Fella

You can use ffmpeg to extract the vorbis audio tracks from the webm videos then make hdiffz patches to convert the oggs back into videos (note that the hdiff patches will be huge since they make up the entire video track).

Only issue is that oggre is buggy af with these oggs, so it may just be easier to use:

srep+lzma2 on the webm

precomps+srep+lolz on everything else.

Precomps = precomp stdin/stdio fork by Nicolas Comerci.

8GB > 7.2GB

I spent all day trying to get oggre working, but it will crash without much of a pattern to go off.

KaktoR

05-06-2023, 09:53

It works but..VERY slow on lz4hc streams.

Hm, no, not for me.

Did it today with latest version.

11:16:00 - Overall input size: 61.98 GB
11:16:00 - Overall output size: 44.43 GB (Ratio 71.68%)
11:16:00 - Overall conversion time: 01:58:29

dixen

05-06-2023, 13:00

Hm, no, not for me.

Did it today with latest version.

11:16:00 - Overall input size: 61.98 GB
11:16:00 - Overall output size: 44.43 GB (Ratio 71.68%)
11:16:00 - Overall conversion time: 01:58:29
Can you share your witcher3.xtl?

Masquerade

06-06-2023, 11:06

Orcs Must Die! 3 [v1.2.1.0]

*.pak:
xtool_OMD3_dd5+lolz
Everything else:
xtool_zlib+razor-stdio

12.6GB ---> 6.8GB

Requires OMD3 database and lz4 library linked here (https://fileforums.com/showpost.php?p=496456&postcount=251).

Dragonis40

07-06-2023, 10:08

Good evening, is there a way to create a .tmp file from a decompressed (or unpacked) file or batch of files in order to use hdiffz.exe and so to create a .diff file?
A lot of videogames have several files that can be unpacked using specific tools, but no way to repack them. Maybe hdiffz.exe is a solution, but it needs a .tmp file. How to create it? Thanks in advance for your help!

Masquerade

07-06-2023, 13:57

HDiffz does not need a .tmp file, read the usage instructions.

Old file and new file makes differential patch.

Masquerade

09-06-2023, 14:42

Hi,
Does anyone have any samples of LZMA compressed Unity Asset Bundles that they could share with me?
Thanks!

KaktoR

10-06-2023, 00:27

Diablo IV
v1.0.2, incl. HD files

10:50:35 - Selected ARC/DS method for Data1a-01.bin was: xtool:dd3+lzma2:d96m:fb273:lc4:mc1000000
10:50:35 - Selected ARC/DS method for Data1b-01.bin was: xtool:mzlib+xtool:mpreflate:dd3+lzma2:d96m:fb273:l c4:mc1000000
-----------------------------------------------------------------------------------------
13:17:45 - Overall input size: 77.06 GB
13:17:45 - Overall output size: 62.72 GB (Ratio 81.38%)
13:17:45 - Overall conversion time: 02:27:08

L33THAK0R

16-06-2023, 06:58

Does anyone know roughly what the average variance is for the final ratio of a given input, that's run through LOLZ?

KaktoR

16-06-2023, 07:55

The Last of Us Part 1
v1.1.0.0

19:08:20 - Selected ARC/DS method for Data1a-01.bin was: xtool:c192mb:mpsarc:core_2.9.9+xtool:mpreflate:dd3 +4x4:lzma
19:08:20 - Selected ARC/DS method for Data1b-01.bin was: xtool:dd3+4x4:lzma
-----------------------------------------------------------------------------------------
21:16:38 - Overall input size: 70.29 GB
21:16:38 - Overall output size: 48.29 GB (Ratio 68.70%)
21:16:38 - Overall conversion time: 02:08:17

You will get better ratio in increasing chunk size for xtool.

kj911

16-06-2023, 14:10

L33THAK0R: Very input data dependent! Older, very older gamez, betterly compressible than previous 10-15 years ago to last years games. Average few percent to upto 10-25% betterly final compression ratio than LZMA. After, many data precompression (ex: xtool), prefiltering (ex: exe/delta) and such more...

Simply, this cannot be said in advance! It is necessary to experiment, it is possible to derive approximate information about the possible degree of compression from existing repacked games online.

Small examples: This very old modded MSTS games *.ACE files with xzlib+srep+lolz combo avg. 25-40% final ratio than uncompressed sizes.
*.RAW files in rare case upto 1.5-2* betterly compressed via delta+(srep)+lolz combo than simple LZMA only.
Sound files final sizes difference its minimal few percentage. Achieve better rates use sound specific compression.

KaktoR: What "mpsarc"??

KaktoR

16-06-2023, 14:50

KaktoR: What "mpsarc"??

https://fileforums.com/showpost.php?p=500982&postcount=352

L33THAK0R

17-06-2023, 02:41

L33THAK0R: Very input data dependent! Older, very older gamez, betterly compressible than previous 10-15 years ago to last years games. Average few percent to upto 10-25% betterly final compression ratio than LZMA. After, many data precompression (ex: xtool), prefiltering (ex: exe/delta) and such more...

Simply, this cannot be said in advance! It is necessary to experiment, it is possible to derive approximate information about the possible degree of compression from existing repacked games online.

Small examples: This very old modded MSTS games *.ACE files with xzlib+srep+lolz combo avg. 25-40% final ratio than uncompressed sizes.
*.RAW files in rare case upto 1.5-2* betterly compressed via delta+(srep)+lolz combo than simple LZMA only.
Sound files final sizes difference its minimal few percentage. Achieve better rates use sound specific compression.

KaktoR: What "mpsarc"??

@@kj911 ah thank you for the explanation, what I meant was more how much could be chalked up to variance due to LOLZ being a non-deterministic compression algorithm (or series of, I'm not too sure on its inner-workings). I like to compare my results to my peers, for some titles, take the 2020 title "Manifold Garden" I got a result of 797 MB, however one of our peers got a result of 792 MB. Now I was working with the latest version of the title and I also included the weight of the installer, with mine being slightly larger (due to some un-optimised storing of the "VCL styles" library that I've yet to figure out how to compress and decompress during run-time), and I'm wondering whether I can chalk up this difference to variance or whether I've missed some data that can be packed more optimally.

kj911

17-06-2023, 08:51

L33THAK0R: From what I have experienced with LOLZ so far, its operation is a bit more unusual compared to the standard LZMA. A given result cannot be reproduced, exactly byte, as with more commercial compressors. (Given compressed file size, like enwik8-10 results.) Certain switches of "Arc.exe" also affect the degree of final compression, I'm thinking here of the "-ds" switch, which is the default in principle or even if you (not) define it the final result may be different, say with "-dses/dsns" than otherwise.. LOLZ can sometimes be better without -dt/dtb, in some cases, than if we set everything to the max.

This small image shows the results of data files precompressed with xZLib+srep:m3f:a0 mode, and separately compressed to 7z, with the same compression settings. The difference between them is whether or not "-ds" or "-dses" switch was used during precompression. We would think that the end result would be the same if LZMA/LOLZ final compression were used for that archive.

Wanterlude

24-06-2023, 12:43

Hi all, can anyone look at the files of the game Bramble The Mountain King, there UE4 and encrypted files inside + kraken.
I tried the latest version of the unreal plugin, but it didn't work. (xtool last ver.). With ucas database the same thing.
Can someone please see if it's the plugin not working or am I crooked?
Key:
0x0D29D2B91A3564686C106FA7A96BE98D3AA13A55C80D4E32 00EE608DA2C105E3

//edit. woks fine with latest xtool

Masquerade

24-06-2023, 15:18

Hi all, can anyone look at the files of the game Bramble The Mountain King, there UE4 and encrypted files inside + kraken.
I tried the latest version of the unreal plugin, but it didn't work. (xtool last ver.). With ucas database the same thing.
Can someone please see if it's the plugin not working or am I crooked?
Key:
0x0D29D2B91A3564686C106FA7A96BE98D3AA13A55C80D4E32 00EE608DA2C105E3

Apologies if you tried already, but please may you try XTool v0.7.1 with Unreal Plugin and Oodle core v2.9.5 with this packcmd? I cpuldn't help but notice you mentioned UE4 whereas the UTOC/UCAS structure is a fearure of UE5 that got added backwards.

xtool precomp -c128mb -t50p -mue5:0D29D2B91A3564686C106FA7A96BE98D3AA13A55C80D4 E3200EE608DA2C105E3+kraken

Wanterlude

24-06-2023, 17:28

Apologies if you tried already, but please may you try XTool v0.7.1 with Unreal Plugin and Oodle core v2.9.5 with this packcmd? I cpuldn't help but notice you mentioned UE4 whereas the UTOC/UCAS structure is a fearure of UE5 that got added backwards.

xtool precomp -c128mb -t50p -mue5:0D29D2B91A3564686C106FA7A96BE98D3AA13A55C80D4 E3200EE608DA2C105E3+kraken
XTool v0.7.1 with Unreal Plugin and Oodle core v2.9.5
xtool precomp -c128mb -t50p -d1 -mue5:0D29D2B91A3564686C106FA7A96BE98D3AA13A55C80D4 E3200EE608DA2C105E3+kraken
Streams: 8910 / 8910
Time: 00:00:18 (CPU 00:01:30)

Size: 6.70 GB >> 6.71 GB
xtool precomp -c128mb -t50p -d1 -mue4:0D29D2B91A3564686C106FA7A96BE98D3AA13A55C80D4 E3200EE608DA2C105E3+kraken
Streams: 984 / 984
Time: 00:00:14 (CPU 00:01:01)

Size: 6.70 GB >> 6.72 GB
*xtool 0.7.2h same results

KaktoR

25-06-2023, 09:07

Just read a topic about unpacking PCK files. Nothing new, honestly, but an idea came in my mind about not just unpacking them, but also encode/decode the WEMs inside the PCK files. So far, nothing new. Just wanted to know, if there is a "elegant" way instead of my idea floating in my head. Maybe somthing more "automated".

I already know that this method works and it's CRC perfect after the last step.

1. Unpack PCK files with quickbms + wwise.bms
2. Copy unpacked files to somewhere else, so you have 2 copys of them
3. Use WEMTool on the first copy in game folder
4. Null all WEM files on the second copy outside of game folder and reimport them back with quickbms to PCK files in game folder. This way you can compress the PCK files* with ~0-1% ratio
5. Compress the game with nulled PCK files* and encoded WEM files

Installing the repack
1. After installing, you have to decode all processed WEM files back to their original with WEMTool again
2. Reimport decoded WEM files and BNK files back to PCK files with quickbms import feature

Masquerade

25-06-2023, 10:12

This will absolutely work however you can replace The QuickBMS reimport with XTool erase/decode since the contents of a PCK aren't compressed so the wems will appear in plain bytes.

I've wanted in the past to find some way of adding a wem plugin to XTool so wems don't need to be extracted to disk (which is a long time when there are often thousands of tiny wems).

KaktoR

26-06-2023, 06:15

Only bad thing left is that quickbms isn't able to create extracted folders in the same (sub)folder where the input files are. In AC games you have several dlc folders with pck files.
In those cases I just use a FOR loop for it, works great. However you have to use "cd /D" command everytime to switch the folder if you are on root level as working directory, otherwise quickbms will create the extracted folders inside root folder without _extracted prefix and you have to move them manually afterwards.

kj911

29-06-2023, 11:07

Any idea, methods, tricks recompressing Trainz 2010 (and PTP2*) game *.CDP files/complete games???

Example file the following CDP format: http://lat-trainz-addons.1w.lv/DR1A-290%20Full%20Pack.cdp.zip

XTool 0.12/0.71 with zlib or zlib+(p)reflate methods or many precomp 0.4.x versions, not handle inflating the following file. GFS found ~19k ZLIB streams packed in level:9 mode.

*Pro Train Perfect 2. Public copy not available. (The game based on TS2010.)

shazzla

30-06-2023, 10:12

This BK2 files cannot be compressed using BPK ?!
Its a new thing to me.
Or i miss something ? (Other BIKs/BK2s compressing well.)

XY@DESKTOP-TSNPU1C d:\repack\pack
$ arc a -mbpk e:\bik.arc "d:\Layers of Fear - 2023-\Cron\Content\Movies\Intro\Intro_4K.bk2"
FreeArc 0.67 (March 15 2014) creating archive: e:\bik.arc
Compressed 1 file, 669,310,488 => 669,310,655 bytes. Ratio 100.00%
Compression time: cpu 1.09 sec/real 4.47 sec = 24%. Speed 149.78 mB/s
All OK

wattafak@DESKTOP-TSNPU1C e:\repack\pack
$ arc a -mbpk e:\bik2.arc "d:\Layers of Fear - 2023-\Cron\Content\Movies\CH5\Cinema\CH5_Cinema_Movie.b k2"
FreeArc 0.67 (March 15 2014) creating archive: e:\bik2.arc
Compressed 1 file, 100,336,964 => 100,336,995 bytes. Ratio 100.00%
Compression time: cpu 0.14 sec/real 0.66 sec = 21%. Speed 151.72 mB/s
All OK

KaktoR

30-06-2023, 10:15

Yes, new bink2 videos cannot be compressed with bpk. Watch out for the "KB2n" file header with hex editor.

kuyhaa

01-07-2023, 22:29

hi everyone

how to extract "disc" file on SOS: A Wonderful Life game and pack back, or plugin for xtool

thanks

Masquerade

02-07-2023, 06:06

hi everyone

how to extract "disc" file on SOS: A Wonderful Life game and pack back, or plugin for xtool

thanks

Not possible with XTool. You can have the game running with extracted disc file but needs EXE patching. Password to decrypt disc can be found here: https://github.com/Masquerade64/SoS-AWL-Password
Just normal ZipCrypto. Compressed data with lzs algo, very similar to early lz4.

KaktoR

02-07-2023, 11:16

For my earlier test, about unpacking PCK files, encoding wem's and such, here is a comparisson for ACOrigins audio files. Excluding processing banks.

SFX + English audio
original
input 3.795.077.936 bytes

Compressed 41 files, 3,795,077,936 => 3,041,411,552 bytes. Ratio 80.14%
Compression time: cpu 2.05 sec/real 223.01 sec = 1%. Speed 17.02 mB/s
All OK

Time 4 minutes

---

diffz+ww files + nulled wem blovks inside PCK files
input 7.422.151.129 bytes)

Compressed 148,175 files, 7,422,151,129 => 2,979,433,554 bytes. Ratio 40.14%
Compression time: cpu 23.94 sec/real 763.29 sec = 3%. Speed 9.72 mB/s
All OK

Time 1 hour 20 minutes

~25%-33% +time per language

L33THAK0R

02-07-2023, 18:14

For my earlier test, about unpacking PCK files, encoding wem's and such, here is a comparisson for ACOrigins audio files. Excluding processing banks.

SFX + English audio
original
input 3.795.077.936 bytes

Compressed 41 files, 3,795,077,936 => 3,041,411,552 bytes. Ratio 80.14%
Compression time: cpu 2.05 sec/real 223.01 sec = 1%. Speed 17.02 mB/s
All OK

Time 4 minutes

---

diffz+ww files + nulled wem blovks inside PCK files
input 7.422.151.129 bytes)

Compressed 148,175 files, 7,422,151,129 => 2,979,433,554 bytes. Ratio 40.14%
Compression time: cpu 23.94 sec/real 763.29 sec = 3%. Speed 9.72 mB/s
All OK

Time 1 hour 20 minutes

~25%-33% +time per language

I wonder if it'd be faster and result in a better ratio to instead extract all the relevant files and pack each PCK as one singular block, then patch the block once its been decoded and reverted back to its original form with the original PCK file. I've found especially with smaller blanked out sections within an archive the final compressor sometimes doesn't achieve as optimal a result as it would when working on a singular patch file. I had this exact issue with Just Cause 3, when I initially patched each FSB (of which there were thousands) and then wrote the now patched FSBs into the archives, which resulted in a fairly lackluster result, compared to combining each valid file extracted from the FSBs into one singular block and creating a hdiffz patch between that block and the original "*.arc" archive.

Enluaphelis

03-07-2023, 01:17

Hello! How to determine which is the best method for compressing "Gravel" game?
I did not find it in the index so I don't know.
It has .mp4 files, .pak files, .exe
Also I don't quite understand how to make an installer after compressing it in DiskSpanGUI. I tried using some random methods but the files I got as an output look weird and I don't know what is the best option: "dvd5" or "unlimited". I don't understand how to pack it in the installer, where to put compressed files in and how to make it work.
I'd appreciate your help.

KaktoR

03-07-2023, 03:10

Use zlib

Streams: 10832 / 10834
Time: 00:00:05 (CPU 00:00:25)

Size: 312 MB >> 669 MB

Fak Eid

03-07-2023, 04:08

Hey, I'm new and don't even know if this question is relevant here. But I need help:

1. How to identify language files from a game. Is there a tool for it? Like for some games like Spider-Man Remastered and GotG, I was not able to locate language files.

2. I'm currently using Ultra compression method of FreeArc (-mx -ld1600m), and using fazip.exe to decompress, but it takes around 1hr to extract Uncharted: Legacy of Thieves Collection. (99.2 GB compressed to 64.4GB). My FreeArc version is 0.67, however my fazip.exe version is 0.0.3 (and a decade old from 2013). Can someone provide me a link to the latest fazip.exe and fazip64.exe files.

3. Also, to decompress, what is the difference in time between fazip.exe and fazip64.exe.

KaktoR

03-07-2023, 04:13

1. You can use steamdb and look inside the depots (https://steamdb.info/app/1817070/depots/). Example for Spider-Man Remastered: the language files have the short language code as file extension. Example: French -> asset_archive\a00s035.fr

2. There is no newer version of freearc nor fazip, sadly.

3. Don't know but I guess fazip64 is using 64-bit architecture and can utilize more resources.

Enluaphelis

03-07-2023, 05:17

Use zlib

Streams: 10832 / 10834
Time: 00:00:05 (CPU 00:00:25)

Size: 312 MB >> 669 MB

For all files you mean? Also how to properly make an installer and what is the best installer for diskspangui? Which split method choose in "dvd-cd-bluray-unlimited" section?

KaktoR

03-07-2023, 05:23

For pak files, but you can use it on all files, doesn't matter.

For supported installers look here
https://fileforums.com/showpost.php?p=492987&postcount=3

Enluaphelis

03-07-2023, 06:00

For pak files, but you can use it on all files, doesn't matter.

For supported installers look here
https://fileforums.com/showpost.php?p=492987&postcount=3

Understood. I've chosen your installator ASIS. I don't quite understand how to configure it and which options I should change in .ini file and where to place compressed files after diskspan?

Enluaphelis

03-07-2023, 08:22

For pak files, but you can use it on all files, doesn't matter.

For supported installers look here
https://fileforums.com/showpost.php?p=492987&postcount=3

I think I understood how to made an installer, but how to get rid of everything except for .bin files? (https://imgur.com/a/dmIqPZL)
Installer doesn't work without these files.
But how does every single repack works without these, only with Setup.exe and data files?

KaktoR

03-07-2023, 11:10

Setup.dll and Records.ini are required. However you can include both inside Setup.exe on compiling.

If you use ASIS, do the following steps:
1. Extract Setup.dll to "[ASIS folder]\DECOMPRESSOR" folder
2. Move Records.ini to the same folder
3. Execute "Create_Setup.dll.bat"
3a. Password input is not visible in CMD window. If you don't want to use a Password, just press ENTER twice
4. Open Script.iss and change line 8
;#define InternalDLL
to
#define InternalDLL
(just remove the semicolon
5. Compile the script. Setup.dll and Records.ini are now included in Setup.exe

If you used a Password for Setup.dll, you have to change line 9 in the script aswell
;#define PasswordDLL "123"
Remove the semicolon and replace 123 with your password (quotation marks have to be there)

However, next time please ask such questions in the topics in which the questions belong to. Thanks.

Enluaphelis

05-07-2023, 08:19

Use zlib

Streams: 10832 / 10834
Time: 00:00:05 (CPU 00:00:25)

Size: 312 MB >> 669 MB

I tried to compressed Gravel game but output size is twice size.
https://imgur.com/a/hm6nbiP
This is the settings I used, I did not find "zlib" so I used xtool:mzlib, I don't know which one I should use: https://imgur.com/a/cbGueNR
What am I doing wrong?

Wanterlude

05-07-2023, 08:40

I tried to compressed Gravel game but output size is twice size.
https://imgur.com/a/hm6nbiP
This is the settings I used, I did not find "zlib" so I used xtool:mzlib, I don't know which one I should use: https://imgur.com/a/cbGueNR
What am I doing wrong?

1) Xtool [currently: zlib] (Only decompresses files for better compression)
2) Deduplicator [srep] (Removes repetitions)
3) Compressor [lolz/lzma/7z/...] (lolz better but slower then others)
Example for DiskSpan: xtool:mzlib+srep_new:1+LOLZ_NORMAL or xtool:mzlib+srep_new:1+lzma2

p.s. You do not need to prescribe anything yourself, everything can be selected in the program itself

KaktoR

05-07-2023, 08:46

xtool:mzlib+srep+LZMA-MT

Output should be around 9.5 GB

Enluaphelis

05-07-2023, 08:47

1) Xtool [currently: zlib] (Only decompresses files for better compression)
2) Deduplicator [srep] (Removes repetitions)
3) Compressor [lolz/lzma/7z/...] (lolz better but slower then others)
Example for DiskSpan: xtool:mzlib+srep_new:1+LOLZ_NORMAL or xtool:mzlib+srep_new:1+lzma2

Thanks, gonna try tomorrow. So if I understand using only xtool:zlib only decompressed files for me but not compressed it again and hence I got that size:D
By the way, what does NORMAL mean after LOLZ? It means compression level or thread priotity?

Enluaphelis

05-07-2023, 08:51

xtool:mzlib+srep+LZMA-MT

Output should be around 9.5 GB

Thanks
Also, what "MT" means?
LZMA2 and LZMA-MT, what is the difference between it? Is there any thread out there where I can read more info about compression methods and these special settings for them?

Wanterlude

05-07-2023, 08:53

Thanks, gonna try tomorrow. So if I understand using only xtool:zlib only decompressed files for me but not compressed it again and hence I got that size:D
By the way, what does NORMAL mean after LOLZ? It means compression level or thread priotity?

https://i.ibb.co/pyzPcC2/184956.png
Lolz: From the bottom up, from the weakest compression to the strongest, but the stronger the compression, the more time will be spent on it. Normal, High, ... its just presets.
You can read all info about lolz here [But in Russian] [You can use the translator] https://krinkels.org/resources/lolz.264/

KaktoR

05-07-2023, 08:55

LZMA2 is used by 7-zip (multithread option is used by default).
LZMA-MT is used by FreeArc (FreeArc LZMA with 4x4 option, literally multithreaded. It will utilize max 16 CPU threads).

As for LZMA2 (it is basically just a predefined config name), you can read possible options somewhere on the 7-zip page I guess.

For the LOLZ_* names: These are just predefined configurations. FreeArc will read LOLZ as follows:
lolz{:options}
Where {:options} includes all valid options for this compressor.

So the preset "LOLZ_NORMAL" is just this:
lolz:d<expr:((0.45*freememory/11)-(32*1048576*2))/min(thread,16)/1048576>:mtt1:mt<expr:min(thread,16)>:mc1023
DSG reads your aviable free RAM and replace the :d option in lolz, same for the :mt option, just with your max CPU threads.

However you can write your own options if you want. lolz options can be read in english in the following folder "Resources\FINAL\LOLZ\Options.txt"

Wanterlude

14-07-2023, 17:50

Jagged Alliance 3 [1.01.338010]

zstd+dedup+lolz

14.6 GB => 10.74 GB

zstd v1.5.2

dixen

15-07-2023, 06:47

Total War SHOGUN 2

Movies*.pack - bkpck (XTool+bink pack)
Sound.pack - msc+precompmt(mp3pack)+srep+lzma2
Local*en & ru.pack - precompmt
Rest *.pack & files - xt76(zlib+reflate)+srep+lolz

RePack size = 9.3 gb

Enluaphelis

15-07-2023, 07:58

Pure (2008)

bpk+srep on .bik files
xtool:mzlib+srep+lolz_normal on everything else

4.90 gb -------> 2.65 gb

Hexagon123

15-07-2023, 11:53

Pure (2008)

4.90 gb -------> 2.65 gb

Pure uses LZSS like compression.

You may try to make a plugin for it.

https://aluigi.altervista.org/bms/splitsecond.bms

Enluaphelis

16-07-2023, 03:44

Pure uses LZSS like compression.

You may try to make a plugin for it.

https://aluigi.altervista.org/bms/splitsecond.bms

Will it benefit in terms of compression ratio? I don't quite understand what is the reason of it and moreover how to make a plugin, I'm beginner:D, using methods present in DiskSpanGUI

Danziel123

17-07-2023, 05:58

Hi, maybe someone know what compress method for dead island 2? Thank you:)

Wanterlude

17-07-2023, 10:05

Hi, maybe someone know what compress method for dead island 2? Thank you:)

unreal plugin(ue5)+kraken+srep/dedup+lolz/lzma
45.8 gb (orig. size)=>108 gb(kraken+unreal.dll)=>38.3 gb(dedup+lolz)
70 mins installation (11400F)

key:
0x014AEC0148FBDEE9640633AAB67521AAB4E1083A98F76FF3 3DDCF78DAD05BE66

Danziel123

17-07-2023, 10:13

unreal plugin+kraken+srep/dedup+lolz/lzma
45.8 gb (orig. size)=>108 gb(kraken+unreal.dll)=>38.3 gb(dedup+lolz)
70 mins installation (11400F)
It using UE4/5?

Wanterlude

17-07-2023, 10:14

It using UE4/5?

use ue5
ue5:k0x014AEC0148FBDEE9640633AAB67521AAB4E1083A98F 76FF33DDCF78DAD05BE66

Snake288

17-07-2023, 11:41

Hello can you help me

Dead Island 2 I'm trying to compress the game
DiskSpan_GUI Dead Island 2 I'm trying to compress the game
What should the correct settings be?

Picture -1
https://i.hizliresim.com/z2klql0.jpg (https://www.hizliresim.com/z2klql0)
Picture -2
https://i.hizliresim.com/bet1crt.jpg[/url

Danziel123

17-07-2023, 19:58

use ue5
ue5:k0x014AEC0148FBDEE9640633AAB67521AAB4E1083A98F 76FF33DDCF78DAD05BE66
Thank you for the info but when i try to compress the game it show error. Maybe you know what cause it? (arledy download unreal engi0ne plugin in DSG)
http://i.ibb.co/b60BvPy/Di-2.png
http://i.ibb.co/ysfD3mv/DI-2-1.png

ScOOt3r

17-07-2023, 20:02

You should try and remove the space in the ue5 line.. no space when you compress. also this game uses oo2core7 file as well in conjunction with it but i dont know diskspan at all as i dont use it for compressions.

Danziel123

17-07-2023, 20:12

You should try and remove the space in the ue5 line.. no space when you compress. also this game uses oo2core7 file as well in conjunction with it but i dont know diskspan at all as i dont use it for compressions.
Sorry, where the space in the ue5 line?

ScOOt3r

17-07-2023, 20:15

ue5:k0x014AEC0148FBDEE9640633AAB67521AAB4E1083A98F 76FF33DDCF78DAD05BE66

Danziel123

17-07-2023, 20:25

ue5:k0x014AEC0148FBDEE9640633AAB67521AAB4E1083A98F 76FF33DDCF78DAD05BE66
I see no space in the ue5 line:confused:
http://i.ibb.co/yWmbSVw/Di-2-3.png
http://i.ibb.co/TrLjkDY/DI-2-4.png

rakeshbro

18-07-2023, 02:53

*.pc
xtool:mzlib+srep_new:1+lolz:d23:mtt1:mt12:mc1023

Other Files
srep_new:1+lolz:d23:mtt1:mt12:mc1023

Installation folder size: 7.73 GB
Compressed game size: 4.20 GB (Ratio 54.35%)

Installation time:
Installation started at 14:38:04
Installation stopped at 14:47:28

Fak Eid

20-07-2023, 06:57

I tried a conversion algo srep_new:3+rzr_mtx from Diskspan GUI v2.0.1.8. What files do I need on my installer to extract the data from it? Can someone also provide me with the files?

KaktoR

20-07-2023, 07:22

All files which are included in Setup.dll

Fak Eid

20-07-2023, 12:01

All files which are included in Setup.dll

Sorry for this noob question. But Setup.dll can't be opened. What tool should I use to open that or am I looking at wrong place? What changes need to be done as part of the 'Arc.ini' file inside my setup installer, if I compress via these files inside my 'unpack' folder

KaktoR

20-07-2023, 13:45

Use DSG. Click the "Tools" button at the bottom, then this appears.

https://i.imgur.com/ymPSfjI.png

For the future, ask such questions in DiskSpan_GUI topic to keep this topic here clean, as it should be only for compression related things. Thanks.

KaktoR

22-07-2023, 07:14

Grand Theft Auto V
13 June 2023

12:19:10 - Selected ARC/DS method for Data1a-01.bin was: bpk+xtool:mreflate+xtool:dd3+4x4:b64mb:lzma:ultra: 64m:bt4:fb273:lc8:mc1000000
12:19:10 - Selected ARC/DS method for Data1b-01.bin was: xtool:dd3+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:m c1000000
-----------------------------------------------------------------------------------------
15:01:02 - Overall input size: 109.44 GB
15:01:02 - Overall output size: 65.85 GB (Ratio 60.17%)
15:01:02 - Overall conversion time: 02:41:51

Data1:
common.rpf
x64*.rpf
update\*.rpf

Data2:
rest of the files

KaktoR

22-07-2023, 13:01

Unholy

20:56:16 - Selected ARC/DS method for Data1a-01.bin was: xtool:mzlib:dd3+4x4:b64mb:lzma:ultra:bt4:fb273:lc8 :mc1000000:64mb
20:56:16 - Selected ARC/DS method for Data1b-01.bin was: xtool:dd3+4x4:b64mb:lzma:ultra:bt4:fb273:lc8:mc100 0000:64mb
-------------------------------------------------------------------------------------------------------------------------------------------
21:53:58 - Overall input size: 30.65 GB
21:53:58 - Overall output size: 22.64 GB (Ratio 73.87%)
21:53:58 - Overall conversion time: 00:57:40

KaktoR

23-07-2023, 07:42

The Long Dark
v2.22

14:53:09 - Selected ARC/DS method for Data1a-01.bin was: xtool:munity,lz4hc,l12:lz4_193+xtool:dd3+4x4:b64mb :lzma:ultra:64m:bt4:fb273:lc8:mc1000000
14:53:09 - Selected ARC/DS method for Data1b-01.bin was: xtool:dd3+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:m c1000000
-------------------------------------------------------------------------------------------------------------------------------------------
15:33:15 - Overall input size: 14.28 GB
15:33:15 - Overall output size: 7.32 GB (Ratio 51.26%)
15:33:15 - Overall conversion time: 00:40:00

WemTool used.
2.34GB -> 2.18GB

KaktoR

23-07-2023, 10:12

DAVE THE DIVER

18:54:02 - Selected ARC/DS method for Data1a-01.bin was: xtool:munity,lz4hc,l12:lz4_193+xtool:dd3+4x4:b64mb :lzma:ultra:64m:bt4:fb273:lc8:mc1000000
18:54:02 - Selected ARC/DS method for Data1b-01.bin was: xtool:dd3+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:m c1000000
-------------------------------------------------------------------------------------------------------------------------------------------
19:07:11 - Overall input size: 8.82 GB
19:07:11 - Overall output size: 1.81 GB (Ratio 20.48%)
19:07:11 - Overall conversion time: 00:13:07
-------------------------------------------------------------------------------------------------------------------------------------------
19:10:44 - All files have been successfully extracted!

KaktoR

23-07-2023, 14:54

Remnant 2

Streams: 658698 / 659636
Time: 00:18:35 (CPU 01:32:56)

Size: 26.3 GB >> 49.6 GB

ue5:k0x0D2FCCB28812883236070C3098006BB686352061FE5 F78DEC6B6186118927193
kraken

dixen

24-07-2023, 03:47

The Sims 3

*.package *.world
XTool v0.76+lolz
20 gb > 32 gb > 29 gb > 23 gb > 13.5 gb

xtool precomp -mpng+zlib+reflate -d3 -dd3 -spa0/0 -c64mb

NOTE.
Some files used lz4 streams but xtool can't decompress it.

KaktoR

24-07-2023, 08:19

Generation Zero
v18.07.2023

15:22:18 - Selected ARC/DS method for Data1a-01.bin was: xtool:c128mb:mzlib:dd3+4x4:b64mb:lzma:ultra:64m:bt 4:fb273:lc8:mc1000000
15:22:18 - Selected ARC/DS method for Data1b-01.bin was: xtool:dd3+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:m c1000000
-------------------------------------------------------------------------------------------------------------------------------------------
16:37:28 - Overall input size: 54.23 GB
16:37:28 - Overall output size: 13.54 GB (Ratio 24.96%)
16:37:28 - Overall conversion time: 01:15:09
-------------------------------------------------------------------------------------------------------------------------------------------
17:14:12 - Overall decompression time: 00:14:04
17:14:12 - All files have been successfully extracted!

FitGirl

25-07-2023, 02:03

The Sims 3

*.package *.world
XTool v0.76+lolz
20 gb > 32 gb > 29 gb > 23 gb > 13.5 gb

NOTE.
Some files used lz4 streams but xtool can't decompress it.
Sims 3 doesn't have LZ4 streams, the game was released before LZ4 was invented. Those are false detections of old custom EA algo, used in many of their games.

Danziel123

25-07-2023, 18:50

Anyone know what best compression for forspoken? Thank you:)

ScOOt3r

25-07-2023, 20:06

i used zlib+srep+lolz and it compressed to 88.8 gb

Danziel123

25-07-2023, 20:13

i used zlib+srep+lolz and it compressed to 88.8 gb
No files can delete to save some space? (Languages files)

ScOOt3r

25-07-2023, 20:39

no as its only about 8 large files about 20gb each..

KaktoR

26-07-2023, 07:46

Viewfinder

15:42:29 - Selected ARC/DS method for Data1a-01.bin was: msc_tak+srep_new:1
15:42:29 - Selected ARC/DS method for Data1b-01.bin was: xtool:dd3+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:m c1000000
-------------------------------------------------------------------------------------------------------------------------------------------
16:01:27 - Overall input size: 14.29 GB
16:01:27 - Overall output size: 4.99 GB (Ratio 34.91%)
16:01:27 - Overall conversion time: 00:18:56

KaktoR

26-07-2023, 07:52

Anyone know what best compression for forspoken? Thank you:)

Cut the first 200mb of one of the big files and send it to me.

Also tell me the file header of the bk2 files if possible with hxd.

Razor12911

26-07-2023, 08:39

Forspoken

-mlz4f:l9

Use liblz4.dll that comes with xtool (v1.8.3)
You must also have xdelta, every stream has 8 byte difference and will not precompress without it.

Compressed 1 file, 104,857,600 => 210,035,729 bytes. Ratio 200.31%
Compression time: cpu 0.08 sec/real 1.15 sec = 7%. Speed 91.17 mB/s

dixen

26-07-2023, 11:26

What about extensionless from Ratchet & Clank: Rift Apart? Same as in SpiderMan?
Sample https://pixeldrain.com/u/cKV4RvZa

Razor12911

26-07-2023, 17:53

Ratchet & Clank: Rift Apart

-minsomniac:rcrads

tex_global
Compressed 1 file, 216,354,837 => 232,416,562 bytes. Ratio 107.42%
Compression time: cpu 0.19 sec/real 1.37 sec = 14%. Speed 157.41 mB/s

actor
Compressed 1 file, 797,350 => 5,150,215 bytes. Ratio 645.92%
Compression time: cpu 0.00 sec/real 0.67 sec = 0%. Speed 1.19 mB/s

Link (https://fileforums.com/showpost.php?p=501722&postcount=372)

Danziel123

26-07-2023, 20:27

no as its only about 8 large files about 20gb each..
Thank you for suggestion, i compress using mzlib + dedub + lzma-MT get 89.4 gb

Cut the first 200mb of one of the big files and send it to me.

Also tell me the file header of the bk2 files if possible with hxd.
Sorry for dumb question, how to cut the big files? Here i send sample earc and bk2 files
bk2: https://drive.google.com/file/d/1ueYfUioJQuaYQ1aCbjccdZNnavXevMW1/view?usp=sharing
earc: https://drive.google.com/file/d/1y8OnOrtPTPEDiQZxCewQLhnyND_yRDWw/view?usp=sharing

Ratchet & Clank: Rift Apart

-minsomniac:rcra

Use liblz4.dll that comes with xtool (v1.8.3)

tex_global
Compressed 1 file, 216,354,837 => 232,416,562 bytes. Ratio 107.42%
Compression time: cpu 0.19 sec/real 1.37 sec = 14%. Speed 157.41 mB/s

actor
Compressed 1 file, 797,350 => 5,150,215 bytes. Ratio 645.92%
Compression time: cpu 0.00 sec/real 0.67 sec = 0%. Speed 1.19 mB/s

Link (https://fileforums.com/showpost.php?p=501722&postcount=372)

PLUGIN RECEIVED A HOTFIX, MAKE SURE YOU USE THAT
Sorry, but how to use it in DSG? Thank you

dixen

26-07-2023, 22:45

Ratchet & Clank: Rift Apart

-minsomniac:rcrads

tex_global
Compressed 1 file, 216,354,837 => 232,416,562 bytes. Ratio 107.42%
Compression time: cpu 0.19 sec/real 1.37 sec = 14%. Speed 157.41 mB/s

actor
Compressed 1 file, 797,350 => 5,150,215 bytes. Ratio 645.92%
Compression time: cpu 0.00 sec/real 0.67 sec = 0%. Speed 1.19 mB/s

Link (https://fileforums.com/showpost.php?p=501722&postcount=372)lz4_1.82 - best result

Compressing zone_openarea 92%
Compressing 862,286,145 bytes with xtool.exe precomp -minsomniac -c128mb -t100p $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
Streams: 11004 / 11004
Time: 00:00:21 (CPU 00:01:20)

Size: 822 MB >> 2.54 GB

100%
Errorlevel=0

Compressed 1 file, 862,286,145 => 2,730,091,250 bytes. Ratio 316.61%
Compression time: cpu 0.27 sec/real 44.57 sec = 1%. Speed 19.35 mB/s
All OK

KaktoR

26-07-2023, 23:27

Thank you for suggestion, i compress using mzlib + dedub + lzma-MT get 89.4 gb

Sorry for dumb question, how to cut the big files? Here i send sample earc and bk2 files
bk2: https://drive.google.com/file/d/1ueYfUioJQuaYQ1aCbjccdZNnavXevMW1/view?usp=sharing
earc: https://drive.google.com/file/d/1y8OnOrtPTPEDiQZxCewQLhnyND_yRDWw/view?usp=sharing

Sorry, but how to use it in DSG? Thank you

There is a file cutter somewhere here on fileforums, shared by Razor12911, which I personally use.

For the earc files use like this
xtool:mlz4f,l9:lz4_183

For the bk2 files you have to use xtool aswell with the bink plugin because bpk will break the file on unpacking.
xtool:c256mb:mbk2:df20p

However I suggest to test all of the bk2 files first. In DSG_Settings.ini set it like this and select the game folder normally (this will only compress *.bk2 files). Test compress and decompress. Maybe it is required to set the correct chunk size too.
[Game1Settings]
Name=Unknown 1
Exec=
EGPath=C:\Program Files (x86)\Unknown <game>
MethodState=0
Data1.Name=Data<game><data>-01.bin
Data1.InclText=*.bk2

Masquerade

26-07-2023, 23:33

VGMToolbox has a FileCutter too afair.

Danziel123

26-07-2023, 23:59

There is a file cutter somewhere here on fileforums, shared by Razor12911, which I personally use.

For the earc files use like this
xtool:mlz4f,l9:lz4_183

For the bk2 files you have to use xtool aswell with the bink plugin because bpk will break the file on unpacking.
xtool:c256mb:mbk2:df20p

However I suggest to test all of the bk2 files first. In DSG_Settings.ini set it like this and select the game folder normally (this will only compress *.bk2 files). Test compress and decompress. Maybe it is required to set the correct chunk size too.
[Game1Settings]
Name=Unknown 1
Exec=
EGPath=C:\Program Files (x86)\Unknown <game>
MethodState=0
Data1.Name=Data<game><data>-01.bin
Data1.InclText=*.bk2
Here it's result compress the bk2 files

-----------------------------------------------------------------------------------------
-------------------------------- CONVERSION DATE 27/07/2023 --------------------------------
-----------------------------------------------------------------------------------------
13:43:53 - You have chosen the unlimited type for the output size.
13:43:53 - ISO images will not be created when conversion is complete.
13:43:53 - The computer will remain on when the conversion is complete.
13:43:53 - Selected ARC/DS method for Data1a-01.bin was: xtool:c256mb:mbk2:df20p
-----------------------------------------------------------------------------------------
13:43:57 - Compression initialized in 27/07/2023 at 13:43:57.
13:43:57 - Starting Data1a-01.bin archive creation.
13:49:39 - Moving Data1a-01.bin file to media folder.
13:49:44 - Compression finished in 27/07/2023 at 13:49:44.
-----------------------------------------------------------------------------------------
13:49:44 - Getting size of the game: Forspoken
13:49:44 - » Installation folder size: 6,84 GB
13:49:44 - » Compressed game size: 6,24 GB (Ratio 91.28%)
-----------------------------------------------------------------------------------------
13:49:44 - Overall input size: 6,84 GB
13:49:44 - Overall output size: 6,24 GB (Ratio 91.28%)
13:49:44 - Overall conversion time: 00:05:46
13:49:44 - Overall conversion media: 1 custom media
-----------------------------------------------------------------------------------------

It's good ratio?

And yes it can install/decompress, here the proof
http://i.ibb.co/kgknfTW/Forspoken.png

KaktoR

27-07-2023, 00:05

BPK is always around ~10% saving, same as oggre.

So yes you can use it.

Danziel123

27-07-2023, 00:23

BPK is always around ~10% saving, same as oggre.

So yes you can use it.
Here is my setting in DSG_Settings, it is correct to compress this game?

[Game1Settings]
Name=Forspoken
Exec=
EGPath=C:\Program Files (x86)\Forspoken
MethodState=0
Data1.Name=Data1a-01.bin
Data1.InclText=*.earc
Data1.ExclText=
Data1.Method1=xtool:mlz4f,l9:lz4_183+xtool:dd3+LZM A-MT
Data2.Name=Data1b-01.bin
Data2.Method1=xtool:c256mb:mbk2:df20p
Data2.InclText=*.bk2

KaktoR

27-07-2023, 00:28

You have forgot something important.

You have 2 data.bin's, just including particular files.

But what about all the other files of the game?

Use ExclText=*.earc|*.bk2 for data3 ;)

Danziel123

27-07-2023, 00:44

You have forgot something important.

You have 2 data.bin's, just including particular files.

But what about all the other files of the game?

Use ExclText=*.earc|*.bk2 for data3 ;)
Sorry i don't get it
Should like this?
[Game1Settings]
Name=Forspoken
Exec=
EGPath=C:\Program Files (x86)\Forspoken
MethodState=0
Data1.Name=Data1a-01.bin
Data1.InclText=*.earc
Data1.ExclText=
Data1.Method1=xtool:mlz4f,l9:lz4_183+xtool:dd3+LZM A-MT
Data2.Name=Data1b-01.bin
Data2.Method1=xtool:c256mb:mbk2:df20p
Data2.InclText=*.bk2
Data3.Name=Data1c-01.bin
Data3.Method1=xtool:dd3+LZM A-MT
Data3.ExclText=*.earc|*.bk2

KaktoR

27-07-2023, 00:48

Yes you got it ;)

Danziel123

27-07-2023, 01:14

Yes you got it ;)
Thank you, still compressing RN and i will report back:)

Edit
It works (Compressing & decompressing) Sorry i can't post the conversion log, because i deleted (my bad)
Using those method get 80.9 GB and install time:
http://i.ibb.co/9YF8rVR/Untitled.png

Danziel123

27-07-2023, 06:31

Sorry if i ask again, anyone know best compression method for ratchet & clank - rift apart? Thank you:)

Masquerade

27-07-2023, 06:41

Sorry if i ask again, anyone know best compression method for ratchet & clank - rift apart? Thank you:)

You could at least read back a page rather than being lazy! :mad:

https://www.fileforums.com/showpost.php?p=501723&postcount=3582

Danziel123

27-07-2023, 06:50

You could at least read back a page rather than being lazy! :mad:

https://www.fileforums.com/showpost.php?p=501723&postcount=3582
Sorry don't mad at me, but where to put those file? In folder XTool/Xtool_2020?

ScOOt3r

27-07-2023, 07:39

Yes those files will have to be copied into your xtool folder and then compress.. hope this helps and good luck compressing..

cheers
ScOOt3r

Danziel123

27-07-2023, 23:29

Yes those files will have to be copied into your xtool folder and then compress.. hope this helps and good luck compressing..

cheers
ScOOt3r
Hi sorry if i ask again, alredy copy those file to folder XTool and compressing with this method:

-----------------------------------------------------------------------------------------
-------------------------------- CONVERSION DATE 28/07/2023 --------------------------------
-----------------------------------------------------------------------------------------
08:43:31 - You have chosen the unlimited type for the output size.
08:43:31 - ISO images will not be created when conversion is complete.
08:43:31 - The computer will remain on when the conversion is complete.
08:43:31 - Selected ARC/DS method for Data1a-01.bin was: xtool:insomniac:rcrads+xtool:dd3+4x4:b64mb:lzma:ul tra:64m:bt4:fb273:lc8:mc1000000
-----------------------------------------------------------------------------------------
08:43:33 - Compression initialized in 28/07/2023 at 08:43:33.
08:43:33 - Starting Data1a-01.bin archive creation.
09:27:40 - Moving Data1a-01.bin file to media folder.
09:27:42 - Compression finished in 28/07/2023 at 09:27:42.
-----------------------------------------------------------------------------------------
09:27:42 - Getting size of the game: Ratchet & Clank - Rift Apart
09:27:42 - » Installation folder size: 38,79 GB
09:27:42 - » Compressed game size: 32,21 GB (Ratio 83.04%)
-----------------------------------------------------------------------------------------
09:27:42 - Overall input size: 38,79 GB
09:27:42 - Overall output size: 32,21 GB (Ratio 83.04%)
09:27:42 - Overall conversion time: 00:44:09
09:27:42 - Overall conversion media: 1 custom media
-----------------------------------------------------------------------------------------

But it not good with ratio, maybe you know what cause it? or i wrong write compress method in DSG? Thank you:)

dixen

27-07-2023, 23:40

But it not good with ratio, maybe you know what cause it? or i wrong write compress method in DSG? Thank you:)

https://fileforums.com/showpost.php?p=501722&postcount=372

Danziel123

28-07-2023, 00:24

https://fileforums.com/showpost.php?p=501722&postcount=372
I alredy download it and copy it to folder RESOURCE/PRE/XTOOL but it not shows in tab xtool plugin, or i wrong place to copy those files?

KaktoR

28-07-2023, 12:09

As for "Forspoken" bink2 videos, you can use a chunk size as 981mb, as the biggest file is 981mb

c256mb
Compressed 331 files, 7,343,656,316 => 6,703,069,558 bytes. Ratio 91.28%
Compression time: cpu 3.03 sec/real 367.24 sec = 1%. Speed 20.00 mB/s
All OK

c512mb
Compressed 331 files, 7,343,656,316 => 6,605,046,614 bytes. Ratio 89.94%
Compression time: cpu 2.98 sec/real 348.98 sec = 1%. Speed 21.04 mB/s
All OK

c1gb
Compressed 331 files, 7,343,656,316 => 6,498,592,363 bytes. Ratio 88.49%
Compression time: cpu 3.36 sec/real 431.03 sec = 1%. Speed 17.04 mB/s
All OK

c981mb
Compressed 331 files, 7,343,656,316 => 6,498,592,343 bytes. Ratio 88.49%
Compression time: cpu 3.20 sec/real 448.67 sec = 1%. Speed 16.37 mB/s
All OK

KaktoR

29-07-2023, 09:48

Ratchet & Clank Rift Apart

For the movie files you can use bink2 plugin with a chunk size like c1gb. The videos are just stored as plain text. You can see the bink2 file headers "KB2j" with HxD but I don't know how big the biggest file is actually. However I guess c1gb should be enough.

c256mb
Compressed 4 files, 4,269,568,000 => 3,891,106,223 bytes. Ratio 91.14%
Compression time: cpu 1.83 sec/real 144.25 sec = 1%. Speed 29.60 mB/s
All OK

c512mb
Compressed 4 files, 4,269,568,000 => 3,843,203,753 bytes. Ratio 90.01%
Compression time: cpu 1.69 sec/real 166.48 sec = 1%. Speed 25.65 mB/s
All OK

c1gb
Compressed 4 files, 4,269,568,000 => 3,708,218,303 bytes. Ratio 86.85%
Compression time: cpu 1.97 sec/real 249.45 sec = 1%. Speed 17.12 mB/s
All OK

c2gb
Compressed 4 files, 4,269,568,000 => 3,708,218,323 bytes. Ratio 86.85%
Compression time: cpu 1.91 sec/real 283.34 sec = 1%. Speed 15.07 mB/s
All OK

Extracted 4 files, 3,708,218,323 => 4,269,568,000 bytes. Ratio 86.85%
Extraction time: cpu 3.02 sec/real 167.80 sec = 2%. Speed 25.44 mB/s
All OK

I guess you can find out the biggest file chunk using xtool verbose mode with the bk2 plugin, but not sure.

Snake288

29-07-2023, 11:35

Hello

Ratchet & Clank Rift Apart I want to compress game
DiskSpan_GUI help with files needed for the filesystem for compression

ravikant

29-07-2023, 20:41

I want to reencode music and videos in some of my old games but all those files are encrypted and if i run ffmpeg on them it will not give me any good results as the enryption is stopping the files to be modified. Any tool to break this encryption in games

At present i am testing on thug 1,2 and THPS 4

ravikant

29-07-2023, 20:44

Hello

Ratchet & Clank Rift Apart I want to compress game
DiskSpan_GUI help with files needed for the filesystem for compression

All the compression tools are already present in diskspan gui you just need to select input and output folder and select one of the preset methods[make sure your system has good processing power when you select a high resource consuming method otherwise it may throw an error in between]

KaktoR

30-07-2023, 01:47

Ratchet & Clank Rift Apart
v1.727

23:07:31 - Selected ARC/DS method for Data1a-01.bin was: xtool:minsomniac,rcrads+xtool:dd3+4x4:b64mb:lzma:u ltra:64m:bt4:fb273:lc8:mc1000000
23:07:31 - Selected ARC/DS method for Data1b-01.bin was: xtool:c1gb:mbk2:df20p
23:07:31 - Selected ARC/DS method for Data1c-01.bin was: xtool:dd3+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:m c1000000
-------------------------------------------------------------------------------------------------------------------------------------------
00:26:12 - Overall input size: 39.40 GB
00:26:12 - Overall output size: 28.05 GB (Ratio 71.20%)
00:26:12 - Overall conversion time: 01:18:39
-------------------------------------------------------------------------------------------------------------------------------------------
>> Unpacking finished successfully!

Bink2 video information (https://fileforums.com/showpost.php?p=501779&postcount=3602)

The soundbanks contain wem files. It is possible to use WemTool on extracted wem files. The same goes for the other wem archives. However some of them are truncated, whatever this means. I have attached the first file from wem00 file.

Dragonis40

30-07-2023, 02:04

Good morning! I've tried to compress "Star Wars Jedi: Fallen Order". In installation process something got wrong: unarc error -12 during .bk2 files installation. Even bink_pack didn't recognize .bk2 file as standard file.
So I've tried to compress .bk2 files in another way, with no good results.
So, are these .bk2 files different? Do I need a specific tool for those? Which one?
Thanks in advance!

KaktoR

30-07-2023, 02:05

Send sample

Edit: According to the file list (https://steamdb.info/depot/1172381/), this game does not contain any bink2 files, just mp4.

Dragonis40

30-07-2023, 02:27

I'll send a sample as soon as i come back home. Anyway, in the deluxe version there are four movies: DDV_1.bk2, DDV_2.bk2, DDV_3.bk2 and DDV_4.bk2.

KaktoR

30-07-2023, 02:34

I see.

Edit: It is a legit KB2j file and should work. However the first part of the file does not belong to a standard bink2 file format. Maybe deleting this part and patching it back again after unpacking do the trick.

https://i.imgur.com/jE5nG3v.png

Bink2 header is "4B 42 32" and the next letter indicates the version and/or codec used. Starting with "n" or later letters are not compatible with binkpack. This way you can always see right off from the start if using binkpack is possible or not.

Edit 2: It works with XTool bink plugin. Normal BPK just failes on extracting.
Compressed 1 file, 327,833,352 => 283,487,172 bytes. Ratio 86.47%
Compression time: cpu 0.06 sec/real 45.62 sec = 0%. Speed 7.19 mB/s
All OK

Extracted 1 file, 283,487,172 => 327,833,352 bytes. Ratio 86.47%
Extraction time: cpu 0.23 sec/real 22.50 sec = 1%. Speed 14.57 mB/s
All OK

Steps:

Open the bk2 files with HxD and search for the first bink2 header and delete the whole part starting from the beginning of the bink2 header "KB2j" to the start of the file and save the file so "KB2j" is on the very beginning of the file (HxD will make a BAK file as backup).
Use XTool bink video plugin
Make patches from Original and Cutted files
After extracting, patch the files back to original with xdelta or hdiff (patch files should be very small)

Snake288

30-07-2023, 02:41

Hello
DiskSpan_GUI I want to get help
I request your support about adding XTool 2020 Plugins DiskSpan_GUI about compressing games
Ratchet & Clank Rift Apart I want to compress

Dragonis40

30-07-2023, 05:20

I see.

Edit: It is a legit KB2j file and should work. However the first part of the file does not belong to a standard bink2 file format. Maybe deleting this part and patching it back again after unpacking do the trick.

https://i.imgur.com/jE5nG3v.png

Bink2 header is "4B 42 32" and the next letter indicates the version and/or codec used. Starting with "n" or later letters are not compatible with binkpack. This way you can always see right off from the start if using binkpack is possible or not.

Edit 2: It works with XTool bink plugin. Normal BPK just failes on extracting.
Compressed 1 file, 327,833,352 => 283,487,172 bytes. Ratio 86.47%
Compression time: cpu 0.06 sec/real 45.62 sec = 0%. Speed 7.19 mB/s
All OK

Extracted 1 file, 283,487,172 => 327,833,352 bytes. Ratio 86.47%
Extraction time: cpu 0.23 sec/real 22.50 sec = 1%. Speed 14.57 mB/s
All OK

Steps:

Open the bk2 files with HxD and search for the first bink2 header and delete the whole part starting from the beginning of the bink2 header "KB2j" to the start of the file and save the file so "KB2j" is on the very beginning of the file (HxD will make a BAK file as backup).
Use XTool bink video plugin
Make patches from Original and Cutted files
After extracting, patch the files back to original with xdelta or hdiff (patch files should be very small)

First of all, THANK YOU for your support.
The only issue I have is how to decompress .bin file using Xtool bink. I think I have to use files i can find in "COMPRESSOR\Resources\TOOLS\Bink" from DiskSpan, but how?

KaktoR

30-07-2023, 05:51

https://i.imgur.com/mkrTTHV.png

Set XTool chunk size like c1gb, should be enough.

Snake288

30-07-2023, 05:55

Ratchet & Clank Rift Apart
v1.727

23:07:31 - Selected ARC/DS method for Data1a-01.bin was: xtool:minsomniac,rcrads+xtool:dd3+4x4:b64mb:lzma:u ltra:64m:bt4:fb273:lc8:mc1000000
23:07:31 - Selected ARC/DS method for Data1b-01.bin was: xtool:c1gb:mbk2:df20p
23:07:31 - Selected ARC/DS method for Data1c-01.bin was: xtool:dd3+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:m c1000000
-------------------------------------------------------------------------------------------------------------------------------------------
00:26:12 - Overall input size: 39.40 GB
00:26:12 - Overall output size: 28.05 GB (Ratio 71.20%)
00:26:12 - Overall conversion time: 01:18:39
-------------------------------------------------------------------------------------------------------------------------------------------
>> Unpacking finished successfully!

Bink2 video information (https://fileforums.com/showpost.php?p=501779&postcount=3602)

The soundbanks contain wem files. It is possible to use WemTool on extracted wem files. The same goes for the other wem archives. However some of them are truncated, whatever this means. I have attached the first file from wem00 file.

Hello

Ratchet & Clank Rift Apart I want to compress game
DiskSpan_GUI help with files needed for the filesystem for compression

dixen

31-07-2023, 23:38

Hi all, can anyone look at the files of the game Bramble The Mountain King, there UE4 and encrypted files inside + kraken.
I tried the latest version of the unreal plugin, but it didn't work. (xtool last ver.). With ucas database the same thing.
Can someone please see if it's the plugin not working or am I crooked?
Key:
0x0D29D2B91A3564686C106FA7A96BE98D3AA13A55C80D4E32 00EE608DA2C105E3
My test with xtool v 0.77 + oodle 2.9.5

Compressing Bramble_TMK-WindowsNoEditor.ucas 99.2%
Compressing 7,111,026,672 bytes with xtool.exe precomp -mue5:k0x0D29D2B91A3564686C106FA7A96BE98D3AA13A55C8 0D4E3200EE608DA2C105E3:n128+kraken -d1 -c128mb -t100p $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
100.0%XTool is created by Razor12911

Streams: 221925 / 222121
Time: 00:06:00 (CPU 00:37:28)

Size: 6.62 GB >> 12.5 GB

100.0%
Errorlevel=0

Compressed 1 file, 7,111,026,672 => 13,462,584,251 bytes. Ratio 189.32%
Compression time: cpu 0.95 sec/real 592.84 sec = 0%. Speed 11.99 mB/s
All OK

Wanterlude

01-08-2023, 03:46

My test with xtool v 0.77 + oodle 2.9.5

Sorry I forgot to edit that...
After Razor updated xtool (or plugin), everything works fine :D

Wanterlude

01-08-2023, 13:17

Hello everyone!
Does anyone know how to fix these errors? (Xtool&UnityPlugin latest versions)
Game: Pathfinder: Wrath of the Righteous (Build 11836620)
Dir: Bundle folder (34 gb)
xtool.exe run as administrator
Game folder not in read only
https://i.ibb.co/njNL95L/231226.png

L0v3craft

01-08-2023, 16:11

Someone tried to precompress Baldur's Gate 3?

Sample (Textures_2.pak):

https://gofile.io/d/xXO3cs

Another sample:

https://gofile.io/d/0n69EX

KaktoR

01-08-2023, 22:57

I guess they using the same engine as in Divinity 2, I bet they still use lz4. Maybe try to use the lspk bms script and make a xtl plugin.

I don't looked at the files so I cannot say anything atm. I am at work lol

L0v3craft

01-08-2023, 23:57

I guess they using the same engine as in Divinity 2, I bet they still use lz4. Maybe try to use the lspk bms script and make a xtl plugin.

I don't looked at the files so I cannot say anything atm. I am at work lol

I didn't try to create the database but I already tried with quickbms and that script. It can't unpack the game files.

Nekhalo22

02-08-2023, 12:36

How to repack game Ultimate Marvel vs. Capcom 3
What Tools ?
I want a video explanation

ultimate Marvel vs capcom 3
method used ztool zlib streams
pzlib+srep:m5f:a1+lolz:dt1:dtp1:dto1:dtm1:dtw1:dtd 1:mtt1:mt6:d100m:tt4
initial size 3.53 gb
Repack size 1.75 gb

How do I use this

ravikant

02-08-2023, 20:07

How to repack game Ultimate Marvel vs. Capcom 3
What Tools ?
I want a video explanation

ultimate Marvel vs capcom 3
method used ztool zlib streams
pzlib+srep:m5f:a1+lolz:dt1:dtp1:dto1:dtm1:dtw1:dtd 1:mtt1:mt6:d100m:tt4
initial size 3.53 gb
Repack size 1.75 gb

How do I use this

You can use this tool (https://fileforums.com/showthread.php?t=103687) to compress your files with your method

Masquerade

03-08-2023, 00:52

How to repack game Ultimate Marvel vs. Capcom 3
What Tools ?
I want a video explanation

I already told you what tools you need to use in your other topic.

Not ztool. It's old. XTool, srep, lolz and oggre.

Nobody will make a video.

Razor12911

03-08-2023, 05:43

Someone tried to precompress Baldur's Gate 3?

Sample (Textures_2.pak):

https://gofile.io/d/xXO3cs

Another sample:

https://gofile.io/d/0n69EX

I made a quick modification to the lspk script, I'm no expert when it comes to bms scripting but eh...

Shared.pak
Compressed 1 file, 78,790,712 => 278,618,150 bytes. Ratio 353.62%
Compression time: cpu 0.05 sec/real 2.87 sec = 2%. Speed 27.47 mB/s

via bg3.xtl (generated using bms2xtl)

bms2xtl.ini
[Config]
SafeMode=1
UserInput=
Exclude=copy
lz4=lz4hc:l9

use liblz4.dll that comes with xtool attachment

Note that on multi pak files, the script will fail that's why safemode is on, for the streams in multi pak files, you have to make sure that freearc captures these files in sequence else the streams will be left behind.

I don't have this game and I can't add support for multi pak files so hope that this approach works.

Fak Eid

03-08-2023, 11:47

xtool:mkraken:8+srep_new+lolz

Grand Theft Auto III: The Definitive Edition - 3.17GB
Grand Theft Auto Vice City: The Definitive Edition - 6.8GB
Grand Theft Auto San Andreas: The Definitive Edition - 14.2GB

Fak Eid

04-08-2023, 06:10

So, I have a question here:

The problem with kraken compressor is that it takes a lot of CPU memory and takes lot of time to extract.

So, while trying to compress GTA Trilogy using kraken:8, it takes around 45 mins to install all three games on average PC. Is there anyway to speed up the extraction process by adding a pre-compressor to the algo?

Nekhalo22

04-08-2023, 10:47

I already told you what tools you need to use in your other topic.

Not ztool. It's old. XTool, srep, lolz and oggre.

Nobody will make a video.

thanks bro

Masquerade

04-08-2023, 11:05

So, I have a question here:

The problem with kraken compressor is that it takes a lot of CPU memory and takes lot of time to extract.

So, while trying to compress GTA Trilogy using kraken:8, it takes around 45 mins to install all three games on average PC. Is there anyway to speed up the extraction process by adding a pre-compressor to the algo?

The Oodle family of compression algorithms are very CPU intensive. You can't speed it up.

XTool or Oodlereck, which processes the kraken streams, is the precompressor. Precompressors decompress data and compress it back during installation.

Illume

05-08-2023, 08:19

for the streams in multi pak files, you have to make sure that freearc captures these files in sequence

it doesn't seems to work
FreeArc is correctly sorted files
Textures.pak
Textures_1.pak
Textures_2.pak
Textures_3.pak

solid .arc, bg3.xtl made of separated pak's
Compressed 4 files, 13,743,119,215 => 16,734,251,958 bytes. Ratio 121.76%

per file (not solid) .arc, bg3.xtl made of separated pak's
Compressed 4 files, 13,743,119,215 => 16,734,252,036 bytes. Ratio 121.76%
Pos Size Compressed Files
------------------------------------------------------
31 4,293,937,007 7,285,063,990 1
7,285,064,021 4,293,777,600 4,293,780,186 1
11,578,844,207 4,294,919,680 4,294,922,266 1
15,873,766,473 860,484,928 860,485,594 1

solid .arc, bg3.xtl made of joined pak's
Compressed 4 files, 13,743,119,215 => 16,734,251,958 bytes. Ratio 121.76%

per file (not solid) .arc, bg3.xtl made of joined pak's
Compressed 4 files, 13,743,119,215 => 13,743,127,639 bytes. Ratio 100.00%

so as you can see only first file (Textures.pak) is decompressed
and it's doesn't matter if you join all Textures*.pak to one file and made bg3.xtl from it

Razor12911

05-08-2023, 18:39

On Textures.pak, Textures_1.pak, Textures_2.pak and Textures_3.pak
XTool is created by Razor12911

Streams: 8495 / 8495
Time: 00:01:25 (CPU 00:14:32)

Size: 12.8 GB >> 20.8 GB

Done!!!

To everyone here plus repackers (I won't say, you know yourselves) who use my tools, I had to set aside time and download 120GB game just so I could tweak a 1KB script by add a few lines (only 6 lines were missing), just simple copy and paste from original script then the script will work.

XTool has so many ways of enabling anyone of any knowledge and skill level to be able to make a plugin that can work on game without requiring the author to every time provide and provide, I thought that making bms2xtl and adding generate feature would at least bridge the gap but it doesn't seem to be case, I spent time on making the documentation, explaining things, I made the source code available but then what's the point? And don't bother replying to this thread (we only post methods here), I'm just venting don't mind me.

I actually hope all games use leviathan level 9 so compressing and repacking games becomes pointless.

KaktoR

06-08-2023, 01:42

Baldurs Gate 3

Bink2 videos just stored as plain text.
https://i.imgur.com/fzQhGm9.png

Unfortunatelly the XTool Bink video plugin will not work here for "GUS_CGI01_Part2.bk2". I tested both versions of the plugin and get extraction errors, not matter what settings I use (c/--diff).
0000000000000028 513393104 Mods\Gustav\Localization\English\Video\GUS_CGI01_P art1.bk2
000000001e99c228 1169576856 Mods\Gustav\Localization\English\Video\GUS_CGI01_P art2.bk2
00000000645015e8 35526400 Video\Splash_Logo_Larian.bk2

For "GUS_CGI01_Part2.bk2" you can go the old way (https://fileforums.com/showthread.php?t=104981) of manually patching. The diff file is 15.1mb.

You can unpack the file with the above modified bms script. Just make sure you null the entire stream in "Data\Gustav_Video.pak" before making the repack and don't forget to patch the file back after installing with either xtool or quickbms.

So basically extract "GUS_CGI01_Part2.bk2" out of the file and patch the entire stream out. Manually make a diff file for it and for the modified "Gustav_Video.pak" use XTool bink plugin.

As for the audio files, they contain wem's, but wemtool can't do anything here.

Edit:
bg3.xtl
Streams: 15486 / 15486
Time: 00:00:23 (CPU 00:00:30)

Size: 1.21 GB >> 1.86 GB

Done!!!

zlib

Streams: 189 / 314
Time: 00:00:10 (CPU 00:00:49)

Size: 1.86 GB >> 1.99 GB

Done!!!

Masquerade

06-08-2023, 05:38

For "GUS_CGI01_Part2.bk2" you can go the old way (https://fileforums.com/showthread.php?t=104981) of manually patching. The diff file is 15.1mb.

How big is the original video file?

Please may you set the --diff parameter to 100p to hypothetically see if the XTool plugin version will accept the stream at all? I'm thinking XTool may just be seeing how large the patch is and skipping - from your descriotion, bink pack seems to handle it alright.

KaktoR

06-08-2023, 05:46

The file is 1.08gb (1169576856 bytes).

BPK works but it will break the file on unpacking stage (crc error).

I already tried to set --diff=100p but to no avail, the ratio is just 100% for some reason (STDIO).
The old plugin works but it's the same as normal BPK, it will break the file on unpacking.

Tried different chunk sizes aswell, from c256mb up to c4gb.

Edit: I got some things mixed up, but nevertheless it doesn't work. Here is the full test.

STDIO plugin
GUS_CGI01_Part1.bk2
Compressed 1 file, 513,393,104 => 467,499,327 bytes. Ratio 91.06%
Compression time: cpu 0.20 sec/real 67.98 sec = 0%. Speed 7.55 mB/s
All OK

Extracted 1 file, 467,499,327 => 513,393,104 bytes. Ratio 91.06%
Extraction time: cpu 0.45 sec/real 32.36 sec = 1%. Speed 15.86 mB/s
All OK

GUS_CGI01_Part2.bk2
Compressed 1 file, 1,169,576,856 => 982,148,782 bytes. Ratio 83.97%
Compression time: cpu 0.69 sec/real 151.34 sec = 0%. Speed 7.73 mB/s
All OK

Extracting 1 file, 1,169,576,856 bytes. Processed 0.0%
ERROR: general (de)compression error in xtool:bk2

Splash_Logo_Larian.bk2
Compressed 1 file, 35,526,400 => 28,360,433 bytes. Ratio 79.83%
Compression time: cpu 0.05 sec/real 5.35 sec = 1%. Speed 6.64 mB/s
All OK

Extracted 1 file, 28,360,433 => 35,526,400 bytes. Ratio 79.83%
Extraction time: cpu 0.05 sec/real 3.01 sec = 2%. Speed 11.79 mB/s
All OK

Non-STDIO plugin
GUS_CGI01_Part2.bk2
Compressed 1 file, 1,169,576,856 => 1,169,577,126 bytes. Ratio 100.00%
Compression time: cpu 0.73 sec/real 2.50 sec = 29%. Speed 467.63 mB/s
All OK

Extracted 1 file, 1,169,577,126 => 1,169,576,856 bytes. Ratio 100.00%
Extraction time: cpu 0.75 sec/real 2.08 sec = 36%. Speed 562.62 mB/s
All OK

And it does not matter if I use xtool stdio mode or not
Compressing 1 file, 1,169,576,856 bytes. Processed 83.1%
Compressing 1,169,576,856 bytes with xtool.exe precomp -c2048mb -mbk2 -t100p --diff=100p InFile OutFile
100.0%XTool is created by Razor12911

Streams: 0 / 1
Time: 00:03:19 (CPU 00:00:02)

Size: 1.09 GB >> 1.09 GB

100.0%
Errorlevel=0
Compressed 1 file, 1,169,576,856 => 1,169,577,126 bytes. Ratio 100.00%
Compression time: cpu 0.81 sec/real 226.68 sec = 0%. Speed 5.16 mB/s
All OK

XTool.ini
[bink]
Encode=bink.exe <filein> <fileout>
Decode=binku.exe <filein> <fileout>

bk2.ini, bink.exe, binku.exe, xdelta3_dll.dll are present

Btw, with -c2gb I get a access violation error in xtool.exe. I have to write -c2048mb to get the 2gb chunk size.

Edit 2: Sample https://pixeldrain.com/u/bnfw8nDp

Hexagon123

06-08-2023, 19:44

Not possible with XTool. You can have the game running with extracted disc file but needs EXE patching. Password to decrypt disc can be found here: https://github.com/Masquerade64/SoS-AWL-Password
Just normal ZipCrypto. Compressed data with lzs algo, very similar to early lz4.

The process is similar to Mega Man Legacy Collection 2 (2017)?

Masquerade

07-08-2023, 00:34

The process is similar to Mega Man Legacy Collection 2 (2017)?

Try this for MMLC2:

https://www.vg-resource.com/thread-41735.html

Fak Eid

07-08-2023, 10:00

Hey guys, I tried the methods mentioned in the forum to compress Scarlet Nexus, but didn't get the required(smaller) file size. Is there any method to compress?

ScOOt3r

07-08-2023, 12:09

Did you use the Plugin for Scarlett Nexus.. that will help, it was posted by Masqerade i believe.

ScOOt3r

Masquerade

07-08-2023, 14:12

Hey guys, I tried the methods mentioned in the forum to compress Scarlet Nexus, but didn't get the required(smaller) file size. Is there any method to compress?

Use the XTool Unreal Engine 5 plugin by Razor12911, or the Unreal Engine UTOC/UCAS database maker.

I've posted a database in the past but it's probably outdated now.

KaktoR

08-08-2023, 07:38

Baldurs Gate 3

11:48:57 - Selected ARC/SPLIT method for Data1a-01.bin was: xtool:dd3+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:m c1000000/$deflatepack=xtool:c128mb:mpreflate:mpng:d1+xtool: dd3+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc10000 00/$mp3pack=7zdll-pm:m0=mp3det:m1=packmp3c
11:48:57 - Selected ARC/DS method for Data1b-01.bin was: xtool:mbg3+xtool:mzlib+xtool:dd3+4x4:b64mb:lzma:ul tra:64m:bt4:fb273:lc8:mc1000000
-------------------------------------------------------------------------------------------------------------------------------------------
15:22:17 - Overall input size: 124.12 GB
15:22:17 - Overall output size: 87.97 GB (Ratio 70.88%)
15:22:17 - Overall conversion time: 03:33:18
-------------------------------------------------------------------------------------------------------------------------------------------
15:22:27 - Starting Data1a-01.bin archive decompression.
15:23:01 - Starting Data1b-01.bin archive decompression.
15:56:45 - Overall decompression time: 00:34:17
15:56:45 - All files have been successfully extracted!

dixen

08-08-2023, 08:26

Does anyone know how to correctly uncompress the PAK files in the State of Decay2 game?
Quickbms + SOD2.bms =7.6 gb > ~11 gb
Xtool + lz4hc:l9 + sod2.xtl = 7.6 gb > 8.1 gb

Tested file pakchunk000004-WindowsNoEditor.pak

NOTE.
Applying Xdelta has no effect

KaktoR

08-08-2023, 08:43

Try with this custom sod2 scripts.

Credits to spiritovod

dixen

08-08-2023, 08:53

Try with this custom sod2 scripts.

Credits to spiritovod

Unfortunately, none of the three worked.

Razor12911

08-08-2023, 16:23

Does anyone know how to correctly uncompress the PAK files in the State of Decay2 game?
Quickbms + SOD2.bms =7.6 gb > ~11 gb
Xtool + lz4hc:l9 + sod2.xtl = 7.6 gb > 8.1 gb

Tested file pakchunk000004-WindowsNoEditor.pak

NOTE.
Applying Xdelta has no effect

If I remember correctly this game uses standard lz4, not lz4hc but I could be mistaken. Also I think -mue4:lz4 (via unreal engine plugin) should work too in case you're having issues with database.

ravikant

08-08-2023, 22:52

How to compress *.naz files in total overdose, i need to make a lossy repack

Masquerade

09-08-2023, 00:21

How to compress *.naz files in total overdose, i need to make a lossy repack

Use naztozip linked here to convert the archive:
https://github.com/Michael0ne/TOD_tools/blob/master/naztozip.exe

From there, you should be able to browse their contents and get a good look at what they contain.

L0v3craft

10-08-2023, 10:47

Someone checked "Atlas Fallen"? Looks like it is compressed with lz4f. XTool detects the streams but is not able to process them.

I tried with the script "thesurge2.bms" to unpack the files selecting the "toc" but quickbms gives error:

https://t88.pixhost.to/thumbs/861/372412985_immagine.png (https://pixhost.to/show/861/372412985_immagine.png)

KaktoR

10-08-2023, 11:26

Just use -mlz4f,l2

Compressed 1 file, 9,574,907 => 34,654,139 bytes. Ratio 361.93%
Compression time: cpu 0.02 sec/real 1.04 sec = 1%. Speed 9.18 mB/s
All OK

Extracted 1 file, 34,654,139 => 9,574,907 bytes. Ratio 361.93%
Extraction time: cpu 0.02 sec/real 1.08 sec = 1%. Speed 8.86 mB/s
All OK

Edit:

[0] Processing lz4f stream at 0000000000000000 (1522893 >> 2066084 >> 1522889) using l2:b5:d0 has failed
There are just a few bytes which not match, so use xdelta library for precompressing and decompressing.

Danziel123

11-08-2023, 00:42

Just use -mlz4f,l2

Compressed 1 file, 9,574,907 => 34,654,139 bytes. Ratio 361.93%
Compression time: cpu 0.02 sec/real 1.04 sec = 1%. Speed 9.18 mB/s
All OK

Extracted 1 file, 34,654,139 => 9,574,907 bytes. Ratio 361.93%
Extraction time: cpu 0.02 sec/real 1.08 sec = 1%. Speed 8.86 mB/s
All OK

Edit:

[0] Processing lz4f stream at 0000000000000000 (1522893 >> 2066084 >> 1522889) using l2:b5:d0 has failed
There are just a few bytes which not match, so use xdelta library for precompressing and decompressing.
Sorry if i joined, I choose this compress method in DSG to compress this game is right or not?

xtool:mlz4f,l2:dd3+LZMA-MT

Thank you:)

Fak Eid

11-08-2023, 10:34

*.dat -> xtool:mlz4f+srep+lolz [22.2GB >> 15.3GB]
*.webm -> mpz_pmt [3.15GB >> 3.10GB]
Rest everything -> srep+lzma

I also think audio\speech-german.fsb is a language file and can be skipped, or included in additional components. I'll run some checks on language and update

ravikant

13-08-2023, 07:29

I was compressing some *.big files and it was in a sounds folder, so i tried to use a media compressor and i run some tests

When i used the media compressor ratio is like 70-80%
When i was not using them i was getting 10-20%

This heavy difference is due to encryption? If anyone knows any tool to decrypt this file please share, these files are commonly found in EA Games

Masquerade

13-08-2023, 07:39

What game and can you share a small sample?

kj911

13-08-2023, 08:52

I was compressing some *.big files and it was in a sounds folder, so i tried to use a media compressor and i run some tests

When i used the media compressor ratio is like 70-80%
When i was not using them i was getting 10-20%

This heavy difference is due to encryption? If anyone knows any tool to decrypt this file please share, these files are commonly found in EA Games

BigGUI (https://dl.fifa-infinity.com/fifa-09/biggui-tool/) or FinalBIG (https://www.cnclabs.com/downloads/details.aspx?id=240) (and the any alternative (https://www.moddb.com/downloads/final-big-editor/)) tools??

ravikant

13-08-2023, 20:18

BigGUI (https://dl.fifa-infinity.com/fifa-09/biggui-tool/) or FinalBIG (https://www.cnclabs.com/downloads/details.aspx?id=240) (and the any alternative (https://www.moddb.com/downloads/final-big-editor/)) tools??

These tools will be useful i myself earlier found such a tool but it could not only decrypt it but not pack them back after i am done with recoding

shazzla

13-08-2023, 22:30

@Fak Eid

Webm container supports only vorbis/opus audio.
Mpz is not for compressing such files! Ok,you gain some megas.
Btw mpz can compress almost everything with little gain. (In my experiences)

Edit :
Sry for "spamming". :)

KaktoR

14-08-2023, 02:56

Tempest Rising Playtest

pak/ucas kraken
bk2 kb2n

I guess it will be still the case in final release.

Masquerade

17-08-2023, 02:46

En Garde!

*.pak
oo2reck+oo2recm+srep+lolz
*.mp4
srep+LZMA2
Everything else
razor

8.2GB ---> 5.7GB

Used Oodle core v2.9.5

hwang4

19-08-2023, 01:08

Best Method Of Compression , plz! :)

Wanterlude

19-08-2023, 02:44

Best Method Of Compression , plz! :)
probably srep+lolz

KaktoR

23-08-2023, 07:05

Dragon Ball Z Kakarot
v2.0

14:05:52 - Selected ARC/DS method for Data1a-01.bin was: xtool:c128mb:mzlib+xtool:dd3+4x4:b64mb:lzma:ultra: 64m:bt4:fb273:lc8:mc1000000
14:05:52 - Selected ARC/DS method for Data1b-01.bin was: xtool:dd3+bcm
14:05:52 - Selected ARC/DS method for Data1c-01.bin was: xtool:dd3+zstd
14:05:52 - Selected ARC/DS method for Data1d-01.bin was: xtool:dd3+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:m c1000000
-------------------------------------------------------------------------------------------------------------------------------------------
15:43:13 - Overall input size: 46.96 GB
15:43:13 - Overall output size: 36.96 GB (Ratio 78.70%)
15:43:13 - Overall conversion time: 01:37:21
15:43:13 - Overall conversion media: 1 custom media
-------------------------------------------------------------------------------------------------------------------------------------------
16:04:51 - Overall decompression time: 00:21:31
16:04:51 - All files have been successfully extracted!

For USM videos you can use bcm.
zstd for AWB files performs best if you don't want to deal with them in other ways.

Danziel123

25-08-2023, 00:33

Hi, maybe someone know what best compress method for armored core VI. Thank you:)

Masquerade

25-08-2023, 02:47

Hi, maybe someone know what best compress method for armored core VI. Thank you:)

It uses Kraken at level 8. Please use core v2.8.14.

IRAQIGHOST

25-08-2023, 03:29

Hi
anyone know best compress for FIFA 23?

KaktoR

25-08-2023, 13:04

Hi
anyone know best compress for FIFA 23?

https://fileforums.com/showpost.php?p=488282&postcount=2

Masquerade

25-08-2023, 14:16

https://fileforums.com/showpost.php?p=488282&postcount=2

I'm sure that's gonna compress really well - Leviathan lvl 9 :D

Danziel123

25-08-2023, 22:55

It uses Kraken at level 8. Please use core v2.8.14.
Sorry if i ask again, is this true compress method for armored core?
xtool:mkraken:core_2.8.14+xtool:db:dd3+4x4:b64mb:l zma:ultra:64m:bt4:fb273:lc8:mc1000000
Or it same with elden ring?
Thank you:)

KaktoR

26-08-2023, 06:48

I'm sure that's gonna compress really well - Leviathan lvl 9 :D

Well, he asked for the best option, and I am sure this is the only best option here :D

KaktoR

26-08-2023, 08:13

It uses Kraken at level 8. Please use core v2.8.14.

Data3.bdt uses l9 :confused:

Masquerade

26-08-2023, 08:30

Yeah I realised something was up from my rubbish size posted above. Dodi repack is 11GB smaller. :(

Wanterlude

26-08-2023, 10:11

Yeah I realised something was up from my rubbish size posted above. Dodi repack is 11GB smaller. :(

I think it's because if you compress archives and video together, srep cuts off a lot. Maybe some of the videos are also inside the archives.
https://i.ibb.co/DtN4VCw/202025.png
Armored Core VI
srep+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000 000

Full game: 59,6 gb
Compressed: 45.7 gb

with kraken ~39-40 gb, but DT: ~75 min [11400F]

KaktoR

26-08-2023, 10:17

^
You can find out if you precompress the files in verbose mode with output and search for bink2 headers in hxd.

Edit: I did the thing explained above and indeed, there are bink2 video files inside the bdt files aswell.

Edit 2: Using bink2 xtool plugin on the unpacked data files
https://i.imgur.com/0wgFfDt.png

Masquerade

26-08-2023, 12:43

You don't need to precompress the data to find the bink2 data. It's there in plaintext.

Cut from mesa:
Possible media at 0x00000002447F695E, size 38398252, type 'bk2'
Possible media at 0x0000000246C9528A, size 7295388, type 'bk2'
Possible media at 0x000000024738A426, size 19681400, type 'bk2'
Possible media at 0x000000024864F49E, size 19954292, type 'bk2'
Possible media at 0x0000000249956F12, size 19175688, type 'bk2'
Possible media at 0x000000024ABA081A, size 23945160, type 'bk2'
Possible media at 0x000000024C2767E2, size 23441796, type 'bk2'
Possible media at 0x000000024D8D1966, size 27112728, type 'bk2'
Possible media at 0x000000024F2ACE7E, size 32755048, type 'bk2'
Possible media at 0x00000002511E9BE6, size 26470388, type 'bk2'
Possible media at 0x0000000252B283DA, size 1535932, type 'bk2'
Possible media at 0x0000000252C9F396, size 4926428, type 'bk2'
Possible media at 0x0000000253151F72, size 348236, type 'bk2'
Possible media at 0x00000002531A6FBE, size 6378408, type 'bk2'
Possible media at 0x00000002537BC366, size 7157732, type 'bk2'
Possible media at 0x0000000253E8FB4A, size 594098992, type 'bk2'
Possible media at 0x000000027752367A, size 838915148, type 'bk2'
Possible media at 0x00000002A9530AC6, size 576688184, type 'bk2'
Possible media at 0x00000002CBB29AFE, size 120534324, type 'bk2'
Possible media at 0x00000002D2E1D032, size 124427520, type 'bk2'
Possible media at 0x00000002DA4C6D32, size 440595344, type 'bk2'
Possible media at 0x00000002F48F60C2, size 299554936, type 'bk2'

Masquerade

26-08-2023, 13:03

They duplicated the entire movies folder in Data0.bdt.

KaktoR

26-08-2023, 13:10

Lol, so 11gb worth of trash. Nice devs.

KaktoR

27-08-2023, 01:09

Found 1010.bk2 in Data0.bdt at 00000000388FE48A
Found 1011.bk2 in Data0.bdt at 00000000392D20C2
Found 1012.bk2 in Data0.bdt at 0000000039CE9D02
Found 1210.bk2 in Data0.bdt at 000000003A762466
Found 14100.bk2 in Data0.bdt at 0000000044FB0862
Found 14110.bk2 in Data0.bdt at 0000000045358AA6
Found 14112.bk2 in Data0.bdt at 0000000045998EF6
Found 14113.bk2 in Data0.bdt at 00000000464AF7A6
Found 14114.bk2 in Data0.bdt at 0000000046CF6866
Found 14115.bk2 in Data0.bdt at 0000000046FACE8A
Found 14120.bk2 in Data0.bdt at 0000000047254EFA
Found 14130.bk2 in Data0.bdt at 0000000047959B86
Found 14135.bk2 in Data0.bdt at 00000000481C8CAE
Found 14200.bk2 in Data0.bdt at 00000000492608B2
Found 14210.bk2 in Data0.bdt at 00000000495538C6
Found 15000.bk2 in Data0.bdt at 0000000049E0E0F6
Found 15001.bk2 in Data0.bdt at 000000004A32084A
Found 15010.bk2 in Data0.bdt at 000000004A97EC32
Found 15020.bk2 in Data0.bdt at 000000004B065B8A
Found 15030.bk2 in Data0.bdt at 000000004B77EB22
Found 15100.bk2 in Data0.bdt at 000000004BA16E7A
Found 15101.bk2 in Data0.bdt at 000000004BF1A762
Found 15102.bk2 in Data0.bdt at 000000004C77040E
Found 15110.bk2 in Data0.bdt at 000000004D1C7926
Found 15120.bk2 in Data0.bdt at 000000004D6B1DE2
Found 15121.bk2 in Data0.bdt at 000000004D9FF84A
Found 15130.bk2 in Data0.bdt at 000000004DC890CE
Found 15140.bk2 in Data0.bdt at 000000004E07CC9E
Found 15200.bk2 in Data0.bdt at 000000004E5F924A
Found 15210.bk2 in Data0.bdt at 000000004EAC4D56
Found 15211.bk2 in Data0.bdt at 000000004EDE4D32
Found 15220.bk2 in Data0.bdt at 000000004F4E97C2
Found 15230.bk2 in Data0.bdt at 000000004FA1F4D2
Found 15300.bk2 in Data0.bdt at 000000005028E34E
Found 15301.bk2 in Data0.bdt at 0000000050873CAA
Found 15310.bk2 in Data0.bdt at 0000000050B9409A
Found 15311.bk2 in Data0.bdt at 000000005127F8EA
Found 15320.bk2 in Data0.bdt at 0000000051892A0A
Found 15325.bk2 in Data0.bdt at 0000000052010542
Found 15400.bk2 in Data0.bdt at 0000000052AE03A2
Found 15410.bk2 in Data0.bdt at 0000000053125B0E
Found 15420.bk2 in Data0.bdt at 0000000053AA461A
Found 15500.bk2 in Data0.bdt at 00000000543278F2
Found 15502.bk2 in Data0.bdt at 0000000054FB7C22
Found 15600.bk2 in Data0.bdt at 0000000055305566
Found 16000.bk2 in Data0.bdt at 0000000056489BFE
Found 16010.bk2 in Data0.bdt at 0000000057BAD4E2
Found 16020.bk2 in Data0.bdt at 000000005A7BAEE2
Found 16100.bk2 in Data0.bdt at 000000005C746F02
Found 16100.bk2 in Data0.bdt at 000000005C8BDEBE
Found 2000.bk2 in Data0.bdt at 000000005CA34E7A
Found 2010.bk2 in Data0.bdt at 000000006217FA7A
Found 20100.bk2 in Data0.bdt at 000000006BCE2992
Found 2020.bk2 in Data0.bdt at 000000006F6B6F56
Found 20200.bk2 in Data0.bdt at 0000000075846F8E
Found 2025.bk2 in Data0.bdt at 000000007A1615AE
Found 2030.bk2 in Data0.bdt at 000000007F1E2FFE
Found 20300.bk2 in Data0.bdt at 0000000084E1D612
Found 2040.bk2 in Data0.bdt at 000000008AC777CA
Found 2050.bk2 in Data0.bdt at 00000000915ED182
Found 20500.bk2 in Data0.bdt at 000000009A3E31C2
Found 20600.bk2 in Data0.bdt at 000000009E057AD6
Found 2070.bk2 in Data0.bdt at 00000000A30E58FA
Found 2080.bk2 in Data0.bdt at 00000000AA30256E
Found 20800.bk2 in Data0.bdt at 00000000AA36ABAA
Found 21000.bk2 in Data0.bdt at 00000000AFECB4C2
Found 21101.bk2 in Data0.bdt at 00000000B5A6C35A
Found 21200.bk2 in Data0.bdt at 00000000B60A53F6
Found 21201.bk2 in Data0.bdt at 00000000BBC685AE
Found 21301.bk2 in Data0.bdt at 00000000BC2A5772
Found 21400.bk2 in Data0.bdt at 00000000BC8EB1FE
Found 21401.bk2 in Data0.bdt at 00000000C03D8DAA
Found 21501.bk2 in Data0.bdt at 00000000C0A1E986
Found 21601.bk2 in Data0.bdt at 00000000C105F6C2
Found 21700.bk2 in Data0.bdt at 00000000C16A578A
Found 21701.bk2 in Data0.bdt at 00000000C6716A22
Found 21800.bk2 in Data0.bdt at 00000000C6D4D832
Found 21801.bk2 in Data0.bdt at 00000000CB7A1376
Found 21901.bk2 in Data0.bdt at 00000000CBDE6E82
Found 22000.bk2 in Data0.bdt at 00000000CC41EB16
Found 22101.bk2 in Data0.bdt at 00000000D2393B5E
Found 22110.bk2 in Data0.bdt at 00000000D37378E2
Found 22191.bk2 in Data0.bdt at 00000000DB9C100A
Found 22200.bk2 in Data0.bdt at 00000000DBF4C1F6
Found 22201.bk2 in Data0.bdt at 00000000E1BB6DE6
Found 22221.bk2 in Data0.bdt at 00000000E1FDD00A
Found 22231.bk2 in Data0.bdt at 00000000E245881E
Found 22300.bk2 in Data0.bdt at 00000000E3013516
Found 22301.bk2 in Data0.bdt at 00000000E8C12CEA
Found 22400.bk2 in Data0.bdt at 00000000E8F9D12E
Found 22500.bk2 in Data0.bdt at 00000000EE969FA2
Found 30110.bk2 in Data0.bdt at 00000000F32CE802
Found 30300.bk2 in Data0.bdt at 00000000F7D5CA46
Found 30430.bk2 in Data0.bdt at 00000000FCED9EC6
Found 32300.bk2 in Data0.bdt at 0000000103961F32
Found 33200.bk2 in Data0.bdt at 000000010F14DA2A
Found 34200.bk2 in Data0.bdt at 00000001171944B2
Found 34300.bk2 in Data0.bdt at 000000011C4D8E3E
Found 35000.bk2 in Data0.bdt at 0000000121DD8166
Found 40000.bk2 in Data0.bdt at 000000012829B81E
Found 40100.bk2 in Data0.bdt at 000000012E966AB2
Found 40200.bk2 in Data0.bdt at 00000001332CA4EA
Found 40210.bk2 in Data0.bdt at 00000001372D798A
Found 40300.bk2 in Data0.bdt at 000000013BB4BC02
Found 41000.bk2 in Data0.bdt at 000000013FA169DE
Found 41100.bk2 in Data0.bdt at 00000001442312E2
Found 50100.bk2 in Data0.bdt at 000000014AD8BCF2
Found 50200.bk2 in Data0.bdt at 00000001520E096A
Found 50300.bk2 in Data0.bdt at 0000000155DE2406
Found 50400.bk2 in Data0.bdt at 000000015BCAF752
Found 50500.bk2 in Data0.bdt at 0000000160AE8DA6
Found 51000.bk2 in Data0.bdt at 0000000166269A22
Found 51100.bk2 in Data0.bdt at 000000016BAD32C2
Found 6000.bk2 in Data0.bdt at 000000017207875A
Found 60000.bk2 in Data0.bdt at 000000017250997A
Found 6010.bk2 in Data0.bdt at 0000000174D5EB5A
Found 60100.bk2 in Data0.bdt at 00000001751E9206
Found 6020.bk2 in Data0.bdt at 000000017724373A
Found 60200.bk2 in Data0.bdt at 00000001776D45EA
Found 6030.bk2 in Data0.bdt at 000000017A3BABF6
Found 6040.bk2 in Data0.bdt at 000000017B20E8CE
Found 6050.bk2 in Data0.bdt at 000000017B6A220E
Found 6060.bk2 in Data0.bdt at 000000017BB305F6
Found 6070.bk2 in Data0.bdt at 000000017BB9DAE2
Found 6080.bk2 in Data0.bdt at 000000017C02E122
Found 6090.bk2 in Data0.bdt at 000000017C4C048A
Found 6100.bk2 in Data0.bdt at 000000017C94E326
Found 6110.bk2 in Data0.bdt at 000000017CDDD98E
Found 6200.bk2 in Data0.bdt at 000000017D26A43A
Found 6210.bk2 in Data0.bdt at 000000017D6EE3A2
Found 6220.bk2 in Data0.bdt at 000000017DB72312
Found 6300.bk2 in Data0.bdt at 000000017DFF80E6
Found 6310.bk2 in Data0.bdt at 000000017E192F22
Found 7000.bk2 in Data0.bdt at 000000017E32DC6E
Found 7010.bk2 in Data0.bdt at 00000001827C815A
Found 7020.bk2 in Data0.bdt at 0000000196D4CBE2
Found 7030.bk2 in Data0.bdt at 00000001AA01586E
Found 7100.bk2 in Data0.bdt at 00000001BEAAA0B2
Found 7110.bk2 in Data0.bdt at 00000001BFF22172
Found 79000.bk2 in Data0.bdt at 00000001C06ED9F2
Found 79100.bk2 in Data0.bdt at 00000001C4437E4A
Found 79200.bk2 in Data0.bdt at 00000001CB307D72
Found 79300.bk2 in Data0.bdt at 00000001CFC50BBA
Found 79400.bk2 in Data0.bdt at 00000001D5DD56D6
Found 79500.bk2 in Data0.bdt at 00000001DD4E7486
Found 79600.bk2 in Data0.bdt at 00000001E18BA296
Found 79700.bk2 in Data0.bdt at 00000001E716F6A2
Found 79800.bk2 in Data0.bdt at 00000001EBD43016
Found 79900.bk2 in Data0.bdt at 00000001EFCE84F2
Found 8000.bk2 in Data0.bdt at 00000001F5388FE6
Found 8001.bk2 in Data0.bdt at 00000001F58C8C02
Found 8002.bk2 in Data0.bdt at 00000001F5E08C22
Found 8003.bk2 in Data0.bdt at 00000001F62A061A
Found 8004.bk2 in Data0.bdt at 00000001F6738506
Found 8005.bk2 in Data0.bdt at 00000001F6BCFE9E
Found 8006.bk2 in Data0.bdt at 00000001F70691A6
Found 8007.bk2 in Data0.bdt at 00000001F75A9632
Found 8008.bk2 in Data0.bdt at 00000001F7B9154E
Found 8009.bk2 in Data0.bdt at 00000001F861124E
Found 8010.bk2 in Data0.bdt at 00000001F9139142
Found 8011.bk2 in Data0.bdt at 00000001F9C6154E
Found 8012.bk2 in Data0.bdt at 00000001FA2F13C2
Found 8013.bk2 in Data0.bdt at 00000001FA96AA2E
Found 8014.bk2 in Data0.bdt at 00000001FAF74796
Found 8015.bk2 in Data0.bdt at 00000001FB4175BE
Found 8016.bk2 in Data0.bdt at 00000001FB99A6F6
Found 8017.bk2 in Data0.bdt at 00000001FBF28EEE
Found 8018.bk2 in Data0.bdt at 00000001FC495AE2
Found 8019.bk2 in Data0.bdt at 00000001FC80AF02
Found 8020.bk2 in Data0.bdt at 00000001FCB72D62
Found 8021.bk2 in Data0.bdt at 00000001FCF413FA
Found 8022.bk2 in Data0.bdt at 00000001FD31B542
Found 8023.bk2 in Data0.bdt at 00000001FDEEB81E
Found 8024.bk2 in Data0.bdt at 00000001FED5C0EE
Found 8025.bk2 in Data0.bdt at 00000001FF7B9E76
Found 8026.bk2 in Data0.bdt at 00000002003898A2
Found 8027.bk2 in Data0.bdt at 00000002008C8ED6
Found 8028.bk2 in Data0.bdt at 0000000200E09B7E
Found 8029.bk2 in Data0.bdt at 00000002029DD346
Found 8030.bk2 in Data0.bdt at 00000002038F60DA
Found 8031.bk2 in Data0.bdt at 00000002049A0ABE
Found 8032.bk2 in Data0.bdt at 000000020569F296
Found 8033.bk2 in Data0.bdt at 000000020637C736
Found 8034.bk2 in Data0.bdt at 00000002071CA822
Found 8035.bk2 in Data0.bdt at 000000020807DCA2
Found 8036.bk2 in Data0.bdt at 0000000208E0291A
Found 8037.bk2 in Data0.bdt at 0000000209C0DBC2
Found 8038.bk2 in Data0.bdt at 000000020B8B5F9E
Found 8039.bk2 in Data0.bdt at 000000020C98237A
Found 8040.bk2 in Data0.bdt at 000000020CE257BE
Found 8041.bk2 in Data0.bdt at 000000020D36564A
Found 8042.bk2 in Data0.bdt at 000000020D990C26
Found 8043.bk2 in Data0.bdt at 000000020DDE6176
Found 8044.bk2 in Data0.bdt at 000000020E303E82
Found 8100.bk2 in Data0.bdt at 000000020EB060E6
Found 8101.bk2 in Data0.bdt at 000000020FAE8686
Found 8102.bk2 in Data0.bdt at 0000000210CE41D6
Found 8103.bk2 in Data0.bdt at 000000021197DFAE
Found 8104.bk2 in Data0.bdt at 0000000212702DEE
Found 8105.bk2 in Data0.bdt at 0000000213877C3A
Found 8106.bk2 in Data0.bdt at 0000000214A948DA
Found 8107.bk2 in Data0.bdt at 0000000215DDFC8A
Found 8108.bk2 in Data0.bdt at 00000002174E30AE
Found 8109.bk2 in Data0.bdt at 0000000217EBB52A
Found 8110.bk2 in Data0.bdt at 0000000218AB8972
Found 8111.bk2 in Data0.bdt at 0000000219F112FA
Found 8112.bk2 in Data0.bdt at 000000021BBDAD6E
Found 8200.bk2 in Data0.bdt at 000000021D44486E
Found 8201.bk2 in Data0.bdt at 000000021DA90ACA
Found 8202.bk2 in Data0.bdt at 000000021E0A4EE2
Found 8203.bk2 in Data0.bdt at 000000021EED1692
Found 8204.bk2 in Data0.bdt at 000000021FC82A5E
Found 8205.bk2 in Data0.bdt at 0000000220FAC9DA
Found 8206.bk2 in Data0.bdt at 00000002226C6BFE
Found 8207.bk2 in Data0.bdt at 0000000222E636BE
Found 8208.bk2 in Data0.bdt at 00000002248AF452
Found 8209.bk2 in Data0.bdt at 000000022609333A
Found 8210.bk2 in Data0.bdt at 00000002272F42B6
Found 8211.bk2 in Data0.bdt at 000000022872A89E
Found 8213.bk2 in Data0.bdt at 0000000229039BAE
Found 8214.bk2 in Data0.bdt at 000000022972E9AE
Found 8215.bk2 in Data0.bdt at 000000022A4B3902
Found 8216.bk2 in Data0.bdt at 000000022B10A3D6
Found 8217.bk2 in Data0.bdt at 000000022C0ECA0A
Found 8218.bk2 in Data0.bdt at 000000022CE399D2
Found 8219.bk2 in Data0.bdt at 000000022DF3345E
Found 8220.bk2 in Data0.bdt at 000000022F4119E6
Found 8221.bk2 in Data0.bdt at 000000023025FFC6
Found 8222.bk2 in Data0.bdt at 00000002312C878E
Found 8223.bk2 in Data0.bdt at 00000002319365EE
Found 8224.bk2 in Data0.bdt at 000000023274162E
Found 8225.bk2 in Data0.bdt at 0000000232DD14EE
Found 8226.bk2 in Data0.bdt at 0000000233701B3E
Found 8227.bk2 in Data0.bdt at 0000000234D950B2
Found 8228.bk2 in Data0.bdt at 0000000235318102
Found 8229.bk2 in Data0.bdt at 000000023676F89E
Found 8230.bk2 in Data0.bdt at 000000023798C43E
Found 8231.bk2 in Data0.bdt at 000000023929E82E
Found 8232.bk2 in Data0.bdt at 00000002398CA326
Found 8233.bk2 in Data0.bdt at 000000023ABD2182
Found 8234.bk2 in Data0.bdt at 000000023B9C69A2
Found 8235.bk2 in Data0.bdt at 000000023C5533D2
Found 8236.bk2 in Data0.bdt at 000000023DDDEB06
Found 8237.bk2 in Data0.bdt at 000000023E8A16B2
Found 8238.bk2 in Data0.bdt at 000000023F6047F6
Found 8300.bk2 in Data0.bdt at 000000023FE6AEB2
Found 8301.bk2 in Data0.bdt at 0000000240302C76
Found 8302.bk2 in Data0.bdt at 00000002407E9EEE
Found 8303.bk2 in Data0.bdt at 0000000240CA399E
Found 8304.bk2 in Data0.bdt at 00000002412F06CE
Found 8305.bk2 in Data0.bdt at 000000024191BD0E
Found 8306.bk2 in Data0.bdt at 0000000241F948EA
Found 8400.bk2 in Data0.bdt at 0000000242EACD76
Found 8401.bk2 in Data0.bdt at 000000024378396A
Found 8402.bk2 in Data0.bdt at 00000002447F695E
Found 8403.bk2 in Data0.bdt at 0000000246C9528A
Found 8500.bk2 in Data0.bdt at 000000024738A426
Found 8501.bk2 in Data0.bdt at 000000024864F49E
Found 8502.bk2 in Data0.bdt at 0000000249956F12
Found 8503.bk2 in Data0.bdt at 000000024ABA081A
Found 8504.bk2 in Data0.bdt at 000000024C2767E2
Found 8505.bk2 in Data0.bdt at 000000024D8D1966
Found 8506.bk2 in Data0.bdt at 000000024F2ACE7E
Found 8507.bk2 in Data0.bdt at 00000002511E9BE6
Found 16100.bk2 in Data0.bdt at 0000000252B283DA
Found menu_monitor_noise.bk2 in Data0.bdt at 0000000252C9F396
Found s00_00_0000_cut0000_01.bk2 in Data0.bdt at 0000000253151F72
Found s00_20_0000_cut0010_00.bk2 in Data0.bdt at 00000002531A6FBE
Found s00_20_0011_cut0010_00.bk2 in Data0.bdt at 00000002537BC366
Found s10_00_0020_cut0010_00.bk2 in Data0.bdt at 0000000253E8FB4A
Found s10_10_0000_cut0070_00.bk2 in Data0.bdt at 000000027752367A
Found s20_00_0000_cut0010_00.bk2 in Data0.bdt at 00000002A9530AC6
Found s20_20_0070_cut0010_00.bk2 in Data0.bdt at 00000002CBB29AFE
Found s30_00_0010_cut0100_00.bk2 in Data0.bdt at 00000002D2E1D032
Found s30_00_0030_cut0110_00.bk2 in Data0.bdt at 00000002DA4C6D32
Found s50_40_0010_cut0035_00.bk2 in Data0.bdt at 00000002F48F60C2

In total they are 276 files inside Data0.bdt, but it's missing "dummyMovie.bk2", so there has to be a file which is not in the movie folder.

However you can cut the files out of Data0.bdt and reimport them back later with xtool again if you want. The game starts normally with cutted files and is playing intro movies (it reads the movies from the movie folder).

For cut files
xtool erase "movies\*.bk2" "Data0.bdt" ac6for.xtl

For reimport files
xtool decode ac6for.xtl "movies" "Data0.bdt"

Edit:
CRC matching after erase/decode
original Data0.bdt = 70E16A53
erase/decode Data0.bdt = 70E16A53

So you can easily use xtool bink2 plugin for the movies folder and cut the files out of Data0.bdt without getting any errors and/or unwanted duplicates for the movies.

Danziel123

27-08-2023, 02:07

Found 1010.bk2 in Data0.bdt at 00000000388FE48A
Found 1011.bk2 in Data0.bdt at 00000000392D20C2
Found 1012.bk2 in Data0.bdt at 0000000039CE9D02
Found 1210.bk2 in Data0.bdt at 000000003A762466
Found 14100.bk2 in Data0.bdt at 0000000044FB0862
Found 14110.bk2 in Data0.bdt at 0000000045358AA6
Found 14112.bk2 in Data0.bdt at 0000000045998EF6
Found 14113.bk2 in Data0.bdt at 00000000464AF7A6
Found 14114.bk2 in Data0.bdt at 0000000046CF6866
Found 14115.bk2 in Data0.bdt at 0000000046FACE8A
Found 14120.bk2 in Data0.bdt at 0000000047254EFA
Found 14130.bk2 in Data0.bdt at 0000000047959B86
Found 14135.bk2 in Data0.bdt at 00000000481C8CAE
Found 14200.bk2 in Data0.bdt at 00000000492608B2
Found 14210.bk2 in Data0.bdt at 00000000495538C6
Found 15000.bk2 in Data0.bdt at 0000000049E0E0F6
Found 15001.bk2 in Data0.bdt at 000000004A32084A
Found 15010.bk2 in Data0.bdt at 000000004A97EC32
Found 15020.bk2 in Data0.bdt at 000000004B065B8A
Found 15030.bk2 in Data0.bdt at 000000004B77EB22
Found 15100.bk2 in Data0.bdt at 000000004BA16E7A
Found 15101.bk2 in Data0.bdt at 000000004BF1A762
Found 15102.bk2 in Data0.bdt at 000000004C77040E
Found 15110.bk2 in Data0.bdt at 000000004D1C7926
Found 15120.bk2 in Data0.bdt at 000000004D6B1DE2
Found 15121.bk2 in Data0.bdt at 000000004D9FF84A
Found 15130.bk2 in Data0.bdt at 000000004DC890CE
Found 15140.bk2 in Data0.bdt at 000000004E07CC9E
Found 15200.bk2 in Data0.bdt at 000000004E5F924A
Found 15210.bk2 in Data0.bdt at 000000004EAC4D56
Found 15211.bk2 in Data0.bdt at 000000004EDE4D32
Found 15220.bk2 in Data0.bdt at 000000004F4E97C2
Found 15230.bk2 in Data0.bdt at 000000004FA1F4D2
Found 15300.bk2 in Data0.bdt at 000000005028E34E
Found 15301.bk2 in Data0.bdt at 0000000050873CAA
Found 15310.bk2 in Data0.bdt at 0000000050B9409A
Found 15311.bk2 in Data0.bdt at 000000005127F8EA
Found 15320.bk2 in Data0.bdt at 0000000051892A0A
Found 15325.bk2 in Data0.bdt at 0000000052010542
Found 15400.bk2 in Data0.bdt at 0000000052AE03A2
Found 15410.bk2 in Data0.bdt at 0000000053125B0E
Found 15420.bk2 in Data0.bdt at 0000000053AA461A
Found 15500.bk2 in Data0.bdt at 00000000543278F2
Found 15502.bk2 in Data0.bdt at 0000000054FB7C22
Found 15600.bk2 in Data0.bdt at 0000000055305566
Found 16000.bk2 in Data0.bdt at 0000000056489BFE
Found 16010.bk2 in Data0.bdt at 0000000057BAD4E2
Found 16020.bk2 in Data0.bdt at 000000005A7BAEE2
Found 16100.bk2 in Data0.bdt at 000000005C746F02
Found 16100.bk2 in Data0.bdt at 000000005C8BDEBE
Found 2000.bk2 in Data0.bdt at 000000005CA34E7A
Found 2010.bk2 in Data0.bdt at 000000006217FA7A
Found 20100.bk2 in Data0.bdt at 000000006BCE2992
Found 2020.bk2 in Data0.bdt at 000000006F6B6F56
Found 20200.bk2 in Data0.bdt at 0000000075846F8E
Found 2025.bk2 in Data0.bdt at 000000007A1615AE
Found 2030.bk2 in Data0.bdt at 000000007F1E2FFE
Found 20300.bk2 in Data0.bdt at 0000000084E1D612
Found 2040.bk2 in Data0.bdt at 000000008AC777CA
Found 2050.bk2 in Data0.bdt at 00000000915ED182
Found 20500.bk2 in Data0.bdt at 000000009A3E31C2
Found 20600.bk2 in Data0.bdt at 000000009E057AD6
Found 2070.bk2 in Data0.bdt at 00000000A30E58FA
Found 2080.bk2 in Data0.bdt at 00000000AA30256E
Found 20800.bk2 in Data0.bdt at 00000000AA36ABAA
Found 21000.bk2 in Data0.bdt at 00000000AFECB4C2
Found 21101.bk2 in Data0.bdt at 00000000B5A6C35A
Found 21200.bk2 in Data0.bdt at 00000000B60A53F6
Found 21201.bk2 in Data0.bdt at 00000000BBC685AE
Found 21301.bk2 in Data0.bdt at 00000000BC2A5772
Found 21400.bk2 in Data0.bdt at 00000000BC8EB1FE
Found 21401.bk2 in Data0.bdt at 00000000C03D8DAA
Found 21501.bk2 in Data0.bdt at 00000000C0A1E986
Found 21601.bk2 in Data0.bdt at 00000000C105F6C2
Found 21700.bk2 in Data0.bdt at 00000000C16A578A
Found 21701.bk2 in Data0.bdt at 00000000C6716A22
Found 21800.bk2 in Data0.bdt at 00000000C6D4D832
Found 21801.bk2 in Data0.bdt at 00000000CB7A1376
Found 21901.bk2 in Data0.bdt at 00000000CBDE6E82
Found 22000.bk2 in Data0.bdt at 00000000CC41EB16
Found 22101.bk2 in Data0.bdt at 00000000D2393B5E
Found 22110.bk2 in Data0.bdt at 00000000D37378E2
Found 22191.bk2 in Data0.bdt at 00000000DB9C100A
Found 22200.bk2 in Data0.bdt at 00000000DBF4C1F6
Found 22201.bk2 in Data0.bdt at 00000000E1BB6DE6
Found 22221.bk2 in Data0.bdt at 00000000E1FDD00A
Found 22231.bk2 in Data0.bdt at 00000000E245881E
Found 22300.bk2 in Data0.bdt at 00000000E3013516
Found 22301.bk2 in Data0.bdt at 00000000E8C12CEA
Found 22400.bk2 in Data0.bdt at 00000000E8F9D12E
Found 22500.bk2 in Data0.bdt at 00000000EE969FA2
Found 30110.bk2 in Data0.bdt at 00000000F32CE802
Found 30300.bk2 in Data0.bdt at 00000000F7D5CA46
Found 30430.bk2 in Data0.bdt at 00000000FCED9EC6
Found 32300.bk2 in Data0.bdt at 0000000103961F32
Found 33200.bk2 in Data0.bdt at 000000010F14DA2A
Found 34200.bk2 in Data0.bdt at 00000001171944B2
Found 34300.bk2 in Data0.bdt at 000000011C4D8E3E
Found 35000.bk2 in Data0.bdt at 0000000121DD8166
Found 40000.bk2 in Data0.bdt at 000000012829B81E
Found 40100.bk2 in Data0.bdt at 000000012E966AB2
Found 40200.bk2 in Data0.bdt at 00000001332CA4EA
Found 40210.bk2 in Data0.bdt at 00000001372D798A
Found 40300.bk2 in Data0.bdt at 000000013BB4BC02
Found 41000.bk2 in Data0.bdt at 000000013FA169DE
Found 41100.bk2 in Data0.bdt at 00000001442312E2
Found 50100.bk2 in Data0.bdt at 000000014AD8BCF2
Found 50200.bk2 in Data0.bdt at 00000001520E096A
Found 50300.bk2 in Data0.bdt at 0000000155DE2406
Found 50400.bk2 in Data0.bdt at 000000015BCAF752
Found 50500.bk2 in Data0.bdt at 0000000160AE8DA6
Found 51000.bk2 in Data0.bdt at 0000000166269A22
Found 51100.bk2 in Data0.bdt at 000000016BAD32C2
Found 6000.bk2 in Data0.bdt at 000000017207875A
Found 60000.bk2 in Data0.bdt at 000000017250997A
Found 6010.bk2 in Data0.bdt at 0000000174D5EB5A
Found 60100.bk2 in Data0.bdt at 00000001751E9206
Found 6020.bk2 in Data0.bdt at 000000017724373A
Found 60200.bk2 in Data0.bdt at 00000001776D45EA
Found 6030.bk2 in Data0.bdt at 000000017A3BABF6
Found 6040.bk2 in Data0.bdt at 000000017B20E8CE
Found 6050.bk2 in Data0.bdt at 000000017B6A220E
Found 6060.bk2 in Data0.bdt at 000000017BB305F6
Found 6070.bk2 in Data0.bdt at 000000017BB9DAE2
Found 6080.bk2 in Data0.bdt at 000000017C02E122
Found 6090.bk2 in Data0.bdt at 000000017C4C048A
Found 6100.bk2 in Data0.bdt at 000000017C94E326
Found 6110.bk2 in Data0.bdt at 000000017CDDD98E
Found 6200.bk2 in Data0.bdt at 000000017D26A43A
Found 6210.bk2 in Data0.bdt at 000000017D6EE3A2
Found 6220.bk2 in Data0.bdt at 000000017DB72312
Found 6300.bk2 in Data0.bdt at 000000017DFF80E6
Found 6310.bk2 in Data0.bdt at 000000017E192F22
Found 7000.bk2 in Data0.bdt at 000000017E32DC6E
Found 7010.bk2 in Data0.bdt at 00000001827C815A
Found 7020.bk2 in Data0.bdt at 0000000196D4CBE2
Found 7030.bk2 in Data0.bdt at 00000001AA01586E
Found 7100.bk2 in Data0.bdt at 00000001BEAAA0B2
Found 7110.bk2 in Data0.bdt at 00000001BFF22172
Found 79000.bk2 in Data0.bdt at 00000001C06ED9F2
Found 79100.bk2 in Data0.bdt at 00000001C4437E4A
Found 79200.bk2 in Data0.bdt at 00000001CB307D72
Found 79300.bk2 in Data0.bdt at 00000001CFC50BBA
Found 79400.bk2 in Data0.bdt at 00000001D5DD56D6
Found 79500.bk2 in Data0.bdt at 00000001DD4E7486
Found 79600.bk2 in Data0.bdt at 00000001E18BA296
Found 79700.bk2 in Data0.bdt at 00000001E716F6A2
Found 79800.bk2 in Data0.bdt at 00000001EBD43016
Found 79900.bk2 in Data0.bdt at 00000001EFCE84F2
Found 8000.bk2 in Data0.bdt at 00000001F5388FE6
Found 8001.bk2 in Data0.bdt at 00000001F58C8C02
Found 8002.bk2 in Data0.bdt at 00000001F5E08C22
Found 8003.bk2 in Data0.bdt at 00000001F62A061A
Found 8004.bk2 in Data0.bdt at 00000001F6738506
Found 8005.bk2 in Data0.bdt at 00000001F6BCFE9E
Found 8006.bk2 in Data0.bdt at 00000001F70691A6
Found 8007.bk2 in Data0.bdt at 00000001F75A9632
Found 8008.bk2 in Data0.bdt at 00000001F7B9154E
Found 8009.bk2 in Data0.bdt at 00000001F861124E
Found 8010.bk2 in Data0.bdt at 00000001F9139142
Found 8011.bk2 in Data0.bdt at 00000001F9C6154E
Found 8012.bk2 in Data0.bdt at 00000001FA2F13C2
Found 8013.bk2 in Data0.bdt at 00000001FA96AA2E
Found 8014.bk2 in Data0.bdt at 00000001FAF74796
Found 8015.bk2 in Data0.bdt at 00000001FB4175BE
Found 8016.bk2 in Data0.bdt at 00000001FB99A6F6
Found 8017.bk2 in Data0.bdt at 00000001FBF28EEE
Found 8018.bk2 in Data0.bdt at 00000001FC495AE2
Found 8019.bk2 in Data0.bdt at 00000001FC80AF02
Found 8020.bk2 in Data0.bdt at 00000001FCB72D62
Found 8021.bk2 in Data0.bdt at 00000001FCF413FA
Found 8022.bk2 in Data0.bdt at 00000001FD31B542
Found 8023.bk2 in Data0.bdt at 00000001FDEEB81E
Found 8024.bk2 in Data0.bdt at 00000001FED5C0EE
Found 8025.bk2 in Data0.bdt at 00000001FF7B9E76
Found 8026.bk2 in Data0.bdt at 00000002003898A2
Found 8027.bk2 in Data0.bdt at 00000002008C8ED6
Found 8028.bk2 in Data0.bdt at 0000000200E09B7E
Found 8029.bk2 in Data0.bdt at 00000002029DD346
Found 8030.bk2 in Data0.bdt at 00000002038F60DA
Found 8031.bk2 in Data0.bdt at 00000002049A0ABE
Found 8032.bk2 in Data0.bdt at 000000020569F296
Found 8033.bk2 in Data0.bdt at 000000020637C736
Found 8034.bk2 in Data0.bdt at 00000002071CA822
Found 8035.bk2 in Data0.bdt at 000000020807DCA2
Found 8036.bk2 in Data0.bdt at 0000000208E0291A
Found 8037.bk2 in Data0.bdt at 0000000209C0DBC2
Found 8038.bk2 in Data0.bdt at 000000020B8B5F9E
Found 8039.bk2 in Data0.bdt at 000000020C98237A
Found 8040.bk2 in Data0.bdt at 000000020CE257BE
Found 8041.bk2 in Data0.bdt at 000000020D36564A
Found 8042.bk2 in Data0.bdt at 000000020D990C26
Found 8043.bk2 in Data0.bdt at 000000020DDE6176
Found 8044.bk2 in Data0.bdt at 000000020E303E82
Found 8100.bk2 in Data0.bdt at 000000020EB060E6
Found 8101.bk2 in Data0.bdt at 000000020FAE8686
Found 8102.bk2 in Data0.bdt at 0000000210CE41D6
Found 8103.bk2 in Data0.bdt at 000000021197DFAE
Found 8104.bk2 in Data0.bdt at 0000000212702DEE
Found 8105.bk2 in Data0.bdt at 0000000213877C3A
Found 8106.bk2 in Data0.bdt at 0000000214A948DA
Found 8107.bk2 in Data0.bdt at 0000000215DDFC8A
Found 8108.bk2 in Data0.bdt at 00000002174E30AE
Found 8109.bk2 in Data0.bdt at 0000000217EBB52A
Found 8110.bk2 in Data0.bdt at 0000000218AB8972
Found 8111.bk2 in Data0.bdt at 0000000219F112FA
Found 8112.bk2 in Data0.bdt at 000000021BBDAD6E
Found 8200.bk2 in Data0.bdt at 000000021D44486E
Found 8201.bk2 in Data0.bdt at 000000021DA90ACA
Found 8202.bk2 in Data0.bdt at 000000021E0A4EE2
Found 8203.bk2 in Data0.bdt at 000000021EED1692
Found 8204.bk2 in Data0.bdt at 000000021FC82A5E
Found 8205.bk2 in Data0.bdt at 0000000220FAC9DA
Found 8206.bk2 in Data0.bdt at 00000002226C6BFE
Found 8207.bk2 in Data0.bdt at 0000000222E636BE
Found 8208.bk2 in Data0.bdt at 00000002248AF452
Found 8209.bk2 in Data0.bdt at 000000022609333A
Found 8210.bk2 in Data0.bdt at 00000002272F42B6
Found 8211.bk2 in Data0.bdt at 000000022872A89E
Found 8213.bk2 in Data0.bdt at 0000000229039BAE
Found 8214.bk2 in Data0.bdt at 000000022972E9AE
Found 8215.bk2 in Data0.bdt at 000000022A4B3902
Found 8216.bk2 in Data0.bdt at 000000022B10A3D6
Found 8217.bk2 in Data0.bdt at 000000022C0ECA0A
Found 8218.bk2 in Data0.bdt at 000000022CE399D2
Found 8219.bk2 in Data0.bdt at 000000022DF3345E
Found 8220.bk2 in Data0.bdt at 000000022F4119E6
Found 8221.bk2 in Data0.bdt at 000000023025FFC6
Found 8222.bk2 in Data0.bdt at 00000002312C878E
Found 8223.bk2 in Data0.bdt at 00000002319365EE
Found 8224.bk2 in Data0.bdt at 000000023274162E
Found 8225.bk2 in Data0.bdt at 0000000232DD14EE
Found 8226.bk2 in Data0.bdt at 0000000233701B3E
Found 8227.bk2 in Data0.bdt at 0000000234D950B2
Found 8228.bk2 in Data0.bdt at 0000000235318102
Found 8229.bk2 in Data0.bdt at 000000023676F89E
Found 8230.bk2 in Data0.bdt at 000000023798C43E
Found 8231.bk2 in Data0.bdt at 000000023929E82E
Found 8232.bk2 in Data0.bdt at 00000002398CA326
Found 8233.bk2 in Data0.bdt at 000000023ABD2182
Found 8234.bk2 in Data0.bdt at 000000023B9C69A2
Found 8235.bk2 in Data0.bdt at 000000023C5533D2
Found 8236.bk2 in Data0.bdt at 000000023DDDEB06
Found 8237.bk2 in Data0.bdt at 000000023E8A16B2
Found 8238.bk2 in Data0.bdt at 000000023F6047F6
Found 8300.bk2 in Data0.bdt at 000000023FE6AEB2
Found 8301.bk2 in Data0.bdt at 0000000240302C76
Found 8302.bk2 in Data0.bdt at 00000002407E9EEE
Found 8303.bk2 in Data0.bdt at 0000000240CA399E
Found 8304.bk2 in Data0.bdt at 00000002412F06CE
Found 8305.bk2 in Data0.bdt at 000000024191BD0E
Found 8306.bk2 in Data0.bdt at 0000000241F948EA
Found 8400.bk2 in Data0.bdt at 0000000242EACD76
Found 8401.bk2 in Data0.bdt at 000000024378396A
Found 8402.bk2 in Data0.bdt at 00000002447F695E
Found 8403.bk2 in Data0.bdt at 0000000246C9528A
Found 8500.bk2 in Data0.bdt at 000000024738A426
Found 8501.bk2 in Data0.bdt at 000000024864F49E
Found 8502.bk2 in Data0.bdt at 0000000249956F12
Found 8503.bk2 in Data0.bdt at 000000024ABA081A
Found 8504.bk2 in Data0.bdt at 000000024C2767E2
Found 8505.bk2 in Data0.bdt at 000000024D8D1966
Found 8506.bk2 in Data0.bdt at 000000024F2ACE7E
Found 8507.bk2 in Data0.bdt at 00000002511E9BE6
Found 16100.bk2 in Data0.bdt at 0000000252B283DA
Found menu_monitor_noise.bk2 in Data0.bdt at 0000000252C9F396
Found s00_00_0000_cut0000_01.bk2 in Data0.bdt at 0000000253151F72
Found s00_20_0000_cut0010_00.bk2 in Data0.bdt at 00000002531A6FBE
Found s00_20_0011_cut0010_00.bk2 in Data0.bdt at 00000002537BC366
Found s10_00_0020_cut0010_00.bk2 in Data0.bdt at 0000000253E8FB4A
Found s10_10_0000_cut0070_00.bk2 in Data0.bdt at 000000027752367A
Found s20_00_0000_cut0010_00.bk2 in Data0.bdt at 00000002A9530AC6
Found s20_20_0070_cut0010_00.bk2 in Data0.bdt at 00000002CBB29AFE
Found s30_00_0010_cut0100_00.bk2 in Data0.bdt at 00000002D2E1D032
Found s30_00_0030_cut0110_00.bk2 in Data0.bdt at 00000002DA4C6D32
Found s50_40_0010_cut0035_00.bk2 in Data0.bdt at 00000002F48F60C2

In total they are 276 files inside Data0.bdt, but it's missing "dummyMovie.bk2", so there has to be a file which is not in the movie folder.

However you can cut the files out of Data0.bdt and reimport them back later with xtool again if you want. The game starts normally with cutted files and is playing intro movies (it reads the movies from the movie folder).

For cut files
xtool erase "movies\*.bk2" "Data0.bdt" ac6for.xtl

For reimport files
xtool decode ac6for.xtl "movies" "Data0.bdt"

Edit:
CRC matching after erase/decode
original Data0.bdt = 70E16A53
erase/decode Data0.bdt = 70E16A53

So you can easily use xtool bink2 plugin for the movies folder and cut the files out of Data0.bdt without getting any errors and/or unwanted duplicates for the movies.
Hi, sorry if i join to discuss. How to do it in DSG? thank you;)

L33THAK0R

27-08-2023, 03:39

When it comes to testing oodle libraries is it just trial and error like with testing LZ4 libraries? Haven't tackled a ton of Oodle titles and currently taking a crack at "Wolfenstein Cyberpilot". Testing with the included "oo2core_6_win64.dll" reveals with verbose mode in xtool that only level 4 streams are successfully processed, however the sample I'm using for testing only ~300/900 streams are processed with this resulting in only a 100.10% output. I should know how to deal with Oodle by now, but in the past I've been lazy and just read through the posts here to figure out which library to use.

KaktoR

27-08-2023, 04:52

@L33THAK0R
With XToolUI you can use library checker.

KaktoR

27-08-2023, 06:02

Armored Core VI Fires of Rubicon

11:30:01 - Selected ARC/DS method for Data1a-01.bin was: xtool:mkraken,l9:core_2.8.14+xtool:dd3+4x4:lzma
11:30:01 - Selected ARC/DS method for Data1b-01.bin was: xtool:c1024mb:mbk2:df20p+xtool:dd3
11:30:01 - Selected ARC/DS method for Data1c-01.bin was: xtool:dd3+4x4:lzma
-------------------------------------------------------------------------------------------------------------------------------------------
xtool erase
11:30:01 - Executing external file (Call Before 1a)...
11:34:22 - Starting Data1a-01.bin archive creation.

xtool decode
14:54:37 - Executing external file (Call After 1c)...
14:59:51 - Compression finished in 27.08.2023 at 14:59:51.
-------------------------------------------------------------------------------------------------------------------------------------------
14:59:52 - Overall input size: 59.69 GB
14:59:52 - Overall output size: 40.99 GB (Ratio 68.68%)
14:59:52 - Overall conversion time: 03:29:42

All files are matching with the original files after installation (use xtool decode + the xtl file).

PS: I don't know what's up with that sd.bdt file. Maybe someone can check it as it is not compressed with kraken.

Masquerade

27-08-2023, 06:10

I think SD.bdt is sound data? Not sure :p

Armored Core VI Fires of Rubicon

1. Xtool erase Movies folder from Data0.BDT

2. Compress:

*.bdt
xtool_kraken_dd5+lolz
*.bk2
xtool_bk2_dd5_fl2
Everything else:
razor

3: Xtool decode database you made in Step 1 to restore Data0.bdt.

59.7GB ---> 38.2GB

shazzla

27-08-2023, 09:40

Guys!

Those duplicated bk2 files in data0.bdt are required to run the game ?
(Im away from my compo)

KaktoR

27-08-2023, 09:48

Not entirely sure. You can start the game normally, and the first videos + intro will show (if you cut bk2 videos from Data0.bdt). But I cannot speak for the whole game.

If you move the movie folder or rename it, the game will not even start.

shazzla

27-08-2023, 10:09

Sorry for offtopic,but kaktor ,can u please check your msgs? A few days ago i sent u a pm.... thanks!:)

L33THAK0R

27-08-2023, 23:30

Uses Kraken, after testing with KaktoR's Oodle library collection, v2.9.0 seems to yield the best results, however even with this library a considerable number of streams are dropped. Against the attached sample 88/231 steams are processed (full recompression currently, as reported by XTool, indicates 6248/7387 streams are processed), verbose output indicates streams are at a depth of L4.

xtool 0.7.8 (-mkraken:l4:n128:t256+zlib+reflate) + srep + lolz

It should be noted that it seems at least with the current library re-compression isn't very effective, with the reinflated output only being ~10MB larger than the initial input. The original Oodle DLL matches the CRC-32 checksum for library v2.6.0, when compared against KaktoR's kindly supplied checksums for all current library.

16,606,744,418 => 14,348,641,273 bytes. Ratio 86.40%

Masquerade

28-08-2023, 00:16

What does the Oodle srream scanner say?

KaktoR

28-08-2023, 07:20

Hi, sorry if i join to discuss. How to do it in DSG? thank you;)

Database preset is now up for download (check in DSG for update).

However in CIU something has to be added in Setup.ini

[CMDCommands]
if exist "{app}\ac6for.xtl" (
"{tmp}\XTool_2020\XTool.exe" decode "{app}\ac6for.xtl" "{app}\Game\movie" "{app}\Game\Data0.bdt"
del /F /S /Q "{app}\ac6for.xtl"
TIMEOUT 3
)

L33THAK0R

28-08-2023, 08:05

What does the Oodle srream scanner say?

Here is the output for the stream scanner (https://pastebin.com/vXRB7HCS), as well as the output for the library checker (https://pastebin.com/kw8vMrf2).

Masquerade

28-08-2023, 08:29

No, I meant the oo2scan_7_win64.exe by Razor12911, as it lists these algorithms:

Universal Oodle stream scanner

Created by Razor12911

[0] = Unknown/Invalid
[1] = Kraken/Hydra
[2] = Mermaid/Selkie/Hydra
[3] = Leviathan/Hydra

If you scan and you see [1] appearing, it could be that you are seeing Hydra streams, not kraken.

Try extracting a single stream from the sample you are testing on using XTool verbose output and a Hex editor and attempt to precompress it with oo2reck. If it *is* a Kraken stream, it will be inflated. Otherwise, try Hydra.

Jarmen_Kell

29-08-2023, 03:31

Hey guys. Is there any suggested method for ARMA III?

L33THAK0R

29-08-2023, 05:38

No, I meant the oo2scan_7_win64.exe by Razor12911, as it lists these algorithms:

Universal Oodle stream scanner

Created by Razor12911

[0] = Unknown/Invalid
[1] = Kraken/Hydra
[2] = Mermaid/Selkie/Hydra
[3] = Leviathan/Hydra

If you scan and you see [1] appearing, it could be that you are seeing Hydra streams, not kraken.

Try extracting a single stream from the sample you are testing on using XTool verbose output and a Hex editor and attempt to pre-compress it with oo2reck. If it *is* a Kraken stream, it will be inflated. Otherwise, try Hydra.

I'm seeing results for both [1] and [3], however using XTool's GUI and testing each library against the sample, now with both the Kraken and Hydra codecs, still results in the same number of streams being successfully pre-compressed, when compared to only using Kraken. I've got a feeling it may in this case be Leviathan & Kraken, and I'm aware of how annoying Leviathan is to process since theres a notable manual element to pre-compressing this Oodle variant. I might investigate this title further, at some point in the future, once I'm more confident with working with undocumented Oodle titles.

KaktoR

29-08-2023, 05:57

If I may quote Razor12911 on this:
He should give up
2.6.0 leviathan
2.9.0 kraken

L33THAK0R

29-08-2023, 08:03

If I may quote Razor12911 on this:
He should give up
2.6.0 leviathan
2.9.0 kraken

I'm nothing if not stubborn, well at least with that information I've got something to go off of, I'll see what I can do.

shazzla

29-08-2023, 09:33

Hey guys. Is there any suggested method for ARMA III?

AFAIK half of this game is RSA encrypted ,so it cannot be recompressed (EBO)

PBOs maybe some kind of LZO?!

Jarmen_Kell

30-08-2023, 09:23

AFAIK half of this game is RSA encrypted ,so it cannot be recompressed (EBO)

PBOs maybe some kind of LZO?!

Then how Fitgirl compressed it from 146GB to 73GB?

Masquerade

30-08-2023, 09:37

Then how Fitgirl compressed it from 146GB to 73GB?

Because she decrypted it... duh.

shazzla

30-08-2023, 09:57

IMHO its impossible ! RSA is asymmetric. So u can obtain the decryption key from the game ,but reencrypt...?! Only the devs knows the encryption key (Maybe its buried accidentally somewhere in a gamefile )...
Or am i wrong ?

btw Masquerade's answer seems good....

Edit:

Maybe start looking here :

https://www.unknowncheats.me/forum/arma-3-a/

KaktoR

31-08-2023, 11:12

Trine 5: A Clockwork Conspiracy

19:18:46 - Selected ARC/DS method for Data1a-01.bin was: xtool:mzstd:zstd_137+xtool:dd3+4x4:lzma
19:18:46 - Selected ARC/DS method for Data1b-01.bin was: xtool:dd3+4x4:lzma
-------------------------------------------------------------------------------------------------------------------------------------------
20:12:09 - Overall input size: 16.92 GB
20:12:09 - Overall output size: 12.42 GB (Ratio 73.38%)
20:12:09 - Overall conversion time: 00:53:21

Masquerade

31-08-2023, 12:09

You can rip voiceovers from audio1.fbq in Trine 5 using this QuickBMS script:

get VERSION long
if VERSION <= 3
comtype zstd
else
comtype lz4
endif
get FILES long
get BASE_OFF long
math BASE_OFF += 0xc
for i = 0 < FILES
get NAME string
get OFFSET long
get NOZIP byte
get SIZE long
get ZSIZE long
get CRC long
math OFFSET += BASE_OFF
if NOZIP == 4
clog NAME OFFSET ZSIZE SIZE
else
log NAME OFFSET SIZE
endif
next i

Made by eaZy41 from XeNTaX community.

Although comtype LZ4 is mentioned, Trine 5 uses only ZSTD and you can use libzstd v1.3.7 (v1.3.6 also seems to work). LZ4 is used on older versions of the FBQ archive structure.

KaktoR

31-08-2023, 12:23

Better use zstd version "1.3.7", as it is stored in trine5.exe.
https://i.imgur.com/q4OTfAM.png

Although it doesn't make any sense, I looked through some of the streams in the texture archives.

#1 174760 > 174752 = 8 bytes
#2 349520 > 349504 = 16 bytes

#3 11184808 > 11184800 = 8 bytes
#4 5592400 > 5592384 = 16 bytes

#5 699048 > 699040 = 8 bytes
#6 1398096 > 1398080 = 16 bytes

#7 43688 > 43680 = 8 bytes
#8 22369616 > 22369600 = 16 bytes
Actually this is the repeating size pattern.

Signature(1),DSize(4),Unknown(8),ZstdHdr(4),Stream
This Unknown 8 bytes are actually the byte size difference for DSize. 8 and 16.

Masquerade

31-08-2023, 15:17

Did your archive decompress alright? I'm getting this error from XTool when trying to decompress my archive:
I was using libzstd v1.3.7.
Unpacking 25,439,279,314 bytes with XTOOL decode -t100p $$arcpackedfile$$.tmp $$arcdatafile$$.tmp
0.0%XTool is created by Razor12911

Time: 00:00:15 (CPU 00:02:30)

0.0%EThreadException: Error in the method 'zstd'

Errorlevel=1
0.0%
ERROR: invalid compression method or parameters in XTZSTD

Wanterlude

31-08-2023, 15:22

Did your archive decompress alright? I'm getting this error from XTool when trying to decompress my archive:
I was using libzstd v1.3.7.
Unpacking 25,439,279,314 bytes with XTOOL decode -t100p $$arcpackedfile$$.tmp $$arcdatafile$$.tmp
0.0%XTool is created by Razor12911

Time: 00:00:15 (CPU 00:02:30)

0.0%EThreadException: Error in the method 'zstd'

Errorlevel=1
0.0%
ERROR: invalid compression method or parameters in XTZSTD
same, xtool just crash on unpack with 1.3.7 zstd lib
"ERROR: write error (disk full?) in compression algorithm zstd" in arc

edit.
1.3.5,1.3.6,1.3.7 - error (crash)
test file: gui1.fbq
Trine 5 [3 fbq files] - https://pixeldrain.com/u/cTSsXMes

Razor12911

31-08-2023, 18:39

same, xtool just crash on unpack with 1.3.7 zstd lib
"ERROR: write error (disk full?) in compression algorithm zstd" in arc

edit.
1.3.5,1.3.6,1.3.7 - error (crash)
test file: gui1.fbq
Trine 5 [3 fbq files] - https://pixeldrain.com/u/cTSsXMes

I won't test as I'm currently busy with something else but maybe try this development version of xtool as a ton of issues were fixed however no update as of yet because it's currently in development so if it works, use it for only this case and revert back to 0.7.8 for everything else. (don't use -dd#, it is bugged in dev build)

I'm nothing if not stubborn, well at least with that information I've got something to go off of, I'll see what I can do.

[Stream1]
// 2.6.0
Name=leviathan
Codec=leviathan:l4
BigEndian=0
Signature=0x0
Structure=Signature(16),DSize(8),CSize(8),OodleHdr (2),Stream
StreamOffset=-2
CompressedSize=CSize
DecompressedSize=DSize
Condition1=OodleHdr = 0x0C8C
Condition2=CSize>=256
Condition3=DSize>CSize

I made quickly made this the other day, not sure if it works very well.

KaktoR

01-09-2023, 01:17

Did your archive decompress alright? I'm getting this error from XTool when trying to decompress my archive:
I was using libzstd v1.3.7.
Unpacking 25,439,279,314 bytes with XTOOL decode -t100p $$arcpackedfile$$.tmp $$arcdatafile$$.tmp
0.0%XTool is created by Razor12911

Time: 00:00:15 (CPU 00:02:30)

0.0%EThreadException: Error in the method 'zstd'

Errorlevel=1
0.0%
ERROR: invalid compression method or parameters in XTZSTD

Yes it works fine here using libzstd v1.3.7, but tbh I am using the development version of xtool since days, there were some fixes.

Extracting could take a while because of zstd l22 btw.

KaktoR

01-09-2023, 06:44

Sid Meier's Civilization VI
AUGUST 2023 UPDATE

14:01:31 - Selected ARC/DS method for Data1a-01.bin was: xtool:c256mb:mbk2:df20p
14:01:31 - Selected ARC/DS method for Data1b-01.bin was: xtool:dd3+lolz:d64:mtt1:mt10:fba4096:mc2
14:01:31 - Selected ARC/SPLIT method for Data1c-01.bin was: xtool:dd3+lolz:d64:mtt1:mt10:fba4096:mc2/$deflatepack=xtool:c128mb:mpreflate:mpng:d1+xtool: dd3+lolz:d64:mtt1:mt10:fba4096:mc2/$oggrepack=oggre
-------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------------------------------------------------
15:42:58 - Overall input size: 25.96 GB
15:42:58 - Overall output size: 8.69 GB (Ratio 33.49%)
15:42:58 - Overall conversion time: 01:41:25

Data1a = *.bk2
Data1b = *.blp
Data1c = rest of the files

PS: I used wemtool by Masquerade for the wem's

KaktoR

01-09-2023, 09:11

Maybe someone know what best compress method for starfield? Thank you:)

Bruh
https://fileforums.com/showpost.php?p=502241&postcount=379

KaktoR

01-09-2023, 12:46

Baldurs Gate 3
v4.1.1.3686210

17:57:24 - Selected ARC/DS method for Data1a-01.bin was: xtool:mbg3+xtool:mzlib+xtool:dd3+4x4:lzma
17:57:24 - Selected ARC/DS method for Data1b-01.bin was: xtool:c1024mb:mbk2:df20p
17:57:24 - Selected ARC/DS method for Data1c-01.bin was: xtool:mbg3+xtool:dd3+zstd
17:57:24 - Selected ARC/SPLIT method for Data1d-01.bin was: xtool:dd3+4x4:lzma/$deflatepack=xtool:c128mb:mpreflate:mpng:d1+xtool: dd3+4x4:lzma/$mp3pack=7zdll-pm:m0=mp3det:m1=packmp3c
-------------------------------------------------------------------------------------------------------------------------------------------
21:42:21 - Overall input size: 137.01 GB
21:42:21 - Overall output size: 88.23 GB (Ratio 64.39%)
21:42:21 - Overall conversion time: 03:40:21

IRAQIGHOST

01-09-2023, 21:19

Starfield

DiskSpan
xt_zlib+xtool:s:mzlib+srep_new:3+LOLZ_ULTRA

Test File (Starfield - Animations.ba2)
Size: 319 MB >> 711 MB
Final Size: 170.56 MB

Danziel123

02-09-2023, 01:47

Bruh
https://fileforums.com/showpost.php?p=502241&postcount=379
Here is my DSG_Settings for starfiled, is this right or not?

Data1.Name=Data1a-01.bin
Data1.InclText=*.ba2
Data1.ExclText=
Data1.Method1=xtool:mstarfield+xtool:mreflate:dd3+ LZMA-MT
Data2.Name=Data1b-01.bin
Data2.Method1=xtool:c256mb:mbk2:df20p+LZMA-MT
Data2.InclText=*.bk2
Data3.Name=Data1c-01.bin
Data3.Method1=xtool:dd3+LZM A-MT
Data3.ExclText=*.ba2|*.bk2

And what about for .bk2 files? is enough using chunk 256mb to compress it? Or i can increase the chunk size? Thank you:)

KaktoR

02-09-2023, 04:37

Should be enough

KaktoR

02-09-2023, 04:42

Starfield

DiskSpan
xt_zlib+xtool:s:mzlib+srep_new:3+LOLZ_ULTRA

Test File (Starfield - Animations.ba2)
Size: 319 MB >> 711 MB
Final Size: 170.56 MB

Why use old xtool with new xtool? And why skip parameter?

IRAQIGHOST

02-09-2023, 05:24

Why use old xtool with new xtool? And why skip parameter?

for try to get more size

Fak Eid

03-09-2023, 04:45

You tried compressing on just a single file. When you compress all the files it will be stuck forever. Do not try this as this is wrong.

Starfield

DiskSpan
xt_zlib+xtool:s:mzlib+srep_new:3+LOLZ_ULTRA

Test File (Starfield - Animations.ba2)
Size: 319 MB >> 711 MB
Final Size: 170.56 MB

IRAQIGHOST

03-09-2023, 05:30

You tried compressing on just a single file. When you compress all the files it will be stuck forever. Do not try this as this is wrong.

Yes, I notice that in some files.

KaktoR

03-09-2023, 06:43

for try to get more size

Wtf. Why not use the correct codecs with appropriate chunk size instead?

And do not use the skip option if you don't know what you do.

Danziel123

03-09-2023, 21:40

Should be enough
Why i get stuck at this?
http://i.ibb.co/sVbn5hm/Starfiled.png
Alredy tried with xtool:mreflate, xtool:mpreflate & xtool:mzlib still same. Maybe you know what cause it? Thank you:)

Fak Eid

04-09-2023, 01:35

Because the codecs can't read the stream of the file. The files contains encoded lz4 that can be compressed only using a certain plugin 'mstarfield' (starfield.dll). Leave out the wwise sound files and compress using that algo.. wwise sound can only be compressed using srep+lzma (as of my knowledge right now)

Why i get stuck at this?
http://i.ibb.co/sVbn5hm/Starfiled.png
Alredy tried with xtool:mreflate, xtool:mpreflate & xtool:mzlib still same. Maybe you know what cause it? Thank you:)

Danziel123

04-09-2023, 01:41

Because the codecs can't read the stream of the file. The files contains encoded lz4 that can be compressed only using a certain plugin 'mstarfield' (starfield.dll). Leave out the wwise sound files and compress using that algo.. wwise sound can only be compressed using srep+lzma (as of my knowledge right now)
Here is my DSG_Settings for starfiled, is this right or not?

Data1.Name=Data1a-01.bin
Data1.InclText=*.ba2
Data1.ExclText=
Data1.Method1=xtool:mstarfield+xtool:mreflate:dd3+ LZMA-MT
Data2.Name=Data1b-01.bin
Data2.Method1=xtool:c256mb:mbk2:df20p+LZMA-MT
Data2.InclText=*.bk2
Data3.Name=Data1c-01.bin
Data3.Method1=xtool:dd3
Data3.ExclText=*.ba2|*.bk2

Masquerade

04-09-2023, 02:13

Quite a lot of audio in Starfield is WWISE PCM - meaning you can use MSC on it.

Other type of WWISE audio is Vorbis, which you can read more about in WemTool topic.

KaktoR

04-09-2023, 02:32

^Yes, all of the *WwiseSounds* are PCM. The Language files are Vorbis it seems.

Streams: 18613 / 18613
Time: 00:03:05 (CPU 00:16:04)
Duplicates: 109 (100 MB) [113 MB >> 35.3 MB]

Size: 19.3 GB >> 6.94 GB >> 6.90 GB

Additional decoding memory is 79.7 MB
Decoding speed improved by up to 1.01x

Done!!!

I used flac plugin with l8 option here. I guess msc is better in terms of ratio but alot slower.

PS: And compression for the other files, files which have "02 00 00 00" in the beginning are compressed with zlib, "03 00 00 00" are compressed with lz4.

KaktoR

04-09-2023, 06:59

As for the WEM files in Starfield, seems like it does not make sense to process them.

xtool:dd3:l4
Compressed 125,234 files, 1,993,236,804 => 1,878,585,133 bytes. Ratio 94.25%
Compression time: cpu 6.13 sec/real 44.43 sec = 14%. Speed 44.87 mB/s
All OK

Compressed 1 file, 1,870,743,269 => 1,839,107,129 bytes. Ratio 98.31%
Compression time: cpu 0.52 sec/real 63.26 sec = 1%. Speed 29.57 mB/s
All OK

Fak Eid

04-09-2023, 07:48

xtool:mstarfield+xtool:mreflate+srep_new+lolz/lzma seems correct for all .ba2 files except WWise Sound files. Also as of now, DSG won't be able to extract the files. You need to do it with your installer only.

Here is my DSG_Settings for starfiled, is this right or not?

Danziel123

04-09-2023, 23:57

xtool:mstarfield+xtool:mreflate+srep_new+lolz/lzma seems correct for all .ba2 files except WWise Sound files. Also as of now, DSG won't be able to extract the files. You need to do it with your installer only.
Thank you for suggestion, change it to xtool:dd now can compress till finish:)

KaktoR

05-09-2023, 10:31

BioShock Infinite
v31.08.2022

16:54:07 - Selected ARC/DS method for Data1a-01.bin was: xtool:c320mb:mbik
16:54:07 - Selected ARC/DS method for Data1b-01.bin was: uelr+xtool:c128mb:mzlib+xtool:dd3+4x4:lzma
16:54:07 - Selected ARC/DS method for Data1c-01.bin was: xtool:dd3+4x4:lzma
------------------------------------------------------------------------------------------
19:08:31 - Overall input size: 43.20 GB
19:08:31 - Overall output size: 15.32 GB (Ratio 35.47%)
19:08:31 - Overall conversion time: 02:14:23

Fak Eid

05-09-2023, 10:55

Why did everyone started using xtool:dd3 instead of srep_new:3. It it better?

BioShock Infinite
v31.08.2022

16:54:07 - Selected ARC/DS method for Data1a-01.bin was: xtool:c320mb:mbik
16:54:07 - Selected ARC/DS method for Data1b-01.bin was: uelr+xtool:c128mb:mzlib+xtool:dd3+4x4:lzma
16:54:07 - Selected ARC/DS method for Data1c-01.bin was: xtool:dd3+4x4:lzma
------------------------------------------------------------------------------------------
19:08:31 - Overall input size: 43.20 GB
19:08:31 - Overall output size: 15.32 GB (Ratio 35.47%)
19:08:31 - Overall conversion time: 02:14:23

KaktoR

05-09-2023, 11:11

Because it's faster.

Edit: "3" in dd# just represents the srep m#f setting m[1..5]. You can even use dd5 to get a bit better results in costs of speed.

Edit2: A bit old, but my test in the past when this feature was introduced: https://fileforums.com/showthread.php?p=499129#post499129

Masquerade

05-09-2023, 15:35

@KaktoR did you try using xtool unreal 3 for the lzo? I'm interesred in the performance vs uelr.

KaktoR

05-09-2023, 23:08

ANNO 1800
Update 18.0

20:37:16 - Selected ARC/DS method for Data1a-01.bin was: xtool:c1500mb:lm:mbk2:df20p+xtool:dd3
20:37:16 - Selected ARC/DS method for Data1b-01.bin was: xtool:mflac,l8+xtool:mzlib:mreflate+xtool:dd3+4x4: lzma
20:37:16 - Selected ARC/SPLIT method for Data1c-01.bin was: xtool:dd3+4x4:lzma/$mp3pack=7zdll-pm:m0=mp3det:m1=packmp3c/$bpkpack=bpk
-----------------------------------------------------------------------------------------
00:12:23 - Overall input size: 101.44 GB
00:12:23 - Overall output size: 30.63 GB (Ratio 30.20%)
00:12:23 - Overall conversion time: 03:34:59

Notes:
- bpk was used on bink1 files which are located in data3.rda
- You have to use -lm for bk2 plugin if you don't have 64GB RAM
- The following files contain bink videos: data3.rda, data11.rda, data12.rda, data13.rda, data16.rda and data18.rda. Use quickbms with "-f "{}.bk2,{}.bik"" file mask to extracrt just bink1/2 files.
- Use XTool erase to create xtl file and zero the files out of the game archives.
- Use XTool decode + xtl file to patch the files back afterwards.
- Some rda files also contain WEM files. You can use wemtool there.
- The language files also contain WEM files, but they are PCM, so you msc/xtool:flac on them.

Fak Eid

06-09-2023, 01:36

Thanks for the update. If dd3 can help me compressing the same size as srep and with less extraction time, I'll go ahead with it. My focus is to improve extraction speed at a cost of minimum difference in compression size. I'll try that for sure.

Because it's faster.

Edit: "3" in dd# just represents the srep m#f setting m[1..5]. You can even use dd5 to get a bit better results in costs of speed.

Edit2: A bit old, but my test in the past when this feature was introduced: https://fileforums.com/showthread.php?p=499129#post499129

L33THAK0R

06-09-2023, 10:38

I know Masquerade made a database recently for the first "Watch Dogs" title, however I've found myself needing to re-generate a new database in my own conversion, as I've opted to include a variety of mods that restore features left on the cutting room floor as well as providing the end-user access to content previously only accessible through online-only interactions. This restoration overwrites a considerable amount of existing files, as such a new database is required.

My current database seems to be considerably smaller than Masquerade's however, with a final result of 3.84 MiB (versus Masquerade's 14.0 MiB). The only issue I can currently see is when the fat2_fat3.bms script attempts to process the "windy_city.fat/.dat" which gives the following error:

Error: the compressed zlib/deflate input is wrong or incomplete (-3)
Info: algorithm 2
offset 494c62b4
input size 0x0000632b 25387
output size 0x00010000 65536
result 0xffffffff -1

Error: uncompressed data (-1) bigger than allocated buffer (224656)
It usually means that data is not compressed or uses another algorithm

I'm assuming this is referring to

elif ZTYPE == 2
comtype deflate

however my bms2xtl.ini configuration file seems to be fine:

[Config]
SafeMode=0
UserInput=
Exclude=lzma_dynamic,snappy
gzip=zlib
deflate=reflate
zlib_noerror=zlib
deflate_noerror=reflate
zlib=zlib
oodle=leviathan
xmemdecompress=xmemcompress
lz4=lz4hc:l9
zenpxp=zenpxp
lzo1x=lzo1x
lzma_dynamic=reflate

I've ensured I've got the latest version of the xmemcompress plugin present, and its appropriate entry within "xtool.ini" is pretty standard

[xmemcompress]
Encode=xcompress64.exe d -window=131072 -partition=524288 <library>
Decode=xcompress64.exe e -window=131072 -partition=524288 <library>

I'm honestly scratching my head trying to figure out what I've messed up this time, even more so by the fact that the file "windy_city.fat/.dat" is one such file that hasn't been modified. Any tips would be greatly appreciated, its definitely been gnawing away at me.

Razor12911

06-09-2023, 10:47

the window and partition for xmemcompress should be 32768 I believe, those configurations are for project cars

[xmemcompress]
Encode=xcompress.exe d -window=32768 -partition=32768 <library>
Decode=xcompress.exe e -window=32768 -partition=32768 <library>

as for the other errors, I'm not so sure

Masquerade

08-09-2023, 01:04

DOOM Eternal v6.66 Rev 2.2

*.streamdb, *.resources, *.lightdb
xtool_kraken:l6_doom7*_dd+xtool_kraken:l6_doom8*_d d+srep+lolz
*.bk2**
xtool_bink2_dd5_fl2
*.bik
xtool_bink_dd5_fl2
Everything else:
razor
* Using these two (https://www.fileforums.com/showpost.php?p=499054&postcount=3267) oodle libraries.

** All bk2 except vs_first.bk2, vs_second.bk2 and doommarinesymbol_wide_fade.bk2.

88.7GB ---> 56GB

Masquerade

08-09-2023, 05:52

The Sims 4 [v1.100.147.1030 + All DLCs]

xtool_zlib_dd5+LZMA2

58.7GB ---> 36.4GB

KaktoR

08-09-2023, 11:42

** All bk2 except vs_first.bk2, vs_second.bk2 and doommarinesymbol_wide_fade.bk2.

Why excluding? Does the modified bink plugin go crazy again? :D

Masquerade

08-09-2023, 12:43

Why excluding? Does the modified bink plugin go crazy again? :D

The modified import does indeed fix motion data warn but it does not fix err. Trying to compress the folder with those 3 videos present will just leave XTool hanging.

KaktoR

08-09-2023, 23:31

Could you send me one of these files? Thanks.

Edit: I looked, and I don't have any problems with the patched bink plugin.

Compressed 3 files, 69,070,112 => 69,070,262 bytes. Ratio 100.00%
Compression time: cpu 0.06 sec/real 1.12 sec = 6%. Speed 61.71 mB/s
All OK

Extracted 3 files, 69,070,262 => 69,070,112 bytes. Ratio 100.00%
Extraction time: cpu 0.05 sec/real 0.60 sec = 8%. Speed 114.63 mB/s
All OK

KaktoR

09-09-2023, 15:44

XCOM 2
version 28 March 2022

20:51:40 - Selected ARC/DS method for Data1a-01.bin was: uelr+xtool:mkraken:core_2.4.1+xtool:mzlib:dd3+4x4: lzma
20:51:40 - Selected ARC/DS method for Data1b-01.bin was: xtool:lm:c1024mb:mbk2:df20p+xtool:dd3
20:51:40 - Selected ARC/SPLIT method for Data1c-01.bin was: xtool:dd3+4x4:lzma/$oggre=oggre/$wemtoolpack=xtool:dd3+zstd
-------------------------------------------------------------------------------------------------------------------------------------------
00:40:08 - Overall input size: 70.96 GB
00:40:08 - Overall output size: 25.26 GB (Ratio 35.59%)
00:40:08 - Overall conversion time: 03:48:26

WEMTool was used for *.wem files.

L33THAK0R

10-09-2023, 18:52

Has anyone else experienced some issues with Oggre when processing a large amount of streams? I'm currently working on "Nioh 2", whose audio can be extracted into a readable Vorbis Ogg format. I individually packed every file (~9000) and compared the original file checksums to determine which files were incompatible and removed them accordingly. In the past this process has never failed me and has always without fail resulted in a successful decompression of those particular assets, however this time around it seems I'm left with a bit of a confusing result. Attempting to pack all the streams together results in the installer, presumably due to the CLS Oggre plugin, crashing.

Investigating the issue by creating a large stored archive of all files in their packable format shows a much clearer image:

Original Input, Stored Archive (7.79 GB) -> Processed with Oggre (5.29 GB) -> Unpacked with Oggre (4.67 GB)

My current solution I'll be employing for my conversion is simply post-install, unpacking the stored archive with the "temp" packable files, then using a patch made between the temp files and a stored archive of the original files.

I'll be then putting these 2 files alongside each other into one large archive, whilst praying I can squeeze some small gains but I'm under no illusion it'll almost certainly be a satisfactory result. Definitely not happy about the amount of additional R/W operations that'll be performed on the end-users drive, but its the best solution I can come up with.

I should mention I was able to process the audio from the original "Nioh" without any trouble, which follows the same structure as "Nioh 2", with a pretty decent final result of 20.3 GB for v1.21.04 of the Complete Edition release for "Nioh".

UPDATE: I ran an additional test with the original unmodified audio archives packed with a generic srep + lolz configuration, when alls said and done, srep actually works well with the duplicated data, resulting in an inflated final result of only 390MB. Given the negligible gains compared to the amount of data that would would need to be written (~16gb) for the original patching approach to work, I think for the time being, and in the interests of the end-user, I'll sacrifice this 390MB.

Masquerade

11-09-2023, 00:17

Yeah, oggre is a mess.

L33THAK0R

12-09-2023, 23:21

Can anyone recommend which library and/or version of XTool to use for processing "Mirror's Edge: Catalyst"'s LZ4 streams? I recently tried running my collection of LZ4 libraries against a 100MB sample, using XTool 0.7.8 however seemingly none of them were of any use (see the following pastebin output) (https://pastebin.com/wsBLzAVx). I've had success in the past with another LZ4 Frostbite 3 title, "Battlefield: Hardline", however attempting to use the same library for Catalyst results in 0 streams being detected/processed.

I can only find one user actually succeeding to pre-compress Catalyst, who indicates they used "xtlz4(v12)" (https://fileforums.com/showpost.php?p=500174&postcount=3379) but I've no idea if this refers to the library or tool.

Any advice would be greatly appreciated, I'm properly stumped with this one.

kj911

13-09-2023, 02:52

Try its probably used old Xtool v0.12 with old lz4 libraries. (or used mixed/replaced in game related lz4 library dll.) Check it...

KaktoR

13-09-2023, 07:04

If I remeber correct, 1.8.0+ should do it.

Edit:
Streams: 1790 / 1815
Time: 00:00:01 (CPU 00:00:00)

Size: 22.8 MB >> 60.2 MB

lz4 library 1.8.3. Try with others aswell

Masquerade

14-09-2023, 00:15

Mary Skelter Finale

xtool_zlib_dd5+lolz/$mp4=srep/$bra=oggre

12.8GB ---> 8.7GB

Masquerade

14-09-2023, 00:16

Golf With Your Friends [Build 12062185]

xtool_unity_dd5+lolz

5.1GB ---> 2.5GB

Masquerade

14-09-2023, 09:08

Wild question, but would anyone happen to have a backup of the tool linked here?
https://forum.xentax.com/viewtopic.php?t=11106

The blog link is offline. The page is on the Wayback Machine, but the actual file download is blocked behind a xentax login so the file isn't on the wayback machine annoyingly.

Razor12911

14-09-2023, 14:09

Can anyone recommend which library and/or version of XTool to use for processing "Mirror's Edge: Catalyst"'s LZ4 streams? I recently tried running my collection of LZ4 libraries against a 100MB sample, using XTool 0.7.8 however seemingly none of them were of any use (see the following pastebin output) (https://pastebin.com/wsBLzAVx). I've had success in the past with another LZ4 Frostbite 3 title, "Battlefield: Hardline", however attempting to use the same library for Catalyst results in 0 streams being detected/processed.

I can only find one user actually succeeding to pre-compress Catalyst, who indicates they used "xtlz4(v12)" (https://fileforums.com/showpost.php?p=500174&postcount=3379) but I've no idea if this refers to the library or tool.

Any advice would be greatly appreciated, I'm properly stumped with this one.

https://fileforums.com/showpost.php?p=488282&postcount=2
Use the frostbite 3 plugin for this game -mfrostbite3:lz4

L33THAK0R

14-09-2023, 17:27

https://fileforums.com/showpost.php?p=488282&postcount=2
Use the frostbite 3 plugin for this game -mfrostbite3:lz4

I should have specified I was having difficulty with finding streams using the frostbite 3 plugin, turns out though I was just testing against a sample that didnt have any streams :o

Once I tried against a larger sample the plugin worked perfectly, many thanks for your hard-work on it!

kj911

15-09-2023, 04:49

Wild question, but would anyone happen to have a backup of the tool linked here?
https://forum.xentax.com/viewtopic.php?t=11106

The blog link is offline. The page is on the Wayback Machine, but the actual file download is blocked behind a xentax login so the file isn't on the wayback machine annoyingly.

Would that be it? -> https://www.moddb.com/downloads/dpk4util

Masquerade

15-09-2023, 08:18

Thanks for the link but that is for DPK4, I was looking for DPK5 version. It's alright now though, the game in question only used zlib so I've done it and moved on.

IRAQIGHOST

18-09-2023, 07:38

Forza Horizon 5 V 1.604.481

xtool:c256mb:mpreflate:mzlib+xtool:dd3+LOLZ_ULTRA

Repack Size: 104 Gb
Final Size: 164 Gb

IRAQIGHOST

19-09-2023, 14:06

PAYDAY 3

xtool:dd3+LOLZ_ULTRA

Repack Size: 18.7 Gb
Final Size: 46 Gb

L33THAK0R

19-09-2023, 22:11

I've been investigating a 2017 title "Lone Echo", which seems to use a previously undocumented Oodle Library, v0.1.0, and whose CRCs don't match any other libraries, as posted by KaktoR.

File: oodle_11_win64.dll
CRC-32: 15a0f169

Attempting to discern which member of the oodle family is used has proven to be a tad tricky, with each sample spitting back a result of [0] (Invalid/Unknown) as reported by Razor's Oodle Stream Scanner (oo2scan_7_win64.exe) utility. Attached is both a sample and the library in question. Since these streams are coming up as Invalid/Unknown is it not possible to re-compress them?