FileForums

FileForums (https://fileforums.com/index.php)
-   Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)
-   -   XTool 2020 (Plugins) (https://fileforums.com/showthread.php?t=103876)

HunterEmrah 31-07-2021 14:22

Quote:

Originally Posted by KaktoR (Post 493525)
Edit: Stucks at 4%. I guess you have to edit frostbite3.ini then.

what exactly needs to be changed?

HunterEmrah 31-07-2021 22:19

Quote:

Originally Posted by macut18 (Post 493532)
xtool_2012_R2 + frostbite3.ini is ok.

Quote:

Originally Posted by macut18 (Post 493532)
-mzstd

you are using only the xtool+zstd

Quote:

Originally Posted by Razor12911 (Post 488282)
Usage
Place frostbite3.ini near xtool.exe along with required libraries and use -mfrostbite3.


Cesar82 01-08-2021 20:25

Could someone let me know if any of these Razor12911 tools "Unreal Engine ucas Database maker" or "Dunia-Disrupt Engine Database maker" generate the *.xtl file for plugin "Unravel" 1 and 2?
Or is there any tool to generate the plugin *.xtl file for this games.


@Razor12911, the file ".ced" (cryengine database) in CryEngine plugin is it necessary to be with XTool during decompression?

PsYcHo_RaGE 01-08-2021 22:19

@HunterEmrah The problem is zstd is slow

It does compress it but after taking a huge amount of time
It's not actually stuck, it looks stuck (DON'T PANIC :D)

We've done tests on Ryzen 32 threaded CPU
It did take us 8 hours to complete the XTool precompression, can take less if you configure the chunk size and -t param perfectly.
And then ofc more for srep and lolz

KaktoR 01-08-2021 23:33

I took the opprtunity to make a test last night with SWBF2 + zstd codec.

Settings used: xtool:zstd+srep:m3f+4x4:lzma

Took about 6 hours, unpacking gave error on srep :rolleyes:

PsYcHo_RaGE 02-08-2021 01:09

Quote:

Originally Posted by KaktoR (Post 493550)
I took the opprtunity to make a test last night with SWBF2 + zstd codec.

Settings used: xtool:zstd+srep:m3f+4x4:lzma

Took about 6 hours, unpacking gave error on srep :rolleyes:

Try xtool:frostbite3 plugin(zstd)+srep:m3f:l512+lzma2 or lzma

:( Sad8669 02-08-2021 10:50

Reminds me of using GrittiBanzli on CB4, that shit took 25 hours to precompress and ended up giving an error xD.

While using reflate, it took less than an hour.

KaktoR 02-08-2021 11:07

Quote:

Originally Posted by PsYcHo_RaGE (Post 493552)
Try xtool:frostbite3 plugin(zstd)+srep:m3f:l512+lzma2 or lzma

Nah, I have better things to do as waiting a whole day for some compression and block my machine from doing anything other then compressing stuff :p

PsYcHo_RaGE 02-08-2021 23:51

Quote:

Originally Posted by KaktoR (Post 493557)
Nah, I have better things to do as waiting a whole day for some compression and block my machine from doing anything other then compressing stuff :p

lol :D

Edit-1: Normal ZSTD Codec works fine on Star Wars Battlefront 2 but not the frostbite3 plugin, looks stuck (Is stuck honestly) but is using CPU and RAM normally even if you keep it like that for 3-5 days :D.

Tested on Ryzen 32 Threaded and Intel i9 16 Threaded CPU

Edit-2: ZSTD is buggy :D

crachlow 02-09-2021 12:49

Hello Razor12911. Please tell an inexperienced user where the keys for UE4 games are taken?

KaktoR 02-09-2021 13:42

From the shipping exe file

elit 08-09-2021 08:05

In Pathfinder Kingmaker, there is a folder called Bundles, containing among other unity files with *_bundle ending but without extension. These are about 16.6gb.

I can inflate them using latest xtool(with unity.dll from here) and liblz4.dll from ztool(~65kb), however I have to set solid = 0 in arc.ini for xtool. With solid ON it stop inflating at around 800mb into, giving only 17.4gb. I don't know if it's bug or unity files are supposed to work without solid. Otherwise it work fine, just wanted to mention it.
(On side note, punity will not work with that liblz4.dll, give exception loading it.)

Now I am thinking its maybe better to turn off solid for most precompressors and engines I use, such as crilayla, unreal, unity, anvil, kraken, zstd... also mp3, ogg, jpg etc.. except deflate? It seems too flaky and I don't have time to test manually every time 10+gb of data whether it inflated correctly everything or stopped at some point. I know this problem existed for long time.
Anyone with experience, for precomp tools alone solid turned off should not have much impact on final size right?

Masquerade 08-09-2021 11:13

elit
The only precompressor I have seen that works better in non-solid mode is the unity one. The rest is fine to run normally.

elit 09-09-2021 08:19

Thank you, I also just found that it's not even viable option after all.
When doing something like xtool:munity+srep+lzma where xtool is set to solid = 0, FA will pass that upstream. That is srep and lzma will also be applied to each file individually before joining archive.

I found out after wasting hours compressing, so now I am doing those files through batch script before packing with FA. I found using ppx2 to parallel xtool(with -t1) for each file instead of doing "tar" through FA with single xtool and -t100p give significant speed boost as well.

Razor12911 28-09-2021 02:20

Plugin(s) updated

Changes

- Added support for FIFA 22 in Frostbite 3 plugin

Notes

Some streams are left behind. :rolleyes:

Just like FIFA 21 wasn't worth it, FIFA 22 is also not worth it.

Code:

Tested 1 file, 233,182,965 => 103,625,666 bytes. Ratio 225.02%
Testing time: cpu 0.14 sec/real 179.17 sec = 0%. Speed 0.58 mB/s

They keep increasing compression, I guess as a form of differentiating FIFA 20 from FIFA 21 from FIFA 22 because it's the same game...

Razor12911 28-09-2021 02:23

1 Attachment(s)
Deathloop

Usage
Place deathloop.ini near xtool.exe along with required libraries and use -mdeathloop

Notes

The configuration file can be discarded after encoding.

oo2core_7_win64.dll and oo2core_8_win64.dll are required

Cesar82 28-09-2021 09:12

Thank you very much for the updates.
Always surprising us with good news...

I look forward to you having time again and if possible bring us some working plugin for the marvels avengers game.
If you are going to try to develop plugin for avengers and need samples let me know that I will upload them, but they (are 4 GB files each).

Razor12911 06-10-2021 15:39

Plugin(s) updated

Changes

- Added support for Battlefield 2042 (based on Open-Beta files) in Frostbite 3 plugin

Notes

The final release of game might have different compression settings. (Make adjustments where necessary)

Battlefield 2042 Open Beta\Data\Win32\installation\levelsinstallpackage\ cas_01.cas:
Code:

Tested 1 file, 405,732,037 => 204,049,188 bytes. Ratio 198.84%
Testing time: cpu 0.20 sec/real 91.37 sec = 0%. Speed 2.23 mB/s

@Cesar82

The tools that are currently available are closed source so unless if there are other methods that I don't know about regarding decryption of Avengers game, perhaps you could make a plugin yourself. (Make it your first xtool plugin) :)

ZAZA4EVER 13-10-2021 15:52

Farcry 6 plugin
 
1 Attachment(s)
Farcry 6 plugin

Usage
Place fc6.xtl near xtool.exe along with required libraries and use -mfc6

Notes

The plugin file can be discarded after encoding.
liblz4 is required
plugin made by Dunia/Disrupt Engine Database maker from here
Thanks to Razor

Files list of plugin
Code:

common.dat
common.fat
fctber_disk.dat
fctber_disk.fat
fctber_dlc.dat
fctber_dlc.fat
fctworlds.dat
fctworlds.fat
installpkg.dat
installpkg.fat

Plugin tested

Code:

Compressed 10 files, 38,471,392,432 => 55,786,073,159 bytes. Ratio 145.01%
Compression time: cpu 72.53 sec/real 1936.74 sec = 4%. Speed 19.86 mB/s
All OK

use Xtool V_0.3.21 and liblz4

daveyrob 20-10-2021 04:04

does anybody know why i cant get the bethesda plugin to work with skyrim? maybe it needs updating or something? it encodes just fine using the r3 version posted earlier with latest xtool but it fails to decode, the command line error is EThredException: error in the method 'skyrim'

i tried the bsa.dll from mega aswell but that fails to do anything, i also tryed multiple versions of xtool with no luck, would somebody kindly help me out??

KaktoR 20-10-2021 04:14

Because the plugin is broken again.

It worked on a specific xtool version, but I don't remember which of them.

daveyrob 20-10-2021 04:22

Quote:

Originally Posted by KaktoR (Post 494530)
Because the plugin is broken again.

It worked on a specific xtool version, but I don't remember which of them.

ahh thank you , glad to know i wasnt doing something wrong, i did notice the r2 bsa.dll was tested on an early post with xtool 0.3.9 but it seems r3 does not work with it, neither did the others i tried, i dont see any links to the r2 bsa either :(

Cesar82 20-10-2021 04:52

1 Attachment(s)
This attachment combination works, but you cannot use the latest versions of XTool with it.

daveyrob 20-10-2021 05:26

Quote:

Originally Posted by Cesar82 (Post 494532)
This attachment combination works, but you cannot use the latest versions of XTool with it.

thank you both, appreciate it :)

Masquerade 28-10-2021 07:23

1 Attachment(s)
Scarlet Nexus: Updated Database for v1.04
Test on a small sample:
Code:

Compressed 8 files, 23,979,472 => 48,613,558 bytes. Ratio 202.73%
Compression time: cpu 0.05 sec/real 11.41 sec = 0%. Speed 2.10 mB/s
All OK


L0v3craft 03-01-2022 18:16

@Razor12911 please can you release an updated plugin for Watch Dogs Legion? I tried with this and with different xtool versions but precompression doesn't work.

prince55 03-01-2022 22:27

Quote:

Originally Posted by Razor12911 (Post 494134)
Deathloop

Usage
Place deathloop.ini near xtool.exe along with required libraries and use -mdeathloop

Notes

The configuration file can be discarded after encoding.

oo2core_7_win64.dll and oo2core_8_win64.dll are required

WDL supported database maker please or updated .xtl :)

:( Sad8669 04-01-2022 03:48

I digged into WDL once, found out that videos.dat contains bink files both HD+4K and they can ripped using HEX. I looked quite into it and realized that it can be reduced to as low as 15~18 GB. Alot of .wem files too, for which Masquerade made a seperate thread.

It would take weeks ofcourse, just to prepare the xdelta patches for each file since BPK compressor ain't coming along very well.


All of these without using XTool.

Razor12911 04-01-2022 12:46

1 Attachment(s)
Quote:

Originally Posted by prince55 (Post 495328)
WDL supported database maker please or updated .xtl :)

as with most of these games, I am either lacking samples or lacking in source materials to even make a plugin for them.

https://fileforums.com/showpost.php?...76&postcount=6

Use this to generate your own database file then use the files in the attachment for precompression, change the liblz4.dll depending on whichever gives better result.

Side note, this is incomplete. (More samples needed for completion)

Use 0.3.21 or earlier, there is a bug in 0.3.22 that prevents search plugins from working and shall be fixed in the next update

prince55 04-01-2022 15:55

Quote:

Originally Posted by Razor12911 (Post 495330)
as with most of these games, I am either lacking samples or lacking in source materials to even make a plugin for them.

https://fileforums.com/showpost.php?...76&postcount=6

Use this to generate your own database file then use the files in the attachment for precompression, change the liblz4.dll depending on whichever gives better result.

Side note, this is incomplete. (More samples needed for completion)

Use 0.3.21 or earlier, there is a bug in 0.3.22 that prevents search plugins from working and shall be fixed in the next update

Streams in Only 23 files detected , some other big files (as london.dat) isn't detected
I used Xtool 0.3.21 with attached dlls in precompress and decompress
only shadersobj.dat expands well
but faced error in decompress :(

Razor12911 07-01-2022 06:31

Quote:

Originally Posted by L0v3craft (Post 495327)
@Razor12911 please can you release an updated plugin for Watch Dogs Legion? I tried with this and with different xtool versions but precompression doesn't work.

After several tests, I've come to the conclusion that precompression on this game is not worth it and here's why:

1. Weak compression is used and a strong compression algorithm does still work on the compressed data
2. A custom lz4 variant was used for the compression of this game meaning the public lz4 decompressor does not work.
3. I did get lz4 decompressor from ProFrager once before, it works on most streams but some get left behind as decompression just straight up fails, worse at times it corrupts the memory buffer causing xtool to crash.
4. All streams that are decompressed will have xdelta diff files contributing to negative ratio worsening the compression ratio you would have gotten otherwise.

So I decided to no longer work on this game as it seems to be a pain in the ass to only gain 1gb (probably at best), not worth the trouble, sorry :(

Snake288 07-01-2022 06:47

Watch Dogs Legion hi how do I do the compression process

Masquerade 09-02-2022 08:17

1 Attachment(s)
Scarlet Nexus: Updated Database for v1.07
  • This database is for game version v1.07 / Steam Build 7988311.
  • Created using Database Maker thanks to Razor12911.
  • Follow the guidance in this post for how to use: https://fileforums.com/showpost.php?...&postcount=166

    Code:

      Compressing pakchunk47-WindowsNoEditor.ucas
    Compressed 1 file, 82,772,046 => 219,071,436 bytes. Ratio 264.67%
    Compression time: cpu 0.13 sec/real 38.91 sec = 0%. Speed 2.13 mB/s
    All OK


mausschieber 09-02-2022 09:27

Quote:

Originally Posted by Masquerade (Post 495768)
Scarlet Nexus: Updated Database for v1.07
[LIST][*]This database is for game version v1.07 / Steam Build 7988311.[*]Created using Database Maker thanks to Razor12911.[*]Follow the guidance in this post for how to use: https://fileforums.com/showpost.php?...&postcount=166

the how to use link do not work

KaktoR 09-02-2022 09:39

https://fileforums.com/showpost.php?...&postcount=166

Razor12911 16-02-2022 22:36

1 Attachment(s)
xmemcompress

Supported games

To be used by QuickBMS scripts...

Razor12911 10-03-2022 08:16

Frostbite 3 Configuration updated

Changes

Updated to support xtool 0.4.4+

Cesar82 10-03-2022 18:14

Quote:

Originally Posted by Razor12911 (Post 496155)
Frostbite 3 Configuration updated

Changes

Updated to support xtool 0.4.4+

I tested it with samples of the ME Andromeda game (cas_08.cas) using the library libzstd.dll v1.2.0 together with XTool 0.3.11 (Before changing parse parameters) and it worked fine (Ratio 171.10%).
XTool 0.4.5 (with libzstd.dll v1.2.0) no does not inflate the precompressed file (Ratio 100.0%).

Code:

Pack.bat
arc.exe a -ep1 -r -ed -s; -w.\temp -mxtool:c32mb:mfrostbite3,zstd data.arc "pack\*"

Arc.ini
[External compressor:xtool]
header    = 0
default  = -c64mb -t100p
packcmd  = "xtool.exe" precomp { -option} --dbase - - <stdin> <stdout>
unpackcmd = "xtool.exe" decode -t100p - - <stdin> <stdout>


Files near XTool.exe
frostbite3.ini
libzstd.dll (v1.2.0)


Razor12911 10-03-2022 18:58

1 Attachment(s)
Endianess can be confusing at times... :) try this

Cesar82 10-03-2022 19:23

1 Attachment(s)
Quote:

Originally Posted by Razor12911 (Post 496164)
Endianess can be confusing at times... :) try this

Thanks.
It worked perfectly now for ME Andromeda.


All times are GMT -7. The time now is 17:37.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2026, vBulletin Solutions Inc.
FileForums @ https://fileforums.com