XTool 2020 (Plugins)
 

Configuration based
Age of Empires III: Definitive Edition
Cyberpunk 2077
Deathloop
EA Frostbite Engine
Saints Row The Third Remastered

Library based
Anvil Engine
Bethesda
CryEngine
Haemimont Engine
Hitman
King Arthur Knight's Tale
Madness Engine
Spiderman Remastered
Unity Engine
Unreal Engine

Database 'based'
Automobilista 2
Castle Of Illusion
Dirt 5
Distant Worlds 2
Orcs Must Die! 3
Project Cars
Project Cars 3
Redout 2
Scarlet Nexus
Super Robot Wars 30
Test Drive: Ferrari Racing Legends
Ubisoft Dunia 2 Engine
Unravel 1 & 2
Ultra Age
WRC10

Executable based
crilayla
Doboz
lz4x
xmemcompress

Notes

*.ini and *.xtl files can be discarded when decoding, they will no be longer needed.
*.dll are required both encoding and decoding

EA Frostbite Engine

Supported games
Dragon Age Inquisition (requires zlibwapi.dll)
FIFA 18, FIFA 19, FIFA 20, FIFA 21, FIFA 22 (require oo2core_#_win64.dll)
NFS Rivals, NFS 2015, NFS Payback (requires liblz4.dll)
Mirror's Edge Catalyst (requires liblz4.dll)
Battlefield 4, Battlefield 1, Battlefield Hardline, Battlefield V (requires liblz4.dll)
Battlefield 2042 Open-Beta (require oo2core_#_win64.dll)
Mass Effect Andromeda, Star Wars Battlefront II, Star Wars Squardon (requires libzstd.dll)
NFS Heat (require oo2core_#_win64.dll)

Usage
Place frostbite3.ini near xtool.exe along with required libraries and use -mfrostbite3.

Saints Row The Third Remastered

Supported games
Saints Row The Third Remastered (requires liblz4.dll)

Usage
Place SR3Remaster.ini near xtool.exe along with required libraries and use -msr3remaster.

Changelog

Ubisoft Dunia 2 Engine

Supported games

Far Cry 3, Far Cry 4 (requires lzo2.dll)
Far Cry 5, Far Cry New Dawn, Far Cry 6 (requires liblz4.dll)
Far Cry Primal, Watch Dog II, Watch Dogs Legion (requires liblz4.dll and libdunia.dll)
The Crew 2 (requires require oo2core_5_win64.dll from game)
Watch Dogs (supported via bms2xtl, use script)

Usage

Generate your own database using this tool

igepatch.dat
ige.dat
common.dat
patchshadersobj.dat
shadersobj.dat
farcry5_hd.dat

Pack

Unpack

Tested without *.fat
Thank you Razor)))

Added The Crew 2 support

Comparison between precompressors + settings:

oo2reck
xtool 2020 -mkraken
xtool 2020 -mkraken:l5
xtool 2020 -mthecrew2

Early rejoiced...
Patch.dat

DELZOREC - 8.3 gb > 11 gb
XTOOL with .xtl db - 8.3 gb > 8.5 gb
Dedup on or off - no matter

Sorry i'm stupid)) It try on R3...R4 all fine

Actually there is a problem :)

It happens when you are multi threading (there is something I miscalculated), it will be fixed in the next release. Thanks for spotting it ;)

Edit: test 2010_R5

Thanks but it tomorrow only

Added Far Cry Primal, Watch Dogs 2, Watch Dogs Legion support

Some results

Far Cry Primal
Watch Dogs 2
Watch Dogs Legion

Notes
This is highly dependent on the temporary libdunia codec xtool just received. I haven't done a proper job so this is what you can get for now. :(

Thank ProFrager, FitGirl and ZAZA4EVER

Yes, this is pretty much what I meant back in 2018: https://fileforums.com/showpost.php?...&postcount=296

Now that I think about it, this could be also a way to make xtool codec-universal. Imagine this:
You would expose certain functions name calls as an open specification - similar to what FA does with CLS, like say:
Next, xtool would always scan for an universal dll name, like custom_cmp.dll or whatever. On the command line, something like "customcodec" would be possible to type as a codec name, which would call for the interested functions from the library.

People would create those dlls on their demand from existing quickbms code amd implement those function names in them, for example copy lzss or yaz0 codec from quickbms source and make a dll with those above functions as a bridge.

Xtool would load custom dll and use those function calls to call the codec. Thogether with custom scripts, one could inflate almost anything now and would make the tool useful to the long future and tor many more current and future codecs and games, even if you cease to work on it one day. Same as FA is still so used today because of its universal container aspect and more.

Cyberpunk 2077

Supported games
Cyberpunk 2077 (requires oo2ext_7_win64.dll)

Usage
Place cp2077.ini near xtool.exe along with required libraries and use -mcp2077

comparison (basegame_4_animation.archive)

oo2reck
xtool 2020 -mkraken
xtool 2020 -mcp2077

Hi Razor12911 Help
 

Cyberpunk 2077 about compression please, I want to get help.

KaktoR control the compression method
getting the error seen in the picture
disk full error my disk free space is close to 650 gb
the computer system information you can see in my picture gipi strong


https://i.imgyukle.com/2020/12/26/aSbuGj.jpg
https://i.imgyukle.com/2020/12/26/aSbS58.jpg

is your free disk space hdd an external drive?

C:\Games\Cyberpunk 2077 game Samsung 960 evo 512 gb m2 installed

I think M2 is not supported, but I don't know exactly. try to install on a different drive

7 Days I Tried Different Method Of Trying To Compress This Game

oo2reck7+srep_new+NLolz I get an error when installing compressed

I've tried them all, I'm getting the same error again

https://i.imgyukle.com/2020/12/30/acYWHH.jpg

https://i.imgyukle.com/2020/12/31/agYNyh.jpg

https://i.imgyukle.com/2020/12/31/agYpm1.jpg

decompress via command line and don't use stdio, I'm interested in the error xtool reports.

Edit: I would understand if you got crc error or if any of other decoding processes terminated but they are still running from the task manager and still working so I would assume the issue is with diskspan or something else. Try recompressing the game without diskspan and then report if the error still persists.

Hi Razor12911 Cyberpunk 2077 about compression please, I want to get help.Cyberpunk 2
 

Cyberpunk 2077 about compression please, I want to get help.

Razor12911 control the compression method
getting the error seen in the picture
disk full error my disk free space is close to 650 gb
the computer system information you can see in my picture gipi strong


https://i.imgyukle.com/2020/12/26/aSbuGj.jpg
https://i.imgyukle.com/2020/12/26/aSbS58.jpg

I have had this experience when using a portable drive as the output.

I repeat, rerun the test without using stdio because I want to see the error message.

Razor12911
how is it done ? test without using stdio do you know an example ?

I guess ???

Unreal Engine

Codecs available

ue3
Use on unreal engine 3 games

This codec works without parameters.
ue4
Use on unreal engine 4/5 games with *.pak file extension

Parameters available:
k#... - to specify decryption key if game is encrypted

You can also directly specify what compression algorithm was used to help xtool with detection and speed up the process.
Algorithms available:
zlib, lz4, lz4hc, lzna, kraken, mermaid, selkie, hydra, leviathan

Example:
ue5
Use on unreal engine 4/5 games with *.utoc/*.ucas format

Parameters available:
k#... - to specify decryption key if game is encrypted

Example:

How this can work without chunk size and block size? It uses the same chunk size of zlib? And about the block size?

chunk size corresponds to -c parameter used in xtool.

Edit: You don't really need to set high values for chunk size because unlike ue4dt, the chunks when split actually cut off some data from being decrypted but in xtool this rarely happens.

@Razor12911, sorry for so many questions, but I'm trying to understand how XTool works.

1) The "liblz4.dll" library of the "Ubisoft Dunia 2 Engine" plugins files (Far Cry Primal, Watch Dogs 2, Watch Dogs Legion) (136 Kb), and different from the "liblz4.dll" library shared with XTool (439 Kb).
Only the "Far Cry 5" plugin use "liblz4.dll" library is the same hash (439 Kb).
Can I use the library in the "_libraries" folder (439 Kb) for all plugins?
2) Do I understand correctly that it is not necessary to include the "* .ini" and "* .xtl" files of the plugins to decompress with XTool.
As an example, if you use the "EA Frostbite Engine" plugin it is not necessary to add anything with XTool to decompress compatible games.
This is correct?
3) Are all plugins compatible with XTool 0.31?
4) Unreal Engine plugin is it not necessary to put "0x" before the key?
5) XTool already has support for methods "LZ4" and "ZSTD" (has the libraries in the folder "_libraries")?

1) Short answer, no. Far Cry Primal, Watch Dogs 2 and Watch Dogs Legion use modified lz4 because the devs thought this was a good idea. Far Cry 5 use the normal version along with New Dawn.

2) Yes, *.ini and *.xtl files are no longer needed after compressing. But the libraries they come with are needed.

3) Yes, apart from Far Cry Primal, Watch Dogs 2, Watch Dogs Legion as these three require libdunia which was removed before and I'm currently busy making a plugin version of it.

4) you can add 0x, $ or # which symbolizes Hexadecimal or just remove it, Up to you. Xtool will check if 0x exists before the key and if it does, it will remove it and if it doesn't, it will convert the key to binary so it uses it on decryption function.

5) "XTool already has support for methods", well the answer is yes and also no. XTool does support these methods yes, but you can't use -mlz4 on a lz4 compressed game and expect xtool to precompress it because scanning for lz4 streams is almost impossible but these "plugins", *.xtl, *.ini and *.dll are the ones that help lz4 by telling it where to find the streams. As for zstd, -mzstd works because scanning for zstd is easier.

You just need to wait for me to update the documentation so you know how to write your own plugin to be able to add support to a game that xtool doesn't have native support for.

@Razor12911, I confused the liblz4.dll libraries with the version I use with the old XTool.
The "Far Cry 5" plugin uses "liblz4.dll" library is the same as the library that came with the old XTool (439 Kb).
The library in the folder "_Libraries" of XTool 2020 is larger and has 494 KB (506,742 bytes).
Sorry for more question.
1) Can I use the library in the "_Libraries" folder of XTool 2020 (494 KB) with the Far cry 5 plugin?
P.S: I'm asking so many questions because I want to include support for all of your XTool plugins in DiskSpan_GUI.
I'm asking so many questions because I want to include support for all of your XTool plugins in DiskSpan_GUI.
As I mentioned before, plugins / library files will be copied before compression and deleted after compressing.
As there are variations of the libraries I would like to be able to use it for all methods (except where it is not possible).
I will wait for the XTool documentation to understand it better.
Thanks!

You do realize that almost each time lz4 is updated on github it compresses data a bit differently? This "bit differently" is why xtool from the beginning allows users to change the library however they want so they pick whichever one gives the best result. When it comes to lz4, there is no universal dll for all games so if you want to add support, you must tell the user to always add the libraries that come with plugins to the DiskSpan_GUI folder.

The lz4 library I ship in xtool is v1.8.3 which is the commonly used library, some games use an older and some use a newer but nonetheless if the streams cannot be restored correctly, xdelta is applied.

If the user uses the incorrect liblz4.dll, xtool tries to use xdelta on the streams and if the user chose the wrong one or does not use the one that comes with a plugin, you end up with negative ratio. Where using a precompressor just gives worse result.

This is seen here, although I am not sure how this situation in particular happened...

Edit: The same is true with zstd, each update causes xtool to use xdelta to try to salvage some streams. Zlib would also be using the same method of xdelta but other libraries such as reflate and preflate exist that handle streams that zlib can't and when it comes to the situation of lz4, there is no such tool.

@Razor12911, could you tell me if the FIFA plugin (EA_Frostbite_Engine) is not yet functional?
It only has a link to FIFA21, but the example was a little confusing and has several filesand a larger XTool (Maybe it was rewritten especially for this game).
I didn't understand where this part of the method came from? -mfrostbite3+zstd.dict:l19
If the plugin is ready, it could post an example with the other previous fifa using XTool 0.31?
How is it determined which section [Stream#] will be used for each game? In frostbite3.ini has 3 commented sections that are for FIFA games
If you want, let me know that I remove my posts from this topic of yours so that I don't have irrelevant content.

I tried to do some tests here, but it seems that XTool has an AI. Even with a lack of plugin libraries, XTool compresses too (no errors were shown), and the extraction was done normally. (It seems that for lack of DLL files, XTool changes the method to avoid errors).
You're the guy, congratulations on the great job.

FIFA is a special case like Far Cry Primal, Watch Dogs 2/Legion, they encrypted the game and used two algorithms, kraken/leviathan + zstd, the zstd used has dictionary. So even before we begin. You have to use FIFA 19 Cas Decryption Tool and then use xtool (I think for FIFA 19 and FIFA 20). As for the method "zstd.dict:l19", it's more or less the same as libdunia situation where I added a temporary codec internally and then remove it, however the xtool with the temporary codec can still be found here

I added these methods because of the scene, I thought they have made a breakthrough in bypassing the latest iteration of Denuvo so I added several methods quickly but as you can see right now, these games are still uncracked so I remove the methods and then slowly put them back via plugin support now that I have made xtool accept them.

Of course checks are made to see if a library can be initialized or not. I'm not sure if you checked the source code included in Unreal Engine plugin

Attachment 28724

Here the plugin checks if a user has given a key and necessary parameters and if nothing is given then using -mue4 then the plugin will be unavailable.

If for example you are just trying to use frostbite3.ini, it has 5 different streams to be detected, if you have liblz4.dll, xtool will search for all 5 different streams but when it tries to add the streams that were found and finds that some libraries are missing, these streams are discarded so that xtool continues working and not raise an error.

Edit:

Alternatively, when using -mfrostbite3 method, you can use it this way -mfrostbite3:zstd, xtool will check what Streams are called "zstd" as they have names, Stream3 is the one that is called zstd, so it will only search using information from Stream3. Same as -mfrostbite3:zstd:zlib, where Stream1 and Stream3 are the ones that are searched. However if no parameter is given, -mfrostbite3, all 5 streams are searched.

To properly explain how xtool behaves. The plugins I post here should be considered "external" plugins, there are also plugins in xtool called "internal" plugins. These are just common compressors used in games, zlib, lz4, lzo and etc. Since some internal plugins have issues detecting streams by themselves, lz4 as an example. What normally happens is, the external plugins I post here give information to the internal plugins. Telling them exactly how the streams are found and where to look for them. So if you use a plugin like frostbite3 and don't add zlibwapi.dll, liblz4.dll and etc. Basically no libraries whatsoever. When the external plugins give information to the internal ones. They will refuse to accept streams or any of the information as they didn't initialize because of missing libraries therefore no error is given however the ratio is affected. You can try it now yourself, use any of the plugins without the required libraries, xtool will just give a result of 100% ratio and no error. Encoding is never a problem, the problem starts when you are decoding, if you used something when encoding but it's missing when decoding. XTool will give you an error telling you a certain method was not found.

Edit:

This is what I told this person here, to run xtool without stdio so that I can see the error message that xtool gives.

Here's an example, I used zlib method when encoding and then I removed zlibwapi.dll and then tried to decode.

@Razor12911, now I understand about the sections [Stream#], the method must be used the value obtained in the key Name=.
The link to the temporary codec is not accessible.
I haven't looked at the Unreal Engine source code (I'll see).
I don't know if only I don't understand. You could explain that.
Check your inbox.
Thanks!

Plugins Updated

Changes

- Updated The Crew 2 to support latest update
- Updated Far Cry Primal, Watch Dogs 2 and Watch Dogs Legion

Nice

i have the same problem when compressing cyberpunk 2077 with cb2077 plugin, it gives error in random progress percent (1.2%, 1.5%, 1.7%)

arc.ini
There is no problem when compressing files separately (tried only these files: basegame_4_animation.archive, lang_en_voice.archive, lang_ru_voice.archive), but I get this error when I compress all ".archive" files at the same time (except what I mentioned before).

also tried without <stdin> <stdout> but it's stuck on %100 after arc.exe compressing files in to one archive.

Try using these settings and what are your system specs?

Also if there are errors that xtool reports, I want to see the error message.

HunterEmrah
"i have the same problem when compressing cyberpunk 2077 with cb2077 plugin, it gives error in random progress percent (1.2%, 1.5%, 1.7%)"


Decreasing "thread usage" may help ! (t90p worked for me)!