XTool 2020 (Plugins)

[Razor12911]

You do realize that almost each time lz4 is updated on github it compresses data a bit differently? This "bit differently" is why xtool from the beginning allows users to change the library however they want so they pick whichever one gives the best result. When it comes to lz4, there is no universal dll for all games so if you want to add support, you must tell the user to always add the libraries that come with plugins to the DiskSpan_GUI folder.

The lz4 library I ship in xtool is v1.8.3 which is the commonly used library, some games use an older and some use a newer but nonetheless if the streams cannot be restored correctly, xdelta is applied.

If the user uses the incorrect liblz4.dll, xtool tries to use xdelta on the streams and if the user chose the wrong one or does not use the one that comes with a plugin, you end up with negative ratio. Where using a precompressor just gives worse result.

This is seen here, although I am not sure how this situation in particular happened...

Edit: The same is true with zstd, each update causes xtool to use xdelta to try to salvage some streams. Zlib would also be using the same method of xdelta but other libraries such as reflate and preflate exist that handle streams that zlib can't and when it comes to the situation of lz4, there is no such tool.

[Cesar82]

@Razor12911, could you tell me if the FIFA plugin (EA_Frostbite_Engine) is not yet functional?
It only has a link to FIFA21, but the example was a little confusing and has several filesand a larger XTool (Maybe it was rewritten especially for this game).
I didn't understand where this part of the method came from? -mfrostbite3+zstd.dict:l19
If the plugin is ready, it could post an example with the other previous fifa using XTool 0.31?
How is it determined which section [Stream#] will be used for each game? In frostbite3.ini has 3 commented sections that are for FIFA games
If you want, let me know that I remove my posts from this topic of yours so that I don't have irrelevant content.

I tried to do some tests here, but it seems that XTool has an AI. Even with a lack of plugin libraries, XTool compresses too (no errors were shown), and the extraction was done normally. (It seems that for lack of DLL files, XTool changes the method to avoid errors).
You're the guy, congratulations on the great job.

[Razor12911]

FIFA is a special case like Far Cry Primal, Watch Dogs 2/Legion, they encrypted the game and used two algorithms, kraken/leviathan + zstd, the zstd used has dictionary. So even before we begin. You have to use FIFA 19 Cas Decryption Tool and then use xtool (I think for FIFA 19 and FIFA 20). As for the method "zstd.dict:l19", it's more or less the same as libdunia situation where I added a temporary codec internally and then remove it, however the xtool with the temporary codec can still be found here

I added these methods because of the scene, I thought they have made a breakthrough in bypassing the latest iteration of Denuvo so I added several methods quickly but as you can see right now, these games are still uncracked so I remove the methods and then slowly put them back via plugin support now that I have made xtool accept them.

Originally Posted by Cesar82

@Razor12911I tried to do some tests here, but it seems that XTool has an AI. Even with a lack of plugin libraries, XTool compresses too (no errors were shown), and the extraction was done normally. (It seems that for lack of DLL files, XTool changes the method to avoid errors).
You're the guy, congratulations on the great job.

Of course checks are made to see if a library can be initialized or not. I'm not sure if you checked the source code included in Unreal Engine plugin

Capture2.PNG

Here the plugin checks if a user has given a key and necessary parameters and if nothing is given then using -mue4 then the plugin will be unavailable.

If for example you are just trying to use frostbite3.ini, it has 5 different streams to be detected, if you have liblz4.dll, xtool will search for all 5 different streams but when it tries to add the streams that were found and finds that some libraries are missing, these streams are discarded so that xtool continues working and not raise an error.

Edit:

[Stream1]
// DAInquisition
Name=zlib
Codec=zlib:l98
BigEndian=1
Signature=0x0270
Structure=Dict(1),DSize(3),Signature(2),CSize(2),ZLibHdr(2),Stream
StreamOffset=0
CompressedSize=CSize - 6
DecompressedSize=DSize
Condition1=DSize > CSize
Condition2=CSize > 64
Condition3=CSize <= 262144
Condition4=Dict=0
Condition5=ZLibHdr = 0x78DA

[Stream2]
// BF4-BFV, NFS2015-NFSHeat, MECatalyst
Name=lz4
Codec=lz4
BigEndian=1
Signature=0x0970
Structure=Dict(1),DSize(3),Signature(2),CSize(2),Stream
StreamOffset=0
CompressedSize=CSize
DecompressedSize=DSize
Condition1=DSize > CSize
Condition2=CSize > 64
Condition3=CSize <= 262144
Condition4=Dict = 0

[Stream3]
// MEAndromeda 
Name=zstd
Codec=zstd:l19
BigEndian=1
Signature=0x0F70
Structure=Dict(1),DSize(3),Signature(2),CSize(2),ZStdHdr(4),Stream
StreamOffset=-4
CompressedSize=CSize
DecompressedSize=DSize
Condition1=DSize > CSize
Condition2=CSize > 64
Condition3=CSize <= 262144
Condition4=Dict = 0
Condition5=ZStdHdr = 0x28B52FFD

[Stream4]
// FIFA 18-20
Name=kraken
Codec=kraken:l8:t128
BigEndian=1
Signature=0x1970
Structure=Dict(1),DSize(3),Signature(2),CSize(2),OodleHdr(2),Stream
StreamOffset=-2
CompressedSize=CSize
DecompressedSize=DSize
Condition1=DSize > CSize
Condition2=CSize > 64
Condition3=CSize <= 262144
Condition4=Dict = 0
Condition5=OodleHdr = 0x8C06

[Stream5]
// FIFA 21
Name=leviathan
Codec=leviathan:l8:t128
BigEndian=1
Signature=0x1970
Structure=Dict(1),DSize(3),Signature(2),CSize(2),OodleHdr(2),Stream
StreamOffset=-2
CompressedSize=CSize
DecompressedSize=DSize
Condition1=DSize > CSize
Condition2=CSize > 64
Condition3=CSize <= 262144
Condition4=Dict = 0
Condition5=OodleHdr = 0x8C0C

Alternatively, when using -mfrostbite3 method, you can use it this way -mfrostbite3:zstd, xtool will check what Streams are called "zstd" as they have names, Stream3 is the one that is called zstd, so it will only search using information from Stream3. Same as -mfrostbite3:zstd:zlib, where Stream1 and Stream3 are the ones that are searched. However if no parameter is given, -mfrostbite3, all 5 streams are searched.

[Razor12911]

To properly explain how xtool behaves. The plugins I post here should be considered "external" plugins, there are also plugins in xtool called "internal" plugins. These are just common compressors used in games, zlib, lz4, lzo and etc. Since some internal plugins have issues detecting streams by themselves, lz4 as an example. What normally happens is, the external plugins I post here give information to the internal plugins. Telling them exactly how the streams are found and where to look for them. So if you use a plugin like frostbite3 and don't add zlibwapi.dll, liblz4.dll and etc. Basically no libraries whatsoever. When the external plugins give information to the internal ones. They will refuse to accept streams or any of the information as they didn't initialize because of missing libraries therefore no error is given however the ratio is affected. You can try it now yourself, use any of the plugins without the required libraries, xtool will just give a result of 100% ratio and no error. Encoding is never a problem, the problem starts when you are decoding, if you used something when encoding but it's missing when decoding. XTool will give you an error telling you a certain method was not found.

Edit:

This is what I told this person here, to run xtool without stdio so that I can see the error message that xtool gives.

Here's an example, I used zlib method when encoding and then I removed zlibwapi.dll and then tried to decode.

[Cesar82]

@Razor12911, now I understand about the sections [Stream#], the method must be used the value obtained in the key Name=.
The link to the temporary codec is not accessible.
I haven't looked at the Unreal Engine source code (I'll see).

run xtool without stdio.

I don't know if only I don't understand. You could explain that.
Check your inbox.
Thanks!

[Razor12911]

Plugins Updated

Changes

- Updated The Crew 2 to support latest update
- Updated Far Cry Primal, Watch Dogs 2 and Watch Dogs Legion

[patricktech]

Nice

[HunterEmrah]

Originally Posted by Snake288

Cyberpunk 2077 about compression please, I want to get help.

i have the same problem when compressing cyberpunk 2077 with cb2077 plugin, it gives error in random progress percent (1.2%, 1.5%, 1.7%)

F:\Games\Cyberpunk 2077\pack\pak>arc a -s; -dsgerpn -lc- -di -hppassword -cfg"C:\Program Files (x86)\FreeArc\bin\xtool2020\arc.ini" -wF:\Temp -mxtool+srep+lzma2 ..\data1.bin * -r
FreeArc 0.67 (March 15 2014) Creating archive: ..\data1.bin using xtool+srep+lzma2
Memory for compression 0b, decompression 0b, cache 1mb
Compressing 16 files, 57,137,917,952 bytes. Processed   1.7%
ERROR: write error (disk full?) in compression algorithm xtool

arc.ini

[External compressor:xtool]
header    = 0
packcmd   = XTool.exe precomp -mcp2077 -c128mb -t100p --dbase - - <stdin> <stdout>
unpackcmd = XTool.exe decode -t100p - - <stdin> <stdout>

[External compressor:srep]
header = 0
packcmd   = srep -m3f -a1 {options} $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = srep -d -mem2048mb - - <stdin> <stdout>

[External compressor:lzma2]
header = 0
packcmd = 7z_fa a -txz -an -mcrc=0 -mx9 -m0=lzma2:a=1:d=350m:mf=bt4:fb=273:mc=4294967295:lc=4:lp=0:pb=2 -mmt=3 -si -so <stdin> <stdout>
unpackcmd = 7z_fa x -txz -an -y -si -so <stdin> <stdout>

There is no problem when compressing files separately (tried only these files: basegame_4_animation.archive, lang_en_voice.archive, lang_ru_voice.archive), but I get this error when I compress all ".archive" files at the same time (except what I mentioned before).

also tried without <stdin> <stdout> but it's stuck on %100 after arc.exe compressing files in to one archive.

[Razor12911]

[External compressor:xtool]
header    = 0
packcmd   = XTool.exe precomp -mcp2077 -c128mb -t100p --dbase $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = XTool.exe decode -t100p - - <stdin> <stdout>

Try using these settings and what are your system specs?

Also if there are errors that xtool reports, I want to see the error message.

[hwang4]

HunterEmrah
"i have the same problem when compressing cyberpunk 2077 with cb2077 plugin, it gives error in random progress percent (1.2%, 1.5%, 1.7%)"


Decreasing "thread usage" may help ! (t90p worked for me)!

[HunterEmrah]

Originally Posted by Razor12911

what are your system specs?

i5-10500 6c12t, 16 GB DDR4 3200MHz, HDD. ok will try later and share result. thanks!

[Cesar82]

@Razo12911, is it not possible for you to include the "crilayla" method to XTool 2020 (x64) via plugin?

[Razor12911]

Plugins Updated

Changes

- Updated Frostbite 3 plugin

@Cesar82

It is possible, I plan to do so in future.

[Cesar82]

@Razor12911, thanks for updates.

1) I did a Frostbite3 plugin test with a sample of the MEAndromeda game.
Using the libzstd.dll library (1,042,500 bytes) that is included with XTool 0.3.2 does not work.
It only works if you use the library that came with the old XTool (627,903 bytes).
It also works with ZTool's libzstd.dll (412,160 bytes) and also with ZSTDRec's precompressor ibzstd.dll (3,647,423 bytes).
It should not work with the library in the _libraries folder of the "xtool_0.3.2.7z" file (1,042,500 bytes)?

2) Could you give me the exact names of the "oo2core_#_win64.dll" libraries required by the frostbite3 plugin for games: FIFA 18, FIFA 19, FIFA 20, FIFA 21, NFS Heat ?
P.S: If anyone has the game library (from game), please send it with game name to me to include in DiskSpan_GUI.

I attached my sample test of MEAndromeda with the plugin Frostbite3.

[Razor12911]

1) https://fileforums.com/showpost.php?...5&postcount=31

The library include is from the latest zstd build, 2020. Mass Effect Andromeda came out in 2017. If you read what I said regarding lz4 that it also applies to zstd, you'll know why the old library works and the new one doesn't.

2) https://fileforums.com/attachment.ph...chmentid=28782

I attached the libraries from the games.