XTool 2020 (Plugins)

[Razor12911]

Originally Posted by Mini

Razor12911
B.AnimRetarget.cache + B.AnimRetarget.toc
https://www17.zippyshare.com/v/spfv8aj1/file.html
58 MB

unpacked size 186 MB (with https://github.com/IcarusXSEC/WF-Evolution-Unpack )
cmd: EvolutionUnpack B.AnimRetarget.cache /game

or use
http://aluigi.altervista.org/bms/darkness2.bms

info
1) https://github.com/FromDarkHell/TheDarknessIIDecompiled
2) https://zenhax.com/viewtopic.php?t=4793

I managed to create plugin, decompressing the game is possible but recompressing with matching crc is a different story.

Compressed 1 file, 60,893,070 => 203,772,032 bytes. Ratio 334.64%
Compression time: cpu 0.02 sec/real 1.17 sec = 1%. Speed 52.01 mB/s

"B.AnimRetarget.cache". File is broken.
Tested 1 file, 203,772,032 => 60,893,070 bytes. Ratio 334.64%
Testing time: cpu 0.09 sec/real 1.06 sec = 9%. Speed 57.33 mB/s
There were 1 warning(s)

[Razor12911]

Here's an alternative doboz library plugin, it may have bugs as this is the first xtool plugin written in Visual Studio rather than RAD Studio.

via executable

Compressed 1 file, 286,247,872 => 1,095,095,861 bytes. Ratio 382.57%
Compression time: cpu 0.20 sec/real 53.75 sec = 0%. Speed 5.33 mB/s

Tested 1 file, 1,095,095,861 => 286,247,872 bytes. Ratio 382.57%
Testing time: cpu 0.22 sec/real 33.81 sec = 1%. Speed 8.47 mB/s

via library

Compressed 1 file, 286,247,872 => 1,095,095,861 bytes. Ratio 382.57%
Compression time: cpu 0.23 sec/real 2.62 sec = 9%. Speed 109.33 mB/s

Tested 1 file, 1,095,095,861 => 286,247,872 bytes. Ratio 382.57%
Testing time: cpu 0.17 sec/real 27.38 sec = 1%. Speed 10.45 mB/s

Refer to the executable version if you have issues which can be found here.

[L33THAK0R]

Originally Posted by Razor12911

Here's an alternative doboz library plugin, it may have bugs as this is the first xtool plugin written in Visual Studio rather than RAD Studio.

Do any titles other than "The Witcher 3: Wild Hunt" or titles running on the PS Vita use Doboz? Only done a quick skim through, but cant seem to find many cases.

[L33THAK0R]

Originally Posted by Razor12911

I managed to create plugin, decompressing the game is possible but recompressing with matching crc is a different story.

Compressed 1 file, 60,893,070 => 203,772,032 bytes. Ratio 334.64%
Compression time: cpu 0.02 sec/real 1.17 sec = 1%. Speed 52.01 mB/s

"B.AnimRetarget.cache". File is broken.
Tested 1 file, 203,772,032 => 60,893,070 bytes. Ratio 334.64%
Testing time: cpu 0.09 sec/real 1.06 sec = 9%. Speed 57.33 mB/s
There were 1 warning(s)

Bit of a silly sounding question but how do you use this plugin? Does xtool.ini need to be modified for this library to be accepted? Interested in seeing if despite the CRC issues anything can be done with the decompressed data either as it is or via an xdelta/hdiffz patch.

[Razor12911]

Originally Posted by L33THAK0R

Do any titles other than "The Witcher 3: Wild Hunt" or titles running on the PS Vita use Doboz? Only done a quick skim through, but cant seem to find many cases.

I don't know either

Originally Posted by L33THAK0R

Bit of a silly sounding question but how do you use this plugin? Does xtool.ini need to be modified for this library to be accepted? Interested in seeing if despite the CRC issues anything can be done with the decompressed data either as it is or via an xdelta/hdiffz patch.

The sample he sent over has a bms script, meaning you first need to use bms2xtl, the comtype used in the game is evolution/lzfx so I just make a plugin that acts a evolution precompressor, however no matter what parameters you use for lzfx, you simply cannot recompress the data with matching crc. I revisited the data set again today after updating the library to patch the streams but the diff files are just too big and wouldn't make sense to precompress this game.

[KaktoR]

LEGO Star Wars - The Skywalker Saga

My first xtool plugin.

GAME6.DAT

-mxtool:legoswtws
Compressed 1 file, 1,643,604,387 => 3,766,491,875 bytes. Ratio 229.16%
Compression time: cpu 1.22 sec/real 82.54 sec = 1%. Speed 19.91 mB/s
All OK

-mxtool:kraken,l6
Compressed 1 file, 1,643,604,387 => 3,629,442,699 bytes. Ratio 220.82%
Compression time: cpu 1.23 sec/real 92.98 sec = 1%. Speed 17.68 mB/s
All OK

-moo2reck
Compressed 1 file, 1,643,604,387 => 3,696,758,986 bytes. Ratio 224.92%
Compression time: cpu 1.80 sec/real 434.00 sec = 0%. Speed 3.79 mB/s
All OK

GAME.DAT

-mxtool:legoswtws
Compressed 1 file, 4,062,980,016 => 9,814,174,667 bytes. Ratio 241.55%
Compression time: cpu 3.77 sec/real 229.08 sec = 2%. Speed 17.74 mB/s
All OK

-mxtool:kraken,l6
Compressed 1 file, 4,062,980,016 => 9,451,336,766 bytes. Ratio 232.62%
Compression time: cpu 3.39 sec/real 246.06 sec = 1%. Speed 16.51 mB/s
All OK

-moo2reck
Compressed 1 file, 4,062,980,016 => 9,595,617,928 bytes. Ratio 236.17%
Compression time: cpu 5.73 sec/real 1036.90 sec = 1%. Speed 3.92 mB/s
All OK

Thanks to Razor12911 for the education and sorry for the headaches

[Masquerade]

Watch Dogs [XCompress]

• Extract the contents of the attached archive to XTool folder and use -mwatchdogs.
• Requires XCompress plugin, you can get that here.
• Previous database shared by Razor12911 does not work with current XTool which is why I am sharing a new database.

Compressing common.dat, 109,476,518 bytes

Compressed 1 file, 109,476,518 => 405,191,786 bytes. Ratio 370.12%
Compression time: cpu 0.11 sec/real 20.00 sec = 1%. Speed 5.47 mB/s
All OK

[Razor12911]

Crilayla

Supported games
Japanese games

Usage
Place crilayla.ini near xtool.exe in conjunction with executable version of the plugin and use -mcrilayla

Persona 3 Portable\data_EN\umd0.cpk

Compressed 1 file, 147,942,064 => 845,153,964 bytes. Ratio 571.27%
Compression time: cpu 0.14 sec/real 41.55 sec = 0%. Speed 3.56 mB/s

[deepshit]

Hey There
Can somebody help me with NFS Shift conversion?

Game uses xmemcompress. I used the bms script to generate xtl db but looks like xtool cant decompress the file.
I also put the required libraries beside xtool.( exe and dll)
sample and script and db are attached.

[Razor12911]

Originally Posted by deepshit

Hey There
Can somebody help me with NFS Shift conversion?

Game uses xmemcompress. I used the bms script to generate xtl db but looks like xtool cant decompress the file.
I also put the required libraries beside xtool.( exe and dll)
sample and script and db are attached.

Refer to the Project Cars 1 configuration by Masquerade for this game, xmemcompress must have these parameters.

[xmemcompress]
Encode=xcompress.exe d -window=131072 -partition=524288 <library>
Decode=xcompress.exe e -window=131072 -partition=524288 <library>

Compressed 1 file, 25,522,432 => 84,558,796 bytes. Ratio 331.31%
Compression time: cpu 0.05 sec/real 4.22 sec = 1%. Speed 6.05 mB/s

[Masquerade]

Bink Video

• XTool plugin adding support for Bink Video compression.
• This plugin uses Bink Pack library by ProFrager.
• Thanks to Razor12911 for the little help.

Advantages of using this plugin versus vanilla Bink Pack

• Multithreading support.
• No CRC check fail since XTool will patch the videos as they are unpacked.
• Incompatible Bink 2 videos (KB2n) are automatically ignored.

Special Notes

• For Bink 2 videos, you may need to make use of the --diff=* parameter as some patches between CRC imperfect videos to the original state can be larger than XTool's default threshold for patches.
• You may need to use a large chunk size to ensure all videos are processed. See example below.

Example Usage

• Extract the attached archive to the XTool folder and use -mbik or -mbk2 depending on your data set.

Test

• Test on the videos from Assassin's Creed 2 (27x Bink 1 videos).
  

  xtool precomp -mbik "AC2Videos\"
  XTool is created by Razor12911
  
  Streams: 13/13
  Time: 00:00:09 (00:00:01)
  Memory: 148 MB (148 MB)
  
  Results: 937 MB >> 928 MB

  xtool precomp -c128mb -mbik "AC2Videos\"
  XTool is created by Razor12911
  
  Streams: 26/26
  Time: 00:02:12 (00:00:04)
  Memory: 391 MB (391 MB)
  
  Results: 937 MB >> 776 MB

  xtool precomp -c256mb -mbik "AC2Videos\"
  XTool is created by Razor12911
  
  Streams: 27/27
  Time: 00:02:25 (00:00:06)
  Memory: 677 MB (677 MB)
  
  Results: 937 MB >> 725 MB

• Test on 05_DAM folder from Shadow Warrior 3 Defintive Edition (6x Bink 2 videos).
  

  xtool precomp -c256mb -mbk2 "SW3\Content\Movies\Cutscenes\05_DAM"
  XTool is created by Razor12911
  
  Streams: 6/6
  Time: 00:01:06 (00:00:03)
  Memory: 406 MB (406 MB)
  
  Results: 214 MB >> 187 MB

[Cesar82]

Originally Posted by Masquerade

For Bink 2 videos, you may need to make use of the --diff=* parameter as some patches between CRC imperfect videos to the original state can be larger than XTool's default threshold for patches.

For me if I use --diff=* in arc.ini it doesn't work, return:
ERROR: write error (disk full?) in compression algorithm xtool:c256mb:mbk2

Arc.ini

--cache=0
[External compressor:xtool]
header    = 0
default   = -c64mb -t100p
packcmd   = "{compressor}.exe" precomp { -option} --diff=* - - <stdin> <stdout>
unpackcmd = "{compressor}.exe" decode -t100p - - <stdin> <stdout>

pack.bat
arc.exe a -ep1 -dses --dirs -s; -lc- -di -i2 -r -w.\temp -m"xtool:c256mb:mbk2" data.arc "pack\*"

I also tested using it directly from the command line after removing --diff=* from arc.ini:
xtool:c256mb:mbk2:-diff=*
xtool:c256mb:mbk2:df*

Arc.ini

--cache=0
[External compressor:xtool]
header    = 0
default   = -c64mb -t100p
packcmd   = "{compressor}.exe" precomp { -option} - - <stdin> <stdout>
unpackcmd = "{compressor}.exe" decode -t100p - - <stdin> <stdout>

pack.bat
arc.exe a -ep1 -dses --dirs -s; -lc- -di -i2 -r -w.\temp -m"xtool:c256mb:mbk2:df*" data.arc "pack\*"
or
arc.exe a -ep1 -dses --dirs -s; -lc- -di -i2 -r -w.\temp -m"xtool:c256mb:mbk2:-diff=*" data.arc "pack\*"

If I don't use --diff=* it compresses normally.

FreeArc 0.67 (March 15 2014) using additional options: --cache=0
Creating archive: data.arc using xtool:c256mb:mbk2
Memory for compression 0b, decompression 0b, cache 0b
Compressing 5 files, 336,582,396 bytes
  Compressing health_screen.bk2
  Compressing ms_logo.bk2
  Compressing ps_studios_long_strawberry_4k30_006.bk2
  Compressing lt_outro_1080p.bk2
  Compressing lt_intro_1080p.bk2
Compressed 5 files, 336,582,396 => 291,316,936 bytes. Ratio 86.55%
Compression time: cpu 0.20 sec/real 29.95 sec = 1%. Speed 11.24 mB/s
All OK

Am I doing something wrong?

[Masquerade]

Originally Posted by Cesar82

Am I doing something wrong?

The --diff parameter in XTool sets a max size in bytes or percentage size of the stream in order for XTool to accept the stream.

The * character is just a placeholder (a wildcard effectively meaning anything/everything). My explanation meant you'd need to change this to an integer.

By default, XTool will not accept a stream if the generated XDelta patch is greater in size than 5% of the stream, but using --diff= allows you to increase this threshold.

You can use --diff=20p to set the max patch size to be less than or equal to 20% of the stream size for XTool to accept the stream. You can also use an integer to specify a max patch size in bytes (e.g. --diff=50000).

[dixen]

XCOM Enemy Unknown

*.bik *.bk2 (With XTool 0.69 + Bink Plugin)

FreeArc 0.67 (March 15 2014) testing archive: data.arc
Tested 168 files, 6,433,251,603 => 7,859,788,952 bytes. Ratio 81.85%
Testing time: cpu 8.44 sec/real 344.19 sec = 2%. Speed 22.84 mB/s
All OK

[Masquerade]

Labyrinth of Galleria: The Moon Society [LZ4]

• Requires LZ4.DLL from the game files. Rename it to liblz4.dll for use with XTool.
• Drop the configuration into XTool folder and use -mLoGTMS

By Razor12911