XTool 2020 (Plugins)

[Cesar82]

Originally Posted by Masquerade

For Bink 2 videos, you may need to make use of the --diff=* parameter as some patches between CRC imperfect videos to the original state can be larger than XTool's default threshold for patches.

For me if I use --diff=* in arc.ini it doesn't work, return:
ERROR: write error (disk full?) in compression algorithm xtool:c256mb:mbk2

Arc.ini

--cache=0
[External compressor:xtool]
header    = 0
default   = -c64mb -t100p
packcmd   = "{compressor}.exe" precomp { -option} --diff=* - - <stdin> <stdout>
unpackcmd = "{compressor}.exe" decode -t100p - - <stdin> <stdout>

pack.bat
arc.exe a -ep1 -dses --dirs -s; -lc- -di -i2 -r -w.\temp -m"xtool:c256mb:mbk2" data.arc "pack\*"

I also tested using it directly from the command line after removing --diff=* from arc.ini:
xtool:c256mb:mbk2:-diff=*
xtool:c256mb:mbk2:df*

Arc.ini

--cache=0
[External compressor:xtool]
header    = 0
default   = -c64mb -t100p
packcmd   = "{compressor}.exe" precomp { -option} - - <stdin> <stdout>
unpackcmd = "{compressor}.exe" decode -t100p - - <stdin> <stdout>

pack.bat
arc.exe a -ep1 -dses --dirs -s; -lc- -di -i2 -r -w.\temp -m"xtool:c256mb:mbk2:df*" data.arc "pack\*"
or
arc.exe a -ep1 -dses --dirs -s; -lc- -di -i2 -r -w.\temp -m"xtool:c256mb:mbk2:-diff=*" data.arc "pack\*"

If I don't use --diff=* it compresses normally.

FreeArc 0.67 (March 15 2014) using additional options: --cache=0
Creating archive: data.arc using xtool:c256mb:mbk2
Memory for compression 0b, decompression 0b, cache 0b
Compressing 5 files, 336,582,396 bytes
  Compressing health_screen.bk2
  Compressing ms_logo.bk2
  Compressing ps_studios_long_strawberry_4k30_006.bk2
  Compressing lt_outro_1080p.bk2
  Compressing lt_intro_1080p.bk2
Compressed 5 files, 336,582,396 => 291,316,936 bytes. Ratio 86.55%
Compression time: cpu 0.20 sec/real 29.95 sec = 1%. Speed 11.24 mB/s
All OK

Am I doing something wrong?

[Masquerade]

Originally Posted by Cesar82

Am I doing something wrong?

The --diff parameter in XTool sets a max size in bytes or percentage size of the stream in order for XTool to accept the stream.

The * character is just a placeholder (a wildcard effectively meaning anything/everything). My explanation meant you'd need to change this to an integer.

By default, XTool will not accept a stream if the generated XDelta patch is greater in size than 5% of the stream, but using --diff= allows you to increase this threshold.

You can use --diff=20p to set the max patch size to be less than or equal to 20% of the stream size for XTool to accept the stream. You can also use an integer to specify a max patch size in bytes (e.g. --diff=50000).

[exorc]

Originally Posted by Masquerade

Bink Video

• XTool plugin adding support for Bink Video compression.
• This plugin uses Bink Pack library by ProFrager.
• Thanks to Razor12911 for the little help.

Advantages of using this plugin versus vanilla Bink Pack

• Multithreading support.
• No CRC check fail since XTool will patch the videos as they are unpacked.
• Incompatible Bink 2 videos (KB2n) are automatically ignored.

Special Notes

• For Bink 2 videos, you may need to make use of the --diff=* parameter as some patches between CRC imperfect videos to the original state can be larger than XTool's default threshold for patches.
• You may need to use a large chunk size to ensure all videos are processed. See example below.

Example Usage

• Extract the attached archive to the XTool folder and use -mbik or -mbk2 depending on your data set.

Test

• Test on the videos from Assassin's Creed 2 (27x Bink 1 videos).
  

  xtool precomp -mbik "AC2Videos\"
  XTool is created by Razor12911
  
  Streams: 13/13
  Time: 00:00:09 (00:00:01)
  Memory: 148 MB (148 MB)
  
  Results: 937 MB >> 928 MB

  xtool precomp -c128mb -mbik "AC2Videos\"
  XTool is created by Razor12911
  
  Streams: 26/26
  Time: 00:02:12 (00:00:04)
  Memory: 391 MB (391 MB)
  
  Results: 937 MB >> 776 MB

  xtool precomp -c256mb -mbik "AC2Videos\"
  XTool is created by Razor12911
  
  Streams: 27/27
  Time: 00:02:25 (00:00:06)
  Memory: 677 MB (677 MB)
  
  Results: 937 MB >> 725 MB

• Test on 05_DAM folder from Shadow Warrior 3 Defintive Edition (6x Bink 2 videos).
  

  xtool precomp -c256mb -mbk2 "SW3\Content\Movies\Cutscenes\05_DAM"
  XTool is created by Razor12911
  
  Streams: 6/6
  Time: 00:01:06 (00:00:03)
  Memory: 406 MB (406 MB)
  
  Results: 214 MB >> 187 MB

How to use with DiskSpan GUI?

[kj911]

Originally Posted by Masquerade

Bink Video

• Extract the attached archive to the XTool folder and use -mbik or -mbk2 depending on your data set.

Whats wrong or am i lame?? (From testing. See two attached pictures.)

Directly use Binkpack.exe or Freearc usaged whole Bink*.exe file and or CLS-based (de)compressors its works in (de)compress in vice versa... Packed 3.07GB huge BIK file correctly to 2.4GB size with Binkpack.
Use Xtool 0.71 directly or Arc with configurations, don't compress smaller.

[KaktoR]

Originally Posted by Masquerade

Bink Video

• XTool plugin adding support for Bink Video compression.
• This plugin uses Bink Pack library by ProFrager.
• Thanks to Razor12911 for the little help.

Advantages of using this plugin versus vanilla Bink Pack

• Multithreading support.
• No CRC check fail since XTool will patch the videos as they are unpacked.
• Incompatible Bink 2 videos (KB2n) are automatically ignored.

Special Notes

• For Bink 2 videos, you may need to make use of the --diff=* parameter as some patches between CRC imperfect videos to the original state can be larger than XTool's default threshold for patches.
• You may need to use a large chunk size to ensure all videos are processed. See example below.

Example Usage

• Extract the attached archive to the XTool folder and use -mbik or -mbk2 depending on your data set.

Updated plugin.

Basically the 'MessageBoxA' was patched out and replaced with some other function from User32.dll. On some bink2 inputs you had encountered some error messages like 'MotionData_warn' or 'err'. I have included two samples for which it is the case. You shouldn't receive error messages anymore with this patched plugin. Inputs which would give these types of errors are just ignored but processed still, but not compressed with binkpack.

Thanks to masquerade and Razor12911.

[Masquerade]

Plugin Update

Added STDIN/STDOUT support for Bink Video plugin (thanks to Gehrman for idea).

Thanks to 78372 for cls wrapper.

Usage has not changed, but its significantly faster.

[dixen]

XCOM Enemy Unknown

*.bik *.bk2 (With XTool 0.69 + Bink Plugin)

FreeArc 0.67 (March 15 2014) testing archive: data.arc
Tested 168 files, 6,433,251,603 => 7,859,788,952 bytes. Ratio 81.85%
Testing time: cpu 8.44 sec/real 344.19 sec = 2%. Speed 22.84 mB/s
All OK

[Masquerade]

Labyrinth of Galleria: The Moon Society [LZ4]

• Requires LZ4.DLL from the game files. Rename it to liblz4.dll for use with XTool.
• Drop the configuration into XTool folder and use -mLoGTMS

By Razor12911

[Masquerade]

NPK2 Archives

Archives ending in .NPK and have a magic no. of NPK2.

Encrypted using Cipherfinal AES_256_CBC which isn't natively supported by XTool.

Examples of games using this scheme:

• Tokyo Necro
• Demonbane
• Sonicomi

Usage

Place npk2.dll with XTool and use

-mnpk2:k###Key in Hexadecimal###+png+preflate -d2

Example for Tokyo Necro [Steam Version]

-mnpk2:k0xAAE616FB7DB7EE84ED424C474AADDAD69A8372799AD576629601EF9B8938E20F+png+preflate -d2

A file is generated during precomp stage called npk2.bin. This must be retained for decoding as well as npk2.dll. npk2.bin contains the initialisation vectors for re-encrypting the data.

By Razor12911

[dixen]

oo2core_9_win64.dll v2.9.8 & v2.9.9

[Razor12911]

Star Ocean: The Divine Force

Supported games
Star Ocean: The Divine Force (requires libzstd.dll)

Usage
Place sodf.dll near xtool.exe along with required libraries and use -msodf+zstd (game is encrypted), so use -d1 as well

Notes

The library is required both when encoding and decoding.
zstd version 1.5.0 seems to work best
using two instances of xtool without -d1 is faster because some encrypted blocks are large and can only fit in 1 or 2 threads which cannot fully utilise the cpu leading to slower decoding

[Razor12911]

Plugin update available

Changes

- improved header decoder in unity plugin
- added ue5 method in unreal plugin to handle ucas/utoc file format without the need of using database creator

Results

Ghostwire Tokyo\Snowfall\Content\Paks\pakchunk0-WinGDK.ucas

-mue5:k0xC26CC5659A24AE8635585C8687DCE552A205DBD72D C4AA0CBE187C60D88921E2+kraken -d1

Streams: 39249 / 39250
Time: 00:07:54 (CPU 00:46:14)

Size: 4.84 GB >> 11.1 GB

I don't know how it compares with Unreal Engine ucas Database maker but this is an alternative for the newer ucas format as the database maker doesn't support it but this plugin does.

[Razor12911]

PlayStation archive

Supported games
PlayStation games?

Usage
Place psarc.dll near xtool.exe along with required libraries and use -mpsarc

Notes

If you plan on capturing the zlib streams in some of the psarc from The Last of Us, it's advisable to use two instances of xtool because the kraken streams are chunked meaning using -d1 will result in the zlib/reflate part of the precompression receiving incomplete streams.

core.psarc

2,11 GB >> 5.16 GB >> 7.53 GB

[Razor12911]

Originally Posted by Razor12911

core.psarc

2,11 GB >> 5.16 GB >> 7.53 GB

Turns out I missed a few streams, R2 uploaded

core.psarc

2.12 GB >> 5.44 GB >> 7.82 GB

Source 2

Supported games
Half-Life: Alyx (requires liblz4.dll)

Usage
Place src2.ini near xtool.exe along with required libraries and use -msrc2

Notes

Sadly I didn't spend too much time on this game and there's really not a lot of lz4 streams and I just totally forgot to upload it after the disappointment.

[Masquerade]

TT Isle Of Man Ride on the Edge 3 [LZ4F]

• Extract the contents of the attached archive and use -mTTIoM3.
• TTIoM3.xtl can be discarded for decoding, but lz4.exe, xtool.ini and xdelta3_dll.dll must be present.

xtool precomp -mTTIoM3 -c128mb -t8 -df20p "WIN32\PKG\CHUNK_0"

XTool is created by Razor12911

Streams: 19100 / 19112
Time: 00:11:36 (CPU 00:03:19)

Size: 8.23 GB >> 13.4 GB