XTool 2020 (Main Project)

[aleison]

How can I add the new version of Xtool to ultraarc to compress using SREP+LZMA+XTOOL? and how to add the same decompression method to an inno setup script? please help I have not been able to follow, thank you very much.

[Razor12911]

Hi, I recommend that you use DiskSpan GUI as the project currently has support and UltraARC has long been abandoned.

[aleison]

Originally Posted by Razor12911

Hi, I recommend that you use DiskSpan GUI as the project currently has support and UltraARC has long been abandoned.

Thank you very much Razor what happens is that when compressing a game using the method Precomp (Ztool) + LOLZ + SREP, I get the following error when decompressing:

An error occured while unpacking: Does not match checksum!
Unarc.dll returned an error code: -12
Error: file D:\Program Files Plague Tale Requiem REPACK JPW\DATAS\SHARED.DPC failed CRC check.

Other games I compress and decompress with this method without problems, but I don't know why I get this error with this game, I have tried to compress it several times and I get the same error when decompressing it. Can you please help me if you can do something? thank you very much for your help.

[Masquerade]

^^
This is the XTool topic and you are asking about ZTool.

ZTool has been long since discontinued and is missing countless improvements added to both old XTool and XTool 2020.

[aleison]

Originally Posted by Masquerade

^^
This is the XTool topic and you are asking about ZTool.

ZTool has been long since discontinued and is missing countless improvements added to both old XTool and XTool 2020.

Thanks for answering, I have another doubt, could you tell me exactly what information I have to enter in ARC.ini to decompress a compressed file using XTOOL(xZlib)+SREP+LOLZ methods? I did everything, but I always get an error: unarc.dll -2 compression method not supported xZlib+SREP.

[Masquerade]

^^
... this is the XTool topic now you are asking about your issues with DiskSpanGUI.

DiskSpanGUI has a built in decompressor area for your archives so you can decompress them without issues.

[leftnomemes]

Sorry if dumb question, but where can I find the documentation for the tool. New to this scene, looking to learn the tool properly.

[shazzla]

Doc (.chm) available in the xtool package.

[Razor12911]

Update available

Changes

- added advanced configuration based plugin support
- added UI mode when xtool.exe is launched with xtoolui.dll present
- added skip verification mode
- xtool now enforces w15 deflate stream detection by default
- fixed oodle scanner exceptions when incomplete stream is detected
- fixed issue with zlib codec not accepting streams from database plugins
- updated command line syntax

Notes

advanced configuration plugin is a way of writing more complex ini files for stream detection without the need of coding skills or the need of creating a library based plugin.

spidey plugin is an example, regular configuration files could not add support for this game however, advanced configuration allows this and here's an example of how that looks like

[StreamList1]
Name=lz4
Codec=lz4
BigEndian=0
Signature=0x1000352415344
Structure1=Signature(8),Count(4),Unk1(4),Unk2(16)
StructureN=DPos(8),CPos(8),DSize(4),CSize(4),Unk3(8)
StructureS=Stream
CounterStart1=1
CounterEnd1=Count
CounterStep1=1
StreamPosition=CPos
StreamOffset=0
CompressedSize=CSize
DecompressedSize=DSize
Condition1=

The user interface mode has been added and it's not meant to compete with tools like DiskSpanGUI or other similar programs but it's meant to help newbies operate the program as everything is made as simple as possible. The program can still be useful for regular users too, you can use the user interface mode to get theoretical outputs without doing actual (pre)compression, if you set output to none (leave blank in cli mode) and enable skip verification then essentially, you are performing a scan on the input (similar to what Drop and Scan for Zlib does) but for all supported codecs.

An example is Cyberpunk 2077 as shown below.
xt1.PNG
xt2.PNG

From here, you can find out the output size if you were to do precompression (in about 5 mins or less, all depends on drive speed).

More uses? Well people have a problem deciding how much chunk size to use, you can change the chunk size and see how the stream found or how the size differs to decide what chunk size to use.

The UI is a bit borked at the moment but the next update will improve upon it.

xtool enforces w15 streams because I figured out that this is what causes reflate to produce crc errors because there are more false positives this way. So how does this affect results, it doesn't because 99% of games use w15 anyways except Dishonored 2 and Dishonored Death of the Outsiders (set w10 for them)

Syntax has been updated as per request from Cesar82:
--dedup can also be -dd
--dedup=#, -dd#
--dbase, -db
--diff=#,-df#
--verbose, -v
--skip, -s

[Razor12911]

Update available

Changes

- added library checker (trial and error)
- improved user interface
- fixed bugs related to oodle scanner
- skip verification no longer applies to encryption codecs

Notes

Library checker allows you to find out what library was used through trial and error, all you do is pick a directory with a list of either lz4, zstd or oodle libraries and set output to none then every single library found in that directory is loaded by xtool one by one while showing you how many streams were processed by each and their respective precompressed outputs. This should allow you to maximize compression in one click rather than doing it manually.
xt3.PNG
xt4.PNG

More syntax changes as per request from Cesar82:
--zlib= can also be -zb
--lz4=#, -l4#
--lzo=#, -lo#
--zstd=#, -zs#
--oodle=#, -od#
--srepmem=75p (when decoding) as requested by Gehrman

Announcement

This is the last update for xtool, I am at a point where I think I have done enough for this project. I have dragged development of the project longer than I should have but I guess it's a habit of mine of not leaving something unfinished and this is the creative vision I had for this tool from the start, it took longer than I expected but I'm glad it's done.

So what does that mean I am leaving the forum? No, I'll stick around for the time being, it's just that the main project is no longer getting updates... and I had to stop at the magic number 69 . So if there are bugs and issues, you'd have to refer to the older releases which are made available on the main post.

[dixen]

Xtool v0.6.9 Elden Ring Edition

Two xtool.exe with injected oo2core_5_win64.dll and oo2core_8_win64.dll

Compressing 1 file, 2,655,581,216 bytes
Compressing Data3.bdt
Compressed 1 file, 2,655,581,216 => 4,846,074,313 bytes. Ratio 182.49%
Compression time: cpu 4.28 sec/real 347.49 sec = 1%. Speed 7.64 mB/s
All OK

[External compressor:xt8,xt5]
header = 0
packcmd = {compressor} precomp -mkraken:l6 -c32mb -t100p - - <stdin> <stdout>
unpackcmd = {compressor} decode -t100p - - <stdin> <stdout>

All with Happy New Year!!! Be Happy!!

[Hexagon123]

Tried using LZ4, but no streams.
Anyone make a plugin for this XTool.

Seen Puyo Puyo Tetris 2, Sonic Frontiers, and a few titles that uses this compression.

Ref1:http://wiki.xentax.com/index.php/Sonic_Games_PAC
Ref2:https://github.com/nickworonekin/puyo-pac

[Razor12911]

Originally Posted by Hexagon123

Tried using LZ4, but no streams.
Anyone make a plugin for this XTool.

Seen Puyo Puyo Tetris 2, Sonic Frontiers, and a few titles that uses this compression.

Ref1:http://wiki.xentax.com/index.php/Sonic_Games_PAC
Ref2:https://github.com/nickworonekin/puyo-pac

The sample you've provided is too small and only contains one stream, can't make a plugin off of this.

XTool is created by Razor12911

[0] Performing scan from block 0000000000000000 to 000000000000081F (2080)
[0] Actual lz4 stream found at 0000000000000040 (2005 >> 9880)

[0] Processing streams on block 0000000000000000 to 000000000000081F (2080)
[0] Processing lz4 stream at 0000000000000040 (2005 >> 9880 >> 2171) using a0 has failed
[0] - Patching stream at 0000000000000040 (2005 >> 2171) [1151] has failed

Streams: 0/1
Time: 00:00:00 (00:00:00)
Memory: 128 MB (128 MB)

[elit]

"I had to stop at the magic number 69.."
^So you are a Scorpio

Anyway I am glad you made my lz_scan (almost)obsolete because this is way better. I will later check in more detail how to do a multi-lib scan. From the pictures it seems you create a version directory and put a dll there, like 2.8.1/codec.dll, 2.8.2/codec.dll and so on, while dlls are all of same name.
As opposite to my lz_scan where I had a single directory with like liblz4_112.dll, liblz4_120.dll etc.

Advanced config is what I love about this new version and will use, really kudos for that.

I recommend if you want to call it a day, don't add a new features anymore but hang around to collect and fix last remaining bugs that users report over time. What you achieved here is a project that gives any institutional budget projects run for their money. I say lets not leave it half backed, after all you do remember Freearc(and srep) fiasco. Both a buggy, half assed projects, but no proper alternative unfortunately(at least for my usage as a GUI archiver).

After some time once project is bug-hardened with no more changes, maybe I try to port it one day to C and/or multiplatform if I decide to switch to Linux in the future. Thank you for all you did for us so far.

[Wanterlude]

Good afternoon, everyone!
I decided to pack Transport Fever 2 using zlib+dedup+lolz, but srep takes up all RAM when unpacking. I looked on the forum to see if this was a problem before and saw the --mem=# parameter. I tried "20p" "800m" "800mb" but nothing ended up working. This is what I have prescribed in arc.

[External compressor:xtool]
header    = 0
default   = -c128mb -t100p
packcmd   = xtool.exe precomp { -moption} -db -dd5 - - <stdin> <stdout>
unpackcmd = xtool.exe decode -t100p --mem=10p - - <stdin> <stdout>

xtool - last ver.
srep 32-bit v392
ASIS script (i try v732,v744)