Delete ?!
Why ?
Not really understand....

Edit:
I mean pass --dedup parameter to chain. I dont want a new arc.ini section.

There were undocumented changes in xtool 0.6.5, --dedup can be used as -dd so you are able to pass this via freearc provided your arc.ini looks like this

then your method will be something like -mxtool:mzlib:dd

Thanks ,Z !
If there are other undocumented changes ,it would be nice to publish! :)

help with Unreal Engine
 

I am trying to compress Madel of Honor Airborne with xtool but it's not inflating the input files.

Here is the sample:

https://www62.zippyshare.com/v/3b32KZZm/file.html

Result with uelr:
Result with xtool:
IDK what is wrong with xtool compression.

XTool niot supported decompress ue3 lzo files

Update available

Changes

- fixed issues with exporting precompression database
- fixed issues with deduplication feature consuming a lot of system memory
- fixed oodle codec from auto enabling selkie method
- fixed reflate related checksum issues due to false positives

XTool 0.6.6

Test Psychonauts 2

Before that, absolutely all versions issued CRC ERROR in zlib+reflate mode.
@Razor12911, thank you for fix

@Razor
Thanks for the update and your continued work :D

@Razor12911 :

Xtool 066 bug.

xtool.exe precomp -mreflate -c32mb -t100p --verbose - - < %1 > %1.out

Streams: 11678/11678
Time: 00:00:25 (00:00:22)
Memory: 291 MB (291 MB)

arc.ini :

xtool066\xtool.exe precomp { -option} {options} -t100p - - <stdin> <stdout>

cmd :

arc a -mxtool:mreflate:c32mb p:\output.arc "p:\input.pak"
FreeArc 0.67 (March 15 2014) creating archive: p:\dd.arc
Compressed 1 file, 185,868,230 => 516,491,144 bytes. Ratio 277.88%
Compression time: cpu 0.17 sec/real 6.18 sec = 3%. Speed 30.10 mB/s
All OK

But when dedup enabled: --dedup, -dd, :dd (via arc.ini ,cmd line pararmeter ,etc. Same effect):

xtool.exe precomp -mreflate -c32mb -dd -t100p --verbose - - < %1 > %1.out

EStreamError: TBufferedStream.Seek not implemented

arc.ini :

xtool066\xtool.exe precomp { -option} {options} -dd -t100p - - <stdin> <stdout>

cmd :

$ arc a -mxtool:mreflate:c32mb p:\output.arc "p:\input.pak"
FreeArc 0.67 (March 15 2014) updating archive: p:\dd.arc
Compressing 1 file, 185,868,230 bytes. Processed 100%
ERROR: general (de)compression error in xtool:mreflate:c32mb

P/Reflate,Zlib ,it doesnt matter ,same error.

Update available

Changes

- fixed oodle scanner exceptions when incorrect library is used
- fixed issues with deduplication feature

New XTOL in Ultraarc
 

How can I add the new version of Xtool to ultraarc to compress using SREP+LZMA+XTOOL? and how to add the same decompression method to an inno setup script? please help I have not been able to follow, thank you very much.

Hi, I recommend that you use DiskSpan GUI as the project currently has support and UltraARC has long been abandoned.

Thank you very much Razor what happens is that when compressing a game using the method Precomp (Ztool) + LOLZ + SREP, I get the following error when decompressing:

An error occured while unpacking: Does not match checksum!
Unarc.dll returned an error code: -12
Error: file D:\Program Files Plague Tale Requiem REPACK JPW\DATAS\SHARED.DPC failed CRC check.

Other games I compress and decompress with this method without problems, but I don't know why I get this error with this game, I have tried to compress it several times and I get the same error when decompressing it. Can you please help me if you can do something? thank you very much for your help.

^^
This is the XTool topic and you are asking about ZTool.

ZTool has been long since discontinued and is missing countless improvements added to both old XTool and XTool 2020.

Thanks for answering, I have another doubt, could you tell me exactly what information I have to enter in ARC.ini to decompress a compressed file using XTOOL(xZlib)+SREP+LOLZ methods? I did everything, but I always get an error: unarc.dll -2 compression method not supported xZlib+SREP.

^^
... this is the XTool topic now you are asking about your issues with DiskSpanGUI.

DiskSpanGUI has a built in decompressor area for your archives so you can decompress them without issues.

Documentation
 

Sorry if dumb question, but where can I find the documentation for the tool. New to this scene, looking to learn the tool properly.

Doc (.chm) available in the xtool package. :)

Update available

Changes

- added advanced configuration based plugin support
- added UI mode when xtool.exe is launched with xtoolui.dll present
- added skip verification mode
- xtool now enforces w15 deflate stream detection by default
- fixed oodle scanner exceptions when incomplete stream is detected
- fixed issue with zlib codec not accepting streams from database plugins
- updated command line syntax

Notes

advanced configuration plugin is a way of writing more complex ini files for stream detection without the need of coding skills or the need of creating a library based plugin.

spidey plugin is an example, regular configuration files could not add support for this game however, advanced configuration allows this and here's an example of how that looks like

The user interface mode has been added and it's not meant to compete with tools like DiskSpanGUI or other similar programs but it's meant to help newbies operate the program as everything is made as simple as possible. The program can still be useful for regular users too, you can use the user interface mode to get theoretical outputs without doing actual (pre)compression, if you set output to none (leave blank in cli mode) and enable skip verification then essentially, you are performing a scan on the input (similar to what Drop and Scan for Zlib does) but for all supported codecs.

An example is Cyberpunk 2077 as shown below.
Attachment 33329
Attachment 33330

From here, you can find out the output size if you were to do precompression (in about 5 mins or less, all depends on drive speed).

More uses? Well people have a problem deciding how much chunk size to use, you can change the chunk size and see how the stream found or how the size differs to decide what chunk size to use.

The UI is a bit borked at the moment but the next update will improve upon it.

xtool enforces w15 streams because I figured out that this is what causes reflate to produce crc errors because there are more false positives this way. So how does this affect results, it doesn't because 99% of games use w15 anyways except Dishonored 2 and Dishonored Death of the Outsiders (set w10 for them)

Syntax has been updated as per request from Cesar82:
--dedup can also be -dd
--dedup=#, -dd#
--dbase, -db
--diff=#,-df#
--verbose, -v
--skip, -s

Update available

Changes

- added library checker (trial and error)
- improved user interface
- fixed bugs related to oodle scanner
- skip verification no longer applies to encryption codecs

Notes

Library checker allows you to find out what library was used through trial and error, all you do is pick a directory with a list of either lz4, zstd or oodle libraries and set output to none then every single library found in that directory is loaded by xtool one by one while showing you how many streams were processed by each and their respective precompressed outputs. This should allow you to maximize compression in one click rather than doing it manually.
Attachment 33335
Attachment 33336

More syntax changes as per request from Cesar82:
--zlib= can also be -zb
--lz4=#, -l4#
--lzo=#, -lo#
--zstd=#, -zs#
--oodle=#, -od#
--srepmem=75p (when decoding) as requested by Gehrman

Announcement

This is the last update for xtool, I am at a point where I think I have done enough for this project. I have dragged development of the project longer than I should have but I guess it's a habit of mine of not leaving something unfinished and this is the creative vision I had for this tool from the start, it took longer than I expected but I'm glad it's done.

So what does that mean I am leaving the forum? No, I'll stick around for the time being, it's just that the main project is no longer getting updates... and I had to stop at the magic number 69 :D. So if there are bugs and issues, you'd have to refer to the older releases which are made available on the main post.

Xtool v0.6.9 Elden Ring Edition

Two xtool.exe with injected oo2core_5_win64.dll and oo2core_8_win64.dll

All with Happy New Year!!! Be Happy!!

Hedgehog Engine 2 titles
 

Tried using LZ4, but no streams.
Anyone make a plugin for this XTool.

Seen Puyo Puyo Tetris 2, Sonic Frontiers, and a few titles that uses this compression.

Ref1:http://wiki.xentax.com/index.php/Sonic_Games_PAC
Ref2:https://github.com/nickworonekin/puyo-pac

The sample you've provided is too small and only contains one stream, can't make a plugin off of this.

"I had to stop at the magic number 69.."
^So you are a Scorpio :)

Anyway I am glad you made my lz_scan (almost)obsolete because this is way better. I will later check in more detail how to do a multi-lib scan. From the pictures it seems you create a version directory and put a dll there, like 2.8.1/codec.dll, 2.8.2/codec.dll and so on, while dlls are all of same name.
As opposite to my lz_scan where I had a single directory with like liblz4_112.dll, liblz4_120.dll etc.

Advanced config is what I love about this new version and will use, really kudos for that.

I recommend if you want to call it a day, don't add a new features anymore but hang around to collect and fix last remaining bugs that users report over time. What you achieved here is a project that gives any institutional budget projects run for their money. I say lets not leave it half backed, after all you do remember Freearc(and srep) fiasco. Both a buggy, half assed projects, but no proper alternative unfortunately(at least for my usage as a GUI archiver).

After some time once project is bug-hardened with no more changes, maybe I try to port it one day to C and/or multiplatform if I decide to switch to Linux in the future. Thank you for all you did for us so far.

Good afternoon, everyone!
I decided to pack Transport Fever 2 using zlib+dedup+lolz, but srep takes up all RAM when unpacking. I looked on the forum to see if this was a problem before and saw the --mem=# parameter. I tried "20p" "800m" "800mb" but nothing ended up working. This is what I have prescribed in arc.

xtool - last ver.
srep 32-bit v392
ASIS script (i try v732,v744)
https://i.ibb.co/L5B5S7Z/sk.png

it should be --srepmem=...

xtool reflate issue from Win7 to Win8.1
 

I would like to bring to your attention an issue I am currently experiencing with xtool.

In the past, I compressed several games with it under Windows 7. Probably most used version by me back then was 047 and 053. I test every archive after creation and back then every one of them tested fine.

However about 2 months ago I switched to Windows 8.1 and I find myself facing decompression errors.
Specifically this almost exclusively relate to reflate(I had one zlib giving me error as well but not all). It is 100% reproducible and do not matter which xtool version I use(I keep ones I compressed anything with just in case). Reflate in archive = 100% fail. Any archive. Due to that I excluded disk issue.

Note that ztool with -cm0 work fine under Win8.1, as well as 32bit xtool-crilayla version. Problem is only xtool->reflate(maybe occasional zlib) from what I seen so far. Thankfully majority of my archives are ztool based due to me stubbornly sticking to it for so long but still quite few of new ones I made use xtool already. xtool doesn't have any fancy switches like -cm0, I tried -t1 to no avail.

I am currently trying all archives to make list of broken ones, I also already installed Win7 under Hyper-V late night and will try more on it later but don't know if environment will be 1:1, maybe xtool is having issues due to memory/exception handling of Win8.1.
Also, I did of course already tried Win7 compatibility(all relevant exe files in chain) as well as running as an admin(real one, enforced through group policy).

If anyone have an idea, I will appreciate it.

That's interesting, so it seems a known issue. Thank you, I will try later after I scan all my archives and report back. I hope this will really fix it.

^Unfortunately, putting both freearc.exe and xtool.exe into registry like above did not help.Attachment 34189

EDIT: Not all reflate are broken, in fact many are fine. I am starting to see potential pattern but will confirm it later whether its case. Ones with the issue so far are the ones with custom settings, like chunk size -c32m etc., whereas all the good ones are just default -mxtool:reflate so far.

EDIT2: I see the pattern. This error affect both reflate and zlib, when high chunk size(>32mb) is used. This was working fine on Win7. Changing thread number doesn't seem to cut it. And I have same rig with enough memory(16G). I will try few more things but it seems as if Win8.1 is handling memory or relevant functions differently.

EDIT3: Windows 7 under Hyper-V did not help. I still believe archives are good and this is because I don't run under native.

@elit
Increase the size of the Windows page file (pagefile.sys).

could you try isolating the bug and send over the sample that causes xtool to bug out

@Gehrman
I have 16G RAM + 16G swap. I also tried -t1 option. This is certainly not the case of RAM but also not of disk space.

@Razor12911
I have looked into it today and it does look like it may have to do with chunk size. All archives that bugged out had -c more than 32mb(64m, 128m), but those without any option or up to 32m were all fine. Best at this point will be for you to see yourself. I will upload 2 of them + FA whole pack that I use. With that you will be able to understand how I use it and try for yourself. Again this all worked under W7 native install. Also I am sure among those hundreds of archives where ztool was used, there had to be at least several with chunk size >32m and those passed fine(I tested my whole collection).
I will upload things somewhere and post the links by tomorrow.

@Razor12911,
Ok I already found a site that was capable to test my gigabit/hdd limits :)

Anyway, this link gives you 2 archives and FA bundle:
https://ufile.io/f/hjbf7

In FA bundle, I use freearc.exe(GUI). In bin\_EC\xtool\ directory are the relevant xtool files. Archives use xtool.exe, so you overwrite one of different versions to that. I believe I used v047 most of the time, but can't say for sure which one I used on those archives. However, none of them successfully unpacked under Win8.1.

You will be able to test 1F.arc to the almost end, which confirm that last chain with big chunk size set is the one problematic(see archive info from GUI). Inside 1R.arc is a single file and will bug out at the very beginning.

Please let me know what you think. Also if anyone will be able to unpack those without error, then it would confirm my suspicion about my Win8.1.

EDIT: Also, when creating new archive under Win8.1(same parameters), it will test and extract correctly. I tested it on "1R" content. Still don't believe this to be bitrot/disk corruption etc. More like different OS environment+HW is causing intolerance in between versions. Before I had Win7 and i5-4690K CPU. Now I have Win8.1 and Ryzen 5800X. This is not a good thing if the case, change of HW/OS should not affect decompression reliability.

Did you ever think about making a linux build?

I once did but it would cause me headaches as a Windows user and as such, the testing would most likely be relegated to Linux users and on this forum, there's really not that many. I mean even tests on Windows are too much for ask for at times :D

png bug
 

With -mpng.. method, xtool may hang, remaining unresponsive. This probably happen with non-png data. For example if doing whole "Data" directory in "Total War: Warhammer 2".

With -mpng though, more than 20k new streams can be found in those .pack(s) and inflation of another 500mb can be achieved.
Test passed without hanging when -c32m was used, but I fear this is more like a workaround/cheating and I fear of using bigger chunks due to risk of not unpacking data sometime in the future again(as it was the case already as mentioned in my posts above).

Attachment 34259

EDIT: I have narrowed down which specific file is causing the bug: https://www.udrop.com/7YDG/local_en_2.7z
It is replicable on my comp. Xtool069 fails/hang with reflate+zlib+png:d1 in its own GUI scanner, while it doesn't hang with c32m added.

Are there any switches to disable the 256kb minimum file size for the "Erase" functionality? I've got some tens of thousands of small ~20kb files that I'm trying to rip/erase but alas the minimum limit is making it a bit tricky.

The filter is for bytes, not kilobytes.

Huh, guess I must have messed something else up then!

Special questions: Hardly ways integrate in last xtool internal (not srep related!) deduplications functions adding to old xtool v0.12 versions?? Or make very tiny xtool compressors, named in xdedup or any names, in has ONE deduplications functions with stdio mode? (No zlib/reflate or many more functions..) Would I make sense?