Grumpy@

more posts here :D I can't find it easy

that's the why studying hard.... ;)

can't wait to test the new xtool :D

Here's an early working version of xtool. I'm still busy adding preflate, reflate and other things so currently only the zlib function works which means this will not work on Dishonored 2 or DOOM but it will work on most titles.

Drag and drop game files that are zlib compressed on the bat file

Things to test:

* Stability
* Can you cancel installation when xtool was used in setup
* If you have 16 threads or more, make xtool use 100% of your cpu to see if it can handle it
* if you have 32GB+ ram, set high chunk size, something like -c1gb and see if xtool was able to allocate the necessary memory
* If you have potato pc, don't worry I got a job for you :) check if there are no problems if you are using 1 thread both in encoding and decoding

Whatever you do, don't place preflate_dll.dll near the exe, not yet. else it will fail, the preflate code hasn't been added yet :D

Notes:

The file check must report
FC: no differences encountered
if the test went successfully, if it failed kindly upload the file it failed on.

This xtool uses slightly more memory than the 2019 version, I opted for stability than less memory usage

xtool now accepts maths expressions in command line, if you wanted it to use all cores minus 1, you can just write -t100p-1 or -t100-2, totally up to you

Next release/test focuses on:

+ preflate
+ reflate (reflate actually isn't replaced by preflate, it sometimes gives better output than preflate so I'll keep it just in case you are after best results)
+ depth setting

Hello Razor, here is my test (I did make a post before but I had to delete it since I used a reflate files by mistake):

My spec: Ryzen 7 2700 (8c/16t) + 16GB RAM.

Bat file editied for 128mb chunk size.

Here's is XTool using all 16 of my threads:

https://i.imgur.com/jGFDfgB.png

Testing on pak file from Astroneer:

Start size: 2.13GB

End Size: 3.75GB

No differences encountered ;)

Not entirely sure how I'd go about testing it in a setup...

tested on just cause2 arc files...
default setting nothing change "ProcProfile64.exe xtool.exe precomp -mzlib -c16mb -t100p-1 %1 %1.out"
system ryzen 3900x with 32gb
*is there arc.ini setting ?

Rayman Orgins
bundle_pc.pak (1.9 gb)

PS. tested on my 2nd pc (fx-4100, 2 cores, 4 threads).

here's preflate test, test on all zlib compressed games + dishonored and doom

tested on Dishonored2 with c16m and 512m and set -t100p-1...
work fine on ryzen 3900x and 32gb use all core and ram...

42.1gb to 57.2gb with c16mb
42.1gb to 59.8gb with c512mb

*used on all game file...

tested unpacking with inno
i get ERROR: archive data corrupted...

I must have messed up stdio mode during decoding but here is the fixed version.

now dec work super fast but i noticed it cpu usage around 50% to 70% of threads unlike compress use 90% to 100% when set to

unpackcmd = xtool.exe decode -t100p-1 - - <stdin> <stdout>

disk write using ssd nvme mp510 write after burst 400 to 500 mb/s...
nvme bottleneck ?
need ps5 ssd to get the max :D

you noticed how much memory xtool uses when compressing compared to when decompressing? that's why. xtool uses less memory when decoding and because of this it can't use up the entire cpu because there are times in between where there is nothing to process from memory. I haven't implemented faster code but in future I will.

I talked to you about preflate quite some time ago :D

glad to see you have made something out of it, will be returning to testing as soon as i am back home!!


have produced several videos on youtube on advanced compression past weeks.

talking to me about it isn't the same as implementing it ;)

@everyone

here's the xtool with reflate support.

I'll once again explain what's the difference. There are 3 deflate precompressors in xtool: zlib, reflate and preflate.

The old xtool/ztool/pzlib all had zlib+reflate but they had so many errors so I added preflate as an alternative if you have issues when you use reflate.

There are other things to this of course. I added all 3 because sometimes one has an advantage such as speed or better size output. Which is up to you to pick.

reflate cannot be used at the same time as preflate or the other way round.

but zlib can be used with reflate or preflate (pick one) like this -mzlib+reflate or -mzlib+preflate

here are my results on games that use the "normal" deflate configuration

using -mzlib:
using -mreflate:
using -mpreflate

Razor,
it would be nice if you could update the "Game File Scanner" with the new xtool. ;)

Dishonored. Death of the outsider (yes, again)

game1.resources
game1_001.resources
game1_patch.resources

-mpreflate

Unpack

I actually have a better idea :rolleyes:

try with reflate

sounds great :D

-mreflate

Test on DOOM - later..

DOOM 2016

snap_gameresources.resources
gameresources.resources

-mxtool 2019 (R3)

-mreflate

Decompress

Unpack

-mpreflate

If you have Saints Row The Third Remastered, can you run this test for me.

Thanks.