XTool 2020 (Main Project)

[daveyrob]

i had the same Disk Full issue, remove libzstd.dll and it will work.

[ScOOt3r]

well i got it working compressing and decompression as someone posted above to remove libzstd.dll.. i did that and it works just fine now.. weird that it was that file
not sure how important libzstd.dll is...

ScOOt3r

[seryogakms]

Originally Posted by ScOOt3r

well i got it working compressing and decompression as someone posted above to remove libzstd.dll.. i did that and it works just fine now.. weird that it was that file
not sure how important libzstd.dll is...

ScOOt3r

arc.exe a -ep1 -r -ed -s; -i2 -w.\temp -mxt22+srep+lolz Packs\data.bin "DATAS\*"

[External compressor:xt22_zstd]
header = 0
packcmd = xtool\zlib\xt22.exe precomp -mzlib+zstd:l19,l22 -c32mb -t100p $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = xt22.exe decode -t100p - - <stdin> <stdout>

[External compressor:lolz]
header = 0
packcmd = lolz_x64.exe -d128 -mc1023 -dtb1 -tt4 -mtt1 -mt8 -mtb128 -fba1024 -oh14 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = lolz_x86.exe $$arcpackedfile$$.tmp $$arcdatafile$$.tmp

[External compressor:srep]
header = 0
packcmd = srep64 -mem512m -l512 -m5f $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = srep -d $$arcpackedfile$$.tmp $$arcdatafile$$.tmp

[:( Sad8669]

ELDEN RING

Oodlerec Used

Compressing 1 file, 2,655,357,776 bytes
  Compressing Data3.bdt
Compressed 1 file, 2,655,357,776 => 4,549,448,231 bytes. Ratio 171.33%
Compression time: cpu 3.81 sec/real 615.49 sec = 1%. Speed 4.31 mB/s
All OK

From Masquerade.
use oodle core from Martha is Dead

[KaktoR]

From a 100mb sample data0.bdt

xtool:kraken
Compressed 1 file, 104,857,600 => 213,800,403 bytes. Ratio 203.90%
Compression time: cpu 0.09 sec/real 15.75 sec = 1%. Speed 6.66 mB/s

xtool:kraken,l6
Compressed 1 file, 104,857,600 => 214,235,226 bytes. Ratio 204.31%
Compression time: cpu 0.09 sec/real 10.01 sec = 1%. Speed 10.48 mB/s

[Razor12911]

Update available

Changes

- added verbose mode
- added feature that allows you to enforce a different library to be loaded
- fixed issues related to imperfect stream patching
- fixed issues with old libraries with missing functions that cause xtool to crash on startup
- updated oodle codec
- updated reflate codec
- updated zstd codec

Notes

You can now enforce xtool to use a specific library rather than renaming the library for it to picked up by xtool.

[External compressor:xtool]
header    = 0
packcmd   = xtool.exe precomp { -moption} -c32mb -t100p --zstd=libzstd148.dll - - <stdin> <stdout>
unpackcmd = xtool.exe decode -t100p - - --zstd=libzstd148.dll <stdin> <stdout>

Available parameters for this are:
--zlib=filename, --lz4=filename, --lzo=filename, --zstd=filename, --oodle=filename

You can also use this to run multiple tests to see which zstd or oodle library gives the best output by running these commands on a small sample:

xtool.exe precomp -mkraken -c32mb -t100p --oodle=oo2core_7_win64.dll - - < %1 > %1.out1
xtool.exe precomp -mkraken -c32mb -t100p --oodle=oo2core_7_win64_2.dll - - < %1 > %1.out2
xtool.exe precomp -mkraken -c32mb -t100p --oodle=oo2core_8_win64.dll - - < %1 > %1.out3
xtool.exe precomp -mkraken -c32mb -t100p --oodle=oo2core_fifa22.dll - - < %1 > %1.out4

Then you can pick whichever produces the best results

___

Oodle code has been updated, if you having issues with stream detection you can use the parameter n1 (kraken,n1), where the old code will be used. it can also be n2 or n3, depending on how many times each stream should be redetected (for improved results but at a cost of speed)
If this doesn't help then use the side project.

___

Verbose mode was added to give information about input, as to what parameters are used for streams within the input and if you see a common trend then you directly use these parameters to speed up xtool and also give it the opportunity to patch streams if there are any invalid streams.

Verbose mode will enforce xtool to 1 thread thus it should not be used by default as it will slow now precompression.

[KaktoR]

How to use verbose mode

@echo off
xtool.exe precomp -mcodec -c32mb -t100p --zlib=filename, lz4=filename, lzo=filename, zstd=filename, oodle=filename --verbose - - < %1 > %1.out1
pause
xtool.exe decode -t100p - - < %1.out > %1.res
del /q %1.out
fc /b %1 %1.res
del /q %1.res
pause

Drag&Drop a file on batch

[:( Sad8669]

ELDEN RING - XTool Update Test (Data3.bdt)

Old XTool - xtool:kraken,l6

Compressed 1 file, 2,655,357,776 => 4,668,458,345 bytes. Ratio 175.81%
Compression time: cpu 4.09 sec/real 673.02 sec = 1%. Speed 3.95 mB/s
All OK

New XTool- xtool:kraken,l6

Compressed 1 file, 2,655,357,776 => 4,551,696,149 bytes. Ratio 171.42%
Compression time: cpu 3.72 sec/real 564.25 sec = 1%. Speed 4.71 mB/s
All OK

New XTool - xtool:kraken,l6,n1

Compressed 1 file, 2,655,357,776 => 4,673,342,606 bytes. Ratio 176.00%
Compression time: cpu 4.59 sec/real 706.91 sec = 1%. Speed 3.76 mB/s
All OK

New XTool - xtool:kraken,l6,n2

Compressed 1 file, 2,655,357,776 => 4,673,136,986 bytes. Ratio 175.99%
Compression time: cpu 4.05 sec/real 672.64 sec = 1%. Speed 3.95 mB/s
All OK

Library Used : oo2core_4_win64.dll

Check previous page and download the library from there.

[Gehrman]

resident evil 4 HD project "*.lfs" Precompressor
https://github.com/emoose/re4-research

Arc.ini

[External compressor:re4-lfs]
header    = 0
solid=0
packcmd   = re4lfs.exe files.lfs files.pack
unpackcmd = re4lfs.exe files.pack files.lfs
datafile   = files.lfs
packedfile = files.pack

--------------------------------------------------------
But it does not work with XTool
XTool.ini

[re4lfs]
Encode=re4lfs.exe files.lfs files.pack
Decode=re4lfs.exe files.pack files.lfs
Continous=0

Is there a way to solve it?

[Razor12911]

can you send sample and the exe?

[Gehrman]

Originally Posted by Razor12911

can you send sample and the exe?

https://github.com/emoose/re4-resear...build-0.1a.zip

[Razor12911]

Originally Posted by Gehrman

https://github.com/emoose/re4-resear...build-0.1a.zip

Sorry for late response but here's an example.

in the generate folder, you'll get the example of how database would be created.

xtool.exe generate -mre4lfs "game\*" "game\*" lfs.xtl

You'll notice that the unpacked data and the original data is the same, that's because this tool accepts game files as they are and doesn't need to extract additional streams.

Note: Chunk size is should be the same size as the largest stream/file

then in precompress folder is where you need to transfer the generated database file from generate folder to precompress folder (lfs.xtl)

Look in xtool.ini to see how the program should be configured based on the information you provided.

[re4lfs]
Encode=re4lfs.exe <filein>.lfs <fileout>.pack
Decode=re4lfs.exe <filein>.pack <fileout>.lfs

Results of the samples you provided

Compressed 5 files, 2,853,066 => 6,511,766 bytes. Ratio 228.24%
Compression time: cpu 0.02 sec/real 1.06 sec = 1%. Speed 2.70 mB/s

[Razor12911]

Originally Posted by Razor12911

Note: Chunk size is should be the same size as the largest stream/file

generate -c##

[Gehrman]

Originally Posted by Razor12911

generate -c##

good work

@echo off
xtool.exe generate -mre4lfs -c512mb "game\*" "game\*" lfs.xtl 
pause

Thank you for this wonderful tool

[dixen]

Razor12911
Is it possible to make lz2k-plugin for XTool?