XTool 2020 (Main Project)

[Razor12911]

Originally Posted by elit

Thanks, so on more technical ground what is to be expected going forward regarding this? Specifically why is old reflate able to work without issues and mzlib not, is it matter of time/rework or is this a implementation limitation that is to be expected from mzlib, as compare to reflate and will not change in future?

there are several implementations of the deflate algorithm, it's very flexible. Think of using lzma to compress something, obviously there are compression levels that it comes with 1-9. -mzlib implementation is based around this idea that a user who uses this would have used a common setting such as compression level so for precompression it will decompress the data and then try to recompress it using these common levels, but of course lzma has methods that people use for other purposes like for stronger compression such as "lzma:ultra:a1:mfbt4:d158m: fb273:mc1000000000:lc8" mzlib cannot process such, similarly to deflate streams in png images as the way deflate set in such a way that it's good compressing pixels of an image but also make it quicker for loading/viewing. Reflate and preflate have no problem processing these as they used an advanced method for precompressing them.

You might say that mzlib is therefore useless, actually it still has its uses. mzlib is faster than both reflate and preflate because of its simple implementation and incurring no overhead furthermore mzlib produces better results because there is no additional header information file produced hence why I've preached several times to mix both zlib with reflate or preflate, if zlib fails to process the stream it just passes the stream to reflate/preflate for it to process it.

mzlib:
Compressed 1 file, 86,347,072 => 444,743,922 bytes. Ratio 515.07%
Compression time: cpu 0.16 sec/real 13.37 sec = 1%. Speed 6.46 mB/s

mreflate,l9:
Compressed 1 file, 86,347,072 => 445,179,589 bytes. Ratio 515.57%
Compression time: cpu 0.14 sec/real 14.33 sec = 1%. Speed 6.03 mB/s

mpreflate:
Compressed 1 file, 86,347,072 => 366,023,129 bytes. Ratio 423.90%
Compression time: cpu 0.14 sec/real 10.25 sec = 1%. Speed 8.42 mB/s

mzlib+reflate,l9:
Compressed 1 file, 86,347,072 => 444,941,384 bytes. Ratio 515.29%
Compression time: cpu 0.06 sec/real 11.74 sec = 1%. Speed 7.35 mB/s

mzlib+preflate:
Compressed 1 file, 86,347,072 => 444,744,743 bytes. Ratio 515.07%
Compression time: cpu 0.11 sec/real 11.23 sec = 1%. Speed 7.69 mB/s

As you can see, zlib perform faster when alone but better when you hook it up with reflate/preflate and preflate fails to process some streams. I've given people options, all up to them how they use xtool.

Originally Posted by elit

I cannot inflate zst(zstd) file from Paper Mario Origami King with xtool. With zstd.exe it does unpack just fine:

Attachment 31395

I was so desperate that I actually downloaded and compiled ALL available zstd versions from github! I am attaching those dll's, zstd.exe and a .zst file itself:
Attachment 31394

PS(could it be due to window size - 48k ?)

use --verbose mode to see which levels are producing the closest result to the original stream.

[0] Processing zstd stream at 0000000000000000 (14692 >> 48079 >> 17101) using l1 has failed
[0] Processing zstd stream at 0000000000000000 (14692 >> 48079 >> 16535) using l2 has failed
[0] Processing zstd stream at 0000000000000000 (14692 >> 48079 >> 16351) using l3 has failed
[0] Processing zstd stream at 0000000000000000 (14692 >> 48079 >> 15784) using l4 has failed
[0] Processing zstd stream at 0000000000000000 (14692 >> 48079 >> 15711) using l5 has failed
[0] Processing zstd stream at 0000000000000000 (14692 >> 48079 >> 15493) using l6 has failed
[0] Processing zstd stream at 0000000000000000 (14692 >> 48079 >> 15462) using l7 has failed
[0] Processing zstd stream at 0000000000000000 (14692 >> 48079 >> 15462) using l8 has failed
[0] Processing zstd stream at 0000000000000000 (14692 >> 48079 >> 15444) using l9 has failed
[0] Processing zstd stream at 0000000000000000 (14692 >> 48079 >> 15442) using l10 has failed
[0] Processing zstd stream at 0000000000000000 (14692 >> 48079 >> 15438) using l11 has failed
[0] Processing zstd stream at 0000000000000000 (14692 >> 48079 >> 15437) using l12 has failed
[0] Processing zstd stream at 0000000000000000 (14692 >> 48079 >> 15428) using l13 has failed
[0] Processing zstd stream at 0000000000000000 (14692 >> 48079 >> 14817) using l14 has failed
[0] Processing zstd stream at 0000000000000000 (14692 >> 48079 >> 14733) using l15 has failed
[0] Processing zstd stream at 0000000000000000 (14692 >> 48079 >> 14689) using l16 has failed
[0] Processing zstd stream at 0000000000000000 (14692 >> 48079 >> 14688) using l17 has failed
[0] Processing zstd stream at 0000000000000000 (14692 >> 48079 >> 14688) using l18 has failed
[0] Processing zstd stream at 0000000000000000 (14692 >> 48079 >> 14688) using l19 has failed
[0] Processing zstd stream at 0000000000000000 (14692 >> 48079 >> 14688) using l20 has failed
[0] Processing zstd stream at 0000000000000000 (14692 >> 48079 >> 14688) using l21 has failed
[0] Processing zstd stream at 0000000000000000 (14692 >> 48079 >> 14688) using l22 has failed

level 17 is close but level 19 is commonly used, then set -mzstd,l19

[0] Processing zstd stream at 0000000000000000 (14692 >> 48079 >> 14688) using l19 has failed
[0] - Patched stream at 0000000000000000 (14692 >> 14688) [28] successfully

Compressed 1 file, 14,692 => 48,239 bytes. Ratio 328.34%
Compression time: cpu 0.00 sec/real 0.55 sec = 0%. Speed 0.03 mB/s

Used libzstd130.dll