XTool 2020 (Main Project)

[Edison007]

Originally Posted by elit

I would suggest crc64 or sha128(or even better VMAC that srep use).

better blake2)

[Masquerade]

Not blake3? ^

[elit]

So I tried xtool on older Telltale game "Back to the future"(gog version) as I "just happen" to be packing it right now. I thought game files should be decrypted first but hey! It inflated them! Specifically I tried on Ep1 file "4_BackToTheFuture101_pc_tx.ttarch" which is 244mb:
1.png

^First I tried -mzlib, it fould 10113 streams and inflated to 632mb in 22s global time(I should have marked that one, ignore red underline on wrong time).

2.png
^Reflate found same number of streams but inflated them to 638mb, which may or may not be actual data(could be overhead which would make it worse than -mzlib). Time was worse at 35s.

3.png
^Preflate processed 3 less streams from all and inflated to 632mb which is same as zlib, but time close to reflate(33s).

4.png
^GrittiBanzli. Now this funny name found all the same number of streams as zlib and reflate, but inflated them to 728mb?! WTF? Unfortunately time was horrible at 440sec. Not sure why same number of actually processed(not just found) streams give such a difference in size. I tried more brute options(including depth) on xtool zlib & reflate and even on ztool but they could not get above 638mb no matter what. This thing can inflate about ~20% more. I wonder how would precomp do here...

5.png
^Finally, good ol' ztool for reference, better inflation size than xtool and same as reflate, but again it could be tool overhead itself not actual streams - which would mean its worse. Time is on par with xtool -mzlib.


There, don't say I never contributed .

[Masquerade]

Just another Oodle test:

Death Stranding: Kraken (oo2core_7_win.dll)

Testing On: 968bf82f34e2b499687c901a888e633a.bin

-mdst+oo2reck:

Compressed 1 file, 550,028,020 => 1,216,560,352 bytes. Ratio 221.18%
Compression time: cpu 1.11 sec/real 80.93 sec = 1%. Speed 6.80 mB/s
All OK

-mdst+xtool (no deduplication or database)

Compressed 1 file, 550,028,020 => 1,213,760,805 bytes. Ratio 220.67%
Compression time: cpu 0.98 sec/real 62.03 sec = 2%. Speed 8.87 mB/s
All OK

[Sebazz]

Originally Posted by Masquerade

Just another Oodle test:

Death Stranding: Kraken (oo2core_7_win.dll)

Testing On: 968bf82f34e2b499687c901a888e633a.bin

-mdst+oo2reck:

Compressed 1 file, 550,028,020 => 1,216,560,352 bytes. Ratio 221.18%
Compression time: cpu 1.11 sec/real 80.93 sec = 1%. Speed 6.80 mB/s
All OK

-mdst+xtool (no deduplication or database)

Compressed 1 file, 550,028,020 => 1,213,760,805 bytes. Ratio 220.67%
Compression time: cpu 0.98 sec/real 62.03 sec = 2%. Speed 8.87 mB/s
All OK

What is mdst ? What settings you use for xtool oodle (kraken) ?

[dixen]

Originally Posted by Sebazz

What is mdst ? What settings you use for xtool oodle (kraken) ?

https://fileforums.com/showthread.php?t=103699

[Masquerade]

-m = freearc method parameter

dst = Death Stranding Decrypt Tool

xtool oodle= use -mkraken (-m feature in xtool, can't remember entire packcmd)

[Razor12911]

Update available

Changes

- added zstd codec
- added lz4, lz4hc, lzna, mermaid, selkie, hydra, leviathan codec placeholders
- added configuration support
- added xdelta support to handle crc mismatch streams

Notes

Configuration support is basically telling xtool how to find streams via an ini file. An example is included (-msr3remaster), I'll document its usage in the next release.
The oodle precompressor in xtool is still inferior to the side project, I'll work on it in the next release.

[KaktoR]

zstd does not seem to work ?

[Razor12911]

Fixed

[KaktoR]

Thanks. However, I had to use a different zstd library to make it work actually

Compressed 1 file, 50,664,187 => 104,234,410 bytes. Ratio 205.74%
Compression time: cpu 0.08 sec/real 2.19 sec = 4%. Speed 23.10 mB/s
All OK

Extracted 1 file, 104,234,410 => 50,664,187 bytes. Ratio 205.74%
Extraction time: cpu 0.05 sec/real 2.08 sec = 2%. Speed 24.37 mB/s
All OK

PS: I made a little compare with ZSTDRec (Side Project)

Compressed 1 file, 50,664,187 => 105,746,716 bytes. Ratio 208.72%
Compression time: cpu 0.08 sec/real 2.35 sec = 3%. Speed 21.52 mB/s
All OK

Extracted 1 file, 105,746,716 => 50,664,187 bytes. Ratio 208.72%
Extraction time: cpu 0.06 sec/real 1.56 sec = 4%. Speed 32.51 mB/s
All OK

[Sebazz]

Thanks ... This settings are correct ?

[External compressor:xzstd]
header    = 0
packcmd   = xtool precomp -mzstd -t100p-1 - - <stdin> <stdout>
unpackcmd = xtool decode -t100p-1 - - <stdin> <stdout>

[Razor12911]

yeah

[Sebazz]

i Have rare problem ... using this setting for precomp - - <stdin> <stdout> sometimes works, sometimes dont, and $$arcdatafile$$.tmp $$arcpackedfile$$.tmp works but stuck the compression ... whats wrong ?

[Razor12911]

try the same input without using Freearc and see if the error persists and if it does, send over the sample and I'll see what is causing the problem.