XTool - Successor of ZTool

[shazzla]

Similar happens on Agony's .PAK files(zlib,v0.7, v0.6 works good)
First file processed well,but the second one looks like simply copied.
Cant upload files because they are more than 10 gigs. :/

[elit]

Originally Posted by doofoo24

i think it may be better for Razor12911 if he develop the tool separately like pzlib/plz4/pzstd/plzo for games...
less of a headache

How exactly is that less headache? This way at least he can re-use a lot of same code/routines for each compressor instead of duplicating and maintaining separate projects with a lot of same code.

[Razor12911]

Update available

Changes

- fixed crilayla bug on naruto games
- fixed issue detecting lzo2a, lzo1c streams

@doofoo24
I have to agree with elit, It's more headache separating the codecs. An example would be ztool itself, normally when I make changes to pzlib, if it's not related to precompression itself, I had to make the same changes in plz4, plzo... sometimes I forget doing this and end up with different sources of each, in one, a bug was fixed then another bug appears in another, now there are bugs all over the codecs, different bugs, you end up trying to fix one and another pops up and so forth. The idea of putting everything together means one code, if one codec is affected, then all will be affected then I'll know the source of the bug.

Originally Posted by Edison007

compr:

afr:v0+srep:m5f:l512:d512m:hash-+lolz:d64m - 37,6 MB (39 463 094 bytes)
xtool+srep:m5f:l512:d512m:hash-+lolz:d64m  - 37,8 MB (39 671 049 bytes)

dec time/speed:

cls-afr, v019 (x86)
t1 - 199.28 sec; ~  5.37 mB/s
t7 -  44.07 sec; ~ 24.29 mB/s
t8 -  42.89 sec; ~ 24.96 mB/s

xtool_x86, v07
t1 - 237.07 sec; ~  4.52 mB/s
t7 -  53.84 sec; ~ 19.88 mB/s
t8 -  51.19 sec; ~ 20.91 mB/s

xtool_x64, v07
t1 - 210.85 sec; ~  5.08 mB/s
t7 -  49.35 sec; ~ 21.69 mB/s
t8 -  46.91 sec; ~ 22.82 mB/s

1'070'530'560 ->   afr: 1'767'925'585 bytes
              -> xtool: 1'768,021'857 bytes

i7-4700MQ, 8gb ram, ram-disk, win7x64.

I wonder if there is still room for improvement.

[Razor12911]

Originally Posted by doofoo24

@Razor12911 xtool 0.7 seems to skip file's test it on ME A with zstd and before with lzo on ac 1...
xtool 0.6 work fine...
any idea ?
i will test with 0.8...

I didn't touch zstd codec in between the updates

ztool_08

Process ID       : 12240
Thread ID        : 19080
Process Exit Code: 0
Thread Exit Code : 0

User Time        :          24.500s
Kernel Time      :           0.734s
Process Time     :          25.234s
Clock Time       :           7.764s

Working Set      :          193016 KB
Paged Pool       :             114 KB
Nonpaged Pool    :              13 KB
Pagefile         :          314972 KB
Page Fault Count : 311084

IO Read          :           43332 KB (in             683 reads )
IO Write         :          101353 KB (in            1584 writes)
IO Other         :               3 KB (in             178 others)

ztool_06

Process ID       : 12748
Thread ID        : 14488
Process Exit Code: 0
Thread Exit Code : 0

User Time        :          24.890s
Kernel Time      :           1.125s
Process Time     :          26.015s
Clock Time       :           8.160s

Working Set      :          177480 KB
Paged Pool       :             115 KB
Nonpaged Pool    :              12 KB
Pagefile         :          229352 KB
Page Fault Count : 316978

IO Read          :           43262 KB (in             677 reads )
IO Write         :          101353 KB (in            1584 writes)
IO Other         :               5 KB (in             270 others)

[pakrat2k2]

ALWAYS choose disable smilies, when posting/ replying ( checkbox below message box )

pain to have to come in & clean up posts & quotes.

[Edison007]

Not sure in terms of results here

It's normal, because i using differents decode methods in cls and exe. In cls i use some tricks, which allow me to get a higher speed. exe-decoder, yes, slower.

I'll check in terms of the number of streams found by xtool vs afr

I think, that numbers of streams found is the same. But maybe in AFR the output container is more successfully made

I wonder if there is still room for improvement.

Yep, but I'm not interested in this engine anymore.

[KaktoR]

Originally Posted by Edison007

Yep, but I'm not interested in this engine anymore.

So, what's your next clue?

[Razor12911]

Originally Posted by doofoo24

test xtool precomp:zlib with 0.6 vs 0.8 on just cause 2

and... which is which?

[Simorq]

dataRegion.cpk 607 MB> 1.59 GB

FreeArc 0.67 (March 15 2014) creating archive: data.arc
Compressed 2 files, 636,697,912 => 1,711,252,663 bytes. Ratio 268.77%
Compression time: cpu 0.89 sec/real 141.42 sec = 1%. Speed 4.50 mB/s
All OK

FreeArc 0.67 (March 15 2014) testing archive: data.arc
Tested 2 files, 1,711,252,663 => 636,697,912 bytes. Ratio 268.77%
Testing time: cpu 0.80 sec/real 118.83 sec = 1%. Speed 5.36 mB/s
All OK

[Sergey3695]

NARUTO SHIPPUDEN Ultimate Ninja STORM 2
FreeArc 0.67 unpacker. Extracting archive: data.arc
Extracting dataRegion.cpk (716396224 bytes)
All OK
129.547s: unarc x -o+ -w.\ -dp_TEST data.arc

[Razor12911]

Update available

Changes

- fixed 2gb+ input issue... (int32 instead of int64 )

xtl06:
Compressed 46 files, 13,586,655,989 => 22,419,187,764 bytes. Ratio 165.01%

xtl07-08:
Compressed 46 files, 13,586,655,989 => 15,710,322,921 bytes. Ratio 115.63%

xtl09:
Compressed 46 files, 13,586,655,989 => 22,419,187,764 bytes. Ratio 165.01%

[oltjon]

[KaktoR]

crilayla only works with 32 bit

[oltjon]

xtool 32bit

[KaktoR]

zstd and oodle are 64bit only. There is no 32bit for these