XTool 2020 (Main Project)

[dixen]

Originally Posted by KaktoR

Did they change compression? I'm sure in some older version of this game they used just zlib or sort of if I remember correctly.

No changes. ZLIB only in 2 *.pak files. RPACK is no compression

[panker1992]

Hello, it's been ages since i had an error !!

This Came directly from Xtool documentation
Xtool version used 6.2

this is my wav.ini
[Stream1]
Name=wav
Codec=wavpack
BigEndian=0
Signature=0x46464952
Structure=Signature(4),FileSize(4),FileType(4),Str eam
StreamOffset=-12
CompressedSize=FileSize + 8
DecompressedSize=0
Condition1=FileType = 0x45564157
Condition2=FileSize >= 4096

this is my xtool.ini
[wavpack]
Encode=wavpack.exe -hh -x4 <filein>.wav <fileout>.wv
Decode=wvunpack.exe <filein>.wv <fileout>.wav

i have been unable to make this even detect the sample FLAC internal codec works and so any other
NOTE: i use this as standalone and not in FA for testing purposes

[Edison007]

xtool.ini:

[wavpack]
Encode=wavpack.exe -hh -x4 <filein> <fileout>
Decode=wvunpack.exe <filein> <fileout>

or

[wavpack]
Encode=wavpack.exe -hh -x4 - - <stdin> <stdout>
Decode=wvunpack.exe - - <stdin> <stdout>

[panker1992]

Originally Posted by Edison007

xtool.ini:

[wavpack]
Encode=wavpack.exe -hh -x4 <filein> <fileout>
Decode=wvunpack.exe <filein> <fileout>

or

[wavpack]
Encode=wavpack.exe -hh -x4 - - <stdin> <stdout>
Decode=wvunpack.exe - - <stdin> <stdout>

i Spoke with Zee

it's a bug confirmed, it's been fixed already

[Razor12911]

Update available

Changes

- added universal lz4f scanner
- fixed issues with database feature
- fixed issues with executable plugin support
- updated lzo codecs

[Razor12911]

Update available

Changes

- fixed issues with lzo2a and lzo1c codecs

[dixen]

XTool v 0.6.4

[0] Processed lzo2a stream at 00000000199339FB (6669 >> 32768 >> 6669) using v999 successfully
[0] Processed lzo2a stream at 000000001993540C (6953 >> 32768 >> 6953) using v999 successfully
[0] Processed lzo2a stream at 0000000019936F39 (6965 >> 32768 >> 6965) using v999 successfully
[0] Processed lzo2a stream at 0000000019938A72 (7049 >> 32768 >> 7049) using v999 successfully
[0] Processed lzo2a stream at 000000001993A5FF (7357 >> 32768 >> 7357) using v999 successfully
[0] Processed lzo2a stream at 000000001993C2C0 (6483 >> 32768 >> 6483) using v999 successfully
[0] Processed lzo2a stream at 000000001993DC17 (6438 >> 32768 >> 6438) using v999 successfully
[0] Processed lzo2a stream at 000000001993F541 (7049 >> 32768 >> 7049) using v999 successfully
[0] Processed lzo2a stream at 00000000199410CE (10483 >> 32768 >> 10483) using v999 successfully
[0] Processed lzo2a stream at 00000000199439C5 (9063 >> 32768 >> 9063) using v999 successfully
[0] Processed lzo2a stream at 0000000019945D30 (11212 >> 32768 >> 11212) using v999 successfully
[0] Processed lzo2a stream at 0000000019948900 (13262 >> 32768 >> 13262) using v999 successfully
[0] Processed lzo2a stream at 000000001994BCD2 (19921 >> 32768 >> 19921) using v999 successfully
[0] Processed lzo2a stream at 0000000019950AA7 (14134 >> 32768 >> 14134) using v999 successfully
[0] Processed lzo2a stream at 00000000199541E1 (15579 >> 26778 >> 15579) using v999 successfully

Streams: 22550/22550
Time: 00:00:41 (00:00:38)
Memory: 264 MB (264 MB)


Errorlevel=0

Compressed 1 file, 429,260,800 => 829,963,641 bytes. Ratio 193.35%
Compression time: cpu 0.33 sec/real 42.34 sec = 1%. Speed 10.14 mB/s
All OK

[Razor12911]

Update available

Changes

- updated oodle scanner
- remove xdelta support from oodle and lzo codecs (crc mismatch often generates large diff files)

Notes

I've been getting reports of xtool taking a long time to process oodle streams (or getting stuck) so I reworked the scanner. I reverse engineered the code of oo2rec (I lost the source code) and rewrote parts of the code so xtool should produce better results while being faster.

There is a parameter rework as well, some people may know of the "n#" parameter which when set increases the chances of xtool capturing a stream. The default value now is 32, can be increased to 64, 128... Up to you, there is no limit just keep in mind that increasing this value also means longer precompression times.

Results on DOOM Eternal's "gameresources_4_1.streamdb"

oo2reck

Compressed 1 file, 1,353,405,353 => 2,584,084,262 bytes. Ratio 190.93%
Compression time: cpu 1.86 sec/real 590.20 sec = 0%. Speed 2.29 mB/s

xtool 0.4.7-0.6.4
Rumour has it that it's still processing as we speak (stuck on 5.7%)

xtool 0.6.5 (-mkraken:l6:n128)

Compressed 1 file, 1,353,405,353 => 2,589,860,614 bytes. Ratio 191.36%
Compression time: cpu 1.80 sec/real 370.32 sec = 0%. Speed 3.65 mB/s

[panker1992]

Originally Posted by Razor12911

Update available

Changes

- updated oodle scanner
- remove xdelta support from oodle and lzo codecs (crc mismatch often generates large diff files)

Notes

I've been getting reports of xtool taking a long time to process oodle streams (or getting stuck) so I reworked the scanner. I reverse engineered the code of oo2rec (I lost the source code) and rewrote parts of the code so xtool should produce better results while being faster.

There is a parameter rework as well, some people may know of the "n#" parameter which when set increases the chances of xtool capturing a stream. The default value now is 32, can be increased to 64, 128... Up to you, there is no limit just keep in mind that increasing this value also means longer precompression times.

Results on DOOM Eternal's "gameresources_4_1.streamdb"

oo2reck

Compressed 1 file, 1,353,405,353 => 2,584,084,262 bytes. Ratio 190.93%
Compression time: cpu 1.86 sec/real 590.20 sec = 0%. Speed 2.29 mB/s

xtool 0.4.7-0.6.4
Rumour has it that it's still processing as we speak (stuck on 5.7%)

xtool 0.6.5 (-mkraken:l6:n128)

Compressed 1 file, 1,353,405,353 => 2,589,860,614 bytes. Ratio 191.36%
Compression time: cpu 1.80 sec/real 370.32 sec = 0%. Speed 3.65 mB/s

it's fixed :P , adding one big file and all libraries inside along with all versions of xtool isnt a good idea

[kuyhaa]

confiused . .

xt 065 , ( same result with other version )

[shazzla]

Hi guys!

I need a little help....
Something is wrong here. Tried to precomp a file with deduplication ,but XTool 0.6.5 doesnt make it. It looks like it ignores the corresponding parameter,even .bin file not created! V0.3.1 works fine!
What am i do wrong?


[External compressor:x065-dedup]
header = 0
packcmd = xtool065\xtool.exe precomp -mzlib -c32mb -t100p --dbase --dedup=chor065.bin - - <stdin> <stdout>

[External compressor:x065-nodedup]
header = 0
packcmd = xtool065\xtool.exe precomp -mzlib -c32mb -t100p - - <stdin> <stdout>

[External compressor:x031-dedup]
header = 0
packcmd = xtool031\xtool.exe precomp -mzlib -c32mb -t100p --dbase --dedup=chor031.bin - - <stdin> <stdout>

[External compressor:x031-nodedup]
header = 0
packcmd = xtool031\xtool.exe precomp -mzlib -c32mb -t100p - - <stdin> <stdout>

Results :

Xtool 0.3.1 ,deduplicated : 40.2 GB
Xtool 0.3.1 ,not deduplicated : 42.9 GB

Xtool 0.6.5 ,deduplicated : 42.9 GB
Xtool 0.6.5 ,not deduplicated : 42.9 GB

All processes finished without error. V0.6.5 is the "pure" version ,directly from Zee's ZIP-file...

Btw where to insert --verbose command?
Only one place is accepted in my experiments ,but no verbose infos are displayed...

[Masquerade]

Originally Posted by shazzla

Btw where to insert --verbose command?
Only one place is accepted in my experiments ,but no verbose infos are displayed...

Only use --verbose for testing. It limits XTool to 1 chunk processed at once. Add it before input / output. You shouldn't combine stdin/stdout with --verbose, use $$arcdatafile$$.tmp/$$arcpackedfile$$.tmp instead.

[KaktoR]

You can make a batch file near xtool.exe and drag&drop a file onto it. This way you don't use freearc for it.

Example

@echo off
cd /D "%~dp0"
xtool.exe precomp -mzlib -c16mb -t100p --verbose - - < %1 > %1.out
pause

[shazzla]

Nobody ?

Found. Damn...
Parameters were changed...

[L33THAK0R]

Quick question, or I guess just asking for some clarification, in regards to "bms2xtl", can you only process a given file with your generated database, if the codec is one that xtool supports? So for example xtool wouldn't be able to process a COMTYPE if it isn't deflate, lz4(hc/x/f)... etc.? I guess what I'm asking in essence is am I correct in saying external, quickbms-native COMTYPEs aren't supported.