XTool 2020 (Main Project) - Page 36

dixen · #1 06-09-2022, 05:26

XTool v 0.6.4

Code:

[0] Processed lzo2a stream at 00000000199339FB (6669 >> 32768 >> 6669) using v999 successfully
[0] Processed lzo2a stream at 000000001993540C (6953 >> 32768 >> 6953) using v999 successfully
[0] Processed lzo2a stream at 0000000019936F39 (6965 >> 32768 >> 6965) using v999 successfully
[0] Processed lzo2a stream at 0000000019938A72 (7049 >> 32768 >> 7049) using v999 successfully
[0] Processed lzo2a stream at 000000001993A5FF (7357 >> 32768 >> 7357) using v999 successfully
[0] Processed lzo2a stream at 000000001993C2C0 (6483 >> 32768 >> 6483) using v999 successfully
[0] Processed lzo2a stream at 000000001993DC17 (6438 >> 32768 >> 6438) using v999 successfully
[0] Processed lzo2a stream at 000000001993F541 (7049 >> 32768 >> 7049) using v999 successfully
[0] Processed lzo2a stream at 00000000199410CE (10483 >> 32768 >> 10483) using v999 successfully
[0] Processed lzo2a stream at 00000000199439C5 (9063 >> 32768 >> 9063) using v999 successfully
[0] Processed lzo2a stream at 0000000019945D30 (11212 >> 32768 >> 11212) using v999 successfully
[0] Processed lzo2a stream at 0000000019948900 (13262 >> 32768 >> 13262) using v999 successfully
[0] Processed lzo2a stream at 000000001994BCD2 (19921 >> 32768 >> 19921) using v999 successfully
[0] Processed lzo2a stream at 0000000019950AA7 (14134 >> 32768 >> 14134) using v999 successfully
[0] Processed lzo2a stream at 00000000199541E1 (15579 >> 26778 >> 15579) using v999 successfully

Streams: 22550/22550
Time: 00:00:41 (00:00:38)
Memory: 264 MB (264 MB)


Errorlevel=0

Compressed 1 file, 429,260,800 => 829,963,641 bytes. Ratio 193.35%
Compression time: cpu 0.33 sec/real 42.34 sec = 1%. Speed 10.14 mB/s
All OK

Razor12911 · #2 09-09-2022, 00:19

Update available

Changes

- updated oodle scanner
- remove xdelta support from oodle and lzo codecs (crc mismatch often generates large diff files)

Notes

I've been getting reports of xtool taking a long time to process oodle streams (or getting stuck) so I reworked the scanner. I reverse engineered the code of oo2rec (I lost the source code) and rewrote parts of the code so xtool should produce better results while being faster.

There is a parameter rework as well, some people may know of the "n#" parameter which when set increases the chances of xtool capturing a stream. The default value now is 32, can be increased to 64, 128... Up to you, there is no limit just keep in mind that increasing this value also means longer precompression times.

Results on DOOM Eternal's "gameresources_4_1.streamdb"

oo2reck

Code:

Compressed 1 file, 1,353,405,353 => 2,584,084,262 bytes. Ratio 190.93%
Compression time: cpu 1.86 sec/real 590.20 sec = 0%. Speed 2.29 mB/s

xtool 0.4.7-0.6.4
Rumour has it that it's still processing as we speak (stuck on 5.7%)

xtool 0.6.5 (-mkraken:l6:n128)

Code:

Compressed 1 file, 1,353,405,353 => 2,589,860,614 bytes. Ratio 191.36%
Compression time: cpu 1.80 sec/real 370.32 sec = 0%. Speed 3.65 mB/s

panker1992 · #3 10-09-2022, 03:29

Quote:

Originally Posted by Razor12911

Update available

Changes

- updated oodle scanner
- remove xdelta support from oodle and lzo codecs (crc mismatch often generates large diff files)

Notes

I've been getting reports of xtool taking a long time to process oodle streams (or getting stuck) so I reworked the scanner. I reverse engineered the code of oo2rec (I lost the source code) and rewrote parts of the code so xtool should produce better results while being faster.

There is a parameter rework as well, some people may know of the "n#" parameter which when set increases the chances of xtool capturing a stream. The default value now is 32, can be increased to 64, 128... Up to you, there is no limit just keep in mind that increasing this value also means longer precompression times.

Results on DOOM Eternal's "gameresources_4_1.streamdb"

oo2reck

Code:

Compressed 1 file, 1,353,405,353 => 2,584,084,262 bytes. Ratio 190.93%
Compression time: cpu 1.86 sec/real 590.20 sec = 0%. Speed 2.29 mB/s

xtool 0.4.7-0.6.4
Rumour has it that it's still processing as we speak (stuck on 5.7%)

xtool 0.6.5 (-mkraken:l6:n128)

Code:

Compressed 1 file, 1,353,405,353 => 2,589,860,614 bytes. Ratio 191.36%
Compression time: cpu 1.80 sec/real 370.32 sec = 0%. Speed 3.65 mB/s

it's fixed :P , adding one big file and all libraries inside along with all versions of xtool isnt a good idea

kuyhaa · #4 15-09-2022, 08:07

confiused . .

xt 065 , ( same result with other version )

shazzla · #5 28-09-2022, 10:20

Hi guys!

I need a little help....
Something is wrong here. Tried to precomp a file with deduplication ,but XTool 0.6.5 doesnt make it. It looks like it ignores the corresponding parameter,even .bin file not created! V0.3.1 works fine!
What am i do wrong?

[External compressor:x065-dedup]
header = 0
packcmd = xtool065\xtool.exe precomp -mzlib -c32mb -t100p --dbase --dedup=chor065.bin - - <stdin> <stdout>

[External compressor:x065-nodedup]
header = 0
packcmd = xtool065\xtool.exe precomp -mzlib -c32mb -t100p - - <stdin> <stdout>

[External compressor:x031-dedup]
header = 0
packcmd = xtool031\xtool.exe precomp -mzlib -c32mb -t100p --dbase --dedup=chor031.bin - - <stdin> <stdout>

[External compressor:x031-nodedup]
header = 0
packcmd = xtool031\xtool.exe precomp -mzlib -c32mb -t100p - - <stdin> <stdout>

Results :

Xtool 0.3.1 ,deduplicated : 40.2 GB
Xtool 0.3.1 ,not deduplicated : 42.9 GB

Xtool 0.6.5 ,deduplicated : 42.9 GB
Xtool 0.6.5 ,not deduplicated : 42.9 GB

All processes finished without error. V0.6.5 is the "pure" version ,directly from Zee's ZIP-file...

Btw where to insert --verbose command?
Only one place is accepted in my experiments ,but no verbose infos are displayed...

Masquerade · #6 28-09-2022, 10:30

Quote:

Originally Posted by shazzla

Btw where to insert --verbose command?
Only one place is accepted in my experiments ,but no verbose infos are displayed...

Only use --verbose for testing. It limits XTool to 1 chunk processed at once. Add it before input / output. You shouldn't combine stdin/stdout with --verbose, use $$arcdatafile$$.tmp/$$arcpackedfile$$.tmp instead.

KaktoR · #7 28-09-2022, 10:55

You can make a batch file near xtool.exe and drag&drop a file onto it. This way you don't use freearc for it.

Example

Code:

@echo off
cd /D "%~dp0"
xtool.exe precomp -mzlib -c16mb -t100p --verbose - - < %1 > %1.out
pause

shazzla · #8 03-10-2022, 09:29

Nobody ?

Found. Damn...
Parameters were changed...

L33THAK0R · #9 14-10-2022, 08:18

Quick question, or I guess just asking for some clarification, in regards to "bms2xtl", can you only process a given file with your generated database, if the codec is one that xtool supports? So for example xtool wouldn't be able to process a COMTYPE if it isn't deflate, lz4(hc/x/f)... etc.? I guess what I'm asking in essence is am I correct in saying external, quickbms-native COMTYPEs aren't supported.

Masquerade · #10 14-10-2022, 11:17

Quote:

Originally Posted by L33THAK0R

Quick question, or I guess just asking for some clarification, in regards to "bms2xtl", can you only process a given file with your generated database, if the codec is one that xtool supports? So for example xtool wouldn't be able to process a COMTYPE if it isn't deflate, lz4(hc/x/f)... etc.? I guess what I'm asking in essence is am I correct in saying external, quickbms-native COMTYPEs aren't supported.

Use xtool.ini to add COMTYPEs that QuickBMS supports. Bear in mind that QuickBMS built in compression can vary wildly from that of what the original input file is compressed with so this is really only suitable for encryption algorithms where you're more likely to get a correct output.

Also, not every algorithm that QBMS can decompress it can also compress - such as LZ2K, of which no compression code exists publicly.

You also have totally different implementations of algos althogether such as LZSS where it's basically different every time you see it.

In short xtool.ini compression with QuickBMS is hit/miss.

shazzla · #11 15-10-2022, 04:59

Hi!

How can i pass the --dedup parameter into the -m chain ?

Masquerade · #12 15-10-2022, 05:31

Quote:

Originally Posted by shazzla

Hi!

How can i pass the --dedup parameter into the -m chain ?

Just put --dedup into XTool packcmd in arc.ini.

shazzla · #13 15-10-2022, 06:40

Currently using that method. Is it possible to pass --dedup ?!

dixen · #14 15-10-2022, 07:24

Quote:

Originally Posted by shazzla

Currently using that method. Is it possible to pass --dedup ?!

Just delete --dedup

shazzla · #15 15-10-2022, 07:37

Delete ?!
Why ?
Not really understand....

Edit:
I mean pass --dedup parameter to chain. I dont want a new arc.ini section.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
[Dev]XTool	Razor12911	Conversion Tutorials	180	23-10-2020 06:26
Project Cars Digital Edition (3xDVD5) (srep+lzma)	GTX590	PC Games - CD/DVD Conversions	10	28-08-2017 08:34
Project IGI Anthology 1xCD700 CIUV2 2039	mausschieber	PC Games - CD/DVD Conversions	0	24-07-2017 15:12
Space Channel 5 Part 2 Translation Project	Christuserloeser	DC Games	0	21-06-2004 18:16

The Following 5 Users Say Thank You to dixen For This Useful Post:
Gehrman (06-09-2022), L0v3craft (06-09-2022), Razor12911 (09-09-2022), ScOOt3r (06-09-2022), Wanterlude (06-09-2022)

The Following 11 Users Say Thank You to Razor12911 For This Useful Post:
BKR-TN (10-09-2022), Cesar82 (09-09-2022), elit (24-09-2022), Gehrman (09-09-2022), KaktoR (09-09-2022), L33THAK0R (09-09-2022), Lord.Freddy (09-09-2022), Masquerade (09-09-2022), ScOOt3r (09-09-2022), shazzla (09-09-2022), Wanterlude (09-09-2022)

The Following User Says Thank You to shazzla For This Useful Post:
ScOOt3r (24-06-2023)

The Following User Says Thank You to Masquerade For This Useful Post:
shazzla (28-09-2022)

The Following 2 Users Say Thank You to KaktoR For This Useful Post:
shazzla (28-09-2022), Wanterlude (28-09-2022)

The Following User Says Thank You to Masquerade For This Useful Post:
L33THAK0R (15-10-2022)