View Full Version : XTool - Successor of ZTool
Razor12911
29-05-2018, 17:31
Yea, that game seems to contain lzopro compressed streams and I only added x86 support for now. Maybe it's AC4, not sure.
Edit:
AC2 was affected due to me not registering what codec to use for restoring, guess it really is a waste of energy.
Your file:
Compressed 1 file, 88,997,888 => 189,610,506 bytes. Ratio 213.05%
Compression time: cpu 0.09 sec/real 2.72 sec = 3%. Speed 32.66 mB/s
Razor12911
29-05-2018, 17:47
sorry brother does not work :D
why bother afr work on all ac game's...
waste of your energy :p
really encouraging words
ZakirAhmad
29-05-2018, 20:06
@Razor: Thanks mate for the update.
Keep up the good work.
Xtool is going to be more than a precompressor, so not adding any game support diverts it from that path.
Sergey3695
30-05-2018, 04:34
- added partial crilayla support for naruto series
https://yadi.sk/d/WNKW73Ht3UpyVN
data.cpk (393 mb)
NARUTO SHIPPUDEN Ultimate Ninja STORM 2
[External compressor:xprecomp]
header = 0
packcmd = xtool.exe e:precomp:c32mb,t1:crilayla - - <stdin> <stdout>
unpackcmd = xtool.exe d:precomp:c32mb,t1:crilayla - - <stdin> <stdout>
Extracting archive: data.arc
Extracting data.cpk (413123988 bytes)
ERROR: file _TEST\data.cpk failed CRC check
:confused:
Hi Razor!
Did you test the sample I sent?
ZakirAhmad
30-05-2018, 08:52
xtool v 0.7 Tried it on a file from AC orgins
it didnt expanded.
sorry to bother oodle was missing from arc.ini setting.
this sorted the issue.
afr expaned it to 163.15% while xtool to 162.91% xtool
restored it in only 14sec while afr took 18 seconds in my system
Now my only problem is with Dunia engine. i wrote dat file address in xtool.ini still no change.
BTW is farcry primal and James Camerons avatar supported.
Yep, didn't expand with :lzo
Maybe wrong codec in arc.ini? lzo2.dll is in place
Edit: You have to use :oodle for Origins:rolleyes::D
DataPC.forge (ACO)
xtool v0.7 oodle: 273.088.512 Bytes => 574.853.120 Bytes
AFR: 273.088.512 Bytes => 574.849.024 Bytes
From this example xtool and AFR have pretty much the same compression speed (AFR is a little bit faster, but just a few secs)
Update available
In terms of the Dunia engine, I advise that you use the latest liblz4.dll to avoid some streams being skipped due to their major difference, this could be because even I myself am not sure if I used the correct parameters for recompression, if you have an idea of what is used, please comment. (I used lz4hc, level 9)
I dont know if you want to go this route, but the way I did my tests when developing lz4 packer for "Raiders of the Broken Planet" game was that I used both original and self-modified quickbms script for unpacking game archive first. I unpacked thus twice - for each.
In modified script only difference was that I disabled decompression by taking out "comtype" command and used "log" instead of "clog" to dump files raw & compressed.
So I had both compressed and decompressed individual files from archive. Then it was only matter of finding right/best cmp settings through hex editor and binary comparison(first on few files and then all) to get best setting. And even wrong(but close) settings were still compatible with 80%+ of all archive.
ShivShubh
30-05-2018, 20:42
I dont know if you want to go this route, but the way I did my tests when developing lz4 packer for "Raiders of the Broken Planet" game was that I used both original and self-modified quickbms script for unpacking game archive first. I unpacked thus twice - for each.
In modified script only difference was that I disabled decompression by taking out "comtype" command and used "log" instead of "clog" to dump files raw & compressed.
So I had both compressed and decompressed individual files from archive. Then it was only matter of finding right/best cmp settings through hex editor and binary comparison(first on few files and then all) to get best setting. And even wrong(but close) settings were still compatible with 80%+ of all archive.
So by saying all that what were you trying to say by this "I dont know if you want to go this route" ? That Razor should find the correct parameters ? I don't think he has that much of an interest in far cry 5 or wants to waste time doing all that.
Razor12911
30-05-2018, 21:28
https://yadi.sk/d/WNKW73Ht3UpyVN
data.cpk (393 mb)
NARUTO SHIPPUDEN Ultimate Ninja STORM 2
[External compressor:xprecomp]
header = 0
packcmd = xtool.exe e:precomp:c32mb,t1:crilayla - - <stdin> <stdout>
unpackcmd = xtool.exe d:precomp:c32mb,t1:crilayla - - <stdin> <stdout>
Extracting archive: data.arc
Extracting data.cpk (413123988 bytes)
ERROR: file _TEST\data.cpk failed CRC check
:confused:
i'll check :)
Edison007
31-05-2018, 03:20
compr:
afr:v0+srep:m5f:l512:d512m:hash-+lolz:d64m - 37,6 MB (39 463 094 bytes)
xtool+srep:m5f:l512:d512m:hash-+lolz:d64m - 37,8 MB (39 671 049 bytes)
dec time/speed:
cls-afr, v019 (x86)
t1 - 199.28 sec; ~ 5.37 mB/s
t7 - 44.07 sec; ~ 24.29 mB/s
t8 - 42.89 sec; ~ 24.96 mB/s
xtool_x86, v07
t1 - 237.07 sec; ~ 4.52 mB/s
t7 - 53.84 sec; ~ 19.88 mB/s
t8 - 51.19 sec; ~ 20.91 mB/s
xtool_x64, v07
t1 - 210.85 sec; ~ 5.08 mB/s
t7 - 49.35 sec; ~ 21.69 mB/s
t8 - 46.91 sec; ~ 22.82 mB/s
1'070'530'560 -> afr: 1'767'925'585 bytes
-> xtool: 1'768,021'857 bytes
i7-4700MQ, 8gb ram, ram-disk, win7x64.
Similar happens on Agony's .PAK files(zlib,v0.7, v0.6 works good)
First file processed well,but the second one looks like simply copied.
Cant upload files because they are more than 10 gigs. :/
i think it may be better for Razor12911 if he develop the tool separately like pzlib/plz4/pzstd/plzo for games...
less of a headache ;)
How exactly is that less headache? This way at least he can re-use a lot of same code/routines for each compressor instead of duplicating and maintaining separate projects with a lot of same code.
Razor12911
31-05-2018, 09:31
Update available
Changes
- fixed crilayla bug on naruto games
- fixed issue detecting lzo2a, lzo1c streams
@doofoo24
I have to agree with elit, It's more headache separating the codecs. An example would be ztool itself, normally when I make changes to pzlib, if it's not related to precompression itself, I had to make the same changes in plz4, plzo... sometimes I forget doing this and end up with different sources of each, in one, a bug was fixed then another bug appears in another, now there are bugs all over the codecs, different bugs, you end up trying to fix one and another pops up and so forth. The idea of putting everything together means one code, if one codec is affected, then all will be affected then I'll know the source of the bug.
compr:
afr:v0+srep:m5f:l512:d512m:hash-+lolz:d64m - 37,6 MB (39 463 094 bytes)
xtool+srep:m5f:l512:d512m:hash-+lolz:d64m - 37,8 MB (39 671 049 bytes)
dec time/speed:
cls-afr, v019 (x86)
t1 - 199.28 sec; ~ 5.37 mB/s
t7 - 44.07 sec; ~ 24.29 mB/s
t8 - 42.89 sec; ~ 24.96 mB/s
xtool_x86, v07
t1 - 237.07 sec; ~ 4.52 mB/s
t7 - 53.84 sec; ~ 19.88 mB/s
t8 - 51.19 sec; ~ 20.91 mB/s
xtool_x64, v07
t1 - 210.85 sec; ~ 5.08 mB/s
t7 - 49.35 sec; ~ 21.69 mB/s
t8 - 46.91 sec; ~ 22.82 mB/s
1'070'530'560 -> afr: 1'767'925'585 bytes
-> xtool: 1'768,021'857 bytes
i7-4700MQ, 8gb ram, ram-disk, win7x64.
I wonder if there is still room for improvement.
Razor12911
31-05-2018, 09:52
@Razor12911 xtool 0.7 seems to skip file's test it on ME A with zstd and before with lzo on ac 1...
xtool 0.6 work fine...
any idea ?
i will test with 0.8...
I didn't touch zstd codec in between the updates
ztool_08
Process ID : 12240
Thread ID : 19080
Process Exit Code: 0
Thread Exit Code : 0
User Time : 24.500s
Kernel Time : 0.734s
Process Time : 25.234s
Clock Time : 7.764s
Working Set : 193016 KB
Paged Pool : 114 KB
Nonpaged Pool : 13 KB
Pagefile : 314972 KB
Page Fault Count : 311084
IO Read : 43332 KB (in 683 reads )
IO Write : 101353 KB (in 1584 writes)
IO Other : 3 KB (in 178 others)
ztool_06
Process ID : 12748
Thread ID : 14488
Process Exit Code: 0
Thread Exit Code : 0
User Time : 24.890s
Kernel Time : 1.125s
Process Time : 26.015s
Clock Time : 8.160s
Working Set : 177480 KB
Paged Pool : 115 KB
Nonpaged Pool : 12 KB
Pagefile : 229352 KB
Page Fault Count : 316978
IO Read : 43262 KB (in 677 reads )
IO Write : 101353 KB (in 1584 writes)
IO Other : 5 KB (in 270 others)
Razor12911
31-05-2018, 10:02
compr:
afr:v0+srep:m5f:l512:d512m:hash-+lolz:d64m - 37,6 MB (39 463 094 bytes)
xtool+srep:m5f:l512:d512m:hash-+lolz:d64m - 37,8 MB (39 671 049 bytes)
dec time/speed:
cls-afr, v019 (x86)
t1 - 199.28 sec; ~ 5.37 mB/s
t7 - 44.07 sec; ~ 24.29 mB/s
t8 - 42.89 sec; ~ 24.96 mB/s
xtool_x86, v07
t1 - 237.07 sec; ~ 4.52 mB/s
t7 - 53.84 sec; ~ 19.88 mB/s
t8 - 51.19 sec; ~ 20.91 mB/s
xtool_x64, v07
t1 - 210.85 sec; ~ 5.08 mB/s
t7 - 49.35 sec; ~ 21.69 mB/s
t8 - 46.91 sec; ~ 22.82 mB/s
1'070'530'560 -> afr: 1'767'925'585 bytes
-> xtool: 1'768,021'857 bytes
i7-4700MQ, 8gb ram, ram-disk, win7x64.
Ran a test using the samples I got from forum. Very small input but got these results:
Input: 278 MB (291,605,504 bytes)
DataPC_Map_Menu.forge (ACI)
DataPC_LoadingRoom2.forge (ACI)
DataPC.forge (ACII)
DataPC.forge (ACIV)
command line:
xtool_x86: d:precomp:t# %1.out1 %1.res
afr19_x86: d -v0 -t# %1.out2 %1.res
Thread(1) = 1 result:
xtool: 17.898s, 18.050s, 17.901s, 18.056s, 17.942s
afr19: 17.853s, 17.875s, 18.173s, 17.865s, 17.879s
Thread(s) = 2 result:
xtool: 9.120s, 9.156s, 9.035s, 9.067s, 9.008s
afr19: 9.394s, 9.356s, 9.318s, 9.248s, 9.291s
Thread(s) = 3 result:
xtool: 6.325s, 6.412s, 6.249s, 6.412s, 6.264s
afr19: 6.603s, 6.511s, 6.696s, 6.517s, 6.691s
Not sure in terms of results here, I ran test on ramdisk using my old core 2 extreme, I didn't include t4 since those are mostly irrelevant.
As for final output after compression, I'd be taking a gamble when I say it's headers since they include stuff like what codec was used since xtool is not just made for one game engine but I'll check in terms of the number of streams found by xtool vs afr
Razor12911
31-05-2018, 10:08
tried xtool 0.8 and 0.7 on multiple files at once like shazzla First file processed well,but the second one looks like simply copied.
lz4 / lzo /zlib
upload files and command line used
pakrat2k2
31-05-2018, 10:09
ALWAYS choose disable smilies, when posting/ replying ( checkbox below message box )
pain to have to come in & clean up posts & quotes.
Edison007
31-05-2018, 10:40
Not sure in terms of results here
It's normal, because i using differents decode methods in cls and exe. In cls i use some tricks, which allow me to get a higher speed. exe-decoder, yes, slower.
I'll check in terms of the number of streams found by xtool vs afr
I think, that numbers of streams found is the same. But maybe in AFR the output container is more successfully made ;)
I wonder if there is still room for improvement.
Yep, but I'm not interested in this engine anymore.
Yep, but I'm not interested in this engine anymore.
So, what's your next clue? :D
dataRegion.cpk 607 MB> 1.59 GB
FreeArc 0.67 (March 15 2014) creating archive: data.arc
Compressed 2 files, 636,697,912 => 1,711,252,663 bytes. Ratio 268.77%
Compression time: cpu 0.89 sec/real 141.42 sec = 1%. Speed 4.50 mB/s
All OK
FreeArc 0.67 (March 15 2014) testing archive: data.arc
Tested 2 files, 1,711,252,663 => 636,697,912 bytes. Ratio 268.77%
Testing time: cpu 0.80 sec/real 118.83 sec = 1%. Speed 5.36 mB/s
All OK
Razor12911
31-05-2018, 11:52
test xtool precomp:zlib with 0.6 vs 0.8 on just cause 2
and... which is which?
Sergey3695
31-05-2018, 13:59
NARUTO SHIPPUDEN Ultimate Ninja STORM 2
FreeArc 0.67 unpacker. Extracting archive: data.arc
Extracting dataRegion.cpk (716396224 bytes)
All OK
129.547s: unarc x -o+ -w.\ -dp_TEST data.arc
Razor12911
31-05-2018, 14:12
Update available
Changes
- fixed 2gb+ input issue... (int32 instead of int64 :()
xtl06:
Compressed 46 files, 13,586,655,989 => 22,419,187,764 bytes. Ratio 165.01%
xtl07-08:
Compressed 46 files, 13,586,655,989 => 15,710,322,921 bytes. Ratio 115.63%
xtl09:
Compressed 46 files, 13,586,655,989 => 22,419,187,764 bytes. Ratio 165.01%
crilayla only works with 32 bit
zstd and oodle are 64bit only. There is no 32bit for these
Razor12911
31-05-2018, 15:19
zstd and oodle are 64bit only. There is no 32bit for these
actually zstd 32-bit is there, just that perhaps the library I included requires a prerequisite.
libgcc_s_dw2-1.dll
as for oodle, I didn't bother with 32-bit version considering the fact that it's memory hungry plus most games ship with the win64 version.
doofoo24
31-05-2018, 15:33
xtool 0.9 precomp:t100p:lz4 :
COD WWII 38.4GB TO 62.3GB to 25.8gb srep+lolz
COD AW 33GB TO 63.8GB to 20.3gb srep+lolz
What about decompression for FarCry 3?
ShivShubh
01-06-2018, 08:04
xtool 0.9 precomp:t100p:lz4 :
COD WWII 38.4GB TO 62.3GB to 25.8gb srep+lolz
COD AW 33GB TO 63.8GB to 20.3gb srep+lolz
Since when COD WWII was 38.4 GB SP Only ? I thought it was around 42 GB after removing MP/ZM files, am I missing something here ? Have you kept xpakfile*.pak ?
doofoo24
01-06-2018, 08:26
xtool seems to work on Wolfenstein The New Order resources files like doom...
but files with .pages file nothing there...
doofoo24
01-06-2018, 08:45
also on The Evil Within tangoresource files work
Razor12911
01-06-2018, 09:22
What about decompression for FarCry 3?
Edit xtool.ini, add all the dat files in there
[Dunia2]
File1=C:\Program Files (x86)\Far Cry 3 Complete Collection\Far Cry 3\data_win32\common.dat
File2=C:\Program Files (x86)\Far Cry 3 Complete Collection\Far Cry 3\data_win32\ige.dat
File3=C:\Program Files (x86)\Far Cry 3 Complete Collection\Far Cry 3\data_win32\patch.dat
File4=C:\Program Files (x86)\Far Cry 3 Complete Collection\Far Cry 3\data_win32\patch_german.dat
File5=C:\Program Files (x86)\Far Cry 3 Complete Collection\Far Cry 3\data_win32\shadersobj.dat
something like this then use lzo codec in arc.ini
[External compressor:xprecomp]
header = 0
packcmd = xtool.exe e:precomp:t4:lzo - - <stdin> <stdout>
unpackcmd = xtool.exe d:precomp:t4 - - <stdin> <stdout>
Compressed 12 files, 700,791,484 => 1,143,391,721 bytes. Ratio 163.16%
Compression time: cpu 1.31 sec/real 104.83 sec = 1%. Speed 6.68 mB/s
Extracted 12 files, 1,143,391,721 => 700,791,484 bytes. Ratio 163.16%
Extraction time: cpu 1.98 sec/real 72.79 sec = 3%. Speed 9.63 mB/s
Same applies to Far Cry 4 and Far Cry 5, except with Far Cry 5, use lz4
Razor12911
01-06-2018, 09:28
what about far cry primal and watch dog ?
I heard Far Cry Primal uses modified lz4 and as for Watch Dogs, I have no idea how to use xcompress library in Delphi otherwise detecting its streams is easy.
If I like that idea, here I leave the game for tests thanks Razor12911
MGSV Ground Zeroes (https://drive.google.com/drive/folders/1RuwdMwit0bHU_UYTK5N-cA5uT2gHgQ-s?usp=sharing)
Hi Razor12911
Unreal Engine 3 LZO may be added in the future?
XCOM 2, Outlast 2, Get Even
alfredd31
01-06-2018, 11:18
to doofoo24: what liblz4 did you use for cod aw?
alfredd31
01-06-2018, 11:47
I already tried this one, it doesn't expand anything (Only tried imagefile1.pak). Did you use xtool.ini?
I used this:
packcmd = xtool.exe e:precomp:t4:v:c32:lz4 - - <stdin> <stdout>
By the way, note that you can find some of my repacks - Quantum Break (23.3 Gb) and outlast 2 (10.2Gb) on ygg if you are interested. You can erase this, I'm sure it's against the rules of the forum.
Razor12911
01-06-2018, 12:11
I already tried this one, it doesn't expand anything (Only tried imagefile1.pak). Did you use xtool.ini?
I used this:
packcmd = xtool.exe e:precomp:t4:v:c32:lz4 - - <stdin> <stdout>
By the way, note that you can find some of my repacks - Quantum Break (23.3 Gb) and outlast 2 (10.2Gb) on ygg if you are interested. You can erase this, I'm sure it's against the rules of the forum.
@everyone
Please make sure the command line is correct first.
"e:precomp:t4:v:c32:lz4" is heavily incorrect
alfredd31
01-06-2018, 12:16
Well, it seems it works with e:precomp:t100p:lz4 with .ff files. For pak files it's ok too. (except for soundfiles which can't be expanded).
Razor12911
01-06-2018, 12:21
Well, it seems it works with e:precomp:t100p:lz4 with .ff files. Doesn't seem to work with .pak files though.
imagefile59.pak
Compressed 1 file, 153,906,476 => 312,811,044 bytes. Ratio 203.25
alfredd31
01-06-2018, 12:22
bik reencoded at 70% indeed. Do you think you could get sth much better? probably not. I did nothing special just used srep and lolz.
I already tried this one, it doesn't expand anything (Only tried imagefile1.pak). Did you use xtool.ini?
I used this:
packcmd = xtool.exe e:precomp:t4:v:c32:lz4 - - <stdin> <stdout>
As razor sayd, options are incorrect.
This one is correct
packcmd = xtool.exe e:precomp:t4,v,c32m:lz4 - - <stdin> <stdout>
{options} must be seperated by a comma (,), not a colon (:).
And you forgot a "m" for chunk size.
Update available
- fixed 2gb+ input issue... (int32 instead of int64 :()
Lol I know alright, same trap happened to me with my rawinjector ^_^.
Edit xtool.ini, add all the dat files in there
[Dunia2]
File1=C:\Program Files (x86)\Far Cry 3 Complete Collection\Far Cry 3\data_win32\common.dat
File2=C:\Program Files (x86)\Far Cry 3 Complete Collection\Far Cry 3\data_win32\ige.dat
...
Now this is nice step forward, to support files like this that are perhaps not easy to detect. I would love to see this going even further in the future, where user could script offsets and data reading with few commands. Example:
HEAD "0f3b4c5b"
SKIP 125
CMPSZ 8
DECMPSZ 8
READ CMPSZ DECMPSZ
Above example would mean: find first/next header with "0f3b4c5b" bytes, then skip 125 bytes, then read 64bit integer that is compressed size, then same for decompressed size, then read and precomp next CMPSZ bytes with above info, repeat until EOF.
Idea is that most engines have similar structures, there are some headers, some data to skip, there is CMP and DECMP sizes info in between and then actual compressed data chunks. So instead of keeping maintenance for every new game/engine which rely on Razor to be here for us and kind enough to work on updates, this would make xtool more forward compatible for the future games without needing to constantly updating it. At least until codecs API's change. I know it will not be enough for every future game but it should cover plenty of them.
This one is correct
[code]packcmd = xtool.exe e:precomp:t4,v,c32m:lz4 - - <stdin> <stdout>
{options} must be seperated by a comma (,), not a colon (:).
I apologize for stupid question but was there specific reason for using extra "precomp" word, aka e:precomp... instead of just e:... d:... as was in ztool? To me it seems redundant as e: and d: already imply default behavior(encode vs decode) and precomp seem'd to me at first confused with precomp tool by schnaader. Also using different separators, aka ',' and ':' for specific areas of cmd make it prone to type error, as happened to me before and I seem to be not the only person. Pure ':'s in ztool were fine and easier for inclussion with other utilities IMO. BTW I am not complaining, this tool is fantastic and ultimate, just being curious about new design.
Don't ask me, ask razor why :D
I apologize for stupid question but was there specific reason for using extra "precomp" word, aka e:precomp... instead of just e:... d:... as was in ztool? To me it seems redundant as e: and d: already imply default behavior(encode vs decode) and precomp seem'd to me at first confused with precomp tool by schnaader.
Only explanation i find is that xtool won't be limited to pre-compression only. I might be talking gibberish here and it's just a redundant word as you say. ;)
ShivShubh
02-06-2018, 20:38
Only explanation i find is that xtool won't be limited to pre-compression only. I might be talking gibberish here and it's just a redundant word as you say. ;)
Razor certainly has plans to implement already exisiting compressors into xtool like razor,lolz etc... with stdio support to make it a universal tool but I might be also talking gibberish here, only Razor knows :confused:
Edit xtool.ini, add all the dat files in there
[Dunia2]
File1=C:\Program Files (x86)\Far Cry 3 Complete Collection\Far Cry 3\data_win32\common.dat
File2=C:\Program Files (x86)\Far Cry 3 Complete Collection\Far Cry 3\data_win32\ige.dat
File3=C:\Program Files (x86)\Far Cry 3 Complete Collection\Far Cry 3\data_win32\patch.dat
File4=C:\Program Files (x86)\Far Cry 3 Complete Collection\Far Cry 3\data_win32\patch_german.dat
File5=C:\Program Files (x86)\Far Cry 3 Complete Collection\Far Cry 3\data_win32\shadersobj.dat
something like this then use lzo codec in arc.ini
[External compressor:xprecomp]
header = 0
packcmd = xtool.exe e:precomp:t4:lzo - - <stdin> <stdout>
unpackcmd = xtool.exe d:precomp:t4 - - <stdin> <stdout>
Compressed 12 files, 700,791,484 => 1,143,391,721 bytes. Ratio 163.16%
Compression time: cpu 1.31 sec/real 104.83 sec = 1%. Speed 6.68 mB/s
Extracted 12 files, 1,143,391,721 => 700,791,484 bytes. Ratio 163.16%
Extraction time: cpu 1.98 sec/real 72.79 sec = 3%. Speed 9.63 mB/s
Same applies to Far Cry 4 and Far Cry 5, except with Far Cry 5, use lz4
Thamks but fc3_main.dat (3.2gb) not decompressing(((
Razor12911
03-06-2018, 12:44
Thamks but fc3_main.dat (3.2gb) not decompressing(((
I'll check
Update available
Changes
- added partial unreal engine lzo support (thanks to Edison007 for lzo1x_99 args)
Notes
This is might as well be a useless update at the moment because I'm pretty sure either old or few UE games use the method that is precompressed by 0.10, I'm still trying to figure out how to precompress the rest of the streams which are common.
I'll check
With 0.10 update - also not decompressing
felice2011
04-06-2018, 08:17
Razor but for the decompression of the archive the folder can only contain the file xtool.exe, without any *.dll right?
ShivShubh
04-06-2018, 08:37
Razor but for the decompression of the archive the folder can only contain the file xtool.exe, without any *.dll right?
Couldn't understand your question but you need to have the dll for the codec you used, in the same folder as xtool.exe. Suppose you used lz4, you just need liblz4.dll not the others.
ZakirAhmad
04-06-2018, 08:38
Razor but for the decompression of the archive the folder can only contain the file xtool.exe, without any *.dll right?
No, for decompression u r going to need same dlls which u used for xompression. that means if u used plz4 during compression u r going to need xtool + liblz4 for decompression.
felice2011
04-06-2018, 09:10
Strange for the decompression seems to work without any DLL, only xtool.exe, tried on a data folder of generic files, that's why I was curious to understand ...:confused:
ShivShubh
04-06-2018, 09:28
Strange for the decompression seems to work without any DLL, only xtool.exe, tried on a data folder of generic files, that's why I was curious to understand ...:confused:
If it had zlib streams then it works without dll as well because xtool has internal zlib/deflate libraries but dont know if razor did that for other codecs too maybe yes he did considering how stupid people can actually get :D
But its always recommended to use latest external libraries for faster processing.
felice2011
04-06-2018, 11:56
wow how many masters and experts in compression lately here in the forum, it's been a while that I'm not here in the forum, it's not about ease of use for people, it's about knowing whether or not to insert the DLL for decompression, "Faster Processing" explain to me what it means because the decompression speed is the same, since we are talking about decompression and not compression.
ShivShubh
04-06-2018, 12:11
wow how many masters and experts in compression lately here in the forum, it's been a while that I'm not here in the forum, it's not about ease of use for people, it's about knowing whether or not to insert the DLL for decompression, "Faster Processing" explain to me what it means because the decompression speed is the same, since we are talking about decompression and not compression.
If I remember correctly, razor once said that internal processing can be actually slower at certain stages when compared to its usage with external libraries. I might be wrong but thats what I remember. Plus I am not even sure what internal libraries are present in xtool, only razor knows, maybe reflate libraries are not present. With ZTool, I noticed size was better with external libraries in some cases and faster too, but its XTool so who knows, maybe Razor improved it. But one thing I can tell, its always better with the external libraries so I don't see the problem in using them.
But really the main reason for internal libraries is for the ease of people. Razor implemented the internal libraries so that processing will still work in case people don't even put the dlls at the correct places, that was the main reason I think.
felice2011
04-06-2018, 12:36
This is what I do not understand, I wanted a concrete motivation, in my concept of coder, a file if needed I use it, if I do not need I do not use it, I do not insert files because I like the dll extension or others, as many they do, without really understanding their use and necessity.
I wait Razor look for a response aimed at my curiosity, thanks anyway for your intervention.
excuse me for post this. i googled before but not good results. is there a link to a tutorial on how to use x-tool i wanna test it to sve some data in my hdd and im totally noob in this .
i ve used the bat file but didnt compress or i really didnt see the difference with the actual file. (a ps2 iso files, even unrared)
It all depends what actual "internal processing" is. Does it mean just dll's included inside exe, does it mean official source code translated to Pascal and included/compiled with project, or does it mean having your own self-written routines?
For 1. case there should be no difference, for case 2. and 3. my guess is that since Razor code in Borland's Delphi, that could be one reason for slower internal routines vs external dll's. Because I used to code in Borland's C++ back then and I remember it was not that fast. Also B. Pascal, just like B. C++ use own libraries, own compiler and both are object based languages. Which will be slower than standard direct C routines, say through MinGW. In most cases that is, few exceptions like STL aside.
Since dll's of codecs like lz4, zstd and most others are written in C and already optimized by its devs, they should be quicker as Delphi is then only used as a bridge for io calling. Yes normally internal routines should be always faster than calling external dll's, but only if under same environment and optimizations. Here we are basically talking B. Pascal vs C overhead and thats assuming routines are perfectly written or translated/optimized to Pascal which they may not be, so you get a speed difference based from both factors.
Dll's also give you ability to switch between different versions as long as they use same API which is another plus.
does it mean official source code translated to Pascal and included/compiled with project, or does it mean having your own self-written routines?
i think pascal compiler's abi is compatible with maybe gcc(if gcc not, i think its borland(embarcadero) c++ itself which sucks at optimizations) so they used compiled objects and directly links with the pascal code see https://github.com/madler/zlib/blob/master/contrib/pascal/zlibpas.pas
Another trickery can be, represents your dll as hexadecmial literals in your code and using some library like (https://github.com/fancycode/MemoryModule), to directly load from memory, quickbms uses it
i think pascal compiler's abi is compatible with maybe gcc(if gcc not, i think its borland(embarcadero) c++ itself which sucks at optimizations) so they used compiled objects and directly links with the pascal code see https://github.com/madler/zlib/blob/master/contrib/pascal/zlibpas.pas
Another trickery can be, represents your dll as hexadecmial literals in your code and using some library like (https://github.com/fancycode/MemoryModule), to directly load from memory, quickbms uses it
For what I have tested the borland compiler is based on clang(msvc edition), as far as minGW posix is not compatible, I just do not know if it is possible with minGW win32. Since you use msys2 then it's posix!
2011 clang which has shitty windows support and they somehow glued it with shitty code, change its core abi to make it work , it's just not right, it can't even compile c++11 correctly even their vector and share_ptr has bugs that why i was using my own implementation in designer and these compiler bugs made me left designers
alfredd31
06-06-2018, 14:18
error with decompression too. Only 1 file, 1.3Gb. Included xtool+liblz4.dll. The installer is stuck at the end, and the file is different: 1,405,353,984 byte vs (1,405,386,216 bytes for the original.
alfredd31
06-06-2018, 15:45
I'm using this...
unpackcmd = xtool.exe d:precomp:t4 - - <stdin> <stdout>
Since others have similar problems, there is probably sth wrong with xtool.
alfredd31
07-06-2018, 03:40
except that I can obtain a better size than doofoo: 22.5Gb. You can select some bik files, even if some are giving an error.
ZakirAhmad
07-06-2018, 05:37
except that I can obtain a better size than doofoo: 22.5Gb. You can select some bik files, even if some are giving an error.
what is ur packing command?
alfredd31
07-06-2018, 06:09
packcmd = xtool.exe e:precomp:t4,lz4 - - <stdin> <stdout>
A script that works with the existing command
I apologize for my bad language in English
ZakirAhmad
07-06-2018, 09:41
packcmd = xtool.exe e:precomp:t4,lz4 - - <stdin> <stdout>
Sometimes srep also gives crc error during decompression. Try srep 3.92 instead of 3.93.
the file which is giving u crc error. try only xtool on it and decompress it(see what u get)
alfredd31
07-06-2018, 10:06
I was using cls-srep (it's srep 3.93a) and xtool 0.9. This srep was working with ztool.
For decompression I'm using this, not the one above...: unpackcmd = xtool.exe d:precomp:t4 - - <stdin> <stdout>
@Ahmad: the installer is not even giving a crc error. It's just stuck. I have the same problem with the whole folder compressed. srep64 and xtool are still in the working processes.
Maybe I'm going to try with smaller files. In both cases it's stuck with the biggest file (imagefile92.pak).
Tested with a smaller file (25mb)...decompression worked.
my scenario for the s&p 500: a big fall. Sell everything.
http://econintersect.com/images/2018/06/66527342welsh.tech.2018.jun.04.fig.04.PNG
http://www.dw.com/image/17095124_303.jpg
alfredd31
07-06-2018, 14:23
Never mind, I was planning to delete codaw anyway. I prefer keeping smaller games. And in this one, it seems no files can be deleted.
ZakirAhmad
07-06-2018, 15:23
When u r debugging some bug u r supposed to check individual files. BTW, the cls srep u r using doesnt only work for srep 3.93. it works with all sreps. The reason i told u this is becase i have experianced this with afr by edison. at last i found it was the problem with srep 3.93a when i replaced it with 3.92b. From that moment onwards i never god any problem.
alfredd31
07-06-2018, 15:38
Well, I'll check that.
Note that I tested xtool with farcry 4 too. And it seems files are not expanded as much as they could be. 24Gb=>30Gb. As a result there is a poor compression. I used a tool made by a certain edison007, and the compression was better, but the script used to decompress all that was a bit more complex. The game is still available on nomaher.com of course.
If you want to make a donation for the administrator, in Palestine, you can contact him. And if you want to see me in Paris to give me 1 million € so that I can fully devote myself to posting on this forum, and stop working (think my boss can't bear to see me any more), you have to know that I agree. We can meet at the procope for example to talk about that.
My target for the s&p 500: 2100 points. The housing and stock bubble will pop, sometime.
https://image.ibb.co/dhb32o/baboon.png
alfredd31 use srep 3.93 (https://www9.zippyshare.com/v/rfOICSRL/file.html)
oodle for ACO
DataPC.forge (273.088.512 Bytes)
with srep64:m3f+lzma2
xtool(:c32m,t4:oodle)
260mb => 548mb => 195mb (205.152.256 Bytes)
Compression time: 192secs
Decompression time: 62secs
AFR(:a2) with in/out patch
260mb => 548mb => 195mb (204.939.264 Bytes)
Compression time: 144secs
Decompression time: 118secs
For decompression used all threads
Only precompress:
xtool
Precompress time: 63secs
Decompress time: 55secs
AFR
Precompress time: 18secs
Decompress time: 105secs
So xtool is at least two times faster than AFR in decompression, but ~3.5 times slower in precompression
masen485
09-06-2018, 02:46
xtool VERSION
0.8,0.7,0.09
xtool(:c32m,t4:oodle) WHICH VERSION
HELP PLEASE
Assassin's Creed Brotherhood
DataPC.forge
masen485
09-06-2018, 02:57
If possible, compress your game
Do you share tools with me?
No.
@razor: Just to let you know, 65+GB data will not be inflated by xtool somehow.
I tested for Origins with c32m,t4:oodle
The first 1GB will be inflated (xtool is over 80% cpu usage), after that, arc only copying files and xtool do nothing anymore. Maybe to much data?
Yes tested with 0.9 and 0.10
Compressing 122 files, 69,051,449,344 bytes. Processed 10.0% ==>> xtool
Compressing 69,353,227,695 bytes with ==>> srep
Correction: xtool is just working for about ~250mb, then stop
masen485
09-06-2018, 11:35
xtool VERSION
0.8,0.7,0.09
xtool(:c32m,t4odle) WHICH VERSION
HELP PLEASE
Assassins Creed IV Black Flag
DataPC.forge
but xtool 0.10 only x86, i thought odle only x64 :confused:
Ok, didn't realized 0.10 was just an update for x86. But problem remains
Edison007
10-06-2018, 00:13
oodle for ACO
DataPC.forge (273.088.512 Bytes)
with srep64:m3f+lzma2
xtool(:c32m,t4:oodle)
260mb => 548mb => 195mb (205.152.256 Bytes)
Compression time: 192secs
Decompression time: 62secs
AFR(:a2) with in/out patch
260mb => 548mb => 195mb (204.939.264 Bytes)
Compression time: 144secs
Decompression time: 118secs
For decompression used all threads
Only precompress:
xtool
Precompress time: 63secs
Decompress time: 55secs
AFR
Precompress time: 18secs
Decompress time: 105secs
So xtool is at least two times faster than AFR in decompression, but ~3.5 times slower in precompression
Give me please this file.)
@Edison
Wait (wrong file)
will upload for you
Razor12911
14-06-2018, 18:01
I apologize for stupid question but was there specific reason for using extra "precomp" word, aka e:precomp... instead of just e:... d:... as was in ztool? To me it seems redundant as e: and d: already imply default behavior(encode vs decode) and precomp seem'd to me at first confused with precomp tool by schnaader. Also using different separators, aka ',' and ':' for specific areas of cmd make it prone to type error, as happened to me before and I seem to be not the only person. Pure ':'s in ztool were fine and easier for inclussion with other utilities IMO. BTW I am not complaining, this tool is fantastic and ultimate, just being curious about new design.
There methods planned apart from precomp to be added in xtool, stuff like preprocessing of data, encryption, data reconstruction (rebuild), compression and etc.
Razor but for the decompression of the archive the folder can only contain the file xtool.exe, without any *.dll right?
xtool requires dll at all times, even zlib requires dll. no code of compresssion algorithms are stored within xtool except for special cases but even those special cases do not execute if a specific dll that goes hand in hand with aren't available.
And?:)
And I think there is something wrong with the parser for FAT v9 (FC3, FC4) for big dat files, I kinda rushed adding this feature without conducting any proper tests.
Yes tested with 0.9 and 0.10
Compressing 122 files, 69,051,449,344 bytes. Processed 10.0% ==>> xtool
Compressing 69,353,227,695 bytes with ==>> srep
Correction: xtool is just working for about ~250mb, then stop
Bugs everything, when are they gonna stop .... -_-
Great tool. Thanks Razor!
What about decompress *.mega2 & *.pages files from DOOM 2016?
Razor12911
21-06-2018, 16:51
Development will resume after the first week of July
ShivShubh
21-06-2018, 21:46
What about decompress *.mega2 & *.pages files from DOOM 2016?
Its pointless to decompress pages files of idTech games for further compression because you will end up having a bad ratio. In previous games they are heavily packed with some texture compression. I tried with DOOM 2016 and as far as I remember ratio was worse compared to their original compressed size.
Its pointless to decompress pages files of idTech games for further compression because you will end up having a bad ratio. In previous games they are heavily packed with some texture compression. I tried with DOOM 2016 and as far as I remember ratio was worse compared to their original compressed size.
Thanks
doofoo24
23-06-2018, 17:21
@Razor12911 i tested xtool 0.10 on mass effect 2 type8 files when i use t100p seems like prime95 THE TEMP are very hot around 90c ?
because when i use t100p with zlib or lz4 it uses all cpu threads but the temp around 70c with lzo around 90c...
i am using i7 8700k with NH-D15...
Razor12911
23-06-2018, 21:23
@Razor12911 i tested xtool 0.10 on mass effect 2 type8 files when i use t100p seems like prime95 THE TEMP are very hot around 90c ?
because when i use t100p with zlib or lz4 it uses all cpu threads but the temp around 70c with lzo around 90c...
i am using i7 8700k with NH-D15...
shows that it's a shitty algorithm causing me headaches for no reason whatsoever
alfredd31
25-06-2018, 14:31
it seems you were right, it wasn't xtool 0.10 which is crashing but srep. I tested srep 3.91, it's the same thing. You can test my latest version of The Evil Within 2, available on nomaher.com.
Note that I'm still very bearish on the s&p500.
https://image.ibb.co/hkbZC8/bb3.jpg
felice2011
25-06-2018, 16:21
@Razor12911 i tested xtool 0.10 on mass effect 2 type8 files when i use t100p seems like prime95 THE TEMP are very hot around 90c ?
because when i use t100p with zlib or lz4 it uses all cpu threads but the temp around 70c with lzo around 90c...
i am using i7 8700k with NH-D15...
Ha Ha ... it seems that xtool 0.10 is great for testing CPU stability in Rock Solid. What the hell with questions !!!!
doofoo24
25-06-2018, 20:34
What the hell with questions !!!!
chill out fratello :D
@Razor12911 i tested xtool 0.10 on mass effect 2 type8 files when i use t100p seems like prime95 THE TEMP are very hot around 90c ?
because when i use t100p with zlib or lz4 it uses all cpu threads but the temp around 70c with lzo around 90c...
i am using i7 8700k with NH-D15...
the 8700k has alot of issues with heat,only way to bring those temps down is to delid the cpu.i have mine delidded at 5.2GHZ @ 1.39v with liquid metal it does not go over 65-70 degress under full load.
oh i do have a full custom water loop. :D
I want to encode far cry 5 with xtool
arc.ini:
[External compressor:xprecomp]
header = 0
packcmd = xtool.exe e:precomp:c32mb,t4:lz4,lzo - - <stdin> <stdout>
unpackcmd = xtool.exe d:precomp:t4 - - <stdin> <stdout>
pack.bat
del /q data.arc
arc.exe a -ep1 -r -ed -s; -w.\temp -mxprecomp data.arc "pack\*"
pause
but ratio: 100%, please help me!
https://fileforums.com/attachment.php?attachmentid=22332&stc=1&d=1531869409
ZakirAhmad
17-07-2018, 18:57
what is pack.ini?
write pack.bat instead of ini
what is pack.ini?
write pack.bat instead of ini
I've brought pack.bat file above.
pack.ini is not!!!!!!!
Please refer to the original subject and please provide guidance.
pack.bat
del /q data.arc
arc.exe a -ep1 -r -ed -s; -w.\temp -mxprecomp data.arc "pack\*"
pause
ZakirAhmad
18-07-2018, 07:38
their r two types of files in farcry 5 .dat and .fat
in xtool.ini (provided by razor)
write locations of all .fat files in xtool.ini
their r two types of files in farcry 5 .dat and .fat
in xtool.ini (provided by razor)
write locations of all .fat files in xtool.ini
Exactly I just want to compress these two file types (.dat and .fat), that this output is obtained!!! :confused::confused:
ZakirAhmad
18-07-2018, 10:16
suppose u want to compress data.fat and data.dat located in c:/games folder.
just put in xtool.ini
c:/games/data.fat
and hit pack.bat, xtool will prexompress it.
suppose u want to compress data.fat and data.dat located in c:/games folder.
just put in xtool.ini
c:/games/data.fat
and hit pack.bat, xtool will prexompress it.
Thank you brother, I put all of the following in xtool.ini, but none of them gave the correct result. Ratio is still 100%. :confused:
I also tried other scenarios not mentioned here, But the result was all ratio 100%.
pack\common.fat
pack\patch.fat
pack\patch_brazilian.fat
File1=pack\common.fat
File2=pack\patch.fat
File3=pack\patch_brazilian.fat
File1=C:\Share\Far Cry 5\xtool_v010\FA_example\pack\common.fat
File2=C:\Share\Far Cry 5\xtool_v010\FA_example\pack\patch.fat
File3=C:\Share\Far Cry 5\xtool_v010\FA_example\pack\patch_brazilian.fat
[Dunia2]
File1=C:\Share\Far Cry 5\xtool_v010\FA_example\pack\common.fat
File2=C:\Share\Far Cry 5\xtool_v010\FA_example\pack\patch.fat
File3=C:\Share\Far Cry 5\xtool_v010\FA_example\pack\patch_brazilian.fat
C:\Share\Far Cry 5\xtool_v010\FA_example\pack\common.fat
C:\Share\Far Cry 5\xtool_v010\FA_example\pack\patch.fat
C:\Share\Far Cry 5\xtool_v010\FA_example\pack\patch_brazilian.fat
[Dunia2]
C:\Share\Far Cry 5\xtool_v010\FA_example\pack\common.fat
C:\Share\Far Cry 5\xtool_v010\FA_example\pack\patch.fat
C:\Share\Far Cry 5\xtool_v010\FA_example\pack\patch_brazilian.fat
[Dunia2]
File1=C:\Share\Far Cry 5\xtool_v010\FA_example\pack\common.fat
File2=C:\Share\Far Cry 5\xtool_v010\FA_example\pack\common.dat
File3=C:\Share\Far Cry 5\xtool_v010\FA_example\pack\patch.fat
File4=C:\Share\Far Cry 5\xtool_v010\FA_example\pack\patch.dat
File5=C:\Share\Far Cry 5\xtool_v010\FA_example\pack\patch_brazilian.fat
File6=C:\Share\Far Cry 5\xtool_v010\FA_example\pack\patch.dat
[Dunia2]
C:\Share\Far Cry 5\xtool_v010\FA_example\pack\common.fat
C:\Share\Far Cry 5\xtool_v010\FA_example\pack\common.dat
C:\Share\Far Cry 5\xtool_v010\FA_example\pack\patch.fat
C:\Share\Far Cry 5\xtool_v010\FA_example\pack\patch.dat
C:\Share\Far Cry 5\xtool_v010\FA_example\pack\patch_brazilian.fat
C:\Share\Far Cry 5\xtool_v010\FA_example\pack\patch.dat
How to use this Program i have very hard time with Ztool also and this Xtool seems complicated too anybody.. here to explain me plz
ZakirAhmad
19-07-2018, 01:29
just go here, Razor has explained everything.
https://fileforums.com/showpost.php?p=471374&postcount=306
BTW in xtool.ini u have to set .dat file not fat
t4:zlib
Precompressing over 45GB data in one go
Write error (disk full). I have over 400GB free
Anyone else?
Cypher23
21-07-2018, 01:52
[External compressor:xprecomp]
header = 0
packcmd = xtool e:precomp:c32mb,t4:zlib,lz4 - - <stdin> <stdout>
unpackcmd = xtool d:precomp:t4 - - <stdin> <stdout>
[External compressor:lolz]
header = 0
packcmd = lolz_x64.exe -mc8 -tt4 -mt2 -mtb512 -d128 -fba4096 -dto0 -dm00 -pc2 -bm4 -blr4 -bll8 -blo8 -lm0 -bc4 -cm1 -mc256 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = cls-lolz_x86 $$arcpackedfile$$.tmp $$arcdatafile$$.tmp
[External compressor:srep]
header = 0
packcmd = srep {options} -m5f -d1g -a2 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = srep -d -s - - <stdin> <stdout>
[External compressor:nz,nanozip,nz64]
;header = 0 -cc=best CM algo -cO=BWT algo ;testing = nz:r:v:cc:m8g:br512m:bw512m:p4:t0:nm
packcmd = nz a -r -v -cc -m2g -p2 -t2 -nm $$arcpackedfile$$.tmp.nz $$arcdatafile$$.tmp
unpackcmd = nz x -m2g -t2 -p0 $$arcpackedfile$$.tmp.nz $$arcdatafile$$.tmp
datafile = $$arcdatafile$$.tmp
packedfile = $$arcpackedfile$$.tmp.nz
;USED for Bink/Smacker files
[External compressor:bpk]
header = 0
packcmd = bink_pack $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = bink_unpack $$arcpackedfile$$.tmp $$arcdatafile$$.tmp
solid = 0
;USED for UNreal
[External compressor:uelr]
header=0
packcmd= uelr uv $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
Command arc.exe a -ep1 -r -ed -s; -w.\temp -mxprecomp+lolz+nz data.arc "pack\*"
but it crashes
Sergey3695
21-07-2018, 02:25
lolz+nz
is stupid
use lolz or nz
pakrat2k2
22-07-2018, 10:39
People, before posting stuff with switches, choose disable smilies under misc options just below submit / preview post !! Pain in the a$$ to keep editing all the posts.
PsYcHo_RaGE
28-07-2018, 11:03
ahhhaaaa i found a new tool.....na na na i am just kidding i thought i would help some buddies here if i got much time, hope that so.
well this xtool seems to be doing better than before whatever Razor really had so much potential....keep this pace buddy :)
is on vacation .. maybe start working on xtool in september
hasandgn34
13-09-2018, 14:28
t4:zlib
Precompressing over 45GB data in one go
Write error (disk full). I have over 400GB free
Anyone else?
i have the same problem on t4:lzo algoritm
:(
Increasing Chunk Size improves crilayla speed.:D
Sonic Forces 16 GB > 31.3 GB
[External compressor:xCriLayla]
header = 0
packcmd = _PE\xCriLayla\XTool e:precomp:t100p:crilayla - - <stdin> <stdout>
unpackcmd = _PE\xCriLayla\XTool d:precomp:t100p - - <stdin> <stdout
P: 4 Hour D: 2 Hour
__________________________________________________ __________
[External compressor:xCriLayla]
header = 0
packcmd = _PE\xCriLayla\XTool e:precomp:t100p,c128m:crilayla - - <stdin> <stdout>
unpackcmd = _PE\xCriLayla\XTool d:precomp:t100p - - <stdin> <stdout
P: 90 Min D: 1 Hour
i have the same problem on t4:lzo algoritm
:(
What is your Arc.ini?
Increasing Chunk Size improves crilayla speed.:D
Sonic Forces 16 GB > 31.3 GB
[External compressor:xCriLayla]
header = 0
packcmd = _PE\xCriLayla\XTool e:precomp:t100p:crilayla - - <stdin> <stdout>
unpackcmd = _PE\xCriLayla\XTool d:precomp:t100p - - <stdin> <stdout
P: 4 Hour D: 2 Hour
__________________________________________________ __________
[External compressor:xCriLayla]
header = 0
packcmd = _PE\xCriLayla\XTool e:precomp:t100p,c128m:crilayla - - <stdin> <stdout>
unpackcmd = _PE\xCriLayla\XTool d:precomp:t100p - - <stdin> <stdout
P: 90 Min D: 1 Hour
I have to cap unpack crilayla to t50p (or t2 for me), otherwise extraction gives error (crc fail).
I have to cap unpack crilayla to t50p (or t2 for me), otherwise extraction gives error (crc fail).
USe XTool 0.9
DataPC_ACE_Egypt_ext.forge 12.4 GB
AFR:A2 > 33.3 GB | P: 45 Min D: 30 Min
xOodle > 33.3 GB | P: 39 Min D: 30 Min
ARC.ini
[External compressor:afr]
header = 0
packcmd = AFR_x64.exe e {options} $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = AFR_x64.exe d #in #out <stdin> <stdout>
[External compressor:xOodle]
header = 0
packcmd = _PE\XTool e:precomp:t100p,c128m:oodle - - <stdin> <stdout>
unpackcmd = _PE\XTool d:precomp:t100p - - <stdin> <stdout>
:LOG
:AFR
Creating archive: afr.Bin using afr:a2
Compressed 1 file, 13,407,420,416 => 35,820,682,539 bytes. Ratio 267.17%
Compression time: cpu 26.64 sec/real 2646.71 sec = 1%. Speed 5.07 mB/s
All OK
Testing archive: afr.Bin
Tested 1 file, 35,820,682,539 => 13,407,420,416 bytes. Ratio 267.17%
Testing time: cpu 28.45 sec/real 1790.77 sec = 2%. Speed 7.49 mB/s
All OK
:XTool 0.9
Creating archive: XTool.Bin using xOodle
Compressed 1 file, 13,407,420,416 => 35,817,965,269 bytes. Ratio 267.15%
Compression time: cpu 22.91 sec/real 2286.35 sec = 1%. Speed 5.86 mB/s
All OK
Testing archive: XTool.Bin
Tested 1 file, 35,817,965,269 => 13,407,420,416 bytes. Ratio 267.15%
Testing time: cpu 11.19 sec/real 1759.10 sec = 1%. Speed 7.62 mB/s
All OK
AMD Ryzen 5 1600 3.2GHz
ZAZA4EVER
18-09-2018, 20:37
What version xtool you used ?
What version xtool you used ?
v09 x64 supported oodle
ZAZA4EVER
19-09-2018, 15:06
https://fileforums.com/showpost.php?p=475772&postcount=406
Hello Every One
I tested Afr(razor) & Poodle & Xoodle (xtool V0.9+ ACO Oodle Dll By Simorq ) On (DataPC_ACE_Egypt_ext.forge 12.4 GB)
Really i get Results As Simorq Post
Afr(Razor)
https://i.imgur.com/qvcblPZ.jpg
Xoodle (xtool V0.9+ ACO Oodle Dll By Simorq )
https://i.imgur.com/IfMWuHi.jpg
Poodle (Not Supoort ACO )
https://i.imgur.com/ruJ09NX.jpg
Final Size And Results
https://i.imgur.com/MmKsh2u.jpg
////////////////////////////////////////////////////////////////////
But When Test This Tools With The same Settings On This File Of the Game
V1.5.1 .... The File Name (DataPC_ACE_DLC_Thebes.forge=12.2Gb)
The File Release With The Last DLC Of The Game Show Results
Afr(Razor)
https://i.imgur.com/vjiYIm4.jpg
Xoodle (xtool V0.9+ ACO Oodle Dll By Simorq )
https://i.imgur.com/vgtx5U3.jpg
Poodle (Not Supoort ACO )
https://i.imgur.com/o1T4Sc3.jpg
Final Size And Results
https://i.imgur.com/Qw34KML.jpg
I want Say Thing ACO The Oodle Dll By Simorq Dont Support Updates Files After Crack Version 1.2.1 In The Game Dont Unpack Oodle Stream in New Updates File ... So You will Get Bad Ratio For ACO V1.5.1 With Xoodle (68.8GB---35GB)
But If You Use Afr You Can Get Good Ratio (68.8GB---29.3GB)
I Think Oodle Dll (ACO_oo2core_4_win64.zip) Want To Update To Unpack New Updates Games Files
Thanks For Help All
Razor12911
20-09-2018, 14:38
Increasing Chunk Size improves crilayla speed.:D
Sonic Forces 16 GB > 31.3 GB
[External compressor:xCriLayla]
header = 0
packcmd = _PE\xCriLayla\XTool e:precomp:t100p:crilayla - - <stdin> <stdout>
unpackcmd = _PE\xCriLayla\XTool d:precomp:t100p - - <stdin> <stdout
P: 4 Hour D: 2 Hour
__________________________________________________ __________
[External compressor:xCriLayla]
header = 0
packcmd = _PE\xCriLayla\XTool e:precomp:t100p,c128m:crilayla - - <stdin> <stdout>
unpackcmd = _PE\xCriLayla\XTool d:precomp:t100p - - <stdin> <stdout
P: 90 Min D: 1 Hour
USe XTool 0.9
DataPC_ACE_Egypt_ext.forge 12.4 GB
AFR:A2 > 33.3 GB | P: 45 Min D: 30 Min
xOodle > 33.3 GB | P: 39 Min D: 30 Min
ARC.ini
[External compressor:afr]
header = 0
packcmd = AFR_x64.exe e {options} $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = AFR_x64.exe d #in #out <stdin> <stdout>
[External compressor:xOodle]
header = 0
packcmd = _PE\XTool e:precomp:t100p,c128m:oodle - - <stdin> <stdout>
unpackcmd = _PE\XTool d:precomp:t100p - - <stdin> <stdout>
:LOG
:AFR
Creating archive: afr.Bin using afr:a2
Compressed 1 file, 13,407,420,416 => 35,820,682,539 bytes. Ratio 267.17%
Compression time: cpu 26.64 sec/real 2646.71 sec = 1%. Speed 5.07 mB/s
All OK
Testing archive: afr.Bin
Tested 1 file, 35,820,682,539 => 13,407,420,416 bytes. Ratio 267.17%
Testing time: cpu 28.45 sec/real 1790.77 sec = 2%. Speed 7.49 mB/s
All OK
:XTool 0.9
Creating archive: XTool.Bin using xOodle
Compressed 1 file, 13,407,420,416 => 35,817,965,269 bytes. Ratio 267.15%
Compression time: cpu 22.91 sec/real 2286.35 sec = 1%. Speed 5.86 mB/s
All OK
Testing archive: XTool.Bin
Tested 1 file, 35,817,965,269 => 13,407,420,416 bytes. Ratio 267.15%
Testing time: cpu 11.19 sec/real 1759.10 sec = 1%. Speed 7.62 mB/s
All OK
AMD Ryzen 5 1600 3.2GHz
I'm away but I like how you're helping towards the project, but most importantly, helping people use the tool.
Keep up the good work :)
I second this. He is always helpfull in compression things. :)
LOL. XTool NOT decompressed *.pak files on Crysis (2007). WTF?
Full Game 26.3 GB > 42.6 GB
Creating Archive: "Ni no Kuni II Revenant Kingdom.Bin" using xCriLayla
Compressed 13,775 files, 28,298,083,576 => 45,805,858,017 bytes. Ratio 161.87%
Compression time: cpu 83.41 sec/real 1136.82 sec = 7%. Speed 24.89 mB/s
All OK
P: 20 Min
Tested 13,775 files, 45,805,858,017 => 28,298,083,576 bytes. Ratio 161.87%
Directory 111,513 => 687,098 bytes. Ratio 16.23%
Testing time: cpu 178.48 sec/real 4984.16 sec = 4%. Speed 5.68 mB/s
All OK
D: 84 Min
[External compressor:xCriLayla]
header = 0
packcmd = _PE\xCriLayla\XTool e:precomp:t100p,c128m,v:crilayla - - <stdin> <stdout>
unpackcmd = _PE\xCriLayla\XTool d:precomp:t100p - - <stdin> <stdout>
LZO, CriLayla works with the -v option without problem:D
Use crilayla.dll in ZTool 1.0
you mean ztool 0.198
no
https://fileforums.com/showpost.php?p=475664&postcount=405
it's dll,i thought razor made new ztool :o
http://uupload.ir/files/nwfc_2018-10-05_231254.png
xtool on mass effect 3 seem to fail at decompression always give crc error,
*i don't remember which file exactly.
Encode without raw2hifdll.dll and hif2rawdll.dll:rolleyes: CRC OK.
masen485
08-10-2018, 10:23
new ztool can you give it to us?
it is possible to use every method I see in the picture.
https://resmim.net/f/nt4WXn.png
felice2011
08-10-2018, 12:33
new ztool can you give it to us?
it is possible to use every method I see in the picture.
https://resmim.net/f/nt4WXn.png
he will post when he is ready and when he wants to, not be petulant with the requests, and sometimes try to to thank and support the work of others.
Razor12911
08-10-2018, 21:16
new ztool can you give it to us?
it is possible to use every method I see in the picture.
https://resmim.net/f/nt4WXn.png
People who use the tools I make sometimes...
This is the cmd output of xtool
XTool version 0.9
Created by Razor12911
Commands:
e - encode
d - decode
Operations and codecs available:
precomp - data precompression
zlib : not loaded
crilayla : not loaded
lz4 : loaded
zstd : not loaded
lzo : not loaded
oodle : not loaded
Options:
precomp - data precompression
c# : chunk size (default 16mb)
t# : number of threads, use p for percentage
dm#: decoding memory [1(low) .. 3(max)]
dt#: diff tolerance (default 0.1)
v : skip verification
Usage : XTool [command]:[operation]:[options]:[codecs] [input] [output]
Example : XTool e:precomp:c32mb,t4:zlib,lz4 Textures.tfc Textures.tfc.xtl
or am I missing something?
FYI...
XTool V0.9 doesnt decompresses Mark of the ninja - Remastered's ZIP-files while ZTool 0.0.19.8 works well !
Damn.
Somethings wrong here. Win10 x64 ,zlib not loaded ,but its there. Any idea ? :/
XTool x86 works well. Interesting.
^ You are using zlibwapi.dll x86.
no.
for Xtool V0.9 x64 , i use the x64 libraries,copied from xtool_v09.7z\_libraries\x64\ folder (not loaded)
btw i doesnt matter.. stick with V10 x86.
thanks.
can somebody give me a full tutorial how to use this ? im clueless
[External compressor:xprecomp]
header = 0
packcmd = xtool.exe e:precomp:c32m,t1:crilayla - - <stdin> <stdout>
unpackcmd = xtool.exe d:precomp:t1 - - <stdin> <stdout>
[External compressor:srep]
;options = l%d (minimal match length, default=512)
header = 0
packcmd = srep {options} -m3f $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = srep -d -s $$arcpackedfile$$.tmp $$arcdatafile$$.tmp
[External compressor:precomp]
header = 0
packcmd = precomp -intense0 -c- {options} -o$$arcpackedfile$$.tmp $$arcdatafile$$.tmp
unpackcmd = precomp -o$$arcdatafile$$.tmp -r $$arcpackedfile$$.tmp
del /q data.arc
arc.exe a -lc1 -ep1 -ed -r -w.\temp -xprecomp+lzma+zlib+srep:m3f+lzma2:d256m:a1:mfbt4:d 1024m:fb256:mc5000:lc1:lp0+diskspan:4350mb:4430mb V7_05.bin "C:\Games\DRAGON.BALL.Fighter.Z\*"
pause
what am i doing wrong?
arc.exe a -lc1 -ep1 -ed -r -w.\temp -mxprecomp........
Use xprecomp+srep+lzma. More than enough.
Btw what is zlib in the chain ?! Its obsolete.
even when i have it only compressed a file from 120mb to 97mb , not much
http://imgur.com/a/Hs7UZFJ
Have the files you want to compress crilayla streams?
Have the files you want to compress crilayla streams?
im trying to understand how it works , but i dont have all the files required in v0.10
darkwolves
24-11-2018, 20:05
the only function i cannot get to work is lzo
i can get it to work on uelr
anyone have arc command examples i could try?
AliReZAx1785
13-12-2018, 08:13
tnx, great tool for best compress data. for this command line version have GUI version relase ? i saw this screenshot from youtube
http://s8.picofile.com/file/8345640292/COMPRESSORMETHOD_001363_2018_12_14_19_36_50_.JPG
http://s6.uplod.ir/i/00943/e2j2sjandzaz.jpg
deepshit
14-12-2018, 03:12
Why xtool lz4 can't detect lz4 streams on gears of war 4?
but lz4 scanner detects lz4 streams in this game.
Or I can't get xtool work.
Razor12911
14-12-2018, 03:19
Why xtool lz4 can't detect lz4 streams on gears of war 4?
but lz4 scanner detects lz4 streams in this game.
Or I can't get xtool work.
it's not generic but specific :), lz4 precompressor in xtool works on games that are either COD, or that use Unity, Frostbite and Dunia2 engine
Chrushev
19-12-2018, 13:22
Trying to compress Dark Souls Remastered with Xtool but when I put the game in pack folder and run pack.bat it starts for a second, detects 6000 files and correct size... then just crashes... any ideas? Tried Xtool 10 and 9 using default pack.bat (no changes made at all, just put the game into pack folder)
darkwolves
02-01-2019, 16:12
GAME: Batman Arkham Knight 1.6.2 Game+All DLC
@yasitha
just use zlib even though it reads lzo they come up as bad lzo when you try to compress the files
darkwolves
02-01-2019, 20:10
thanks, i already done that.
you know what is causing this error ?
i am not certain... i get the same problem on the first injustice game as well.
i use uelr on it it tells me the lzo is bad just like on arkham knight.
Xtool lzo seems to have little to no effect on either game.
it will however work on the other batman arkham games oddly...
Love the XTool....
Hope New Update Come out Soon... :)
PUNISHMAN
20-01-2019, 23:57
xtool 0.10 not work windows 7 x64.
http://i63.tinypic.com/a0z6lj.png
Sergey3695
21-01-2019, 03:05
:D panicman
ROKA1969
26-01-2019, 22:16
Compressor v1.2.0:
https://i.imgur.com/8cejqPT.jpg
Identically v.2.1 does not work:
https://i.imgur.com/zZ4vIQh.jpg
I do not have a compression method with precomp. Compression stops at 11.6%. What is it? I'm using Compressor v1.2.0 and BlackBox v2 Designer. Other methods work.
And is it possible to add xtool to this compressor, if so, in what way?
Arc.ini:
https://i.imgur.com/zPzJah7.jpg
I added all necessary libraries to the folder with the compressor.And without result.
I do not have a compression method with precomp. Compression stops at 11.6%. What is it? I'm using Compressor v1.2.0 and BlackBox v2 Designer. Other methods work.
And is it possible to add xtool to this compressor, if so, in what way?
Everything seems OK to me, you just need patience
ROKA1969
27-01-2019, 08:41
I agree, everything is fine, just let me explain why the compression has stopped in place for almost two hours, the size of the game file has also increased from precomp. Example
800 mb: with mprecomp srep64 delta lzma64
700mb: msrep64 delta lzma64
xtool 0.10 not work windows 7 x64.
Put these 2 dll's to your xtool dir:
23987
PS(btw HI everyone -_^)
Sergey3695
28-01-2019, 07:29
21-01-2019, 10:57
Today, 16:46
u funny ).
xtool 0.10 not work windows 7 x64.
http://i63.tinypic.com/a0z6lj.png
Try replacing the libzstd.dll with this one
doofoo24
10-02-2019, 07:25
i done retesting xtool on large files dat far cry 5...
using xtool on multiple files at once like
[Dunia2]
File1=GAME\data_final\pc\worlds\farcry5.dat
File2=GAME\data_final\pc\patch.dat
seem to skip files (just copy)
using xtool on one file at time like
[Dunia2]
File1=GAME\data_final\pc\worlds\farcry5.dat
seem to work...
:confused:
using xtool 0.9 with setting
packcmd = xtool.exe e:precomp:t4,v,c32m:lz4 - - <stdin> <stdout>
^Old problem. The same with oodle codec. It just copy files if you precompress all at once. File by file works. Output will be bigger in this case, remind this.
doofoo24
10-02-2019, 11:16
ok so using xtool 0.6 on far cry 5 with "precomp:t4:lz4 - - <stdin> <stdout>"...
test it on 2 files
[Dunia2]
File1=pack\data_final\pc\patch.dat
File2=pack\data_final\pc\worlds\farcry5.dat
size 26.2gb after xtool from 18.9gb :eek:
Razor12911
27-02-2019, 14:50
Update available
Changes
- updated dunia engine lz4hc precompression support
Development
The source is messed up pretty badly :D so if there are certain affected codecs, use the older versions, those should work because the only thing I did here was update lz4 codec specifically the dunia2 support.
Results
FC New Dawn:
File1=common.dat
File2=patch.dat
File3=patch_english.dat
File4=patchshadersobj.dat
File5=shadersobj.dat
Compressed 12 files, 474,373,538 => 1,524,039,214 bytes. Ratio 321.27%
Compression time: cpu 0.84 sec/real 342.03 sec = 0%. Speed 1.39 mB/s
You can just go ahead and run a test on the rest of the game, edit the ini file to add more dat files if some weren't specified
I only installed the game today, updated the source just so it could work for it, didn't run full test because I don't have space :D
FC New Dawn
installpkg.dat
Compressed 2 files, 9,482,587,589 => 12,326,480,636 bytes. Ratio 129.99%
Compression time: cpu 24.69 sec/real 2642.03 sec = 1%. Speed 3.59 mB/s
All OK
Far Cry 5
common.dat
Compressed 2 files, 182,333,526 => 541,484,042 bytes. Ratio 296.97%
Compression time: cpu 0.36 sec/real 94.06 sec = 0%. Speed 1.94 mB/s
All OK
Far Cry New Dawn
fcbworlds.dat
fcbworlds.fat
fcbworlds_russian.dat
fcbworlds_russian.fat
fcbworlds_russian_feminine.dat
fcbworlds_russian_feminine.fat
installpkg.dat
installpkg.fat
installpkg_russian.dat
installpkg_russian.fat
installpkg_russian_feminine.dat
installpkg_russian_feminine.fat
Compressing 12 files, 19,853,255,088 => 27,315,496,302 bytes. Ratio 138.8%
can i skip
_feminine.dat * 0 bytes * .dat files?
i mean not include to xtool.ini?
Yes of course
Thanks bro.
I don't know but for me inflation just stops after a few percents if I took all files.
Can't Compress Far Cry 5 with new Xtool 011
File, one by one for testing it works...
but when i put all (.dat) files to xtool.ini files.
It takes 2h for decompressing, but actually it won't decompress files.. I don't know why.
Of anyone knows please share information..
so far What I've try,
I have check all files one by one, then i saw some files ratio 100% so i remove them from xtool.ini and,
all 0 bytes dat files too, then i have all 15 files and i try i though it works nope not working..
It seems xtool has a bug...
Please share if you know more information about this..
A little observation on the last XTool(v011) (Far Cry New Dawn)
[Dunia2]
BaseDir=pack\data_final\pc
File1=common.dat
File2=patch.dat
File3=patchshadersobj.dat
File4=shadersobj.dat
File5=worlds\fcbworlds.dat
File6=worlds\installpkg.dat-m=xtool:c16mb,t8:lz4
Compressed 12 files, 19,557,312,354 => 19,956,114,780 bytes. Ratio 102.04%
-m=xtool:c32mb,t8:lz4
Compressed 12 files, 19,557,312,354 => 28,107,674,136 bytes. Ratio 143.72%
-m=xtool:c64mb,t8:lz4
Compressed 12 files, 19,557,312,354 => 28,107,669,872 bytes. Ratio 143.72%
-m=xtool:c128mb,t8:lz4
Compressed 12 files, 19,557,312,354 => 28,107,668,960 bytes. Ratio 143.72%
-m=xtool:c512mb,t8:lz4
Compressed 12 files, 19,557,312,354 => 28,107,668,088 bytes. Ratio 143.72%
can you please fix for Far Cry 5?
I have Try more then 5, 6 Different way to pack all files, nope can't do that. but File by file can.
I got sorted out my previous problem. Whenever I include the HD pack file, it doesn't work. I will try with only hd pack file soon (maybe too large input?).
I got sorted out my previous problem. Whenever I include the HD pack file, it doesn't work. I will try with only hd pack file soon (maybe too large input?).
Yes maybe we need to leave smaill ratio and hd files.
Small ratio like under 105% cause i think that what cause this problem... I've try everything but i didn't try that one.
I have tried without HD files but i didn't try with low ratio like unde 105%,
on full game 106% ratio is not right.
Yes maybe we need to leave smaill ratio and hd files.
Small ratio like under 105% cause i think that what cause this problem... I've try everything but i didn't try that one.
I have tried without HD files but i didn't try with low ratio like unde 105%,
on full game 106% ratio is not right.
HD files have no compression at all.
HD files have no compression at all.
Yes i have done all testing...the problem is
When i pack all dat files at once with xtool.ini
Ratio like103% / 106% / 107%
What is happening?
you know what causing this?
I have test all the files. one at a time,
Most of them have under 105% ratio. So finally i have 10 files but still 107% on all files but one by one ratio like 124% / 224% / 142% / 151%
If you could help I'll appreciate it ☺
I have use this settings for all test..
"xtool.exe e:precomp:c128mb,t8:lz4"
but still can't pack all files at once,
when i do that, i get only 107% Ratio that's it!
if anyone could find a way to do that?
We can just use xtoil for FC5... simple..
I tell this problem to KaktoR and
He's working on it.
he had the same issues.
Can't pack add dat files at once
Compressed 8 files, 591,359,833 => 1,328,100,701 bytes. Ratio 224.58%
Compression time: cpu 0.56 sec/real 70.54 sec = 1%. Speed 8.38 mB/s
All OK
File1=common.dat
File2=patchshadersobj.dat
File3=shadersobj.dat
File4=ingameeditor\dlcpack0.dat
================================================== =======================
Compressed 2 files, 11,428,699,615 => 16,325,667,463 bytes. Ratio 142.85%
Compression time: cpu 15.08 sec/real 1088.64 sec = 1%. Speed 10.50 mB/s
All OK
File1=farcry5.dat
================================================== =======================
Compressed 2 files, 10,490,400,710 => 13,634,127,931 bytes. Ratio 129.97%
Compression time: cpu 14.75 sec/real 1675.14 sec = 1%. Speed 6.26 mB/s
All OK
File1=installpkg.dat
================================================== =======================
Compressed 2 files, 2,369,686,646 => 3,136,552,876 bytes. Ratio 132.36%
Compression time: cpu 2.67 sec/real 272.47 sec = 1%. Speed 8.70 mB/s
All OK
File1=igepack.dat
================================================== =======================
Compressed 2 files, 4,371,045,970 => 6,638,770,784 bytes. Ratio 151.88%
Compression time: cpu 5.27 sec/real 1233.67 sec = 0%. Speed 3.54 mB/s
All OK
File1=dlc_vietnam.dat
================================================== =======================
Compressed 2 files, 4,908,269,998 => 7,304,500,533 bytes. Ratio 148.82%
Compression time: cpu 6.83 sec/real 1384.01 sec = 0%. Speed 3.55 mB/s
All OK
File1=dlc_mars.dat
================================================== =======================
Compressed 2 files, 8,969,186,606 => 11,724,942,902 bytes. Ratio 130.72%
Compression time: cpu 13.00 sec/real 1007.75 sec = 1%. Speed 8.90 mB/s
All OK
File1=patch.dat
================================================== =======================
Compressed 20 files, 43,128,649,378 => 46,243,558,086 bytes. Ratio 107.22%
Compression time: cpu 73.31 sec/real 2781.96 sec = 3%. Speed 15.50 mB/s
All OK
File1=common.dat
File2=patch.dat
File3=patchshadersobj.dat
File4=shadersobj.dat
File5=downloadcontent\dlc_mars\dlc_mars.dat
File6=downloadcontent\dlc_vietnam\dlc_vietnam.dat
File7=ingameeditor\dlcpack0.dat
File8=ingameeditor\igepack.dat
File9=worlds\farcry5.dat
File10=worlds\installpkg.dat
i have leave 0 bytes files and none lzo,lz4 streams files
and under 105% ratio files. so that is why total files 10 :)
================================================== =========================
All (.Dat) files
common.dat Found Streams lz4
ige.dat - NO STREAMERS FOUND
igepatch.dat - NO STREAMERS FOUND
patch.dat Found Streams lz4
patch_english.dat - 103.87% low Streams lz4
patchshadersobj.dat Found Streams lz4
shadersobj.dat Found Streams lz4
downloadcontent\dlc_mars\dlc_mars.dat Found Streams lz4
downloadcontent\dlc_mars\dlc_mars_english.dat 102.53% low Streams lz4
downloadcontent\dlc_mars\dlc_mars_hd.dat - NO STREAMERS FOUND
downloadcontent\dlc_vietnam\dlc_vietnam.dat Found Streams lz4
downloadcontent\dlc_vietnam\dlc_vietnam_english.da t 102.20% low Streams lz4
downloadcontent\dlc_vietnam\dlc_vietnam_hd.dat - NO STREAMERS FOUND
downloadcontent\dlc_zombies\dlc_zombies.dat - NO STREAMERS FOUND
ingameeditor\dlcpack0.dat Found Streams lz4
ingameeditor\dlcpack0_hd.dat - NO STREAMERS FOUND
ingameeditor\igepack.dat Found Streams lz4
ingameeditor\igepack_hd.dat - NO STREAMERS FOUND
worlds\farcry5.dat Found Streams lz4
worlds\farcry5_english.dat 103.54% low Streams lz4
worlds\farcry5_hd.dat - NO STREAMERS FOUND
worlds\installpkg.dat Found Streams lz4
worlds\installpkg_english.dat 104% low Streams lz4
worlds\installpkg_hd.dat - NO STREAMERS FOUND
I have tested aswell with FC5 now.
Output is way bigger if you unpack all files one by one then unpack all at once (keep archives).
Something is wrong :D
I have tested aswell with FC5 now.
Output is way bigger if you unpack all files one by one then unpack all at once (keep archives).
Something is wrong :D
so this is xtool problem? :D
I don't know. Without doing math, average should be around 130% - 150% on all files.
I will try with xtool 0.6 (it was introducing dunia engine support).
I don't know. Without doing math, average should be around 130% - 150% on all files.
I will try with xtool 0.6 (it was introducing dunia engine support).
Oky try it, i think we can reduce 8GB or 10GB... 🙂
If this works..
It's a known bug with FC5 files, it's discussed above many times. Until Razor takes a look at it, nothing can be done.
It's a known bug with FC5 files, it's discussed above many times. Until Razor takes a look at it, nothing can be done.
so how did you manage to pack the game?
anyone knows how to Pack Watch Dogs 1 and 2 using Precompressor? I jave try WD2 won't work..
Streams were used on game:
Watch Dogs 1 LZX
Watch Dogs 2 LZ4
so how did you manage to pack the game?
Delzorec
Razor12911
06-03-2019, 02:15
It's a known bug with FC5 files, it's discussed above many times. Until Razor takes a look at it, nothing can be done.
xtool is broken, it has to be rewritten from scratch :D
xtool is broken, it has to be rewritten from scratch :D
Can you Do it? :D any soon?
doofoo24
06-03-2019, 05:58
xtool is broken, it has to be rewritten from scratch :D
isn't better to just separate xtool to pzlib/plzo/plz4, less headache to keep updating xtool every time new game comes...
just release separate tools for any new game like you been doing for zstd/oddle...
Razor12911
06-03-2019, 08:51
isn't better to just separate xtool to pzlib/plzo/plz4, less headache to keep updating xtool every time new game comes...
just release separate tools for any new game like you been doing for zstd/oddle...
no it isn't because if there is an issue that I haven't come across before then it means when I try to fix it for that particular codec or source in what you want me to try and do, I will have to fix it on all of them, the reason I combined them in the first place is that when I add a feature or fix an issue, all receive the same treatment. 90% of the code will be repeated in all the separate sources, it's actually this 90% that requires fixing because bugs related to multi threading, threads getting stuck on some loop or them not responding because of some error they couldn't handle or when a user cancels setup and it doesn't respond. The 10% is actually the code that gives precompression capabilities, this one isn't affected at all. Since it is 10%, it means it is small code, I didn't see the reason not to combine everything.
doofoo24
06-03-2019, 08:54
good to know :)
Waiting for the new Release of Xtool_012 :D with bug fixed
@Razor12911
Could you add support for DOS2DE? It use lz4hc but doesn't work with latest xtool version unfortunatelly.
Spoiler Alert. Xtool 012 will be release soon... :D
Razor12911
09-03-2019, 14:06
@Razor12911
Could you add support for DOS2DE? It use lz4hc but doesn't work with latest xtool version unfortunatelly.
tf is DOS2DE? :confused:
Divinity Original Sin 2 Devinitive Edition :D
Razor12911
10-03-2019, 00:58
Divinity Original Sin 2 Devinitive Edition :D
sample?
sample?
Hi Razor can u do Metro Exodus!?
Razor12911
10-03-2019, 01:22
Hi Razor can u do Metro Exodus!?
I know 0% about this game and I don't even have it
Hi Razor12911
Could you add support for Traveller's Tales games DAT files?
QuickBMS Script
# Traveller's Tales games DAT files extractor (script 0.9.11a)
# Lego Movie: The Video Game
# LEGO Batman 1
# LEGO Batman 2
# LEGO Star Wars
# LEGO Star Wars III
# LEGO Indiana Jones
# LEGO Harry Potter
# LEGO Pirates of the Caribbean
# LEGO Lord of the Rings
# Transformers
# LEGO Worlds
# LEGO MARVELs Avengers
# LEGO Dimensions
# Lego Marvel Super Heroes
# Lego Star Wars: The Force Awakens
# Lego NinjaGO
# others, check them on http://www.ttgames.com
#
# In case of problems with the extraction try setting NAMELESS to 1
# thanx to who fixed the handling of the names!
#
# Note that the script may not work with the compressed files of
# Lego Movie for PS3, in case of crashes or other issues you can
# extract only the non compressed files by setting
# EXTRACT_COMPRESSED to 0.
#
# script for QuickBMS http://quickbms.aluigi.org
math FORCE_CRC_BITS = 0 # use 32, 64 or 0 for auto-guess, just in case auto-guess fails
quickbmsver "0.8.0"
set NAMELESS long 0 # set to 1 to extract the files without names
set EXTRACT_COMPRESSED long 1 # set to 0 to extract ONLY the non-compressed files
math HAVE_CRCS = 0
math CRC_FNV_OFFSET = 0x811c9dc5
math CRC_FNV_PRIME = 0x199933
set SIGN_1234567a binary "\x7a\x56\x34\x12"
getdstring SIGN 4
goto 0
if SIGN u== SIGN_1234567a
goto 0
callfunction EXTRACT_1234567a 1
cleanexit
endif
string SIGN f SIGN
math ONE_FILE = 0
if SIGN == "LZ2K"
math ONE_FILE = 1
elif SIGN == "DFLT"
math ONE_FILE = 1
elif SIGN == "ZLIB" # currently doesn't exist
math ONE_FILE = 1
elif SIGN == "LZMA" # currently doesn't exist
math ONE_FILE = 1
elif SIGN == "ZIPX"
math ONE_FILE = 1
elif SIGN == "RFPK"
math ONE_FILE = 1
elif SIGN == "RNC_"
math ONE_FILE = 1
elif SIGN == "Defl" # "Deflate_v1.0"
goto 0
math OFFSET = 0
get SIZE asize
get NAME basename
get EXT extension
string NAME + "_unpack."
string NAME + EXT
callfunction DEFLATE_UNPACK 1
cleanexit
endif
if ONE_FILE != 0
math OFFSET = 0
get ZSIZE asize
math SIZE = ZSIZE
callfunction UNPACK
log "" 0 SIZE MEMORY_FILE2
cleanexit
endif
get HDR_NAME basename
string HDR_NAME += ".hdr"
open FDSE HDR_NAME 1 EXISTS
if EXISTS != 0
get NAME filename
if NAME == HDR_NAME
get NAME basename
string NAME += ".dat"
open FDSE NAME
endif
callfunction HANDLE_COMPRESSED_DAT 1
get HDR_SIZE asize 1
log MEMORY_FILE 0 HDR_SIZE 1
get DUMMY long MEMORY_FILE
else
callfunction HANDLE_COMPRESSED_DAT 1
get INFO_OFF long
if INFO_OFF & 0x80000000
math INFO_OFF ^= 0xffffffff
math INFO_OFF <<= 8
math INFO_OFF += 0x100
endif
get INFO_SIZE long
log MEMORY_FILE INFO_OFF INFO_SIZE
endif
get TYPE_BOH signed_long MEMORY_FILE
get FILES long MEMORY_FILE
math NEW_FORMAT = 1
if TYPE_BOH != 0x3443432e # game.hdr
if TYPE_BOH != 0x2e434334
if FILES != 0x3443432e # game.dat
if FILES != 0x2e434334
math NEW_FORMAT = 0
endif
endif
endif
endif
if NEW_FORMAT == 1 # ".CC400TAD"
goto 0 MEMORY_FILE
endian big
get HDR_SIZE long MEMORY_FILE
idstring MEMORY_FILE ".CC4"
idstring MEMORY_FILE "0TAD"
get TYPE_BOH signed_long MEMORY_FILE # -8
get NEW_FORMAT_VER long MEMORY_FILE # 1
get FILES long MEMORY_FILE
get NAMES long MEMORY_FILE
get NAMES_SIZE long MEMORY_FILE
savepos NAMES_OFF MEMORY_FILE
math OFFSET = NAMES_OFF
math OFFSET + NAMES_SIZE
goto OFFSET MEMORY_FILE
putarray 10 NAMES "" # necessary and useful
putarray 1 NAMES "" # necessary and useful
get DUMMY long MEMORY_FILE
math MYID = 0
xmath LAST_NAME_WORKAROUND "NAMES - 1"
for i = 0 < NAMES
get NAME_OFF long MEMORY_FILE
get FOLDER_ID short MEMORY_FILE
if NEW_FORMAT_VER >= 2
get DUMMY_ID short MEMORY_FILE
endif
get SOME_ID signed_short MEMORY_FILE
get FILE_ID short MEMORY_FILE
if NAME_OFF != 0xffffffff
savepos TMP MEMORY_FILE
math NAME_OFF + NAMES_OFF
goto NAME_OFF MEMORY_FILE
get NAME string MEMORY_FILE
goto TMP MEMORY_FILE
# I will find a solution another day maybe...
if i == LAST_NAME_WORKAROUND
math FILE_ID = MYID
endif
getarray PATH 1 FOLDER_ID
string NAME p "%s\%s" PATH NAME
if FILE_ID != 0
putarray 10 MYID NAME
math MYID + 1
else
putarray 1 i NAME
endif
endif
next i
get TYPE_BOH signed_long MEMORY_FILE # -8
get FILES long MEMORY_FILE
# performance
putarray 2 FILES 0
putarray 3 FILES 0
putarray 4 FILES 0
putarray 5 FILES 0
for i = 0 < FILES
if TYPE_BOH <= -11
get OFFSET longlong MEMORY_FILE
else
get OFFSET long MEMORY_FILE
endif
get ZSIZE long MEMORY_FILE
get SIZE long MEMORY_FILE
if TYPE_BOH <= -10
math PACKED = SIZE
math SIZE & 0x7fffffff
math PACKED u>> 31
if PACKED != 0
math PACKED = 2 # useless, but we need & 2
endif
else
get PACKED byte MEMORY_FILE
get ZERO short MEMORY_FILE
get OFFSET2 byte MEMORY_FILE
math OFFSET u<< 8
math OFFSET | OFFSET2
endif
putarray 2 i OFFSET
putarray 3 i ZSIZE
putarray 4 i SIZE
putarray 5 i PACKED
next i
callfunction GET_CRCS 1
get EXT extension
if EXT & "DAT" # both DAT and DAT2
else
open FDDE "DAT" # in case HDR has been opened
endif
for i = 0 < FILES
getarray FULLNAME 10 i
callfunction GET_NAME 1
getarray OFFSET 2 IDX
getarray ZSIZE 3 IDX
getarray SIZE 4 IDX
getarray PACKED 5 IDX
callfunction EXTRACT_FILE 1
next i
else
savepos INFO_OFF MEMORY_FILE
math TMP = FILES
math TMP *= 16
math NAME_INFO = INFO_OFF
math NAME_INFO += TMP
goto NAME_INFO MEMORY_FILE
get NAMES long MEMORY_FILE
savepos NAME_INFO MEMORY_FILE
math NAME_FIELD_SIZE = 8
if TYPE_BOH <= -5
math NAME_FIELD_SIZE = 12
endif
math TMP = NAMES
math TMP *= NAME_FIELD_SIZE
math NAME_OFF = NAME_INFO
math NAME_OFF += TMP
goto NAME_OFF MEMORY_FILE
get NAMECRC_OFF long MEMORY_FILE
savepos NAME_OFF MEMORY_FILE
math NAMECRC_OFF += NAME_OFF
goto NAMECRC_OFF MEMORY_FILE
callfunction GET_CRCS 1
if TYPE_BOH <= -2
get DUMMY1 signed_long MEMORY_FILE
get DUMMY2 long MEMORY_FILE
endif
# print "files: %FILES%"
# print "names: %NAMES%"
# print "info_off: %INFO_OFF%"
# print "info_size: %INFO_SIZE%"
# print "name_info: %NAME_INFO%"
# print "name_off: %NAME_OFF%"
# print "namecrc_off: %NAMECRC_OFF%"
set NAMEZ long 0
set FULLNAME string ""
set FULLPATH string ""
for i = 0 < FILES
callfunction SET_NAME 1
callfunction GET_NAME 1
math IDX *= 16
math IDX += INFO_OFF
goto IDX MEMORY_FILE
get OFFSET long MEMORY_FILE
get ZSIZE long MEMORY_FILE
get SIZE long MEMORY_FILE
get PACKED threebyte MEMORY_FILE
get OFFSET2 byte MEMORY_FILE
if TYPE_BOH == -1
# do nothing
else
math OFFSET <<= 8
endif
math OFFSET += OFFSET2
callfunction EXTRACT_FILE 1
next i
endif
startfunction GET_CRCS
math HAVE_CRCS = 0
putarray 0 FILES 0
putarray 11 FILES 0
get TMP1 asize MEMORY_FILE
savepos TMP2 MEMORY_FILE
if TMP2 u< TMP1
xmath TMP4 "TMP2 + (FILES * 4)"
xmath TMP8 "TMP2 + (FILES * 8)"
savepos TMP MEMORY_FILE
goto TMP4 MEMORY_FILE
get ZERO long MEMORY_FILE
if ZERO == 0 # this is just a test because there is no good way to guess if it's 64 or 32bit
math TMP8 = TMP1 # disable 64bit false positives
endif
goto TMP MEMORY_FILE
math CRC_BITS = 32
if TMP8 u< TMP1 && NEW_FORMAT_VER >= 2 # Lego NinjaGo
math CRC_BITS = 64
endif
if FORCE_CRC_BITS >= 32
math CRC_BITS = FORCE_CRC_BITS
endif
if CRC_BITS == 64
callfunction QUICKBMS_4GB_CHECK 1
math CRC_FNV_OFFSET = 0xcbf29ce484222325
math CRC_FNV_PRIME = 1099511628211
math HAVE_CRCS = 2
for i = 0 < FILES
get CRC longlong MEMORY_FILE
putarray 0 i CRC
putarray 11 i 0 # file has been extracted?
next i
elif TMP4 u< TMP1 # no need to check CRC_BITS == 32
math HAVE_CRCS = 1
for i = 0 < FILES
get CRC long MEMORY_FILE
putarray 0 i CRC
putarray 11 i 0 # file has been extracted?
next i
endif
endif
if HAVE_CRCS == 0
for i = 0 < FILES
putarray 0 i 0
putarray 11 i 0 # file has been extracted?
next i
endif
endfunction
startfunction EXTRACT_FILE
endian little # necessary, they are little endian on big endian archives too
goto OFFSET
getdstring SIGN 4
string SIGN f SIGN
if SIGN == "LZ2K"
set PACKED long 2
elif SIGN == "DFLT"
set PACKED long 3
elif SIGN == "ZLIB"
set PACKED long 3
elif SIGN == "LZMA"
set PACKED long 3
elif SIGN == "ZIPX"
set PACKED long 3
elif SIGN == "RFPK"
set PACKED long 3
elif SIGN == "RNC_"
set PACKED long 3
else
set PACKED long 0
endif
if PACKED & 2 # includes both 2 and 3
if EXTRACT_COMPRESSED != 0
callfunction UNPACK
log FULLNAME 0 SIZE MEMORY_FILE2
endif
else
if SIZE != 0
if ZSIZE != 0
if SIZE != ZSIZE
print "SIZE (%SIZE%) and ZSIZE (%ZSIZE%) differ at offset %OFFSET|x%, contact me"
cleanexit
endif
endif
endif
log FULLNAME OFFSET SIZE
endif
endfunction
startfunction SET_NAME
do
goto NAME_INFO MEMORY_FILE
get NEXT signed_short MEMORY_FILE
get PREV signed_short MEMORY_FILE
get OFF signed_long MEMORY_FILE
if TYPE_BOH <= -5 # if NAME_FIELD_SIZE >= 12
get DUMMY long MEMORY_FILE
endif
savepos NAME_INFO MEMORY_FILE
if OFF < 0
set NAME string ""
else
math OFF += NAME_OFF
goto OFF MEMORY_FILE
get NAME string MEMORY_FILE
endif
# used only for LEGO the game if you don't use the hdr file
getvarchr TMP0 NAME 0
if TMP0 >= 0xf0
set NAME string ""
endif
#string NAME u= NAME # needed for the crc check, but doesn't improve performances so much
if PREV != 0
getarray FULLPATH 1 PREV
endif
putarray 1 NAMEZ FULLPATH # putarray 1 NAMEZ NAME
if NEXT > 0 # folder
getarray TMP 1 PREV
if TMP != "" # long story to avoid things like 2foldername that gives problems to QuickBMS
set OLDNAME string \ # do not use /
string OLDNAME += TMP
string OLDNAME += \ # do not use /
string FULLPATH >>= OLDNAME
endif
if NAME != ""
#string FULLPATH += \ # do not use /
string FULLPATH += NAME
string FULLPATH += \ # do not use /
endif
endif
math NAMEZ += 1
while NEXT > 0
set FULLNAME string FULLPATH
string FULLNAME += NAME
endfunction
startfunction GET_CURRENT_NAME
for j = 0 < FILES
getarray TMP 11 j
if TMP == 0 # the first file not extracted yet
math IDX = j
break
endif
next j
endfunction
startfunction GET_NAME
if HAVE_CRCS == 0
callfunction GET_CURRENT_NAME 1
else
getvarchr TMP0 FULLNAME 0
if TMP0 == '\\'
string FULLNAME <<= 1
endif
math IDX = i
if NAMELESS == 0
strlen NAMESZ FULLNAME
string FULLNAME u= FULLNAME
string FULLNAME R= / \
math CRC = CRC_FNV_OFFSET
for j = 0 < NAMESZ
getvarchr CHR FULLNAME j
math CRC ^= CHR
math CRC *= CRC_FNV_PRIME
next j
if HAVE_CRCS <= 1
math CRC &= 0xffffffff # quickbms_4gb_files
endif
for j = 0 < FILES
getarray TMP 0 j # CRC database
if CRC == TMP
math IDX = j
break
endif
next j
if j >= FILES
print "Alert: the crc of the file %FULLNAME% has not been found, I extract the current file"
callfunction GET_CURRENT_NAME 1
endif
else
set FULLNAME string ""
endif
endif
putarray 11 IDX 1 # the file has been extracted
endfunction
startfunction UNPACK
putvarchr MEMORY_FILE2 SIZE 0
log MEMORY_FILE2 0 0
append
for TMPSZ = 0 < ZSIZE
goto OFFSET
getdstring SIGN 4
string SIGN f SIGN
if SIGN == "LZ2K"
comtype lz2k
get CHUNK_SIZE long
get CHUNK_ZSIZE long
elif SIGN == "DFLT"
comtype dflt
get CHUNK_ZSIZE long
get CHUNK_SIZE long
elif SIGN == "ZLIB" # currently doesn't exist
comtype zlib
get CHUNK_ZSIZE long
get CHUNK_SIZE long
elif SIGN == "LZMA" # currently doesn't exist
comtype lzma
get CHUNK_ZSIZE long
get CHUNK_SIZE long
elif SIGN == "ZIPX"
comtype deflate # ??? not tested
get CHUNK_ZSIZE long
get CHUNK_SIZE long
string TMP = CHUNK_ZSIZE
encryption rc4 TMP "" 0 4
elif SIGN == "RFPK"
comtype rfpk
get CHUNK_ZSIZE long
get CHUNK_SIZE long
elif SIGN == "RNC_"
comtype rnc
math CHUNK_SIZE = SIZE
math CHUNK_ZSIZE = ZSIZE
goto OFFSET
else
print "Error: the compressing signature at offset %OFFSET% (%SIGN%) is not known, contact me"
cleanexit
endif
savepos OFFSET
if CHUNK_ZSIZE == CHUNK_SIZE
log MEMORY_FILE2 OFFSET CHUNK_SIZE
else
clog MEMORY_FILE2 OFFSET CHUNK_ZSIZE CHUNK_SIZE
endif
encryption "" ""
math OFFSET += CHUNK_ZSIZE
savepos TMP MEMORY_FILE2
math TMPSZ += 12
math TMPSZ += CHUNK_ZSIZE
next
append
endfunction
startfunction HANDLE_COMPRESSED_DAT
goto 0
getdstring TMP 16
if TMP == "CMP2CMP2CMP2CMP2"
comtype lzma
getdstring DAT_HASH 16
get DAT_SIZE longlong
savepos DAT_OFFSET
get DAT_ZSIZE asize
math DAT_ZSIZE - DAT_OFFSET
clog TEMPORARY_FILE DAT_OFFSET DAT_ZSIZE DAT_SIZE
open "." TEMPORARY_FILE
endif
goto 0
endfunction
startfunction EXTRACT_1234567a
idstring "\x7a\x56\x34\x12" # 0x1234567a
get FILES long
get PAK_SIZE long
get CRC long
get ZERO long
get ZERO long
for i = 0 < FILES
get NAME_OFF long
get OFFSET long
get SIZE long
get DUMMY long # it's ever 4 in any case
get ZERO long
get CRC long
get ZERO long
savepos TMP_OFF
goto NAME_OFF
get NAME string
goto OFFSET
callfunction DEFLATE_UNPACK 1
goto TMP_OFF
next i
endfunction
startfunction DEFLATE_UNPACK
getdstring SIGN 0x20
if SIGN == "Deflate_v1.0"
comtype dflt
get XSIZE long
math OFFSET + 0x24
math SIZE - 0x24
clog NAME OFFSET SIZE XSIZE
else
log NAME OFFSET SIZE
endif
endfunction
startfunction QUICKBMS_4GB_CHECK
math TMP64 = 0x10000000
math TMP64 * 16
if TMP64 == 0
print "you must use quickbms_4gb_files.exe with this archive"
cleanexit
endif
endfunction
sample?
Here are two samples
Snake288
10-03-2019, 03:44
https://i.hizliresim.com/jgDn9W.jpg
Hi Razor can u do Metro Exodus!?
Here are two samples
Hi Razor12911
Could you add support for Traveller's Tales games DAT files?[/CODE]
All compressors used in TT games are custom ones and have only decompressors available. So there is no way to pack them back.
hey razor you should get paid for all the work you do:D:D;)
hey razor you should get paid for all the work you do:D:D;)
I have told about that long ago same thing :D
Razor12911
10-03-2019, 14:10
Here are two samples
These don't seem to follow, the 16-byte header which is this part 24320
hey razor you should get paid for all the work you do:D:D;)
yea I thought about that before but then you should also pay guys like ProFrager, Edison007 and etc. None of those guys get anything so I don't want to differ ;)
^Strange because export tool unpack and pack with lz4hc (byte perfect). At least this is not a Unity Engine game at all.
Thanks for your efforts anyway ;)
Razor12911
10-03-2019, 16:54
the headers are placed elsewhere bro. The files you gave me do have signs of lz4hc being used but there are no headers for me to reference for information as to where streams start and where they end. This will be a job for a universal lz4 precompressor but not at the moment because lz4 is a piece of shit when it comes to that department, as you can see in the screenshot I uploaded.
gives two byte header "F2 00" that I have no idea what it means then followed by parts of the original data, "44 44 53 20" which is a header for a dds texture which means lz4hc was indeed used but that's it. no game engine headers located or anything that could help for me to be able to add support.
ShivShubh
10-03-2019, 21:47
Hi Razor can u do Metro Exodus!?
I don't think Metro Exodus has any compression. It just uses a type of texture encoding that can't be further compressed that much.
I don't think Metro Exodus has any compression. It just uses a type of texture encoding that can't be further compressed that much.
Metro Exodus use, Zlib but encrypted,
Try game scanner with Dynamic enabled you'll see...
and i have able to get
47GB (51,323MB)
to
29.3 (31,541MB)
Using srep + lolz
@yasitha
Even if you scan a TXT file you will get dynamic streams. It means nothing.
@Razor12911
Thanks bro!
@yasitha
Even if you scan a TXT file you will get dynamic streams. It means nothing.
@Razor12911
Thanks bro!
So you mean no compressor used on Metro Exodus at all?
ShivShubh
11-03-2019, 08:17
So you mean no compressor used on Metro Exodus at all?
Nope unless FitGirl has something else to say here. As I said before, its textures can't be compressed that much when compared to normal DXT textures.
IgorKolesnik
14-03-2019, 00:04
help please solve the problem. When using xtool for large files, this error occurs during installation.
I use this method: xtool+srep+delta+lolz
options xtool:
[External compressor:xtool]
header = 0
packcmd = xtool e:precomp:t2:zlib - - <stdin> <stdout>
bat file:
arc.exe a -w.\ -ep1 -dses --dirs -s; -lc- -di -i2 -r -mxtool+srep+delta+lolz data1.bin "packeddata2\*"
xtool v011
https://i1.imageban.ru/out/2019/03/13/8e1215a0bf73996d9f41dab3b81b57d7.jpg
help please solve the problem. When using xtool for large files, this error occurs during installation.
I use this method: xtool+srep+delta+lolz
options xtool:
[External compressor:xtool]
header = 0
packcmd = xtool e:precomp:t2:zlib - - <stdin> <stdout>
bat file:
arc.exe a -w.\ -ep1 -dses --dirs -s; -lc- -di -i2 -r -mxtool+srep+delta+lolz data1.bin "packeddata2\*"
xtool v011
https://i1.imageban.ru/out/2019/03/13/8e1215a0bf73996d9f41dab3b81b57d7.jpg
Use xtool+srep+lolz don't use delta.
Try xtool e:precomp:c32mb,t75p:zlib
srep:m3f:l512:c512:a4+lolz:dtb1:d128m:mtt1:mt4
Mt4 need 4 threads. If you have 8 threads use 8 instead of 4
Tip: try single larger file and try decompress. If it works try full game...
Try change chunk size to 128 (worked for all games until now)
xprecomp:t2,c128m:zlib
IgorKolesnik
14-03-2019, 03:58
Thanks guys, the problem seems to be gone. I will test.
By the way, I noticed that xtool needs vcc 2013 to work, vcc 2015 and newer for some reason does not work.
bunti_o4u
14-03-2019, 23:01
Try change chunk size to 128 (worked for all games until now)
xprecomp:t2,c128m:zlib
Tested but it's not working..
I have compress single 35GB file with c32mb,
It's not the problem. You get Decompressing error right?
If you have any other PC. try decompressing on that pc.
Maybe your compression is fine your pc is the problem...
doofoo24
15-03-2019, 00:36
nah!
i'm guessing that he use different version of xtool for install...
just use the same one if you compress with 0.9 use 0.9 for install...
bunti_o4u
15-03-2019, 00:58
I have compress single 35GB file with c32mb,
It's not the problem. You get Decompressing error right?
If you have any other PC. try decompressing on that pc.
Maybe your compression is fine your pc is the problem...
It's not working means the setting has no effect and files are not getting deflated..
Xtool abandoned?
where's the Update?
Xtool abandoned?
where's the Update?
I don't think xtool has anything else to offer
You may only need to fix some bugs and fully reconcile old ZTool items so ZTool can be removed from CIU.
Before I asked KaktoR:
Why keep ZTool and XTool in CIU, if XTool is ZTool's successor? Does XTool have bugs to keep both in CIU?
Kaktor replied:
No XTool is better (only bug is that big data input does not get unpacked with oodle and lzo codec. All other codecs work).
ZTool for example don't support crilayla codec (which is mostly used in the "anime-like" games like DragonBall, OnePiece, Sonic and etc).
So if you like to remove one of them, then remove ZTool.
L0v3craft
10-05-2019, 13:15
Hi guys, someone knows how to pre-compress ...\Content\Paks\*.pak files when they are crypted in games developed with unreal engine ? Usually we can use ztool or xtool/zlib to pre compress the pak files, but when they are crypted ? What to do :confused:
Edit: i found the way to get the decryption key(a man from zenhax is able to get it), but how to use the key with xtool ?
L0v3craft, as far as i know you need to unpack the archive with the key, compress it's content and then re-import the files back.
L0v3craft
10-05-2019, 19:38
L0v3craft, as far as i know you need to unpack the archive with the key, compress it's content and then re-import the files back.
I can extract it with quickbms, but you mean re import with quickbms ? Is possible to implement it with inno in the way that inno can shows the installation with a progress bar ?
you mean re import with quickbms ?
Yes.
Is possible to implement it with inno in the way that inno can shows the installation with a progress bar ?
Maybe, haven't tried yet.
Razor12911
16-06-2019, 04:54
Update available (.. or rather update was available, I forgot to post this)
Changes
- added mk11 lzna support
Status
Project is halted, and to be replaced with this (https://fileforums.com/showthread.php?p=481382)
Thanks for this. Did you fixed the big input bug yet? Or is this on your to-do list? :)
Razor12911
16-06-2019, 05:03
to do list, I have chosen to rewrite the project from scratch, surely that will fix many if not all issues but who knows :)
darkwolves
16-06-2019, 19:33
Update available (.. or rather update was available, I forgot to post this)
Changes
- added mk11 lzna support
Status
Project is halted, and to be replaced with this (https://fileforums.com/showthread.php?p=481382)
inflation test of mk11 successful i am now repacking it and will do a test unpacking
inflation test of mk11 successful i am now repacking it and will do a test unpacking
Don't expect good ratio, LZNA is the best compression algo in Oodle family.
Razor12911
20-06-2019, 15:12
@everyone
I'm now focusing on the main code of XTool 2019, so now it's a good time to mention all the bugs the current xtool has so I can try to avoid as I write the new code from scratch and what new features would you like to be added in the upcoming release.
^^
For oodle codec on AC:Origins and Odyssey: XTool only unpack a small amount of forge files in the beginning, then it just copy the rest of the files (ratio is somewhat by ~102% overall if I remember right).
I think there was something with lz4hc on Far Cry too (I just don't remember what it was exactly, I'm sorry).
Would be good if you can add lzo for unreal engine games (borderlands, ME1+2, etc) and for the older forge engine games aswell.
Btw, what about oodle3 codec? Destiny 2 is using this and I can't get it with xtool nor your oodle tools released previously (oo2recx).
Razor12911
21-06-2019, 10:36
I think it is also best if you provide samples and the required resources to make precompression possible, like dlls. You could start with oodle... I just hope the headers are still the same
Here are a few samples
Destiny 2 (oodle 3) (https://drive.google.com/open?id=1qRN0tU7ADAfupLdgffoCUuWFBY00q897)
Borderlands 2 Remastered (unreal engine lzo) (https://drive.google.com/open?id=1cBQ5Yic9QvKXOvFCPTImfP8GDHJDhOJ8)
Borderlands PreSequel Remastered (unreal engine lzo) (https://drive.google.com/open?id=1zwNQYg1XlW6xNhGgL2r13CX9q_ybA2b8)
Mass Effect (unreal engine lzo) (https://drive.google.com/open?id=115OOjeeNvtyhuGyctJSPqXo4qIv5pWH7)
Mass Effect 2 (unreal engine lzo) (https://drive.google.com/open?id=1xb8tkVrS3cwXaoFOMsojhZNq-KqB2DUo)
AC Odyssey (forge engine oodle 4) (https://drive.google.com/open?id=1lIt36in13dtef8RAFbOtxeW0eatSYLoz)
Maybe more to come...
Razor12911
21-06-2019, 18:09
Here are a few samples
Destiny 2 (oodle 3) (https://drive.google.com/open?id=1qRN0tU7ADAfupLdgffoCUuWFBY00q897)
Borderlands 2 Remastered (unreal engine lzo) (https://drive.google.com/open?id=1cBQ5Yic9QvKXOvFCPTImfP8GDHJDhOJ8)
Borderlands PreSequel Remastered (unreal engine lzo) (https://drive.google.com/open?id=1zwNQYg1XlW6xNhGgL2r13CX9q_ybA2b8)
Mass Effect (unreal engine lzo) (https://drive.google.com/open?id=115OOjeeNvtyhuGyctJSPqXo4qIv5pWH7)
Mass Effect 2 (unreal engine lzo) (https://drive.google.com/open?id=1xb8tkVrS3cwXaoFOMsojhZNq-KqB2DUo)
AC Odyssey (forge engine oodle 4) (https://drive.google.com/open?id=1lIt36in13dtef8RAFbOtxeW0eatSYLoz)
Maybe more to come...
if it's an unreal engine game compressed with lzo then for precompression, you must provide lzopro, standard lzo cannot restore any of the streams. If you use ztool/xtool but skip verification, you'll notice that the precompressor does see the streams but just for the fact that it cannot restore them with perfect crc, it just skips all of them. The reason I added skip verification was the fact that if it ever came to a point where one cannot really precompress a game with perfect crc, it should then at least allow you to try to pack a game with different crc but then also allow you to repack it, example of this was DmC: Devil May Cry, also unreal engine game compressed with lzo, I couldn't use xtool to precompress game but used skip verification, the game files were different but the game played from start to finish without crashing plus I managed to save space after repacking it.
as for Odyssey, probably some internal issue but the scanner does see all the streams in the samples
DataPC_ACD_Greece_DX11.forge:
Universal Oodle stream scanner
Created by Razor12911
[0] = Unknown/Invalid
[1] = Kraken/Hydra
[2] = Mermaid/Selkie/Hydra
[3] = Leviathan/Hydra
0001[2]| Pos: 00004CE4, Size: 13359
0002[2]| Pos: 00008117, Size: 13271
0003[2]| Pos: 0000B4F2, Size: 25162
0004[2]| Pos: 00011740, Size: 25029
....
0387[2]| Pos: 00856A2E, Size: 26334
0388[2]| Pos: 0085D110, Size: 13314
0389[2]| Pos: 00860568, Size: 21767
0390[2]| Pos: 00865A73, Size: 23507
Done!!!
for Destiny, I think lzna was used on this game just like mk11 (I see a lot of 0x058c headers everywhere), support for lzna is not added at the moment, but I obviously need to confirm this.
hungeuro
21-06-2019, 20:45
Wooh nice.
Thank you!
thanks so much
if it's an unreal engine game compressed with lzo then for precompression, you must provide lzopro, standard lzo cannot restore any of the streams. If you use ztool/xtool but skip verification, you'll notice that the precompressor does see the streams but just for the fact that it cannot restore them with perfect crc, it just skips all of them. The reason I added skip verification was the fact that if it ever came to a point where one cannot really precompress a game with perfect crc, it should then at least allow you to try to pack a game with different crc but then also allow you to repack it, example of this was DmC: Devil May Cry, also unreal engine game compressed with lzo, I couldn't use xtool to precompress game but used skip verification, the game files were different but the game played from start to finish without crashing plus I managed to save space after repacking it.
as for Odyssey, probably some internal issue but the scanner does see all the streams in the samples
DataPC_ACD_Greece_DX11.forge:
Universal Oodle stream scanner
Created by Razor12911
[0] = Unknown/Invalid
[1] = Kraken/Hydra
[2] = Mermaid/Selkie/Hydra
[3] = Leviathan/Hydra
0001[2]| Pos: 00004CE4, Size: 13359
0002[2]| Pos: 00008117, Size: 13271
0003[2]| Pos: 0000B4F2, Size: 25162
0004[2]| Pos: 00011740, Size: 25029
....
0387[2]| Pos: 00856A2E, Size: 26334
0388[2]| Pos: 0085D110, Size: 13314
0389[2]| Pos: 00860568, Size: 21767
0390[2]| Pos: 00865A73, Size: 23507
Done!!!
for Destiny, I think lzna was used on this game just like mk11 (I see a lot of 0x058c headers everywhere), support for lzna is not added at the moment, but I obviously need to confirm this.
Yeah I used oo2recm for Odyssey and it worked fine (AFR works too but has no stdin support for compression, means temp files are over 250GB in size before srep even starts). With XTool:oodle it was just game size + ~2gb temp file (in task manager xtool just stop working after a certain point just in the beginning, no cpu or ram load, only hdd load for copying).
We already can use uelr for the unreal engine lzo games listed above, but the main problem is, that it has no stdin too and only specific files can be unpacked with it (unlike xtool, you can't just pack the whole game with uelr if files are present which don't have lzo streams, for example binaries or text files. You have to use masking or a seperate archive if you use diskspan for example. With xtool we can just pack the whole game without seperate it by files or streams).
Razor12911
22-06-2019, 16:11
Thanks I'll look into it :)
I guess those are all the issues
Razor12911
25-06-2019, 02:42
can someone bless me with a sample of Dragon Age Inquisition, it's required for development of XTool 2019
Lemme download it. I will send you some samples later (if no one else do it).
Did you need any libraries or just archives?
Razor12911
25-06-2019, 02:54
The game should be Zlib compressed, I guess only archives. I just need one cas archive, you can run a scanner or something like that to confirm on whether the sample you are uploading does indeed contain zlib streams. If you want to quickly upload the cas file since most definitely they are 1GB, you can really slice the file and give the first 100MB or something like that, I only need it to see the header structure to improve speed for games that use frostbite engine.
Ok. Download is on 70% right now.
I have one question: According to the Bethesda BSA extractor, Skyrim Special Edition use lz4 compression, but the game file scanner didn't find anything and I can't find any usefull information about it on the usual sites like xentax etc.
Did you know the trick? Are these archives crypted somehow? Most of the files aren't compressable in the usual way (ratio is always at 99.9%).
vBulletin® v3.8.11, Copyright ©2000-2026, vBulletin Solutions Inc.