It's the same way Microsoft releases versions for its Windows 10. And yes, year 2020 and 10th month. 2010

hey,razor uh i dont know coding and stuff but what does this do ? create repacks or compress game files and how can i use it?

and also what is source code? ( sorry if asking in wrong place) when i see some projects they say an update is out and give some kind of source code. what is that?

This tools are for compress game files, in the case of XTool has functionally of inflate compatible streams of game engine, then srep compresses it and finally lzma or lolz or lzma2 compresses it even more.

Here are best methods for compress games ... usually i use srep+lzma2 for fast compression, but in some cases i add xtool depend of game.

https://fileforums.com/showthread.php?t=101639

You can get all other tools here.

https://www.fileforums.com/showthread.php?p=478349

Razor12911, can you please add , (or some other sign) as alternate option to specify combining codecs? I'm actually trying to use arc.exe by specifying codecs from command line rather than modifying arc.ini, so I'm trying to use packcmd in arc.ini like this:
...
[External compressor:xtool]
header = 0
packcmd = xtool precomp -m{option} -c32mb -t100p-1 - - <stdin> <stdout>
..so I can use, for example, arc.exe a -mxtool:zlib+reflate ...
But since you choose to use + sign to combine codecs this will not work....or if somebody have another idea how to specify different codecs that requires + sign from command line and not by editing arc.ini everytime, I would much appreciate advice...

Thanks in advance,
regards

A heavy Thanks But....
 

A heavy Thanks But i already have razor 's quick archive packer which is okay for me but how can i use custom installer ultimate ( the second link u gave ) . what can i do with it? and also whenever razor or y_thelastknight release an update for a project how can i update it? and what is source code? :confused::confused::confused:

sorry if i am asking too many questions.:D:D:(:(:D

Just another oodle test:

Sekiro v1.05 Data4.bdt:

oo2reck (oo2core_4_win64.dll):

Xtool Kraken (oo2core_4_win64.dll):

I'll add "+" as a resource string and if you want to change it as a user, you'd have to do it yourself. in the next update, that is

Update available

Changes

- added database search
- updated zlib scanner
- fixed reflate bug
- fixed 2GB memory limit

Notes
Database search is a feature similar to configuration support, a demonstration on Far Cry 5 and Watch Dogs Legion will be posted to show how it works.

Updated zlib scanner to eliminate some pesky false positives that cause reflate bugs, verification of reflate removed and should work faster now.

2010_R2 update broke reflate, not sure if you noticed but it's now fixed.

Report made by Kaktor on some file that gave outrageous ratios.
https://fileforums.com/showpost.php?...&postcount=559
Turns out the issue was caused by this
https://community.idera.com/develope...ream-2gb-limit

So I created a custom TMemoryStream that can handle more than 2GB data in x64

Whats parameters you use in your arc.ini settings ? Thanks

> Turns out the issue was caused by this
https://community.idera.com/develope...ream-2gb-limit

because of these obscure bugs, I didn't continue with Delphi, especially ide is shit compared with other. You're still holding grounds nice work :)

Thank you very much:D If I can make a small comment, isn't resource string editing "overkill"? I mean, simple "-mzlib,reflate" syntax is all that is needed...(currently, only possible is, for example, -mzlib+reflate)

Anyway, thanks again,
regards

Well I mostly evade the bugs by just sticking to WinAPI functions and a few times by using System functions but then with these I usually have to write custom code that do the same thing just that the one Delphi has is crap. :)

the problem is commas are used for something else

example -mzlib:l98,w15+kraken:l9:t128...

if we replace + with , then syntax will look like this -mzlib:l98,w15,kraken:l9,t128..
zlib will start considering everything as one of its parameters. :p

Edit:

This is what you need to change via Resource Hacker in xtool.exe

Update available

Changes

- updated search/config support

Notes

You can now use database files (*.xtl) posted in this thread to precompress games which the program does not have a native support for.

An example is Far Cry 5, the game is lz4 compressed and as there is no universal scanner for these streams, you can use a generated database to precompress the game.

Results on farcry5.dat:

I understand now and I agree comma is not a good choice, but how about & or # sign?...I never asked for , specifically, it does not matter as long as it is not +

Anyway, it is just a suggestion, ResourceHacker editing is not a problem, changed to & and it is working great (it's even more logical to me with & sign) :)...
Thank you very much for your work,regards

Greetings Razor

Does the Dunia codec support for Far Cry Primal?

I haven't seen any mention of it here or on the DELZOREC thread on krinkels.org.

I tried using DELZOREC on the HD Texture file, but it certainly didn't work (3GB file somehow unpacked to 1GB).

If anything needs testing, I can upload some small samples. If there's a way you could show me to see what kind of compression is already on the files, that would also be appreciated.

Just Srep+LOLZ

Yes, but being properly precompressed wil get an even smaller packed version, which is what I'm going for here.

I had totally forgotten about Primal, I'm only guessing when I say that I think it uses the same stuff that Watch Dogs 2 and Legion uses, so I think after I'm done with these, I'll check it out.

Update available

Changes

- fixed search/config support bug (thanks dixen)

Tested FC5 *.dat with XTool R4 and...It closed on 10-20%
https://i2.imageban.ru/out/2020/11/0...b05b076a45.jpg

Well it's not like I didn't say something was wrong and fixed it in the next release... :rolleyes:

But...If tested without SREP or LOLZ - ALL fine unpacked and XT20 process used about 300-400 mb RAM.
UPD. XT20(R5) + SREP = same bug

try unpacking with temps (disable cls/stdio), maybe we can see what is causing the problem.

Edit: I may need to make cls of xtool perhaps just to avoid issues.

20 gb data.arc - same bug.

OFF stdin/stdout - in progress

UPD

With this parameters - same bug - xtool is closed on 2.5-2.7 gb RAM
R4 or R5 - no matter..
Offensively

could you try without deduplication, too many points where an error could occur.

Of course...waiting
UPD.
Yes without dedup - the test was completed successfully and xt used about 500-600 mb RAM (XT20 R5)
https://i1.imageban.ru/out/2020/11/0...a3ca83d1e4.jpg

You may want to be interested in this.

I was trying to inflate cpk file from Diablo 3 from nintendo switch. Normaly these are crilayla but gfs detected lz. So I tried old ztool, some older xtool(date modified says 26.april 2019) and xtool2009R3.

For reference I tried single file "Act1.cpk" of 814mb size. GFS detected potential of inflation of up to ~1800mb.

Ztool could not inflate past few 10's of mb if any, regardless of setting like m2, m3 etc.

Later xtool2009R3 depended on codec, most were at about 100mb extra only, except reflate which did go to 1.1gb. However final packed size after srep+xlzma was about same which make me think of possible huge overheat?

Finally, that older xtool with :high setting was able to inflate to.. 1.7gb!!
I tried to compress that and got about ~40mb better compression.

General final compression after codecs was around 740mb+, this one went to ~704mb. I wonder if this was real inflated data to 1.7gb or too much overheat. I must also say that older xtool was able to do it only with :high option which unlike ztool is not even know or documented and probably unofficial. Also it took very long time compare to say ztool:high, it seems to work differently.

I have uploaded the file act1.cpk for further research if you are interested:
https://megaup.net/1zEg1/Act1.cpk

Is there any incompatibility between "CIU 3.x+ZTOOL+XTOOL" extraction process & new AMD CPU series ?

Anybody have tested ZTOOL with below CPU ?

AMD Ryzen™ 9 3900X "12-Core 3.8 GHz" (4.6 GHz Max Boost) Socket AM4 105W

My installer created with "CIU 3.x+SrepInsidex64+ZToolx64+LOLZx64" works pretty well on ALL of mid range CPUs(Corei3+Corei5+Corei7) but when extracting on NEW AMD Ryzen™ 9 3900X, installer stuck on "0.0" percent progress bar & "ZTool.exe" won't launch on task manager!!!

Is "ZTool.exe" incompatible with new 12-Core CPUs ?

Thanks @Razor12911

Did you try to unpack with bat file`? If this works then the problem could be inside CIU code somewhere.

my tools can handle up to 256 threads, if you have problems it's mainly because you're using ztool instead of the newer xtool which had MT bugs fixed.

Thanks @KaktoR for batch script! i tried that unpack script but it gives me some errors ! can you please take a look at them :

I have a lot of HDD space [200GB] ,but still gives me errors !

I have compressed my files in below names & put them in "OUTPUT" folder of DiskSpan_UNPACK script:

using below compression method :

I only modified archive name in "_DiskSpan_UNPACK (Test Only).bat" , should i change other parameters ? for example compression switch !

Try this

Update available

Changes

- added temporary libdunia codec (thanks ProFrager and FitGirl)

Thanks Razor / ProFrager / FitGirl for the new dunia codec!

Here's a test on primal_main.dat (6.32GB)

Working fast and good.

WD Legion

london.dat
shadersobj.dat
common.dat
patch.dat
london_preload.dat
london_cache.dat

13 gb > 17.6 gb

Log not saved)) Sorry

Having a few problems decompressing with the fcp option, I have tried a few different unpackcmds:

Yes, fcp.xtl and liblz4.dll are with xtool.exe

Errors like:

This is just on 1 file with no extra compressors.

Is there something wrong with my unpackcmd?

Update: srep+lolz is smaller than fcp+srep+lolz

Guys ,how can i set the

--dedup=FC.bin

parameter in -m chain ?
is it possible ?

So I was dealing with crilayla in Persona 5 cpk's. Normally for these I just use bms but for the first time I got errors in some archives. So i tried your older xtool that still had it(in fact that's why I kept it). And man.. this thing is fantastic! I bring it because you said you dropped it because of speed. Well I don't know if I can change your mind but let me try.

I tested 4.4gb cpk with :t4 on my 4960k 4.2ghz, which is pretty normal today. Inflating took about 16min and re-encoding about 14min. This gives the speed of around 4.5-5.5mb\s. Look, this is not bad at all! There are lot of crazy people out there abusing lzma/lolz with outright stupid settings like mc1000+ that can slow you to crawl with no benefits, yet unlike those this tool is extremely beneficial.

Maybe you meant that for decompression its too slow. Well, repacking cpk back with single threaded bms is just as slow if not more + all the work around it, I did it this way till now. I even had to use xdelta because bms compressor write something into header that I have not seen in single game yet(other than zeroes, I think it was offset 16 and ~6 bytes long string).

So your cri xtool is excellent, work well and have reasonable speed including de/compression - considering alternatives. Also future CPU cores are only going to be more, not less.

This is my 2 cents, of course you do what you feel like. But there is no other cri tool like this and compression is probably used second most often after deflate. There already are plenty of deflate tools and libraries, even original ztool is just perfectly fine for this. Its those other codecs that would be beneficial. Crilayla, maybe also lzss and certainly yaz0 to name few.

In fact you already completed the work, why abandon it? So for the love of god I beg you, put it back in latest version!

PS(Zstd and lz4 are temporary and will stop working after time because of their retarded design. Not sure about lzo(that is actually a question, why is standard lzo not working in dunia and unreal engines and need special treatment? I thought its a stable design like zlib?). But crilayla, lzss, yaz0 and such should remain compatible.

-grittibanzli inflate completely randomly. Sometimes tested 20mb inflated zip file is 34mb, other times 31.6mb, another time 33mb atd. exactly same setting and just repeating same command on console.

-zlib codec did not inflated zip file at all!(I thought it was supposed to handle deflate?)

-preflate and reflate both did to same ~27mb, but preflate took ~6sec while reflate did it in ~1sec.

Looks like good old reflate is still the best zlib codec to stick.


EDIT(and ztool is also able to inflate to 27mb without :high or :m2 !) <<UPDATE(BS, scrap that claim, need m2 or high indeed)
EDIT2(ok I checked manual again and learned about zlib+reflate combination benefits. Maybe this is best way. It took ~3sec compared to ~1sec on deflate alone though. Not sure how big those benefits are in real application yet.)
EDIT3(have to explicitly state 'mb' in -c option instead of just 'm', e.g. -c128m vs -c128mb. No big deal but used to ztool, was more flexible. PS: I am really liking the idea of zlib+reflate in one go, before I used to pass twice in test to find out which one to use.)