I tested xtool in Divinity Original Sin 1 EE and 2.

GameMaster.pak[DOS2] => 426MB
PAK format

Sample?

Call of Duty WWII
bulge.fd
bulge.ff
imagefile19.pak

link

Please help with the decompression program

Update available

Changes

- added ea frostbite 3 oodle and zstd support for fifa 18
- added dunia 2 lzo support for fc series
- added iw engine lz4hc support
- added partial crilayla support for naruto series
- added anvil engine lzo and oodle support
- fixed crilayla inflation bug
- fixed non-stdio mode bug
- updated zlib precompression
- removed MT scanning for some codecs

The samples you uploaded:

DataPC_LoadingRoom2.forge

afr19: 10,780,672 => 13,917,326 => 8,970,681 bytes
xtool: 10,780,672 => 13,917,332 => 8,983,511 bytes

DataPC_Map_Menu.forge

afr19: 1,605,632 => 6,956,755 => 821,299 bytes
xtool: 1,605,632 => 6,957,991 => 820,636 bytes

:confused::confused::confused:

have you tested xtool on the samples you uploaded?

Have you tried to increase chunk size?

Skip verification to see what you're really suppose to get, if it's more then try using an older liblz4.dll since AW came before the version of the library in the attachment.

Use the x86 version for that

Yea, that game seems to contain lzopro compressed streams and I only added x86 support for now. Maybe it's AC4, not sure.
Edit:
AC2 was affected due to me not registering what codec to use for restoring, guess it really is a waste of energy.

Your file:
Compressed 1 file, 88,997,888 => 189,610,506 bytes. Ratio 213.05%
Compression time: cpu 0.09 sec/real 2.72 sec = 3%. Speed 32.66 mB/s

really encouraging words

@Razor: Thanks mate for the update.
Keep up the good work.
Xtool is going to be more than a precompressor, so not adding any game support diverts it from that path.

https://yadi.sk/d/WNKW73Ht3UpyVN
data.cpk (393 mb)
NARUTO SHIPPUDEN Ultimate Ninja STORM 2

[External compressor:xprecomp]
header = 0
packcmd = xtool.exe e:precomp:c32mb,t1:crilayla - - <stdin> <stdout>
unpackcmd = xtool.exe d:precomp:c32mb,t1:crilayla - - <stdin> <stdout>

Extracting archive: data.arc
Extracting data.cpk (413123988 bytes)
ERROR: file _TEST\data.cpk failed CRC check
:confused:

Hi Razor!

Did you test the sample I sent?

xtool v 0.7 Tried it on a file from AC orgins

it didnt expanded.
sorry to bother oodle was missing from arc.ini setting.
this sorted the issue.

afr expaned it to 163.15% while xtool to 162.91% xtool
restored it in only 14sec while afr took 18 seconds in my system

Now my only problem is with Dunia engine. i wrote dat file address in xtool.ini still no change.

BTW is farcry primal and James Camerons avatar supported.

Yep, didn't expand with :lzo

Maybe wrong codec in arc.ini? lzo2.dll is in place

Edit: You have to use for Origins:rolleyes::D

From this example xtool and AFR have pretty much the same compression speed (AFR is a little bit faster, but just a few secs)

I dont know if you want to go this route, but the way I did my tests when developing lz4 packer for "Raiders of the Broken Planet" game was that I used both original and self-modified quickbms script for unpacking game archive first. I unpacked thus twice - for each.

In modified script only difference was that I disabled decompression by taking out "comtype" command and used "log" instead of "clog" to dump files raw & compressed.

So I had both compressed and decompressed individual files from archive. Then it was only matter of finding right/best cmp settings through hex editor and binary comparison(first on few files and then all) to get best setting. And even wrong(but close) settings were still compatible with 80%+ of all archive.

So by saying all that what were you trying to say by this "I dont know if you want to go this route" ? That Razor should find the correct parameters ? I don't think he has that much of an interest in far cry 5 or wants to waste time doing all that.

i'll check :)

compr:
dec time/speed:

Similar happens on Agony's .PAK files(zlib,v0.7, v0.6 works good)
First file processed well,but the second one looks like simply copied.
Cant upload files because they are more than 10 gigs. :/

How exactly is that less headache? This way at least he can re-use a lot of same code/routines for each compressor instead of duplicating and maintaining separate projects with a lot of same code.

Update available

Changes

- fixed crilayla bug on naruto games
- fixed issue detecting lzo2a, lzo1c streams

@doofoo24
I have to agree with elit, It's more headache separating the codecs. An example would be ztool itself, normally when I make changes to pzlib, if it's not related to precompression itself, I had to make the same changes in plz4, plzo... sometimes I forget doing this and end up with different sources of each, in one, a bug was fixed then another bug appears in another, now there are bugs all over the codecs, different bugs, you end up trying to fix one and another pops up and so forth. The idea of putting everything together means one code, if one codec is affected, then all will be affected then I'll know the source of the bug.

I wonder if there is still room for improvement.

I didn't touch zstd codec in between the updates

ztool_08

ztool_06

Ran a test using the samples I got from forum. Very small input but got these results:

Input: 278 MB (291,605,504 bytes)
DataPC_Map_Menu.forge (ACI)
DataPC_LoadingRoom2.forge (ACI)
DataPC.forge (ACII)
DataPC.forge (ACIV)

Not sure in terms of results here, I ran test on ramdisk using my old core 2 extreme, I didn't include t4 since those are mostly irrelevant.

As for final output after compression, I'd be taking a gamble when I say it's headers since they include stuff like what codec was used since xtool is not just made for one game engine but I'll check in terms of the number of streams found by xtool vs afr

upload files and command line used

It's normal, because i using differents decode methods in cls and exe. In cls i use some tricks, which allow me to get a higher speed. exe-decoder, yes, slower.

I think, that numbers of streams found is the same. But maybe in AFR the output container is more successfully made ;)

Yep, but I'm not interested in this engine anymore.

So, what's your next clue? :D

NSUNS3 XTool 0.8
 

and... which is which?

NARUTO SHIPPUDEN Ultimate Ninja STORM 2
FreeArc 0.67 unpacker. Extracting archive: data.arc
Extracting dataRegion.cpk (716396224 bytes)
All OK
129.547s: unarc x -o+ -w.\ -dp_TEST data.arc

Update available

Changes

- fixed 2gb+ input issue... (int32 instead of int64 :()

:confused:

crilayla only works with 32 bit

:confused: xtool 32bit

zstd and oodle are 64bit only. There is no 32bit for these

actually zstd 32-bit is there, just that perhaps the library I included requires a prerequisite.

libgcc_s_dw2-1.dll

as for oodle, I didn't bother with 32-bit version considering the fact that it's memory hungry plus most games ship with the win64 version.

xtool 0.9 precomp:t100p:lz4 :
COD WWII 38.4GB TO 62.3GB to 25.8gb srep+lolz
COD AW 33GB TO 63.8GB to 20.3gb srep+lolz