XTool - Successor of ZTool

[Razor12911]

Update available

Changes

- added dunia engine lz4hc stream precompression support (Far Cry 5)
- fixed unity engine support

Notes

In terms of the Dunia engine, I advise that you use the latest liblz4.dll to avoid some streams being skipped due to their major difference, this could be because even I myself am not sure if I used the correct parameters for recompression, if you have an idea of what is used, please comment. (I used lz4hc, level 9)

Only Far Cry 5 is what I think is supported currently, that's the only game I had installed so run tests to see if all is good, if so then I'll be moving on to Far Cry 3 and 4. Add the dat files in xtool.ini, make sure in that same location, the dat files are with the fat files else they'll be errors.

Parsing the fat file is a pain in the ass especially for the big files (farcry5.dat and installpkg.dat), so xtool might seem stuck when running those, at most it should take 5-10 minutes before it begins actual precompression.

[darkwolves]

@Razor12911
excellent you have been on a roll with this tool the past couple of days

[Razor12911]

Originally Posted by Razor12911

Update available

Changes

- added dunia engine lz4hc stream precompression support (Far Cry 5)
- fixed unity engine support

Notes

In terms of the Dunia engine, I advise that you use the latest liblz4.dll to avoid some streams being skipped due to their major difference, this could be because even I myself am not sure if I used the correct parameters for recompression, if you have an idea of what is used, please comment. (I used lz4hc, level 9)

Only Far Cry 5 is what I think is supported currently, that's the only game I had installed so run tests to see if all is good, if so then I'll be moving on to Far Cry 3 and 4. Add the dat files in xtool.ini, make sure in that same location, the dat files are with the fat files else they'll be errors.

Parsing the fat file is a pain in the ass especially for the big files (farcry5.dat and installpkg.dat), so xtool might seem stuck when running those, at most it should take 5-10 minutes before it begins actual precompression.

Doesn't seem like I'll be adding FC3-4 support since I think lzopro is required for them just like Unreal Engine games.

As for current progress, I added support for FIFA 18.

The idea of binding codecs together came when I saw that this game is compressed with two different algorithms, oodle (kraken) and zstd

This is the result:

when using:

oodle - - <stdin> <stdout>

Compressed 1 file, 317,249,339 => 586,677,625 bytes. Ratio 184.93%
Compression time: cpu 0.59 sec/real 172.68 sec = 0%. Speed 1.84 mB/s

however when using:

oodle,zstd - - <stdin> <stdout>

Compressed 1 file, 317,249,339 => 592,734,832 bytes. Ratio 186.84%
Compression time: cpu 0.66 sec/real 185.29 sec = 0%. Speed 1.71 mB/s

[doofoo24]

can't you make FC3/4 like FC5 file by file without lzopro ?

[Razor12911]

Originally Posted by doofoo24

can't you make FC3/4 like FC5 file by file without lzopro ?

Streams never restore to their original form bro, that's the issue. Applying techniques of xdelta don't work either because there are a lot of differences between.

I could be missing something, who knows...

[doofoo24]

i guess also games with engine Disrupt (watch_dog) are the same ?

[Razor12911]

Originally Posted by doofoo24

i guess also games with engine Disrupt (watch_dog) are the same ?

I have not tried that game before so I can't say for sure.

[ShivShubh]

Originally Posted by doofoo24

can't you make FC3/4 like FC5 file by file without lzopro ?

It doesnt work like that. FC2/3/4 uses LZOPRO while FC5 uses LZ4HC.

[darkwolves]

Originally Posted by ShivShubh

It doesnt work like that. FC2/3/4 uses LZOPRO while FC5 uses LZ4HC.

looks like someone is going to have to break down and get LZOPRO for razor...
i will check my monthly budget..

[elit]

Originally Posted by Razor12911

Update available
In terms of the Dunia engine, I advise that you use the latest liblz4.dll to avoid some streams being skipped due to their major difference, this could be because even I myself am not sure if I used the correct parameters for recompression, if you have an idea of what is used, please comment. (I used lz4hc, level 9)

I dont know if you want to go this route, but the way I did my tests when developing lz4 packer for "Raiders of the Broken Planet" game was that I used both original and self-modified quickbms script for unpacking game archive first. I unpacked thus twice - for each.

In modified script only difference was that I disabled decompression by taking out "comtype" command and used "log" instead of "clog" to dump files raw & compressed.

So I had both compressed and decompressed individual files from archive. Then it was only matter of finding right/best cmp settings through hex editor and binary comparison(first on few files and then all) to get best setting. And even wrong(but close) settings were still compatible with 80%+ of all archive.

[ShivShubh]

Originally Posted by elit

I dont know if you want to go this route, but the way I did my tests when developing lz4 packer for "Raiders of the Broken Planet" game was that I used both original and self-modified quickbms script for unpacking game archive first. I unpacked thus twice - for each.

In modified script only difference was that I disabled decompression by taking out "comtype" command and used "log" instead of "clog" to dump files raw & compressed.

So I had both compressed and decompressed individual files from archive. Then it was only matter of finding right/best cmp settings through hex editor and binary comparison(first on few files and then all) to get best setting. And even wrong(but close) settings were still compatible with 80%+ of all archive.

So by saying all that what were you trying to say by this "I dont know if you want to go this route" ? That Razor should find the correct parameters ? I don't think he has that much of an interest in far cry 5 or wants to waste time doing all that.