LZ4 MOD for *certain* game's data archives | repacks

[Razor12911]

PrinceGupta there is a reason ztool comes with external libraries, it’s for user to change them until he finds one that was used in a particular game. Decompression is never a problem, but recompression is. When it comes lz4 and zstd, everything must be precise else you will get crc mismatch every time. Perhaps a bit with ztool since it comes with internal diff patching functions when user used a dll that was not used for compression in first place

[elit]

Originally Posted by Razor12911

zstd has this problem as well. lzo, not so sure but it shouldn't have this problem since all variants of lzo come with binaries/libraries, lzo1x, lzo2a and so on.
zlib doesn't have this.

Now I see the pattern, both lz4 and zstd are developed by same author. Well, at least his code is nicely organized and easy to learn.

Originally Posted by Razor12911

PrinceGupta there is a reason ztool comes with external libraries, it’s for user to change them until he finds one that was used in a particular game.

OMG I kill you man, thats great! How come I did not know.. Time to do my games all over again .
EDIT: Well actually, thinking of it it may not be enough with lz4/zstd. I dont know how you coded it but in this case for example, default library would read block structure wrong even if compression match because of mentioned crc position which is part of the block, unless your tool is skipping it and can focus only on game data.
Still, it gave me an idea that one perhaps doesnt need to repack whole game archives with quickbms anymore, because depending on how your ztool works, maybe all I need is to modify library as I did here and ztool could function again. That would be great and would made ztool universal, then all you do is compile all lz4 versions and recycle them forever. But I would like to know more in detail how your tool work/read data exactly to understand, though its probably your secret so I am not going to ask.

[Razor12911]

Originally Posted by elit

OMG I kill you man, thats great! How come I did not know.. Time to do my games all over again .
EDIT: Well actually, thinking of it it may not be enough with lz4/zstd. I dont know how you coded it but in this case for example, default library would read block structure wrong even if compression match because of mentioned crc position which is part of the block, unless your tool is skipping it and can focus only on game data.
Still, it gave me an idea that one perhaps doesnt need to repack whole game archives with quickbms anymore, because depending on how your ztool works, maybe all I need is to modify library as I did here and ztool could function again. That would be great and would made ztool universal, then all you do is compile all lz4 versions and recycle them forever. But I would like to know more in detail how your tool work/read data exactly to understand, though its probably your secret so I am not going to ask.

Yes, tool focuses on game data, error codes are returned all the time, like you said, library reading block structure incorrectly, it happens a lot, but it still returns decompressed data.
I can tell you how it works if you want, it's no secret.

[elit]

Originally Posted by Razor12911

...I can tell you how it works if you want, it's no secret.

That would be helpful, you can PM me or post it here whatever you prefer.

I actually tried to use lib from this version that I know works with the game and could not get it unpack anything with ztool(output same size). But I am not sure if I did it right, you see your lz4.dll have ~60kb while my compiled had around ~200+kb and I made .dll by manually changing function in makefile code - in function "add_library" where I added "SHARED" flag, something like this:

add_library(liblz4 ${LZ4_SRCS_LIB}) >> add_library(liblz4 SHARED ${LZ4_SRCS_LIB})

Now the thing is if I dont do that and just check to make lib in cmake gui, it does output library around same size as yours(~60kb), but its a .lib not windows dll. When I tried that big dll ztool did not complain with errors but output same size, if I rename lib to dll it output error(obviously not same). Maybe someone can hint me here what I did wrong, you can also try yourself if you can compile more properly.

Additionally, I found on zenhax forums that zstd may not be so bad after all, it does keep track of different version in header:
"zstd
It starts with a little endian 32bit magic number, when seen with a hex editor only the first byte (the low 8bit) is different because it depends by the version of the algorithm:

1e b5 2f fd    v0.1
22 b5 2f fd    v0.2
23 b5 2f fd    v0.3
24 b5 2f fd    v0.4
25 b5 2f fd    v0.5
26 b5 2f fd    v0.6
27 b5 2f fd    v0.7
28 b5 2f fd    v0.8, current version"

Unfortunately lz4 dont do it, but its the only one at least. Of course, there still have to be a header present in game resource to know.