LZ4 MOD for *certain* game's data archives | repacks

[panker1992]

Sorry to revive this graved topic from the afterlife.

I have briefly some questions as i am trying to get into lz4 for the " i dont know i lost count " time.

you give some tools above a story and an explanation.

1) how to use the script to Import the files back in?? the files are decompressed and they will give error.
Q)Where does it say how to import them and with what parameters ?

2) How do you zero out the entire space of the files you are willing to remove? for example sound.
Q)Castlevania is mixed with sounds and they are wav/ogg files and they take up alot of space? i only want to zero out those and then reimport them but your script gives error.

[elit]

Panker, I simply wrote bat script:
recode_RotBP.zip


As for null, either with quickbms -Z or rawinjector I made. You have to figure things the rest for yourself, it was a long time.

[panker1992]

Thanks for giving guidance !

I have posted several youtube videos on very advanced compressing and analyzing data

sorry to bother this old grave of a post, let it rest in peace

[L33THAK0R]

Ah fuck I didn't realise LZ4 was such a bitch, also on what Razor said about Z/XTool LZ4 support not being designed to be universal that explains a lot of my troubles with the codec. I've been trying to pack "Demon's Souls" (contains LZ4 streams) and was wondering why Z/XTool wouldn't work with it, I guess this is the reason why.

[panker1992]

Last time i checked demon souls isnt out for pc yet !!!

are you telling me you have a secre that it may come out ?

[KaktoR]

@panker
https://fileforums.com/showpost.php?...&postcount=396

[panker1992]

if it's possible i want sample data of lz4 streams

i think it's possible

[L33THAK0R]

Originally Posted by panker1992

Last time i checked demon souls isnt out for pc yet !!!

are you telling me you have a secre that it may come out ?

@panker1992, sorry I should have clarified. I'm currently repacking console exclusives, as I've figured out how to pack some trickier microsoft/sony container-types, and have been getting good ratios. Although Xenia (Xbox 360 emulator) progress is slow, RPCS3 (PS3 emulator) development has had a decent uptick in progress. "Demon's Souls" for the PS3 (not PS5) contains LZ4 streams, however this is based entirely off of investigating the methods used in 2 repacks of the title (A Darack team-member, & Gnarly), the former of which is currently offline due to personal issues, and the latter being unable to recall their methodology. I currently can't say which files (of which there are ~6000) contain the LZ4 streams, as the GFS tool is unable to detect any.

I'm also not sure how to identify if LZ4 compression was used, like how the Authors other project ZSTD can be, by examining the first 4 bytes in a hex editor.

[Masquerade]

LZ4 can be used without header info. This also confused me at first. I'm out of country currently, but I definitely intend to do some more research when I get back.

Btw you could encounter problems analysing the data since desktop PCs are little endian whereas the PS3 had a CPU made by PowerPC which is big endian iirc.

[L33THAK0R]

Originally Posted by Masquerade

LZ4 can be used without header info.

This is both fascinating and slightly depressing, just when I think I'm starting to understand the basics of these algorithms, I realise I'm yet to even scratch the surface.

Originally Posted by Masquerade

Btw you could encounter problems analysing the data since desktop PCs are little endian whereas the PS3 had a CPU made by PowerPC which is big endian iirc.

Damn I hadn't considered the architecture differences. I suppose though it shouldn't be too hard to determine if a mainstream compression algorithim is being used, through trial and error, should a universal methodology for LZ4 pre-compression be developed (which from the sounds of things would be no easy feat).

[Masquerade]

Originally Posted by L33THAK0R

I realise I'm yet to even scratch the surface.

Welcome to the party.

[FitGirl]

LZ4 has been released only in 2011, it could not exist in Demon's Souls. If any LZ algo is used in there - it's something custom, like LZSS, Japanese devs love it.

[L33THAK0R]

Originally Posted by FitGirl

LZ4 has been released only in 2011, it could not exist in Demon's Souls. If any LZ algo is used in there - it's something custom, like LZSS, Japanese devs love it.

Rats! I hadn't considered that, that begs the question how ztool was used in the repacks I mentioned to be able to pack these streams. It'll definitely be interesting to see if anyones able to figure this one out, but given how niche this hobby is, and more niche still with console game-data compression, I think this might just remain a mystery for a while longer!

[Masquerade]

Originally Posted by L33THAK0R

Rats! I hadn't considered that, that begs the question how ztool was used in the repacks I mentioned to be able to pack these streams. It'll definitely be interesting to see if anyones able to figure this one out, but given how niche this hobby is, and more niche still with console game-data compression, I think this might just remain a mystery for a while longer!

Well if you can see what plugin was used for ZTool, that might help you.

[L33THAK0R]

Originally Posted by Masquerade

Well if you can see what plugin was used for ZTool, that might help you.

All I was able to gather from the installer and testing the archives was that, at least for Gnarys repack of the title, "plz4" was used. I haven't been able to look at the Darack teams repack of it as no mirrors of it exist currently, but I theorise a similar method was used.