How do you find out?
Also what I'm looking for is some sort of tool that you can take Unity assets off of and precompress, and during install they're decompressed and put back on the Unity files. Is it possible?

You can use the XTool unity plugin to precompress lz4 compressed Unity Asset Bundles.

Assassins Creed 2 *.dat files blocks addresses doesn't match
 

I extract the DataPC.forge file using quickBMS script and after that i try to put them back into where it was using raw injector but it doesn't work out because the all extracted *.dat files doesn't have the same hexadecimals address so my question is how do i solve this & why blocks are isn't the same?

Trying to find the 87.dat block Start Offset and End Offset in DataPC.forge using HxD
https://drive.google.com/file/d/15Og...atcpOML7D/view

But it doesn't work out?
https://drive.google.com/file/d/1ME-...GABq2mgRJ/view

These are the unpack files
https://drive.google.com/file/d/1ova...U1YYmZxn-/view

Can someone look at this kind of files and what I can do with them? They have "RIFF8-��WAVEfmt".

Sample as attachment

Not sure what it is. VGMStream cannot detect what it is and wwise2ogg will only give a 17kb output.

Golf With Your Friends (Build 11145867)

4.6GB ---> 2.3GB

Hi all !

Frostpunk V1.6.1 - GOTY

Latest XTool : Zero inflation. Any kind of variations of ZLib,Preflate,Reflate -cXXmb ,no success.

But this version inflates the data well !

XTool version 0.9

Created by Razor12911

Commands:
e - encode
d - decode

Operations and codecs available:
precomp - data precompression
zlib : loaded
crilayla : not loaded
lz4 : loaded
zstd : loaded
lzo : loaded
oodle : loaded

Options:
precomp - data precompression
c# : chunk size (default 16mb)
t# : number of threads, use p for percentage
dm#: decoding memory [1(low) .. 3(max)]
dt#: diff tolerance (default 0.1)


Other game's ZLib files are OK.
What am i do wrong ?

^
I can confirm.

Sample should be around 106%

try xtool 0.12

@kuyhaa : i would like to use the newest version,btw thank you !:)


Sample from GOTY V1.6.1 :

XTool v0.9 :

arc a -mxprecomp32 e:\animations32.arc e:\animations.dat
FreeArc 0.67 (March 15 2014) creating archive: e:\animations.arc
Compressed 1 file, 29,260,989 => 50,177,682 bytes. Ratio 171.48%
Compression time: cpu 0.02 sec/real 12.71 sec = 0%. Speed 2.30 mB/s
All OK

arc t e:\animations32.arc
FreeArc 0.67 (March 15 2014) testing archive: e:\animations32.arc
Tested 1 file, 50,177,682 => 29,260,989 bytes. Ratio 171.48%
Testing time: cpu 0.02 sec/real 0.85 sec = 2%. Speed 34.25 mB/s
All OK

Latest XTool :
Nothing.

Edit:

XTool V0.7.1 : Works fine ! Thank you Razor12911 !

Marvels Spiderman Remastered [Build 10619930]

Archive 1 content: All files from asset_archive folder except dag, toc and languages.
Archive 2 content: All remanining files except languages.

Archive 1:
Archive 2:
Languages:
Repack

Thanks to Razor12911 for XTool, ProFrager for lolz, Bulat Ziganshin for FreeArc and srep.

Warhammer 40,000: Boltgun [1.17.38829.471]

4.38 gb => 2.96 gb

You can also use this database I made using bms2xtl:
https://bayfiles.com/D6bdh7taze/Boltgun_xtl

Which Version of oo2core.dll , should i use ?

This is a custom wwise bnk archive.
It contains multiple wem files.
I converted one of them.

State of Decay 2
 

Has anyone had any luck with any LZ4 library for "State of Decay 2"? Threads around the net indicate it uses some sort of "custom" LZ4 solution, however QuickBMS is able to parse it with it's built in LZ4 comtype. Currently exhausted a good collection of ~20 libraries through trials and I'm yet to pick up or detect a single stream.

Attached is a sample ".pak" archive should this compression quandary tickle anyone else's fancy.

Shame Legacy

*.pak:
Everything else:
9.9GB ---> 6.9GB

Used CLS-UE4D library by Shegorat.
Thanks to Razor12911 and ProFrager for XTool and Lolz.
Thanks to Bulat Ziganshin for FreeArc and srep.

Anyone can help me with srep options to use? I'm getting stuck here a bit.

Edit: -l# option seems to be the trick.

Hitman 3 test using rpkg tool (because no plugin is available atm)

Input is bigger because chunk0.rpkg was extracted (lz4 variant is used). However output is smaller. But that time to process is insane and I recommend not to do this at all.

The time above was achived on a m2 ssd for a single test. Here again it is not recommended to do this kind of stuff on a ssd, even it is much faster.

Extracting all rpkg's on a HDD took barely 4 hours.
Next step is to make TAR files out of extracted files (this will take another bunch of hours), because somehow freearc seems to have problems with nearly 2 million extracted files.

This is just a info test out of curiosity and I wanted to share some infos here.

Which version xtool and liblz4.dll needed?
With W3.xtl - decompression error for me:confused:

XTool 0.6.9 + xdelta3_dll.dll + lz4_193 + doboz.dll

bms2xtl config

It works but..VERY slow on lz4hc streams.

Repella Fella

You can use ffmpeg to extract the vorbis audio tracks from the webm videos then make hdiffz patches to convert the oggs back into videos (note that the hdiff patches will be huge since they make up the entire video track).

Only issue is that oggre is buggy af with these oggs, so it may just be easier to use:

srep+lzma2 on the webm

precomps+srep+lolz on everything else.

Precomps = precomp stdin/stdio fork by Nicolas Comerci.

8GB > 7.2GB

I spent all day trying to get oggre working, but it will crash without much of a pattern to go off.

Hm, no, not for me.

Did it today with latest version.

Can you share your witcher3.xtl?

Orcs Must Die! 3 [v1.2.1.0]

*.pak:
Everything else:
12.6GB ---> 6.8GB

Requires OMD3 database and lz4 library linked here.

Good evening, is there a way to create a .tmp file from a decompressed (or unpacked) file or batch of files in order to use hdiffz.exe and so to create a .diff file?
A lot of videogames have several files that can be unpacked using specific tools, but no way to repack them. Maybe hdiffz.exe is a solution, but it needs a .tmp file. How to create it? Thanks in advance for your help!

HDiffz does not need a .tmp file, read the usage instructions.

Old file and new file makes differential patch.

Hi,
Does anyone have any samples of LZMA compressed Unity Asset Bundles that they could share with me?
Thanks!

Diablo IV
v1.0.2, incl. HD files

Does anyone know roughly what the average variance is for the final ratio of a given input, that's run through LOLZ?

The Last of Us Part 1
v1.1.0.0

You will get better ratio in increasing chunk size for xtool.

L33THAK0R: Very input data dependent! Older, very older gamez, betterly compressible than previous 10-15 years ago to last years games. Average few percent to upto 10-25% betterly final compression ratio than LZMA. After, many data precompression (ex: xtool), prefiltering (ex: exe/delta) and such more...

Simply, this cannot be said in advance! It is necessary to experiment, it is possible to derive approximate information about the possible degree of compression from existing repacked games online.

Small examples: This very old modded MSTS games *.ACE files with xzlib+srep+lolz combo avg. 25-40% final ratio than uncompressed sizes.
*.RAW files in rare case upto 1.5-2* betterly compressed via delta+(srep)+lolz combo than simple LZMA only.
Sound files final sizes difference its minimal few percentage. Achieve better rates use sound specific compression.

KaktoR: What "mpsarc"??

@@kj911 ah thank you for the explanation, what I meant was more how much could be chalked up to variance due to LOLZ being a non-deterministic compression algorithm (or series of, I'm not too sure on its inner-workings). I like to compare my results to my peers, for some titles, take the 2020 title "Manifold Garden" I got a result of 797 MB, however one of our peers got a result of 792 MB. Now I was working with the latest version of the title and I also included the weight of the installer, with mine being slightly larger (due to some un-optimised storing of the "VCL styles" library that I've yet to figure out how to compress and decompress during run-time), and I'm wondering whether I can chalk up this difference to variance or whether I've missed some data that can be packed more optimally.

L33THAK0R: From what I have experienced with LOLZ so far, its operation is a bit more unusual compared to the standard LZMA. A given result cannot be reproduced, exactly byte, as with more commercial compressors. (Given compressed file size, like enwik8-10 results.) Certain switches of "Arc.exe" also affect the degree of final compression, I'm thinking here of the "-ds" switch, which is the default in principle or even if you (not) define it the final result may be different, say with "-dses/dsns" than otherwise.. LOLZ can sometimes be better without -dt/dtb, in some cases, than if we set everything to the max.

This small image shows the results of data files precompressed with xZLib+srep:m3f:a0 mode, and separately compressed to 7z, with the same compression settings. The difference between them is whether or not "-ds" or "-dses" switch was used during precompression. We would think that the end result would be the same if LZMA/LOLZ final compression were used for that archive.

Hi all, can anyone look at the files of the game Bramble The Mountain King, there UE4 and encrypted files inside + kraken.
I tried the latest version of the unreal plugin, but it didn't work. (xtool last ver.). With ucas database the same thing.
Can someone please see if it's the plugin not working or am I crooked?
Key:
//edit. woks fine with latest xtool

Apologies if you tried already, but please may you try XTool v0.7.1 with Unreal Plugin and Oodle core v2.9.5 with this packcmd? I cpuldn't help but notice you mentioned UE4 whereas the UTOC/UCAS structure is a fearure of UE5 that got added backwards.

XTool v0.7.1 with Unreal Plugin and Oodle core v2.9.5
*xtool 0.7.2h same results

Just read a topic about unpacking PCK files. Nothing new, honestly, but an idea came in my mind about not just unpacking them, but also encode/decode the WEMs inside the PCK files. So far, nothing new. Just wanted to know, if there is a "elegant" way instead of my idea floating in my head. Maybe somthing more "automated".

I already know that this method works and it's CRC perfect after the last step.