Current compressing techniques guidance

[Spexxos]

I am also trying to get into the whole compression thing. In my case its primarily for compressing savegames. For that matter I would be interested in the current state of things aswell. I think a might be a bit ahead of you in my knowledge, JoeKkerr, and since I dont want to open up another thread with the same topic I am going to add my questions here. I am sure we both can benefit from this. Here we go:

1. What is DELTA and when to use it?
2. When to use what precompressor or any of the programs following? (precomp, srep, pzlib, ztool, xtool, PrecompMT, msc )
I have seen different versions of precompressors delivering different results. Sometimes older versions worked better than newer ones?
3. Why is FreeArc alsways the recommended tool? Depending on my files I got better results with also 7z (lzma2) and rar? Is it because of the ease of use for external compressors?
4. Whats are the benefits of xz? (In my unterstanding its just a container, am i right?)
5. Is there any benefit in using FreeArcs successor (FreeArcNext(?)?)
6. Most importantly: Are there any already written guides that I have missed that have answers to my questions?

Cheers

[joekkerr]

Originally Posted by Spexxos

I am also trying to get into the whole compression thing. In my case its primarily for compressing savegames. For that matter I would be interested in the current state of things aswell. I think a might be a bit ahead of you in my knowledge, JoeKkerr, and since I dont want to open up another thread with the same topic I am going to add my questions here. I am sure we both can benefit from this. Here we go:

1. What is DELTA and when to use it?
2. When to use what precompressor or any of the programs following? (precomp, srep, pzlib, ztool, xtool, PrecompMT, msc )
I have seen different versions of precompressors delivering different results. Sometimes older versions worked better than newer ones?
3. Why is FreeArc alsways the recommended tool? Depending on my files I got better results with also 7z (lzma2) and rar? Is it because of the ease of use for external compressors?
4. Whats are the benefits of xz? (In my unterstanding its just a container, am i right?)
5. Is there any benefit in using FreeArcs successor (FreeArcNext(?)?)
6. Most importantly: Are there any already written guides that I have missed that have answers to my questions?

Cheers

precomp is used when there is a file already compressed (i dont know the technical word) . But a file that is , if you try freearc you wont get a good result because its already compressed with low level compression, this makes high level compression lose its effectiveness. Hence, precomp..deflates the file, i.e removes the low level compression and allows it to be compressed by high level compression freearc offers.

an example would be, try RAR ing a normal text file with medium compression, then 7zip that file, you wont get any good result.
now try taht with using precomp first or on raw text file without winrar.

last time i checked (which was a long time ago) LZMA (7zip) was better than freearc for text files.
That brings us to your next question, why freearc you ask ?
Some files get better compression ratios with Freearc , some with 7zip. Mostly its freearc hands down, but some game files like Assassins creed files get better result with 7zip. Depends on game

[78372]

Originally Posted by Spexxos

I am also trying to get into the whole compression thing. In my case its primarily for compressing savegames. For that matter I would be interested in the current state of things aswell. I think a might be a bit ahead of you in my knowledge, JoeKkerr, since I dont want to open up another thread with the same topic I am going to add my questions here. I am sure we both can benefit from this. Here we go:

1. What is DELTA and when to use it?
2. When to use what precompressor or any of the programs following? (precomp, srep, pzlib, ztool, xtool, PrecompMT, msc )
I have seen different versions of precompressors delivering different results. Sometimes older versions worked better than newer ones?
3. Why is FreeArc alsways the recommended tool? Depending on my files I got better results with also 7z (lzma2) and rar? Is it because of the ease of use for external compressors?
4. Whats are the benefits of xz? (In my unterstanding its just a container, am i right?)
5. Is there any benefit in using FreeArcs successor (FreeArcNext(?)?)
6. Most importantly: Are there any already written guides that I have missed that have answers to my questions?

Cheers

1. DELTA is a preprocessor for executables and wav audio files. You get better ratio by using delta+lzma rather than just lzma for those files.
2. Know your data, use Razor12911's GFS for testing what to use. If your data contains zlib, use pzlib from ztool/xtool/normal pZlib v3, if contains lz4, use plz4 from ztool, if it has .forge archives from AC games, use AFR, if game is from unreal engine, use uelr and etc. etc. Use msc before anything if your data contains multimedia data.
3. Yep, external compressors, you can't use them with rar and you need to learn coding for using them with 7zip, and with FreeArc you can use them just by editing arc.ini
4. Yeah, xz is a container for LZMA2 compression which has better decompression speed and multithreading which doesn't exist in lzma
5. Yes, you get more speed, but they aren't much matured yet.
6. Check the forum, you may find more.



Joekkerr and others, if you want to know something, ask questions, don't say that I want to know everything, be specific, it will be help for others to answer.

[joekkerr]

Ok i'll be as specific as possible
I remember using precomp + srep and freearc combo on gta 5, but it failed, failed as in compression ratio was barely 70% i.e only 30% got reduced.
IIRC precomp didnt deflate much of the files.

second qn. I looked at the 'specific games technique thread' , i can understand the commands for an executable (-mx, -m2500 ,etc) but i saw some new programs for example here this line

pzlib:m3:max:c128m:t100p+srep:m2f:a2:lolz:d32 without audio file size 16.8gb.

i googled LOLZ, from my understanding, its must be similar to Freearc.
Pzlib is like precomp (correct me if im wrong)
i know srep.
So what is lolz, & why is it preferred over freearc?
am i correct about pzlib ?

Know your data, use Razor12911's GFS for testing what to use. If your data contains zlib, use pzlib from ztool/xtool/normal pZlib v3, if contains lz4, use plz4 from ztool, if it has .forge archives from AC games, use AFR, if game is from unreal engine, use uelr and etc. etc. Use msc before anything if your data contains multimedia data.

are the programs you mentioned are preprocessors like precomp ?

whats an 'msc'?

from my understanding, freearc+precomp+srep was the combo to go, whats the current ones? (i understand different for specific games, but in general ?)

and with FreeArc you can use them just by editing arc.ini

i assume you're talking about using other programs just like we do in a 'batch' file

Yeah, xz is a container for LZMA2 compression which has better decompression speed and multithreading which doesn't exist in lzma

i'll tell you what i udnerstood and then you correct me.
LZMA type of compression is from 7zip, i suppose this xz is a program (which you refer to as container) has lzma2 with decomp speed and multithread.

 wav audio files

I think using lame.exe for temporary converting them into mp3 then unpacking them into wav would be much efficient right ?

but they aren't much matured yet

as in ? unstable ?