Masked Compression (Ultimate compression in one go)

[panker1992]

@Lucas its fine to use reflate only, tho as i said this version of inno setup i use refuses to see reflating as external compressor.
but if you have found a way to use it that is good news

did you notice any difference in ratio or speed while using the Masks ?? will be glad to know

@vint i saw that now i only check the entire data.bin for ratio

this is because some data are non compressible

in this case there are 3 scenarios
1) turn precomp to brute mode from intensive
2) lossy compress the files manually
3) if it is a packed format of games for example .pack formats used by all Total Wars
then you extract the content and compress it and then make a bat to restore it

[Lucas65]

Originally Posted by panker1992

@Lucas its fine to use reflate only, tho as i said this version of inno setup i use refuses to see reflating as external compressor.
but if you have found a way to use it that is good news

did you notice any difference in ratio or speed while using the Masks ?? will be glad to know

This type of compression is very good and also very fast. In my case I have that problem with precomp but fortunately I have, more or less, solved.
With reflate I only did the test and it would be really great if you could find a way to decompress in inno because in many cases exceptional results.
I need another help if you can: if for example use reflate alone, with this line arc.exe a -ep1 -r -ed -s; -mreflate data.arc "D:\Giochi\Gta san andreas\*, then how can I decompress with a bat file? I honestly have no idea.
I thank you in advance Panker!

[panker1992]

I haven't tested -mreflate alone I mean without srep and lzma or lzma2

But I have found a glitch
Download my decompress for ripping and masked compression

Copy arc.ini from my masked compression zip to decompress for ripping and replace it

Then try to decompress
The error that didn't let rwflate work is solved :P

[Lucas65]

Great. Now I work in almost all ... It seems also precomp. Thank you very much for all your help and for the work you've done!

[panker1992]

Lucas your welcome

i always try to troubleshoot, and try to know how the programs work so when i see errors
i know where they come from

you seemed to have the most difficult situation in the forum, i am really glad you managed to get it right

any feedback would be nice on the methods, have you seen ratio or at least speed improvements ??? has masked comp/decomp worked well ??

[hans314]

Dear Panker,

I would also like to thank you for your great script+grouping of files! The idea is great!

I´m having a little problem with some files, getting the message:
##################
Recompressed streams: 0/6
PNG streams (multi): 0/1
zLib streams (intense mode): 0/5

None of the given compression and memory levels could be used.
There will be no gain compressing the output file.

ERROR: general (de)compression error in precomp
##################
it occurs on Win8.1+Win10+VMWare (blank installation of Win8.1)

I just remarked, that it hungs up by converting a bnk file (but other bnk files worked) and in another case an ico and a wmv file has caused the problem.

Should I need to sort them out or change anything in the precomp settings?

Thanks for your help!

[panker1992]

thanks for feedback

dear friend all files and how they get compressed are chosen by the file named arc.groups

the name of the filter starts with $text for example
and then all *.txt *.ini and all files that are text alike will be compressed by what you put in front of /$text=rep:500+zstd for example this will take all texts and compress them separetaly then put them back in the entire arc or bin or cab whatever

now there is a filter that takes all files that are unknown to the mask and compresses them generally by the usage of the

-mmsc+srep64+...+lz77 for example


if you open your arc.group you will find that wmv files are under void = uncompressible data

if you want add *.bnk files under void too
and replace this line in general comp /$void=msc+srep64+zstd thus ensuring that precomp will never see bnk files

bnk is uncompressible data format is has nothing to do with precomp.exe

i hope that you got me
sorry for long text

[hans314]

Dear Panker,

thanks for your fast reply, unfortunately it doesn´t work for me, he still thinks that he needs to use precomp, altough I´ve added a wmv and ico entry in the $void section... (the wmv entry was already there like you said). For example when I try to compress only one wmv file, it still uses precomp.

Thanks again!

P.S. you can test it with the attached icon

[panker1992]

[External compressor:precomp]
header = 0
packcmd = precomp -slow -t-j -o$$arcpackedfile$$.tmp $$arcdatafile$$.tmp
unpackcmd = precomp -o$$arcdatafile$$.tmp -r $$arcpackedfile$$.tmp


i had already began to test precomp 3.8 as a replacement for 4.2 which made it little bit easier

i will update all projects to precomp 3.8 as it is more stble

[hans314]

Great it works thanks a lot! Maybe one idea: what about a "comparing" tool with all precomp versions? Maybe you can integrate it in your great script

I also had the idea, that the batch finds out the biggest files and do some compressing tests with intense or brute method (to also get GTA5 files compressed). After the test run, there will be a short message saying for example" Precomp 4.2 with intense mode" is the best option.

I think it will be hard to implement automatically, but you could do a big script for all precomp versions and options so that you will be able to test it on one big file which you manually choose. Then you can compare the results and set the best option.

I hope you understand what I mean

[panker1992]

precomp takes a lot of time but what you say i can do it with some help

it was the 9 hours that precomp takes that forced to make zstd part of my project then xz and then masks just wait a little longer

i have been working 40 hours straight to get a working Inno setup with

precomp + reflate + srep64 + fazip with all goodies + nanozip + zpaq + zstd ......i can go on forever i am almost done


Here take a look <3

[hans314]

Wow this sounds quite good, I appreciate your good and hard work!

I love the idea of masking files and bringing a tool to handle every compression method and I will test it a lot but I also like the scripts, so if both could be possible we will have a new wonder of the world

[panker1992]

hans i assure you that by the end of this year i will be "Panker the Compression Manipulator "

just kidding

razor went and created a damn hard program again but it's so appealing that i decided to work on it and maybe have Inno + masks altogether

hope you enjoy it when i give it to public

[hans314]

Hehe

I´m sure I will enjoy it cannot keep waiting for it

[panker1992]

alright brand new Update Inno_v3 is up

big thanks to razor as i couldnt have done it without him