Masked Compression (Ultimate compression in one go)

[panker1992]

I need to post, a very nasty bug i fixed yesterday but it has nothing to do with decompression

it's high likely that you have a leak somewhere please post what you compress
which method and what parameters, any modification ....

[romulus_ut3]

Here's my "Advanced_Data_Comp.bat"

@echo off
color 0c
Title Compressor - by Panker1992
echo.   **************************************************************************
echo.   *    External Options for  Compressor  By Panker1992                     *
echo.   * Modes: Options stored in arc.ini see below for best results            *
echo.   * 1) fast =super fast, optimal =best w/o precomp, max =optimal+precomp *  
echo.   * smartmax =masked precomp, smartlzt =smartmax + lzt, legacy =oldmethod  *
echo.   * devastate = heavy duty comp, will take so much time you will freak     *
echo.   * 2) I added txt, void and jpeg as external for custom made bat files    *
echo.   **************************************************************************
pause
@echo Recompressing Data...
arc a -lc1024 -ld1024 -ep1 -ed -r -hp -pYourpassword -w.\ Data\data.bin -mstrong  "G:\Sniper Elite V2\*"
::examples, I recommend optimal and smartmax for ultimate results 
::devastate has ultimate compression but its so slow that its best only when you store long term
::arc a -lc1024 -ld1024 -ep1 -ed -r -hp -pYourpassword -w.\ Data\data1.bin -mfast  "D:\Games\Steam\*"
::arc a -lc1024 -ld1024 -ep1 -ed -r -hp -pYourpassword -w.\ Data\data1.bin -mstrong  "D:\Games\Steam\*"
::arc a -lc1024 -ld1024 -ep1 -ed -r -hp -pYourpassword -w.\ Data\data1.bin -mdevastate  "D:\Games\Steam\*"
::arc a -lc1024 -ld1024 -ep1 -ed -r -hp -pYourpassword -w.\ Data\data1.bin -mlegacy  "D:\Games\Steam\*"
echo *******************************************************************************
echo.
echo                Compressing is Completed
echo.
echo *******************************************************************************
pause

And here's my arc.ini:

[Compression methods]
; For Tiny SFX usage, dont use anything else other than that !!! 
sfx       =  msc+srep+delta+exe2+4x4:lzma:100m:max/$text=dict:p:128m+lzp:64m:105:d1m:s32:h22+ppmd:12:192m
sfxmax    =  precomp+msc+srep+delta+exe2+4x4:lzma:100m:max/$text=dict:p:128m+lzp:64m:105:d1m:s32:h22+ppmd:12:192m
lzma2     =  4x4:lzma:a1:mfbt4:d256m:fb273:mc1000:lc8
oldtxt    =  dict:p:128m+lzp:64m:105:d1m:s32:h22+ppmd:12:192m
; For Masked Compression Usage   (precomp:zl98:d1)
legacy    = msc+srep+delta+exe2+4x4:lzma:a1:mfbt4:d256m:fb273:mc1000:lc8
sonido    = msc+srep64+delta+exe2+zstd/$jpgsolid=jpg+zstd/$mp3solid=mp3+zstd/$void=srep64+zstd/$text=dict:p:128m+lzp:64m:105:d1m:s32:h22+ppmd:12:192m
fast      = msc+srep64+delta+exe2+xz/$jpgsolid=jpg+zstd/$mp3solid=mp3+zstd/$void=srep64+zstd/$text=nz/$precomp=msc+srep64+delta+zstd
optimal   = msc+srep64+delta+exe2+xz/$jpgsolid=jpg+zstd/$mp3solid=mp3+zstd/$void=srep64+zstd/$text=nz
lztmax    = msc+srep64+delta+exe2+lzt/$jpgsolid=jpg+zstd/$mp3solid=mp3+zstd/$void=srep64+zstd/$text=nz/$precomp=precomp+msc+srep64+delta+lzt
strong    = msc+srep64+delta+exe2+xz/$jpgsolid=jpg+zstd/$mp3solid=mp3+zstd/$void=srep64+zstd/$text=nz/$precomp=precomp+msc+srep64+delta+lz77     
ultra     = precomp+msc+srep64+delta+exe2+lz77/$jpgsolid=jpg+zstd/$mp3solid=mp3+zstd/$void=srep64+zstd/$text=nz
; these 2 methods provide ultimate compression ratios but they are BETA at the moment 
refl8     = msc+srep64+delta+exe2+xz/$jpgsolid=jpg+zstd/$mp3solid=mp3+zstd/$void=srep64+zstd/$text=nz/$precomp=reflate+msc+srep64+delta+lz77 
devastate = precomp+msc+srep64+delta+exe2+nzmax/$jpgsolid=jpg+zstd/$mp3solid=mp3+zstd/$void=srep64+zstd/$text=nz

[External compressor:srep]
header = 0
packcmd   = srep -m3f -a1 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = srep -d -s -- <stdin> <stdout>

[External compressor:srep64]
header = 0
packcmd   = srep64 {options} -m3f -d1g -a2 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = srep64 -d -s -- <stdin> <stdout>

[External compressor:xz]
header = 0
packcmd = 7z a -txz -an -mcrc=0 -m1=lzma2:d27:fb=273:mf=bt4:mc=1000000:lc=4:lp=0 -mmt=on -mx9 -si -so <stdin> <stdout>
unpackcmd = dec x -txz -an -y -si -so <stdin> <stdout>

[External compressor:precomp]
header = 0
packcmd   = precomp -slow -t-j -o$$arcpackedfile$$.tmp  $$arcdatafile$$.tmp
unpackcmd = precomp -o$$arcdatafile$$.tmp -r $$arcpackedfile$$.tmp

[External compressor:reflate]
header = 0
packcmd    = reflate -process $$arcdatafile$$.tmp $$arcpackedfile$$.tmp 
unpackcmd  = reflate -restore123 $$arcpackedfile$$.tmp $$arcdatafile$$.tmp

[External compressor:zstd]
header = 0
packcmd   = zstd -f -9 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = zstd -d -f <stdin> <stdout>

[External compressor:rep,lzma,4x4,delta,exe]

header = 0

packcmd   = fazip -i0   compress:{compressor}{:option} <stdin> <stdout>

unpackcmd = fazip -i0 decompress:{compressor}{:option} <stdin> <stdout>


[External compressor:jpg,mp3]
header = 0
packcmd   = msr -fast -cn -t+j3 -o$$arcpackedfile$$.tmp  $$arcdatafile$$.tmp
unpackcmd = msr -o$$arcdatafile$$.tmp -r $$arcpackedfile$$.tmp
solid = 0

[External compressor:lz77]
header = 0
packcmd = 7z a -txz -an -mcrc=0 -m1=lzma2:d200m:fb=273:mf=bt4:mc=10000:lc=4:lp=0 -mmt=2 -mx9 -si -so <stdin> <stdout>
unpackcmd = dec x -txz -an -y -si -so <stdin> <stdout>

[External compressor:msc]
;Alt: c -v -f Forced //-frog=9 use this instead of -tak=9
header = 0
packcmd = MSC c -v -wav=1 -raw=1 -bmp=1 -ddsraw=1 -ddsdxt=1 -mp3=1 -bmf=9s -tak=9 -dxt=1 -lzma=hc4,lc8,lp2,pb2,fb273,mc1000 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp

[External compressor:nz]
;options -cc=best -m3g=memory -nm 
packcmd   = nz a -r -v -cc -m3g -br128m -bw128m -p2 -t0 -nm $$arcpackedfile$$.tmp.nz $$arcdatafile$$.tmp   
unpackcmd = nz x -m1g -t0 -p0 $$arcpackedfile$$.tmp.nz $$arcdatafile$$.tmp     
datafile   = $$arcdatafile$$.tmp
packedfile = $$arcpackedfile$$.tmp.nz

[External compressor:nzmax]
;options -cc=best CM algo -m3g=memory -nm ; -cO fast option BWT algo
packcmd   = nz a -r -v -cc -m8g -br128m -bw128m -p4 -t0 -nm $$arcpackedfile$$.tmp.nz $$arcdatafile$$.tmp   
unpackcmd = nz x -m1g -t0 -p0 $$arcpackedfile$$.tmp.nz $$arcdatafile$$.tmp     
datafile   = $$arcdatafile$$.tmp
packedfile = $$arcpackedfile$$.tmp.nz

[External compressor:lzt]
;x64 version ;fast: -32 ;max:-p1 -49 -b200
packcmd   = lzt -p1 -49 -b200 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp   
unpackcmd = lzt -d $$arcpackedfile$$.tmp $$arcdatafile$$.tmp   
datafile   = $$arcdatafile$$.tmp
packedfile = $$arcpackedfile$$.tmp

[External compressor:precomp045]
; This is a method in test, dont use this unless you are a developer or Beta tester, I DO NOT PROVIDE PUBLIC SUPPORT YET (Felice dont update DLG with this :P )
header = 0
packcmd   = msr -intense0 -cn -zl98 -d1 -o$$arcpackedfile$$.tmp  $$arcdatafile$$.tmp
unpackcmd = msr -o$$arcdatafile$$.tmp -r $$arcpackedfile$$.tmp

Here's my Unpacker.bat:

@echo off
setlocal enableextensions 


unarc.exe x -o+ -pYourpassword -dptest data.bin 

pause

And here's the arc.ini from the Decomp Tools folder:

[External compressor:srep]
header = 0
packcmd   = srep -m3f -a1 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = srep -d -s -- <stdin> <stdout>

[External compressor:srep64]
header = 0
packcmd   = srep64 {options} -m3f -d1g -a2 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = srep64 -d -s -- <stdin> <stdout>

[External compressor:xz,lz77]
header = 0
packcmd   = 7z a -txz -an -mcrc=0 -m1=lzma2:d450m:fb=273:mf=bt4:mc=1000000:lc=4:lp=0 -mmt=4 -mx9 -si -so <stdin> <stdout>
unpackcmd = dec x -txz -an -y -si -so <stdin> <stdout>

[External compressor:precomp]
header = 0
packcmd   = precomp -slow -t-j -o$$arcpackedfile$$.tmp  $$arcdatafile$$.tmp
unpackcmd = precomp -o$$arcdatafile$$.tmp -r $$arcpackedfile$$.tmp

[External compressor:reflate]
header = 0
packcmd    = reflate -process $$arcdatafile$$.tmp $$arcpackedfile$$.tmp 
unpackcmd  = reflate -restore123 $$arcpackedfile$$.tmp $$arcdatafile$$.tmp

[External compressor:zstd]
header = 0
packcmd   = zstd -f -9 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = zstd -d -f <stdin> <stdout>

[External compressor:jpg,mp3,precomp045]
; to use precomp045 in this case you need to delete packjpg_dll1.dll packjpg_dll.dll and cls-precomp.dll from this menu
header = 0
packcmd   = msr -fast -cn -t+j3 -o$$arcpackedfile$$.tmp  $$arcdatafile$$.tmp
unpackcmd = msr -o$$arcdatafile$$.tmp -r $$arcpackedfile$$.tmp
solid = 0

[External compressor:msc]
;Alt: c -v -f Forced //-frog=9 use this instead of -tak=9
header = 0
packcmd = MSC c -v -wav=1 -raw=1 -bmp=1 -ddsraw=1 -ddsdxt=1 -mp3=1 -bmf=9s -tak=9 -dxt=1 -lzma=hc4,lc8,lp2,pb2,fb273,mc1000 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp

[External compressor:nz,nzmax]
;options -cc=best -m3g=memory -nm 
packcmd   = nz64 a -r -v -cc -m3g -br128m -bw128m -p2 -t0 -nm $$arcpackedfile$$.tmp.nz $$arcdatafile$$.tmp   
unpackcmd = nz64 x -m1400m -t0 -p0 $$arcpackedfile$$.tmp.nz $$arcdatafile$$.tmp     
datafile   = $$arcdatafile$$.tmp
packedfile = $$arcpackedfile$$.tmp.nz

[External compressor:lzt]
;x64 version ;fast: -32 ;max:-p1 -49 -b200
packcmd   = lzt -p1 -49 -b200 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp   
unpackcmd = lzt -d $$arcpackedfile$$.tmp $$arcdatafile$$.tmp   
datafile   = $$arcdatafile$$.tmp
packedfile = $$arcpackedfile$$.tmp

[Andrey167]

is it possible to somehow make the progress bar during the installation smoothly? When using this method of compression is not a smooth progress bar and then hangs up and goes to 100%

[coveiro]

Decomp Tools error



This error occurs on Windows 7, I tested it on Windows 8.1 works, how to fix it?

[panker1992]

Hello Dear Community, as of yesterday i am fixing a VERY VERY nasty BUG

i haven't finished fixing it , took me longer than i expected

but i hope when its done, Masks will surely look different

regarding this bug, it fixes masked compression file misdetection

i have found that if many masks are used in parallel detection will not work and files will be compressed using different method

so i decided to make new masks from scratch and write a new arc.groups
to make packing detect correctly and use all tools in the process

i am so taken in by this new work, that i didn't even log in the forums :/

allow me some time, and please use absolutely masks 2.1 legacy to work with

thanks for understanding

@andrey
i am working on a solution for smooth out the progress (progress.inf)

this will show the correct progress in the future and new precomp0.45 will be used extensively

[Amsal]

Panker1992, see this


I used Masked Compression v2.2 and with Blackbox v2 updated one, which is modified for Masked Compression v2.2 by you! So can you tell me any solution for this problem!
Thanks in advance

[panker1992]

I am in the middle of fixing a very serious bug, i have almost finished and my solution works

please use masked compression v 2.1, or wait for v2.3

i am testing my masks and my new v2.3 might be up by tomorrow

i cannot promise anything but i hope tomorrow it will be there


edit: Masked Compression v2.3 is up and running

i think the most solid work i did so far

This is a silent Release, none will notice it :P

[Amsal]

Please help me with this!

And bro, one more question, I found two folders of Masked Compression v2.3 in the archive, so what is the difference or they are same

[panker1992]

I forgot to include msr program in the folder i am updating the 7z file with msr inside

[Amsal]

Bro, I tried the new updated Masked Compression v2.3 but Still I am getting this error(I used lztmax) Please help me bro!



Thanks in advance

[panker1992]

i will help you gladly but .... provide info please ....

you said method lztmax what did you compress, did you turn off antivirus ??

was there any mp3 or jpg file to compress?? if you find a file names thumb.db delete it
it causes errors on some occasions

[Amsal]

Originally Posted by panker1992

i will help you gladly but .... provide info please ....

you said method lztmax what did you compress, did you turn off antivirus ??

was there any mp3 or jpg file to compress?? if you find a file names thumb.db delete it
it causes errors on some occasions

Bro, I compressed a software of name Advanced Installer 11.7, and I don't have antivirus, even my windows defender is off

[panker1992]

ok ok

remove this line

solid = 0 from arc.ini
its down on msr

this will end your problem though i dont recommend to turn off this feature :/

[Amsal]

Originally Posted by panker1992

ok ok

remove this line

solid = 0 from arc.ini
its down on msr

this will end your problem though i dont recommend to turn off this feature :/

ok bro, it helped me, thanks, one more advice is needed, which compression algo is best in masked compression? And bro, whenever I use lztmax then I got a freearc353465363.tmp folder when installation is completed, but when i use Ultra or brute then I didn't got this problem!

[panker1992]

There is no best algo, every algo play within 3 dimensions

one is compression speed
second is ratio
third is decomp speed

Every algo can have 2 out 3 from the above virtues, so a masked process will always win over the classic process

no algo can have all the above, nanozip has best compression ratio but in terms of speed both comp and decomp it's useless

if there was a perfect algo then this masked thing would be useless :/

I hope i helped Amsal