FileForums

FileForums (https://fileforums.com/index.php)
-   Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)
-   -   Masked Compression (Ultimate compression in one go) (https://fileforums.com/showthread.php?t=97530)

panker1992 07-01-2016 11:25

2 Attachment(s)
it only appears in windows 10 guys, i have no problems in windows 8.1

ramazan and lolaya both your windows 10 produce this error

test method -mmax or reverse exe2 and delta experiment with this i dont have windows 10 to try :(


Edit 1) there you go it works perfectly ....... what are you doing wrong ?

edit 2) Both of you and windows 10 user try devastate just to clarify if this works

ramazan19833 07-01-2016 11:42

Hello
 
I have a great interest in their work panker1992 in the work that I'm trying to follow the forum who is in a lot of Labor and help you the friends who I owe a debt of thanks to updates and future updates that you're giving a place in gelek: I'm interested to know when Deni....

1) Add a method in the related Inno
2) use the new building the meta-DTD 4.4
3) add the string to stdout and lzt
4) update the bin Re-encoder, fix better menu , better options with the help of some kind should also add side by side
5) update for Windows zstd Compression

panker1992 07-01-2016 12:48

windows defender makes this problem ??



good to know

panker1992 07-01-2016 13:12

generally windows defender is useless, self compiled programs are scanned and killed


programs i have compiled myself
7z.exe and reduced version
zstd
packjpg
lz77 custom version
lzham
maybe more

most of them are found inside my compression pack
so no wonder that is doent work :/

small bug fix, extract this to the v2 folder and replace precomp.exe

edit update 1 moved to main post :)

ramazan19833 07-01-2016 14:18

Hello,Help
 
Hello,Help

http://i.hizliresim.com/knvW6m.jpg

Mask Data Compression

@echo off
color 0c
Title Compressor - by Panker1992
echo. ************************************************** ************************
echo. * External Options for Compressor By Panker1992 *
echo. * Modes: Options stored in arc.ini see below for best results *
echo. * 1) sonido =super fast, optimal =best w/o precomp, max =optimal+precomp *
echo. * smartmax =masked precomp, smartlzt =smartmax + lzt, legacy =oldmethod *
echo. * devastate = heavy duty comp, will take so much time you will freak *
echo. * 2) I added txt, void and jpeg as external for custom made bat files *
echo. ************************************************** ************************
pause
@echo Recompressing Data...
arc a -lc1024 -ld1024 -ep1 -ed -r -hp -pYourpassword -w.\ Data\data1.bin -moptimal "C:\Program Files (x86)\Adobe\*"
::examples, I recommend optimal and smartmax for ultimate results
::devastate has ultimate compression but its so slow that its best only when you store long term
::arc a -lc1024 -ld1024 -ep1 -ed -r -hp -pYourpassword -w.\ Data\data1.bin -msonido "D:\Games\Steam\*"
::arc a -lc1024 -ld1024 -ep1 -ed -r -hp -pYourpassword -w.\ Data\data1.bin -msmartmax "D:\Games\Steam\*"
::arc a -lc1024 -ld1024 -ep1 -ed -r -hp -pYourpassword -w.\ Data\data1.bin -mdevastate "D:\Games\Steam\*"
::arc a -lc1024 -ld1024 -ep1 -ed -r -hp -pYourpassword -w.\ Data\data1.bin -mlegacy "D:\Games\Steam\*"
echo ************************************************** *****************************
echo.
echo Compressing is Completed
echo.
echo ************************************************** *****************************
pause


MainScriptEnglish.iss

arc.ini

[External compressor:reflate]
header = 0
unpackcmd = reflate -restore123 $$arcpackedfile$$.tmp $$arcdatafile$$.tmp

[External compressor:lz77]
header = 0
unpackcmd = lz77 d -si -so <stdin> <stdout>
[External compressor:precomp]
header = 0
packcmd = precomp -intense0 -c- {options} -o$$arcpackedfile$$.tmp $$arcdatafile$$.tmp
unpackcmd = precomp -o$$arcdatafile$$.tmp -r $$arcpackedfile$$.tmp

[External compressor:srep]
header = 0
packcmd = srep {options} -m3f -d1g -a2 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = srep {options} -d -s - - <stdin> <stdout>

[External compressor:lzmh]
packcmd = 7z.exe a -t7z -mcrc=0 -m0=bcj2 -m1=lzmh:{option}:fb256 -mx $$arcpackedfile$$.7z $$arcdatafile$$.dll
unpackcmd = 7z.exe x -y $$arcpackedfile$$.7z $$arcdatafile$$.dll
packedfile = $$arcpackedfile$$.7z
datafile = $$arcdatafile$$.dll

[External compressor:7z]
packcmd = 7z.exe a -t7z -mcrc=0 -m0=bcj2 -m1=lzma:fb256 -mx9 $$arcpackedfile$$.7z $$arcdatafile$$.dll
unpackcmd = 7z.exe x -y $$arcpackedfile$$.7z $$arcdatafile$$.dll
packedfile = $$arcpackedfile$$.7z
datafile = $$arcdatafile$$.dll

[External compressor:bcj2]
packcmd = 7z.exe a -mcrc=0 -m0=bcj2 $$arcpackedfile$$.7z $$arcdatafile$$.dll
unpackcmd = 7z.exe x -y $$arcpackedfile$$.7z $$arcdatafile$$.dll
packedfile = $$arcpackedfile$$.7z
datafile = $$arcdatafile$$.dll


[External compressor:rar]
;-m5:1024
;-md512m
cmem = 130
dmem = 130
default=32
packcmd = rar a -s -m5 -ma5 -md1024m -mt4 $$arcpackedfile$$.tmp.rar $$arcdatafile$$.tmp
unpackcmd = rar e -mt4 $$arcpackedfile$$.tmp.rar
datafile = $$arcdatafile$$.tmp
packedfile = $$arcpackedfile$$.tmp.rar

[External compressor:packet]
;x64 version
packcmd = packet.exe a -v -r -mx -b512 -h4 $$arcpackedfile$$.tmp $$arcdatafile$$.tmp
unpackcmd = packet.exe x -v $$arcpackedfile$$.tmp $$arcdatafile$$.tmp
datafile = $$arcdatafile$$.tmp
packedfile = $$arcpackedfile$$.tmp

[External compressor:split]
;x64 version
packcmd = split.exe s $$arcpackedfile$$.tmp 1024 $$arcdatafile$$.tmp
unpackcmd = split.exe m $$arcpackedfile$$.tmp $$arcdatafile$$.tmp
datafile = $$arcdatafile$$.tmp
packedfile = $$arcpackedfile$$.tmp

[External compressor:zm]

packcmd = zmolly64.exe -b99 -e $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = zmolly64.exe -d $$arcpackedfile$$.tmp $$arcdatafile$$.tmp

[External compressor:plzma]
;x64 version
packcmd = plzma c2 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp 26 1024 273 8 0 0 4096 128 16 272
unpackcmd = plzma d $$arcpackedfile$$.tmp $$arcdatafile$$.tmp
datafile = $$arcdatafile$$.tmp
packedfile = $$arcpackedfile$$.tmp

[External compressor:bssc]
;x64 version
packcmd = bssc.exe e $$arcdatafile$$.tmp $$arcpackedfile$$.tmp -b1024aeflzrs
unpackcmd = bssc.exe d $$arcpackedfile$$.tmp $$arcdatafile$$.tmp
datafile = $$arcdatafile$$.tmp
packedfile = $$arcpackedfile$$.tmp

[External compressor:rzm]
;x64 version
packcmd = rzm.exe c $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = rzm.exe d $$arcpackedfile$$.tmp $$arcdatafile$$.tmp
datafile = $$arcdatafile$$.tmp
packedfile = $$arcpackedfile$$.tmp

[External compressor:smoke]
;x64 version
packcmd = smoke.exe $$arcdatafile$$.tmp $$arcpackedfile$$.tmp

[External compressor:pause]
;x64 version
packcmd = cmd.exe "type exit and press ENTER please"

[External compressor:cls]
;x64 version
packcmd = start /b cmd.exe cls


[External compressor:lzturbo]
;x64 version
packcmd = lzturbo {options} $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = lzturbo -d $$arcpackedfile$$.tmp $$arcdatafile$$.tmp
datafile = $$arcdatafile$$.tmp
packedfile = $$arcpackedfile$$.tmp


[External compressor:exdupe]
;x64 version
packcmd = exdupe.exe -x0 -m512 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = exdupe.exe -R $$arcpackedfile$$.tmp $$arcdatafile$$.tmp
datafile = $$arcdatafile$$.tmp
packedfile = $$arcpackedfile$$.tmp

[External compressor:zpaq]
;-method s4.3ci1
;x64 version
packcmd = zpaq64.exe a $$arcpackedfile$$.tmp $$arcdatafile$$.tmp -method 46,36,26
unpackcmd = zpaq64.exe x $$arcpackedfile$$.tmp $$arcdatafile$$.tmp -summary
datafile = $$arcdatafile$$.tmp
packedfile = $$arcpackedfile$$.tmp

[External compressor:xdelta3]
;x64 version
;-S djw9
packcmd = xdelta3.exe -0 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = xdelta3.exe -d $$arcpackedfile$$.tmp $$arcdatafile$$.tmp
datafile = $$arcdatafile$$.tmp
packedfile = $$arcpackedfile$$.tmp

[External compressor:bro]
;brotli google compressor
packcmd = bro.exe -q 9 -v -f -i $$arcdatafile$$.tmp -o $$arcpackedfile$$.tmp
unpackcmd = bro.exe -d -i $$arcpackedfile$$.tmp -o $$arcdatafile$$.tmp
datafile = $$arcdatafile$$.tmp
packedfile = $$arcpackedfile$$.tmp


[External compressor:zh]
;x64 version
packcmd = zhuff.exe -c2t2 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = zhuff.exe -d $$arcpackedfile$$.tmp $$arcdatafile$$.tmp
datafile = $$arcdatafile$$.tmp
packedfile = $$arcpackedfile$$.tmp

[External compressor:nz]
;x64 version
packcmd = nz.exe a -r -v -cc -m512m -t0 -br128m -bw128m -p2 $$arcpackedfile$$.tmp.nz $$arcdatafile$$.tmp
unpackcmd = nz.exe x -m256m -t0 -p4 $$arcpackedfile$$.tmp.nz $$arcdatafile$$.tmp
datafile = $$arcdatafile$$.tmp
packedfile = $$arcpackedfile$$.tmp.nz

http://www.dosya.tc/server6/npl0gb/Dark_Blue.rar.html

panker1992 07-01-2016 14:29

razaman i didnt provide the above files, i have uploaded a blackbox Inno in page 1 for you to download

i cannot rewrite this sorry :/ no time

Amsal 09-01-2016 00:55

I am stuck from 1 day in MSC, Panker1992 can you tell the solution for this? I am repacking SIMS 4 game!
http://i.imgur.com/zI6Azdc.png

panker1992 09-01-2016 02:19

my first impression is that you have -f switch inside arc.ini

-f switch is no limit on input size
then msc tries to compress something very big and fails

try to disable the command

here is the correct one
[External compressor:msc]
header = 0
packcmd = MSC c -v -wav=1 -raw=1 -bmp=1 -ddsraw=1 -ddsdxt=1 -mp3=1 -bmf=9s -tak=9 -dxt=1 -lzma=hc4,lc8,lp2,pb2,fb273,mc1000 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp

if this doent solve the case then disable msc and leave everything else alone to do the job :D

i havent tested sims myself, and i am not the author of msc
Though i hope Shegorat will make a new msc sometime in the future :)

Amsal 09-01-2016 04:46

Quote:

Originally Posted by panker1992 (Post 446056)
my first impression is that you have -f switch inside arc.ini

-f switch is no limit on input size
then msc tries to compress something very big and fails

try to disable the command

here is the correct one
[External compressor:msc]
header = 0
packcmd = MSC c -v -wav=1 -raw=1 -bmp=1 -ddsraw=1 -ddsdxt=1 -mp3=1 -bmf=9s -tak=9 -dxt=1 -lzma=hc4,lc8,lp2,pb2,fb273,mc1000 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp

if this doent solve the case then disable msc and leave everything else alone to do the job :D

i havent tested sims myself, and i am not the author of msc
Though i hope Shegorat will make a new msc sometime in the future :)

It is still stuck there after doing this modification

panker1992 09-01-2016 07:03

Read This, Important
 
I am constantly asked about MSC problems, so i decided to give some kind of guide ( troubleshooting) !!!!


First things first i am not the author of msc , nor i can make it magically work
many of these programs arent easy to use on Windows 10 especially !!

I myself have tested windows 10 and i had many errors/warnings etc so i had to roll back to win 8.1 !!! :cool:

From here on I will not help/troubleshoot anything that i havent provided,
there are people here that ask for help with tools and methods that someone else has provided,

Here is a troubleshooting guide, Do those before you decide to post an error!!

1) disable all kinds of antivirus
2) disable the -f switch in msc if you have it on
3) windows defender reported to be delivering many errors on precomp, xz , and msc disable it for good and forget that it ever excisted
4)when a method gives an error, try a different approach

5) Do not by any means mix tools together, from different methods the only sure is that it will break
every author has different approach if you mix them all together it will never work

truerepacks 11-01-2016 03:25

latest version of msc can't use bmf compression, even cls-msc.dll can't extract it
for more information visit http://krinkels.org/threads/msc-medi...mpressor.1587/

panker1992 11-01-2016 04:57

truerepacks thanks for this info ....

unfortunately i dont speak russian nor i can read or understand them,

krinkels.org is a treasure forum in which i cannot participate

please if you have info regarding msc or any other tool that is used inform us like you did just now

panker1992 11-01-2016 05:48

lolaya this sovles your problem ....


remove msc until we get an update on this , i hope in 2016 Shegorat will give us a new one

about your other problem i have found a nice alternative

use lzma:max:512m

this will give you about the same compress ratio as my method xz and lz77 but its gonna be so slow.

xz method uses 4 cores as i have provided making it super fast (lzma2)


post some feedback to let me know if lzma:max:512m worked !!!

hans314 15-01-2016 00:46

Dear Panker,

are you going to update your Masked Data Compression tool with the new Precomp 0.44 version (decomp as well)? I am wondering if there is a better performance + ratio

panker1992 15-01-2016 02:00

Hello Hans,

I will update it in time, i properly havent tested it myself yet.

Christian the author of precomp is working on Multithreaded bruteforce to make this process faster,

Now to make this clear, precomp 0.44 doesnt support stdin to stdout support
until a proper stdin and stdout are implemented, or a cls-precomp.dll is written
i will stay with 0.38 because its faster.

Up untill now i havent seen any improvements on precomp, my tests took the same time with 0.38 and 0.44.

if you want to update your tools you will have to
1) download my alpha precomp i posted it.
2) delete old precomp.exe and packjpg_dll.dll and copy-paste new
3) change the line in arc.ini from -slow -t-j ~~~> -intense0 -cn

Inno changes are
1) delete precomp.exe both packjpg_dll.dll and packjpg_dll1.dll and cls-precomp.dll
2) copy paste new precomp and leave arc.ini as it is :) it will work

NOTE:
Due to the lack of proper stdin - stdout and cls-precomp.dll it will never show progress in Inno, and will take x2 time to unpack

Good luck :)

Amsal 15-01-2016 04:51

And panker, give a new arc.ini for 4gb ram,6gb ram, 8gb ram, because you added update 1, which will change the arc.ini, so upload the arc.ini for 4gb ram, 6gb ram, 8gb ram AGAIN!

Do it fast as possible

panker1992 15-01-2016 05:38

New Optimizations (requested)
 
Quote:

Originally Posted by Amsal (Post 446116)
And panker, give a new arc.ini for 4gb ram,6gb ram, 8gb ram, because you added update 1, which will change the arc.ini, so upload the arc.ini for 4gb ram, 6gb ram, 8gb ram AGAIN!

Do it fast as possible

Just like lightning, i strike again :P .. . . . . . just kidding


New arc.ini for 8G of ram, 12G and 16G

Best optimizations for those ram

i decided to drop 4G ram support because it doesnt worth it,
i dont think that anyone has 4G of ram !!!

you will not find a 12G.7z because i have 12G of ram so it comes optimized for it.

enjoy :)

Amsal 15-01-2016 05:39

Quote:

Originally Posted by panker1992 (Post 446117)
Just like lightning, i strike again :P .. . . . . . just kidding


New arc.ini for 8G of ram, 12G and 16G

Best optimizations for those ram

i decided to drop 4G ram support because it doesnt worth it,
i dont think that anyone has 4G of ram !!!

you will not find a 12G.7z because i have 12G of ram so it comes optimized for it.

enjoy :)

Bro, I has 4gb ram, and in other pc 6gb ram, so please please upload that!

panker1992 15-01-2016 05:43

i will write you for 6G ram :)

your 4G pc will most likely crash :(

lzt(lzturbo) uses minimum 5G

it's up and running

6G ram added per requested :)

anything less than that will freeze, and probably crash your pc to death :(

Amsal 15-01-2016 06:21

http://i.imgur.com/41o2OU7.png?1
http://i.imgur.com/9cDpn3H.png?1

VirtualAlloc failed!

Please tell me a solution for it

panker1992 15-01-2016 06:36

Not enough ram amsal :(

this is error related to not enough ram to allocate :(

let me see if i can lower a bit more ram allocation will post in few minutes

edit

[External compressor:lzt]
;x64 version ;fast: -32 ;max:-p1 -49 -b200
packcmd = lzt -32 -b200 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = lzt -d $$arcpackedfile$$.tmp $$arcdatafile$$.tmp
datafile = $$arcdatafile$$.tmp
packedfile = $$arcpackedfile$$.tmp

@Amsal

lzt doesnt have a direct memory allocation limitation
try this here

and wish for success

Amsal 15-01-2016 06:56

Quote:

Originally Posted by panker1992 (Post 446123)
Not enough ram amsal :(

this is error related to not enough ram to allocate :(

let me see if i can lower a bit more ram allocation will post in few minutes

edit

[External compressor:lzt]
;x64 version ;fast: -32 ;max:-p1 -49 -b200
packcmd = lzt -32 -b200 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = lzt -d $$arcpackedfile$$.tmp $$arcdatafile$$.tmp
datafile = $$arcdatafile$$.tmp
packedfile = $$arcpackedfile$$.tmp

@Amsal

lzt doesnt have a direct memory allocation limitation
try this here

and wish for success

Bro, thanks a lot, it works now THANKS THANKS!! bro, also change this update to the 6G arc.ini, which you uploaded on first page

panker1992 15-01-2016 07:15

i will try to test this, a bit more before i post it on 6G ram

lzt is hard program to control, due to not having direct memory allocation control like lzma and lzma2 and nanozip

felice2011 15-01-2016 07:17

Hey Panker, as always it is discriminated those under of 4GB of RAM...:(:o:D
If you want to test your masks quickly and work with white gloves, I can send you in private, something useful preview, still under development, I need some consulting on your part before you finish the job..;)
I also intend to to give the possibility of choosing quickly in choosing the quantity of RAM to be used during compression (4,8,12,16 and 32gb) instead of modifying the ini file manually.
Something like this..

http://i65.tinypic.com/34tbg54.png

But you need a few days to finish the demo.:)

panker1992 15-01-2016 07:27

i always try to optimize for all ram systems, problem is that some programs refuse to work especially the closed source x64 bit lzturbo :(


Send me the review :D I like the colour already

i will help you as best as i can

rep:512m is obsolete, no need to be used razor kindly explained why :)

zpaq is replaced by nanozip due to better handling on text files, i can always recompile zpaq to test it of course also nanozip uses less ram, 4G ram users can compress with nanozip easily

hans314 15-01-2016 07:38

@Panker, thanks for your feedback, I will wait and hope that there will be a multithreaded precomp (would be really great!).

@Felice: Your creating tool looks quite cool, a lot of possible options integrated

Please both keep up your good work!

panker1992 15-01-2016 07:41

1 Attachment(s)
precomp 0.45 is on the way


trying to figure it and optimize this

it brings packmp3 to the table lossless audio on a very low API :)


i optimized packmp3.exe quite some time ago it was 13% faster than the original

lets see time will tell

felice2011 15-01-2016 07:47

Quote:

Originally Posted by panker1992 (Post 446127)
i always try to optimize for all ram systems, problem is that some programs refuse to work especially the closed source x64 bit lzturbo :(


Send me the review :D I like the colour already

i will help you as best as i can

rep:512m is obsolete, no need to be used razor kindly explained why :)

zpaq is replaced by nanozip due to better handling on text files, i can always recompile zpaq to test it of course also nanozip uses less ram, 4G ram users can compress with nanozip easily

Ok when it's finished you can to test the preview, you tell me if you remove, add or change something.;)

panker1992 15-01-2016 07:50

dont worry :)

You will get a log fully detailed

also you will get
packjpg.exe my new optimized x64 bit version
zstd new optimized version
precomp 0.44 alpha and if i manage precomp 0.45 beta with low api mp3 recompression

Rahmani1811 15-01-2016 08:59

panker bro how to solve this
http://i.imgur.com/pXIKwJt.png

panker1992 15-01-2016 09:03

[External compressor:xz]
header = 0
packcmd = 7z a -txz -an -mcrc=0 -m1=lzma2:d27:fb=273:mf=bt4:mc=1000000:lc=4:lp=0 -mmt=on -mx9 -si -so <stdin> <stdout>
unpackcmd = dec x -txz -an -y -si -so <stdin> <stdout>

used value instead of memory allocation || and changed multithread to automatic

test and feedback please :)

coveiro 15-01-2016 09:12

this configuration still get the same error that always xz,

panker1992 15-01-2016 09:23

this is ram related problem :/

what version of 7z.exe and 7z.dll you use ??




edit) added a 7z.exe reduced that will do the job ;)

feedback if tha tsolved it

coveiro 15-01-2016 09:36

Quote:

Originally Posted by panker1992 (Post 446141)
this is ram related problem :/

what version of 7z.exe and 7z.dll you use ??




edit) added a 7z.exe reduced that will do the job ;)

feedback if tha tsolved it


Worked here, thank you :D

panker1992 15-01-2016 09:38

so the 7z.exe 15.14 version was causing the fuss :/

ok so now i know the problem, i will either post 15.12 or replace with this reduced format i gave you

will see how this behaves

Amsal 15-01-2016 17:46

Now, help me bro, I am running on 6gb ram, and I got this error while unpacking with decompress tools! And bro, one more request, how can I disable packJPG of precomp in the masked compression?

http://i.imgur.com/MmkxU4L.png?1

panker1992 16-01-2016 02:54

if you use my tools for decompression then edit arc.ini

and leave only the options you have used for compression

and for precomp you can use the witch -t-j after -intense0

precomp -intense0 -t-j .......... rest leave it as it is

panker1992 16-01-2016 06:23

New updates on Zstd compressor for Windows

i am working on bink updates right now so stay tuned :)

ramazan19833 16-01-2016 06:46

Zstd_Ultra_Compressor / how to use can you describe it ?
 
Zstd_Ultra_Compressor / how to use can you describe it ?

panker1992 16-01-2016 07:17

1 Attachment(s)
install the program

then go to a file or folder

right click and press compress using zstd :)

there you go :) very very easy

right now i have a bink.bat that is 50 lines code :P will be epic

Teaser :)

New bink and bk2 bat will be able to use efficient your cpu

smart headroom to be used showing the number of files to be processd and curently processing

featuring a smart crop system to perform smart resize and upscaling to 1080p

eye lossless as i call it :) stay tuned


All times are GMT -7. The time now is 09:42.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2026, vBulletin Solutions Inc.
FileForums @ https://fileforums.com