FileForums

FileForums (https://fileforums.com/index.php)
-   Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)
-   -   Masked Compression (Ultimate compression in one go) (https://fileforums.com/showthread.php?t=97530)

Amsal 16-01-2016 18:19

Bro, I repack on LINUX os UBUNTU, and I doesn't have C: drive, so can you give me how to use it on desktop, and bro can you give me a tutorial of repacking with bink?
Desktop Path - /home/DARCK/Desktop
Thanks in advance

ramazan19833 16-01-2016 21:31

Hello
 
panker1992

brother have a very strong code that you think ?
The more powerful do you have one ?


-mprecomp:precomp+msc+srep64+delta+exe2+nzmax/$jpgsolid=jpg+zstd/$void=srep64+zstd/$text=srep64+rep:77mb+nz

felice2011 17-01-2016 02:00

I await confirmation from Panker, I tested various forms of data files of approximately 200mb

Code:

Pre-Standard = msc+srep64+delta+exe2+xz/$jpgsolid=jpg+zstd/$void=srep64+zstd/$text=srep64+zpaq

Pre-Optimal = msc+srep64+delta+exe2+lzt/$jpgsolid=jpg+zstd/$void=srep64+zstd/$text=srep64+rep:77mb+nz/$precomp=precomp+msc+srep64+delta+lzt

Pre-Max = msc+srep64+delta+exe2+xz/$jpgsolid=jpg+zstd/$void=srep64+zstd/$text=srep64+nz/$precomp=precomp+msc+srep64+delta+lz77

    OR = msc+srep64+delta+exe2+zstd/$jpgsolid=jpg+zstd/$void=srep64+zstd/$text=srep64+nz

Pre-Strong = precomp+msc+srep64+delta+exe2+xz/$jpgsolid=jpg+zstd/$void=srep64+zstd/$text=nz

Pre-Ultra = reflate+msc+srep64+delta+exe2+xz/$jpgsolid=jpg+zstd/$void=srep64+zstd/$text=srep64+nz

Pre-Devastate = precomp+msc+srep64+delta+exe2+nzmax/$jpgsolid=jpg+zstd/$void=srep64+zstd/$text=srep64+nz

The masks will be inserted in pre-configuration of the new "M_S_Kompressor"

panker1992 17-01-2016 03:38

good evening everybody :)

@Amsal i repack in arch linux most of the time

here are the steps that need to do
1) create a folder with the name "pack"
2) then change this line
arc a -lc1024 -ld1024 -ep1 -ed -r -hp -pYourpassword -w.\ Data\data1.bin -moptimal "pack\*"

also there is no BINK repacking, bink.bat and bk2.bat take video and re-encode it to minimize it's size, you can read the "readme.log" i have put inside all details are there

@Razaman the strongest compression you can find is inside my tools named "devastate"

but there are 2 problems to fix with this method: doesnt show progress on Inno setup
and it leaves a temporary folder then it finishes its compression i may change this with zpaq.exe i am testing how this will go and update my tools in a few days :D


@Felice i made some corrections that fix many bugs

1) removed "pre-" as it may cause arc not being able to find it,

Fast = msc+srep64+delta+exe2+xz/$jpgsolid=jpg+zstd/$void=srep64+zstd/$text=nz//$precomp=msc+srep64+delta+zstd
Notes-Fast: This will only heavily compress compresible data, without low api precompression, This is entirely based on speed

Standard = msc+srep64+delta+exe2+lzt/$jpgsolid=jpg+zstd/$void=srep64+zstd/$text=nz/$precomp=precomp+msc+srep64+delta+lzt
Notes-Standard: Only corrected text format, left lzt untouched :) i like lzt too


Optimal = msc+srep64+delta+exe2+xz/$jpgsolid=jpg+zstd/$void=srep64+zstd/$text=srep64+nz/$precomp=precomp+msc+srep64+delta+lz77
Notes-Optimal : renamed this as now optimal is masked precomp, will become even better with up coming precomp0.45

{removed max entirely, you dont really need max strong ultra and devastate people will get comfused with so many ultimate ultra ...etc etc } :P

Strong = precomp+msc+srep64+delta+exe2+xz/$jpgsolid=jpg+zstd/$void=srep64+zstd/$text=nz
Notes-Strong: left as it is really innovate precomp will run on all data regardless

Ultra = reflate+msc+srep64+delta+exe2+xz/$jpgsolid=jpg+zstd/$void=srep64+zstd/$text=nz
Notes-Ultra: really like this one left as it is ....does inno + reflate really work for you ??

Devastate = precomp+msc+srep64+delta+exe2+nzmax/$jpgsolid=jpg+zstd/$void=srep64+zstd/$text=nz
Notes-Devastate : i am working on this method it does have some issues like it leaves a temp folder .... really pain ... need to see how i can fix this

felice2011 17-01-2016 04:03

Code:

1) removed "pre-" as it may cause arc not being able to find it
The "pre-" is just a Title, was for pre-configuration, so that the user has a configuration ready, and not trying to break with with multiple algorithms, does not affect the command line arc.;)
Code:

Notes-Ultra: really like this one left as it is ....does inno + reflate really work for you ??
Reflate is an excellent data compressor, use not only for games but also for other types of data compression, do not use, and does not exist, only "inno" as decompressor and installer, but other types of custom installers, so I think that the support must stay:)

I test various masks that you posted, thank you.:D

Amsal 17-01-2016 05:06

Panker1992, whenever I install repack, then the installation completes fine, but then when I go to the game directory, then there are extra temp folder like freearc1656238312.tmp, so can you tell me how to delete them, or tell me the fix of that, I use -smartlzt for repacking

panker1992 17-01-2016 05:07

TEST SAMPLE 1 = 100mb of random Photos

Precomp 0.45 x86 optimized build
Install Started - 14:44:37.31
Install Finished - 14:45:42.12
Total = 65 seconds
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Precomp 0.45 x64 optimized build
Install Started - 14:46:38.97
Install Finished - 14:47:38.42
Total = 60 seconds

Approx speed comparison x64 was 9,2% faster :)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Precomp 0.44 original Build
Install Started - 14:50:05.75
Install Finished - 14:51:09.70
Total = 64 seconds

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
TEST SAMPLE 1 = 115mb of 7z files
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Precomp 0.44 original Build
Install Started - 14:54:18.04
Install Finished - 14:55:01.72
Total = 43 seconds
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Precomp 0.45 x64 optimized build
Install Started - 14:55:59.44
Install Finished - 14:56:42.13
Total = 43 seconds
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Precomp 0.45 x86 optimized build
Install Started - 14:59:31.22
Install Finished - 15:00:15.10
Total = 44 seconds

Seems we have a winer :D

Method used was same on those 2 test samples, -mprecomp+srep+xz

i made this for you Felice :D please read this carefully and let me know what you think

i had some time today so i tested the precomp 0.45

panker1992 17-01-2016 05:08

@Amsal

the folder is there because you need to change

this

$text=nz

if you this one

$text=srep64+rep:100+nz it leaves a folder of trash :(

Amsal 17-01-2016 05:13

Quote:

Originally Posted by panker1992 (Post 446188)
@Amsal

the folder is there because you need to change

this

$text=nz

if you this one

$text=srep64+rep:100+nz it leaves a folder of trash :(

Sorry bro, i use only nz, and bro, listen me clearly, when I install game with blacbox v2, then it install install fine, but when I go to the install directory then I see tmp folder like this free23423532.tmp folder. So please tell me the fix of this

panker1992 17-01-2016 05:17

you can replace nz with similar very nice method

replace
$text=dict:p:128m+lzp:64m:105:d1m:s32:h22+ppmd:12: 192m

and free329049309 will vanish :D

Amsal 17-01-2016 05:18

Quote:

Originally Posted by panker1992 (Post 446190)
you can replace nz with similar very nice method

replace
$text=dict:p:128m+lzp:64m:105:d1m:s32:h22+ppmd:12: 192m

and free329049309 will vanish :D

So, i have to repack again

felice2011 17-01-2016 05:22

Great speed tests between "044" and "045" optimized, but the compression ratio remains the same?

Amsal 17-01-2016 05:22

Quote:

Originally Posted by panker1992 (Post 446184)
good evening everybody :)

@Amsal i repack in arch linux most of the time

here are the steps that need to do
1) create a folder with the name "pack"
2) then change this line
arc a -lc1024 -ld1024 -ep1 -ed -r -hp -pYourpassword -w.\ Data\data1.bin -moptimal "pack\*"

Bro, I mean that, can you give me tutorial for using bik and bk2 on ubuntu desktop? Because in my ubuntu, there is no D:/ drive. So, i want to use bink, on the desktop!

panker1992 17-01-2016 05:23

what game is it ?

i will give you optimals :D and yes repack again, since nz leaves trash behind i may have to switch to something more bugless

nanozip is no longer supported, its source is closed and i can do nothing about it

i think that zpaq may be on the menu again :)

panker1992 17-01-2016 05:41

Quote:

Originally Posted by felice2011 (Post 446192)
Great speed tests between "044" and "045" optimized, but the compression ratio remains the same?

Yes. Ratio is the same

That is why I didn't post a ratio test yet

But all in good time :)

Amsal 17-01-2016 05:53

Quote:

Originally Posted by panker1992 (Post 446194)
what game is it ?

i will give you optimals :D and yes repack again, since nz leaves trash behind i may have to switch to something more bugless

nanozip is no longer supported, its source is closed and i can do nothing about it

i think that zpaq may be on the menu again :)

It is Mortal Kombat X with update 20, can you tell me the optimals, and can you tell me some tips/tricks to detect the optimals for the specific games?

panker1992 17-01-2016 06:11

i havent tested mortal kombat myself but , try smartmax

and change the text format from nz to the one i provided in my last post

it will perform best, at least this is what i would do if i were you :)

Amsal 17-01-2016 07:03

Quote:

Originally Posted by panker1992 (Post 446197)
i havent tested mortal kombat myself but , try smartmax

and change the text format from nz to the one i provided in my last post

it will perform best, at least this is what i would do if i were you :)

Okay, I will try that, and bro, almost 9 out 10 times, the msc stuck on the different types of games repacking, so can you tell me the optimal setting for the msc!

panker1992 17-01-2016 07:07

from what i have read msc isnt being maintained anymore

and also i read that it has problems on a 2010 un-maintained bmf format

if you see fit or if you dont care about it disable it :)

Amsal 17-01-2016 08:20

Quote:

Originally Posted by panker1992 (Post 446197)
i havent tested mortal kombat myself but , try smartmax

and change the text format from nz to the one i provided in my last post

it will perform best, at least this is what i would do if i were you :)

Thanks bro, it is fixed now! now no freearc32543534.tmp and it's folder are created after the installation! And bro, there is one more thing I want to tell you, about 13gb are movies in Mortal Kombat X and they all are of format .bk2, but when I try to encode from your Bink technique, then it open and after clicking enter, it close in 3~4 seconds and there is no size difference also. I am running on Ubuntu 14.04 LTS 64bit with 6gb ram and it has only 1 drive partition(there is no C: drive)

1234567890123 17-01-2016 10:59

I'm waiting masked comp. v3 impatiently

panker1992 17-01-2016 14:05

@Amsal my bink Re-encoding isnt made for linux ....

i am arch linux user myself :D i can share my work with you , because i compile programs for linux environment too, i will make a bink re-encoding for linux too soon


for example i have zpaq, lz4, lz5, zstd precomp 0.45 on linux natively
but at the moment linux will have to wait because i work on new project :)


now as for masked compression v3

there is one thing that bothers me, unless Razor decides to help we will have a huge problem

i have contacted Christian the author of precomp for tak, Wav sound lossless codec
to be implemented on precomp

the problem is this : there is no stdout support on precomp .... and neither a working cls-precomp.dll

this causes a real problem, decompression will take twice the amount of time to decomp

and progress bar on inno will never work .....

ramazan19833 17-01-2016 14:39

Hello,Help
 
Originally Posted by ramazan19833
I send you the file to make it compatible compression ultimate masked getirer do you know that ? and ,also Data1.bin,Data2.bin,Data3.Data4 thousand.bin,Data5.bin,Data6.only data1.bin files.bin and loads of others to see please don't know, I would ask your help

http://s8.dosya.tc/server/vww0tv/Ultimate_Game.rar.html
i will try to make this compatible, give me a day or 2 and i will send you back !!!

panker1992 17-01-2016 15:07

Razaman ....
imagine 500 people sending files to me to optimize for them ....
i provided a tool and 2 Inno setups to support it along with a "Decomp Tool"
i cannot forever optimize all source codes that are sent to me,

please give me time to respond, i work on several projects , have a job , and i optimize and troubleshoot for everyone here

I have posted before that I will not ever again optimize someone else's code and i mean it

I can see that you have everything mixed up again :(

Amsal 17-01-2016 17:43

Quote:

Originally Posted by panker1992 (Post 446209)
@Amsal my bink Re-encoding isnt made for linux ....

i am arch linux user myself :D i can share my work with you , because i compile programs for linux environment too, i will make a bink re-encoding for linux too soon


for example i have zpaq, lz4, lz5, zstd precomp 0.45 on linux natively
but at the moment linux will have to wait because i work on new project :)


now as for masked compression v3

there is one thing that bothers me, unless Razor decides to help we will have a huge problem

i have contacted Christian the author of precomp for tak, Wav sound lossless codec
to be implemented on precomp

the problem is this : there is no stdout support on precomp .... and neither a working cls-precomp.dll

this causes a real problem, decompression will take twice the amount of time to decomp

and progress bar on inno will never work .....

Okay, thanks for the reply

Amsal 17-01-2016 18:10

Quote:

Originally Posted by panker1992 (Post 446214)
Razaman ....
imagine 500 people sending files to me to optimize for them ....
i provided a tool and 2 Inno setups to support it along with a "Decomp Tool"
i cannot forever optimize all source codes that are sent to me,

please give me time to respond, i work on several projects , have a job , and i optimize and troubleshoot for everyone here

I have posted before that I will not ever again optimize someone else's code and i mean it

I can see that you have everything mixed up again :(

Panker1992, if you are tired, then you can forward the private messages to me, I will do that, even I done that with my script also, and now it support! So if you need help of optimizing inno then you can contact me!

panker1992 18-01-2016 02:57

Greet Masked Compression v 2.1

many bugs fixes and new updates and optimizations
pacckjpg.exe, zstd

also i rewrote the front page, people should read that first before posting

@Amsal

personally i try to help everyone, but i cannot optimize every inno that i receive
i simply dont have soooo much time do that
can you help your friend razaman with his work ?? he needs to optimize the script

Amsal 18-01-2016 03:26

Quote:

Originally Posted by panker1992 (Post 446223)
Greet Masked Compression v 2.1

many bugs fixes and new updates and optimizations
pacckjpg.exe, zstd

also i rewrote the front page, people should read that first before posting

@Amsal

personally i try to help everyone, but i cannot optimize every inno that i receive
i simply dont have soooo much time do that
can you help your friend razaman with his work ?? he needs to optimize the script

Bro, can you upload new 6G,8G,12G,16G ram optimization, because I seen that The arc.ini is a lot different from the ram optimization. So upload new one!

panker1992 18-01-2016 03:30

aaa yes :D i forgot to change that

give me 2 minutes :)


its ok now , ram optimals are working again

Amsal 18-01-2016 03:42

Now bro, can you tell me which method is best for repacking

panker1992 18-01-2016 04:16

strong or ultra :D

refl8 and devastate are both BETA
do not attempt to use for generic usage just yet :)

i have 2 methods for fast
optimal and 2 ultra ones

and 2 BETA refl8 and devastate are not for public yet

Amsal 18-01-2016 04:41

And bro, when you have time, tell me please which is the best compression algo for the Mortal Kombat X with update 20

Razor12911 18-01-2016 06:19

Quote:

Originally Posted by panker1992 (Post 446209)
@Amsal my bink Re-encoding isnt made for linux ....

i am arch linux user myself :D i can share my work with you , because i compile programs for linux environment too, i will make a bink re-encoding for linux too soon


for example i have zpaq, lz4, lz5, zstd precomp 0.45 on linux natively
but at the moment linux will have to wait because i work on new project :)


now as for masked compression v3

there is one thing that bothers me, unless Razor decides to help we will have a huge problem

i have contacted Christian the author of precomp for tak, Wav sound lossless codec
to be implemented on precomp

the problem is this : there is no stdout support on precomp .... and neither a working cls-precomp.dll

this causes a real problem, decompression will take twice the amount of time to decomp

and progress bar on inno will never work .....

What is it bro?

panker1992 18-01-2016 06:35

Tutorial cls-.dll c++ knowledge needed
 
1 Attachment(s)
Razor bro,

here you are this is a cls-test.dll

its included inside freearc, for cls.dll tutorial

i am sending you also precomp 0.45 code along with a exe file
can you help with cls-precomp.dll

cls-precomp.dll can be in delphi ~~~~~ because you speak delphi

romulus_ut3 18-01-2016 06:38

No love for 4G ram users?

panker1992 18-01-2016 06:43

Quote:

Originally Posted by romulus_ut3 (Post 446238)
No love for 4G ram users?


your pc will die :P

but you can change the parameters in arc.ini

nanozip there a memory limiter where is says memory instead of 2g replace with 1g

xz and lz77

use memory = d100 for lz77
and memory = d50 for xz

and prey it works :P

lzt will never work so dont even bother :P

Razor12911 18-01-2016 07:43

Panker, you do realize that cls isn't the only way to make precomp show progress during installation. If cls is a problem or is a difficult approach , why not make use handle streams for decompression?

The arc.ini will look like this
precomp -r -o- - <stdin> <stdout>.

panker1992 18-01-2016 08:38

it's the first time i hear handle streams for decompression :/

precomp -r -o- - <stdin> <stdout>

this command never worked, precomp doesnt support it

took an error

Razor12911 19-01-2016 06:38

You have to edit the source and make it possible. That cls thing is very complicated even for me,
I don't know about CPP but on delphi, there is a type of stream that works with handles.
You just detect a certain type of handle, which is simple, just detect stdin and stdout.
You don't have to worry of how to exchangd data between freearc and precomp, with handles all you have to do is read using the stdin handle stream thenfrom there you let precomp handle the data then you send the output back to freearc, simple as that, progress will move automatically. The only tip I can give you is to never write anything on cmd window because that is also regarded as stdout data.

Here is a simple example but in delphi

var
hs1, hs2: THandleStream;

begin
hs1:= Thandlestream.create(GetStdHandle(STD_INPUT_HANDLE )) ;
hs2:= THandleStream.create(GetStdHandle(STD_OUTPUT_HANDL E));
hs1.read.... or use copyfrom to get data directly from freearc <stdin>
//precomp should process the data here...
hs2.write or hs2.copyfrom to send the processed/unpacked data back to freearc //this moves the progress bar.


If I knew the header structure of precomp, I could make precomp that uses stdin and stdout for unpacking to show progress.

panker1992 19-01-2016 06:45

this post will get me somewhere :) i am at the university at the moment giving advance economic exams :P

damn i will get to this after i finish my studies here because it seems very interesting

thanks razor

at the moment i want to do this and edit precomp source to make it use stdin name for tempfiles so it can be used parallel

for example

for *.zip || ppx2 -P %Number_of_Processors% precomp intense0 -cn -zl98 -d1 file.pcf

will be able to not conflict between multiusage :)


All times are GMT -7. The time now is 01:35.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2026, vBulletin Solutions Inc.
FileForums @ https://fileforums.com