lzma2 on freearc as external

[panker1992]

Finally, after working on this one for about 2 months, I managed to use lzma2 as external command for freearc.
It uses cpu to the limit so expect a full 4 times faster compression, please consider to share any settings you might find that does improve this method enjoy !!!

Properly Intro zstd

~~~~~> Check Pic Below <~~~~~
Metal Gear Solid 5
Method: srep64+zstd 19 Gigs total file
Decompressed in less that 3 minutes
Decomp Speed 275 mb/sec

well for those who have used my method you might have seen a compressor named zstd
since i havent seen anyone using it before, i will right down here what it does and how to use
like my lzma2 method it supports stdin and stdout, so decomp starts within seconds from data*.bin.
zstd is a poor compressor and its usage isnt for compressing but storing data that cannot be further compressed lossless
its capable of delivering 490mb/sec per thread... meaning a full 1.8 giga/sec on an 4 core cpu; assuming you dont have a bottlenech on
i/o(aka HDD/SSD).
Just for the record srep64 can deliver a full 1giga/sec, thus making srep64+zstd method a really bad @ss method to use.
hope you like it and use it.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Properly Intro lzham
Lzham just like lzma2 is a heavy compressor, the reason i have chose to upload this tool is because i have seen that uses 55% less ram than lzma2.
Thus making it a reasonable tool for those that utilize 8 gigs of ram or less
The ratio is 0.20% bigger nothing to worry there...Decomp speed is a little bit faster than lzma2 ..or that least for my pc !!!!
and of course ... i kept this for last !! :P it utilizes andler32 (something similar to checksum for those familiar with this )
ENJOY

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Update 1:
Added inno to make it work all in one go !!!
This is from Kurutcu v 1.8 blackbox all credits for this goes to him.
well you need to modify it to support your given password and the number of bin files you created !!! more of this later !!!

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

[ Compressing Tutorial ]
Well as i got too many requests for a tutorial i decided even now to share one with you!!
how to use this:
1)) select a game you want to compress then
2) decide how to make ex: put all videos and data that generally doesnt compress well in 1 folder and all other data that will compress heavily into folder 2
A) method selection examples max compression: -mprecomp+msc+srep64+xz or fast decompression for data that doesnt compress: -mmsc+srep64+zstd
B) put there the path to the folder you want to compress
C) the number of bin files data1.bin, data2.bin ......data4.bin etc..
D) you can name data1.bin or data1.arc or data1.cab choose whatever you want. i prefer bin files :P
E: you can choose to have password protected data or not its your decision to make i use password to my bins
3) after you have done that you can you save the bat file and run it !!!

[ Decompression Tutorial ]
A1) The name of your data files example: if you made data1.bin and data2.bin then uncomment 2 lines of code for data1 and data2
B1) If you have used password on compression it must be reverse-used to decompress them as well, put it there i have shown on pic !!
C1) This is the progressbar detector if you have 1 file then 100% goes to one file if you have 2 files 50 for one and another 50 for the other.
{REMEMBER} you can have 7 bin files or whatever you want, make sure when you sum this up you need to have 100 combined

A2) The name of the exe file in your game folder, this is used to make a desktop shortcut so be careful its case sensitive
B2) The amount of space that the game reserves, in this case lords of the fallen takes 17 gigs* so the amount that you point there is the amount of megabytes
plus 1000 more to point correctly!!!
C2) The name of the game and the name of the folder that this inno will create...

PS: All tools and Inno Setup have been updated!!! also made some corrections.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Properly Intro Bink and Bk2 Encoding

Just as i promised that i will keep updating this thread.
I will show you how to properly make some lossy compression in order to achieve even greater Ratios.
i wrote down that Script myself in order to take bik and bk files as input and properly reduce them without you making a fuss..

This script is optimised to Show how many inputs you inserted and show progress while processing them thus its title will show at which
bink or bk file it is, also i made that it deletes the input file after compression in order to save time.

~~~~~~ Here is How to use ~~~~~~

Place the bat files and bink.exe inside a folder that you make along with the files.
then run bink_enc or bk2_enc depending which type you are Rebuilding. make sure you back up files just in case!!!

Enjoy !!!

THIS TOPIC IS MOVED PERMANENTLY ON MY NEW MASKED COMPRESSION THREAD

[kenzo34]

thank you very good job. good result and the speed is faster. what should is the bat file to decompress? I try with this command:. arc.exe x -w \ -dpunpacked \ "data1.bin" but I found an error

[panker1992]

Originally Posted by kenzo34

thank you very good job. good result and the speed is faster. what should is the bat file to decompress? I try with this command:. arc.exe x -w \ -dpunpacked \ "data1.bin" but I found an error

you will need unarc.exe or arc.exe to do the unpacking
therefore these are the commands
unarc x -pyourpassword -o+ data1.bin (optional command -dpFOLDER) to extract into a specific folder

arc x data1.bin -y -pYourpassword


or you can modify kurutcus 1.8 to extract from there i will be uploading ways to extract if you ask for them both inno and ripping


PS1: the error is either password error or you need to rename 7z.exe ~> dec.exe using 7z.dll as well ... or copy paste and replace with the update 1 i provided

[kenzo34]

thank's you panker1992 I like your compression method.
I will try with your board
Thank you

[panker1992]

this is a result of a very hard work lzma2 is hard to work on command line. tho i have to admit that part of this inspiration came from razor's work i wanted to use this with cls-reflater.dll !! lets see if people decide to use this method on mine

[Carldric Clement]

Originally Posted by panker1992

Finally, after working on this one for about 2 months, I managed to use lzma2 as external command for freearc. It uses cpu to the limit so expect a full 4 times faster compression, please consider to share any settings you might find that does improve this method enjoy !!!


update1:
[External compressor:xz]
;lzma2:mfbt4:d27:fb128:mc1000 ;-m1=lzma2:fb=256
header = 0
packcmd = 7z a -txz -an -mcrc=0 -mx9 -m1=lzma2:d27:fb=273:mf=bt4:mc=1000 -mmt=on -si -so <stdin> <stdout>
unpackcmd = 7z x -txz -an -y -si -so <stdin> <stdout>

Very well bro.
I think i got idea to test with isdone installer.

[kenzo34]

it would be great if you could combine your method with reflate. Bravo for your work sincerely

[panker1992]

Originally Posted by Carldric Clement

Very well bro.
I think i got idea to test with isdone installer.

my method supports stdin and stdout so it works with isdone i have tested it i will be uploading 2 ways to use it effectively so no worries

[ShadowEagle]

Originally Posted by panker1992

my method supports stdin and stdout so it works with isdone i have tested it i will be uploading 2 ways to use it effectively so no worries

Excited to combine / integrate it into UltraARC

[panker1992]

use my method with the inno i provided above so you will see how it performs on stdin and stdout compared to just lzma :P soon i will be providing the other method. This inno supports precomp+msc+srep64+xz all in one go stdin to stdout

[rinaldo]

interesting

[panker1992]

I want to see results on those method, can anyone post ?? also I will be updating as much as i can so keep an eye on this

[gozarck]

Hello panker1992 thnks for share your work.
i just add lc4 and lp0 you gain very little bytes in compression.

[External compressor:7z]
packcmd = 7z.exe a -t7z -mcrc=0 -m1=lzma2:fb=273:mf=bt4:mc=1000:lc=4:lp0 -mx9 $$arcpackedfile$$.7z $$arcdatafile$$.dll
unpackcmd = 7z.exe x -y $$arcpackedfile$$.7z $$arcdatafile$$.dll
packedfile = $$arcpackedfile$$.7z
datafile = $$arcdatafile$$.dll

if you see the code i use -t7z instead -txz because lzma2 is default compression in 7zip since 2012.
Also i inject your 7z.dll into 7z.exe check the attachment, just drag and drop any file in the bat file to test.

[panker1992]

thanks gozarck this method works pretty nice !!! feel free to uploads any settings you have discovered that may improve this ...

PS1: but sorry to disappoint you :P to trick 7z to stdin and stdout xz is the only option i tried for many weeks to make it 7z stdin and stdout and i always got an error in decomp. couldnt make it work at all .. so xz=lzma2 best option available :P plus it supports precomp+msc+srep64 all in one go :P

[ShadowEagle]

How to use it in InnoSetup? I use CIU2 - how to do?