FileForums

FileForums (https://fileforums.com/index.php)
-   Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)
-   -   Simple splitting and compress by DLG (https://fileforums.com/showthread.php?t=97303)

panker1992 05-12-2015 04:21

Quote:

Originally Posted by RamiroCruzo (Post 445130)
Lets compare ourselves which is better for our needs....:)

Source:Encode.ru

| Compressor name | Compression| Decompress.| Compr. size | Ratio |
| --------------- | -----------| -----------| ----------- | ----- |
| memcpy | 8533 MB/s | 8533 MB/s | 104857600 |100.00 |
| lz4 r131 | 480 MB/s | 2275 MB/s | 64872315 | 61.87 |
| lz4hc r131 -1 | 82 MB/s | 1896 MB/s | 59448496 | 56.69 |
| lz4hc r131 -3 | 54 MB/s | 1932 MB/s | 56343753 | 53.73 |
| lz4hc r131 -5 | 41 MB/s | 1969 MB/s | 55271312 | 52.71 |
| lz4hc r131 -7 | 31 MB/s | 1969 MB/s | 54889301 | 52.35 |
| lz4hc r131 -9 | 24 MB/s | 1969 MB/s | 54773517 | 52.24 |
| lz4hc r131 -11 | 20 MB/s | 1969 MB/s | 54751363 | 52.21 |
| lz4hc r131 -13 | 17 MB/s | 1969 MB/s | 54744790 | 52.21 |
| lz4hc r131 -15 | 14 MB/s | 2007 MB/s | 54741827 | 52.21 |
| lz5 r131 | 195 MB/s | 939 MB/s | 55884927 | 53.30 |
| lz5hc r131 -1 | 32 MB/s | 742 MB/s | 52927122 | 50.48 |
| lz5hc r131 -3 | 20 MB/s | 716 MB/s | 50970192 | 48.61 |
| lz5hc r131 -5 | 10 MB/s | 701 MB/s | 49970285 | 47.66 |
| lz5hc r131 -7 | 5.54 MB/s | 682 MB/s | 49541511 | 47.25 |
| lz5hc r131 -9 | 2.69 MB/s | 673 MB/s | 49346894 | 47.06 |
| lz5hc r131 -11 | 1.36 MB/s | 664 MB/s | 49266526 | 46.98 |
| zstd v0.3 | 257 MB/s | 547 MB/s | 51231016 | 48.86 |
| zstd_HC v0.3 -1 | 257 MB/s | 553 MB/s | 51231016 | 48.86 |
| zstd_HC v0.3 -3 | 76 MB/s | 417 MB/s | 46774383 | 44.61 |
| zstd_HC v0.3 -5 | 40 MB/s | 476 MB/s | 45628362 | 43.51 |
| zstd_HC v0.3 -9 | 14 MB/s | 485 MB/s | 44840562 | 42.76 |
| zstd_HC v0.3 -13 | 9.34 MB/s | 469 MB/s | 43114895 | 41.12 |
| zstd_HC v0.3 -17 | 6.02 MB/s | 463 MB/s | 42989971 | 41.00 |
| zstd_HC v0.3 -21 | 3.35 MB/s | 461 MB/s | 42956964 | 40.97 |
| zstd_HC v0.3 -23 | 2.33 MB/s | 463 MB/s | 42934217 | 40.95 |

Impressive but this test utilize the HC format of the zstd program
currently i test zstd v4.2

i have eaten all my day to test, test and test this :P

download the lz4 installer for windows and rename zstd ~~> lz4.exe and
replace in programs/lz4 folder

use this and you will see that decomp speed is all but fake i/o cant handle
2.2 gigs/sec because of limitation unless you have some x6 SSD array :P

sayang123 05-12-2015 05:49

Is there any installer script which support the unpack command from this app?

sorry for my bad english

rinaldo 05-12-2015 08:23

also applies in the ComboBox style. the success is in knowing how to match colors
(Too bright and too colorful are crap)
lit are used to such mail alert .. design still do not like you know.

felice2011 05-12-2015 09:48

also applies in the ComboBox style. the success is in knowing how to match colors
(Too bright and too colorful are crap)
lit are used to such mail alert .. design still do not like you know.[/QUOTE]

Depending on your point of view there are those who see the world in color and some in black and white... I see the world in color, anyway I see what you need.;)

felice2011 05-12-2015 12:17

Quote:

Originally Posted by sayang123 (Post 445136)
is there any installer script which support the unpack command from this app?

Sorry for my bad english

In attachment it is an example of installing proprietary 'not InnoSetup' with a data file compressed using the reflate method 'reflate+srep+delta+lzma:a1:mfbt4:d158m:fb273:mc10 00000000:lc8'
For what concerns me and for my use decompression or installation or setup of a compressed data folder, with the excellent method reflate, works beautifully.

coming soon, new universal extractor in new thread

Lucas65 10-12-2015 14:26

Hello, thank you felice2011 for this application. It is fantastic!

In my windows 7 64 bit is good but in another my PC with XP SP3 32 bit does not work with this error: "Split_Kompress.exe is win32 application invalid".

There is solutions to this problem?

Thank you in advance, sorry for my english and congratulations again for your work.

panker1992 10-12-2015 17:31

@Lucas this will never work for 32 bit windows let alone xp ...

the programs attached to this compressor,
well at least xz and Zstd i compiled myself and are only 64 bit supported

at this point {Note} that at least 8 gigs of ram is the absolute minimum for solid compression !!!

felice2011 11-12-2015 00:35

Quote:

Originally Posted by Lucas65 (Post 445369)
Hello, thank you felice2011 for this application. It is fantastic!

In my windows 7 64 bit is good but in another my PC with XP SP3 32 bit does not work with this error: "Split_Kompress.exe is win32 application invalid".

There is solutions to this problem?

Thank you in advance, sorry for my english and congratulations again for your work.

The program is optimized for 64-bit systems with minimum 4GB of RAM, XP is now obsolete, and makes the most of 3.5GB of RAM as it is a 32bit system.
Even if it would work 32-bit its potential for compression would be reduced by 60%.

Lucas65 11-12-2015 04:44

Ahn ok. So if I wanted to use this code: -mprecomp+reflate+srep+delta+lzma:a1:mfbt4:d158m:fb 273:mc100000:lc8 in .bat file on my XP the volum of compression would be reduced by 60% however, or not?

felice2011 11-12-2015 05:40

Quote:

Originally Posted by Lucas65 (Post 445382)
Ahn ok. So if I wanted to use this code: -mprecomp+reflate+srep+delta+lzma:a1:mfbt4:d158m:fb 273:mc100000:lc8 in .bat file on my XP the volum of compression would be reduced by 60% however, or not?

No one questions that this method does not work with xp 32 bit, but with the modified method 64-bit in this way "-msrep64+delta+lzma64:a1:mfbt4:d158m:fb273:mc1000:l c8" you may take advantage of more RAM and more cores of a good processor with a system 64 bit, significantly reducing the waiting time during a large compression.
I can not and not convenient modify the source code of the utility to fit on a 32-bit system, moreover dated, we try to create advanced software able to improve and lighten the work of compression to you users, making it more easy and intuitive, especially with system compatibility in view of present and futur...not for the past...hi:)

Lucas65 11-12-2015 09:24

I understand and I thank you for giving me clear the concept. Recently I have become aware of these compression methods so it's all new to me.
I would have another problem that after using this great program I was presented. Using the method "reflate", at the end of compression gives me this error:

http://i.imgur.com/c0EVIWm.png

And after the error did not save the compressed file. The library 7z.dll I never touched. The .tmp and .raw file rest in the _tmp folder.
What may be due? Sorry again for your trouble.

felice2011 11-12-2015 10:21

7z dll error due to a block loading...
Manually delete the temporary files in the folder '_tmp'.
Manually delete also the folder 'Resources' and recopy the one contained in the 7z file I posted, so be sure that no files were modified.
Disable your antivirus when using, some files it can be recognized as false positive.
Please try the same method with a data folder smallest and see if it works.

Lucas65 11-12-2015 11:10

Thanks for the help felice but unfortunately still no go.
I then downloaded all the files from .7z you posted and tried to shut down anti-virus on a file of 200 MB. First I tried on about 4 gb. Task manager is clear.
However I get the exact same error.

felice2011 11-12-2015 11:55

I tested with the version I posted, using the same settings on a folder of 200mb I have not received any error.
P.S: Tested on w7 x64 bit.

http://i65.tinypic.com/2guzlgw.png

Try with this copy and replace.

Attachment 13060

Lucas65 12-12-2015 04:34

Thanks for the help but I tried and tried and unfortunately continues to give me that error.
I tried changing file types, destinations and extensions of the output but keeps the damn error 7z.dll.
If you have any other advice on which to rest thank you, while I keep trying, hoping to find a solution.

felice2011 12-12-2015 05:48

Quote:

Originally Posted by Lucas65 (Post 445418)
Thanks for the help but I tried and tried and unfortunately continues to give me that error.
I tried changing file types, destinations and extensions of the output but keeps the damn error 7z.dll.
If you have any other advice on which to rest thank you, while I keep trying, hoping to find a solution.

Excuse satisfy my curiosity, try directly with reflate edited by Razor, http://www.fileforums.com/attachment...9&d=1442362824 or at the most uses only this command line "-mreflate+srep+delta+lzma" with my utility.;)

Lucas65 12-12-2015 09:26

Fine, I tried both options with your utility: with that string, I get the same error while the reflate razor everything works well until the end. What does it mean?
Thanks again for the help.

Lucas65 13-12-2015 04:35

Great Felice! It worked last! I did some tests and now work all the various compression options. In the end I saw that it was enough to take the file 7z.dll razor replacing it in the "Resource" folder to get it going. Thank you so much for the help and patience!
I would have another curiosity as long as there are: but various versions of Precomp have substantial differences, or what?

felice2011 13-12-2015 08:44

Great, the important thing is to solve....I personally use the precomp042 and 043, substantial differences between the two, the precomp042 has a greater compatibility of compression with data files and with the various application for decompression,on the contrary, with 043 you could get errors of incompatibility on some files, during the compression, but you get more compression as unlike to the 042.
the other information you can read here http://schnaader.info/precomp_changes.php

Lucas65 13-12-2015 10:39

Thanks a lot for everything felice. Now I'm trying to figure out how to unzip these methods, especially with reflate. I am using the script of Rinaldo and I get an error decompression. If I continue to have these problems I can ask here, in the thread of Rinaldo or open a new topic?

Meanwhile, thanks for everything, you were very kind. Hello!

felice2011 13-12-2015 10:58

Quote:

Originally Posted by Lucas65 (Post 445451)
Thanks a lot for everything felice. Now I'm trying to figure out how to unzip these methods, especially with reflate. I am using the script of Rinaldo and I get an error decompression. If I continue to have these problems I can ask here, in the thread of Rinaldo or open a new topic?

Meanwhile, thanks for everything, you were very kind. Hello!

Here we discuss problems related to my utility, open a new thread, or better directly in the thread of Rinaldo, he's the expert of script with inno setup will give you valid information.;)

panker1992 17-12-2015 22:29

Zstd Compressor for windows & zstd v 4.4
 
i decided to give away my Zstd compression for windows
because i think it can go head to head with lzma :eek:
yes it can beat lzma normal and high modes but not ultra

special code it is pipelined through srep for maximum performance test the glory ;)


also i updated zstd to 4.4 the latest its getting better and better


Notes : zstd compressor for windows is only 64 bit, but i have posted 32 bit support on encode. ru forums i also gave away inno setup for it there too


Enjoy :)

felice2011 19-12-2015 10:37

Updated Now " Split Kompress v1.3d "

;)

panker1992 19-12-2015 10:47

packcmd = lz77 e -si -so <stdin> <stdout> -a1 -d30 -fb273 -mc10000 -lc8 -lp4 -mfbt4

this command is the top you can get that means lz77 aka(lzma renamed) will use best dictionary available


in short terms that means you must have at least 20 gigs of ram ..... yeah lzma can use bigger dictionary than 1024 mb ...
if i remember correct 1.5 gigs or now after the update around 4 which means 15 gigs and 40 gigs respectively .....


depending on your machine use -d15 instead of 30 or -d20 test a little

enjoy :)

EDIT1: well decided to give you many options since i am to the bone in lzma and lzma2
-mfbt4 command on lzma is power-very-hungry ram user and nasty one to use especially if your ram is limited
it will use 10 gigs of ram per 1gig of dictionary .... in short you can switch to -mfhc4 that uses 1.5 gigs ram per 1 gig dictionary
personally i havent used -mfhc4 at all so if you pull some tests please let me know

EDIT2: sorry forgot to tell above :P please dont yell for many edits i only have one mind
with this out of the way, lz77 replaces freearc-lzma in all aspects, this is why i posted it originally
freearc-lzma is outdated to the neck so why use it ?? arc.exe has lzma inside it already and you can just call it without having lzma in ini thi is why i renamed to lz77 to reverse conflict
the lzma inside arc.exe is also outdated to fazip.exe respectively :P

EDIT3: ok now because i want to make your live easy i will tell all my secrets like boss :P
fazip also hasnt been updated in while so its outdated to lz77
lz77 is the same code used in streams to produce lzma2 i use in xz command so all in all
if you wanna do it right delete lz77.exe and use

[External compressor:lz77]
header = 0
packcmd = xz a -txz -an -mcrc=0 -m1=lzma2:d1024m:fb=273:mf=bt4:mc=10000:lc=4:lp=0 -mmt=2 -mx9 -si -so <stdin> <stdout>
unpackcmd = dec x -txz -an -y -si -so <stdin> <stdout>

this line will produce a lzma variant exactly like lz77 and fazip and the lzma code inside arc.exe this is the most updated lzma code to this day
1gig dictionary usage 10 gigs ram is required

Final Edit :

Tests on your program and Corrections

Felice i had time today so i gave a go on your program which is very nice especially for new people in compression

i also give you my endorsement for considering advanced users in this section as well

i decided to make some corrections on this in order to help minimize its source

you dont need dll files as i injected them inside the applications exe now carry their dlls on their backs no need to have them in a file separetely

i removed some programs as well as they dont fit :P

i also updated srep 32 and 64 bit to the latest best version 3.93

Enjoy :)

felice2011 20-12-2015 05:46

@Panker the file "arc.ini" is updated when you start each application, so the changes that you made in the folder resource are lost and do not work at the next startup.
Moreover, with files that have modified in the folder 'resource' utility goes error with the methods that contain '+ lzma64', we must give the user the possibility of choosing any algorithm, and not the one that we like.:)
If you have any changes to the utility or any other updates, please send me the files, I will think about modify as best he can pleasure..;)

panker1992 20-12-2015 06:23

Felice i know what you mean .... let people have many choices is right i didnt remove the choices that you let people have but software updates fast even faster that we imagine

you can replace freearc-lzma with fazip

fazip excels in lzma coding by around 13% for 32bit and around 17% in 64 bit so :P

felice2011 20-12-2015 06:33

Quote:

Originally Posted by panker1992 (Post 445608)
Felice i know what you mean .... let people have many choices is right i didnt remove the choices that you let people have but software updates fast even faster that we imagine

you can replace freearc-lzma with fazip

fazip excels in lzma coding by around 13% for 32bit and around 17% in 64 bit so :P

This is great, I understand, and other users will have understand, but if the utility has several choices let these operate without errors, and that the user can choose, on the contrary I would not have entered.;)
Moreover if there would be the possibility to enter the option lz77: "7z a -txz -an -mcrc=0 -m1=lzma2:d512m:fb=273:mf=bt4:mc=10000:lc=4:lp=0 -mmt=2 -mx9" OR xz: "7z a -txz -an -mcrc=0 -m1=lzma2:d27:fb=273:mf=bt4:mc=1000:lc=4:lp=0 -mmt=on -mx9"
Directly in the textbox of the utility would be a great thing, so that the user can modify the choice in the utility.

panker1992 20-12-2015 06:38

Felice i am uploading my masked compression project i am making my last adjustments


everything i am trying to tell you was for this moment you dont have to compress using one method when you can use everything all at once

did you know you can use reflate precomp lz77 zstd xz and deflating method all on 1 large bin file ???


sorry forgot to tell you :P

please let people modify the dictionary

because its ram dependant so people with less ram will never make it with lzma2 which can use 4 times the ram that lzma1 uses

felice2011 22-12-2015 01:55

Further Update "Split Kompress Update_2" On The Front Page.

Sorry I try of perfecting.

:)

gozarck 22-12-2015 07:25

Very nice update felice. This is a very powerfull tool.

rinaldo 22-12-2015 13:54

1 Attachment(s)
I tried this program only SREP and was fine ..
it only works with SREP ... why?
when it crashes, always the same error .. and when I stop him from the button because it blocked ... from them is no longer moving.
Sin looked good your program :)

ChronoCross 22-12-2015 17:31

I really want to test, but i have an error "System.IO.FileNotFoundException
". the program crash quickly.
Win7x64

felice2011 23-12-2015 00:37

Quote:

Originally Posted by rinaldo (Post 445692)
I tried this program only SREP and was fine ..
it only works with SREP ... why?
when it crashes, always the same error .. and when I stop him from the button because it blocked ... from them is no longer moving.
Sin looked good your program :)

Quote:

Originally Posted by ChronoCross (Post 445694)
I really want to test, but i have an error "System.IO.FileNotFoundException
". the program crash quickly.
Win7x64

@rinaldo from what I see from the photo do not use it correctly, although I tested the utility in your way, it works equally..;)
Simply create a folder to save the compressed files, and not in a root directory of the system, where you have hundreds of other files, if you press' STOP 'while the program is running to read, for compressed files, risks of a crash' utility, seems obvious.:)
the utility is tested at 99% with the methods included in "Simple", attention that the algorithm contains "precomp" natively sometimes finds it difficult to compress some folders that contain files such as * .dll and others, it happened to me with a program called Picasa seems.

@ChronoCross checks that the root of the utility contains "MetroFramework.dll":)

rinaldo 23-12-2015 06:05

only pre-stressed errors amazing .. use 043 only, otherwise they will always incompatible, because 043 is all in one and not the others, so the test will be more accurate with only a test exe .. regarding other methods, you should absolutely use the bat or programs in symbiosis.

felice2011 28-12-2015 03:33

" Split Kompress Update_3 (only exe) "

' Fixed Reading Of Only Files Processed In The Temporary Control Window '

' New Self-Configuration Setting "records.ini" At Packet Completed '

" Enjoy "

ChronoCross 30-12-2015 21:12

Nice felice2011 this is a great tool.

1234567890123 02-01-2016 13:45

which script i can use for this program

felice2011 03-01-2016 00:40

Any installation script, example installers "RG Catalyst" By @Bilawal or "New Installer" By @Rinaldo, for Installations With And Without Password.

weltall 04-01-2016 16:54

O.O a new DLG compressor? I definitely have to try it! Thank you so much for all the work and effort you put to make these :D I am glad they are compatible with Rinaldo's installers :)

-Edit-
Excuse my fanboy moment; Woooooow this new tool is so cool! Interface looks great and configuring it is extremely easy, even for people that do not understand much about such things. Thank you so much! Happy New Year :)

felice2011 20-01-2016 03:57

" Update SK v1.4c "

' Added Precomp 0.4.4 Alpha '

' Fixed Writing In "records.ini" Created For Precomp From "0.38" To "0.44" '

' Added Compression Algorithm "NanoZip" In Advanced Mode '

' Added Methods In Simple Mode "precomp+msc+srep64+delta+nz" & "msc+srep64+exe2+nz" '

( Personally The Compression Algorithm "NZ" In Combination With Different Methods Is One Of The Better )

:)

( Copy And Replace On Original Path All The Folders And Files )


All times are GMT -7. The time now is 08:12.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2026, vBulletin Solutions Inc.
FileForums @ https://fileforums.com