FileForums

FileForums (https://fileforums.com/index.php)
-   Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)
-   -   Simple splitting and compress by DLG (https://fileforums.com/showthread.php?t=97303)

panker1992 04-12-2015 07:59

Quote:

Originally Posted by RamiroCruzo (Post 445110)
Amigos if you wanna so fast compression method, why don't you take a look at LZ4 which goes beyond 1GB/s without I/O overwrapping & multithreading.....


I have tested lz4 and lz5 long time ago, zstd is a step ahead of lz4 lz5 and zhuff at the moment its steady and it has a lot of support the one i posted is v3.6 and the one i am testing now is 4.2 and it has 2.2 gigs/sec around
( 593mb/sec/thread = 2.2 gigs/sec )

felice2011 04-12-2015 08:35

Quote:

Originally Posted by RamiroCruzo (Post 445110)
Amigos if you wanna so fast compression method, why don't you take a look at LZ4 which goes beyond 1GB/s without I/O overwrapping & multithreading.....

Quote:

Originally Posted by panker1992 (Post 445118)
I have tested lz4 and lz5 long time ago, zstd is a step ahead of lz4 lz5 and zhuff at the moment its steady and it has a lot of support the one i posted is v3.6 and the one i am testing now is 4.2 and it has 2.2 gigs/sec around
( 593mb/sec/thread = 2.2 gigs/sec )

Friends you do your choices, I'm happy to update my utility just as you wish, and it however you like.
Panker do not look at only the speed, but also the compression level of the various methods, we must try to find a balance in both, to satisfy everyone.

panker1992 04-12-2015 09:39

Quote:

Originally Posted by felice2011 (Post 445119)
Friends you do your choices, I'm happy to update my utility just as you wish, and it however you like.
Panker do not look at only the speed, but also the compression level of the various methods, we must try to find a balance in both, to satisfy everyone.

This is why i chose zstd ...because it features advanced API that can compress decent.

To be more specific zstd has at least 30% ~ 35% better compression ratio
tho everyone i see use my original -8 switch ....it can easily go as high as -20 but its ratio doesnt compare to lzma at all :P

All in all zstd gives 30% ~ 35% better compression AND it decompressed as fast as lz4

HINT : zstd and all tools i use have porpose nothing i choose is coincidence this all is part of my upcoming big project call "Masked Compression"

soon !!!

felice2011 04-12-2015 10:10

Great Bro ... at least we know that LZ4 and LZ5 is now out ;)

Razor12911 04-12-2015 12:22

Quote:

Originally Posted by ShadowEagle (Post 445111)
Ok, i do understand. You created a tool, posted it and don`t help to get it work. I see no tutorial. I`ll give 1 star for bad support, most bad support i`ve ever seen.

took me 4 minutes to figure out how this program works, from my perspective, this program is easier than ultra arc and it is reliable.

ShadowEagle 04-12-2015 14:32

Yes i`m stupid and bad guy :(

Razor12911 04-12-2015 16:58

Quote:

Originally Posted by ShadowEagle (Post 445126)
Yes i`m stupid and bad guy :(

not really, you just have to engage with the program even further and use logic, so far so good, you are showing interests with this program only thing left is to make attempts to make the program work using logic then you'll get there.

gozarck 04-12-2015 18:58

Yes felice your tool is very powerfull and easy to use.you need to make the UI more atractive for newbies. Do you was tried to integrate metroframework.dll (metroUI) in yours projects?.

RamiroCruzo 04-12-2015 22:14

Quote:

Originally Posted by felice2011 (Post 445119)
Friends you do your choices, I'm happy to update my utility just as you wish, and it however you like.
Panker do not look at only the speed, but also the compression level of the various methods, we must try to find a balance in both, to satisfy everyone.

Lets compare ourselves which is better for our needs....:)

Source:Encode.ru

| Compressor name | Compression| Decompress.| Compr. size | Ratio |
| --------------- | -----------| -----------| ----------- | ----- |
| memcpy | 8533 MB/s | 8533 MB/s | 104857600 |100.00 |
| lz4 r131 | 480 MB/s | 2275 MB/s | 64872315 | 61.87 |
| lz4hc r131 -1 | 82 MB/s | 1896 MB/s | 59448496 | 56.69 |
| lz4hc r131 -3 | 54 MB/s | 1932 MB/s | 56343753 | 53.73 |
| lz4hc r131 -5 | 41 MB/s | 1969 MB/s | 55271312 | 52.71 |
| lz4hc r131 -7 | 31 MB/s | 1969 MB/s | 54889301 | 52.35 |
| lz4hc r131 -9 | 24 MB/s | 1969 MB/s | 54773517 | 52.24 |
| lz4hc r131 -11 | 20 MB/s | 1969 MB/s | 54751363 | 52.21 |
| lz4hc r131 -13 | 17 MB/s | 1969 MB/s | 54744790 | 52.21 |
| lz4hc r131 -15 | 14 MB/s | 2007 MB/s | 54741827 | 52.21 |
| lz5 r131 | 195 MB/s | 939 MB/s | 55884927 | 53.30 |
| lz5hc r131 -1 | 32 MB/s | 742 MB/s | 52927122 | 50.48 |
| lz5hc r131 -3 | 20 MB/s | 716 MB/s | 50970192 | 48.61 |
| lz5hc r131 -5 | 10 MB/s | 701 MB/s | 49970285 | 47.66 |
| lz5hc r131 -7 | 5.54 MB/s | 682 MB/s | 49541511 | 47.25 |
| lz5hc r131 -9 | 2.69 MB/s | 673 MB/s | 49346894 | 47.06 |
| lz5hc r131 -11 | 1.36 MB/s | 664 MB/s | 49266526 | 46.98 |
| zstd v0.3 | 257 MB/s | 547 MB/s | 51231016 | 48.86 |
| zstd_HC v0.3 -1 | 257 MB/s | 553 MB/s | 51231016 | 48.86 |
| zstd_HC v0.3 -3 | 76 MB/s | 417 MB/s | 46774383 | 44.61 |
| zstd_HC v0.3 -5 | 40 MB/s | 476 MB/s | 45628362 | 43.51 |
| zstd_HC v0.3 -9 | 14 MB/s | 485 MB/s | 44840562 | 42.76 |
| zstd_HC v0.3 -13 | 9.34 MB/s | 469 MB/s | 43114895 | 41.12 |
| zstd_HC v0.3 -17 | 6.02 MB/s | 463 MB/s | 42989971 | 41.00 |
| zstd_HC v0.3 -21 | 3.35 MB/s | 461 MB/s | 42956964 | 40.97 |
| zstd_HC v0.3 -23 | 2.33 MB/s | 463 MB/s | 42934217 | 40.95 |

ShadowEagle 05-12-2015 02:07

Used this method:

http://ultraimg.com/images/Unbenannt2b400e.png

I tried to extract with inno setup. I copied all files from "Resources" folder into includes folder in inno setup script and added the reflate files into the script:

Code:

Source: Include\reflate.exe; DestDir: {tmp}; Flags: dontcopy
Source: Include\rawrest.exe; DestDir: {tmp}; Flags: dontcopy
Source: Include\rawdet.exe; DestDir: {tmp}; Flags: dontcopy
Source: Include\raw2hif.exe; DestDir: {tmp}; Flags: dontcopy

But i get this error:

http://ultraimg.com/images/Unbenannt4ecef.png

Pls help. What i`m doing wrong?

panker1992 05-12-2015 04:21

Quote:

Originally Posted by RamiroCruzo (Post 445130)
Lets compare ourselves which is better for our needs....:)

Source:Encode.ru

| Compressor name | Compression| Decompress.| Compr. size | Ratio |
| --------------- | -----------| -----------| ----------- | ----- |
| memcpy | 8533 MB/s | 8533 MB/s | 104857600 |100.00 |
| lz4 r131 | 480 MB/s | 2275 MB/s | 64872315 | 61.87 |
| lz4hc r131 -1 | 82 MB/s | 1896 MB/s | 59448496 | 56.69 |
| lz4hc r131 -3 | 54 MB/s | 1932 MB/s | 56343753 | 53.73 |
| lz4hc r131 -5 | 41 MB/s | 1969 MB/s | 55271312 | 52.71 |
| lz4hc r131 -7 | 31 MB/s | 1969 MB/s | 54889301 | 52.35 |
| lz4hc r131 -9 | 24 MB/s | 1969 MB/s | 54773517 | 52.24 |
| lz4hc r131 -11 | 20 MB/s | 1969 MB/s | 54751363 | 52.21 |
| lz4hc r131 -13 | 17 MB/s | 1969 MB/s | 54744790 | 52.21 |
| lz4hc r131 -15 | 14 MB/s | 2007 MB/s | 54741827 | 52.21 |
| lz5 r131 | 195 MB/s | 939 MB/s | 55884927 | 53.30 |
| lz5hc r131 -1 | 32 MB/s | 742 MB/s | 52927122 | 50.48 |
| lz5hc r131 -3 | 20 MB/s | 716 MB/s | 50970192 | 48.61 |
| lz5hc r131 -5 | 10 MB/s | 701 MB/s | 49970285 | 47.66 |
| lz5hc r131 -7 | 5.54 MB/s | 682 MB/s | 49541511 | 47.25 |
| lz5hc r131 -9 | 2.69 MB/s | 673 MB/s | 49346894 | 47.06 |
| lz5hc r131 -11 | 1.36 MB/s | 664 MB/s | 49266526 | 46.98 |
| zstd v0.3 | 257 MB/s | 547 MB/s | 51231016 | 48.86 |
| zstd_HC v0.3 -1 | 257 MB/s | 553 MB/s | 51231016 | 48.86 |
| zstd_HC v0.3 -3 | 76 MB/s | 417 MB/s | 46774383 | 44.61 |
| zstd_HC v0.3 -5 | 40 MB/s | 476 MB/s | 45628362 | 43.51 |
| zstd_HC v0.3 -9 | 14 MB/s | 485 MB/s | 44840562 | 42.76 |
| zstd_HC v0.3 -13 | 9.34 MB/s | 469 MB/s | 43114895 | 41.12 |
| zstd_HC v0.3 -17 | 6.02 MB/s | 463 MB/s | 42989971 | 41.00 |
| zstd_HC v0.3 -21 | 3.35 MB/s | 461 MB/s | 42956964 | 40.97 |
| zstd_HC v0.3 -23 | 2.33 MB/s | 463 MB/s | 42934217 | 40.95 |

Impressive but this test utilize the HC format of the zstd program
currently i test zstd v4.2

i have eaten all my day to test, test and test this :P

download the lz4 installer for windows and rename zstd ~~> lz4.exe and
replace in programs/lz4 folder

use this and you will see that decomp speed is all but fake i/o cant handle
2.2 gigs/sec because of limitation unless you have some x6 SSD array :P

sayang123 05-12-2015 05:49

Is there any installer script which support the unpack command from this app?

sorry for my bad english

rinaldo 05-12-2015 08:23

also applies in the ComboBox style. the success is in knowing how to match colors
(Too bright and too colorful are crap)
lit are used to such mail alert .. design still do not like you know.

felice2011 05-12-2015 09:48

also applies in the ComboBox style. the success is in knowing how to match colors
(Too bright and too colorful are crap)
lit are used to such mail alert .. design still do not like you know.[/QUOTE]

Depending on your point of view there are those who see the world in color and some in black and white... I see the world in color, anyway I see what you need.;)

felice2011 05-12-2015 12:17

Quote:

Originally Posted by sayang123 (Post 445136)
is there any installer script which support the unpack command from this app?

Sorry for my bad english

In attachment it is an example of installing proprietary 'not InnoSetup' with a data file compressed using the reflate method 'reflate+srep+delta+lzma:a1:mfbt4:d158m:fb273:mc10 00000000:lc8'
For what concerns me and for my use decompression or installation or setup of a compressed data folder, with the excellent method reflate, works beautifully.

coming soon, new universal extractor in new thread


All times are GMT -7. The time now is 09:38.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2026, vBulletin Solutions Inc.
FileForums @ https://fileforums.com