FileForums
Page 2 of 2 1 2
Show 40 post(s) from this thread on one page

FileForums (https://fileforums.com/index.php)
-   Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)
-   -   CLS-ZStd (https://fileforums.com/showthread.php?t=98121)

Razor12911 08-10-2016 10:01
Quote:
Originally Posted by felice2011 (Post 452585)
Code:
-↓- [ CMD Bench.Test.Info v0.0.6c ] -↓- Compressed Archive Completed At -↓- 08/10/2016 13:54:37
=============================================================================================================================================
 ○ [ ALGORITHM ] ○ [ INPUT SIZE ] ○ [ OUTPUT SIZE ] ○ [ RATIO.% ] ○ [ ARCHIVED FILES ] ○ [ COMP.TIME ] ○ [ SYNTAX - OPTIONS - ARGUMENTS ] ○

        Arc+zstd          95,4°MB          26,2°MB      27,46 %              1°Files    00:00:44:940  a -ma9 -ds -lc1024 -ld1024 -ep1 -di -i2 -ed -r -s; -w"temp" -mzstd
=============================================================================================================================================

-↓- [ CMD Bench.Test.Info v0.0.6c ] -↓- Compressed Archive Completed At -↓- 08/10/2016 13:56:59
=============================================================================================================================================
 ○ [ ALGORITHM ] ○ [ INPUT SIZE ] ○ [ OUTPUT SIZE ] ○ [ RATIO.% ] ○ [ ARCHIVED FILES ] ○ [ COMP.TIME ] ○ [ SYNTAX - OPTIONS - ARGUMENTS ] ○

Arc+zstd32_gcc620        95,4°MB          26,2°MB      27,46 %              1°Files    00:00:43:959  a -ma9 -ds -lc1024 -ld1024 -ep1 -di -i2 -ed -r -s; -w"temp" -mzstd
=============================================================================================================================================

-↓- [ CMD Bench.Test.Info v0.0.6c ] -↓- Compressed Archive Completed At -↓- 08/10/2016 13:59:05
=============================================================================================================================================
 ○ [ ALGORITHM ] ○ [ INPUT SIZE ] ○ [ OUTPUT SIZE ] ○ [ RATIO.% ] ○ [ ARCHIVED FILES ] ○ [ COMP.TIME ] ○ [ SYNTAX - OPTIONS - ARGUMENTS ] ○

Arc+zstd64_gcc620        95,4°MB          26,2°MB      27,46 %              1°Files    00:00:43:956  a -ma9 -ds -lc1024 -ld1024 -ep1 -di -i2 -ed -r -s; -w"temp" -mzstd
=============================================================================================================================================

-↓- [ CMD Bench.Test.Info v0.0.6c ] -↓- Compressed Archive Completed At -↓- 08/10/2016 14:09:55
=============================================================================================================================================
 ○ [ ALGORITHM ] ○ [ INPUT SIZE ] ○ [ OUTPUT SIZE ] ○ [ RATIO.% ] ○ [ ARCHIVED FILES ] ○ [ COMP.TIME ] ○ [ SYNTAX - OPTIONS - ARGUMENTS ] ○

Arc+pzstd_gcc620          95,4°MB          26,7°MB      27,99 %              1°Files    00:00:09:946  a -ma9 -ds -lc1024 -ld1024 -ep1 -di -i2 -ed -r -s; -w"temp" -mpzstd
=============================================================================================================================================
All test ZSTD Same Options :
Code:
Level=19
ChunkSize=100M
Threads=2
Test PZSTD Options
Code:
pzstd -19 -f
felice, chunk size is set to 100m, input is less than 100m, obviously no comparison with pzstd here because cls-zstd was single threaded either way.

Chunk size works this way.

It feeds a specified chunk size to a thread.
If you have 4 threads for example, you set 100m chunk size on a 360mb input.
thread 1-3 will get 100mb
thread 4 will get 60mb
for your case, it really doesn't matter how many threads you set, compression will always be single threaded because chunk size is bigger than main input, all this just goes to 1 thread, all other threads have nothing to process.

felice2011 08-10-2016 15:25
Excellent elucidation, when I have time I'll just repeat the test..;)..though honestly I think the difference will be minimal, but I hope to be wrong.

felice2011 15-10-2016 05:29
Today, having a little free time I repeated the test...:)...All Test Input Data 420MB.

Here we have as a benchmark compression via ZSTD v1.0 no CLS.
Code:
-↓- [ CMD Bench.Test.Info v0.0.6c ] -↓- Compressed Archive Completed At -↓- 15/10/2016 09:58:52
=============================================================================================================================================
 ○ [ ALGORITHM ] ○ [ INPUT SIZE ] ○ [ OUTPUT SIZE ] ○ [ RATIO.% ] ○ [ ARCHIVED FILES ] ○ [ COMP.TIME ] ○ [ SYNTAX - OPTIONS - ARGUMENTS ] ○

    Arc+zstd_1.0        419,6°MB          195,9°MB      46,69 %            14°Files    00:03:22:986  a -ma9 -ds -lc1024 -ld1024 -ep1 -di -i2 -ed -r -s; -w"temp" -mzstd
=============================================================================================================================================
Here the version tested ZSTD540 with CLS, with variables set "MT = 2 or 4" and chunksize in a range from 50m to 500m.
Code:
-↓- [ CMD Bench.Test.Info v0.0.6c ] -↓- Compressed Archive Completed At -↓- 15/10/2016 12:31:18
=============================================================================================================================================
 ○ [ ALGORITHM ] ○ [ INPUT SIZE ] ○ [ OUTPUT SIZE ] ○ [ RATIO.% ] ○ [ ARCHIVED FILES ] ○ [ COMP.TIME ] ○ [ SYNTAX - OPTIONS - ARGUMENTS ] ○

Arc+zstd540mt2cs50        419,6°MB          235,6°MB      56,15 %            14°Files    00:00:02:946  a -ma9 -ds -lc1024 -ld1024 -ep1 -di -i2 -ed -r -s; -w"temp" -mzstd
=============================================================================================================================================

-↓- [ CMD Bench.Test.Info v0.0.6c ] -↓- Compressed Archive Completed At -↓- 15/10/2016 12:32:47
=============================================================================================================================================
 ○ [ ALGORITHM ] ○ [ INPUT SIZE ] ○ [ OUTPUT SIZE ] ○ [ RATIO.% ] ○ [ ARCHIVED FILES ] ○ [ COMP.TIME ] ○ [ SYNTAX - OPTIONS - ARGUMENTS ] ○

Arc+zstd540mt4cs50        419,6°MB          235,6°MB      56,15 %            14°Files    00:00:02:932  a -ma9 -ds -lc1024 -ld1024 -ep1 -di -i2 -ed -r -s; -w"temp" -mzstd
=============================================================================================================================================

-↓- [ CMD Bench.Test.Info v0.0.6c ] -↓- Compressed Archive Completed At -↓- 15/10/2016 12:34:25
=============================================================================================================================================
 ○ [ ALGORITHM ] ○ [ INPUT SIZE ] ○ [ OUTPUT SIZE ] ○ [ RATIO.% ] ○ [ ARCHIVED FILES ] ○ [ COMP.TIME ] ○ [ SYNTAX - OPTIONS - ARGUMENTS ] ○

Arc+zstd540mt2cs64        419,6°MB          235,6°MB      56,15 %            14°Files    00:00:02:948  a -ma9 -ds -lc1024 -ld1024 -ep1 -di -i2 -ed -r -s; -w"temp" -mzstd
=============================================================================================================================================

-↓- [ CMD Bench.Test.Info v0.0.6c ] -↓- Compressed Archive Completed At -↓- 15/10/2016 12:35:18
=============================================================================================================================================
 ○ [ ALGORITHM ] ○ [ INPUT SIZE ] ○ [ OUTPUT SIZE ] ○ [ RATIO.% ] ○ [ ARCHIVED FILES ] ○ [ COMP.TIME ] ○ [ SYNTAX - OPTIONS - ARGUMENTS ] ○

Arc+zstd540mt4cs64        419,6°MB          235,6°MB      56,15 %            14°Files    00:00:02:951  a -ma9 -ds -lc1024 -ld1024 -ep1 -di -i2 -ed -r -s; -w"temp" -mzstd
=============================================================================================================================================

-↓- [ CMD Bench.Test.Info v0.0.6c ] -↓- Compressed Archive Completed At -↓- 15/10/2016 12:15:35
=============================================================================================================================================
 ○ [ ALGORITHM ] ○ [ INPUT SIZE ] ○ [ OUTPUT SIZE ] ○ [ RATIO.% ] ○ [ ARCHIVED FILES ] ○ [ COMP.TIME ] ○ [ SYNTAX - OPTIONS - ARGUMENTS ] ○

Arc+zstd540mt2cs100        419,6°MB          199,2°MB      47,47 %            14°Files    00:01:35:918  a -ma9 -ds -lc1024 -ld1024 -ep1 -di -i2 -ed -r -s; -w"temp" -mzstd
=============================================================================================================================================

-↓- [ CMD Bench.Test.Info v0.0.6c ] -↓- Compressed Archive Completed At -↓- 15/10/2016 12:22:12
=============================================================================================================================================
 ○ [ ALGORITHM ] ○ [ INPUT SIZE ] ○ [ OUTPUT SIZE ] ○ [ RATIO.% ] ○ [ ARCHIVED FILES ] ○ [ COMP.TIME ] ○ [ SYNTAX - OPTIONS - ARGUMENTS ] ○

Arc+zstd540mt4cs100        419,6°MB          235,6°MB      56,15 %            14°Files    00:00:02:942  a -ma9 -ds -lc1024 -ld1024 -ep1 -di -i2 -ed -r -s; -w"temp" -mzstd
=============================================================================================================================================

-↓- [ CMD Bench.Test.Info v0.0.6c ] -↓- Compressed Archive Completed At -↓- 15/10/2016 12:23:41
=============================================================================================================================================
 ○ [ ALGORITHM ] ○ [ INPUT SIZE ] ○ [ OUTPUT SIZE ] ○ [ RATIO.% ] ○ [ ARCHIVED FILES ] ○ [ COMP.TIME ] ○ [ SYNTAX - OPTIONS - ARGUMENTS ] ○

Arc+zstd540mt2cs250        419,6°MB          235,6°MB      56,15 %            14°Files    00:00:02:944  a -ma9 -ds -lc1024 -ld1024 -ep1 -di -i2 -ed -r -s; -w"temp" -mzstd
=============================================================================================================================================

-↓- [ CMD Bench.Test.Info v0.0.6c ] -↓- Compressed Archive Completed At -↓- 15/10/2016 12:26:36
=============================================================================================================================================
 ○ [ ALGORITHM ] ○ [ INPUT SIZE ] ○ [ OUTPUT SIZE ] ○ [ RATIO.% ] ○ [ ARCHIVED FILES ] ○ [ COMP.TIME ] ○ [ SYNTAX - OPTIONS - ARGUMENTS ] ○

Arc+zstd540mt4cs250        419,6°MB          235,6°MB      56,15 %            14°Files    00:00:02:937  a -ma9 -ds -lc1024 -ld1024 -ep1 -di -i2 -ed -r -s; -w"temp" -mzstd
=============================================================================================================================================

-↓- [ CMD Bench.Test.Info v0.0.6c ] -↓- Compressed Archive Completed At -↓- 15/10/2016 12:28:34
=============================================================================================================================================
 ○ [ ALGORITHM ] ○ [ INPUT SIZE ] ○ [ OUTPUT SIZE ] ○ [ RATIO.% ] ○ [ ARCHIVED FILES ] ○ [ COMP.TIME ] ○ [ SYNTAX - OPTIONS - ARGUMENTS ] ○

Arc+zstd540mt2cs500        419,6°MB          235,6°MB      56,15 %            14°Files    00:00:02:951  a -ma9 -ds -lc1024 -ld1024 -ep1 -di -i2 -ed -r -s; -w"temp" -mzstd
=============================================================================================================================================

-↓- [ CMD Bench.Test.Info v0.0.6c ] -↓- Compressed Archive Completed At -↓- 15/10/2016 12:29:56
=============================================================================================================================================
 ○ [ ALGORITHM ] ○ [ INPUT SIZE ] ○ [ OUTPUT SIZE ] ○ [ RATIO.% ] ○ [ ARCHIVED FILES ] ○ [ COMP.TIME ] ○ [ SYNTAX - OPTIONS - ARGUMENTS ] ○

Arc+zstd540mt4cs500        419,6°MB          235,6°MB      56,15 %            14°Files    00:00:02:941  a -ma9 -ds -lc1024 -ld1024 -ep1 -di -i2 -ed -r -s; -w"temp" -mzstd
=============================================================================================================================================
Here the version tested zstd620 with CLS, with the same variables set "MT = 2 or 4" and the chunksize in a range from 50m to 512m.
Code:
-↓- [ CMD Bench.Test.Info v0.0.6c ] -↓- Compressed Archive Completed At -↓- 15/10/2016 12:38:55
=============================================================================================================================================
 ○ [ ALGORITHM ] ○ [ INPUT SIZE ] ○ [ OUTPUT SIZE ] ○ [ RATIO.% ] ○ [ ARCHIVED FILES ] ○ [ COMP.TIME ] ○ [ SYNTAX - OPTIONS - ARGUMENTS ] ○

Arc+zstd620mt2cs50        419,6°MB          235,6°MB      56,15 %            14°Files    00:00:02:947  a -ma9 -ds -lc1024 -ld1024 -ep1 -di -i2 -ed -r -s; -w"temp" -mzstd
=============================================================================================================================================

-↓- [ CMD Bench.Test.Info v0.0.6c ] -↓- Compressed Archive Completed At -↓- 15/10/2016 12:41:04
=============================================================================================================================================
 ○ [ ALGORITHM ] ○ [ INPUT SIZE ] ○ [ OUTPUT SIZE ] ○ [ RATIO.% ] ○ [ ARCHIVED FILES ] ○ [ COMP.TIME ] ○ [ SYNTAX - OPTIONS - ARGUMENTS ] ○

Arc+zstd620mt4cs50        419,6°MB          235,6°MB      56,15 %            14°Files    00:00:02:932  a -ma9 -ds -lc1024 -ld1024 -ep1 -di -i2 -ed -r -s; -w"temp" -mzstd
=============================================================================================================================================

-↓- [ CMD Bench.Test.Info v0.0.6c ] -↓- Compressed Archive Completed At -↓- 15/10/2016 12:42:46
=============================================================================================================================================
 ○ [ ALGORITHM ] ○ [ INPUT SIZE ] ○ [ OUTPUT SIZE ] ○ [ RATIO.% ] ○ [ ARCHIVED FILES ] ○ [ COMP.TIME ] ○ [ SYNTAX - OPTIONS - ARGUMENTS ] ○

Arc+zstd620mt2cs64        419,6°MB          235,6°MB      56,15 %            14°Files    00:00:02:944  a -ma9 -ds -lc1024 -ld1024 -ep1 -di -i2 -ed -r -s; -w"temp" -mzstd
=============================================================================================================================================

-↓- [ CMD Bench.Test.Info v0.0.6c ] -↓- Compressed Archive Completed At -↓- 15/10/2016 12:43:55
=============================================================================================================================================
 ○ [ ALGORITHM ] ○ [ INPUT SIZE ] ○ [ OUTPUT SIZE ] ○ [ RATIO.% ] ○ [ ARCHIVED FILES ] ○ [ COMP.TIME ] ○ [ SYNTAX - OPTIONS - ARGUMENTS ] ○

Arc+zstd620mt4cs64        419,6°MB          235,6°MB      56,15 %            14°Files    00:00:02:944  a -ma9 -ds -lc1024 -ld1024 -ep1 -di -i2 -ed -r -s; -w"temp" -mzstd
=============================================================================================================================================

-↓- [ CMD Bench.Test.Info v0.0.6c ] -↓- Compressed Archive Completed At -↓- 15/10/2016 12:45:15
=============================================================================================================================================
 ○ [ ALGORITHM ] ○ [ INPUT SIZE ] ○ [ OUTPUT SIZE ] ○ [ RATIO.% ] ○ [ ARCHIVED FILES ] ○ [ COMP.TIME ] ○ [ SYNTAX - OPTIONS - ARGUMENTS ] ○

Arc+zstd620mt2cs128        419,6°MB          235,6°MB      56,15 %            14°Files    00:00:02:950  a -ma9 -ds -lc1024 -ld1024 -ep1 -di -i2 -ed -r -s; -w"temp" -mzstd
=============================================================================================================================================

-↓- [ CMD Bench.Test.Info v0.0.6c ] -↓- Compressed Archive Completed At -↓- 15/10/2016 12:46:23
=============================================================================================================================================
 ○ [ ALGORITHM ] ○ [ INPUT SIZE ] ○ [ OUTPUT SIZE ] ○ [ RATIO.% ] ○ [ ARCHIVED FILES ] ○ [ COMP.TIME ] ○ [ SYNTAX - OPTIONS - ARGUMENTS ] ○

Arc+zstd620mt4cs128        419,6°MB          235,6°MB      56,15 %            14°Files    00:00:03:04  a -ma9 -ds -lc1024 -ld1024 -ep1 -di -i2 -ed -r -s; -w"temp" -mzstd
=============================================================================================================================================

-↓- [ CMD Bench.Test.Info v0.0.6c ] -↓- Compressed Archive Completed At -↓- 15/10/2016 12:47:33
=============================================================================================================================================
 ○ [ ALGORITHM ] ○ [ INPUT SIZE ] ○ [ OUTPUT SIZE ] ○ [ RATIO.% ] ○ [ ARCHIVED FILES ] ○ [ COMP.TIME ] ○ [ SYNTAX - OPTIONS - ARGUMENTS ] ○

Arc+zstd620mt2cs256        419,6°MB          235,6°MB      56,15 %            14°Files    00:00:02:941  a -ma9 -ds -lc1024 -ld1024 -ep1 -di -i2 -ed -r -s; -w"temp" -mzstd
=============================================================================================================================================

-↓- [ CMD Bench.Test.Info v0.0.6c ] -↓- Compressed Archive Completed At -↓- 15/10/2016 12:48:40
=============================================================================================================================================
 ○ [ ALGORITHM ] ○ [ INPUT SIZE ] ○ [ OUTPUT SIZE ] ○ [ RATIO.% ] ○ [ ARCHIVED FILES ] ○ [ COMP.TIME ] ○ [ SYNTAX - OPTIONS - ARGUMENTS ] ○

Arc+zstd620mt4cs256        419,6°MB          235,6°MB      56,15 %            14°Files    00:00:02:957  a -ma9 -ds -lc1024 -ld1024 -ep1 -di -i2 -ed -r -s; -w"temp" -mzstd
=============================================================================================================================================

-↓- [ CMD Bench.Test.Info v0.0.6c ] -↓- Compressed Archive Completed At -↓- 15/10/2016 12:49:40
=============================================================================================================================================
 ○ [ ALGORITHM ] ○ [ INPUT SIZE ] ○ [ OUTPUT SIZE ] ○ [ RATIO.% ] ○ [ ARCHIVED FILES ] ○ [ COMP.TIME ] ○ [ SYNTAX - OPTIONS - ARGUMENTS ] ○

Arc+zstd620mt2cs512        419,6°MB          235,6°MB      56,15 %            14°Files    00:00:02:944  a -ma9 -ds -lc1024 -ld1024 -ep1 -di -i2 -ed -r -s; -w"temp" -mzstd
=============================================================================================================================================

-↓- [ CMD Bench.Test.Info v0.0.6c ] -↓- Compressed Archive Completed At -↓- 15/10/2016 12:50:02
=============================================================================================================================================
 ○ [ ALGORITHM ] ○ [ INPUT SIZE ] ○ [ OUTPUT SIZE ] ○ [ RATIO.% ] ○ [ ARCHIVED FILES ] ○ [ COMP.TIME ] ○ [ SYNTAX - OPTIONS - ARGUMENTS ] ○

Arc+zstd620mt4cs512        419,6°MB          235,6°MB      56,15 %            14°Files    00:00:02:948  a -ma9 -ds -lc1024 -ld1024 -ep1 -di -i2 -ed -r -s; -w"temp" -mzstd
=============================================================================================================================================
As you can see the difference between the versions ZSTD 540 & 620 with CLS is nothing in percentage of compression, and of a few thousandths of seconds in compression time.
A distinct difference between the ZSTD version v1.0, and versions with no CLS 540 and CLS 620, with the latter results in a marked improvement in speed at the expense of better compression with ZSTD v1.0, so the choice is purely subjective, whether to use the version with or without CLS, but personally have not seen anything of sensational in these versions with CLS support.
Razor your lesson on the block of memory and the allocation thereof is correct, but in this case will be interpreted theoretically, in practice way I did not have any difference of advantage or disadvantage, both as a percentage of compression, that compression time , whatever of the range utilized for the memory blocks.

doofoo24 16-06-2017 02:14
is there update for cls-zstd to the latest ver 1.2 ?

Razor12911 26-06-2017 06:23
Quote:
Originally Posted by doofoo24 (Post 459798)
is there update for cls-zstd to the latest ver 1.2 ?
I'm pretty sure dropping in a v1.2 dll wouldn't be much of a problem. ;)


All times are GMT -7. The time now is 03:24.
Page 2 of 2 1 2
Show 40 post(s) from this thread on one page

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2026, vBulletin Solutions Inc.
FileForums @ https://fileforums.com