CLS-ZStd
 

Here's CLS version of ZStd, ran some tests, it's fast during compression because no temps are created, also added multi threading feature, it's only effective if you use a high level to compress, also if the chunk size is high as well, then it is effective, if you're using level 1 for example, and you try to use multi threading, just know that it is going to be slow, since level 1 is almost like copying a file, just adding multi threaded copying doesn't mean speed will be fast, it will drop.
During decompression, I ran tests as well, it's also fast, you can easily get more than 200mb/s. Decompression is single threaded.

This is nice i'll test.
Update.............
Nice speed.
I was tried chunks like 50m or 1g and 8b and the percent freeze in 9% or 100%. But in 1024k works very well. Maybe only accept k in chunks. Also pack and unpack no errors. Tested with 80 gb folder.
Threads 6
level 21
chunks 1024k

we need the felice2011 benchmark here!!!

Your PC freezes due to lack of RAM ;) ZStd is also an LZ-Compressor which uses about 10-20 times your Chunk size for better results. So, If ya enter 1 GB as chunk size, you'll need 20 GB RAM

Ok thanks for the tip.
With 8b the progress stack in 9% and complete the job without errors.
In 50m percent goes quick to 100% and stack to complete the job. In 1g arc.exe crash. In any case works this cls works faster.

1G chunks? Lol
And using 6 threads. That's 1GBx2x6 = 12GB Ram usage. X86 is limited to 2GB.
Memory usage is calculated as chunk size x 2 x number of threads

Hey wow ok. it's a test men. My tests goes from 0 to the limit of the program.

I place an quick test of compression and decompression with enwik8, with and without cls.dll, with a block of memory resident in the 16384k heap, to equal the same compression ratio in response to the use of zstd without the cls.

I also tried to use 8 threads, the speed of compression increases considerably, but the cpu ... squirts to hell, with freeze pc if not adequately cooled.

zstd without cls

zstd only cls

With the use of the cls the compression time decreases by almost 60% while maintaining the same compression ratio with and without cls.
On the contrary, the decompression time has a lower speed of 15-17% with the use of cls, but in this case I think you should test larger files.
These dll alternative by razor are a real boon from sky ...;)

Hello mates, I was not here from long time due to my studies actually, I managed some time today, so thought for a modified version by me in ZSTD :D

What I did?

1. Actually increased speed around atleast 2mb/s with the same cls through modified zstdlib_x86.dll without any ratio loss.

2. Added latest compiled version of Zstd x32 and x64, additionally PZstd also!

3. I did some benchmarks also, for all tests level 19 was used, was tested for compression only and threads were set as 2(It doesn't matter in PZstd case), I have maximum 2 threads only.


Test with new and older DLL

Tests Between Zstd and Pzstd with the Modified DLL


Edit: I will prefer to use zstd32_gcc620.exe or zstd64_gcc620.exe as zstd.exe for now with current Razor CLS.

You can run bench again.

With the files posted by @MAK ?

Yup. I doubt that there will much diff though.

All test ZSTD Same Options :
Test PZSTD Options

no matey, he actually mean, test between cls with zstd and cls without zstd for speed

This ZSTD v1.0.0 normal NO-CLS, the previous post the tests already carried out with CLS.
same all settings "-19" performed with enwik8.
Try you also with the same input file:rolleyes:

So, the conclusion in the last is, we finally made a zstd cls with no drawback of speed and ratio, and with full support to Inno, Am happy my zstdlib_x86.dll was useful here ;)

felice, chunk size is set to 100m, input is less than 100m, obviously no comparison with pzstd here because cls-zstd was single threaded either way.

Chunk size works this way.

It feeds a specified chunk size to a thread.
If you have 4 threads for example, you set 100m chunk size on a 360mb input.
thread 1-3 will get 100mb
thread 4 will get 60mb
for your case, it really doesn't matter how many threads you set, compression will always be single threaded because chunk size is bigger than main input, all this just goes to 1 thread, all other threads have nothing to process.

Excellent elucidation, when I have time I'll just repeat the test..;)..though honestly I think the difference will be minimal, but I hope to be wrong.

Today, having a little free time I repeated the test...:)...All Test Input Data 420MB.

Here we have as a benchmark compression via ZSTD v1.0 no CLS.
Here the version tested ZSTD540 with CLS, with variables set "MT = 2 or 4" and chunksize in a range from 50m to 500m.
Here the version tested zstd620 with CLS, with the same variables set "MT = 2 or 4" and the chunksize in a range from 50m to 512m.
As you can see the difference between the versions ZSTD 540 & 620 with CLS is nothing in percentage of compression, and of a few thousandths of seconds in compression time.
A distinct difference between the ZSTD version v1.0, and versions with no CLS 540 and CLS 620, with the latter results in a marked improvement in speed at the expense of better compression with ZSTD v1.0, so the choice is purely subjective, whether to use the version with or without CLS, but personally have not seen anything of sensational in these versions with CLS support.
Razor your lesson on the block of memory and the allocation thereof is correct, but in this case will be interpreted theoretically, in practice way I did not have any difference of advantage or disadvantage, both as a percentage of compression, that compression time , whatever of the range utilized for the memory blocks.

is there update for cls-zstd to the latest ver 1.2 ?

I'm pretty sure dropping in a v1.2 dll wouldn't be much of a problem. ;)