FileForums - CLS - Fast LZMA2

Page 1 of 2

Show 40 post(s) from this thread on one page

FileForums (https://fileforums.com/index.php)

- Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)

- - CLS - Fast LZMA2 (https://fileforums.com/showthread.php?t=103052)

78372

26-09-2019 01:39

CLS - Fast LZMA2

3 Attachment(s)

Hi, here's another cls for fast lzma2.
I haven't yet able to convert the streaming functions to delphi, so this cls uses 64MB chunks.

Here is the conversion

Here's the cls anyways.

If you are able to convert the whole c header file to delphi, please send me a copy.

[Update]
Added FPC Build. More optimized and uses less memory in some cases.

PSP17

09-11-2019 10:12

How to use it with freearc ? what should be add in arc.ini and temp files of installer ?

78372

10-11-2019 03:28

Nothing. Just keep cls-fl2.dll and fast-lzma2.dll

PSP17

16-11-2019 16:23

Hi, did test on "INSOMNIA The Ark" 10 GB file, and got nothing :(

packed with cls | 1133.47 sec = 7%. Speed 9.17 mB/s
unpack with cls | 141.39 sec = 48%. Speed 73.48 mB/s

packed with no cls | 1125.90 sec = 8%. Speed 9.23 mB/s
unpack with no cls | 140.49 sec = 55%. Speed 73.95 mB/s

Maybe I am doing something wrong ?

78372

17-11-2019 07:22

What do you mean by packed without cls? What was your compression method? It should not be lzma, it should be fl2.

PSP17

17-11-2019 11:39

Quote:

What do you mean by packed without cls?

It mean not using cls to see if it is any difference in pack/unpack speed.

Quote:

What was your compression method?

here is the whole chain:
batch file

Code:

arc.exe a -r --dirs -s; -ep1 -i1 -lc- -w.\_tmp -msrep+lzma2 "data.bin" "pack\*"

arc.ini

Code:

[External compressor:lzma2]

header = 0

default = :d25:fb=273:mf=bt4:mc=1000000:lc=4:lp=0

packcmd = "7-Zip" a -txz -an -mcrc=0 -m1=lzma2{:option}:mf=bt4 -mmt=on -mx9 -si -so <stdin> <stdout>

78372

17-11-2019 19:20

It should be -msrep+fl2 to use the cls.

Gupta

19-11-2019 08:34

You should try shared memory. I did implement that for zstd and this and it does provide speed boosts but that have deadlocks (like 1 out of 10) and I didn't find enough motivation to debug. Also, I did implement that from the ground up. You can use some library or something to make it work XD

Anyway nice work lately.

darkwolves

19-11-2019 09:25

it packs just fine for me but it will not let me unpack gives me a -2 error

78372

20-11-2019 06:36

64 bit FL2 - 256MB Chunks

1 Attachment(s)

Quote:

FreeArc 0.67 (March 15 2014) creating archive: data.arc
Compressed 1 file, 943,130,025 => 125,282,928 bytes. Ratio 13.28%
Compression time: cpu 8.30 sec/real 57.45 sec = 14%. Speed 16.42 mB/s
All OK

FreeArc 0.67 (March 15 2014) testing archive: data.arc
Tested 1 file, 125,282,928 => 943,130,025 bytes. Ratio 13.28%
Testing time: cpu 2.28 sec/real 8.42 sec = 27%. Speed 112.01 mB/s
All OK

64bit FL2 Compile, about 2MB sized exe :p

Quote:

[External compressor:fl2]
header = 0
packcmd = fl2 c - - <stdin> <stdout>
unpackcmd = fl2 d - - <stdin> <stdout>

darkwolves

20-11-2019 08:23

Quote:

Originally Posted by 78372 (Post 483594)

64bit FL2 Compile, about 2MB sized exe :p

i will try this i rather enjoyed the compression speed

darkwolves

20-11-2019 17:25

oh i like this i was even able to put in my lzma2 commands behind it and they worked
thank you kindly

doofoo24

20-11-2019 22:39

@78372 do you know if there a way to use 7-Zip-zstd in <stdin> <stdout> with (Fast LZMA2, Zstandard, Brotli...)...
https://github.com/mcmilk/7-Zip-zstd/releases

78372

20-11-2019 22:59

ZSTD and Brotli already supports stdio afaik.
And lz4/brotli aren't suitable for game compression

doofoo24

20-11-2019 23:11

sorry i ask the wrong question, :o
do you know the cmd for (Zstd-Brotli) in 7-Zip-zstd to use with freearc like normal 7z...
like "packcmd = "7-Zip" a -txz -an -mcrc=0 -m1=lzma2{:option}:mf=bt4 -mmt=on -mx9 -si -so <stdin> <stdout>"

All times are GMT -7. The time now is 15:55.

Page 1 of 2

Show 40 post(s) from this thread on one page