Helping thread :)

[FitGirl]

The example string works just fine
srep+4x4:t4:b128m:lzma:128m

Never use large dict size for 4x4
4x4 means you're running 4 instances of LZMA, each with 400Mb dict size, that leads to 400*4*11 = 17600 MB of RAM to compress.

On user machine ISDone will require four continuous memory blocks of 400 MB each.

Also, using 4x4 hurts compression ratio due to blocking. That's why "some guy" repacks are larger than any other. You can try to use LZMA2 instead of LZMA1 (gives you 2 decompression threads by default), but it has it's own downsides.

I've personally used 4x4:LZMA in some of my early repacks, but gave it up. Number of SSD users is not that high, and unpacking in four streams on HDD often leads to overload.

[MCWRX]

I tried this string

srep+4x4:t4:b128m:lzma:128m

but I am not able to unpack it.

[aswadd]

Originally Posted by MCWRX

I tried this string

srep+4x4:t4:b128m:lzma:128m

but I am not able to unpack it.

why you're making it like this 4x4:t4:b128m:lzma:128m
why not 4x4:lzma:b128m:128m

if you used fazip for 4x4 , lzma then you must use it again for unpacking
see (arc.ini)

[MCWRX]

Originally Posted by aswadd

why you're making it like this 4x4:t4:b128m:lzma:128m
why not 4x4:lzma:b128m:128m

if you used fazip for 4x4 , lzma then you must use it again for unpacking
see (arc.ini)

thanks, I will test.

I use LZMA x64

[aswadd]

Originally Posted by MCWRX

thanks, I will test.

I use LZMA x64

one here mentioned that don't use lzmax64.exe it's not good i don't remember why but he said that as he tested use 7z lzma2 (xz) or fazip for (4x4,lzma)

[Bulat]

Originally Posted by aswadd

if you used fazip for 4x4 , lzma then you must use it again for unpacking
see (arc.ini)

it's opposite - fazip can implement any internal arc.exe algorithm as external one. this means that you can compress with fazip and decompress internally, or vice versa.

in practice, i recommend to use fazip for large compression methods (rep, lzma, ppmd). you may also prefer to use fazip for decompression (even in installer) since it's yet another way to break 2/4 GB barrier of 32-bit programs

[Bulat]

Originally Posted by FitGirl

The example string works just fine
srep+4x4:t4:b128m:lzma:128m

Never use large dict size for 4x4
4x4 means you're running 4 instances of LZMA, each with 400Mb dict size, that leads to 400*4*11 = 17600 MB of RAM to compress.

On user machine ISDone will require four continuous memory blocks of 400 MB each.

Also, using 4x4 hurts compression ratio due to blocking. That's why "some guy" repacks are larger than any other. You can try to use LZMA2 instead of LZMA1 (gives you 2 decompression threads by default), but it has it's own downsides.

I've personally used 4x4:LZMA in some of my early repacks, but gave it up. Number of SSD users is not that high, and unpacking in four streams on HDD often leads to overload.

i agree with almost everything you said except for two things:

1. 4x4 doesn't mean "use exactly 4 threads both for compression and decompression". let's see - data are split into independently compressed blocks, so you can use any number of threads at compression stage and, independent of that, any number of threads at decompression stage. both arc and unarc find largest contiguous block available (in 64-bit windows it will be 2047 MB), divide its size by the memory required for single (de)compression thread, and run so many threads. so, on 64-bit windows it will use more threads, on 32-bit windows it will use less threads

2. lzma2 is different from lzma in two things:
- it stores incompressible chunks of data
- like 4x4, it can split input data into independent blocks. unfortunately, it always uses only 1 thread for decompression. When manual says "LZMA2 uses: 1 thread for each chunk in x1 and x3 modes; and 2 threads for each chunk in x5, x7 and x9 modes" it means only comperssion stage

So, compared to 4x4:lzma, lzma2 may give slightly better ratio and single-thread speed, but it can't decompress even in 2 threads. but probably you meant compression w/o splitting into independent blocks, where lzma2 indeed is slightly better, but still single-threaded on decompression side. unfortunately, it's inherent property of lzma algorithm - you can't use morte than 1 thread to decompress single lzma compressed stream

3. that said, inability to decompress data compressed with 4x4:lzma is a bug in my program. I already have a better algorithm for InsertTempfile procedure, but ATM it's not yet ideal and still wrtitten in Lua. But i will fix that sooner or later.

4. these days, most of user computers run Win64, so probably it's time to switch to 64-bit unarc.dll in most repacks? i will try to build one.