PDA

View Full Version : cls-precomp - for any version (Multithreaded)


BLACKFIRE69
23-07-2020, 13:21
cls-precomp - for any version


This is a CLS library for Precomp that can be used as multithreads.

Tested on Precomp 0.38, 0.40, 0.41, 0.42, 0.43, 0.44, 0.45, 0.46, 0.47, 0.48. (may work on future versions. :D)



User can select different detection methods (and other options).
Detection methods,

1. Intense
2. Brute


Test Results on Precomp 0.48 x32:


-intense0 : 1.51 GB --> 2.35 GB
-intense2 : 1.51 GB --> 2.63 GB
-brute : 1.51 GB --> 2.64 GB


Old PrecompX(v1.0 and v2.0) uses -intense0.

You can get different compression ratio by changing the Chunk size and the Options.

Precomp v0.4.8 - options

-[option]:

c[lbn] Compression method to use, l = lzma2, b = bZip2, n = none <l>
lm[amount] Set maximal LZMA memory in MiB <1024>
lt[count] Set LZMA thread count <auto-detect: 4>
lf[+-][xpiatsd] Set LZMA filters (up to 3 of them can be combined) <none>
lf+[xpiatsd] = enable these filters, lf- = disable all
X = x86, P = PowerPC, I = IA-64, A = ARM, T = ARM-Thumb
S = SPARC, D = delta (must be followed by distance 1..256)
llc[bits] Set LZMA literal context bits <3>
llp[bits] Set LZMA literal position bits <0>
The sum of lc and lp must be inside 0..4
lpb[bits] Set LZMA position bits, must be inside 0..4 <2>
n[lbn] Convert a PCF file to this compression (same as above) <off>
d[depth] Set maximal recursion depth <10>
pfmeta[amount] Split deflate streams into meta blocks of this size in KiB <2048>
pfverify Force preflate to verify its generated reconstruction data
intense Detect raw zLib headers, too. Slower and more sensitive <off>
brute Brute force zLib detection. VERY Slow and most sensitive <off>
t[+-][pzgnfjsmb3] Compression type switch <all enabled>
t+ = enable these types only, t- = enable all types except these
P = PDF, Z = ZIP, G = GZip, N = PNG, F = GIF, J = JPG
S = SWF, M = MIME Base64, B = bZip2, 3 = MP3
f Fast mode, use first found compression lvl for all streams <off>
i[pos] Ignore stream at input file position [pos] <none>
s[size] Set minimal identical byte size to [size] <4 (64 intense mode)>
pdfbmp[+-] Wrap a BMP header around PDF images <off>
progonly[+-] Recompress progressive JPGs only (useful for PAQ) <off>
mjpeg[+-] Insert huffman table for MJPEG recompression <on>

You can use an optional number following -intense and -brute to set a
limit for how deep in recursion they should be used. E.g. -intense0 means
that intense mode will be used but not in recursion, -intense2 that only
streams up to recursion depth 2 will be treated intense (3 or higher in
this case won't). Using a sensible setting here can save you some time.

github
31-01-2021, 13:09
Thanks.

Is there an option to specify the output directory of the compressed data?
before using this, my bat is
arc.exe data/data1.bin

but with this cls-precomp.ini and cls-precomp.dll, mt works however the compressed file will write in to the place where .ini/.dll or arc.exe is, it doesn't do the data/data1.bin anymore.
Same happens to the temporary file folders and log file.

Edit:
I gave this a go with various file sets, on a folder with some .zip and .xlsx files, compressing is fine and about 25% faster than without this.
But decompressing always fail with a particular .zip or a .xlsx file. :(

github
06-02-2021, 13:42
By giving the absolute path the file output positions are ok now.

This one seems to be the most functional and most reliable multi thread precomp dll, thank you very much for your great work