FileForums - NEW LOLZ V22c4

FileForums (https://fileforums.com/index.php)

- Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)

- - NEW LOLZ V22c4 (https://fileforums.com/showthread.php?t=102509)

Quote:

Originally Posted by KaktoR (Post 496735)

You want "best compression" and "fast speed". Both together aren't possible.

Default lolz settings are well balanced in most cases. So try lolz:mt# (# = number cpu threads)

Code:

Data detection options:

-dt  [0..1] - enables / disables the pos_ctx / dxt / raw detection. no headers, everything is detected based on the analysis of data statistics. Default: -dt1;

-dtp [0..1] - enables / disables transmission of statistics from previous blocks to subsequent blocks in the detector. Default: -dtp1;

-dtb [0..1] - enables / disables enumeration of all options regardless of heuristics. Default: dtb0;

-dto [0..1] - enables / disables detection of the best positional o1 context. Default: -dto1;

-dtm [0..1] - enables / disables raw graphics multimedia detection. Default: -dtm1;

-dtw [0..1] - enables / disables width detection for raw graphics and dxt textures;

-dtd [0..1] - enables / disables detection of dxt textures;



Multithreading options:

-mtt [0..1]   - for multithreading, specifies the operating mode to be used. When set to 0, the dictionary size must be at least 2 times the block size. In this mode, data for each stream will be loaded alternately in block size. In this mode, in most cases, you can achieve better compression than in the second, however, decompression requires as many streams as in compression. When 1, each block is compressed separately, without dependencies on neighboring data, respectively, the compression here is usually worse than in the first mode, but you can specify any number of streams for unpacking. It is for this mode that the options from cls.ini MaxThreadsUsage and MaxMemoryUsage are used. Default: -mtt0;

-mt  [1..16]  - sets the number of threads to process. With -mt1 and -mtt0, normal sequential lossless compression is obtained by dividing the stream into blocks. Default: -mt1;

-mtb [2..512] - sets the block size in MB. When -mt1 -mtt0 also plays a role, but minimal. And more does not mean better. Usually, for -mtt0, the optimal value is about 32-64MB, respectively, the size of the dictionary should be more than 2 times larger. For mtt1 mode, the dictionary size must be no more than the block size;



Main options:

-d   [16..2032] - dictionary size in MB. Default: -d64;

-tt  [1..256]   - the number of paths to consider in the optimal parser. It greatly affects the speed and compression ratio, but not unpacking. You shouldn't ask more than 16, I assure you it's not worth it. Default: -tt4;

-oh  [8..14]    - sets the maximum number of bytes that the optimal parser will process at a time (2 ^ X). Default: -oh12;

-os  [0 .. -oh] - sets the minimum number of bytes that the parser will process at a time (2 ^ X). Default: -os8;

-fba [0..4096]  - sets the size of the minimum match, at which the parser will not bother much in the calculations. Compression is decently accelerated (2 times) with a small loss of compression. At 0, these simplifications are disabled. Default: -fba256;

-fbb [0..4096]  - THIS OPTION DOES NOT WORK AT THE MOMENT. Asked for even greater simplifications;

-al  [0..1]     - enables / disables the calculation of the literal price even if rep0 matches. Default: -al1;

-x   [0..2]     - turns on the slow modes of the parser with the calculation of (almost) all lengths of the found match, as well as the options match + lit + rep0match. Very slow and merciless. And the benefit is very small. It's easier to add -tt; Default: -x0;



Matchfinder options:

-rt [0..2]    - THIS OPTION DOES NOT WORK AT THE MOMENT. I set the type of matchfinder - lz, rolz or hybrid mode, but rolz did not meet expectations and I made all the changes without taking it into account, so it does not work now; Default: -rt0;

-mc [2..1023] - Specifies the maximum number of traversals of the binary tree of matches, after which matches for the given position are no longer searched; Default: -mc128;



Model options:

-cm  [0..1] - enables / disables a simple context mixer in some critical places, which mixes a couple of models in each place. When enabled, it improves compression, but slows down decompression. Default: -cm1;

-bc  [0..8] - sets the level of influence of the previous byte on the mixer; Default: -bc4;

-lm  [0..4] - THIS OPTION DOES NOT WORK AT THE MOMENT. Specifies the type of an "elementary" literal. Complex high-order models with cm did not perform well, so I abandoned them, as did rolz. Default: -lm0;

-blo [0..8] - sets the degree of influence of the previous byte on the encoding of the upper part of the literal. Default: -blo8;

-bll [0..8] - sets the degree of influence of the previous byte on the encoding of the lower part of the literal. Default: -bll8;

-blr [0..8] - sets the degree of influence of the rep0lit byte on the encoding of the upper part of the literal. Default: -blr4;

-bm  [0..8] - sets the degree of influence of the rep0lit byte on the encoding of the match type flag. Default: -bm4;

-pc  [0..4] - sets the positional context for all encoding operations. It is automatically ignored when the detector is on. Default: -pc2;

-dmXY(X [0..3], Y [0..4]) - sets the model for encoding color pairs (X) and alpha channel (Y) pairs. At the maximum value of each parameter, adaptive switching between models is used, while the decompression speed is reduced, but the compression is better in most cases. Default: -dm34;

-gmXY(X [0..2], Y [0..1]) - X - sets the model for encoding raw graphics. at the maximum value, adaptive switching between models is enabled. However, in this case, it is rare to see a gain over the adaptive mode. Generally, mode 0 is in the lead, but its unpacking is 2 times slower than mode 1. Y - enables updating of statistics of models when they were not used (for example, there was a long match). For X0 and X1, it usually gives a small gain in compression, but the speed drops by a factor of 2 (it all depends on the data). In general, the most appropriate is -gm00, he - and the default mode;

i have readed this but it didnt go well with me

can u send one for fast compression and no matter for size
and one for slow compression but best size
like dodi and fitgirl

Fast but worse ratio

Code:

lolz:dt0:mtt1:mt#:d64:oh14:os14:fba4096

Slow but good ratio

Code:

lolz:dtb1:d256m:mt#:mc1023

PS: All depends of input data ofc

Quote:

Originally Posted by KaktoR (Post 496738)

Fast but worse ratio

Code:

lolz:dt0:mtt1:mt#:d64:oh14:os14:fba4096

Slow but good ratio

Code:

lolz:dtb1:d256m:mt#:mc1023

PS: All depends of input data ofc

OK Thanks i will give it a try
Do you have any social media link for u?

+ the input data perform well with xtool:mermaid+srep+lolz but the problem is when xtool stucks a lot means too slow i literally waited 6 hrs for 22gb and reached 84% for xtool only

hey guys, anyone know how to compress,
Monster.Hunter.World.v15.11.01
(105,712,123,904 bytes)

i tried Oodle Kraken, as the previous version but won't work.
I tried all oodle core 4 - 7 and
oo2core_7_win64.dll this one seems working fine but it won't compress.

For example.

There's a file 170MB and it will become 898mb once i use oo2core_7_win64.dll but it won't do any effect on lolz or z stranded. Won't compress.
if i try lolz it will become 168.5mb,
but with z standard it will become 186mb much worse..

If anyone know how to compress this
Please let me know..

Thank you 😇

Quote:

Originally Posted by yasitha (Post 496822)

It could be one of this games
https://fileforums.com/showpost.php?...postcount=2926

Quote:

Originally Posted by KaktoR (Post 496827)

It could be one of this games
https://fileforums.com/showpost.php?...postcount=2926

So there's nothing we can do about it?

hey guys i need best lolz configuration .

compressing slow is ok, i need best ratio configuration please write down.....

Can someone shed some light on what kind of data types the previous old versions of LOLZ compress better than the last two public versions? I read about this in several places, but a specific description, about the specific data types, could not be found in this regard. The same can be said for v21a7 vs. also for v22c4b versions. Are there specific examples of this?

What is certain in the case of a game, when tested, is that the current version compresses ~3% better and ~10% slower than the previous public version. But what about even older versions? It would be nice to test it. Many public compressors its tested, the specific dataset. (Not widely available on public.)

Hi guys!

When i use the ldmf1 parameter,many times ,the resulted files cannot be decompressed! (Freearc)
Somewhere in the middle it stop with an error.
Sadly i cant write the exact message.
I dont use the cls-ini file,because i have much more than enough RAM...
RAM modules are good,no faults,lot of free space on my drives...

Anyone faced this ?
Ideas?

Ups,forget : default LOLZ settings used all the time,only dictionary size modified,up to 2 Gb,but it doesnt matter.

Does anyone have the lolly v19j version binaries handy, in cls format? (two x86/64 exe and dll files. And working properly from WinXP.)

Original firstly publicated cls-based exe's, without dll file, dont't work, crashing now with decompression testing big 3+ GB size archive. The x86 exe, totally not working from WinXP. Win7 SP1 its ok, without arc usage methods decompressing.* (Newer version its have better comp. ratio in my testings.)

*Directly packed any data via lolz, ex: lolz_x64.exe <switches> data.dat packed.lolz
Decomprssing via cls-lolz_x86/64.exe packed.lolz data.dat

Hi guys. These are the parameters that usually I use:

-d256mb -dtb1 -mtt1 -tt4 -mt10 -mc1023

today I tried to add two parameters: -fba4096 -gm21

and the decompression doesn't work, doesn't start and I get unarc error.

Someone knows the reason?

Quote:

Originally Posted by L0v3craft (Post 502781)

well, the (-gm21) parameter doesn't work, use (-gm20) instead.

Compression challenge!

I wonder who would be able to compress the contents of the archive linked below (extracted into 100MB a *.arc file!) to a size smaller than 30MB or 30000KB... 25MB or less would not be bad. (Although, would it be of any practical use, in addition to benchmarking, to the given compressor/archiver if it is not capable of decompressing at a speed of 5-10+ MB/s.)

Compression (last v22c4b): lolz_x64.exe -dt -d64 -fba4096 -mc1023

Whatever you can use, you can start with the PAQ family.*

Link: https://www.upload.ee/files/16721310/TEST0.lolz.html

Decompressing use: cls-lolz_x64/86.exe TEST0.lolz TEST0.arc

*Note: FP8_v6 or LPAQ or MCM not better than LOLZ, 5-10% ratio less.

Question: Is there an asymmetric CM compressor?

^^
You'd be better off posting this on encode.su - it's not really suitable here. Nobody uses PAQ based compressors for PC games.

>> You'd be better posting this site from many results, publicly or private...

This encode.su focused on any not game related data, ex: enwik8-10, calgary and more datasets... 5-6 years ago in released last LOLZ/Razor versions.

What I would like to know:

Obviously, there have been various compressors/archivers under development in recent years, which are not yet public. Well, in connection with these, I would like to know whether, in the case of the data set above, we can still hope for a size reduction of at least another 20-30%. (Just as, to any extent, with other types of data.)

So, anyone who still has an idea/trick up their sleeve can test it, even with programs that have never been seen before. (rough example: Razor 1.6 developer version, LOLZ2, etc... Provided that you don't have to wait long until the public release of v2.0.) But, previous LOLZ versions can also be considered, up to the December 2016 release retroactively. Or anything else. If the given result is not public, it can also come privately.

Compare lists:

LOLZ v22c4b: 33 492 307 byte
RAZOR 1.03.7 with -d 64M: 37 446 869 byte (~331 sec) / -d 128M: 37 319 110 byte (~326 sec)
SCM v0.0.1d: 34 578 565 byte (used profile file. 10-25* slower than LOLZ.)
WinRK 3.12: 33 326 819 byte (~98 minutes!)
Paq8px_v208fix1 with level9: 30 505 692 byte (21 h 2 min!)
EMMA v0.1.25 x64 Fast/1024MB/order6/128MB/None/Low: 39 855 061 byte (~1056 sec)
EMMA v0.1.25 x64 (aka v1.25) modded Audio model profile: 32 522 027 byte (less than 1 hour)*
EMMA v0.1.25 x64 used preconfigured Audio (Max) profile: 31 835 173 byte (11298,3 sec)**

Note: Barely harder (slower) LOLZ settings given minimally better (~150-200kB) results than WinRK.

*Note2: No fast long matches search and Ludicrous complexity mode off. (Options: 1024MB/order9)
**Note3: Try it "Executables (Max)" or "Best" preconfigured settings, probably get in minimally better results. (estimately 200-500kB smaller size. Compress times around 6 to 9 hours.)