TARZ — Texture Adaptive RANS

[panker1992]

The internal naming is the name of the project i am replacing, for example addex is replacing srep and the internal name has many srep counterparts in it.

Addex is also heavily inspired from srep. And Tarz is also heavily inspired from Lolz.

I am fixing Decompression because Tarz stdio decompression has some bugs.
CLS should work.

Let me explain how my multithreading works, the data is split in exactly as many threads as you request.

For example if you have an input of 16GB and you request 16 threads. this automatically means you split the data in 16 parts of 1gb each, you create 16 sliding windows, 16 pairs of models and of course you will need exactly 16 threads to decompress the data.

The reason i hard limited the threads to 16 is because of that because in the beginning i decied that even 8 will be alot.

Tarz doesn't advertise encoding speed, however the decode speed is boosted when using multithreading because the load is decoded evenly.

i don't yet have code to make this a block based encoding possible. something which lolz can do. there is a possibility to add this later.

let me explain how ram works.

each dictionary you create, needs * 10 the ram + number of block + 64mb for LRM.

if you use 128mb dictionary and one thread, this means around 1600 mb of ram. to decompress this you need 128mb + 64mb for decoding.

which if you ask me is quite efficient. it's highly assymetric.

[FitGirl]

RAM reqs are close to LZMA/LOLZ, usually it's always 10-11 times + some extra.

As for threads - yes, block-based mode is a must then. Otherwise repacking is usable only for the system which created it, running decompressing in current 16t mode on 4t machine would probably be a bad idea

[panker1992]

For the time being this project is a TEST only - Demo.

i am currently working on it, i am fixing decode stdio.

what you guys in this forum can do to help me out is simple. Try it ..Combine it, let me know of the problems you see. so i can fix those.

what i need is a sample in which lolz does better, if for example i havent added a model that can bypass Tanz in prediction.

The block based Tarz will be a future addition which i also consider atm seriously.

The sweat spot is 4 threads, it gives the best ratio and decompression speed ratio available

as for lolz strings they will be gone soon i am rewriting this project clean.