Archiver, 7z Alternative and FreeArc Replacement

[panker1992]

This is a project i have been designing for the past year,

this is practically a brand new archiver, it has custom everything,

Brand new Header, like Freearc which with its utility has taken this forum for the past decade and more
this project's plan is to practically replace it.

I have added a ton of stuff, including the ability to use external algorithms and tools, and i have also included xtool 7.9 from Razor Github page into it as an internal means to precompress.

it comes with 2 modes,

Solid Block, and Blocked compression

you can rely on solid block to pack your games while a block compression can be used to maintain archives.

I plan to expand this with GUI and a DLL for Inno.

================================================== ================================================== ================================================== ================================================== ===
Backlog:
================================================== ================================================== ================================================== ================================================== ===
Unsolveable:
1) zstd, cannot use window > 1GB in 32 bit, hence :w30 is max for 32 bit systems. it cannot even decompress it.
2) filtering has been suspended because its slow, this has been solved by using the precompression
3) packmp3 is single threaded due to init sequence they are having build in and their buffers automatically clear each other out. > waiting on YadeWira if he will provide a working DLL
4) Precomp Codec will never be able to work in multi succession, you can only have one instance per chain, but you can use whatever you want in it as many cases and whatever depth (max:10)

Known Cases Future Fix maybe:
32bit executable uses 64bit placeholders, originally a 32 bit wasn't in the plans. this is a future fix.
New code for delphi lzma2/lzma/xz/7z is added via sdk 26.01. Possibility of adding it, right now it's kinda buggy and a total mess, compressing 100mb takes 4 hours.

Bugs - To Do list:
Bugs: Brunsli Codec in Precomp, out of bound bugs, best replacement is packjpg by YadeWira which is the only one that is maintained.
Bugs: New patching Engine, release 0.1, which will be replacing zstd one that is actually present. thought they give same result on my tests. so is there any good reason to do it ?


Changes - WIP 7.5:
1) BWT - Preprocessor is added, Initial Test, 1 gb of input data: bwt+lz4 28% ratio, 31% lz4 only, Works with weak algos
2) Further Cleanup on the code, and optimization. Masks are being integrated, Counter is 65k
3) New Deduplication engine using rzip, available in blocked compression and Content Mixing.
4) turborep is being remade to use less ram and be faster, but has less scanning capabilities, has multithreading.
5) External Variable added to enable external only Codecs, this will effectively bypass all internal stuff, works on encode/decode
6) Absolute Offset is now on by default, in a multisream header.
7) Multithreading on packjpg is on, thanks to YadeWira, Encode.su, that provided multithreading code and DLL for the Library.
Fixed a bug in store that didn't allow the usage of passwords. (Critical)
9) LZMA2 once again is activated, i am actually fed up with this after many trials and errors which almost threw this out of hand.
I decided to write my own code to handle it, here is how it works, normally when you go with lzma it will create one stream of data.
LZMA2 works by dividing those data into chunks and packing them using lzma as a stream using the lzma compression props.
However making it streaming is actually tricky. My LZMA2 will write to a an internal Pipe letting it slide and glide, i have given decompression multithreading as well.
10) I also added bzip3 library to the project. lets see what this can do though. at this point i will add everything compression wise


This is <<<experimental>>>
Added a self made variant, or basically an idea i had for a long time, lzma3,
What it does, it takes chunks of data made into big blocks for example 4 times the dictionary, then it applies fast deduplication, which reduces it significantly, then applies lzma on top with dictionary
you can manipulate, lzma dictionary and level, and block and deduplication level.
This is <<<experimental>>>