Oodle MT Compressor for FreeArc

[78372]

Oodle Compressor 2.5.2 by RAD Game Tools

This is a wrapper to use it with freearc and stdio

Comprssion levels:
None = 0,
SuperFast = 1,
VeryFast = 2,
Fast = 3,
Normal = 4,
Optimal1 = 5,
Optimal2 = 6,
Optimal3 = 7,
Count = 8

None don't compress, just copy raw bytes
SuperFast super fast mode, compression can be very poor; try VeryFast first
VeryFast fastest LZ mode for normal use, good for runtime encoding
Fast fast - good for daily use
Normal standard medium speed LZ mode
Optimal1 optimal parse level 1 (faster optimal encoder)
Optimal2 optimal parse level 2 <- recommended
Optimal3 optimal parse level 3 (slower optimal encoder)

Compression methods:

None = 3,
Kraken = 8,
Leviathan = 13,
Mermaid = 9,
Selkie = 11,
LZNA = 7,
BitKnit = 10,
LZB16 = 4,
Hydra = 12,
LZBLW = 5,
LZH = 0,
LZHLW = 1,
LZNIB = 2,
LZA = 6,
Count = 14

None None = memcpy
Kraken Fast decodes, high compression, amazing! NOTE : LARGE QUANTUM
Leviathan Leviathan = Kraken's big brother with higher compression. NOTE : LARGE QUANTUM
Mermaid Mermaid is between Kraken & Selkie - crazy fast, still decent compression. NOTE : LARGE QUANTUM
Selkie Selkie is a super-fast relative of Mermaid. Faster than LZB16/LZ4 but more compression. NOTE : LARGE QUANTUM ; NOTE : Selkie will show up as Mermaid data in the decoder
LZNA LZNA : the highest compression option in Oodle, comparable to LZMA (7zip) but much faster to decode
BitKnit BitKnit ; usually close to LZNA compression levels but faster. Particularly great on some types of structured binary, such as structs of DWORD/float.
LZB16 LZB16 = LZ-Bytewise ; 64k window ; fast, low compression. Generally prefer Selkie unless you need the 64k window
Hydra Hydra, the many-headed beast = Kraken, Mermaid, or Selkie
LZBLW OLD : LZBLW = LZB-LargeWindow ; often a free win over LZB16
LZH OLD : LZH , 128k sliding window ; generally use LZHLW instead
LZHLW OLD : LZH-LargeWindow ; fast to decode, good compression
LZNIB OLD : LZ-Nibbled ; fast to decompress + medium compression
LZA DO NOT USE : LZA = LZ-Arithmetic ; use LZNA instead


Multithreading is done by chunks, remember that highest RAM usage for decompression = Chunksize*2*threads

May contain bugs, not fully tested with multicore cpus, I only have 2 cores

Updated oodlemt

Changelog
1. Fixed some threading bugs.
2. Added a not-so-good progress indicator if you don't want to use stdio
3. Compatibility broken

OodleMT 1.3 (Final)

ChangeLog

1. Memory usage optimized a little.
2. DLL is now loaded from external file. Therefore other updated dlls can be used too.
3. 32 bit edition is now available too.
4. Default options changed. It's optimal for game compression and speed for now. You can always change though

[shazzla]

Maybe a stupid question ,but is this x64 version ?
If no ,can u make an x64 version?
Atm im from mobile...
Thanks in advance !

[Jiva newstone]

@shazzla its already x64 bit complied

[masen485]

Oodle Compressor 2.5.2 by RAD Game Tools
HOW TO USE THE KNOWLEDGE
REQUIRED TO USE WITH THE EXAMPLE

[DiCaPrIo]

[External compressor:oodle]
header = 0
packcmd = oodlemt.exe e -c256 -m6 -l9 -t4 - - <stdin> <stdout>
unpackcmd = oodlemt.exe d - - <stdin> <stdout>

[masen485]

Oodle Compressor 2.5.2 by RAD Game Tools
which games are used
information and sample usage on a few games

[Sergey3695]

Leviathan = 13 - don't work for me.

[78372]

Leviathan = 13 - don't work for me.

XD I just copied from oodle docs, probably oodle 2.5.2 didn't contain Leviathan, that's from 2.6 I think

[Jiva newstone]

it is a compresstor and not pre-compresstor - if u want to use it,
Just like this
srep+oodle or srep+lzna
it has faster decompresstion, so can use instead of using lolz or lzma

[Jiva newstone]

use lzna or kraken instead of using old lzma or xz

[Simorq]

Thanks 78372

Only Count = 14 Not Work

[78372]

Sorry everyone, I actually didn't tested all codecs, I just tested kraken and copied other codec and method names from oodle documentation, I don't know if they exist in 2.5.2 core or not.
Anyways, if you find any codec not working at all, just post here and I will remove it from first post, sorry again.

[shazzla]

Interesting...

[External compressor:lzna]
header = 0
packcmd = oodlemt.exe e -m7 {options} - -

arc a -mlzna:c128:l7:t4 pack.arc data.dat

uses only basic(?) thread ,while running oodlemt.exe from command line t4 works fine.
any idea ? :(
(i5-6600)

Edit :

[External compressor:oodle]
header = 0
packcmd = oodlemt.exe e {options} - -

arc a -moodle:c128:m7:l7:t4 pack.arc data.dat

it works !
strange....
:confused:

[78372]

Some information at last


Try this, it may fix the error, works here for two threads, no idea if it will work for more than 2 or not, test it please.



Use -t4 and keep an eye on task manager, make sure your data is 4x than chunksize.

Test with both old and new edition and report.

Test compression only.

[shazzla]

Nothings changed here ,still the same.
Console version works fine with same file.
Old and new version with -t4 works from console ,but not with arc.exe (arc.exe uses only 25% of CPU ,console uses 100%).
Dataset is 1.2 GB.

[78372]

Nothings changed here ,still the same.
Console version works fine with same file.
Old and new version with -t4 works from console ,but not with arc.exe (arc.exe uses only 25% of CPU ,console uses 100%).
Dataset is 1.2 GB.

Console uses 100%? :| How much is used by oodlemt.exe? Both for old and new edition?

[shazzla]

i use process explorer to monitor my tasks.
if i use console version with -t4 ,cpu usage is 9x% (oodlemt ,of course).
with default t2 ,its 25%.

damn ,i have no time to experiment. :/
btw thank you very much !

[78372]

Updated oodlemt

Changelog
1. Fixed some threading bugs.
2.Added a not-so-good progress indicator if you don't want to use stdio
3. Compatibility broken

[78372]

High ram usage in this edition. It reached 2GB usage using only one thread and 32mb chunks (rep:1600mb+oodlemt:m8:l9:t1:c32).

Compression speed is not satisfactory, although I am using a celeron dual core pc. Decompression is fast enough as expected(used 2 threads and around 260MB RAM).

[78372]

Probably the last update uploaded.

OodleMT 1.3 (Final)


1. Memory usage optimized a little.
2. DLL is now loaded from external file. Therefore other updated dlls can be used too.
3. 32 bit edition is now available too.
4. Default options changed. It's optimal for game compression and speed for now. You can always change though.

Check first post for downloading.

No longer working on this anymore. It will not receive any future updates except bug-fixing

[Simorq]

@78372
-t#: Threads(Default = 2) to -t#: Threads(Default = t50p)
is it possible?

[78372]

@78372
-t#: Threads(Default = 2) to -t#: Threads(Default = t50p)
is it possible?

No, at least not in this edition

[kj911]

This noob question from bumping the thread:

Newest Oodle Core v6/7 and 8 (x64 only) have better compression ratio from various codec compare to v5?? OOZ taken from GitHub its working from compress any data? 32bit compile?

Any users, tested the few codecs variants from find the best versions from any data?? Textures, text data, sound files and more?