FileForums

FileForums (https://fileforums.com/index.php)
-   Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)
-   -   Cost vs return - aka worthiness debate (https://fileforums.com/showthread.php?t=99560)

78372 13-01-2018 20:55

Quote:

Originally Posted by doofoo24 (Post 465897)
@78372 by any chance do you know the setting for Brotli in Zip-zstd broti MT ?
https://github.com/mcmilk/7-Zip-zstd/releases
is it possible to use zstd/Brotli within 7z.dll/exe to compress and decompress with <stdin> <stdout>...

perhaps I can try to make a stdio cls/exe but currently I am too much busy.

doofoo24 13-01-2018 21:05

but do you know setting for arc.ini does it work ?

[External compressor:7zip]
header = 0
packcmd = 7za a -txz -an -mcrc=0 -mf=off -myx=0 -mmt=4 -m0=lzma2{:option} -si -so <stdin> <stdout>
unpackcmd = 7za x -txz -an -y -si -so <stdin> <stdout>

*what to change to brotli ??

78372 14-01-2018 05:29

Quote:

Originally Posted by doofoo24 (Post 465899)
but do you know setting for arc.ini does it work ?

[External compressor:7zip]
header = 0
packcmd = 7za a -txz -an -mcrc=0 -mf=off -myx=0 -mmt=4 -m0=lzma2{:option} -si -so <stdin> <stdout>
unpackcmd = 7za x -txz -an -y -si -so <stdin> <stdout>

*what to change to brotli ??

Probably it doesn't work for those methods but I haven't tested

elit 15-01-2018 13:33

Today I tried something different. Rather than focusing on general compressors, I decided to compare default multimedia compressor of FA vs alternatives. Specifically in this test, I was focusing on graphics .bmp files and default $bmp of FA which use grzip for this purpose. I compared it to similar alternatives suitable for graphics as well as more general compressors such as lolz.

Test consisted of around 1.3gb of .bmp files. First and most important thing, by default FA process each file separately. I tarred them into single file and compared default FA's $bmp algo how it cope with separated files vs 1 put together:

original: 1.33gb
-FA -mbmp on dir of files(internally each processed separately and put into archive = default behavior): 539mb

From now all following results are on tarred single file:
Code:

-FA -mbmp:        469.61mb !!!
-bcm:            531.96mb
-bsc:            518.54mb
-bsc -m5:        518.86mb
-dlz:            489.85mb
-uharc -m3:      457.73mb
-lolz:            451.42mb
-FA -xppmd:      483.05mb

First of all, even though uharc and lolz were(only ones) better than tarred FA's default $bmp, speed was like ~8x worse. Especially uharc which was slowest of them all. From my test it wasnt worth it, even for lolz not. Forget them, for that little gain they were way too slow. What was surprising to me was that even bsc and bcm did not beat default FA. In fact not even dlz did which was also very slow.

Now, dlz and lolz re specialized on dds textures and there they kick ass ~10%(I already tried), but for common image formats, dont bother.

Default FA's grzip rules, it is internal thus supporting pipelining so no disk trashing, it is extremely fast, multi-threaded and is just great overall.
However, as you see in this test default behavior of processing images separately hurt *big time*. How to solve it? Simple, add rep before bmp:

$bmp=bmp >> $bmp=5rep+bmp
Then when you use something like: FA -m5/$bmp=5rep+bmp -mc:rep/srep... what will happen is that FA will tar and srep images first before applying grzip. And btw, adding rep or srep help even further, it doesnt hurt at all.

elit 15-01-2018 15:45

And finally, I tested audio(wav) part of mm compressor of FA. Here results were unexpected. Although FA apply tta compressor on each(any) wav file, results are not so clear as you will see.

But first, let me tell you that originally I tried to compare tta to flac. On specific game I tried(IL2 Sturmovik), flac refused to recognize file format even though it have RIFF header and is played fine in foobar2000. These wav's must have been compressed or something. On top of that, you cannot "tar" files and apply flac on it, but tta is able to. Because of issues and compatibility problems, I came to conclussion that it is not worth replacing internal tta with external compressor. But thats not all, I tried tta vs lzma and... you will see. But lets start from beginning.

Original *.wav folder 44.72mb:
FA -mwav: 43.92mb < Clearly not working

Now tarred wavs to single file:
Code:

FA -mwav:                    42.84mb (small gain when tarred but not working properly)
FA -mxlzma:                  20.52mb !!!
FA -mdelta+xlzma:            20.52mb
FA -xlzma+tta:              20.52mb
FA -m5rep+delta+lzma:        21.14mb
FA -mtta+delta+xlzma:        42.84mb
FA -mtta+xlzma:              42.84mb (trying all kind of order as you see)
FA -mdelta+tta:              42.84mb

You can see something is wrong here. Neither tta nor flac seem to be able to process these wav's at all. Could be compressed? But how can then lzma achieve half the size?! This is problem because FA doesnt recognize wav's internally and apply tta based on extension. I couldnt help it so I downloaded random wav file from internet to confirm.

Code:

11k16bitpcm.wav(298kb):
FA -mtta:        203kb
FA -mxlzma:      260kb
FA -m5rep+tta:  203kb

Aha!
So this means:

-do continue using internal tta but verify it first if your wav files are "standard" ones or not, test and compare with lzma!

-this thing still need more testing to get right idea, I am particularly curious about combination of delta+tta on "real" wav's as well as effect of rep etc...
EDIT: Since tta doesnt work properly on tarred wav's, this argument is done now, no need for rep or delta on top of it $wav.

-(feel free to tar files together same way as $bmp I described above, like: $wav=5rep+wav for example(or better 0+tta because unlike $bmp here can be a slight loss of compression), tta is great in that unlike other codecs, it work on raw data regardless of extension. This could also help with some data packs that contain wav files inside as well as some audio banks.)
EDIT: ^Do not tar wav files after all, even though it worked on my test dir, during actual game repack test it resulted in decompression error. Keep standard $wav=wav setting.


All times are GMT -7. The time now is 09:10.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2026, vBulletin Solutions Inc.
FileForums @ https://fileforums.com