balrogg: lossless Vorbis/Opus recompressor

[_contextmodel]

balrogg losslessly recompresses Ogg Vorbis and Opus files. Archives are typically 8-12% smaller than .ogg files and 3-8% smaller than .opus files. After decompression they are reconstructed identically.

The source code (GPLv3) and binaries are available here: https://github.com/iczelia/balrogg.

Hopefully some find this useful. The Windows binaries are attached to the thread (64-bit, 32-bit modern Windows, 32-bit Windows 95 and newer).

Edit (10/09/26):
- v1.4 is now attached. It adds solid mode for Opus processing. Until v2.0 is reached, the releases will be neither forwards- nor backwards-compatible.

Edit (07/09/26):
- v1.3 is now attached. It improves compression and decompression speed and memory usage. More slightly malformed files are accepted. Vorbis and Opus comments are now supported. Until v2.0 is reached, the releases will be neither forwards- nor backwards-compatible.

Edit (06/09/26):
- v1.2 is now attached. It improves compression and decompression speed and memory usage. More slightly malformed files are accepted. Until v2.0 is reached, the releases will be neither forwards- nor backwards-compatible.

Edit (05/09/26):
- v1.1 is now attached. It slightly improves compression and decompression speed and memory usage. It also accepts more valid Vorbis and Opus files. Until v2.0 is reached, the releases will be neither forwards- nor backwards-compatible.

[kj911]

Very good work!

In the case of the OGG container format, 5 different encoding profiles were tested (see sample files, description), where in the case of small >30s sample files, it processed the files flawlessly, while on a ~2 hour test file (different source material), it only partially succeeded, with various error messages or endless encoding loops. The memory usage was always between ~200,000KB and ~270,000KB, slowly increasing, over and over again, without writing an output file. Compared to OGGRE, it consumes 2-3-4* more memory, but often with better results. It also brought more than 10% size reduction on the tested OPUS file.

[_contextmodel]

kj911, thank you for the test. Would you mind sharing the corpus used, so that the failures can be pinned down? The high memory usage, among other issues that you observed, is unintended (unless working in parallel).

[kj911]

The test corpus is a bit problematic (and the net too, up: <20kB/s!), at least in terms of public and copyright.

The small samples are from ESB, I'm now testing this quickly to see how the program reacts to converting the entire 2-hour file to OGG. (Source It's in 256kbps AC3.) The problematic cases written in TXT occurred on the ANH audio. (It's in PCM as part of a 10+ GB MKV file.) What is questionable is whether the error appears depending on the given audio and/or just the time length. I also tested it on a 100+ minute OPUS file, without error. (With audio copied from a WEBM video, taken from a video hosting site.)

The conversion/copying was done using XMedia Recode 3.2.7.1 x86 (last XP-compatible release).

UPD: Infinite loop again!

Source: ESB / 2hr / 256kbps AC3

128kbps, Vorbis/ABR OGG via l1: Works!
128kbps, Quality (q4) OGG via l9: Infinite compression loop. Level4 works. (watch longer than 2-4minutes times the Windows Process Explorer.)

The errors you experienced can be easily reconstructed if you have ~2 hours (SW, etc...) of audio files and test all the OGG encoding settings that XMediaRecode allows when encoding audio files to OGG.

UPD2: Infinite loop = Various compression methods used, forced it??

First 10minutes piece from ANH audio, encoded to 128kbps/q4 OGG file: Memory fluctuation are present it avg. 8-12 or upto times (~18-24MB), compression finished! Default cases, 100MB's OGG handled it 1-2minutes times. And one compression steps. Level5 only two memory fluctuation and finished it 17 seconds. (5.8MB's OGG) Level1 in less than 2sec.

Other bugs, not easily reproducible. (packet ends/unpaired bits or any.)

[_contextmodel]

Can you try v1.1 with the progress tracking enabled, and re-measure the encoding/decoding time and memory usage? I have done some tests on larger and more complex Vorbis files.

[kj911]

Recreating problematic files again (few hours ago deleted it):

Encoding times avg. 13-15min per file from Xeon PC..

First test "packet ends" errors via v1.1 are fixed! (128kbps ABR Vorbis OGG case) ~25% reductions! (level1)

The "18 unparsed audio bits" error has also been fixed!

It looks like the error reported as "infinite loop" was most likely a mistake!
Running v1.1 once at level 9 on a 2-hour OGG file took about 20 minutes, if not longer, to compress.

The resulting file sizes, where I compared them, are a few hundred bytes larger for v1.1 than for files compressed by v1.0. (if this was the price of the bug fix, then OK!)

The circumstances of the following case in the first test are a bit confusing:

"128kbps Quality Vorbis OGG:
balrogg-i686.exe -1 e track3.ogg track3.blr
Works! Level 5 to 9 infinite looping compress, no output file. (not finished in below 5min)"

There is such a line in the first TXT: "89 206 180 track3.ogg"

For this file, it did not appear, it finished with it quickly. (both v1.0/v1.1 tested)

Maybe this was "75 731 994 track4.ogg" that problematic file??? I have not managed to reproduce it byte-by-byte so far.

I have not tested the OPUS file now.

[KaktoR]

Looks like --jobs=N does not have any effect? Or how is it supposed to be used?

Edit: Here are some files for which balrogg does not work. They are ogg vorbis but something is off with them. You can check them out if you want.
https://pixeldrain.com/u/TD8wjGrG

Edit: Never mind, I used it wrong. --jobs=N is working.

[KaktoR]

Looks like -5 is the sweet spot here, at least for vorbis set I have.

Input 90.419.200 Bytes
Setting --jobs=6 -b e
-1
Clock Time : 2.907s
Output 81.047.039 Bytes

-5
Clock Time : 25.681s
Output 79.436.915 Bytes

-9
Clock Time : 109.501s
Output 79.430.984 Bytes

[kj911]

Kaktor: Does OGGRE process the problematic files? Have you tested the program on OPUS files (compressed or not)?

A more general question, regarding games...

Since you are the main tester of the forum, you have already tested many games: How common is it in games of the last 20 years (ex: GTA: SA) and are there any examples of it, that OGG or OPUS audio files of 1-2 CDs (or even more) are found? Which is the most glaring case?

_contextmodel: Would it be possible to add streaming/stdio support in the later ones, so that they can be used like, say, OGGRE, via FreeArc? Can the files be unpacked with CLS? (without writing temporary files)

[_contextmodel]

kj911, KaktoR,

I am working on v1.2 (to make the tool a bit faster, work better on Windows, and so on). In principle due to the specifics of how OGG files work, without seekable stream support, they have to be buffered. Perhaps there is a way around this. I have never used FreeArc and I don't know what CLS is.

[KaktoR]

Kaktor: Does OGGRE process the problematic files? Have you tested the program on OPUS files (compressed or not)?

A more general question, regarding games...

Since you are the main tester of the forum, you have already tested many games: How common is it in games of the last 20 years (ex: GTA: SA) and are there any examples of it, that OGG or OPUS audio files of 1-2 CDs (or even more) are found? Which is the most glaring case?





Oggre does not work on them either.

I already tested opus and it works.

I think vorbis is more common than opus.

Test on the attachment. It is opus.

[kj911]

In principle due to the specifics of how OGG files work, without seekable stream support, they have to be buffered. Perhaps there is a way around this. I have never used FreeArc and I don't know what CLS is.

Here in this post (https://fileforums.com/showpost.php?p=483676&postcount=4), there is a small archive that you can use to test in practical conditions how to use OGGRE with FreeArc when repackaging games.

Also study the DiskSpan GUI tool (https://fileforums.com/showthread.php?t=104507), you will find a lot of interesting things.

[KaktoR]

It already works with freearc. However it looks like you have to use solid=0, therefore you cannot use --jobs option I guess.

[External compressor:balrogg]
header = 0
solid = 0
packcmd = balrogg64.exe -1 e $$arcdatafile$$.ogg $$arcdatafile$$.blr
unpackcmd = balrogg64.exe d $$arcdatafile$$.blr $$arcdatafile$$.ogg
datafile = $$arcdatafile$$.ogg
packedfile = $$arcdatafile$$.blr

[_contextmodel]

v1.2 is now attached. It should be markedly faster.




parallel (79.3 MB, 1047 files) newmusic_piano3.ogg (2.5 MB)
tool | e/time | ratio (size) | mem | e/time | ratio (size) | mem
1.3 -1 | 1.61 s | 90.4% (75,256kB) | 15M | 0.38 s | 83.3% (2,221kB) | 6M
-2 | 1.78 s | 89.2% (74,259kB) | 16M | 0.69 s | 79.4% (2,118kB) | 7M
-3 | 1.91 s | 88.9% (74,035kB) | 17M | 0.77 s | 79.2% (2,113kB) | 8M
-4 | 1.99 s | 88.6% (73,743kB) | 17M | 0.82 s | 78.9% (2,106kB) | 8M
-5 | 4.19 s | 88.3% (73,506kB) | 17M | 2.67 s | 78.6% (2,095kB) | 9M
-6 | 6.17 s | 88.2% (73,421kB) | 17M | 4.27 s | 78.3% (2,089kB) | 10M
-7 | 8.12 s | 88.2% (73,417kB) | 17M | 5.84 s | 78.3% (2,089kB) | 11M
-8 | 10.06 s | 88.2% (73,411kB) | 17M | 7.50 s | 78.3% (2,089kB) | 11M
-9 | 14.15 s | 88.2% (73,393kB) | 17M | 10.60 s | 78.3% (2,088kB) | 11M

1.2 -1 | 1.63 s | 90.4% (75,274kB) | 15M | 0.42 s | 83.3% (2,221kB) | 7M
-2 | 1.85 s | 89.2% (74,276kB) | 15M | 0.77 s | 79.4% (2,118kB) | 7M
-3 | 1.97 s | 89.0% (74,052kB) | 17M | 0.82 s | 79.2% (2,113kB) | 8M
-4 | 2.06 s | 88.6% (73,761kB) | 17M | 0.90 s | 78.9% (2,106kB) | 8M
-5 | 4.24 s | 88.3% (73,524kB) | 17M | 2.78 s | 78.6% (2,095kB) | 9M
-6 | 6.31 s | 88.2% (73,438kB) | 17M | 4.55 s | 78.3% (2,089kB) | 10M
-7 | 8.39 s | 88.2% (73,435kB) | 17M | 6.25 s | 78.3% (2,089kB) | 11M
-8 | 10.65 s | 88.2% (73,428kB) | 17M | 8.00 s | 78.3% (2,089kB) | 11M
-9 | 15.11 s | 88.2% (73,428kB) | 17M | 11.73 s | 78.3% (2,089kB) | 11M

1.1 -1 | 0.93 s | 90.4% (75,270kB) | 11M | 0.45 s | 83.3% (2,221kB) | 6M
-2 | 1.30 s | 89.2% (74,272kB) | 11M | 0.84 s | 79.4% (2,118kB) | 7M
-3 | 1.41 s | 88.9% (74,048kB) | 13M | 0.90 s | 79.2% (2,113kB) | 8M
-4 | 1.64 s | 88.6% (73,757kB) | 13M | 0.96 s | 78.9% (2,106kB) | 8M
-5 | 4.39 s | 88.3% (73,520kB) | 13M | 3.01 s | 78.6% (2,095kB) | 10M
-6 | 6.89 s | 88.2% (73,434kB) | 13M | 5.00 s | 78.3% (2,089kB) | 10M
-7 | 9.44 s | 88.2% (73,431kB) | 13M | 6.84 s | 78.3% (2,089kB) | 10M
-8 | 11.92 s | 88.2% (73,424kB) | 13M | 8.56 s | 78.3% (2,089kB) | 10M
-9 | 16.75 s | 88.2% (73,424kB) | 13M | 12.22 s | 78.3% (2,089kB) | 10M

1.0 -1 | 0.92 s | 90.4% (75,238kB) | 37M | 0.43 s | 83.3% (2,221kB) | 12M
-2 | 1.25 s | 89.2% (74,241kB) | 37M | 0.86 s | 79.4% (2,118kB) | 12M
-3 | 1.40 s | 88.9% (74,017kB) | 38M | 0.91 s | 79.2% (2,113kB) | 13M
-4 | 1.63 s | 88.6% (73,726kB) | 38M | 1.00 s | 78.9% (2,105kB) | 13M
-5 | 5.68 s | 88.3% (73,489kB) | 46M | 3.13 s | 78.6% (2,095kB) | 46M
-6 | 9.66 s | 88.2% (73,404kB) | 47M | 5.07 s | 78.3% (2,089kB) | 46M
-7 | 13.30 s | 88.2% (73,401kB) | 47M | 7.00 s | 78.3% (2,089kB) | 46M
-8 | 16.91 s | 88.2% (73,394kB) | 47M | 9.02 s | 78.3% (2,089kB) | 46M
-9 | 24.04 s | 88.2% (73,394kB) | 48M | 13.08 s | 78.3% (2,089kB) | 48M


Mean of 5 on AMD Ryzen 9 5950X. v1.2+: calls fsync() and FlushFileBuffers() for
more durable I/O handling, explaining the minor parallel regression.

[KaktoR]

--jobs=6 -b -5
Input 175.717.750 Bytes

Only encode

v1.0

Process ID : 16572
Thread ID : 15524
Process Exit Code: 0
Thread Exit Code : 0

User Time : 0.031s
Kernel Time : 0.000s
Process Time : 0.031s
Clock Time : 42.996s

Working Set : 4456 KB
Paged Pool : 40 KB
Nonpaged Pool : 11 KB
Pagefile : 4072 KB
Page Fault Count : 1191

IO Read : 0 KB (in 0 reads )
IO Write : 0 KB (in 26 writes)
IO Other : 19 KB (in 857 others)


v1.1

Process ID : 8148
Thread ID : 7800
Process Exit Code: 0
Thread Exit Code : 0

User Time : 0.015s
Kernel Time : 0.000s
Process Time : 0.015s
Clock Time : 44.663s

Working Set : 4460 KB
Paged Pool : 40 KB
Nonpaged Pool : 11 KB
Pagefile : 4076 KB
Page Fault Count : 1193

IO Read : 0 KB (in 0 reads )
IO Write : 0 KB (in 26 writes)
IO Other : 19 KB (in 857 others)


v1.2

Process ID : 14760
Thread ID : 17260
Process Exit Code: 0
Thread Exit Code : 0

User Time : 0.000s
Kernel Time : 0.031s
Process Time : 0.031s
Clock Time : 38.332s

Working Set : 4148 KB
Paged Pool : 40 KB
Nonpaged Pool : 10 KB
Pagefile : 3996 KB
Page Fault Count : 1112

IO Read : 0 KB (in 0 reads )
IO Write : 0 KB (in 0 writes)
IO Other : 19 KB (in 842 others)



Edit: Some more specific clock times. There you see speed increase way better. Approx 7secs faster than before.

v1.0

balrogg: 14 files, 12 jobs

Process ID : 13988
Thread ID : 3280
Process Exit Code: 0
Thread Exit Code : 0

User Time : 0.015s
Kernel Time : 0.000s
Process Time : 0.015s
Clock Time : 37.073s

Working Set : 4068 KB
Paged Pool : 34 KB
Nonpaged Pool : 12 KB
Pagefile : 4028 KB
Page Fault Count : 1064

IO Read : 36 KB (in 1 reads )
IO Write : 0 KB (in 27 writes)
IO Other : 17 KB (in 804 others)

v1.1

balrogg: 14 files, 12 jobs

Process ID : 12796
Thread ID : 11432
Process Exit Code: 0
Thread Exit Code : 0

User Time : 0.015s
Kernel Time : 0.015s
Process Time : 0.030s
Clock Time : 37.827s

Working Set : 4068 KB
Paged Pool : 34 KB
Nonpaged Pool : 11 KB
Pagefile : 4020 KB
Page Fault Count : 1062

IO Read : 81 KB (in 1 reads )
IO Write : 0 KB (in 27 writes)
IO Other : 17 KB (in 804 others)

v1.2

balrogg: 14 files, 12 jobs

Process ID : 14460
Thread ID : 17000
Process Exit Code: 0
Thread Exit Code : 0

User Time : 0.015s
Kernel Time : 0.015s
Process Time : 0.030s
Clock Time : 30.973s

Working Set : 3780 KB
Paged Pool : 32 KB
Nonpaged Pool : 10 KB
Pagefile : 3956 KB
Page Fault Count : 988

IO Read : 87 KB (in 1 reads )
IO Write : 0 KB (in 0 writes)
IO Other : 17 KB (in 792 others)

[kj911]

v1.2: Don't like CMD window forced closing, after execute "balrogg.exe" printing helps or (de)compress.

[FitGirl]

Thanks for a new tool. I've tested it a little. My Opus tests, while showing some result don't make it usable in real repacking - the gain is tiny, no solid mode support (yet?).

Test corpus: Wwise Opus converted to generic opus: 14440 files
Opus: 581718877 bytes
Ballroggs: 567165459 bytes

Since opus files don't have codebooks which could be stored once in a solid mode and where oggre does it's job good, with opus it won't work. And rn since ballrogg doesn't support solid mode, one can't expect big gain on a smaller files concatenated into one stream.

I also tested initial ballrogg v1.0 on vorbis files in single/solid mode. It compressed files fine in single mode, but with the same files concatenated it gave an error.

Will do more tests with fresh builds when I have time.
Unfortunately for repackers, Opus is already a very effective format, with it's range-coder near Shannon limit and space for improvement is really tiny.

[kj911]

FitGirl: Wow! Those OPUS files are so compact! 2.5% size reduction is really not much.
For you, what would be the "sweet pot", in Kaktor's words, where you would use it, for a Repack? 5% or 10% or more?

Doesn't SREP or "lzp:8mb:3:h22:d64mb" give minimal results either? (min. 1-2KB.)

_contextmodel: Do you have a rough idea of ​​what the realistic gain would be if there were a public recompressor for WMV/WMA or MPEG-1/2 videos? (For MPEG2, there is a packer called "oca_mpeg06", but it is too slow.) Can the 2-5-10 or upto MB/s speed be achieved for these cases, as with BinkPack? You don't include H264 in the target list?

[FitGirl]

FitGirl: Wow! Those OPUS files are so compact! 2.5% size reduction is really not much.
For you, what would be the "sweet pot", in Kaktor's words, where you would use it, for a Repack? 5% or 10% or more?
Around 10% with a reasonable MT-speed would be nice. But I doubt it's achievable with Opus. It's REALLY very good lossy format.

As for using it instead of oggre in repacks, will wait until the format matures and gets a stable version. Since the developer is interested in that project, it looks like just a matter of time.

[_contextmodel]

Fitgirl,

Thank you for your response.

> My Opus tests, while showing some result don't make it usable in real repacking - the gain is tiny, no solid mode support (yet?).

The gain is small as Opus files tend to be quite well compressed already, sadly. However, it remains within realm of possibilities, and does happen at a fine-ish performance cost.

> Since opus files don't have codebooks which could be stored once in a solid mode and where oggre does it's job good, with opus it won't work. And rn since ballrogg doesn't support solid mode, one can't expect big gain on a smaller files concatenated into one stream.
> I also tested initial ballrogg v1.0 on vorbis files in single/solid mode. It compressed files fine in single mode, but with the same files concatenated it gave an error.

I will look into implementing this.

> Will do more tests with fresh builds when I have time. Unfortunately for repackers, Opus is already a very effective format, with it's range-coder near Shannon limit and space for improvement is really tiny.

Indeed. I would say that Vorbis repacking is a slightly better selling point.

[_contextmodel]

kj911, Fitgirl:

> _contextmodel: Do you have a rough idea of ​​what the realistic gain would be if there were a public recompressor for WMV/WMA or MPEG-1/2 videos? (For MPEG2, there is a packer called "oca_mpeg06", but it is too slow.) Can the 2-5-10 or upto MB/s speed be achieved for these cases, as with BinkPack? You don't include H264 in the target list?

I will think about it.

> As for using it instead of oggre in repacks, will wait until the format matures and gets a stable version. Since the developer is interested in that project, it looks like just a matter of time.

I will let you know when this happens.

[_contextmodel]

I have released v1.4. Solid mode for Opus files is now supported.

[panker1992]

Around 2 months ago, i started working on an ogg recompression tool myself,

Unfortunately i never finished it to the point that opus is supported because i found bad results and ended up with negative ratio at that time.

I will provide a test here with the capabilities of it.

and i also provided DLLs with header if you want to use this piece of code.

Source Code will be available in the future when i am done with it.


PS. If the author of balrogg can allow i will use it instead of mine .... i think its better in ratio.

[_contextmodel]

Feel free to use my code for any purpose as long as the terms of the license (GNU GPLv3) are followed and authorship is clearly stated/preserved.

[panker1992]

i was under the idea that this code was shared here didn't actually think you are the author.

now that we established that, i did some checking on the sample i provided yesterday.

well done on that 12.5% ratio!!!

[kj911]

balrogg/archiver_ogg_recomp tested from GTASA game "STREAMS" audio files? (Contains many OGGs.

Note: SAAT Tool, losslessly decompress and recreate the whole game archives from prepare better compression. My test in few years ago resulted avg. 1006-1008 million bytes. (Extracted OGGs, compresses it oggre+srep+lolz)