FileForums - Best Compression Methods for 'Specific' Games. Q&A

FileForums (https://fileforums.com/index.php)

- Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)

- - Best Compression Methods for 'Specific' Games. Q&A (https://fileforums.com/showthread.php?t=99554)

Quote:

Originally Posted by KaktoR (Post 498312)

What is NBA2k23 using? Can't find any samples yet and don't want to download 122gb just for testing.

Switch version uses multiple types of Oodle family, but can't comment on PC version. Sorry.

Quote:

Originally Posted by KaktoR (Post 498312)

What is NBA2k23 using? Can't find any samples yet and don't want to download 122gb just for testing.

Zlib

My test from BMP files portion the Space Empires V gamez:

This x86 ONLY compressors, its wins from ratio: MSC+DLZ
2nd place: MSC+BMP or MSC+GRZIP with any alterations.

This x64 ONLY or x86/x64 compressors, its wins from ratio: LOLZ
2nd place: MSC+LOLZ. Results ~200kB's smaller than MSC+DLZ combo.

Results:

Code:

This is For [BMP] files:



-----------------------------------------------------------------------------------------------------

Compression Method - 'msc'



FreeArc 0.67 (March 15 2014) testing archive: bmp_test0.arc

Tested 164 files, 17,164,036 => 83,606,836 bytes. Ratio 20.53%

Testing time: cpu 109.84 sec/real 111.33 sec = 99%. Speed 0.75 mB/s

All OK

-----------------------------------------------------------------------------------------------------

Compression Method - 'bmp'



FreeArc 0.67 (March 15 2014) testing archive: bmp_test1.arc

Tested 164 files, 14,300,345 => 83,606,836 bytes. Ratio 17.10%

Testing time: cpu 17.81 sec/real 19.20 sec = 93%. Speed 4.35 mB/s

All OK

-----------------------------------------------------------------------------------------------------

Compression Method - 'msc+grzip:8388096b:m1:l2048:h15:a'



FreeArc 0.67 (March 15 2014) testing archive: bmp_test2.arc

Tested 164 files, 12,463,714 => 83,606,836 bytes. Ratio 14.91%

Testing time: cpu 129.11 sec/real 130.56 sec = 99%. Speed 0.64 mB/s

All OK

-----------------------------------------------------------------------------------------------------

Compression Method - 'msc+lzma'



FreeArc 0.67 (March 15 2014) testing archive: bmp_test3.arc

Tested 164 files, 12,661,353 => 83,606,836 bytes. Ratio 15.14%

Testing time: cpu 126.25 sec/real 128.22 sec = 98%. Speed 0.65 mB/s

All OK

-----------------------------------------------------------------------------------------------------

Compression Method - 'msc+lzma:a1:mfbt4:d72mb:fb273:lc8:mc1024'



FreeArc 0.67 (March 15 2014) testing archive: bmp_test3-2.arc

Tested 164 files, 12,607,076 => 83,606,836 bytes. Ratio 15.08%

Testing time: cpu 126.55 sec/real 127.92 sec = 99%. Speed 0.65 mB/s

All OK

-----------------------------------------------------------------------------------------------------

Compression Method - 'msc+lzp:64mb:3:h22:d128mb'



FreeArc 0.67 (March 15 2014) testing archive: bmp_test4.arc

Tested 164 files, 15,073,522 => 83,606,836 bytes. Ratio 18.03%

Testing time: cpu 124.94 sec/real 126.44 sec = 99%. Speed 0.66 mB/s

All OK

-----------------------------------------------------------------------------------------------------

Compression Method - 'msc+grzip:8388096b:m1:l2048:h15:a+lzp:64mb:3:h22:d128mb'



FreeArc 0.67 (March 15 2014) testing archive: bmp_test5.arc

Tested 164 files, 12,462,437 => 83,606,836 bytes. Ratio 14.91%

Testing time: cpu 131.30 sec/real 135.17 sec = 97%. Speed 0.62 mB/s

All OK

-----------------------------------------------------------------------------------------------------

Compression Method - 'msc+dlz:slp:d64m:lc8:lp0:fb337:mc8192'



FreeArc 0.67 (March 15 2014) testing archive: bmp_test6.arc

Tested 164 files, 12,410,208 => 83,606,836 bytes. Ratio 14.84%

Testing time: cpu 114.17 sec/real 115.78 sec = 99%. Speed 0.72 mB/s

All OK

-----------------------------------------------------------------------------------------------------

Compression Method - 'msc+grzip:8388096b:m1:l2048:h15:a+dlz:slp:d64m:lc8:lp0:fb337:mc8192'



FreeArc 0.67 (March 15 2014) testing archive: bmp_test7.arc

Tested 164 files, 12,464,540 => 83,606,836 bytes. Ratio 14.91%

Testing time: cpu 122.03 sec/real 123.52 sec = 99%. Speed 0.68 mB/s

All OK

-----------------------------------------------------------------------------------------------------

Compression Method - 'msc+lolz:dt:dtb1:dtw1:dtm1:dto1:dtd1:mt4:d160m:fba4096:tt16:mc1023'



FreeArc 0.67 (March 15 2014) testing archive: bmp_test8.arc

Tested 181 files, 12,309,746 => 83,606,836 bytes. Ratio 14.72%

Testing time: cpu 115.23 sec/real 117.13 sec = 98%. Speed 0.71 mB/s

All OK

-----------------------------------------------------------------------------------------------------

Compression Method - 'lolz:dt:dtb1:dtw1:dtm1:dto1:dtd1:mt4:d160m:fba4096:tt16:mc1023'



FreeArc 0.67 (March 15 2014) testing archive: bmp_test9.arc

Tested 181 files, 11,111,413 => 83,606,836 bytes. Ratio 13.29%

Testing time: cpu 0.16 sec/real 3.45 sec = 5%. Speed 24.21 mB/s

All OK

-----------------------------------------------------------------------------------------------------

Compression Method - 'lzma'



FreeArc 0.67 (March 15 2014) testing archive: bmp_test10.arc

Tested 164 files, 14,359,017 => 83,606,836 bytes. Ratio 17.17%

Testing time: cpu 2.39 sec/real 2.47 sec = 97%. Speed 33.87 mB/s

All OK

-----------------------------------------------------------------------------------------------------

Compression Method - 'lzma:a:72mb:normal:bt4:273:mc1024:lc8'



FreeArc 0.67 (March 15 2014) testing archive: bmp_test11.arc

Tested 164 files, 13,281,704 => 83,606,836 bytes. Ratio 15.89%

Testing time: cpu 2.41 sec/real 2.67 sec = 90%. Speed 31.29 mB/s

All OK

-----------------------------------------------------------------------------------------------------



MSC OutPut Console Window:

<--------------------------- B M P ---------------------------->

Total founded BMP: 142

BMP with not supported header value "bit_per_pixel": 0

BMP with not supported line size (should be line_size_bytes mod 4 = 0): 6

BMP filtered by size: 0



processed BMP: 136 (77'310'176 bytes)

Quote:

Originally Posted by kj911 (Post 498322)

Can you use this compressor?

Code:

[External compressor:bliz]

cmem      = 600

dmem      = 600

default   = 100

packcmd   = "Blizzard\bliz.exe" c $$arcdatafile$$.tmp $$arcpackedfile$$.tmp {option}000000

unpackcmd = "Blizzard\bliz.exe" d $$arcpackedfile$$.tmp $$arcdatafile$$.tmp

Lord.Freddy: Nothing. I haven't met him yet, the Blizzard compressor. Testing now in next hours/days.

DLZ = (D)ds modeled (LZ)ma v0.2.2 by profrager

Quick tests: 2* Faster than LZMA and comparable ratio.

Code:

Tested 164 files, 13,760,715 => 83,606,836 bytes. Ratio 16.46%

Testing time: cpu 0.16 sec/real 29.28 sec = 1%. Speed 2.86 mB/s

Tested again, packed all files in one ARC file, in store mode and packing in Blizzard. Small, default blocksize its better!?

Code:

bliz.exe c bmp_store.arc bmp_.blz 300000000  ---> 13 762 777 byte

bliz.exe f bmp_store.arc bmp_.blz 300000000  ---> 13 963 348 byte

bliz.exe f bmp_store.arc bmp_.blz  ---> 13 604 705 byte

bliz.exe c bmp_store.arc bmp_.blz ---> 13 396 429byte

Quick note: Best results get in use 8MB blocksize (smaller/bigger, worse results): 13 369 598 byte

While we on this discusion right now, what's a good method for text files btw? Such as txt, xml, ini, etc.

Quote:

Originally Posted by KaktoR (Post 498327)

While we on this discusion right now, what's a good method for text files btw? Such as txt, xml, ini, etc.

Code:

----‐--------‐-------------‐--------‐-------------‐--------‐---------

1) ' nanozip:cc ' => Ultra

2) ' bsc ' => maximum

3) ' dict:p+lzp+ppmd:20:500mb:r1 ' => High

----‐--------‐-------------‐--------‐-------------‐--------‐---------

Good evening, I've extracted a .archive file and this is composed of several .bin files. Which is the best method to compress these .bin files? I've tried with precomp v0.4.7, then with xzlib, with no compression results at all. Is there a specific compressor or a mix of them? Thank you!

You should know what these bin files include.

The Death

srep+lolz

7.4GB ----> 2.5GB

Quote:

Originally Posted by KaktoR (Post 498331)

You should know what these bin files include.

Thank you for your reply! The .archive file i've extracted is a file from Cyberpunk 2077 and it's a language file, so I think .bin files inside are audio file or sound file. Do you think I have to use MSC or similar tools to compress them?

Quote:

Originally Posted by Dragonis40 (Post 498344)

It's WWise Vorbis

Quote:

Originally Posted by Masquerade (Post 498346)

It's WWise Vorbis

You mean I should compress .bin files with "How to correctly process WWise Vorbis Audio | Guide + WemTool" guide?

1GB's BMP compression test:

Code:

Original archive: bmp.arc (244,937,914 byte)

Methods: lzma:192mb:normal:bt4:128:lc8

Packed from grzip (with -mgrzip:256mb:h15:l2048:a):

Code:

5,483 files, 1,069,598,818 bytes, 320,784,780 compressed (not solid!)

tempfile+grzip..: 1,069,598,818 bytes, 321,101,123 compressed (solid!)

Packed from DLZ: 239,147,583 byte, Methods: -slp -d64m -lc8 -lp0 -fb337 -mc8192

Packed from bliz:

Code:

100m bytes dict. size: 258,506,104 byte

with 4096kB Dict size: 270,019,193 byte

tempfile+mm+bliz: 279,418,278 byte

mm+tempfile+bliz: 289,750,196 byte (not solid archive!)

Packed from MSC (BMF 2.0.1 with slow mode, bmf=9s) VERYSLOW:

Code:

Compressed 5,483 files, 1,069,598,818 => 340,840,020 bytes. Ratio 31.87%

Compression time: cpu 108.97 sec/real 17445.55 sec = 1%. Speed 0.06 mB/s

BMF Results:

Code:

<--------------------------- B M P ---------------------------->

Total founded BMP: 4549

BMP with not supported header value "bit_per_pixel": 0

BMP with not supported line size (should be line_size_bytes mod 4 = 0): 132

BMP filtered by size: 0



processed BMP: 4417 (919'082'710 bytes)

Packed again the MSC/BMF-packed previous archive:

Code:

7z/LZMA: 232 253 503 byte

DLZ only: 228 604 204 byte

Note: DLZ its finished in ~41 mins from compression. BLIZ its faster.

and any compressors:

RAZOR 1.03.7 x64 with 512m Dict. size: 195 443 238 byte

LOLZ v22c4b (Brute detect disabled!):

Code:

default lolz settings (ldmf1, 64MB dict ...): 187,784,213 byte

lolz:dt:mtt1:mt4:d160m:fba4096:tt16:mc1023 -> 187,109,257 byte

All compression tests, except RZ/LOLZ use only from very old x86 PC with 1GB RAM. RZ/LOLZ used x64 laptop with 8GB RAM.

I belive the *.ggpack* archives in Return To Monkey Island are XOR encrypted. None of the current keys from this list appear to be correct.