Best Compression Methods for 'Specific' Games. Q&A

[ZakirAhmad]

Originally Posted by doofoo24

using srep with setting srep:m2f:l128:a1 reduce alot of memory decompression with minimum compression ratio hit as long used with lolz, the setting that i used is lolz:mc1023:fba4096:bc8:blr8:bm8h14c4:mtt1:d12 8m:mt8 (change mt depend on cpu core)...
i test mad max pzlib:max:c256m+srep:m2f:l128:a1+lolz:mc1023:fba40 96:bc8:blr8:bm8h14c4:mtt1:mt8:d128m.
i get 34,354,331,688 => 4,266,484,514 bytes. Ratio 12.42%

*srep with m3f need 21g of ram for decompression with m2f:l128:a1 it reduce to 2696mb...
the difference ratio in the end between m3f and m2f 250mb but a lot less memory usage in decompresion and much faster...
total installation time 6 min

can u please post ur srep which u used for compression.
thanks.

[doofoo24]

srep 3.93a beta srep:m2f:l128:a1 but you need about 18600mb of ram to compress if you have 16gb use l256

[ZakirAhmad]

Originally Posted by doofoo24

srep 3.93a beta srep:m2f:l128:a1 but you need about 18600mb of ram to compress if you have 16gb use l256

Thanks .
can u tell me how to launch compressors from setup. like u posted in above screen shot of setup in task manager.

[ZakirAhmad]

Originally Posted by doofoo24

using srep with setting srep:m2f:l128:a1 reduce alot of memory decompression with minimum compression ratio hit as long used with lolz, the setting that i used is lolz:mc1023:fba4096:bc8:blr8:bm8:oh14:pc4:mtt1:d12 8m:mt8 (change mt depend on cpu core)...
i test mad max pzlib:max:c256m+srep:m2f:l128:a1+lolz:mc1023:fba40 96:bc8:blr8:bm8:oh14:pc4:mtt1:mt8:d128m.
i get 34,354,331,688 => 4,266,484,514 bytes. Ratio 12.42%

as in the picture lolz need 261mb per thread no need for more than 2 thread to use with pzlib becase pzlib is the bottleneck...

*srep with m3f need 21g of ram for decompression with m2f:l128:a1 it reduce to 2696mb...
the difference ratio in the end between m3f and m2f 250mb but a lot less memory usage in decompresion and much faster...
total installation time 6 min

how did u make compressors like ztool lolz group with setup in above taskmanager screenshot.

[doofoo24]

is like m3f:l512 try m2f:l512:a1 it shoud be less than 8gb but the ratio you may need to increase lolz to d256m but make sure to be mt2 if you don't have enough ram i used lolz wit d128:mt8 it fast but need around 12gb or ram in compression...

[ZakirAhmad]

Compressed Mad Max Fury Road with ztool srep lolz to 3.90 from 32gb.<br>
The srep dictionary size is only 2200mb. I used settings srep:m2f:t1:l512:c512.<br>
srep compressed it to 9.9gb then lolz to 3.90gb. <br>
The only problem with doofoo settings is that he used multithreaded srep because srep with m1 and m2 uses multithreading hence reduction in co.pression ration. He also used l32 and c 32 whre as i used l512 and c 512 because my ram is only 4gb otherwise i might have gained may be better than this. <br>
The whole point is to limit creation of temp file which interupts fast installation.<br>
<br>
How can we achieve fast installation?<br>
The things i recommend is <br>
1. If useing prexompressor with multithreaded support, do not set processor useage to 100% because their r other tools like lolz srep etc in compression chain which also use cpu. If u set precompressor(1st tool in chain) usage to 100%,, it will restrict other compressors like srep and lolz to use cpu in parallel and hence the compressors will run one after another not parallel.<br>
#2. If a game is heavely compressed with srep like in mad max use m2f setting instead of m3f or m5f. It will in most cases restrict decompression dectionary to managable size.<br>
#3 useing multithreading for lolz when using precompressor is of no use in most cases i guess because if precompressor cant process much, it will automatically restrict lolz data processing or lazma etc.<br>
<br>
#4 use latest srep by profrager. And set memory usage in cls as 100%- 1250m. Why 1250m?, i observed if ur win 10 pc is idle, it uses only (1250mb ) of ram (at least in my case), what if we use 100% memory usage or 100% - 256m? <br>
I would not recommand doing this. U known in win 10 their r two ways to free RAM when it senses RAM is full, one is to use page file and another is to use compression and then page file. So when u set ram usage to 100% and if srep needs ur 100% of ram, it will first fill ur 100% ram and then will create virtual temp file file. But in prder to iuse 100% ram windows has to move own components to page file and or compress, this will increse instillation time exponentially and will make using pc hell impossible!<br>
However I have read that srep uses memory from free pool but i havent foumd doing it that. It just uses full ram if I do not use it that way.<br>
Also m5f -ia- swotches of srep promises to ise less ram without the cost of speed, so if any one has got fast computer please compress madmax with m5f -ia- switch and report decompression memory isage by srep back here. Use srep 3.93 because it provides decompression ram usage after processing the file.<br>
One more thing, please do press thanks, if u find this helpfull.<br>
<br>
<br>

[Simorq]

Mad Max
pzlib+srep:m3f:l1024+lolz:d1024:al1:mc1023:tt16 (3.95 GB)

[78372]

LIMBO
Original Size = 95MB
1. Method = pzlib+srep:m3f+rz:128m
2. Method = pzlib+srep:m3f+lolz(default)
1.Final Size = 50MB
2. Final Size = 55.1MB
pZlib inflated the game to 567MB, then srep made it 167MB, then rz made it 50MB at last/lolz made it 55.1MB at last.

While lolz was significantly slow for my potato pc(157KB/S), rz gave me more speed(267KB/S), extraction speed was almost same because I extracted on a regular hdd and probably lolz bottlenecked with it.

[doofoo24]

Originally Posted by ZakirAhmad

Compressed Mad Max Fury Road with ztool srep lolz to 3.90 from 32gb.<br>
The srep dictionary size is only 2200mb. I used settings srep:m2f:t1:l512:c512.<br>
srep compressed it to 9.9gb then lolz to 3.90gb. <br>
The only problem with doofoo settings is that he used multithreaded srep because srep with m1 and m2 uses multithreading hence reduction in co.pression ration. He also used l32 and c 32 whre as i used l512 and c 512 because my ram is only 4gb otherwise i might have gained may be better than this. <br>
The whole point is to limit creation of temp file which interupts fast installation.<br>
<br>
How can we achieve fast installation?<br>
The things i recommend is <br>
1. If useing prexompressor with multithreaded support, do not set processor useage to 100% because their r other tools like lolz srep etc in compression chain which also use cpu. If u set precompressor(1st tool in chain) usage to 100%,, it will restrict other compressors like srep and lolz to use cpu in parallel and hence the compressors will run one after another not parallel.<br>
#2. If a game is heavely compressed with srep like in mad max use m2f setting instead of m3f or m5f. It will in most cases restrict decompression dectionary to managable size.<br>
#3 useing multithreading for lolz when using precompressor is of no use in most cases i guess because if precompressor cant process much, it will automatically restrict lolz data processing or lazma etc.<br>
<br>
#4 use latest srep by profrager. And set memory usage in cls as 100%- 1250m. Why 1250m?, i observed if ur win 10 pc is idle, it uses only (1250mb ) of ram (at least in my case), what if we use 100% memory usage or 100% - 256m? <br>
I would not recommand doing this. U known in win 10 their r two ways to free RAM when it senses RAM is full, one is to use page file and another is to use compression and then page file. So when u set ram usage to 100% and if srep needs ur 100% of ram, it will first fill ur 100% ram and then will create virtual temp file file. But in prder to iuse 100% ram windows has to move own components to page file and or compress, this will increse instillation time exponentially and will make using pc hell impossible!<br>
However I have read that srep uses memory from free pool but i havent foumd doing it that. It just uses full ram if I do not use it that way.<br>
Also m5f -ia- swotches of srep promises to ise less ram without the cost of speed, so if any one has got fast computer please compress madmax with m5f -ia- switch and report decompression memory isage by srep back here. Use srep 3.93 because it provides decompression ram usage after processing the file.<br>
One more thing, please do press thanks, if u find this helpfull.<br>
<br>
<br>

*srep 3.93 when you use m3f:l512:c256 always fail in decompression...
also i found that using m3f is the best for final results after applying lzma/lolz better than m5f...
*you will get better results with m5f alone but seems to me that lzma or lolz mange to compress better srep:m3f over m5f...

[78372]

INSIDE

Original Size = 2.5G

Method = msc:tak+srep:m3f+rz:128m

Final Size = 930MB

[doofoo24]

Originally Posted by 78372

INSIDE

Original Size = 2.5G

isn't INSIDE 2.15G english only ?

[78372]

Probably 2.19G, my mistake

[ZakirAhmad]

Originally Posted by 78372

Probably 2.19G, my mistake

fitgirl repacked it from 2.3gb to 939mb. if urs is only 2.19g then lolz is winner.

[doofoo24]

@78372 did you try dispack070+delta ?
when i try dispack070+delta with lzma work but with lolz does not seem to work, i am using fazip but it did'nt run ?
with msc+srep:m3f:a2+dispack070+delta+lzma:a1:mfbt4:d51 2m:fb273:mc10000:lc8 you will get 908mb...
with msc+srep:m3f:a2+lolz 864mb...

[78372]

Originally Posted by ZakirAhmad

fitgirl repacked it from 2.3gb to 939mb. if urs is only 2.19g then lolz is winner.

My repack is based on her repack, I actually don't remember the actual size, not on pc now