Best Compression Methods for 'Specific' Games. Q&A

[KaktoR]

Originally Posted by dixen

What decompression method for DAT TCU?

https://www.patreon.com/posts/crew-offline-138995143

[KaktoR]

Looks like lzo

I don't know really much about lzo so this could be false positives. Someone else has to check

[JustFun]

Tom Clancy's Splinter Cell: Chaos Theory
Ripped: Foreign Languages, Versus Mode

For Videos use:

xtool:c1792mb:mbik+bcm

For Sounds use:

msc_tak+srep_new:490g+bcm

For rest of the files use:

srep_new:490g+lolz

------------------------------------------------------------------------------------------

Game size: 2.95 GB
Compressed game size: 1.67 GB

[JustFun]

Tom Clancy's Splinter Cell
Ripped: Foreign Languages

For Videos use:

xtool:c1792mb:mbik+bcm

For Sounds use:

msc_tak+srep_new:490g+bcm

For rest of the files use:

srep_new:490g+lolz

------------------------------------------------------------------------------------------

Game size: 1.61 GB
Compressed game size: 952 MB

[JustFun]

Far Cry 3 [Complete, Incl. DLC]

For Sounds use:

oggre+srep_new:490g+bcm

For everything else use:

bpk+xtool:c32mb:mfc3+srep_new:490g+lolz

------------------------------------------------------------------------------------------

Game size: 8.25 GB
Compressed game size: 4.5 GB

[L0v3craft]

Hi guys. I'm trying to compress the game "WUCHANG: Fallen Feathers" v1.6 (179243).

It has two pak files (one is 41.6GB and the other one 14GB).

They are encrypted but not at all seems, because simply using srep+lolz the compression ratio is very good.

So I have used srep+lolz on all game files, compressed from 56.2GB to 18.4GB.

Another repacker (Elamigos) compressed the same game version (I compared the hashes of all game files) but his repack size is 17.5GB. He included also the soundtrack in the compression (so additional 137MB of mp3).

So I opened his archive and he used like me simply srep+lolz:





Now I don't think at all it is about some parameter of lolz or srep, because my repacks and his repacks usually are the same and in some case I compress a little better. In many cases we both compressed only using srep+lolz and no difference in final size.

My srep parameters: srep64 -m3f -a2

My lolz parameters: -d256mb -dtb1 -mtt1 -tt4 -mt10 -mc1023

Someone can help me to understand, please? Thanks.

[KaktoR]

aes encryption is used.

Try xtool ue plugin with key and srep+lolz

[L0v3craft]

Originally Posted by KaktoR

aes encryption is used.

Try xtool ue plugin with key and srep+lolz

I found the password for his archive and I used that password to open it with Freearc. Freearc is showing exactly the method he used (like me srep+lolz). And as confirmation I have installed his repack monitoring the installation from the task manager and XTool is not running, only srep and lolz:



However I tried what you suggested with the plugin "unrealengine_R13"":

XTool v0.8.9 here, only testing the decryption part and it stucks at some point:



I tried also adding zlib+reflate after the key and it finds many more streams but stucks at the same point and anyway as you can see the size is not increasing, maybe the streams found are very small:



this is with XTool v0.7.9, it doesn't stuck but it finds only 13 streams and I don't think this decryption can give some benefit:



I guess the plugin "unrealengine_R13" is not compatible with these pak files format. I tried also extracting them with quickbms and "unreal_tournament_4_0.4.27e_paks_only.bms" but it is not able to decompress all the files...

[JustFun]

@L0v3craft
Check his installed game size to yours, maybe it's lower?

[L0v3craft]

Originally Posted by JustFun

@L0v3craft
Check his installed game size to yours, maybe it's lower?

I have already checked, I have created the checksums file on my installation and verified on his installation, the files are the same and in addition he included 137MB for the soundtrack... so his repack is 1GB or so smaller than my repack.

If you are interested send me a pm for the files... so you can try too. Thanks.

[KaktoR]

I will check this as soon as I have downloaded the game.

[kj911]

L0v3craft: Try it use LDMF and 1(!) threads only mode from LOLZ!

example:

lolz:dt:dtb1:dtw1:dtm1:dto1:dtd1:d256m:fba4096:mc1023:tt16:mt1:ldmf1:ldc0:ldl5  /or tt8 its barely faster/

and really recommend use defeault SREP level/chunks sizes, aka l512 !

Compressible PCM audio founds or not the archives? Video files? (checking via MSC.)

Quckly testing add delta filters before others, checking compression ratio gains or not. /gain yes = delta+srep+lolz/

[L0v3craft]

Originally Posted by kj911

L0v3craft: Try it use LDMF and 1(!) threads only mode from LOLZ!

example:

lolz:dt:dtb1:dtw1:dtm1:dto1:dtd1:d256m:fba4096:mc1023:tt16:mt1:ldmf1:ldc0:ldl5  /or tt8 its barely faster/

and really recommend use defeault SREP level/chunks sizes, aka l512 !

Compressible PCM audio founds or not the archives? Video files? (checking via MSC.)

Quckly testing add delta filters before others, checking compression ratio gains or not. /gain yes = delta+srep+lolz/

I will test those lolz parameters this evening/tonight. Thanks.

Ok I need to add -l512 to my srep parameters, because I use srep 3.93 and -l is zero by default.

msc_tak works , infact it compressed from 55.7GB to 54.9GB.

But I tested srep+4x4/lzma -> 20.3GB and after adding msc_tak before srep and the final size is smaller only of 100MB.

I never tested "delta" before than srep+lolz. I can test it with 4x4/lzma like srep+delta+4x4/lzma. But again Freearc is showing that Elamigos used only srep+lolz, I don't see "delta" in his method.

[kj911]

Use the delta algorithm at the beginning of the chain. It may improve the ratio, it may not.

Use 3.92 from SREP! (The default value of the "l/c" parameters is 512, if not defined separately.) You can play a little more with the a0/a1/a2 parameters, although it does not multiply much. (v3.93 contains decompression bug.)

Did you run MSC with only WAV=1? With the RAW=1 option, how many audio streams does it find? True, unpacking can be a little more problematic this way, but usually it works. What can be a problem if there is no error during packing, but when unpacking it does, because problematic RAW/PCM audio files can appear that run with errors.

It is interesting to note that the small test results show that with MSC there is only an extra 100MB gain at the end of the compression chain, after the initial 800MB size reduction.

In the case of LOLZ, in the case of compression on 1 thread as described above, be prepared for it to take up to ~2-3 days! It might only bring ~5%, but it might be more, since you've run it on 10 threads so far, it might even be a bigger profit.

ps: Hint, try use DLZ replacement from LZMA, from faster compression times testing than LOLZ. Use 256-512MB dictionary.

My preferred switches: -slp -d512m -lp0 -fb337 -mc8192 -cu3 -cm43

[L0v3craft]

Originally Posted by kj911

Use 3.92 from SREP! (The default value of the "l/c" parameters is 512, if not defined separately.) You can play a little more with the a0/a1/a2 parameters, although it does not multiply much. (v3.93 contains decompression bug.)

I can't believe it... srep 3.2 is the solution.

srep(3.93)+4x4/lzma -> 20.3GB

srep(3.2)+4x4/lzma -> 19.4GB

but in decompression srep 3.2 doesn't work for me, so I tried srep 3.93 in decompression and works.