FileForums

FileForums (https://fileforums.com/index.php)
-   Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)
-   -   Best Compression Methods for 'Specific' Games. Q&A (https://fileforums.com/showthread.php?t=99554)

KaktoR 07-04-2025 08:26

Good to see someone else steps in and give such detailed results. I like it, especially the parts where you dig deeper into such formats such as extracting audio files.

Keep it up.

KaktoR 09-04-2025 10:56

1 Attachment(s)
Age of Mythology Retold
They switched to lz4. However not much benefit because most of the data seems to be not compressed at all or very weak compression.

Code:

[Stream1]
Name=lz4hc
Codec=lz4hc:l9
BigEndian=0
Signature=0x347A6C61
Structure=Signature(4),DSize(4),CSize(4),Unk(4),Stream
StreamOffset=0
CompressedSize=CSize
DecompressedSize=DSize


JustFun 10-04-2025 07:48

25 to Life - (Retail)

Code:

Selected ARC/DS method for Data1a-01.bin was:
xtool:mpreflate+srep_new:490g+lolz
------------------------------------------------------------------------------------------
Overall input size: 1,43 GB
Overall output size: 589,62 MB (Ratio 40.27%)


JustFun 11-04-2025 08:21

Tom Clancy's Ghost Recon (2001) - (Retail)

Code:

Selected ARC/DS method for Data1a-01.bin was:
xtool:mpreflate+msc_tak+srep_new:490g+lolz
------------------------------------------------------------------------------------------
Overall input size: 0,93 GB
Compressed game size: 334,62 MB (Ratio 35.20%)


IRAQIGHOST 11-04-2025 08:24

Call of Duty Advanced Warfare

Game Size: 44GB
Repack Size: 29.9GB

.ff files
Quote:

xt_lz4+srep_new:393a+LOLZ_NORMAL
.pak files (Images)
Quote:

xt_lz4+srep_new:393a+LOLZ_NORMAL
.pak files (Sound)
Quote:

xt_lz4+msc_tak
Rest files
Quote:

srep_new:393a+4x4:b96mb:lzma:ultra:bt4:fb273:lc8:m c1000000:96mb

KaktoR 11-04-2025 10:49

Why not combine ff and pak files if you use the same method on them? Pretty sure you missing some duplicates with srep.

IRAQIGHOST 11-04-2025 11:01

Quote:

Originally Posted by KaktoR (Post 507243)
Why not combine ff and pak files if you use the same method on them? Pretty sure you missing some duplicates with srep.

Just to clarify between files (Images) and (Sounds)

L33THAK0R 16-04-2025 08:10

Rain World
 
GOG Release - Build ID 58500446910738020

Original Size: 5.07 GiB
Final Size: 2.72 GiB (Incl. installer)
Ripped Files: None.
Tools used:

Xtool v0.8.9 (Codecs: png, reflate, unity, zlib)
Srep
Lolz
Msc
Oggre

Additional notes:

MSC-related streams are decompressed post-install, due to CRC errors when decompressing. To minimise the size of patch files for the relevant containers, organised and efficient sorting of loose files is recommended, a similar approach was taken for my "Seasons after Fall" compression, more details can be found in my write-up for the title.

Quite happy with how this one turned out, given it's a respectable 12.25% smaller than the currently smallest published results. Despite making the post-install process quite fast, the downside is that the relevant containers are effectively written to disk 3 times (~approx. 5.5 GiB); once to build the oldFile, once for patching to the decompressed newFile, and once to restore the decompressed newFile to the original container. I wanted it to be CRC-perfect, but if additional DLC is released and I have to update my results, I'll experiment with decompressing the containers, as I've had success in the past with having titles accept decompressed *.bundle containers.

KaktoR 17-04-2025 09:39

Terminator Dark Fate Defiance
Update 1.06.1107

Code:

16:46:48 - Selected ARC/DS method for Data1a-01.bin was: oggre
 16:46:48 - Selected ARC/DS method for Data1b-01.bin was: xtool:mflac,l8
 16:46:48 - Selected ARC/DS method for Data1c-01.bin was: xtool:c128mb:mreflate:mpng:d1:dd3:l10,d128mb
 16:46:48 - Selected ARC/DS method for Data1d-01.bin was: xtool:dd3:l10,d128mb
 16:46:48 - Selected ARC/DS method for Data1e-01.bin was: 1
--------------------------------------------------------------------------
 17:36:23 - Overall input size: 35.26 GB
 17:36:23 - Overall output size: 13.45 GB (Ratio 38.15%)
 17:36:23 - Overall conversion time: 00:46:07

Just a fast compression.
You need to exclude "basis.pak\sounds\dialogs\10_mission\10th_sintu_la st_mission_1_1.wav" from xtool:mflac (or msc) otherwise you will get crc error on this paticular file.

password
Code:

oKoo$]bnGTKJLMNBA9A
Recreate paks with
Code:

7z.exe a -mx5 -tzip -aoa -r -poKoo$]bnGTKJLMNBA9A
The recreated files are not crc perfect but the game works fine nevertheless.

KaktoR 17-04-2025 12:59

Tempest Rising

Code:

0x9A6DF7547946158E6A6210A26F3F81BB7BF33A513BB369A38E41287249246577

Danziel123 21-04-2025 22:37

hello everyone, please ask. what is the right compress method for the game fatal fury city of the wolves? i can't check because the file is encrypted & here is a sample file:
https://drive.google.com/file/d/1dek...ew?usp=sharing
Thank you:)

KaktoR 22-04-2025 07:35

Quote:

Originally Posted by Danziel123 (Post 507373)
hello everyone, please ask. what is the right compress method for the game fatal fury city of the wolves? i can't check because the file is encrypted & here is a sample file:
https://drive.google.com/file/d/1dek...ew?usp=sharing
Thank you:)

Code:

0x2732554A3AE54CFCB7BDCE9A635E756CB9BB4D58B1B69585AF2EEAC75AD9E0CB
+ kraken

Danziel123 22-04-2025 18:39

Quote:

Originally Posted by KaktoR (Post 507378)
Code:

0x2732554A3AE54CFCB7BDCE9A635E756CB9BB4D58B1B69585AF2EEAC75AD9E0CB
+ kraken

Thank you for the advice. Allow me to ask, how to know if a game uses oodle kraken, mermaid or others if the file is encrypted?

dixen 23-04-2025 03:36

Quote:

Originally Posted by Danziel123 (Post 507385)
Thank you for the advice. Allow me to ask, how to know if a game uses oodle kraken, mermaid or others if the file is encrypted?

Check **shipping.exe with AES finder, then Xtool
-mue4(5):k(key)+kraken

KaktoR 23-04-2025 07:02

Simply use xtoolui, make settings and go. Enable "Skip verification" to speed up the process, then you know what method to use. After this find out correct library, check "Stop after # streams" checkbox (personally I use 25-50).

KaktoR 23-04-2025 08:37

TESIV Oblivion Remastered

mermaid+kraken pak,ucas

mp3
Code:

OblivionRemastered\Content\Dev\ObvData\Data\Oblivion - Voices1.bsa
OblivionRemastered\Content\Dev\ObvData\Data\Oblivion - Voices2.bsa
OblivionRemastered\Content\Dev\ObvData\Data\DLCShiveringIsles - Voices.bsa

msc
Code:

OblivionRemastered\Content\Dev\ObvData\Data\Oblivion - Sounds.bsa
OblivionRemastered\Content\Dev\ObvData\Data\DLCShiveringIsles - Sounds.bsa


Danziel123 23-04-2025 18:39

Quote:

Originally Posted by dixen (Post 507391)
Check **shipping.exe with AES finder, then Xtool
-mue4(5):k(key)+kraken

Thank you
Quote:

Originally Posted by KaktoR (Post 507393)
Simply use xtoolui, make settings and go. Enable "Skip verification" to speed up the process, then you know what method to use. After this find out correct library, check "Stop after # streams" checkbox (personally I use 25-50).

Sorry if I ask again, is the meaning of “make settings and go” as dixen said?

KaktoR 23-04-2025 22:48

Yes
https://i.imgur.com/tWFxdTN.png

alqawasmy 24-04-2025 08:34

Quote:

Originally Posted by JustFun (Post 469251)
7 Sins - 790 MB to 386 MB

msc+srep+lzma2 on sound files
pzlib+srep+lolz on everything else

thank you

JustFun 24-04-2025 20:55

Quote:

Originally Posted by alqawasmy (Post 507421)
thank you

BTW, there's a bit better and easier method shared as well.
Check the Index

st0rm 26-04-2025 03:40

Condemned : Criminal Origins [2006]
 
Code:

Game Version : 1.0

Ripped - Re-encoded Files : Nothing

Original Size : 6.23 GB

Compressed Size : 2.99 GB

Install Time : 5 Mins on HDD (Depends your system)

Method : srep:m3f+lzma2


KaktoR 26-04-2025 07:57

@st0rm
There are wav and bik files in arch00 files.

Code:

Streams: 5858 / 5858
Time: 00:06:07 (CPU 00:24:48)
Duplicates: 260 (24.1 MB) [33.0 MB >> 15.7 MB]
Srep decompression memory: 204 MB

Size: 6.22 GB >> 5.27 GB >> 5.26 GB >> 4.46 GB >> 2.51 GB

Done!!!


JustFun 29-04-2025 01:11

Prince of Persia: The Two Thrones - (Retail)

Code:

Selected ARC/DS method for Data1a-01.bin was:
xtool:c1792mb:mbik+xtool:mzlib+srep_new:490g+lolz
------------------------------------------------------------------------------------------
Overall input size: 1,21 GB
Compressed game size: 786,55 MB (Ratio 63.59%)


JustFun 29-04-2025 01:40

Agatha Christie - Murder on the Orient Express (2006) - (Retail)

Code:

Selected ARC/DS method for Data1a-01.bin was:
xtool:c1792mb:mbik+xtool:mpreflate+msc_tak+srep_new:490g+lolz
------------------------------------------------------------------------------------------
Overall input size: 751,74 MB
Compressed game size: 550,75 MB (Ratio 73.26%)


JustFun 29-04-2025 01:53

Hitman: Codename 47

Code:

Selected ARC/DS method for Data1a-01.bin was:
xtool:c1792mb:mbik+xtool:mpreflate+msc_tak+srep_new:490g+lolz
------------------------------------------------------------------------------------------
Overall input size: 266,94 MB
Compressed game size: 130,93 MB (Ratio 49.05%)


JustFun 29-04-2025 02:14

Hitman: Contracts

Code:

Selected ARC/DS method for Data1a-01.bin was:
xtool:c1792mb:mbik+xtool:mpreflate+msc_tak+srep_new:490g+lolz
------------------------------------------------------------------------------------------
Overall input size: 859,45 MB
Compressed game size: 580,22 MB (Ratio 67.51%)


JustFun 29-04-2025 03:20

Hitman 2: Silent Assassin

Code:

Selected ARC/DS method for Data1a-01.bin was:
xtool:c1792mb:mbik+xtool:mpreflate+msc_tak+srep_new:490g+lolz
------------------------------------------------------------------------------------------
Overall input size: 492,81 MB
Compressed game size: 182,00 MB (Ratio 36.93%)


JustFun 29-04-2025 05:04

The Saboteur - (GOG Version, English Only)

Code:

Selected ARC/DS method for Data1a-01.bin was:
xtool:c1792mb:mbik+xtool:mpreflate+srep_new:490g+lolz
------------------------------------------------------------------------------------------
Overall input size: 5,86 GB
Compressed game size: 2,32 GB (Ratio 39.56%)


trillestdwane 29-04-2025 14:27

Crysis 3 (non remastered)
 
I am having a bit of trouble trying to compress this game. When I use the Cry Engine plugin (specifically for Crysis 3) i end up getting a "runtime error 216" message box popping up at 10.2%, then

Code:

ERROR: general (de)compression error in xtool:c32mb:mc 10.2%
I still get the same error when I only do it on the games .pak files. :confused:

wareck 30-04-2025 01:09

Day Gones Broken Road
 
I use DiskspanGUI, and I done this :
-go in ADDON diskspangu folder, unzip the special oodle library oo2core_7_win64.dll and oo2core_8_win64.dll in the oodle library folder root

-in the game folder, removed language I don"t use (only big .pak files)

Data1:
Name Data3.bin
xtool:mkraken:7:mmermaid:8:mreflate+srep_new:490g+ LOLZ_ULTRA => NOTICE
(only one file, it's just to let diskspangui to copy dll in xtool folder and use them for the next step)

Data2
Name Data1.bin
xtool:mkraken,t400:mreflate+srep_new:490g+LOLZ_NOR MAL =>*.pak

Data3
Name Data2.bin
xtool:mkraken:mreflate+srep_new:490g+LOLZ_NORMAL => all other files

Results is around 23GB

If you wants to keep languages for future update:
Data4
xtool:mkraken,t400:mzlib:mreflate:mpng:d1+srep_new :490g+LOLZ_NORMAL and add only files (ar.pak , de.pak ...) and do not forget to disable them in Data1.bin
You also can add task and several data to make installer with multiple choice
for language
Language pack is around 15.6GB compressed

JustFun 30-04-2025 07:24

Call of Juarez - (Steam Version)

Code:

Selected ARC/DS method for Data1a-01.bin was:
xtool:mpreflate+oggre+srep_new:490g+lolz
------------------------------------------------------------------------------------------
Overall input size: 2,28 GB
Compressed game size: 1,09 GB (Ratio 47.92%)


L33THAK0R 30-04-2025 21:34

Metal Gear Solid V: Ground Zeroes
 
2 Attachment(s)
Scene Release - CPY

Original Size: 2.82 GiB
Final Size: 1.80 GiB (Incl. installer) (1,940,176,780 bytes)
Ripped Files: None.
Tools used:

Xtool v0.8.9 (Codecs: reflate, zlib)
Srep
Lolz
Oggre

Additional notes:

"GzsTool.exe" & "sdp_tool.exe", and WEM processing tools are attached below.

"data_01.g0s" & "data_02.g0s" contain majority of assets, "data_00.g0s" just contains a FMV cut-scene. Archives are XOR-ed, in layman's terms a simple bit-shifting operation is used to encrypt the archive contents.

My solution isn't ideal (data for each relevant archive is written to the disk twice), however I couldn't find a better approach. It should also be noted this isn't a CRC-perfect packing process but it is deterministic from my testing, ensure no additional files are within the target directory for packing, nor is it missing any original files. "GzsTool.exe" is used to unpack the archives, and then repack them post-install. Within these archives are a variety of file-types, notably ".wem" and ".sdp". The former file-type is self explanatory, and can be processed (for those who are unsure) with the following chain

Code:

ww2ogg.exe "XXX.wem" --pcb "packed_codebooks_aoTuV_603.bin" -o "XXX.tmp"
oggre_enc.exe "XXX.tmp" "XXX.packed"
oggre_dec.exe "XXX.packed" "XXX.unpacked"
hdiffz -c-lzma2-9 -C-no "XXX.unpacked" "XXX.wem" "XXX.patch"

To eliminate ".wem" files that are inflated by packing, I run a basic test/filter for file diff. that I wrote in python, as I've found the <100KiB rule that I've been using for a good while doesn't always hold true.

".sdp" files are just ".bnk" files with an additional header, they can be extracted quickly using "sbp_tool.exe", there you can isolate the wem files, process them and make a patch. Processing ".sdp" archives only nets around ~30MiB (rough estimate, actual figure is under ~100MiB but I've forgotten the exact number).

I've been trying to make any additional gains to at least match the best publicly available size (CorePack's release coming in at 1.73GiB), but I'm unable to determine where further gains can be made. I do have their "Phantom Pain" release and can confirm for that title they use a similar unpacking/repacking method, however I'm unable to source their "Ground Zeroes" release for further analysis. I feel like I'm missing something glaringly obvious but crawling through community documentation has netted nothing that jumps out at me. 2nd place is a nice consolation prize but it would be nice to at least know what they did for a 70MiB reduction.

I'm currently finalisng my "Metal Gear Solid V: The Phantom Pain" approach, and will publish the results soon.

JustFun 03-05-2025 08:09

Dracula: Origin (Steam Version)

Code:

Selected ARC/DS method for Data1a-01.bin was:
xtool:c1792mb:mbik+xtool:mpreflate+oggre+srep_new:490g+lolz
------------------------------------------------------------------------------------------
Overall input size: 1,90 GB
Overall output size: 1,36 GB (Ratio 71.38%)


JustFun 03-05-2025 09:32

NARC (2005) (Retail)

Code:

Selected ARC/DS method for Data1a-01.bin was:
xtool:c1792mb:mbik+xtool:mpreflate+srep_new:490g+lolz
------------------------------------------------------------------------------------------
Overall input size: 1,74 GB
Overall output size: 1,17 GB (Ratio 67.41%)


JustFun 03-05-2025 10:08

Manhunt

Code:

Selected ARC/DS method for Data1a-01.bin was:
xtool:c1792mb:mbik+xtool:mpreflate+srep_new:490g+lolz
------------------------------------------------------------------------------------------
Overall input size: 1,89 GB
Overall output size: 1,21 GB (Ratio 64.17%)


KaktoR 03-05-2025 10:44

Quote:

Originally Posted by L33THAK0R (Post 507489)
Scene Release - CPY

Original Size: 2.82 GiB
Final Size: 1.80 GiB (Incl. installer) (1,940,176,780 bytes)
Ripped Files: None.
Tools used:

Xtool v0.8.9 (Codecs: reflate, zlib)
Srep
Lolz
Oggre

Additional notes:

"GzsTool.exe" & "sdp_tool.exe", and WEM processing tools are attached below.

"data_01.g0s" & "data_02.g0s" contain majority of assets, "data_00.g0s" just contains a FMV cut-scene. Archives are XOR-ed, in layman's terms a simple bit-shifting operation is used to encrypt the archive contents.

My solution isn't ideal (data for each relevant archive is written to the disk twice), however I couldn't find a better approach. It should also be noted this isn't a CRC-perfect packing process but it is deterministic from my testing, ensure no additional files are within the target directory for packing, nor is it missing any original files. "GzsTool.exe" is used to unpack the archives, and then repack them post-install. Within these archives are a variety of file-types, notably ".wem" and ".sdp". The former file-type is self explanatory, and can be processed (for those who are unsure) with the following chain

Code:

ww2ogg.exe "XXX.wem" --pcb "packed_codebooks_aoTuV_603.bin" -o "XXX.tmp"
oggre_enc.exe "XXX.tmp" "XXX.packed"
oggre_dec.exe "XXX.packed" "XXX.unpacked"
hdiffz -c-lzma2-9 -C-no "XXX.unpacked" "XXX.wem" "XXX.patch"

To eliminate ".wem" files that are inflated by packing, I run a basic test/filter for file diff. that I wrote in python, as I've found the <100KiB rule that I've been using for a good while doesn't always hold true.

".sdp" files are just ".bnk" files with an additional header, they can be extracted quickly using "sbp_tool.exe", there you can isolate the wem files, process them and make a patch. Processing ".sdp" archives only nets around ~30MiB (rough estimate, actual figure is under ~100MiB but I've forgotten the exact number).

I've been trying to make any additional gains to at least match the best publicly available size (CorePack's release coming in at 1.73GiB), but I'm unable to determine where further gains can be made. I do have their "Phantom Pain" release and can confirm for that title they use a similar unpacking/repacking method, however I'm unable to source their "Ground Zeroes" release for further analysis. I feel like I'm missing something glaringly obvious but crawling through community documentation has netted nothing that jumps out at me. 2nd place is a nice consolation prize but it would be nice to at least know what they did for a 70MiB reduction.

I'm currently finalisng my "Metal Gear Solid V: The Phantom Pain" approach, and will publish the results soon.

Looks like you cannot use the sbp tool in a for loop with /r option sadly.

st0rm 03-05-2025 15:09

Double Dragon Gaiden: Rise Of The Dragons [2023]
 
Code:

Game Version : v2024.06.21

Ripped - Re-encoded Files : Nothing

Original Size : 2.68 GB

Compressed Size : 476 MB

Install Time : 2 Mins on HDD (Depends your system)

Method : srep:m3f+lzma2


L33THAK0R 03-05-2025 18:58

Quote:

Originally Posted by KaktoR (Post 507546)
Looks like you cannot use the sbp tool in a for loop with /r option sadly.

Yeah it's a touch annoying, I just generated a checksum for all the loose files and then isolated the .sbp entries and ran as a batch script. Thankfully it's only needed to extract the initial WEMs, and a hdiffz patch takes care of the rest post-install.

L33THAK0R 05-05-2025 19:30

Metal Gear Solid V: The Phantom Pain
 
1 Attachment(s)
Scene Release - CODEX

Original Size: 22.9 GiB
Final Size: 10.8 GiB (Incl. installer)
Ripped Files: Online MP Assets
Tools used:

Xtool v0.8.9 (Codecs: reflate, zlib)
Srep
Lolz
Oggre

Additional notes:

Used very similar processing methods to my "Ground Zeroes" results, see here for more details

Used a different fork of "GzsTool", this same version is attached at the end of this post.

There's around ~100MiB that can be gained, however, like with "Ground Zeroes" I'm unable to determine where these gains can be found.

JustFun 06-05-2025 11:34

Sherlock Holmes: The Mystery of the Persian Carpet (Steam Version)

Code:

Selected ARC/DS method for Data1a-01.bin was:
xtool:mpreflate+oggre+srep_new:490g+lolz
------------------------------------------------------------------------------------------
Overall input size: 93,70 MB
Overall output size: 81,07 MB (Ratio 86.53%)



All times are GMT -7. The time now is 00:57.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2026, vBulletin Solutions Inc.
FileForums @ https://fileforums.com