FileForums

FileForums (https://fileforums.com/index.php)
-   Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)
-   -   Best Compression Methods for 'Specific' Games. Q&A (https://fileforums.com/showthread.php?t=99554)

:( Sad8669 21-01-2022 23:45

Forza Horizon 5 - Precompression

Code:

Overall input size: 92.32 GB
Overall output size: 150.84 GB

Compressed 11,938 files, 99,133,025,380 => 161,958,571,033 bytes. Ratio 163.37%
Compression time: cpu 190.63 sec/real 7941.73 sec = 2%. Speed 12.48 mB/s
All OK


:( Sad8669 25-01-2022 00:48

Yakuza Kiwami 2 - From Yakuza's Series 2/7
 
Code:

Method : srep:m3f:l512+lzma:a1:mfbt4:256m:fb273:mc1000000000:lc8
Compression Time : 6 Hours 39 Minutes and 47 seconds
Decompression Time : 52 Minutes (4T)
Initial Size : 41.02 GB
Final Size: 30.24 GB


:( Sad8669 25-01-2022 06:19

God Of War - BNK Sound Files
 
1 Attachment(s)
Well, if anyone is interested in hearing/extracting GOW sounds, you can do the following.

They are found here, GodOfWar\exec\wad\pc_le\soundbanks with extension .sdp

Open any of the file with HxD, select everything and scroll down until you see BKHD. Delete all hex code to the point you see "BKHD", also delete the highlighted area.

Now, you have a bank file. Which can be extracted using armyoftwo.bms (QuickBMS).

OR

You can use bnkextr, its faster.

https://github.com/eXpl0it3r/bnkextr
After extracting the bank, you will find alot of .wem files named as .wav

Convert them to OGG, if you want to hear them.

KaktoR 25-01-2022 08:29

1 Attachment(s)
Thanks for the hint.

You can repack extracted files with plain zip (0 compression, just storing) and create a diff file to patch back to original. Now find a way to do the hex editing for hundreds of files in a batch :p

:( Sad8669 25-01-2022 08:49

Creating diff was also in my mind, but there is no bnk repacker so you can't hope to use WemTool on the inside .wem files, since there would be no way to rebuild them.

If there was a bnkrepacker, we might have saved 150~200 MB (only 1~2 MB compression with WemTool on 80 MB files)

BKHD header occurs at different places in different files, i think there is no pattern + we can ignore some 1 KB ~ 1 MB files :D

Masquerade 25-01-2022 13:58

Serious Sam: Siberian Mayhem

Start Size: 25.5GB

Method:
Code:

xtool+msc+srep+lolz
Final Size: 17.3GB

Notes:

*.gro archives are Zip files.

dixen 26-01-2022 00:34

Quote:

Originally Posted by Masquerade (Post 495552)
Final Size: [Compressing...]

~ 18 gb.
XTool v0.40 + Reflate = All *.gro - 28 gb > 50 gb..but..error unpack from lolz (it just closed on 6.4%):confused:
UPD
XTool + zlib+preflate
Quote:

Tested 1 file, 18,629,519,581 => 26,137,702,000 bytes. Ratio 71.27%
Testing time: cpu 28.02 sec/real 500.07 sec = 6%. Speed 52.27 mB/s
All OK

:( Sad8669 26-01-2022 00:52

God of War - English Only [Day 1 Patch Not Included]
 
Code:

Method : srep:m3f:l512+lzma:a1:mfbt4:256m:fb273:mc1000000000:lc8
Compression Time : 8.4 Hours
Decompression Time : 55+ Minutes (4T)
Initial Size : 61 GB
Final Size: 27.9 GB
Ripped : All Lang files except ENG
Possibilities : You can use BPK on bink files.


Masquerade 26-01-2022 06:17

Kapia
 
Code:

srep+lolz
12.6GB ---> 2.7GB

KaktoR 26-01-2022 13:19

Quote:

Originally Posted by Masquerade (Post 495552)
Serious Sam: Siberian Mayhem

Start Size: 25.5GB

Method:
Code:

xtool+msc+srep+lolz
Final Size: 17.3GB

Notes:

*.gro archives are Zip files.

There are some ogg's too

Masquerade 26-01-2022 13:23

I saw those. I think may be a good idea to have another go and use QuickBMS to import the oggs back to their correct spot in the archive. Using oggre in the pipeline would probably cause issues.

KaktoR 26-01-2022 13:27

I guess you can just make a xtool database for this.

Masquerade 26-01-2022 23:50

There's only about 350mb of OGG audio in the game.

Masquerade 27-01-2022 10:27

Neon Abyss

Step 1: 7x bundle files (ignoring StandaloneWindows64) - Use UBT to decompress them, then recompress each bundle and make a patch for each. I ended up with ~16mb of HDiffZ patches in total. When decompressed, the game requires 2.3GB of disk space despite only being 1GB when compressed.

Step 2: Compress:

Code:

srep+lolz/$mp4=srep+zstd/$bank=srep+razor/$hdiffz=srep
Final Size: 250MB

Masquerade 27-01-2022 13:28

Rugby 22

Code:

srep+lolz/$wav=msc+srep+lzma2/$mp4=srep+zstd
23.5GB ----> 3.7GB

I experienced issues with srep v3.92b with this game so I used srep v3.2 instead.

panker1992 29-01-2022 04:43

what files in Hitman 3 are the patch files ?? i want to replace the originals with patches and save traffic

Masquerade 29-01-2022 06:01

panker1992
All the files with "patch" in their name:
https://i.ibb.co/jrBXqfJ/Capture.png

Masquerade 29-01-2022 07:56

HighFleet
Code:

precomp+srep+lolz/$wav=msc+srep+lzma/$mp3=mpz+srep/$ogg=oggre
1.3GB ----> 773MB

KaktoR 30-01-2022 13:05

Galactic Civilizations IV (Beta)

Code:

21:37:14 - Selected ARC/SPLIT method for Data1a-01.bin was: srep_new+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000
/$textpack=dict+lzp+lzma:a1:mfbt4:256m:fb273:mc1000000000:lc8
/$deflatepack=xt_zlib+srep_new+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000
/$ddspack=srep_new+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000
/$bpkpack=bpk
/$mp3pack=srep_new+mpz_mtx
/$libpack=srep_new+razor_mtx
 21:37:14 - Selected ARC/DS method for Data1b-01.bin was: srep_new+1
-------------------------------------------------------------------------------------------------------------------------------------------
 22:04:24 - Overall input size: 11.94 GB
 22:04:24 - Overall output size: 8.40 GB (Ratio 70.40%)
 22:04:24 - Overall conversion time: 00:27:07

Exclude from bpk
Code:

Movies\homescreen_logo2.bk2
Movies\Large_Frame.bk2
Movies\End Game\Defeat_Event.bk2
Movies\Events\Event_Promethion.bk2
Movies\Events\Event_TradeRouteRaid.bk2
Movies\PlanetBinks\HorizontalGraph.bk2
Movies\Techs\GC3_Armor_Temp.bk2
Movies\Techs\GC3_Beam_Temp.bk2
Movies\Techs\GC3_Colonization_Temp.bk2
Movies\Techs\GC3_Drive_Systems_Temp.bk2
Movies\Techs\GC3_Economics_Temp.bk2
Movies\Techs\GC3_Interstellar_Craft_Temp.bk2
Movies\Techs\GC3_Invasion_Temp.bk2
Movies\Techs\GC3_Kinetic_Temp.bk2
Movies\Techs\GC3_Life_Support_Temp.bk2
Movies\Techs\GC3_lndustrail_Temp.bk2
Movies\Techs\GC3_Marines_Temp.bk2
Movies\Techs\GC3_Missle_Temp.bk2
Movies\Techs\GC3_Planetary_Defense_Temp.bk2
Movies\Techs\GC3_Point_Defense_Temp.bk2
Movies\Techs\GC3_Research_Temp.bk2
Movies\Techs\GC3_Sensors_Temp.bk2
Movies\Techs\GC3_Shields_Temp.bk2
Movies\Techs\GC3_Starbase_Defense_Temp.bk2
Movies\Terran\Terran_ResearchAdviser_Approach.bk2
Movies\Terran\Terran_ResearchAdviser_Idle.bk2

Thanks to Masquerade with helping me looping -mbpk for arc

itsthorin 31-01-2022 03:26

Wanna learn highly compressed repacks
 
Hey there, I wanna try repacking for fun like fitgirl and every other repacker. If anybody can help please explain to me step by step everything from compressing the game to creating an installer package for it, your help will mean a lot to me I've been trying this for days but failed every time.

Masquerade 31-01-2022 10:17

LEGO Worlds [Build 3833060 - ENGLISH ONLY]

Step 1:

Unpack game files using QuickBMS + Modified BMS script posted here.

Step 2:

Patch the game exe as shown here. This will enable the game to run in an unpacked state. Developers deliberately tried to prevent this!

Step 2.5 [Optional]:

You can separate Voiceover files inside "AUDIO\VO\DX\NARRATOR" directory.

Step 3:

Compress

Code:

srep+lolz/$ogg=srep+lzma2
4.8GB ===> 1.6GB

Notes:
  • Don't use oggre, it doesn't decompress.
  • The sizes above are for English only repack, if you add all other voiceovers, ofc it will be larger.
  • A collective thanks to aluigi, acidicoala and our mutual friend m0xf for the unpacking tools and EXE patching.

fabrieunko 31-01-2022 11:17

Quote:

Originally Posted by Masquerade (Post 495577)
Rugby 22

Code:

srep+lolz/$wav=msc+srep+lzma2/$mp4=srep+zstd
23.5GB ----> 3.7GB

I experienced issues with srep v3.92b with this game so I used srep v3.2 instead.


how to use this command with DiskSpanGui2007? when i copy

srep+lolz/$wav=msc+srep+lzma2/$mp4=srep+zstd

DiskSpanGui2007 answers me the "$wav=msc" method is not supported.

KaktoR 31-01-2022 11:28

$wav = $mscpack
$mp4 = $storepack

dixen 02-02-2022 04:56

Life is Strange: Before the Storm - Remastered

*.Bytes - XTool(v0.12)+srep+lolz
16.3 gb > 19.5 gb > 12,5 gb > 7.8 gb

*.usm - srep+lzma2
4.6 gb > 3.6 gb

*.awb, *.acb - srep+zstd
1.7 gb > 1.1 gb

Rest files - srep+lolz

Repack size - 12.7 gb
Install size - 22 gb
Install time - 5 min (R5 3600)

Note. For precompression *.bytes - you need liblz4 v1.9.3
Early versions are not as effective

Masquerade 02-02-2022 13:49

For Life Is Strange Remastered, you can extract WWise Audio to use WemTool + separate JP voiceovers. You need QuickBMS:

BMS Script: https://bayfiles.com/L046w1Ffx9/unreal_bms

To extract:
Code:

quickbms_4gb_files -f "*.wem" unreal.bms pakchunk0-WindowsNoEditor.pak tmp
Now make a copy of the tmp folder, call it tmp2 or whatever.
Then eliminate the wem files from the archive with the following commands:
Code:

for /f "tokens=*" %%i in ('dir /b /s /a *.wem') do cd.>"%%i"
quickbms -w -r unreal.bms pakchunk0-WindowsNoEditor.pak tmp

Thanks to Dixen for the first command to null the wems.

This will reimport empty space where the wems originally were.

Then, run wemtool over the extracted wems to compress them.

Finally, during decompression, decode the wems with wemtool then reimport them with quickbms
Code:

quickbms -w -r unreal.bms pakchunk0-WindowsNoEditor.pak tmp2

fabrieunko 03-02-2022 00:32

Code:

for /f "tokens=*" %%i in ('dir /b /s /a *.wem') do cd.>"%%i"
quickbms -w -r unreal.bms pakchunk0-WindowsNoEditor.pak tmp


Hello masquerade, when I use this command, windows gives me an error: %% was unexpected

Masquerade 03-02-2022 05:16

Quote:

Originally Posted by fabrieunko (Post 495641)
Code:

for /f "tokens=*" %%i in ('dir /b /s /a *.wem') do cd.>"%%i"
quickbms -w -r unreal.bms pakchunk0-WindowsNoEditor.pak tmp


Hello masquerade, when I use this command, windows gives me an error: %% was unexpected

Swap %%i to %i

fabrieunko 03-02-2022 08:12

I got confused in the manipulation to do, can you confirm the steps:

I do :

1: quickbms_4gb_files -f "*.wem" unreal.bms pakchunk0-WindowsNoEditor.pak tmp

2: I copy everything in TMP to TMP2

3: for /f "tokens=*" %i in ('dir /b /s /a *.wem') do cd.>"%i"
4: quickbms -w -r unreal.bms pakchunk0-WindowsNoEditor.pak tmp

5: I launch WemTool -e in the TMP2 folder

I don't understand this part:
Finally, during decompression, decode the wems with wemtool then reimport them with quickbms

6 ????

7: quickbms -w -r unreal.bms pakchunk0-WindowsNoEditor.pak tmp2

KaktoR 03-02-2022 08:15

Quote:

Originally Posted by fabrieunko (Post 495650)
I don't understand this part:
Finally, during decompression, decode the wems with wemtool then reimport them with quickbms

6 ????

WemTool with -d parameter:
WemTool.exe -d

fabrieunko 03-02-2022 10:51

excuse me for insisting but I do not understand after this command:

for /f "tokens=*" %i in ('dir /b /s /a *.wem') do cd.>"%i"
quickbms -w -r unreal.bms pakchunk0-WindowsNoEditor.pak tmp

what do i do with wemtool?

:( Sad8669 03-02-2022 21:08

Read this : https://fileforums.com/showthread.php?t=104859

:( Sad8669 04-02-2022 02:55

God of War BNK Sound Files- Update II
 
I have already posted about .sbp files of GOW and how to extract them, however i failed to provide a method to compress them while keeping CRC. Here it is,

What do you need?

- HxD Editor
- QuickBMS & armyoftwo.bms
- HDiffz or any delta patcher

Notes :
- This is not efficient at all.
- This method is based on theory (It should work though).
- Barely a 2~3 MB gain from 80 MB files (there are two of them).
- WemTool doesn't work on localized sound banks.
- Time Consuming
- While Nulling the files, make sure to use *.wav and *.dat

Method:

1. First you have to make the .sbp files compatible with QuickBMS. Follow the link.

2. After converting the .sbp to a compatible version, create diff patches between them, since you deleted a good amount of Hex code, although the patch should be smaller.

3. Extract the new compatible bank using QuickBMS, create a copy of the extracted folder, and null the inside files, follow this guide. Upon extraction, you will see alot of .wavs and .dats, meaning that you are in trouble and you will need to rename them to .wem to make it work with WemTool. (Use PowerRename or anything)

4. During installation, decode the the files using WemTool, then rename them back to their original names . Reimport them using QuickBMS and use the diff patches to patch the data you removed in step 1.

Quick Tip:

If you are willing the convert the .sbp to compatible version, open it with HxD, search for "BKHD", convert the code to dec, click on the "dot" before BKHD, note the offset, Use Ctrl+E then select from 0 to whatever offset you noted when you searched for BKHD.

For Example :
Code:

lvl_stn000.sbp  | 0 - 18527 |
lvl_helr000.sbp | 0 - 16735 |
lvl_peak000.sbp | 0 - 21599 |
lvl_hel000.sbp  | 0 - 14687 |
lvl_alf000.sbp  | 0 - 44127 |
lvl_foot000.sbp | 0 - 7007  |
lvl_riv000.sbp  | 0 - 22367 |
lvl_stn950.sbp  | 0 - 5471  |

Thanks to Masquerade, dixen and everyone.

KaktoR 05-02-2022 11:38

Dying Light 2
*.bik -> bpk works
*.rpack, navmesh -> just use srep+lolz/lzma/...
*.dat -> zlib
streams_<lang>.aesp -> some ogg files are inside (~97% ratio)

:( Sad8669 05-02-2022 21:17

Dying Light 2 - Movies

Uses Bink 1 format so bpk works for sure, also stated by KakTor.
This game comes with two versions of movies, FHD and 4K.

Movies Folder [1.51 GB] [FHD only]
Code:

Initial Size : 1.51 GB
Final Size : 1.13 GB
Method: BPK

Movies Folder [4.40 GB] [Both 4K + FHD]
Code:

Initial Size : 4.40 GB
Final Size : 3.21 GB
Method: BPK


fabrieunko 05-02-2022 23:06

Quote:

Originally Posted by KaktoR (Post 495685)
Dying Light 2
*.bik -> bpk works
*.rpack, navmesh -> just use srep+lolz/lzma/...
*.dat -> zlib
streams_<lang>.aesp -> some ogg files are inside (~97% ratio)



Hello, when there are several styles of files how to make a single BIN?

:( Sad8669 06-02-2022 00:13

Masked compression is your good friend.

Check this out : https://fileforums.com/showthread.php?t=97530

Don't read too much, just the bit where its about masked compression.

A game contains files of different formats, so as you said several styles of algorithms are used with desired formats.

Masquerade 06-02-2022 00:32

Quote:

Originally Posted by fabrieunko (Post 495693)
Hello, when there are several styles of files how to make a single BIN?

xtool+srep+lolz/$bink=bpk/$aesp=oggre

Make an arc.groups file in the same location as arc.exe and put this:

Code:

$Default

$bink
*.bik
*.bk2

$aesp
*.aesp


Masquerade 06-02-2022 01:40

The LEGO Movie 2 Videogame

Step 1: Unpack DAT archives using QuickBMS and this script.

Step 2: Apply TTMagic patch (Koalyptus Framework).

Step 3: Compress:

Code:

srep+lolz/$ogg=srep+LZMA2/$fmv=srep+LZMA2
  • You can rip the NXG folder inside the MOVIES folder because since only the DX11 game executable is patched by Koalyptus framework, the NXG videos will never be used.

14GB ===> 4GB

fabrieunko 06-02-2022 02:42

OUPSsss sorry

Masquerade 06-02-2022 04:21

Quote:

Originally Posted by fabrieunko (Post 495700)
Hello I would like to test to learn but where to find the game?

Forum Rule 1 :mad:
https://www.fileforums.com/showthread.php?t=54193


All times are GMT -7. The time now is 12:42.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2026, vBulletin Solutions Inc.
FileForums @ https://fileforums.com