FileForums

FileForums (https://fileforums.com/index.php)
-   Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)
-   -   Best Compression Methods for 'Specific' Games. Q&A (https://fileforums.com/showthread.php?t=99554)

Masquerade 25-07-2021 02:05

Quote:

Originally Posted by :( Sad8669 (Post 493404)
Is there a way to compress .awb audio files?

1:1 CRC Perfect.

No, AWB is a lossy format so even when you extract the wav audio, there's no way you can patch it back.

:( Sad8669 26-07-2021 06:15

Edit : Nothing.

Masquerade 27-07-2021 06:12

SnowRunner [v14 + All DLCs]
 
Code:

xzlib+srep+lolz
27.2GB -----> 13.2GB

Masquerade 27-07-2021 09:38

Samurai Warriors 5
 
Code:

xzlib+srep+lolz
for movie folder, use
Code:

srep+lzma
15.1GB ---> 9.6GB

MV059.bin and MV060.bin are credits videos, so you can rip them to give you a final of 9.1GB.

KaktoR 28-07-2021 11:36

The Forgotten City

Code:

19:37:28 - Selected ARC/DS method for Data1a-01.bin was: srep_new+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000
----------------------------------------------------------------------------------------------------------------------------------------
 20:35:24 - Overall input size: 25.84 GB
 20:35:24 - Overall output size: 9.43 GB (Ratio 36.50%)
 20:35:24 - Overall conversion time: 00:57:55

Code:

19:28:10 - Selected ARC/DS method for Data1a-01.bin was: m1
----------------------------------------------------------------------------------------------------------------------------------------
 19:36:38 - Overall input size: 25.84 GB
 19:36:38 - Overall output size: 12.46 GB (Ratio 48.20%)
 19:36:38 - Overall conversion time: 00:08:25


dixen 28-07-2021 12:05

Outlast 2 (GOG) (English voice only)

*.TFC, *.UPK - uelr+srep+lolz
23,2 gb > 36 gb > 24 gb > 11,6 gb

*.bnk, *.ol2 - srep+lzma2
1,7 gb > 1,3 gb

Rest files - srep+lolz
115 mb > 55 mb

Repack size - 13 gb
Install size - 25 gb
Install time - 4 mins. (r5 3600, HDD)

dixen 29-07-2021 06:18

Crysis 3 (DDE) (RUS+ENG)

*.pak - XTool (with CRYEngine plugin)+SREP+LOLZ
10.8 gb > 16.5 gb > 11 gb > 6.9 gb

Rest files - SREP+LOLZ
43 mb > 13 mb
Quote:

packcmd = xt21\xt21 precomp -mcrysis3+zlib+preflate -d1 -c256mb -t100p - - <stdin> <stdout>

:( Sad8669 30-07-2021 06:09

  • Naruto - Ultimate Ninja Storm - Precompression

dataRegion.cpk
Code:

Compressed 1 file, 436,892,316 => 1,358,762,846 bytes. Ratio 311.01%
Compression time: cpu 1.22 sec/real 26.37 sec = 5%. Speed 16.57 mB/s
All OK

Codec : Crilayla

Some .cpk like movie and sound contains no streams, so you can save time by separating them.

:( Sad8669 01-08-2021 08:06

Naruto - Ultimate Ninja Storm [English Only][v1.0]
 
Code:

Method : xtool_crilayla+srep:m3f:l512+lolz:dtb1:d128:mtt0:mt4:mc1023
Compression Time : 2 Hours 21 Minutes 28 Seconds
Decompression Time : 9 Minutes 10 Seconds
Initial Size : 7.53 GB
Final Size : 2.43 GB
Ripped Files : Europe - Japanese Movies + JP Sound Files + Credit Video
Re-encoded : All Main Movies.

Code:

-----------------------
Movie File | Size : 2.02 GB                | Blocks/Offsets
-----------------------
opening_EU.usm   --->    3586646016 - 3776301631
opening_JP.usm    --->    3776303104 - 3965540352
da1050_JP.usm    --->    2699368448 - 3310075263
da1049_JP.usm     --->  1485172736 - 2097358367
da1001_JP.usm      --->  596582400 -  872960095
da1001_EU.usm    --->    298297344 - 596580511
da1001.usm          --->  4096 - 298295615
------------------------

Note : Game is tested and is playable, but before you start the match you must select the sound which you didn't rip out otherwise it will crash the game.


Tools Used + Thanks to : - QuickBms - HxD Hex Editor - Scaleform Encoder - VGMTool Box - Masquerade - panker1992 - Everyone

panker1992 01-08-2021 13:05

Quote:

Originally Posted by :( Sad8669 (Post 493545)
Code:

Method : xtool_crilayla+srep:m3f:l512+lolz:dtb1:d128:mtt0:mt4:mc1023
Compression Time : 2 Hours 21 Minutes 28 Seconds
Decompression Time : 9 Minutes 10 Seconds
Initial Size : 7.53 GB
Final Size : 2.43 GB
Ripped Files : Europe - Japanese Movies + JP Sound Files + Credit Video
Re-encoded : All Main Movies.

Code:

-----------------------
Movie File | Size : 2.02 GB                | Blocks/Offsets
-----------------------
opening_EU.usm   --->    3586646016 - 3776301631
opening_JP.usm    --->    3776303104 - 3965540352
da1050_JP.usm    --->    2699368448 - 3310075263
da1049_JP.usm     --->  1485172736 - 2097358367
da1001_JP.usm      --->  596582400 -  872960095
da1001_EU.usm    --->    298297344 - 596580511
da1001.usm          --->  4096 - 298295615
------------------------

Note : Game is tested and is playable, but before you start the match you must select the sound which you didn't rip out otherwise it will crash the game.


Tools Used + Thanks to : - QuickBms - HxD Hex Editor - Scaleform Encoder - VGMTool Box - Masquerade - panker1992 - Everyone



It seems that you have gotten the idea !!! interesting

i would like to see future repacks made this way, as i find it very resource wasting to have videos 30mbps and audio of the highest quality with no real reason

KaktoR 13-08-2021 01:39

Assassin's Creed Valhalla
v1.3.0, 1 language, all DLCs

Code:

23:23:14 - Selected ARC/DS method for Data1a-01.bin was: oo2recm7+srep_new+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000
 23:23:14 - Selected ARC/DS method for Data1b-01.bin was: srep_new+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000
--------------------------------------------------------------------------------------
 23:23:18 - Starting Data1a-01.bin archive creation.
 09:20:36 - Moving Data1a-01.bin file to media folder.
 09:20:40 - Moving Data1a-02.bin file to media folder.
 09:20:40 - Starting Data1b-01.bin archive creation.
 09:32:41 - Moving Data1b-01.bin file to media folder.
--------------------------------------------------------------------------------------
 10:04:24 - Overall input size: 104,93 GB
 10:04:24 - Overall output size: 38,42 GB (Ratio 36.62%)
 10:04:24 - Data files creation time: 10:09:27


Masquerade 15-08-2021 02:01

Jupiter Hell (Build 7187894)
 
Start size: 1.7GB

Code:

precomp:cn:brute:d1+srep+lolz
Final Size: 1GB

KaktoR 18-08-2021 00:25

Grand Theft Auto V
v1.0.231.0 / 1.54

Code:

01:08:30 - Selected ARC/DS method for Data1a-01.bin was: bpk+xtool:mreflate+srep_new+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000
 01:08:30 - Selected ARC/DS method for Data1b-01.bin was: xtool:mreflate+srep_new+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000
 01:08:30 - Selected ARC/DS method for Data1c-01.bin was: srep_new+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000
----------------------------------------------------------------------------------------------------------------------------------------
 05:39:13 - Overall input size: 102.55 GB
 05:39:13 - Overall output size: 62.48 GB (Ratio 60.93%)
 05:39:13 - Data files creation time: 04:30:42

Data1a contents:
update\update.rpf
update\x64\dlcpacks\patchday3ng\dlc.rpf
update\x64\dlcpacks\patchday17ng\dlc.rpf
update\x64\dlcpacks\patchday19ng\dlc.rpf
update\x64\dlcpacks\patchday20ng\dlc.rpf
update\x64\dlcpacks\patchday21ng\dlc.rpf
update\x64\dlcpacks\patchday22ng\dlc.rpf
update\x64\dlcpacks\patchday24ng\dlc.rpf
update\x64\dlcpacks\patchday25ng\dlc.rpf
x64a.rpf
x64d.rpf

Data1b contents:
*.rpf
exclude Data1a contents and x64\*

Data1c contents:
rest of the files


KaktoR 18-08-2021 02:16

HUMANKIND

Code:

10:24:57 - Selected ARC/DS method for Data1a-01.bin was: srep+LZMA-MT
----------------------------------------------------------------------------------------------------------------------------------------
 11:15:53 - Overall input size: 27.22 GB
 11:15:53 - Overall output size: 10.20 GB (Ratio 37.48%)
 11:15:53 - Overall conversion time: 00:50:53


KaktoR 20-08-2021 00:31

Cyberpunk 2077
v1.30

Code:

22:43:05 - Selected ARC/DS method for Data1a-01.bin was: xtool:c32mb:mcp2077+srep_new+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000
 22:43:05 - Selected ARC/DS method for Data1b-01.bin was: srep_new+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000
--------------------------------------------------------------------------------------
 05:54:53 - Overall input size: 66.50 GB
 05:54:53 - Overall output size: 43.73 GB (Ratio 65.76%)
 05:54:53 - Data files creation time: 06:44:23

Data1a contents:
*.archive

Data1b contents:
rest of the files


Masquerade 20-08-2021 13:32

QUAKE Enhanced
 
Code:

precomp+srep+lolz/$ogg=oggre
1.4GB ---> 370MB

Masquerade 20-08-2021 13:33

RIDE 4 [Build 7159412]
 
Code:

XZLIB+SREP+LOLZ
46.6GB ---> 22.3GB

I used -d3 param with zlib+reflate.

Masquerade 20-08-2021 13:33

Mortal Shell [Build 7180006]
 
Code:

xzlib+srep+lolz
14.9GB ---> 5.5GB

Masquerade 20-08-2021 13:34

Wall Of Insanity [Build 7196424]
 
Code:

srep+lolz
3.5GB ---> 1.1GB

Masquerade 20-08-2021 13:35

303 Squadron: Battle of Britain [v2.0.1]
 
Code:

xzlib+srep+lolz
4.9GB ---> 3GB

Masquerade 21-08-2021 09:21

Nayati River
 
Code:

xzlib+srep+lolz
Now, this is a unity game and you only need srep and lolz, however SREP would crash when trying to run on this game files, so I put xtool in front and it worked (failure on both srep 3.92b and srep 3.2). XTool does nothing to actually improve compression here.

18.2GB ---> 3.8GB

L0v3craft 22-08-2021 00:48

Quote:

Originally Posted by Masquerade (Post 493723)
Mortal Shell [Build 7180006]

Code:

xzlib+srep+lolz
14.9GB ---> 5.5GB

Please write the full parameters that you used for lolz and which version of xtool have you used.

KaktoR 22-08-2021 10:30

Dying Light Platinum Edition

Code:

16:27:24 - Selected ARC/DS method for Data1a-01.bin was: bpk+srep_new+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000
 16:27:24 - Selected ARC/DS method for Data1b-01.bin was: xtool:mreflate+srep_new+lzma:a1:mfbt4:256m:fb273:mc1000000000:lc8
----------------------------------------------------------------------------------------------------------------------------------------
 19:00:21 - Overall input size: 35.08 GB
 19:00:21 - Overall output size: 17.70 GB (Ratio 50.44%)
 19:00:21 - Overall conversion time: 02:32:55


Data1b contents:
*.pak, *.rpack

Data1a contents:
rest of the files


mr_gnar 24-08-2021 10:14

Has anyone tried Serious Sam 3 (BFE)? I have tried the method on here (by dixen), but for some reason, it doesn't work as well.

Quote:

Xtool (zLib codec)+msc+srep+lolz - 6 gb > 3.3 gb
6 GB > 5.55GB for me. I tried changing out my version of XT and srep/lolz as well but it gets almost the same result. :confused:

Masquerade 24-08-2021 10:33

Quote:

Originally Posted by mr_gnar (Post 493746)
Has anyone tried Serious Sam 3 (BFE)? I have tried the method on here (by dixen), but for some reason, it doesn't work as well.



6 GB > 5.55GB for me. I tried changing out my version of XT and srep/lolz as well but it gets almost the same result. :confused:

Make sure you don't have LZMA enabled in your MSC configs, since it will compress everything input with LZMA after any media streams.

KaktoR 26-08-2021 08:32

Small language update for Spyro Reignited Trilogy (if anyone cares at all :D).

Was able to figure out all languages by Audio Track ID

Code:

ID  Language
1    English
2    Arabic
3    Danish
4    German
5    Spanish Mexico *
6    Spanish Spain *
7    Finnish
8    French
9    Italian
10  Dutch
11  Norwegian
12  Polish
13  Brazil **
14  Swedish

*Example: Spoken "[...]es una criatura simplona."
But I'm not sure what is Spain and Mexico. We need some guidance from Spain/Mexican people here :D

**In Portuguese it is "[...]é uma criatura simples."

I will do this same shit for Crash Bandicoot.

KaktoR 26-08-2021 14:30

Language update for Crash Bandicoot 4 It's about Time

Code:

ID  Language
1    English
2    Arabic
3    German
4    Spanish Mexico ???
5    Japanese
6    Spanish
7    French
8    Italian
9    Portuguese Brazil

I couldn't figure out what language ID 4 is yet.

PS: ID 4 is somewhat spanish mexico or whatever. They say "si" for "yes". But other keywords don't match with spanish (spain) or italian.

lomesmo 26-08-2021 18:04

Quote:

Originally Posted by *Example: Spoken "[...
es una criatura simplona."
But I'm not sure what is Spain and Mexico. We need some guidance from Spain/Mexican people here :D

**In Portuguese it is "[...]é uma criatura simples."

I will do this same shit for Crash Bandicoot.

Spanish (Mexico) is Spanish for Latin America. Spanish (Spain) is European, the difference lies in words and adjectives.

For example, mobile phone (Spain) and cell phone (Latin America), things like that

Telefono movil (Espaņa) - Celular (Latinoamerica)(Mexico)

lomesmo 26-08-2021 18:08

It is essentially the same only that in Latin America and in Spain different words are used to express the same idea

KaktoR 27-08-2021 09:23

Makes sense, but because I don't speak spanish I can't tell you any deeper differences between Spanish and Latam ;)

KaktoR 30-08-2021 11:39

Total War Rome Remastered
 
Quote:

Originally Posted by Masquerade (Post 491701)
Code:

xtool+srep+lolz
On *.webm and *.feral
Code:

srep+lzma2
*feral is ogg audio archives, so you could use oggre but no idea of the benefit (or any issue in decompression)

50.1GB ----> 17.3GB :D

You can rip languages from webm videos with ffmpeg (thanks to devs streams are labeled).

Only 1 language: 1.78 GB -> 1.68 GB

You can use MSC on the following files
Code:

Contents\Resources\Data\data\sounds\sfx.dat.feral

Contents\Resources\Data\data\sounds\localised\de\voice_battle.dat.feral
Contents\Resources\Data\data\sounds\localised\de\voice_strat.dat.feral

Contents\Resources\Data\data\sounds\localised\en\voice_battle.dat.feral
Contents\Resources\Data\data\sounds\localised\en\voice_strat.dat.feral

Contents\Resources\Data\data\sounds\localised\it\voice_battle.dat.feral
Contents\Resources\Data\data\sounds\localised\it\voice_strat.dat.feral

Contents\Resources\Data\data\sounds\localised\es\voice_battle.dat.feral
Contents\Resources\Data\data\sounds\localised\es\voice_strat.dat.feral

Contents\Resources\Data\data\sounds\localised\ru\voice_battle.dat.feral
Contents\Resources\Data\data\sounds\localised\ru\voice_strat.dat.feral

Contents\Resources\Data\data\sounds\localised\fr\voice_battle.dat.feral
Contents\Resources\Data\data\sounds\localised\fr\voice_strat.dat.feral


KaktoR 31-08-2021 05:59

Total War Rome Remastered
v2.0.2, 1 language, other languages ripped, no hd pack

Code:

10:53:41 - Selected ARC/SPLIT method for Data1a-01.bin was:
srep_new+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000
$deflatepack=xtool:mreflate+srep_new+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000
$mscromeremastered=msc_tak:mp3=1:wav=1:raw=1:ddsraw=1:bmf=9:TAK=9:bmp=1:lzma,bt4,fb128,lc8
$textpack=srep_new+lzma:a1:mfbt4:256m:fb273:mc1000000000:lc8
$precomppack=precomp_mtx+srep_new+lolz:mtt1:mt<thread>:tt1:mc2
$libpack=srep_new+razor_mtx
----------------------------------------------------------------------------------------------------------------------------------------
 13:29:13 - Overall input size: 43.22 GB
 13:29:13 - Overall output size: 19.46 GB (Ratio 45.02%)
 13:29:13 - Overall conversion time: 02:27:30

$deflatepack: *.png
$msc: see post above
$textpack: *.txt, *.xml, *.json, *.strings
$precomppack: *.gif
$libpack: *.dll, *.exe

video files for ffmpeg: Contents\Resources\Data\data\fmv\intros (1 language: 1.78 GB -> 1.68 GB)

KaktoR 03-09-2021 01:10

Far Cry 5
v1.011, 2 languages, incl HD pack

Code:

21:41:34 - Selected ARC/DS method for Data1a-01.bin was: xtool:mreflate+srep_new+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000
----------------------------------------------------------------------------------------------------------------------------------------
 02:20:26 - Overall input size: 91.60 GB
 02:20:26 - Overall output size: 39.95 GB (Ratio 43.61%)
 02:20:26 - Overall conversion time: 04:38:50

Code:

Used DELZORec_x64 because xtool with fc5 plugin does not work properly atm.

DELZORec times:
8min ssd
25min hdd

PS: DELZORec does not work on any hd game files (*_hd.*)

Code:

DELZORec usage from batch (common.dat example):
DELZORec_x64.exe u "data_final\pc\common.fat" "data_final\pc\common.dat" "data_final\pc\common.unp"
DELZORec_x64.exe r "data_final\pc\common.unp" "data_final\pc\common.dat" "data_final\pc\common.fat"


hwang4 06-09-2021 12:07

WRC 10 FIA World Rally Championship
 
What's Best Method for compression?

Masquerade 06-09-2021 13:41

Quote:

Originally Posted by hwang4 (Post 493880)
What's Best Method for compression?

This game's PKG archives are compressed using LZ4, so first you need to find a way to decompress files inside the archives. Since the header of the files appear to be encrypted, unlike previous games, there's no BMS script to unpack the PKG.

hwang4 07-09-2021 06:13

Masquerade , can you show me your method plz !:)

I tried (xtool+srep+lolz), i got 25,5gb (english+french)!
Your method is better (21gb)
thx!

Masquerade 07-09-2021 13:29

Quote:

Originally Posted by hwang4 (Post 493889)
Masquerade , can you show me your method plz !:)

I tried (xtool+srep+lolz), i got 25,5gb (english+french)!
Your method is better (21gb)
thx!

I used VGM toolbox to "extract" the PKGs, since every lz4 stream is very easy to detect with the 0x04224D18 header with PKGB before the steam and PKGE after the stream. When each file is extracted, you can use lz4 exe to decompress the extracted lz4 streams, then compress with xtool:zlib+srep+lolz.

Then, to remake the pkgs, you just need to compress each file again individually with the lz4 exe using -9 --no-frame-crc arguments. Then, concatenate the files together (I use bincat for this since it's fast and simple, not sure of native cmd option) and xdelta.

Make your xdeltas before compressing your archive, since you will need to decompress all of the extracted files from the pkgs, compress them, then concatenate them to make a temp file. You can compare the temp file with the original pkg to make a delta.

For the sound files, I used razor.

Hexagon123 09-09-2021 14:25

I found out that Yoku's Island Express use ZLIB?

dixen 12-09-2021 14:54

Life Is Strange: Before The Storm (GOG)

*.bytes - xtool (v0.12 + lz4 codec) + srep + lolz

15 gb > 33,6 gb > 15 gb > 4.6 gb

*.usm, *.awb, *.acb - Srep + zstd (cls)
6 gb > 4.6 gb

Rest files - srep + lolz
105 mb > 31 mb

RePack size - 9.3 gb
Install time - 6 min. (R5 3600, 16 gb RAM, HDD>HDD)

NOTE. Why ZSTD? Easy)))

Quote:

Tested 290 files, 5,035,921,557 => 6,416,827,196 bytes. Ratio 78.48%
Testing time: cpu 17.61 sec/real 17.95 sec = 98%. Speed 357.46 mB/s
All OK

Walker96082 13-09-2021 12:21

Dragons Dogma Dark Arisen HD Edition
 
Using FreeArc in Ultra Profile
from 25.7 GB to 7.98 GB:D


All times are GMT -7. The time now is 22:56.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2026, vBulletin Solutions Inc.
FileForums @ https://fileforums.com