FileForums

FileForums (https://fileforums.com/index.php)
-   Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)
-   -   Best Compression Methods for 'Specific' Games. Q&A (https://fileforums.com/showthread.php?t=99554)

rbn1120 06-04-2023 04:02

Satisfactory | 19,90GB -> 6,08GB

Code:

.pak
srep:m3f:a2:l256+lolz:dt:dtb1:dtw1:dtm1:dto1:dtd1:mtt1:mt4:d128m:fba4096

rest
srep:m3f:a2:l256+oodleMT:m0:l3:c128:t6


KaktoR 07-04-2023 08:27

EVERSPACE 2

zlib
Code:

Streams: 927573 / 927957
Time: 00:08:21 (CPU 00:36:08)
Duplicates: 42779 (352 MB) [663 MB >> 2.60 GB]
Srep decompression memory: 619 MB [1.15 GB*]

Size: 28.0 GB >> 57.2 GB >> 54.6 GB >> 46.6 GB

Edit ---

Code:

21:04:04 - Selected ARC/DS method for Data1a-01.bin was: xtool:mzlib:odb:odd3+4x4:lzma
 21:04:04 - Selected ARC/DS method for Data1b-01.bin was: msc_tak
 21:04:04 - Selected ARC/DS method for Data1c-01.bin was: xtool:odb:odd3+4x4:lzma
-------------------------------------------------------------------------------------------------------------------------------------------
 22:20:13 - Overall input size: 34.27 GB
 22:20:13 - Overall output size: 24.85 GB (Ratio 72.51%)
 22:20:13 - Overall conversion time: 01:16:08
-------------------------------------------------------------------------------------------------------------------------------------------
 22:20:19 - Starting Data1a-01.bin archive decompression.
 22:28:59 - Starting Data1b-01.bin archive decompression.
 22:29:14 - Starting Data1c-01.bin archive decompression.
 22:31:15 - Overall decompression time: 00:10:55
 22:31:15 - All files have been successfully extracted!


KaktoR 08-04-2023 12:36

Hearts of Iron IV
v1.12.10

Code:

20:54:11 - Selected ARC/DS method for Data1a-01.bin was: srep_new:1+lolz:d33:mtt1:mt12:mc1023
 20:54:11 - Selected ARC/DS method for Data1b-01.bin was: msc_tak
 20:54:11 - Selected ARC/DS method for Data1c-01.bin was: oggre
 20:54:11 - Selected ARC/DS method for Data1d-01.bin was: 1
 20:54:11 - Selected ARC/SPLIT method for Data1e-01.bin was: srep_new:1+lolz:d33:mtt1:mt12:mc1023/$jpgpack=xtool:mbrunsli/$mp3pack=7zdll-pm:m0=mp3det:m1=packmp3c/$deflatepack=xtool:c128mb:mpreflate:mpng:d1+srep_new:1+lolz:d33:mtt1:mt12:mc1023/$storepack=1/$textpack=srep_new:1+lzma:a1:mfbt4:256m:fb273:mc1000000000:lc8
-------------------------------------------------------------------------------------------------------------------------------------------
 21:18:08 - Overall input size: 6.38 GB
 21:18:08 - Overall output size: 3.03 GB (Ratio 47.49%)
 21:18:08 - Overall conversion time: 00:23:56
-------------------------------------------------------------------------------------------------------------------------------------------
 21:34:34 - Overall decompression time: 00:06:05
 21:34:34 - All files have been successfully extracted!


---


Data1a = *.dds
Data1b = *.wav, *.bmp
Data1c = *.ogg (exclude "dlc\dlc035_no_step_back_preorder_bonus\music\Katyusha.ogg")
Data1d = "dlc\dlc035_no_step_back_preorder_bonus\music\Katyusha.ogg"
Data1e = rest of the files - masked


L33THAK0R 10-04-2023 23:19

Bit of a basic question but how exactly does ZSTD re-compression work? Within XTool, is it a library-based like LZ4 or more universal like reflate? Additionally is ripping a block, say for example the first 200mb of a larger archive a suitable method for creating a sample to run tests on? I'm currently giving most of the RE Engine titles a crack and have been having some difficulties with testing different ZSTD libraries, as I've had no success with the stock XTool v0.5.3 ZSTD codec library, and its a bit of a waste to write so much data to the disk just for it to be for nothing.

dixen 12-04-2023 02:03

What about *.vpk from Half-Life Alyx? LZ4 algo?

Ra7eN 12-04-2023 14:26

Skyrim SE
 
definitely wrong area. Please delete this.

KaktoR 13-04-2023 13:24

X4 Foundations v6.0

Maybe someone might find this usefull

Code:

zlib+msc (oggre will crash)
01.dat
05.dat
06.dat
07.dat
extensions\ego_dlc_boron\ext_01.dat
extensions\ego_dlc_boron\ext_02.dat
extensions\ego_dlc_pirate\ext_01.dat
extensions\ego_dlc_pirate\ext_02.dat
extensions\ego_dlc_split\ext_01.dat
extensions\ego_dlc_terran\ext_01.dat
extensions\ego_dlc_terran\ext_02.dat

msc
02.dat

oggre will crash
03.dat
04.dat


L0v3craft 15-04-2023 02:13

Quote:

Originally Posted by dixen (Post 500809)
What about *.vpk from Half-Life Alyx? LZ4 algo?

Seems to have lz4 compression but for some reason xtool is not able to process the streams:

https://t85.pixhost.to/thumbs/94/347471141_immagine.png

Masquerade 15-04-2023 06:04

Please may someone upload a sample, such as
game/hlvr/maps/a1_intro_world/a1_intro_world_skybox.vpk?

Thanks!

L0v3craft 15-04-2023 06:14

Quote:

Originally Posted by Masquerade (Post 500830)
Please may someone upload a sample, such as
game/hlvr/maps/a1_intro_world/a1_intro_world_skybox.vpk?

Thanks!

here

Code:

https://gofile.io/d/EafcDT

Zyzz 17-04-2023 09:07

Kerbal Space Program 2
 
I am using Diskspan with xtool+srep+lzam(1 and 2) and i am getting this error
Code:

WARNING: can't open file "E:\Kerbal Space Program 2\KSP2_x64_Data\StreamingAssets\aa\StandaloneWindows64\parts-base_assets_generator_0v_thermoelectric_radioisotope_icon.png_62877585fabce190dba4ae03f3ade21f.b  9.4%
WARNING: can't open file "E:\Kerbal Space Program 2\KSP2_x64_Data\StreamingAssets\aa\StandaloneWindows64\parts-base_assets_generator_3v_thermoelectric_radioisotope_icon.png_d09dd8a19b5eddfc5024b06104f62283.b 10.8%

i was thinking of using lolz but it takes too long. i tried the xtool unity plugin but that didnt seem to help.

Masquerade 17-04-2023 09:49

Your compression tools aren't incorrect here, I think this issue is because you are hitting the filepath limit in Windows.

This quick registry hack should fix it.

Also, do use the Unity plugin because afaik this game uses lz4 compressed bundles.

KaktoR 17-04-2023 10:04

^
Just the small ones. All of the big files are not compressed it seems.

Code:

Compressed 1,762 files, 32,373,757,878 => 37,763,790,489 bytes. Ratio 116.65%
Compression time: cpu 14.59 sec/real 877.50 sec = 2%. Speed 36.89 mB/s
All OK


Razor12911 21-04-2023 15:46

Star Ocean: The Divine Force

Code:

Compressed 1 file, 349,845,408 => 1,256,090,567 bytes. Ratio 359.04%
Compression time: cpu 0.30 sec/real 21.82 sec = 1%. Speed 16.03 mB/s

more info here

KaktoR 22-04-2023 07:11

Could someone send me a small sample, like "chunk_0\prop.cpk", from "Star Ocean The Divine Force"?

Would be appreciated :)

Masquerade 22-04-2023 07:42

Quote:

Originally Posted by KaktoR (Post 500906)
Could someone send me a small sample, like "chunk_0\prop.cpk", from "Star Ocean The Divine Force"?

Would be appreciated :)

https://hexupload.net/un3syuvgoidh

kj911 24-04-2023 02:54

Quote:

Originally Posted by kj911 (Post 496879)
This piggy and fbz compressor tell someone something?

fbz.exe => Blizzard v0.24b! Available from encode.su
piggy.dll -> https://github.com/kaby76/Piggy ??

Does anyone know what "cls-sndpack.dll" could be used for in some repacks? I came across it once, 3kB in size.

Masquerade 25-04-2023 12:30

Strayed Lights

Ripped *.PDB

xtool_zlib+msc_tak+srep+lolz

*.bk2 = xtool_dedup5_fl2

7.5GB ---> 3GB

KaktoR 29-04-2023 06:35

Star Wars Jedi Survivor

kraken
Code:

pakchunk15optional-WindowsNoEditor.ucas

Streams: 9944 / 9950
Time: 00:08:15 (CPU 00:46:45)

Size: 3.56 GB >> 9.73 GB


KaktoR 30-04-2023 00:20

The Last of Us Part 1
v1.0.4.1

Code:

14:05:49 - Selected ARC/DS method for Data1a-01.bin was: xtool:mkraken,l4,n256:core_2.9.9+xtool:mpreflate+xtool:odd3+4x4:lzma
 14:05:49 - Selected ARC/DS method for Data1b-01.bin was: xtool:odd3+4x4:lzma
-----------------------------------------------------------------------------------------
16:36:26 - Overall input size: 69.43 GB
 16:36:26 - Overall output size: 50.22 GB (Ratio 72.32%)
 16:36:26 - Overall conversion time: 02:18:58

With xtool plugin

Code:

22:45:38 - Selected ARC/DS method for Data1a-01.bin was: xtool:mpsarc+xtool:mpreflate:odd3+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000
 22:45:38 - Selected ARC/DS method for Data1b-01.bin was: xtool:odd3+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000
-----------------------------------------------------------------------------------------
00:43:06 - Overall input size: 69.43 GB
 00:43:06 - Overall output size: 48.51 GB (Ratio 69.87%)
 00:43:06 - Overall conversion time: 01:57:27


dixen 30-04-2023 04:23

KaktoR
Did you tested *.pak RE 4 Remake? 80% ratio for me(
zstd+zlib

KaktoR 30-04-2023 04:58

@dixen
No I don't have this game.

Dragonis40 04-05-2023 06:33

Good afternoon! I've been trying to get some info about best .wad files compression method in Batman Arkham Knight, but i've found very little info on web and nothing on fileforums site.
Is there any way to massively losslessy compress (and decompress) these files? I've tried almost every tool in Diskspan, but the only good compression mask seems to be to usual srep + lolz.
Tham you in advance!

KaktoR 04-05-2023 07:42

Send some samples here, so we can tell you something usefull :)

Dragonis40 04-05-2023 09:46

I'd like to compress all .wad files, not specific files. So I don't know which file I can send you.

KaktoR 04-05-2023 09:50

Looking on steamdb, it seems that these files are actually sound files. But I don't have the game so I cannot say for sure, just looking at the file names it seem to be sound files actually.

Maybe wwise in some unreal engine format or whatever. Cannot tell you anything about it.

Masquerade 04-05-2023 11:02

Not to mention this is just pure awkwardness - if you want help to compress all the files, upload a small one so we can at least see what type of file it is rather than giving some dumb excuse. File Extensions mean nothing, what you really need is a Hex Editor to see what the file actually is.

Something like

BmGame/CookedPCConsole/SFX/SFX_4.wad

Might be good.

Dragonis40 05-05-2023 05:37

https://we.tl/t-e6tlDpZfJ1

FitGirl 05-05-2023 16:36

Quote:

Originally Posted by Dragonis40 (Post 501028)

WWise Ogg, can be processed manually by Oggre, check Masquerade's tutorial.

Average gain ~10% of initial size.

KaktoR 06-05-2023 10:05

Lego Star Wars The Skywalker Saga
Update 04.05.2023

Code:

17:08:51 - Selected ARC/DS method for Data1a-01.bin was: xtool:mlegoswtws+xtool:dd3+4x4:lzma
-------------------------------------------------------------------------------------------------------------------------------------------
 18:32:13 - Overall input size: 30.48 GB
 18:32:13 - Overall output size: 22.88 GB (Ratio 75.05%)
 18:32:13 - Overall conversion time: 01:23:20
-------------------------------------------------------------------------------------------------------------------------------------------
 18:57:23 - Overall decompression time: 00:25:04
 18:57:23 - All files have been successfully extracted!


Dragonis40 07-05-2023 01:19

Quote:

Originally Posted by FitGirl (Post 501035)
WWise Ogg, can be processed manually by Oggre, check Masquerade's tutorial.

Average gain ~10% of initial size.

are you sure? How can I use wWWise ogg procedure on .wad files? I've tried to use wemtool, but of course the exe file found nothing.

L33THAK0R 07-05-2023 09:22

How to rip compressed assets, when only having the decompressed assets?
 
Currently I'm taking a swing at the various Quantic Dreams titles that were given a proper PC release, however I've run into a bit of a roadblock. The archives for these titles are only compressed with a basic zlib/deflate algorithm, with XTool being able to process the vast majority of streams.

Unfortunately unlike every other title I've worked on, even the localised Vorbis audio is compressed within these archives, leaving me stumped on how to rip it to make the various localised packs into selective offerings.

Currently the only method I could think to exploit was decompressing the archives as a singular block with XTool, blanking out the offending data and then try and restore the block, however, as I feared I was only met with decompression errors and was unsuccessful.

I should mention I the QuickBMS scripts for Quantic Dream titles doesn't work at all with any of the PC ports that I tested, making for scanning compressed blocks with QuickBMS currently not possible.

Has anyone had a similar problem, and if so would such an individual be willing to share their solution, I'd greatly appreciate it, these blasted titles are doing nothing but keeping me awake at night!

Gehrman 09-05-2023 21:03

Assassins Creed Valhalla Complete v1.7.0
 
Code:

*.forge 134 GB > 44.7 GB
Code:

xtool:manvil:db:dd3:l10
Code:

Other Files 10 GB > 7.22 GB
Code:

xrep:db:dd3:l10
Full
144 GB > 51.9 GB

CT: 04:05:19
DT: 02:26:47



Code:

[External compressor:xtool,xrep]
header    = 0
unpackcmd = "XTool.exe" decode -t100p -dm10p -sm10p - - <stdin> <stdout>


adrianskiloses 10-05-2023 02:15

Request
 
Offworld Trading Company please.
Thank you.

Masquerade 10-05-2023 03:45

Quote:

Originally Posted by adrianskiloses (Post 501060)
Offworld Trading Company please.
Thank you.

srep+lolz

adrianskiloses 10-05-2023 05:21

Quote:

Originally Posted by Masquerade (Post 501064)
srep+lolz

Thank you very much.
Can you also suggest compression methods for FLAC files (for DLCs)?

Masquerade 10-05-2023 08:02

FLACs cannot be losslessly compressed, but you can use ffmpeg to recode them into compressed audio formats like MP3

adrianskiloses 10-05-2023 08:24

Quote:

Originally Posted by Masquerade (Post 501067)
FLACs cannot be losslessly compressed, but you can use ffmpeg to recode them into compressed audio formats like MP3

There is an OST DLC that has both mp3 and FLAC versions. So should I ignore compressing the FLAC entirely in that case?

Also is there any guides for compressing Unity games in general?
Thanks

Hexagon123 10-05-2023 11:41

Quote:

Originally Posted by adrianskiloses (Post 501068)
There is an OST DLC that has both mp3 and FLAC versions. So should I ignore compressing the FLAC entirely in that case?

Also is there any guides for compressing Unity games in general?
Thanks

Depends on the game,

Some use LZ4, LZMA or none

SREP+LOLZ

IRAQIGHOST 11-05-2023 06:19

Resident Evil 4 Remake
 
*.pak
xtool:mzlib:mzstd+srep_new:3+LOLZ_NORMAL

other files
srep_new:3+LZMA-MT*

Repack Size: 42 Gb
Final Size: 57 Gb (Deluxe Edition)


All times are GMT -7. The time now is 22:55.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2026, vBulletin Solutions Inc.
FileForums @ https://fileforums.com