FileForums

FileForums (https://fileforums.com/index.php)
-   Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)
-   -   Best Compression Methods for 'Specific' Games. Q&A (https://fileforums.com/showthread.php?t=99554)

Danziel123 28-01-2025 05:20

Hi everyone, may I ask. Maybe someone knows the right compression method for the Ace Attorney Investigations Collection game? This game uses the unity engine and this is a sample file:
https://drive.google.com/file/d/168I...ew?usp=sharing
Thank you:)

dyren 28-01-2025 06:09

lz4hc,l12 for bundle files
srep+lolz for rest

Danziel123 29-01-2025 22:59

hello everyone, let me ask again. what is the right compression method for the game Suicide Squad Kill the Justice League? here is the sample file:
https://drive.google.com/file/d/1tTg...ew?usp=sharing
Thank you:)

ScOOt3r 30-01-2025 04:09

Hi Danziel

Suicide Squad Kill the Justice League uses. zlib+srep+lolz
this game will not get compressed alot i believe that you will save about 20% compression..

ScOOt3r

KaktoR 30-01-2025 07:42

No Man's Sky
They switched from zlib to zstd

Danziel123 31-01-2025 04:18

Hello everyone, let me ask again. what is the right compression method for the game Marvels Spider-Man 2? here is the sample file:
https://drive.google.com/file/d/1VPa...ew?usp=sharing
Thank you

KaktoR 31-01-2025 06:41

insomniac spiderman:rolleyes:

Wanterlude 31-01-2025 16:01

Quote:

Originally Posted by Danziel123 (Post 506719)
Hello everyone, let me ask again. what is the right compression method for the game Marvels Spider-Man 2? here is the sample file:
https://drive.google.com/file/d/1VPa...ew?usp=sharing
Thank you

Quote:

Originally Posted by KaktoR (Post 506722)
insomniac spiderman:rolleyes:

tex_building file
Quote:

insomniac:spiderman
Streams: 3 / 1898
Time: 00:00:01 (CPU 00:00:00)
Size: 1.29 GB >> 1.29 GB
Quote:

insomniac:rcra
Streams: 1898 / 1898
Time: 00:00:02 (CPU 00:00:01)
Size: 1.29 GB >> 1.34 GB
Quote:

insomniac:rcrads
Streams: 5515 / 5515
Time: 00:00:51 (CPU 00:01:57)
Size: 1.29 GB >> 2.85 GB
Full game [ru+en vo] (Marvel's Spider-Man 2)
Quote:

insomniac:rcrads + dd5 + lzma2
111 gb > ~220-260 gb (don't remember exactly) > 73.4 gb
inst.time[11400f]: ~35 min

KaktoR 04-02-2025 12:23

KCD2 same as KCD

infovs 05-02-2025 02:50

Quote:

Originally Posted by ScOOt3r (Post 506709)
Hi Danziel
Suicide Squad Kill the Justice League uses. zlib+srep+lolz
this game will not get compressed alot i believe that you will save about 20% compression..
ScOOt3r

Based on what is this zlib? Just because somebody included xtool into compression method it does not means that xtool is infact used, nothing is expanded, it just gets skipped and it does nothing. You will get exactly the same "gain" if not using xtool:zlib at all, all gain (small) is from srep+lolz, xtool:zlib is just placebo.
This game uses xtool:kraken (pakchunk51-WindowsNoEditor.pak) but you have to decrypt it with xtool unreal plugin, use known key.

Danziel123 05-02-2025 05:44

Quote:

Originally Posted by infovs (Post 506764)
Based on what is this zlib? Just because somebody included xtool into compression method it does not means that xtool is infact used, nothing is expanded, it just gets skipped and it does nothing. You will get exactly the same "gain" if not using xtool:zlib at all, all gain (small) is from srep+lolz, xtool:zlib is just placebo.
This game uses xtool:kraken but you have to decrypt pak with xtool unreal plugin, use known key.

Maybe you can give the right library info for this game? because I don't know how to check it. Thank you:)

infovs 05-02-2025 06:30

Quote:

Originally Posted by Danziel123 (Post 506766)
Maybe you can give the right library info for this game? because I don't know how to check it. Thank you:)

Use oo2core_9_win64.dll v2.9.5 or 2.9.9 with xtool:kraken and unreal plugin (search for key on known sites, I'm typing this from memory...) on
pakchunk51-WindowsNoEditor.pak
But other .ucas & .pak files are not encrypted and oodle compressed..unfortunately did not have time to test it more. But they are not xtool:zlib also, no gain over srep+lolz only method. Maybe there's something I'm missing...

KaktoR 05-02-2025 06:39

You guys still deal with unreal engine slop?

It's always the same with UE:
Either zlib or oodle is used
crypted or not (use aes_finder on shipping exe to get the key if crypted)
If oodle is used, mostly library 2.9.5 or 2.9.10_lib2
Poor ratio especially if oodle is used (> 80%)

Masquerade 07-02-2025 10:50

Danziel123 just doesn't learn. That's what.

L33THAK0R 07-02-2025 22:43

Quote:

Originally Posted by Danziel123 (Post 506766)
Maybe you can give the right library info for this game? because I don't know how to check it. Thank you:)

XTool's GUI mode functions as a way to test library-based decompression, cut 500mb from the largest archive of a given title and then run that against your collection of libraries. For someone who has been on this forum since 2013 with almost 1200 posts, it's a bit worrying you seem to be unaware of these basic testing methods.

KaktoR 08-02-2025 16:42

L33THAK0R
You don't need to cut anything from a file. There is a checkbox "Stop after # streams".
Just select folder which contains all oo2core v2.9.X libraries and let it run. Based on the results (processed streams and time) you know what library you have to use.
Mostly it's 2.9.5, sometimes 2.9.10 (lib2) and for the newer UE5 games in some cases 2.9.12 (last time I checked at least).

PS: In any case, never ever use library 2.9.6. This version has a memory bug you will notice.

Danziel123 10-02-2025 04:44

Quote:

Originally Posted by L33THAK0R (Post 506799)
XTool's GUI mode functions as a way to test library-based decompression, cut 500mb from the largest archive of a given title and then run that against your collection of libraries. For someone who has been on this forum since 2013 with almost 1200 posts, it's a bit worrying you seem to be unaware of these basic testing methods.

If the file is encrypted I don't know how to check the library used. If it's not encrypted, I can check it myself. The file name used to cut a large file if I may know what is the file name?

KaktoR 10-02-2025 08:29

https://i.imgur.com/IXhSjvH.png

For pak files you have to use ue4, not ue5.

JustFun 14-02-2025 11:17

Tomb Raider IV-VI Remastered - Steam Version (Only 1 Language)

Code:

Selected ARC/DS method for Data1a-01.bin was:
srep_new:393b+lolz
------------------------------------------------------------------------------------------
Overall input size: 10,10 GB
Overall output size: 4,09 GB (Ratio 40.55%)


L33THAK0R 15-02-2025 05:27

Quote:

Originally Posted by Danziel123 (Post 506808)
If the file is encrypted I don't know how to check the library used. If it's not encrypted, I can check it myself. The file name used to cut a large file if I may know what is the file name?

Swiss File Knife (SFK), it's an exceptionally useful tool, as well as HDiffPatch, these two tools alone are very useful for more intermediate repacking processes, such as when specific data types need to be ripped to be processed separately from other data types, and then patched back in post-install.

Hexagon123 15-02-2025 11:40

1 Attachment(s)
It seems like the game: DreamWorks Kung Fu Panda - Showdown of Legendary Legends (2015) likely uses the LZ4 compression.

It was also used in Adventure Time: Finn & Jake Investigations (2015) running on the Vicious Engine 2.

However, older games which used an older compression.

The rundown:
Magic for the file at 0x0: 56 C0 A9 2F
Compressed header at 0x4: 04 22 4D 18
Uncompressed size at 0x8
Unknown at 0xC
File offset at 0x10

KaktoR 15-02-2025 11:51

Pretty sure it is lz4hc

Danziel123 04-03-2025 19:55

Hello everyone, please ask. Does anyone know the right compression method for the ETS 2 game? Because I have tried all compression methods starting from xtool:mzlib:mgdeflate:xtool:dd3+LZMA-MT, fast lzma2 or lzma2 still error or not finished (already waiting for 5 hours) and for gta v enhanced is the compression method still the same as the legacy version? Thank you

mausschieber 05-03-2025 05:40

Quote:

Originally Posted by Danziel123 (Post 506904)
Hello everyone, please ask. Does anyone know the right compression method for the ETS 2 game? Because I have tried all compression methods starting from xtool:mzlib:mgdeflate:xtool:dd3+LZMA-MT, fast lzma2 or lzma2 still error or not finished (already waiting for 5 hours) and for gta v enhanced is the compression method still the same as the legacy version? Thank you

Maybe this will help here

wareck 21-03-2025 22:31

sonic unleashed
 
Hello
I'm working on sonic unleashed.
I'm near the solution but I'm not enough skilled to finish

The game is compressed and packed
xmemcomp (xbox) for compress and arsys (segs archive) for pack.

for decompress:
https://aluigi.altervista.org/bms/xcompress_file.bms

unpack :
https://aluigi.altervista.org/bms/arcsys.bms

there is no problem for xcompress side bms2xtl works perfectly and give me a database.

so the first step is ok, but I don't know how use the second step (arcsys)

Masquerade 23-03-2025 05:44

^^
The files that you have attached are compressed only with xmemcompress.

Do you have any bigger samples?

In any case, the arcsys bms script only mentions support for zlib/deflate/gzip so XTool's zlib+reflate codec should have no bother parsing that data without the need of a special database.

wareck 23-03-2025 06:51

Here an another sample.
but I think you're right most of the files are gzipped after xcompress.

https://mega.nz/file/WlJ3wDAC#O3pO6D...I_wSgVGHWc2ikw

Masquerade 23-03-2025 08:54

^^
That file you sent contains sound data.

KaktoR 31-03-2025 00:57

X4 Foundations
v7.50

Code:

20:52:18 - Selected ARC/DS method for Data1a-01.bin was: xtool:mzlib:dd3+4x4:lzma
 20:52:18 - Selected ARC/DS method for Data1b-01.bin was: msc
 20:52:18 - Selected ARC/DS method for Data1c-01.bin was: oggre
 20:52:18 - Selected ARC/DS method for Data1d-01.bin was: xtool:dd3+FA-MIN
 20:52:18 - Selected ARC/DS method for Data1e-01.bin was: xtool:dd3:l10,d128mb
---------------------------------------------------------------------------------------
 21:48:52 - Overall input size: 32.33 GB
 21:48:52 - Overall output size: 18.57 GB (Ratio 57.42%)
 21:48:52 - Overall conversion time: 00:56:33

You have to filter out some of the OGG files since they will cause oggre to crash. The following list is for game version 7.50. Other versions might change this list. Not tested with French and Russian languages.

Code:

04.dat
0000000006339381.ogg
000000001212EBDD.ogg

06.dat
00000000026B4B51.ogg
0000000006DFAED3.ogg
0000000007553031.ogg
000000000758F062.ogg
0000000007621FDA.ogg
0000000007A0FD8C.ogg
0000000007A32F36.ogg
0000000007A9CDA1.ogg
0000000007BF888F.ogg
0000000007C35E87.ogg
0000000007DB2AC1.ogg
000000000BC6F8B6.ogg

ego_dlc_boron\ext_02.dat
0000000000000000.ogg
0000000000D68742.ogg
000000000188BF76.ogg
00000000041CA372.ogg
0000000004E2B710.ogg
00000000056C8303.ogg
0000000008697024.ogg
000000000B61F5FE.ogg
000000000B678133.ogg
000000000B79804C.ogg
000000000B87D8CD.ogg
000000000D1C9B6C.ogg
000000000D1FFC8B.ogg
000000000D22E54B.ogg
000000000D283532.ogg
000000000D2E95A8.ogg
000000000D32E5C9.ogg
000000000D389BD7.ogg
000000000D3AA153.ogg
000000000D3E5CF1.ogg
000000000D430C00.ogg
000000000D49C44A.ogg
000000000D5082B9.ogg
000000000D80042E.ogg
000000000DAB3D6A.ogg
000000000DB9DD2A.ogg
000000000DBEA7ED.ogg
000000000DC8901B.ogg
000000000DC9BF6A.ogg
000000000DE106B3.ogg
000000000DE2A86A.ogg

ego_dlc_mini_01\ext_02.dat
0000000000937321.ogg
0000000002CC1CA9.ogg

ego_dlc_pirate\ext_02.dat
0000000003939CBB.ogg

ego_dlc_terran\ext_02.dat
000000000299C5C0.ogg
000000000529020E.ogg
0000000007596E61.ogg

ego_dlc_timelines\ext_02.dat
0000000006B12A2B.ogg
000000000757B341.ogg
00000000081746FF.ogg
00000000094831AB.ogg


L33THAK0R 31-03-2025 23:08

Tomb Raider (2013)
 
GOG Release - Build ID 54388455997567214

Original Size: 10.87 GiB
Final Size: 5.66 GiB
Ripped Files: Non-EN Loc.
Tools used:

Xtool v0.8.9 (Codecs: zlib, relfate, bink)
Srep
Lolz
Msc

Notes: I've been revisiting titles using the Foundation Engine after deciding to put the time in to properly processing the *.mul found in earlier titles (More recent titles such as "Shadow of the Tomb Raider" use the *.wem format instead, and offer far less impressive gains). The *.mul file-type is just a basic FSB4 container with a few hundred bytes added on as a header and are stored uncompressed within the *.tiger archives, making offset patching very straight forward, I highly recommend using MParallel for post-install patching as there are just shy of 10,000 files for this smaller title. Actual gains aren't too impressive as my previous compression for this title came in at 5.80 GiB, with an additional approx. 6 GiB of writes needed for this patching method, but I think it's interesting none-the-less seeing how small some titles can be made.

Further gains could be made by ripping multiplayer-exclusive assets, I identified ~90 MiB of audio but was unable to immediately identify any other exclusive assets. I didn't get around to experimenting with any issues from ripping this assets, but my immediate thoughts are it should be fine considering the GOG release already has the multiplayer component removed.

prince55 03-04-2025 16:23

The Last of us Part II
 
Sample
https://pixeldrain.com/u/3rq5jVRz

I tried both psarc plugin and oo2reck, but no precompress, i don't know what i am missing, maybe dll versions ?:confused:

dixen 03-04-2025 22:26

Quote:

Originally Posted by prince55 (Post 507141)
Sample
https://pixeldrain.com/u/3rq5jVRz

I tried both psarc plugin and oo2reck, but no precompress, i don't know what i am missing, maybe dll versions ?:confused:

minsomniac:rcrads+gdeflate

Wanterlude 03-04-2025 22:27

Quote:

Originally Posted by prince55 (Post 507141)
Sample
https://pixeldrain.com/u/3rq5jVRz

I tried both psarc plugin and oo2reck, but no precompress, i don't know what i am missing, maybe dll versions ?:confused:

use insomniac plugin for xtool
insomniac:rcra or insomniac:rcrads

Quote:

Originally Posted by dixen (Post 507145)
minsomniac:rcrads+gdeflate

insomniac:rcrads - uses gdeflate, so its additional use is pointless.

Code:

test file: sp-common.psarc (19.8 gb)

insomniac:rcra

Streams: 130981 / 130981
Time: 00:01:12 (CPU 00:05:34)

Size: 19.9 GB >> 31.8 GB

insomniac:rcrads

Streams: 130981 / 130981
Time: 00:01:05 (CPU 00:05:41)

Size: 19.9 GB >> 31.8 GB

insomniac:rcra + gdeflate

Streams: 130981 / 130982
Time: 00:01:08 (CPU 00:05:46)

Size: 19.9 GB >> 31.8 GB


IRAQIGHOST 03-04-2025 23:47

BLEACH Rebirth of Souls

72GB --> 52GB

Quote:

xtool:mzlib:mreflate:dd3+LOLZ_NORMAL
I got CRC error with these two folders
Quote:

01MIDDLE - 02LOW
so I used this method
Quote:

xt_zlib+srep_new:393a+LOLZ_NORMAL

shazzla 04-04-2025 06:59

Yeah ,sometimes reflate fails for me,too.

KaktoR 04-04-2025 07:32

Try with d1 parameter

prince55 04-04-2025 11:35

Quote:

Originally Posted by dixen (Post 507145)
minsomniac:rcrads+gdeflate

Thanks works perfect

prince55 04-04-2025 11:37

Quote:

Originally Posted by Wanterlude (Post 507146)
use insomniac plugin for xtool
insomniac:rcra or insomniac:rcrads


insomniac:rcrads - uses gdeflate, so its additional use is pointless.

Code:

test file: sp-common.psarc (19.8 gb)

insomniac:rcra

Streams: 130981 / 130981
Time: 00:01:12 (CPU 00:05:34)

Size: 19.9 GB >> 31.8 GB

insomniac:rcrads

Streams: 130981 / 130981
Time: 00:01:05 (CPU 00:05:41)

Size: 19.9 GB >> 31.8 GB

insomniac:rcra + gdeflate

Streams: 130981 / 130982
Time: 00:01:08 (CPU 00:05:46)

Size: 19.9 GB >> 31.8 GB


Yeah, i tried all, same result. thanks for help

L33THAK0R 07-04-2025 03:04

Seasons after Fall
 
GOG Release - Build ID 52358852926631596

Original Size: 3.83 GiB
Final Size: 1.00 GiB (Incl. installer)
Ripped Files: None.
Tools used:

Xtool v0.8.9 (Codecs: zlib, relfate)
Srep
Lolz
Msc
Oggre

Notes: Briefly considered ripping localised audio but all localisations amount to less than 100 MiB. Used a new patching approach that turned out to net a 34.64% reduction (from 1.53 GiB). This involved building the oldPath post-install, rather than ripping one of the audio formats, for separate processing, from an existing oldPath and patching it post-install. A handful of the "*.wav" and ".ogg" audio don't play well with their respective compressor, so (like with many titles), validation tests will be required. I unfortunately didn't save the checksums for the offending files, as this particular title has been scattered across different drives for a handful of weeks now (due to unsatisfactory initial results), but thankfully given the size of the title, the validation tests don't take long.

I'm unsure of room for further gains, the patch file is a close to perfectly optimised as I can currently manage, but if there is room by processing individual asset types separately, I'm not currently aware of them.

L33THAK0R 07-04-2025 08:23

Indika
 
GOG Release - Build ID 57836190042333563

Original Size: 48.5 GiB
Final Size: 15.1 GiB (16,313,675,776 bytes) (Incl. installer)
Ripped Files: None.
Tools used:

Xtool v0.8.9 (Codecs: zlib) (reflate is a touch hit or miss with UE titles, so I opt to play it safe and rely on zlib)
Srep
Lolz
Oggre

Notes: Additional gains were made by patching audio post-install. Unreal Engine 4 titles, in regards to titles that use regular Vorbis OGG audio store it without exception within the "*.uexp" files, archives were first unpacked then using a basic (but fast) python script I threw together, a 10-byte string was searched for within each extracted "*.uexp", resulting in only 772 MiB of writes for an approx. 110 (+/- 10) MiB improvement over current leading results.

Honestly the real challenge was figuring out what I wanted to do for the game icon, I think it turned out alright but the art assets were surprisingly difficult to extend or frame so I went for a compromise using an asset from the official art book as a frame. https://i.imgur.com/5NsstHX.png

I don't want to clog up the forums so I might make a bulk post of the few hundred compressions I've done in the last couple months, but whilst they are quite impressive for the most part in terms of ratio, there's nothing really interesting or unique about the processing methods.


All times are GMT -7. The time now is 09:43.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2026, vBulletin Solutions Inc.
FileForums @ https://fileforums.com