FileForums

FileForums (https://fileforums.com/index.php)
-   Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)
-   -   Best Compression Methods for 'Specific' Games. Q&A (https://fileforums.com/showthread.php?t=99554)

DODI 31-10-2024 18:49

Quote:

Originally Posted by KaktoR (Post 506169)
No problem, and have fun

Code:

FreeArc 0.67 (March 15 2014) creating archive: Conversion_Output\Data1a-01.bin
Compressed 1 file, 494,902,306 => 1,014,185,526 bytes. Ratio 204.93%
Compression time: cpu 0.27 sec/real 107.59 sec = 0%. Speed 4.60 mB/s
All OK

FreeArc 0.67 (March 15 2014) extracting archive: Conversion_Output\CONVERSION\Data1a-01.bin
Extracted 1 file, 1,014,185,526 => 494,902,306 bytes. Ratio 204.93%
Extraction time: cpu 0.70 sec/real 93.77 sec = 1%. Speed 5.28 mB/s
All OK


Thanks :)

KaktoR 01-11-2024 05:44

Horizon Zero Dawn Remastered

Same as Forbidden West

IRAQIGHOST 07-11-2024 09:07

Quote:

Forza Horizon 5
Quote:

Version 656.386
Quote:

xtool:c256mb:mpreflate:mzlib+xtool:dd3+LOLZ_ULTRA
Without 4K movies

143GB --> 85.4GB

demon964 11-11-2024 00:47

Quote:

Originally Posted by IRAQIGHOST (Post 506227)
Without 4K movies

143GB --> 85.4GB

fitgirl repack 73.5 gb. How ?

dyren 11-11-2024 05:45

Quote:

Originally Posted by demon964 (Post 506243)
fitgirl repack 73.5 gb. How ?

Using LOLZ_ULTRA usually give worse results rather than using lolz. Plus she has more experience.

KaktoR 11-11-2024 05:54

+ you can compress video files and audio banks too.

dyren 11-11-2024 06:06

^^ Oh yeah, that also

shazzla 11-11-2024 09:21

Plus FG uses 2047m:mt1.

FitGirl 11-11-2024 16:59

Quote:

Originally Posted by shazzla (Post 506248)
Plus FG uses 2047m:mt1.

No, I don't :)

shazzla 11-11-2024 23:39

Quote:

Originally Posted by FitGirl (Post 506250)
No, I don't :)

Then something like that. nevermind.. :D

dyren 15-11-2024 08:49

Incursion Red River

Code:

srep+lolz
Game size: 53.4GB
Compressed size: 13.6GB

Paks are not compressed and are not encrypted.
Compression took ~21 hours on 4c/8t CPU

L33THAK0R 15-11-2024 20:14

Cryptmaster
 
1 Attachment(s)
Spanish (ES) language files ripped

srep+lolz+oggre

This title was an interesting one as the language files (stored as fsb containers) do not have any consistent ordering within the Unity resource archives, majority. An AI/ML model, specifically an ASR model was used in conjunction with some Python scripting to determine whether a given input contained English or Spanish dialogue. Approx. 200 files had to be processed manually as they contained only a single word, or had a heavy post-processing filter applied, making it difficult to get a definitive result, overall success rate was approx. 94%. English was preferred in analysis to reduce likelihood of English files being accidentally misidentified and ripped.

Final total size of archives: 965 MiB

Based on v1.042, GOG Release

I estimate there might still be a few Spanish files I was not able to identify, with more aggressive packing techniques I think it's definitely possible to reduce overall size to ≤900 MiB.

Attached is a checksum file with all ripped files, you're welcome.

IRAQIGHOST 17-11-2024 04:20

Assassin's Creed Mirage (English and Arabic Only)

*.*.forge files
Quote:

xtool:manvil:core_2.8.4:dd3+LOLZ_NORMAL
Sound files
Quote:

xtool:dd3+bcm
rest files
Quote:

srep_new:393b+4x4:b96mb:lzma:ultra:bt4:fb273:lc8:m c1000000:96mb
38.6GB --> 20.5GB

hwang4 18-11-2024 22:26

Dragon's Dogma 2
 
what's the best method ? thx!

KaktoR 19-11-2024 07:25

I would say zstd

L33THAK0R 19-11-2024 19:04

Been digging through the files myself, zstd+deflate streams

IRAQIGHOST 20-11-2024 09:54

s.t.a.l.k.e.r. 2: heart of chornobyl

best method?

KaktoR 20-11-2024 10:33

Send sample man. Everytime the same.

Edit: This is again a shit UE5 game. Try the usual things.

Toximan69 20-11-2024 11:55

Quote:

Originally Posted by KaktoR (Post 506332)
Send sample man. Everytime the same.

Edit: This is again a shit UE5 game. Try the usual things.

sample from Stalker 2
https://drive.google.com/file/d/1AZe...usp=drive_link

dixen 20-11-2024 14:31

Quote:

Originally Posted by IRAQIGHOST (Post 506331)
s.t.a.l.k.e.r. 2: heart of chornobyl

best method?

Code:

mue5:k0x33A604DF49A07FFD4A4C919962161F5C35A134D37EFA98DB37A34F6450D7D386:n128+kraken:l8+mermaid -lm -d3
NOTE. Decompression is very slow
oo2core_9_win64.dll 2.9.6 or 2.9.9

KaktoR 21-11-2024 07:14

Do not use 2.9.6.

2.9.5 is the correct one.

Danziel123 21-11-2024 07:31

Quote:

Originally Posted by KaktoR (Post 506338)
Do not use 2.9.6.

2.9.5 is the correct one.

Sorry to join the discussion, so what should i write in dsg to compress stalker 2?

dyren 21-11-2024 08:01

Quote:

Originally Posted by Danziel123 (Post 506339)
Sorry to join the discussion, so what should i write in dsg to compress stalker 2?

Code:

xtool:c32mb:d1:mue5,k0x33A604DF49A07FFD4A4C919962161F5C35A134D37EFA98DB37A34F6450D7D386:mkraken:core_2.9.5+srep_new:393b+lolz
for *.pak files

KaktoR 25-11-2024 04:31

Alan Wake 2
v1.2.4, nothing ripped

Code:

20:16:46 - Selected ARC/DS method for Data1a-01.bin was: xtool:maw2:dd3:l10,d128mb
 20:16:46 - Selected ARC/DS method for Data1b-01.bin was: xtool:dd3:l10,d128mb
 20:16:46 - Selected ARC/DS method for Data1c-01.bin was: xtool:lm:c1500mb:mbik:mbk2:df20p
-----------------------------------------------------------------------------------------------------------------------------------------
 01:36:40 - Overall input size: 146.06 GB
 01:36:40 - Overall output size: 82.67 GB (Ratio 56.60%)
 01:36:40 - Overall conversion time: 05:19:53

Bink2 videos were ripped out of game files first and then compressed afterwards for reasons, hence the input size. Original size is 111GB.

ZAZA4EVER 09-12-2024 23:36

Indiana Jones and the Great Circle
 
1 Attachment(s)
Indiana Jones and the Great Circle (Test)

Files Test (50.7MB)

Code:

"base\gameresources_50.streamdb"
"base\gameresources_patch1_pc.resources"
"base\gameresources_10_patch1.streamdb"
"base\gameresources_20_hq_patch1.streamdb"
"base\gameresources_45_patch1.streamdb"

Method Kraken (oodle file attached)

Code:

-moo2reck
Result

Code:

FreeArc 0.67 (March 15 2014) creating archive: ZAZA.Test.V5.bin
Compressed 5 files, 53,250,548 => 122,929,176 bytes. Ratio 230.85%
Compression time: cpu 0.05 sec/real 18.41 sec = 0%. Speed 2.89 mB/s
All OK

You can use oodle lib version 2.9.12 (attached with game files)
or oodle lib version 2.9.10 all both is the same result
thanks to Kaktor for oodle lib versions Here

ZAZA4EVER 09-12-2024 23:57

Red Dead Redemption
 
1 Attachment(s)
Red Dead Redemption (Test)

Files Test (381MB)

Code:

"mapres.rpf"
"blackwater.rpf"

Method ZSTD (ZSTD file attached)

Code:

-mzstd,l19
Result

Code:

FreeArc 0.67 (March 15 2014) creating archive: ZAZA.Test.bin
Compressed 2 files, 399,803,872 => 1,213,731,228 bytes. Ratio 303.58%
Compression time: cpu 0.61 sec/real 44.09 sec = 1%. Speed 9.07 mB/s
All OK

You can use Zlib+Zstd and get some streams

Code:

-mzlib+zstd,l19
Code:

FreeArc 0.67 (March 15 2014) creating archive: ZAZA.Test.bin
Compressed 2 files, 399,803,872 => 1,224,533,926 bytes. Ratio 306.28%
Compression time: cpu 0.61 sec/real 46.82 sec = 1%. Speed 8.54 mB/s
All OK

You can use Xtool Ver 0.4.3 to Ver 0.6.0 all works fine with ZSTD

Special thanks to Masquerade , Fitgirl and Kaktor

IRAQIGHOST 17-12-2024 10:31

1 Attachment(s)
Indiana Jones and the Great Circle

.streamdb and .resources files
Quote:

oo2reck:7+srep_new:490g+LOLZ_NORMAL
Sound files
Quote:

msc_tak+srep_new:490g+LOLZ_NORMAL
rest of files
Quote:

srep_new:490g+4x4:b96mb:lzma:ultra:bt4:fb273:lc8:m c1000000:96mb
Game size: 112GB
Repack Size: 86GB

use the file attached
thanks to Kaktor for oodle lib

L33THAK0R 22-12-2024 05:58

Sunset Overdrive
 
1 Attachment(s)
Based on latest CSF

xtool (marginal zlib gains) + srep + lolz

Original Size: 22.9 GiB
Without Ripped Videos: 12.9 GiB
With Ripped Videos: 14.4 GiB

Non-EN loc., 894 MiB of "optional videos" (start-up cinematics and a 853 MiB promotional video that was used for the "Dawn of the Rise of the Fallen Machines" DLC (same video as seen in this YouTube video), as well as the credits cinematic. Should be noted for ripped videos a dummy Bink video needs to be written from the start offset of a given ripped video, otherwise the title softlocks.

Bink videos are also unable to be packed despite being KB2j variants, either with the xtool plugin or with bpk, bpk refuses outright, xtool plugin at least detects the streams, decoding with the RAD Video Tools shows no visual or audio artefacts strangely.

Attached are some modding scripts that are useful if anyone wants to dig around the files further, I know the original XBOX release had assets compressed with LZ4 (From watching GDC talks where Insomniac developers directly reference this) however I was unable to find any LZ4 streams in either the asset archives or in the decompressed assets, zlib streams are largely in the toc/dag files from testing.

Further gains could likely be made by compressing the audio, encoded with the *.wem structure. Spent a long time trying different approaches, very disappointed with how this one turned out but I've got quite the backlog of projects and alas I have to move on.

KaktoR 22-12-2024 08:09

@L33THAK0R

I was able to pack the bink videos. There are 3, one of them is BK2, the others BIK.

Code:

FreeArc 0.67 (March 15 2014) creating archive: Conversion_Output\Data1a-01.bin
Compressed 3 files, 1,580,482,272 => 1,292,223,837 bytes. Ratio 81.76%
Compression time: cpu 1.08 sec/real 154.58 sec = 1%. Speed 10.22 mB/s
All OK

As for the WEM files. I tested a bit with the language files and you will gain just a small amount of ratio even if you group+squash the files into one single file.

g00s001 WEM files 733MB -> 672MB

Just make sure to exclude WEM files which are less than 100kb in size if you process them. There is always a magic line for the WEM files for which - if they less than size X - processing them will yield in zero ratio or sometimes even in negative ratio.

JustFun 22-12-2024 22:42

Dreamfall: The Longest Journey - Steam Version

Code:

Selected ARC/DS method for Data1a-01.bin was:
xtool:c1792mb:mbik+xtool:mpreflate+srep_new:393b+lolz
------------------------------------------------------------------------------------------
Overall input size: 5.92 GB
Overall output size: 1.74 GB (Ratio 29.30%)


L33THAK0R 23-12-2024 00:19

Quote:

Originally Posted by KaktoR (Post 506523)
@L33THAK0R

I was able to pack the bink videos. There are 3, one of them is BK2, the others BIK.

Code:

FreeArc 0.67 (March 15 2014) creating archive: Conversion_Output\Data1a-01.bin
Compressed 3 files, 1,580,482,272 => 1,292,223,837 bytes. Ratio 81.76%
Compression time: cpu 1.08 sec/real 154.58 sec = 1%. Speed 10.22 mB/s
All OK

As for the WEM files. I tested a bit with the language files and you will gain just a small amount of ratio even if you group+squash the files into one single file.

g00s001 WEM files 733MB -> 672MB

Just make sure to exclude WEM files which are less than 100kb in size if you process them. There is always a magic line for the WEM files for which - if they less than size X - processing them will yield in zero ratio or sometimes even in negative ratio.

I feel very silly now! It just so happens out of the 3 videos I sampled all had the KB2j signature, talk about rough luck. I need to start packing WEM files, never had massive luck with them due to the annoying packing process, appreciate the advice!

KaktoR 23-12-2024 03:24

Don't hassle with the language files. Imo they aren't worth to process with wemtool (just ~15mb per language if you skip all files below 100kb).

You can extract the WEM files with this tool I guess (I use another tool without 2GB limitation)
https://www.fileforums.com/showpost....6&postcount=82

A bit messy but if you get any errors with that 2GB file limitation, just search for the first riff wavefmt stream somewhere in mid-file offset, then split this file into two halfs and use the extractor if you don't have other tools by hand. Like I said this is a bit messy but should work as a workaround. After split and extracting WEM files, erase WEM files with xtool erase command. Afterwards just merge the two files back into one file, then decode back WEM files afterwards with xtool. The rest is up to you.

If I remember correctly the total save with processed WEM files is about 200-250MB in total without language files (input ~1.27 GB).

JustFun 26-12-2024 12:36

Crash Time 2 - Steam Version (Only 1 Language)

Code:

Selected ARC/DS method for Data1a-01.bin was:
xtool:mpreflate+msc_tak+srep_new:393b+lolz
------------------------------------------------------------------------------------------
Overall input size: 1.95 GB
Overall output size: 890.81 MB (Ratio 44.58%)


dyren 31-12-2024 06:54

The Crust

Code:

xtool:mreflate,l9+srep+lolz
Game size: 5.16GB
Compressed size: 3.60GB

Paks are not encrypted, uses ZLib compression method.
Compression took ~10 hours on 4c/8t CPU.

kj911 03-01-2025 08:16

Old TRS2004-2012 recompression issues
 
2 Attachment(s)
Any tools/idea from precompressing old TRS game series? Aka TRS2004 to 2012 games used *.ja archives. Newer TRS2012 SP1 and newer 20xx versions use (probably uncompressed) only *.tzarc game files.

Very old TRS2004 related *.ja extractor added and *.ja archive sample from the TRS2012 game.

In last stage, the overall best methods from *.ja files:

RAZOR only (stdio version) with 1023mb dictionary size, the files shrunked to 4.22GB.

Any rest files use in: xZLib/xtool+msc+delta+srep+lolz or xZLib/xtool+RAZOR

Output size differences abysmal, less than ~0.5MB's compressed size. (RZ used 256MB dict size. Larger probably get in 0.5-2MB extra size reduction.) The game not easily fit to onto ONE DVD5 sizes.

dyren 08-01-2025 09:43

Finnish Cottage Simulator

For paks:
Code:

xtool:mkraken:ue5:k0x22269AD77003B4D9225C8FE775B448B8BEACA9D76D3AB881E5BD99D1584E4C76+srep+lolz
Rest:
Code:

srep+lolz
Game size: 4.66GB
Compressed size: 2.93GB

Paks are encrypted, uses Oodle, kraken compression method.
Compression took ~3.5 hours on 4c/8t CPU.
Oodle version: 2.9.10, [Unreal Engine 5] oo2core_9_win64.dll (from Masquerade's xtool archive)

wareck 09-01-2025 07:15

Six Days in Fallujah
 
*.pak xtool:c32mb:mue4:mzlib+xtool:dd3+lolz:d62:mtt1:mt1 6:mc1023
*.bk2 xtool:dd3+bcm (not compatible with bkp bk2 compression)
other files : xtool:mzlib:mreflate:dd3+razor_mtx

14:58:28 - Overall input size: 21,09 GB
14:58:28 - Overall output size: 9,85 GB (Ratio 46.70%)
14:58:28 - Overall conversion time: 01:00:27
14:58:28 - Overall conversion media: 1 custom media

IRAQIGHOST 16-01-2025 02:43

Hi
What is the best method for Bodycam?
simple
https://www.swiftuploads.com/QN9mE6N736dW/file

KaktoR 16-01-2025 06:16

Maybe use key from Bodycam-Win64-Shipping.exe to find out.

IRAQIGHOST 16-01-2025 12:13

Quote:

Originally Posted by KaktoR (Post 506601)
Maybe use key from Bodycam-Win64-Shipping.exe to find out.

Thanks I found it


All times are GMT -7. The time now is 09:43.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2026, vBulletin Solutions Inc.
FileForums @ https://fileforums.com