FileForums

FileForums (https://fileforums.com/index.php)
-   Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)
-   -   Best Compression Methods for 'Specific' Games. Q&A (https://fileforums.com/showthread.php?t=99554)

Masquerade 25-06-2023 10:12

This will absolutely work however you can replace The QuickBMS reimport with XTool erase/decode since the contents of a PCK aren't compressed so the wems will appear in plain bytes.

I've wanted in the past to find some way of adding a wem plugin to XTool so wems don't need to be extracted to disk (which is a long time when there are often thousands of tiny wems).

KaktoR 26-06-2023 06:15

Only bad thing left is that quickbms isn't able to create extracted folders in the same (sub)folder where the input files are. In AC games you have several dlc folders with pck files.
In those cases I just use a FOR loop for it, works great. However you have to use "cd /D" command everytime to switch the folder if you are on root level as working directory, otherwise quickbms will create the extracted folders inside root folder without _extracted prefix and you have to move them manually afterwards.

kj911 29-06-2023 11:07

Any idea, methods, tricks recompressing Trainz 2010 (and PTP2*) game *.CDP files/complete games???

Example file the following CDP format: http://lat-trainz-addons.1w.lv/DR1A-...20Pack.cdp.zip

XTool 0.12/0.71 with zlib or zlib+(p)reflate methods or many precomp 0.4.x versions, not handle inflating the following file. GFS found ~19k ZLIB streams packed in level:9 mode.

*Pro Train Perfect 2. Public copy not available. (The game based on TS2010.)

shazzla 30-06-2023 10:12

This BK2 files cannot be compressed using BPK ?!
Its a new thing to me.
Or i miss something ? (Other BIKs/BK2s compressing well.)

XY@DESKTOP-TSNPU1C d:\repack\pack
$ arc a -mbpk e:\bik.arc "d:\Layers of Fear - 2023-\Cron\Content\Movies\Intro\Intro_4K.bk2"
FreeArc 0.67 (March 15 2014) creating archive: e:\bik.arc
Compressed 1 file, 669,310,488 => 669,310,655 bytes. Ratio 100.00%
Compression time: cpu 1.09 sec/real 4.47 sec = 24%. Speed 149.78 mB/s
All OK

wattafak@DESKTOP-TSNPU1C e:\repack\pack
$ arc a -mbpk e:\bik2.arc "d:\Layers of Fear - 2023-\Cron\Content\Movies\CH5\Cinema\CH5_Cinema_Movie.b k2"
FreeArc 0.67 (March 15 2014) creating archive: e:\bik2.arc
Compressed 1 file, 100,336,964 => 100,336,995 bytes. Ratio 100.00%
Compression time: cpu 0.14 sec/real 0.66 sec = 21%. Speed 151.72 mB/s
All OK

KaktoR 30-06-2023 10:15

Yes, new bink2 videos cannot be compressed with bpk. Watch out for the "KB2n" file header with hex editor.

kuyhaa 01-07-2023 22:29

hi everyone

how to extract "disc" file on SOS: A Wonderful Life game and pack back, or plugin for xtool

thanks

Masquerade 02-07-2023 06:06

Quote:

Originally Posted by kuyhaa (Post 501444)
hi everyone

how to extract "disc" file on SOS: A Wonderful Life game and pack back, or plugin for xtool

thanks

Not possible with XTool. You can have the game running with extracted disc file but needs EXE patching. Password to decrypt disc can be found here: https://github.com/Masquerade64/SoS-AWL-Password
Just normal ZipCrypto. Compressed data with lzs algo, very similar to early lz4.

KaktoR 02-07-2023 11:16

For my earlier test, about unpacking PCK files, encoding wem's and such, here is a comparisson for ACOrigins audio files. Excluding processing banks.


SFX + English audio
Code:

original
input 3.795.077.936 bytes

Compressed 41 files, 3,795,077,936 => 3,041,411,552 bytes. Ratio 80.14%
Compression time: cpu 2.05 sec/real 223.01 sec = 1%. Speed 17.02 mB/s
All OK

Time 4 minutes

---

diffz+ww files + nulled wem blovks inside PCK files
input 7.422.151.129 bytes)

Compressed 148,175 files, 7,422,151,129 => 2,979,433,554 bytes. Ratio 40.14%
Compression time: cpu 23.94 sec/real 763.29 sec = 3%. Speed 9.72 mB/s
All OK

Time 1 hour 20 minutes

~25%-33% +time per language


L33THAK0R 02-07-2023 18:14

Quote:

Originally Posted by KaktoR (Post 501451)
For my earlier test, about unpacking PCK files, encoding wem's and such, here is a comparisson for ACOrigins audio files. Excluding processing banks.


SFX + English audio
Code:

original
input 3.795.077.936 bytes

Compressed 41 files, 3,795,077,936 => 3,041,411,552 bytes. Ratio 80.14%
Compression time: cpu 2.05 sec/real 223.01 sec = 1%. Speed 17.02 mB/s
All OK

Time 4 minutes

---

diffz+ww files + nulled wem blovks inside PCK files
input 7.422.151.129 bytes)

Compressed 148,175 files, 7,422,151,129 => 2,979,433,554 bytes. Ratio 40.14%
Compression time: cpu 23.94 sec/real 763.29 sec = 3%. Speed 9.72 mB/s
All OK

Time 1 hour 20 minutes

~25%-33% +time per language


I wonder if it'd be faster and result in a better ratio to instead extract all the relevant files and pack each PCK as one singular block, then patch the block once its been decoded and reverted back to its original form with the original PCK file. I've found especially with smaller blanked out sections within an archive the final compressor sometimes doesn't achieve as optimal a result as it would when working on a singular patch file. I had this exact issue with Just Cause 3, when I initially patched each FSB (of which there were thousands) and then wrote the now patched FSBs into the archives, which resulted in a fairly lackluster result, compared to combining each valid file extracted from the FSBs into one singular block and creating a hdiffz patch between that block and the original "*.arc" archive.

Enluaphelis 03-07-2023 01:17

Hello! How to determine which is the best method for compressing "Gravel" game?
I did not find it in the index so I don't know.
It has .mp4 files, .pak files, .exe
Also I don't quite understand how to make an installer after compressing it in DiskSpanGUI. I tried using some random methods but the files I got as an output look weird and I don't know what is the best option: "dvd5" or "unlimited". I don't understand how to pack it in the installer, where to put compressed files in and how to make it work.
I'd appreciate your help.

KaktoR 03-07-2023 03:10

Use zlib

Code:

Streams: 10832 / 10834
Time: 00:00:05 (CPU 00:00:25)

Size: 312 MB >> 669 MB


Fak Eid 03-07-2023 04:08

Hey, I'm new and don't even know if this question is relevant here. But I need help:

1. How to identify language files from a game. Is there a tool for it? Like for some games like Spider-Man Remastered and GotG, I was not able to locate language files.

2. I'm currently using Ultra compression method of FreeArc (-mx -ld1600m), and using fazip.exe to decompress, but it takes around 1hr to extract Uncharted: Legacy of Thieves Collection. (99.2 GB compressed to 64.4GB). My FreeArc version is 0.67, however my fazip.exe version is 0.0.3 (and a decade old from 2013). Can someone provide me a link to the latest fazip.exe and fazip64.exe files.

3. Also, to decompress, what is the difference in time between fazip.exe and fazip64.exe.

KaktoR 03-07-2023 04:13

1. You can use steamdb and look inside the depots. Example for Spider-Man Remastered: the language files have the short language code as file extension. Example: French -> asset_archive\a00s035.fr

2. There is no newer version of freearc nor fazip, sadly.

3. Don't know but I guess fazip64 is using 64-bit architecture and can utilize more resources.

Enluaphelis 03-07-2023 05:17

Quote:

Originally Posted by KaktoR (Post 501455)
Use zlib

Code:

Streams: 10832 / 10834
Time: 00:00:05 (CPU 00:00:25)

Size: 312 MB >> 669 MB


For all files you mean? Also how to properly make an installer and what is the best installer for diskspangui? Which split method choose in "dvd-cd-bluray-unlimited" section?

KaktoR 03-07-2023 05:23

For pak files, but you can use it on all files, doesn't matter.

For supported installers look here
https://fileforums.com/showpost.php?...87&postcount=3

Enluaphelis 03-07-2023 06:00

Quote:

Originally Posted by KaktoR (Post 501460)
For pak files, but you can use it on all files, doesn't matter.

For supported installers look here
https://fileforums.com/showpost.php?...87&postcount=3

Understood. I've chosen your installator ASIS. I don't quite understand how to configure it and which options I should change in .ini file and where to place compressed files after diskspan?

Enluaphelis 03-07-2023 08:22

Quote:

Originally Posted by KaktoR (Post 501460)
For pak files, but you can use it on all files, doesn't matter.

For supported installers look here
https://fileforums.com/showpost.php?...87&postcount=3

I think I understood how to made an installer, but how to get rid of everything except for .bin files? (https://imgur.com/a/dmIqPZL)
Installer doesn't work without these files.
But how does every single repack works without these, only with Setup.exe and data files?

KaktoR 03-07-2023 11:10

Setup.dll and Records.ini are required. However you can include both inside Setup.exe on compiling.

If you use ASIS, do the following steps:
1. Extract Setup.dll to "[ASIS folder]\DECOMPRESSOR" folder
2. Move Records.ini to the same folder
3. Execute "Create_Setup.dll.bat"
3a. Password input is not visible in CMD window. If you don't want to use a Password, just press ENTER twice
4. Open Script.iss and change line 8
Code:

;#define InternalDLL
to
#define InternalDLL

(just remove the semicolon
5. Compile the script. Setup.dll and Records.ini are now included in Setup.exe

If you used a Password for Setup.dll, you have to change line 9 in the script aswell
Code:

;#define PasswordDLL "123"
Remove the semicolon and replace 123 with your password (quotation marks have to be there)

However, next time please ask such questions in the topics in which the questions belong to. Thanks.

Enluaphelis 05-07-2023 08:19

Quote:

Originally Posted by KaktoR (Post 501455)
Use zlib

Code:

Streams: 10832 / 10834
Time: 00:00:05 (CPU 00:00:25)

Size: 312 MB >> 669 MB


I tried to compressed Gravel game but output size is twice size.
https://imgur.com/a/hm6nbiP
This is the settings I used, I did not find "zlib" so I used xtool:mzlib, I don't know which one I should use: https://imgur.com/a/cbGueNR
What am I doing wrong?

Wanterlude 05-07-2023 08:40

Quote:

Originally Posted by Enluaphelis (Post 501507)
I tried to compressed Gravel game but output size is twice size.
https://imgur.com/a/hm6nbiP
This is the settings I used, I did not find "zlib" so I used xtool:mzlib, I don't know which one I should use: https://imgur.com/a/cbGueNR
What am I doing wrong?

1) Xtool [currently: zlib] (Only decompresses files for better compression)
2) Deduplicator [srep] (Removes repetitions)
3) Compressor [lolz/lzma/7z/...] (lolz better but slower then others)
Example for DiskSpan: xtool:mzlib+srep_new:1+LOLZ_NORMAL or xtool:mzlib+srep_new:1+lzma2

p.s. You do not need to prescribe anything yourself, everything can be selected in the program itself

KaktoR 05-07-2023 08:46

xtool:mzlib+srep+LZMA-MT

Output should be around 9.5 GB

Enluaphelis 05-07-2023 08:47

Quote:

Originally Posted by Wanterlude (Post 501510)
1) Xtool [currently: zlib] (Only decompresses files for better compression)
2) Deduplicator [srep] (Removes repetitions)
3) Compressor [lolz/lzma/7z/...] (lolz better but slower then others)
Example for DiskSpan: xtool:mzlib+srep_new:1+LOLZ_NORMAL or xtool:mzlib+srep_new:1+lzma2

Thanks, gonna try tomorrow. So if I understand using only xtool:zlib only decompressed files for me but not compressed it again and hence I got that size:D
By the way, what does NORMAL mean after LOLZ? It means compression level or thread priotity?

Enluaphelis 05-07-2023 08:51

Quote:

Originally Posted by KaktoR (Post 501511)
xtool:mzlib+srep+LZMA-MT

Output should be around 9.5 GB

Thanks
Also, what "MT" means?
LZMA2 and LZMA-MT, what is the difference between it? Is there any thread out there where I can read more info about compression methods and these special settings for them?

Wanterlude 05-07-2023 08:53

Quote:

Originally Posted by Enluaphelis (Post 501512)
Thanks, gonna try tomorrow. So if I understand using only xtool:zlib only decompressed files for me but not compressed it again and hence I got that size:D
By the way, what does NORMAL mean after LOLZ? It means compression level or thread priotity?

https://i.ibb.co/pyzPcC2/184956.png
Lolz: From the bottom up, from the weakest compression to the strongest, but the stronger the compression, the more time will be spent on it. Normal, High, ... its just presets.
You can read all info about lolz here [But in Russian] [You can use the translator] https://krinkels.org/resources/lolz.264/

KaktoR 05-07-2023 08:55

LZMA2 is used by 7-zip (multithread option is used by default).
LZMA-MT is used by FreeArc (FreeArc LZMA with 4x4 option, literally multithreaded. It will utilize max 16 CPU threads).

As for LZMA2 (it is basically just a predefined config name), you can read possible options somewhere on the 7-zip page I guess.

For the LOLZ_* names: These are just predefined configurations. FreeArc will read LOLZ as follows:
Code:

lolz{:options}
Where {:options} includes all valid options for this compressor.

So the preset "LOLZ_NORMAL" is just this:
Code:

lolz:d<expr:((0.45*freememory/11)-(32*1048576*2))/min(thread,16)/1048576>:mtt1:mt<expr:min(thread,16)>:mc1023
DSG reads your aviable free RAM and replace the :d option in lolz, same for the :mt option, just with your max CPU threads.

However you can write your own options if you want. lolz options can be read in english in the following folder "Resources\FINAL\LOLZ\Options.txt"

Wanterlude 14-07-2023 17:50

1 Attachment(s)
Jagged Alliance 3 [1.01.338010]

Code:

zstd+dedup+lolz
14.6 GB => 10.74 GB

zstd v1.5.2

dixen 15-07-2023 06:47

Total War SHOGUN 2

Movies*.pack - bkpck (XTool+bink pack)
Sound.pack - msc+precompmt(mp3pack)+srep+lzma2
Local*en & ru.pack - precompmt
Rest *.pack & files - xt76(zlib+reflate)+srep+lolz

RePack size = 9.3 gb

Enluaphelis 15-07-2023 07:58

Pure (2008)

Quote:

bpk+srep on .bik files
xtool:mzlib+srep+lolz_normal on everything else
4.90 gb -------> 2.65 gb

Hexagon123 15-07-2023 11:53

Quote:

Originally Posted by Enluaphelis (Post 501589)
Pure (2008)



4.90 gb -------> 2.65 gb

Pure uses LZSS like compression.

You may try to make a plugin for it.

https://aluigi.altervista.org/bms/splitsecond.bms

Enluaphelis 16-07-2023 03:44

Quote:

Originally Posted by Hexagon123 (Post 501593)
Pure uses LZSS like compression.

You may try to make a plugin for it.

https://aluigi.altervista.org/bms/splitsecond.bms

Will it benefit in terms of compression ratio? I don't quite understand what is the reason of it and moreover how to make a plugin, I'm beginner:D, using methods present in DiskSpanGUI

Danziel123 17-07-2023 05:58

Hi, maybe someone know what compress method for dead island 2? Thank you:)

Wanterlude 17-07-2023 10:05

Quote:

Originally Posted by Danziel123 (Post 501612)
Hi, maybe someone know what compress method for dead island 2? Thank you:)

unreal plugin(ue5)+kraken+srep/dedup+lolz/lzma
45.8 gb (orig. size)=>108 gb(kraken+unreal.dll)=>38.3 gb(dedup+lolz)
70 mins installation (11400F)

key:
Code:

0x014AEC0148FBDEE9640633AAB67521AAB4E1083A98F76FF33DDCF78DAD05BE66

Danziel123 17-07-2023 10:13

Quote:

Originally Posted by Wanterlude (Post 501613)
unreal plugin+kraken+srep/dedup+lolz/lzma
45.8 gb (orig. size)=>108 gb(kraken+unreal.dll)=>38.3 gb(dedup+lolz)
70 mins installation (11400F)

It using UE4/5?

Wanterlude 17-07-2023 10:14

Quote:

Originally Posted by Danziel123 (Post 501614)
It using UE4/5?

use ue5
ue5:k0x014AEC0148FBDEE9640633AAB67521AAB4E1083A98F 76FF33DDCF78DAD05BE66

Snake288 17-07-2023 11:41

Dead Island 2 I'm trying to compress the game
 
Hello can you help me

Dead Island 2 I'm trying to compress the game
DiskSpan_GUI Dead Island 2 I'm trying to compress the game
What should the correct settings be?

Picture -1
https://i.hizliresim.com/z2klql0.jpg
Picture -2
https://i.hizliresim.com/bet1crt.jpg[/url

Danziel123 17-07-2023 19:58

Quote:

Originally Posted by Wanterlude (Post 501615)
use ue5
ue5:k0x014AEC0148FBDEE9640633AAB67521AAB4E1083A98F 76FF33DDCF78DAD05BE66

Thank you for the info but when i try to compress the game it show error. Maybe you know what cause it? (arledy download unreal engi0ne plugin in DSG)
http://i.ibb.co/b60BvPy/Di-2.png
http://i.ibb.co/ysfD3mv/DI-2-1.png

ScOOt3r 17-07-2023 20:02

You should try and remove the space in the ue5 line.. no space when you compress. also this game uses oo2core7 file as well in conjunction with it but i dont know diskspan at all as i dont use it for compressions.

Danziel123 17-07-2023 20:12

Quote:

Originally Posted by ScOOt3r (Post 501618)
You should try and remove the space in the ue5 line.. no space when you compress. also this game uses oo2core7 file as well in conjunction with it but i dont know diskspan at all as i dont use it for compressions.

Sorry, where the space in the ue5 line?

ScOOt3r 17-07-2023 20:15

ue5:k0x014AEC0148FBDEE9640633AAB67521AAB4E1083A98F 76FF33DDCF78DAD05BE66

Danziel123 17-07-2023 20:25

Quote:

Originally Posted by ScOOt3r (Post 501620)
ue5:k0x014AEC0148FBDEE9640633AAB67521AAB4E1083A98F 76FF33DDCF78DAD05BE66

I see no space in the ue5 line:confused:
http://i.ibb.co/yWmbSVw/Di-2-3.png
http://i.ibb.co/TrLjkDY/DI-2-4.png


All times are GMT -7. The time now is 13:04.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2026, vBulletin Solutions Inc.
FileForums @ https://fileforums.com