FileForums

FileForums (https://fileforums.com/index.php)
-   Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)
-   -   Best Compression Methods for 'Specific' Games. Q&A (https://fileforums.com/showthread.php?t=99554)

KaktoR 23-05-2024 07:08

Quote:

Originally Posted by Masquerade (Post 504601)
^^
With that amount of zeroes, that is almost certainly not the correct AES key for this game (or any game).

Run the oodle scanner over the pak or ucas file and you will see streams appear.

Code:

mmermaid,l7:mkraken,l7:9:mzlib
This is where things are going wrong. Just use kraken and mermaid and zlib algos without any extra arguments.

It is. Take "Lords of the Fallen" as an example, where the key looks funny aswell
Code:

0x0200000003000000020000000200000002000000020000000300000003000000
With that key you will get almost all streams and output is better than without. I was surprised aswell.

Furthermore if using DSG, you have to set correct oodle library as parameter (which is just a fake parameter for xtool). In this case the correct parameter should be as follow:
Code:

:core_2.9.10
But I said you have to use the other v2.9.10 library, and even in this case there is a fake parameter possible, namely
Code:

:core_2.9.10:lib_2
which just tells DSG which library to copy near xtool.exe for compressing/decompressing (in this case "oo2core_9_win64_2.dll", which will be renamed to "oo2core_9_win64.dll" from "core_2.9.10" folder).

Edit: I still prefer to run multiple instances for xtool for different codecs because sometimes things will get broken if you use -d1 option and more than one or two methods, at least with current public xtool version and my testing (archive data corrupted and things like that). But honestly I don't know if this has anything to do with xtool or srep or DSG.

In this case I would use something like this for DSG
Code:

xtool:d1:mue5,k0x09000A000A000A000B000C000D000D000D000E000E000F001000120012001200:mmermaid:mkraken:core_2.9.10:lib_2+xtool:mzlib+xtool:dd3+...
Edit 2:
mermaid+kraken / ue5<key>+mermaid+kraken
Code:

Streams: 770978 / 770992
Time: 00:42:03 (CPU 03:21:13)

Size: 29.8 GB >> 51.2 GB

Done!!!

Streams: 865592 / 877159
Time: 00:45:44 (CPU 03:40:22)

Size: 29.8 GB >> 51.9 GB

Done!!!

Did not tested with zlib

Gillian 07-06-2024 09:31

Quote:

Originally Posted by KaktoR (Post 469315)
Here a few others

Final Fantasy XII: The Zodiac Age
ztool pzlib:m2:st50p:t4:c32m:rx:x+srep+lzma:a1:mfbt4:d25 6m:fb273:mc1000000000:lc8
Size: 19.4 GB
Nothing ripped


Final Fantasy Type-0 HD
ztool pzlib:m2:st50p:t4:c32m:rx:x+srep+lzma:a1:mfbt4:d25 6m:fb273:mc1000000000:lc8
Size: 16.3 GB
Nothing ripped

Wrong GAME name on this one needs redoing.

KaktoR 07-06-2024 09:53

Quote:

Originally Posted by Gillian (Post 504730)
Wrong GAME name on this one needs redoing.

Don't know what you talking about, but fixed.

Fak Eid 07-06-2024 13:25

@Kaktor
Some of my files for Starfield .ba2 keeps erroring out during decompression. Can you help?

Like: Starfield - Terrain01.ba2, Starfield - Textures01.ba2, etc.
I use DSG 2.0.2.0
Method: mstarfield+mreflate+dd+lolz

Should I be compressing them separately?

KaktoR 07-06-2024 13:32

Quote:

Originally Posted by Fak Eid (Post 504750)
@Kaktor
Some of my files for Starfield .ba2 keeps erroring out during decompression. Can you help?

Like: Starfield - Terrain01.ba2, Starfield - Textures01.ba2, etc.
I use DSG 2.0.2.0
Method: mstarfield+mreflate+dd+lolz

Should I be compressing them separately?

Do not compress them separately, as you will miss many duplicates.

Use starfield+zlib+reflate instead. All streams which cannot be processed by zlib will processed by reflate instead so "wrong" streams will processed with the correct method.

However in those cases always make testing with a single file, just don't waste your time. If you get errors on a specific file than take that "Starfield - Terrain01.ba2" file to some folder and use starfield+reflate and on a second test starfield+zlib+reflate and test decompression.

I could go even further and tell you to see what specific stream is making problems and extract it from the file, then test things with that specific stream but this would go a bit far for this case I guess.

Fak Eid 07-06-2024 14:07

Quote:

Originally Posted by KaktoR (Post 504751)
Do not compress them separately, as you will miss many duplicates.

Use starfield+zlib+reflate instead. All streams which cannot be processed by zlib will processed by reflate instead so "wrong" streams will processed with the correct method.

However in those cases always make testing with a single file, just don't waste your time. If you get errors on a specific file than take that "Starfield - Terrain01.ba2" file to some folder and use starfield+reflate and on a second test starfield+zlib+reflate and test decompression.

I could go even further and tell you to see what specific stream is making problems and extract it from the file, then test things with that specific stream but this would go a bit far for this case I guess.

I'm using the plugin on the DSG, and verified that it already has zlib dll. Should i use that same plugin and external zlib dll too?
Like: mstarfield (this already has zlib) + mzlib + mreflate?

Whatever you said I'll try and DM tomorrow in case of help.

KaktoR 08-06-2024 01:54

Yes, for DSG it is "xtool:mstarfield+xtool:mzlib:mreflate:dd[1..5]" then.

Fak Eid 10-06-2024 01:28

Quote:

Originally Posted by KaktoR (Post 504758)
Yes, for DSG it is "xtool:mstarfield+xtool:mzlib:mreflate:dd[1..5]" then.

@kaktor

For file: Starfield - LODTextures.ba2
The compression method with xtool:mstarfield plugin is failing.
Should I be compressing it with zlib+preflate only? Or seperately?

KaktoR 10-06-2024 01:59

Quote:

Originally Posted by Fak Eid (Post 504775)
@kaktor

For file: Starfield - LODTextures.ba2
The compression method with xtool:mstarfield plugin is failing.
Should I be compressing it with zlib+preflate only? Or seperately?

What exact method you are using? Works fine here

Code:

10:50:30 - Selected ARC/DS method for Data1a-01.bin was: xtool:mstarfield+xtool:mzlib:mreflate

---------------------------------------------------------------------------------------------------------------------------------------------------------
 Creating Data1a-01.bin file, Please Wait...
---------------------------------------------------------------------------------------------------------------------------------------------------------
FreeArc 0.67 (March 15 2014) creating archive: Conversion_Output\Data1a-01.bin
Compressed 1 file, 4,220,084,222 => 9,653,812,057 bytes. Ratio 228.76%
Compression time: cpu 3.98 sec/real 293.03 sec = 1%. Speed 14.40 mB/s
All OK

---------------------------------------------------------------------------------------------------------------------------------------------------------
 Extracting Data1a-01.bin file, Please Wait...
---------------------------------------------------------------------------------------------------------------------------------------------------------
FreeArc 0.67 (March 15 2014) extracting archive: Conversion_Output\CONVERSION\Data1a-01.bin
Extracted 1 file, 9,653,812,057 => 4,220,084,222 bytes. Ratio 228.76%
Extraction time: cpu 6.53 sec/real 143.69 sec = 5%. Speed 29.37 mB/s
All OK


wareck 14-06-2024 07:14

hello
I've got some question, its about "terminator dark fate"
the is a file called "basis.pak".
I'ts a PKZIP file (zip 2.0 with a password)
this file contains several files (animation,textures,sound ect)

In this file I extracted with sfk/partcopy a block from byte 882682388 to 4507627948.

I obtain a new file called sound.pak with is aroung 3.7Go.
it's again a PKZIP file with the same password of the big file...

I would like to know how to compress it , my first idea was to use xtool/flac or msc, but seems to not working (probably because password and packed)

I think the right way is to uncompress/decode compress then during installation use batch with sfk to re inject sound in basis.pak ...

thanks for your help

KaktoR 14-06-2024 08:48

Quote:

Originally Posted by wareck (Post 504799)
hello
I've got some question, its about "terminator dark fate"
the is a file called "basis.pak".
I'ts a PKZIP file (zip 2.0 with a password)
this file contains several files (animation,textures,sound ect)

In this file I extracted with sfk/partcopy a block from byte 882682388 to 4507627948.

I obtain a new file called sound.pak with is aroung 3.7Go.
it's again a PKZIP file with the same password of the big file...

I would like to know how to compress it , my first idea was to use xtool/flac or msc, but seems to not working (probably because password and packed)

I think the right way is to uncompress/decode compress then during installation use batch with sfk to re inject sound in basis.pak ...

thanks for your help

password
Code:

oKoo$]bnGTKJLMNBA9A
Thanks to masquerade

PS: The password is stored as plain text in trm.exe
https://i.imgur.com/fF5pLvn.png

wareck 14-06-2024 09:05

yes I have already found the password.
Now I try to find a way to compress/decompress the separate pak files

I need to unpack zip, compress with arc then make a batch to rebuild files likes on the game inital structure ?

KaktoR 14-06-2024 09:55

I would try to compress the files with zip with same zip settings and password, then make diff patch to patch files back to original. Not sure if this will work, could be difficult because of the password protection.

wareck 14-06-2024 10:31

yes it's more or less what I tried

In fact this file, basis.pak was uses as a mounted drive/partition
There is a structure like if they done an usb/harddrive image.

I'm starting a test, maybe the game do not check the crc of this file.
in this case probably I may unpack/repack without problem

I'll give an update this evening

Masquerade 14-06-2024 12:42

Game runs with unpacked data.

wareck 14-06-2024 22:40

Ok, it works
it's not a realy nice way to pack/unpack but it works
I think the problem is when use zip with password, it introduce some random number in header and data of the final zip
So there is no easy way to use xdelta or hdiff to rebuild original file

what i've done for v1.04.987:
uncompress basis.pak in a temporary folder
uncompress patch_003.pak in temporary folder
uncompress patch_005.pak in temporary folder
remove basis.pak patch_003.pak patch_005.pak

pack with diskspangui

then in my unpack batch, in the temporary folder :
.\7z.exe -mx5 -tzip -spe -poKoo$]bnGTKJLMNBA9A a patch_003.pak .\patch_003\*
.\7z.exe -mx5 -tzip -spe -poKoo$]bnGTKJLMNBA9A a patch_005.pak .\patch_005\*
.\7z.exe -mx5 -tzip -spe -poKoo$]bnGTKJLMNBA9A a basis.pak .\basis\*
move basis.pak ..\
move patch_005.pak ..\
move patch_003.pak ..\
delete temporay folder

I'm around 10go compressed game with setup.exe
game works without issue

LILU 17-06-2024 17:42

Sker Ritual v 1.0.2.20980
Code:

22:14:35 - You have chosen the DVD5 type for the media size.
 22:14:35 - ISO images will not be created when conversion is complete.
 22:14:35 - The computer will remain on when the conversion is complete.
 22:14:35 - Selected ARC/DS method for Data1a-01.bin was: xtool:c32mb:munity+4x4:b96mb:lzma:ultra:bt4:fb273:lc8:mc1000000:96mb
----------------------------------------------------------------------------------------------------------------------------------------------------------
 22:14:36 - Compression initialized in 17.06.2024 at 22:14:36.
 22:14:36 - Starting Data1a-01.bin archive creation.
 22:33:58 - Moving Data1a-01.bin » Data1a-02.bin files to media folder.
 22:33:58 - Compression finished in 17.06.2024 at 22:33:58.
 22:33:58 - Updating new files into the Setup.dll file.
----------------------------------------------------------------------------------------------------------------------------------------------------------
 22:33:59 - Getting size of the game: Sker Ritual
 22:33:59 - » Installation folder size: 19,95 GB
 22:33:59 - » Compressed game size: 8,41 GB (Ratio 42.13%)
----------------------------------------------------------------------------------------------------------------------------------------------------------
 22:33:59 - Overall input size: 19,95 GB
 22:33:59 - Overall output size: 8,41 GB (Ratio 42.13%)
 22:33:59 - Overall conversion time: 00:19:22
 22:33:59 - Overall conversion discs: 2 DVD5

Code:

17:11:32 - Deleting files from the output folder.
 17:11:33 - Files have been deleted.
----------------------------------------------------------------------------------------------------------------------------------------------------------
 17:11:33 - You have chosen the DVD5 type for the media size.
 17:11:33 - ISO images will not be created when conversion is complete.
 17:11:33 - The computer will remain on when the conversion is complete.
 17:11:33 - Selected ARC/DS method for Data1a-01.bin was: xtool:c32mb:munity+lolz:dtb1:d77:mtt1:mt12:mc1023
----------------------------------------------------------------------------------------------------------------------------------------------------------
 17:11:33 - Compression initialized in 18.06.2024 at 17:11:33.
 17:11:33 - Starting Data1a-01.bin archive creation.
 19:43:03 - Moving Data1a-01.bin » Data1a-02.bin files to media folder.
 19:43:03 - Compression finished in 18.06.2024 at 19:43:03.
 19:43:03 - Updating new files into the Setup.dll file.
----------------------------------------------------------------------------------------------------------------------------------------------------------
 19:43:03 - Getting size of the game: Sker Ritual
 19:43:03 - » Installation folder size: 19,95 GB
 19:43:03 - » Compressed game size: 7,44 GB (Ratio 37.30%)
----------------------------------------------------------------------------------------------------------------------------------------------------------
 19:43:03 - Overall input size: 19,95 GB
 19:43:03 - Overall output size: 7,44 GB (Ratio 37.30%)
 19:43:03 - Overall conversion time: 02:31:30
 19:43:03 - Overall conversion discs: 2 DVD5


Danziel123 19-06-2024 04:17

Hello everyone, maybe someone knows the compression method for the Crime Boss Rockay City game? this is an example file:
Ucas: https://drive.google.com/file/d/141x...usp=drive_link
Pak: https://drive.google.com/file/d/1P7Q...usp=drive_link
Thank you:)

LILU 19-06-2024 05:05

HORROR TALES: The Beggar v 14729589
Code:

----------------------------------------------------------------------------------------------------------------------------------------------------------
---------------------------- CONVERSION DATE 2024.06.19 --------------------------------------------------------
----------------------------------------------------------------------------------------------------------------------------------------------------------
 11:41:33 - Deleting files from the output folder.
 11:41:33 - Files have been deleted.
----------------------------------------------------------------------------------------------------------------------------------------------------------
 11:41:33 - You have chosen the DVD5 type for the media size.
 11:41:33 - ISO images will not be created when conversion is complete.
 11:41:33 - The computer will remain on when the conversion is complete.
 11:41:33 - Selected ARC/DS method for Data1a-01.bin was: xtool:c32mb:t50p:mkraken:mmermaid:mselkie:7:mzlib:dd3+lolz:dtb1:d94:mtt1:mt12:mc1023
----------------------------------------------------------------------------------------------------------------------------------------------------------
 11:41:33 - Compression initialized in 19.06.2024 at 11:41:33.
 11:41:33 - Starting Data1a-01.bin archive creation.
 13:05:36 - Moving Data1a-01.bin » Data1a-02.bin files to media folder.
 13:05:36 - Compression finished in 19.06.2024 at 13:05:36.
 13:05:36 - Updating new files into the Setup.dll file.
----------------------------------------------------------------------------------------------------------------------------------------------------------
 13:05:37 - Getting size of the game: HORROR TALES: The Beggar
 13:05:37 - » Installation folder size: 10,54 GB
 13:05:37 - » Compressed game size: 5,12 GB (Ratio 48.57%)
----------------------------------------------------------------------------------------------------------------------------------------------------------
 13:05:37 - Overall input size: 10,54 GB
 13:05:37 - Overall output size: 5,12 GB (Ratio 48.57%)
 13:05:37 - Overall conversion time: 01:21:38
 13:05:37 - Overall conversion discs: 2 DVD5
----------------------------------------------------------------------------------------------------------------------------------------------------------


KaktoR 19-06-2024 07:00

Quote:

Originally Posted by Danziel123 (Post 504840)
Hello everyone, maybe someone knows the compression method for the Crime Boss Rockay City game? this is an example file:
Ucas: https://drive.google.com/file/d/141x...usp=drive_link
Pak: https://drive.google.com/file/d/1P7Q...usp=drive_link
Thank you:)

zlib

Masquerade 19-06-2024 07:46

zlib+reflate gives slightly better ratio:

Code:

xtool precomp -mzlib data
XTool is created by Razor12911

Version 0.8.7

Streams: 20523 / 20974
Time: 00:00:03 (CPU 00:00:25)

Size: 406 MB >> 924 MB

Code:

xtool precomp -mzlib+reflate data
XTool is created by Razor12911

Version 0.8.7

Streams: 20974 / 20974
Time: 00:00:02 (CPU 00:00:24)

Size: 406 MB >> 927 MB


KaktoR 20-06-2024 09:49

Quote:

Originally Posted by wareck (Post 504805)
Ok, it works
it's not a realy nice way to pack/unpack but it works
I think the problem is when use zip with password, it introduce some random number in header and data of the final zip
So there is no easy way to use xdelta or hdiff to rebuild original file

what i've done for v1.04.987:
uncompress basis.pak in a temporary folder
uncompress patch_003.pak in temporary folder
uncompress patch_005.pak in temporary folder
remove basis.pak patch_003.pak patch_005.pak

pack with diskspangui

then in my unpack batch, in the temporary folder :
.\7z.exe -mx5 -tzip -spe -poKoo$]bnGTKJLMNBA9A a patch_003.pak .\patch_003\*
.\7z.exe -mx5 -tzip -spe -poKoo$]bnGTKJLMNBA9A a patch_005.pak .\patch_005\*
.\7z.exe -mx5 -tzip -spe -poKoo$]bnGTKJLMNBA9A a basis.pak .\basis\*
move basis.pak ..\
move patch_005.pak ..\
move patch_003.pak ..\
delete temporay folder

I'm around 10go compressed game with setup.exe
game works without issue

I did a deep check today and found out there are at least 1 wav file which will crash using msc or flac.

KaktoR 20-06-2024 09:56

This file will cause a crash:
Code:

basis.pak\sounds\dialogs\10_mission\10th_sintu_last_mission_1_1.wav
PS: I can't edit my previous post because forum software is bugged.

wareck 20-06-2024 12:57

yes in fact there are something like 12 wav files that can crash msc/flac
that's why I switched to wavpack which does 100% wav conversions

There's another little trick
If you make a batch script to automate file creation before compression.
some files have names in Cyrillic
So I added this command to my batch file:
Quote:

chcp 65001

wareck 20-06-2024 12:59

I can post my batch files if you want
it's not really optimized I've started to try conversion something like 2 month ago...

LILU 20-06-2024 14:00

TRAIL OUT v 3.03
Code:

10:41:58 - You have chosen the DVD5 type for the media size.
 10:41:58 - ISO images will not be created when conversion is complete.
 10:41:58 - The computer will remain on when the conversion is complete.
 10:41:58 - Selected ARC/DS method for Data1a-01.bin was: xtool:c32mb:d1:mue4,k0x0EEE0509FABDC7BA25C96DDFA7FFBF60924D969D609354809D74C07809AB5944,zstd:mzlib:mreflate:mpreflate+lolz:d77:mtt1:mt12:mc1023
 10:41:58 - Selected ARC/DS method for Data1b-01.bin was: xtool:c32mb:t90p:mzlib+4x4:b96mb:lzma:ultra:bt4:fb273:lc8:mc1000000:96mb
----------------------------------------------------------------------------------------------------------------------------------------------------------
 10:41:58 - Compression initialized in 20.06.2024 at 10:41:58.
 10:41:59 - Starting Data1a-01.bin archive creation.
 16:58:06 - Moving Data1a-01.bin » Data1a-04.bin files to media folder.
 16:58:06 - Starting Data1b-01.bin archive creation.
 16:58:41 - Moving Data1b-01.bin file to media folder.
 16:58:41 - Compression finished in 20.06.2024 at 16:58:41.
 16:58:41 - Updating new files into the Setup.dll file.
----------------------------------------------------------------------------------------------------------------------------------------------------------
 16:58:42 - Getting size of the game: TRAIL OUT
 16:58:42 - » Installation folder size: 28,52 GB
 16:58:42 - » Data1a-01.bin file size: 15,90 GB
 16:58:42 - » Data1b-01.bin file size: 115,33 MB
 16:58:42 - » Compressed game size: 16,01 GB (Ratio 56.14%)
----------------------------------------------------------------------------------------------------------------------------------------------------------
 16:58:42 - Overall input size: 28,52 GB
 16:58:42 - Overall output size: 16,01 GB (Ratio 56.14%)
 16:58:42 - Overall conversion time: 06:16:42
 16:58:42 - Overall conversion discs: 4 DVD5


Danziel123 21-06-2024 06:03

Quote:

Originally Posted by KaktoR (Post 504845)
zlib

Thank you for telling me, but why is the compression process always stuck here? I've been waiting for 4 hours and it hasn't finished
http://i.ibb.co.com/1R3NDjh/Crime.png

KaktoR 21-06-2024 06:48

See if srep.exe is in use (taskmanager)

KaktoR 21-06-2024 06:57

Quote:

Originally Posted by wareck (Post 504864)
I can post my batch files if you want
it's not really optimized I've started to try conversion something like 2 month ago...

Would be good for general information for other users, so if you want just post it ;)

Danziel123 21-06-2024 07:14

Quote:

Originally Posted by KaktoR (Post 504870)
See if srep.exe is in use (taskmanager)

It doesn't work (down to 0) and I've tried using srep as well as stuck, can you help me analyze this game?

KaktoR 21-06-2024 10:50

Use only mzlib and see what happens

Danziel123 21-06-2024 23:43

Quote:

Originally Posted by KaktoR (Post 504875)
Use only mzlib and see what happens

still the same, have tried from mzlib to xt_zlib still stuck:confused:
http://i.ibb.co.com/DVYBZz6/Crime-boss.png
maybe someone can help?

mausschieber 22-06-2024 04:50

1 Attachment(s)
Here it works fine

Danziel123 22-06-2024 06:12

Quote:

Originally Posted by mausschieber (Post 504882)
Here it works fine

Try compressing using the method
Quote:

xtool:mzlib:mreflate:dd3+4x4:b64mb:lzma:ultra:bt4: fb273:lc8:mc1000000:64mb

KaktoR 22-06-2024 06:47

I see srep64. See if srep is working normally. Otherwise try with +srep+ instead of :dd3

Danziel123 22-06-2024 08:08

Quote:

Originally Posted by KaktoR (Post 504885)
I see srep64. See if srep is working normally. Otherwise try with +srep+ instead of :dd3

I've tried using srep new but it's still stuck, should I use it lolz?

KaktoR 22-06-2024 11:16

Must be something on your end. Works fine here

Code:

16:32:22 - Selected ARC/DS method for Data1a-01.bin was: xtool:mzlib:mreflate:dd3:l10,d128mb
-----------------------------------------------------------------------------------------------------------------------------------------
 19:48:47 - Overall input size: 86.49 GB
 19:48:47 - Overall output size: 60.39 GB (Ratio 69.82%)
 19:48:47 - Overall conversion time: 03:16:23


Danziel123 23-06-2024 22:41

Quote:

Originally Posted by KaktoR (Post 504888)
Must be something on your end. Works fine here

Code:

16:32:22 - Selected ARC/DS method for Data1a-01.bin was: xtool:mzlib:mreflate:dd3:l10,d128mb
-----------------------------------------------------------------------------------------------------------------------------------------
 19:48:47 - Overall input size: 86.49 GB
 19:48:47 - Overall output size: 60.39 GB (Ratio 69.82%)
 19:48:47 - Overall conversion time: 03:16:23


I've tried moving it to another drive and trying to compress using: xtool:mzlib:mreflate:dd3+4x4:b64mb:lzma:ultra:bt4: fb273:lc8:mc1000000:64mb and it's still stuck and tried your method and it managed to compress to completion but When you want to test install an error appears with code -11:(

Edit: Finally, changing from xtool:mzlib to xt_zlib can also be installed until it's finished

wareck 24-06-2024 13:28

hello daniel , it's for what game ?
I've got some free time tonight to try

mausschieber 24-06-2024 13:35

Quote:

Originally Posted by wareck (Post 504906)
hello daniel , it's for what game ?
I've got some free time tonight to try

i think its for Crime Boss Rockay City


All times are GMT -7. The time now is 04:34.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2026, vBulletin Solutions Inc.
FileForums @ https://fileforums.com