FileForums - Best Compression Methods for 'Specific' Games. Q&A

Page 81 of 119

« First

Last »

Show 40 post(s) from this thread on one page

FileForums (https://fileforums.com/index.php)

- Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)

- - Best Compression Methods for 'Specific' Games. Q&A (https://fileforums.com/showthread.php?t=99554)

Edison007

05-10-2022 14:18

Quote:

Came back to this one after a bit of a break and after learning about xtool's "verbose" mode, turns out LZO is used on the archives, however it seems that no matter what level depth setting or library I use no streams are able to be processed, does anyone know the reason behind this? Attached is an image supporting my findings.

No, in the game does not use LZO.
Тhis is a private(?) LZ algorithm. You can read this thread: https://forum.xentax.com/viewtopic.php?f=10&t=9481

Quote:

hi, how to TEXTURE Rebuild and compression .dds file use "crunch" ?

crunch = lossy, not suitable for repacking;)

L33THAK0R

05-10-2022 16:39

Quote:

Originally Posted by Edison007 (Post 498629)

No, in the game does not use LZO.
Тhis is a private(?) LZ algorithm. You can read this thread: https://forum.xentax.com/viewtopic.php?f=10&t=9481

I remember reading through that entire thread when I was first researching this bugger of a title, had a hope perhaps the lads over at the Xentax forums had been wrong, and that the PMCQ algorithim was just a fork of LZO. I actually managed to talk to Ekey (a very talented developer who's quite experienced with developing various decompression suites for a variety of titles), but he unfortunately lost his source code for his Sleeping Dogs decompression tool. I'm not sure if you'd be able to answer this but do you know why xtool might be reporting lzo1x streams? Is this just as simple as being an error on xtool's part? Honestly might try again to adapt his original decompression function (which we still thankfully have the source code for) as an xtool plugin, but given my lack of success in previous attempts I doubt I'll make much progress.

Edison007

06-10-2022 01:11

Quote:

ad a hope perhaps the lads over at the Xentax forums had been wrong, and that the PMCQ algorithim was just a fork of LZO.

No :) This is a different algorithm.

Quote:

Is this just as simple as being an error on xtool's part?

Yep. False positive detect.

Quote:

Honestly might try again to adapt his original decompression function

You also need a compress function. ;)

L33THAK0R

06-10-2022 05:51

Quote:

Originally Posted by Edison007 (Post 498633)

No :) This is a different algorithm.

Drat! It's always bizarre how complex some titles can be to properly process!

Quote:

Originally Posted by Edison007 (Post 498633)

You also need a compress function. ;)

Thats going to be a fun time trying to figure out, heres hoping its lying around somewhere on the web!

kj911

07-10-2022 09:07

Best compression methods from Stalker: Shadow of Chernobyl?? The game compressibble in smaller than 2-2.5GB's size? (with or without MP files. Multilang version installed size in has ~9.5GB's. Installer smaller than ~3.2GB with INNO only.)

Tested one gamedata archive file (*.db5), in few methods. This db5 archive contents extracted in specialized tools and extracted files compressed again into separated 3 archives and results its bigger than compress the one *.db5 file. Not understood!

File: gamedata.db5
Size: 677 500kB

CAB/LZX:21: 520 571kB
7z: 498 055kB
7z/PPMD: 529 581kB
SQX2: 492 547kB
DLZ only: 456 021kB
OGGRE+DLZ: 462 367kB

MSC only (BMP/DDSRAW/DXT=1): 550 356kB* (archive broken! Any data portions its missing??) --> Crunch in DLZ: 387 155kB (probably false...)
MSC only (DDSRAW=0, DXT=1): 677 377kB --> Crunch in DLZ: 478 327kB

Extracted the db5 file:

*.OGM: 68 290kB (6 file) --> DLZ: 59 064kB (Note: OGGRE+DLZ: 59 061kB)
*.OGG: 174 445kB (2052 file) --> OGGRE: 146 989kB
*.DDS: 446 886kB (1249 file) --> MSC+DLZ: 317 108kB / DLZ: 294 630kB
other: 908kB (17 file) --> MSC+DLZ: 103kB
__________________
Total: 690 528kB --> 523 264kB / 500 786kB (WTF!)

Masquerade

11-10-2022 11:47

Overcooked! 2

Code:

xtool_unity+srep+lolz

7.9GB ----> 3.7GB

liblz4 v1.8.0 used for best ratio.

Masquerade

11-10-2022 11:49

Prison Architect

Paradox Launcher files ripped.
All DAT files unpacked with WinRAR.

Code:

precomp:d0+srep+lolz/$ogg=oggre

670MB ---> 306MB

Thanks Nicolas Comerci for precomp STDIO fork.

L33THAK0R

12-10-2022 10:50

Grim Dawn LZ4 dropping streams

Hey all,

I've been giving "Grim Dawn" a looksie and was wondering if anyone might be able to clear something up

Code:

FreeArc 0.67 (March 15 2014) creating archive: Data.arc

Compressing 1 file, 145,766,328 bytes. Processed   0%

Compressing 145,766,328 bytes with "C:\Compressor_Resources\XTool_2020\xtool_0.6.5\XTool.exe" precomp -mgrimTEST -c512mb -t95p $$arcdatafile$$.tmp $$arcpackedfile$$.tmp

XTool is created by Razor12911



Streams: 1396/14028

Time: 00:00:10 (00:00:36)

Memory: 160 MB (160 MB)



100%

Errorlevel=0

Compressed 1 file, 145,766,328 => 160,143,671 bytes. Ratio 109.86%

Compression time: cpu 0.09 sec/real 10.60 sec = 1%. Speed 13.75 mB/s

All OK

I've tried levels 1 - 9 so far and I only get maybe ~1 additional stream for each level increment, and I've tested essentially every version of LZ4 under the sun, (other versions only resulting in ~<100 streams being picked up), would anyone happen to know if its possible to pick up more streams or is it more likely that LZ4 was used very lightly and the other streams are inconsequential.

KaktoR

12-10-2022 14:23

Sample?

L33THAK0R

12-10-2022 18:21

1 Attachment(s)

Quote:

Originally Posted by KaktoR (Post 498703)

Sample?

Ah whoops I forgot to attach it, within the following attachment is a sample, a BMS script to decompress "Grim Dawn"'s assets (since the native LZ4 stream detection fails to pick up any at all), as well as the LZ4 library I've found success with.

L33THAK0R

18-10-2022 02:39

Ghost Recon Wildlands recompression woes

Hi all has anyone else had any issues with recompressing "Ghost Recon Wildlands" .forge archives? I've tried the following:

XTool 0.6.5 (Anvil) + Srep + Lolz

Code:

74,326,347,887 bytes -> 20,662,955,336 bytes

as well as

AFR v0.0.1.9 (v0.0.2.0a7 detects and seemingly picks up streams, but spits out an error code and fails to write to disk) + srep + lolz

Code:

74,326,347,887 bytes -> 21,147,452,370 bytes

I've had decent success with previous Anvil Engine titles (save for "Assassin's Creed III" (the original, I'm currently testing remastered)), however it seems like Wildlands is giving me a whole lot of strife. Attempting to decompress the XTool-based test yields an almost immediate installer crash, as does attempting to decompress the AFR test when using the CLS plugin or AFR v0.0.1.9 (the latter at least offers a "-11" error code instead of an unceremonious CTD) , however attempting to use v0.0.2.0a7 results in a partially successful decompression, however failure occurs around the halfway point. Has anyone else experienced any similar behavior, at this point I'm a bit reluctant to attempt any further attempts due to the write endurance costs resulting from this behemoth of a title.

Masquerade

18-10-2022 03:39

I did Wildlands either last year or year before and had no issues with AFR.

KaktoR

18-10-2022 04:02

A Plague Tale Requiem

*.dpc -> zlib

Masquerade

18-10-2022 04:15

Quote:

Originally Posted by KaktoR (Post 498770)

A Plague Tale Requiem

*.dpc -> zlib

Different voice packs are all contained in VO_MEDIA.PC.PCK - use WWise PCK BMS script from here to extract and then use XTool erase command to remove unneeded voice packs.

KaktoR

18-10-2022 04:44

Does not work here, or lets say only partial.

Seems like xtool just erase some files, as I can see in hex editor filled sectors with zeros. But if I extract the archive once again, all files are still there.

Maybe I do it just wrong lol

Edit: Oh ok, the extracted files are zero'ed, now I got it.

Edit 2: Forget my prevous edit. Seems like this one worked correctly (before I was using wildcards for the folders/files)

Code:

xtool erase "VO_MEDIA.PC.PCK_extract\chinese(simplified)" "VO_MEDIA.PC.PCK" test.xtl

xtool erase "VO_MEDIA.PC.PCK_extract\french(france)" "VO_MEDIA.PC.PCK" test.xtl

xtool erase "VO_MEDIA.PC.PCK_extract\german" "VO_MEDIA.PC.PCK" test.xtl

xtool erase "VO_MEDIA.PC.PCK_extract\spanish(spain)" "VO_MEDIA.PC.PCK" test.xtl

L33THAK0R

18-10-2022 06:21

Quote:

Originally Posted by Masquerade (Post 498769)

I did Wildlands either last year or year before and had no issues with AFR.

I'm gonna give the EMPRESS release a shot, was originally basing my tests off the Steampunks release, fingers crossed I have better luck with it, out of curiosity, do you happen to remember which version of AFR you used?

Masquerade

18-10-2022 06:22

Quote:

Originally Posted by L33THAK0R (Post 498774)

do you happen to remember which version of AFR you used?

Latest Edison version. P.S. don't discuss Warez on this forum - it's against the rules!

Edison007

18-10-2022 08:14

Quote:

Originally Posted by L33THAK0R (Post 498774)

AFR versions above 019 are unstable. Crashed when using mt-unpacking.

Razor12911

18-10-2022 08:59

Quote:

Originally Posted by L33THAK0R (Post 498767)

Code:

unpackcmd = xtool.exe decode --verbose $$arcpackedfile$$.tmp $$arcdatafile$$.tmp

Try xtool in verbose mode to see what error message it produces.

I have tested this game once before and found no errors having only lzo2.dll in directory with xtool.exe

DataPC.forge

Code:

Tested 1 file, 10,925,907,848 => 8,042,348,544 bytes. Ratio 135.85%

Testing time: cpu 8.38 sec/real 238.37 sec = 4%. Speed 33.74 mB/s

Cesar82

18-10-2022 09:47

Quote:

Originally Posted by Razor12911 (Post 498779)

Code:

unpackcmd = xtool.exe decode --verbose $$arcpackedfile$$.tmp $$arcdatafile$$.tmp

Try xtool in verbose mode to see what error message it produces.

I have tested this game once before and found no errors having only lzo2.dll in directory with xtool.exe

DataPC.forge

Code:

Tested 1 file, 10,925,907,848 => 8,042,348,544 bytes. Ratio 135.85%

Testing time: cpu 8.38 sec/real 238.37 sec = 4%. Speed 33.74 mB/s

I don't know if the extraction done by @L33THAK0R is being done on Windows 11, but I couldn't get lzo2.dll to work on windows 11, maybe I need a recompilation of the lzo2.dll library to make it compatible with Windows 11.
Also the old library zlibwapi.dll (from the old xtool) doesn't work, however the new library zlibwapi.dll shared with the new xtool works perfectly with Windows 11.

fabrieunko

18-10-2022 11:55

Quote:

Originally Posted by Masquerade (Post 498771)

Different voice packs are all contained in VO_MEDIA.PC.PCK - use WWise PCK BMS script from here to extract and then use XTool erase command to remove unneeded voice packs.

can you tell me if i think i'm doing it right?

I launch quickbms, I open the WWise.bms file then the VO_MEDIA.PC.PCK file

it is extracted from wma files in directories (chinese(simplified)/english(us)/french(france)/german/spanish(spain)

if i want to delete the german i type in a back window (from the post of KaktoR)

xtool erase "VO_MEDIA.PC.PCK_extract\german" "VO_MEDIA.PC.PCK" test.xtl

KaktoR

18-10-2022 12:06

Quote:

Originally Posted by fabrieunko (Post 498781)

Make batch files for this, so you don't have to type all manually in cmd.

unpack.bat for quickbms

Code:

for %%f in (*.pck) do quickbms_4gb_files.exe -d -o -Y wwise.bms "%%f"

pause

xtool_erase.bat for xtool

Code:

xtool erase "VO_MEDIA.PC.PCK_extract\german" "VO_MEDIA.PC.PCK" test.xtl

This will delete all german voice files from the file.

PS: Make a backup of the file first ;)

L33THAK0R

18-10-2022 19:02

Quote:

Originally Posted by Cesar82 (Post 498780)

I couldn't get lzo2.dll to work on windows 11, maybe I need a recompilation of the lzo2.dll library to make it compatible with Windows 11.

Quote:

Originally Posted by Razor12911 (Post 498779)

[CODE]
I have tested this game once before and found no errors having only lzo2.dll in directory with xtool.exe

I'm so sorry for wasting your time lads, bit of backstory, I use a separate ASIS installation to test my archives, I'd just finished wrapping up my "Assassin's Creed" "Odyssey/Origins" compressions and was using the same setup.dll which didn't contain lzo2.dll, after updating the setup.dll archive decompression was successful with my xtool-based test.

Masquerade

19-10-2022 09:47

Uncharted Legacy of Thieves Collection Precompression

Using the oo2core_9_win64.dll from game folder:

Code:

Compressing animtex0.psarc, 47,187,477 bytes



xtool v0.6.5 (-mkraken)



Compressed 1 file, 47,187,477 => 180,655,223 bytes. Ratio 382.85%

Compression time: cpu 0.05 sec/real 6.76 sec = 1%. Speed 6.98 mB/s

All OK



xtool v0.6.5 (-mkraken:n256)



Compressed 1 file, 47,187,477 => 183,093,362 bytes. Ratio 388.01%

Compression time: cpu 0.06 sec/real 11.78 sec = 1%. Speed 4.01 mB/s

All OK



oo2reck (rename oo2core_9_win64.dll to oo2core_7_win64.dll)



Compressed 1 file, 47,187,477 => 183,176,064 bytes. Ratio 388.19%

Compression time: cpu 0.08 sec/real 15.15 sec = 1%. Speed 3.11 mB/s

All OK

dixen

19-10-2022 15:21

Quote:

Originally Posted by Masquerade (Post 498789)

Uncharted Legacy of Thieves Collection Precompression
Using the oo2core_9_win64.dll from game folder:

And zlib also:D

XTool v0.53 (zlib+kraken)

Quote:

Compressing 4 files, 58,792,062 bytes. Processed 0%
Compressing 58,792,062 bytes with xtool.exe precomp -mzlib+reflate+kraken -c128mb -t100p $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
XTool is created by Razor12911

Streams: 6445/6445
Time: 00:00:50 (00:00:49)
Memory: 331 MB (331 MB)

100%
Errorlevel=0
Compressed 4 files, 58,792,062 => 211,162,930 bytes. Ratio 359.17%
Compression time: cpu 0.02 sec/real 50.73 sec = 0%. Speed 1.16 mB/s
All OK

XTool v0.53 (only kraken)

Quote:

Compressing 4 files, 58,792,062 bytes. Processed 0%
Compressing 58,792,062 bytes with xtool.exe precomp -mkraken -c128mb -t100p $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
XTool is created by Razor12911

Streams: 6370/6370
Time: 00:00:45 (00:00:44)
Memory: 298 MB (298 MB)

100%
Errorlevel=0
Compressed 4 files, 58,792,062 => 197,760,883 bytes. Ratio 336.37%
Compression time: cpu 0.00 sec/real 46.67 sec = 0%. Speed 1.26 mB/s
All OK

Wanterlude

20-10-2022 07:51

Uncharted Legacy of Thieves Collection (v.1.0.20122 | English/Russian lang.)

Code:

XTool (zlib+kraken*)+srep+lolz



Compressed 81 files, 106,547,733,553 => 51,633,688,075 bytes. Ratio 48.46%

Compression time: cpu 1323.59 sec/real 33653.46 sec = 4%. Speed 3.17 mB/s

Overall conversion time: 09:20:55 (i5 11400F)



99.23 GB => 48.09 GB

*Use oo2core_9_win64.dll from game folder

fabrieunko

20-10-2022 09:05

Quote:

Originally Posted by Wanterlude (Post 498801)

Uncharted Legacy of Thieves Collection (v.1.0.20122 | English/Russian lang.)

Code:

XTool (zlib+kraken*)+srep+lolz



Compressed 81 files, 106,547,733,553 => 51,633,688,075 bytes. Ratio 48.46%

Compression time: cpu 1323.59 sec/real 33653.46 sec = 4%. Speed 3.17 mB/s

Overall conversion time: 09:20:55 (i5 11400F)



99.23 GB => 48.09 GB

*Use oo2core_9_win64.dll from game folder

how to use your :Use oo2core_9_win64.dll with dsg?

KaktoR

20-10-2022 10:42

Quote:

Originally Posted by fabrieunko (Post 498804)

how to use your :Use oo2core_9_win64.dll with dsg?

DSG contains fake parameters for xtool. Use "xtool:mkraken:9+xtool:mzlib" to use oo2core_9_win64.dll library.

KaktoR

20-10-2022 14:14

Anyone knows what compression "The Valiant" is using? unity lz4/lz4hc does not seem to have any effect.

Sample

Masquerade

20-10-2022 14:22

Quote:

Originally Posted by KaktoR (Post 498811)

Anyone knows what compression "The Valiant" is using? unity lz4/lz4hc does not seem to have any effect.

Sample

I believe it's LZ4, but Razor might need to take a look at this one because the unity plugin could be bugged.

shazzla

21-10-2022 21:00

Hi!

Anyone knows what is the correct LZ4 library for COD : Infinite warfare ?

Masquerade

23-10-2022 02:54

Treasures of the Aegean

Code:

xtool_unity+srep+lolz

1.5GB ---> 850MB

liblz4 v1.8.3 used.

KaktoR

25-10-2022 15:18

Mount & Blade II Bannerlord

*.bin -> reflate

Code:

8,471,456,167 bytes -> 141,154,813,359 bytes

*.gtp|*.tpac not checked yet

Masquerade

26-10-2022 01:00

@KaktoR
TPAC compressed with LZ4. XTool plugin probably required for precompression.

exorc

26-10-2022 06:53

Quote:

Originally Posted by Masquerade (Post 496967)

Dolmen

*.PAK

Code:

oo2recm+srep+lolz
*.MP4

Code:

srep
Everything else:

Code:

srep+razor

Use this oodle core.

7.3GB ---> 5.7GB [Game + 60FPS + 30FPS videos]

6.8GB ---> 5.1GB [Game + 60 FPS videos]

6.6GB ---> 5GB [Game + 30 FPS videos]

url expired, can you reupload?

thanks.

Masquerade

26-10-2022 07:06

Quote:

Originally Posted by exorc (Post 498865)

url expired, can you reupload?

thanks.

Done, aded a new link to the post.

Masquerade

26-10-2022 08:51

It doesn't matter ^^

CLS-SREP will decompress srep data no matter which version of srep you use for compression.

Make sure you use the latest version of CLS-SREP. I'm assuming you're using ProFrager's version and not Razor's version. The latest CRCs of those files are:

Code:

    File: CLS-SREP_X64.EXE

  CRC-32: ae1cd4f0



    File: CLS-SREP.DLL

  CRC-32: d30b67ba

kuyhaa

26-10-2022 22:01

.pak gotham knights method please ? :)

KaktoR

27-10-2022 01:15

Victoria 3

For bink2 files you have to follow this guide. However there is not much benefit unfortunatelly.

1.14 GB -> 1.04 GB

Masquerade

27-10-2022 01:32

^^
Yeah, not all of the vids even decompress properly so you end up with big diff files so it's best to ignore the vids that don't compress.

The banks don't compress with oggre either, oggre will just crash (you need to use the python script to convert FSB5 to vorbis).

All times are GMT -7. The time now is 02:57.

Page 81 of 119

« First

Last »

Show 40 post(s) from this thread on one page