FileForums

FileForums (https://fileforums.com/index.php)
-   Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)
-   -   Best Compression Methods for 'Specific' Games. Q&A (https://fileforums.com/showthread.php?t=99554)

KaktoR 06-07-2022 09:22

The Riftbreaker

Code:

17:37:17 - Selected ARC/DS method for Data1a-01.bin was: msc_tak+oggre
 17:37:17 - Selected ARC/DS method for Data1b-01.bin was: xtool:mreflate+srep_new:1+lzma2:d96m:fb273:lc4:mc1000000
-------------------------------------------------------------------------------------------------------------------------------------------
 18:10:58 - Overall input size: 6.53 GB
 18:10:58 - Overall output size: 4.98 GB (Ratio 76.31%)
 18:10:58 - Overall conversion time: 00:33:37

Data1a -> packs\*audio.zip

Code:

Extracted 7 files, 719,712,551 => 1,123,637,547 bytes. Ratio 64.05%
Extraction time: cpu 103.39 sec/real 107.77 sec = 96%. Speed 10.43 mB/s
All OK

Extracted 194 files, 4,628,120,468 => 5,884,339,798 bytes. Ratio 78.65%
Extraction time: cpu 11.06 sec/real 121.15 sec = 9%. Speed 48.57 mB/s
All OK


Masquerade 08-07-2022 10:59

Platypus Adventures

*.PAK
Code:

oo2reck+srep+lolz
Everything else:
Code:

srep+razor
1.6GB ----> 1.1GB

KaktoR 09-07-2022 06:01

Stellaris
v3.4.5, incl all DLC

Code:

10:41:35 - Selected ARC/SPLIT method for Data1a-01.bin was:
srep_new:1+lolz:mtt1:mt12
/$dlcpack=xtool:mreflate+msc_tak+oggre
/$mscpack=msc_tak
/$oggrepack=oggre
/$mp3pack=mpz
/$ddspack=srep_new:1+lolz:mtt1:mt12
/$deflatepack=precomp_pmt+srep_new:1+lolz:mtt1:mt12
/$textpack=dict+lzp+ppmt
-------------------------------------------------------------------------------------------------------------------------------------------
 13:08:00 - Overall input size: 16.48 GB
 13:08:00 - Overall output size: 6.17 GB (Ratio 37.45%)
 13:08:00 - Data file creation time: 02:20:30

$dlcpack = dlc\*.zip
The zip files are compressed so first you have to unpack them with xtool, otherwise msc and oggre will not work

$mscpack = *.wav

$deflatepack = *.pdf, *.png

Code:

Extracted 25,373 files, 6,626,124,086 => 17,691,563,211 bytes. Ratio 37.45%
Extraction time: cpu 184.66 sec/real 679.58 sec = 27%. Speed 26.03 mB/s
All OK


Masquerade 13-07-2022 05:54

Warriors Orochi 3 Ultimate Definitive Edition
  • *.bin
    Code:

    srep
  • *.g1l
    Code:

    mtx:c64m:t16:bcm
  • Everything else:
    Code:

    xzlib+srep+lolz

The following files are credits videos and can be ripped if you want:
  • 018_2d.bin
  • 019_2d.bin
  • 020_2d.bin
  • 038_2d.bin
  • 039_2d.bin
  • 040_2d.bin

Repack with nothing ripped: 20.5GB ----> 12.1GB

KaktoR 19-07-2022 09:08

Stray
pak oo2reck + xtool reflate

bk2 files are KB2n, so bpk has no effect here.

Here are some tests (I have splitted the pak into 50mb parts for faster testing)
Code:

oodle 8 library
Compressed 1 file, 52,428,800 => 78,163,078 bytes. Ratio 149.08%
Compression time: cpu 0.16 sec/real 51.14 sec = 0%. Speed 1.03 mB/s
All OK

+ reflate
Compressed 1 file, 52,428,800 => 87,504,892 bytes. Ratio 166.90%
Compression time: cpu 0.08 sec/real 51.14 sec = 0%. Speed 1.03 mB/s
All OK

Extracted 1 file, 87,504,892 => 52,428,800 bytes. Ratio 166.90%
Extraction time: cpu 0.13 sec/real 3.77 sec = 3%. Speed 13.92 mB/s
All OK

Oodle 7 + 9 libraries gives worse ratio, so the way to go is 8.

Here are some other tests with oo2reck
Code:

xtool:reflate+oo2reck
Compressed 1 file, 52,428,800 => 87,730,149 bytes. Ratio 167.33%
Compression time: cpu 0.05 sec/real 67.07 sec = 0%. Speed 0.78 mB/s
All OK

oo2reck+xtool:reflate
Compressed 1 file, 52,428,800 => 88,079,648 bytes. Ratio 168.00%
Compression time: cpu 0.08 sec/real 68.80 sec = 0%. Speed 0.76 mB/s
All OK

Extracted 1 file, 88,079,648 => 52,428,800 bytes. Ratio 168.00%
Extraction time: cpu 0.08 sec/real 3.27 sec = 2%. Speed 16.02 mB/s
All OK

PS: If you want to make a database use "unreal_tournament_4_0.4.27c_paks_only.bms"

dixen 19-07-2022 10:13

Stray

XTool 0.53 (zlib+kraken v8)
5,944,563,799 bytes >> 14,406,848,518 bytes

oo2core_8_Win64.dll (938.5 kb but not 1 mb)

Masquerade 19-07-2022 10:27

Quote:

Originally Posted by dixen (Post 497694)
Stray

XTool 0.53 (zlib+kraken v8)
5,944,563,799 bytes >> 14,406,848,518 bytes

oo2core_8_Win64.dll (938.5 kb but not 1 mb)

Thanks for the test, I can back you up here that XTool gives better ratio + speed boost than oo2reck (only minor though). :)

dixen 19-07-2022 11:44

Quote:

Originally Posted by Masquerade (Post 497695)
Thanks for the test, I can back you up here that XTool gives better ratio + speed boost than oo2reck (only minor though). :)

well..
Quote:

Compressed 1 file, 5,944,563,799 => 4,211,654,134 bytes. Ratio 70.85%
Compression time: cpu 20.86 sec/real 6223.63 sec = 0%. Speed 0.96 mB/s
All OK
:D
To be honest, there is no point in repacking at all. The difference is 1.5 GB and the installation takes almost 10 minutes. Such a pleasure

Masquerade 19-07-2022 14:06

Stray

  • *.pak
    Code:

    xtool:mzlib+kraken + srep + lolz
  • *.bk2
    Code:

    srep+LZMA2
  • Everything else:
    Code:

    srep+razor

Bink videos have KB2n header so no BPK support here.

6.5GB ----> 4.8GB

kuyhaa 21-07-2022 04:32

anyone, what the precompressor for fifa 17 ?

thanks

Masquerade 25-07-2022 01:10

StarCraft Remastered v1.23.9

Code:

xzlib+msc:tak+srep+lolz
9.8GB -----> 7GB

Dragonis40 27-07-2022 13:29

Cyberpunk 2077 v1.52 HF2:

Good evening everybody. I'd like to know if there is any difference between v1.50 and v1.52 HF2 in compression. It seems that v1.50 is more compressible, isn't it?
If you download Fitgirl version from his/her site (v1.50, one language and Bonus content included), the size is about 36.8 GB. After several tests, my size is 37.8 GB (v1.52 HF2, one language and Bonus content included). For sure, Fitgirl uses specific softwares to compress a lot and not only LOLZ, SREP, XTOOL 2020.
Does anyone know if there is something specific to better compress Cyberpunk 2077? Thanks in advance!

Masquerade 27-07-2022 14:29

Quote:

Originally Posted by Dragonis40 (Post 497779)
For sure, Fitgirl uses specific softwares to compress a lot and not only LOLZ, SREP, XTOOL 2020.
Does anyone know if there is something specific to better compress Cyberpunk 2077? Thanks in advance!

You've answered your own question. FitGirl's repack is smaller because she uses additional tools outside of the FreeArc process to properly compress the game files.

For example:

- Game uses BINK videos that BPK will produce CRC imperfect files, so you must HDiffPatch them (definitely sure this will work)

So there you go, how to shrink your repack.

Dragonis40 27-07-2022 23:54

Thank you for replying me! Audio_1 and Audio_2 are composed of a lot of .wem files, but I you try to convert .wem files from Audio_1, you'll get worse compression ratio (don't know why). If you convert .wem files from Audio_2, not all them will be converted, however compression ratio is much better!
For bpk files, only some basegame_5 bpk files are compressible with no error, lot of others are not compressible cause of crc error or motion data error (or something like that).
So, how I can use HDiffPatch to fix bpk issues? Thank you!

KaktoR 30-07-2022 03:16

Can anyone tell me how to deal with files in "motor_package\data" for "STRANGER OF PARADISE FINAL FANTASY ORIGIN" game?

FitGirl 30-07-2022 17:50

Quote:

Originally Posted by KaktoR (Post 497841)
Can anyone tell me how to deal with files in "motor_package\data" for "STRANGER OF PARADISE FINAL FANTASY ORIGIN" game?

Encrypted audio and unencrypted videos. Basically you can't do anything good with those files. And even if decryption would be possible - most probably it's ogg vorbis inside.

Masquerade 02-08-2022 02:45

Quote:

Originally Posted by KaktoR (Post 497841)
Can anyone tell me how to deal with files in "motor_package\data" for "STRANGER OF PARADISE FINAL FANTASY ORIGIN" game?

If you remove the first 56 bytes of any of the video files, you can view them in any media player. You can then determine which videos are animation tests and are completely unused by the game.

ScOOt3r 04-08-2022 04:40

where can i find these files for Ghostwire -> mgwtokyo & the .xtl file

thanks
ScOOt3r

Masquerade 04-08-2022 05:37

Quote:

Originally Posted by ScOOt3r (Post 497860)
where can i find these files for Ghostwire -> mgwtokyo & the .xtl file

thanks
ScOOt3r

You need to make the database yourself using XTool Unreal Engine Database creator

Gehrman 04-08-2022 05:39

Resident Evil 4 HD Project | The Complete HD Remaster of Resident Evil 4
 
Thanks Shegorat
Code:

*.LFS 26.7GB
Code:

red4hp+rep+srep:m3f:l128+delta+lolz:d96:dm00:fba0:gm20:mtt1:mt6:mc1023
8.82 GB

Code:

Tested 4,945 files, 9,478,004,882 => 28,682,410,574 bytes. Ratio 33.04%   
Testing time: cpu 15455.02 sec/real 1983.74 sec = 779%. Speed 14.46 mB/s
All OK

CLS.ini For Compression and Decompression
Code:

[RED4]
ThreadCount=12

Code:

*.xwb;*.xsb;*.xgs;*.snd 2.39 GB
Code:

msc:raw=1
1.23 GB

Code:

Other Files
Srep+Lolz

Useless files:
Code:

BIO4\movie\adaend_c.sfd
BIO4\movie\adaend_m.sfd
BIO4\movie\demo0.sfd
BIO4\movie\opening.sfd
BIO4\movie\opening_usa.sfd
BIO4\movie\r51es20.sfd
BIO4\movie\r503s00.sfd

At the end of 16 or 17 GB

ScOOt3r 04-08-2022 06:44

Thanks @Masquerade for the info, i will give it a try

Cheers
ScOOt3r

KaktoR 09-08-2022 13:56

Starship Troopers - Terran Command

Code:

22:28:25 - Selected ARC/SPLIT method for Data1a-01.bin was:
srep_new:1+lzma2:d96m:fb273:lc4:mc1000000
/$mscpack=msc_tak
/$storepack=m1
/$textpack=dict+lzp+lzma
/$deflatepack=xtool:mpng:d1:mpreflate+srep_new:1+lzma2:d96m:fb273:lc4:mc1000000
-------------------------------------------------------------------------------------------------------------------------------------------
 22:46:07 - Overall input size: 8.71 GB
 22:46:07 - Overall output size: 5.07 GB (Ratio 58.25%)
 22:46:07 - Overall conversion time: 00:17:40

$storepack = *.mp4
$deflatepack = *.png


Hexagon123 09-08-2022 14:29

Quote:

Originally Posted by KaktoR (Post 497915)
Starship Troopers - Terran Command

Code:

22:28:25 - Selected ARC/SPLIT method for Data1a-01.bin was:
srep_new:1+lzma2:d96m:fb273:lc4:mc1000000
/$mscpack=msc_tak
/$storepack=m1
/$textpack=dict+lzp+lzma
/$deflatepack=xtool:mpng:d1:mpreflate+srep_new:1+lzma2:d96m:fb273:lc4:mc1000000
-------------------------------------------------------------------------------------------------------------------------------------------
 22:46:07 - Overall input size: 8.71 GB
 22:46:07 - Overall output size: 5.07 GB (Ratio 58.25%)
 22:46:07 - Overall conversion time: 00:17:40

$storepack = *.mp4
$deflatepack = *.png


It's a Unity engine based game, decompress bundle with UBT or XTool Unity?

KaktoR 09-08-2022 14:52

UBT did not work, it will not compress with lz4 (same size as decompressed file).

ScOOt3r 09-08-2022 16:27

I did Starship Troopers - Terran Command with srep+lolz only on data files and rzr on data2
Compressed size: 4.80
Original Size: 8.71

ScOOt3r

Dragonis40 09-08-2022 22:35

Good morning! I'm compressing "Alan Wake" and i'd like to know which is the best compression method for .avi and .wmv files. As far as I know, MSC doesn't compress too much. Thank you in advance!

L33THAK0R 10-08-2022 00:23

Quote:

Originally Posted by Dragonis40 (Post 497926)
Good morning! I'm compressing "Alan Wake" and i'd like to know which is the best compression method for .avi and .wmv files. As far as I know, MSC doesn't compress too much. Thank you in advance!

Best you can do is recode them at a slightly lower bit rate, I've only ever repacked the original two titles, which both used BInk1 videos, if the new ones are using H.265 for the ".avi" files you can probably get some pretty decent lossy gains, as for the WMV9s recoding them can be a bit hit or miss, you can usually get away with 5000kbps AVB, really depends on the source though, matter of fact is lossy is your only route for any significant gains.

Hexagon123 13-08-2022 01:24

Marvel's Spider-Man Remastered, info:

These files are packed inside the game file resources.

Structure is similar to Sunset Overdrive, another Insomniac game.

Tried edge_of_nowhere.bms, didn't work. In some files, BPK failed during compression.

Audio have .bnk [BKHD], .wem [RIFF]

Video have .bk2 with KB2j magic [4B 42 32 6A] size info at 0x4, data at 0x8, width at 0x14, heigh at 0x18. Plenty of resolutions like FHD, QHD, and 4K.

dag, and toc have some zlib

Most extensionless files have LZ4, use plugin.

0x0 first with KB2j magic, then 0x4 size in decimal, 0x8 rest of data.

dixen 13-08-2022 11:29

Which files contans 2k & 4k video?

Hexagon123 13-08-2022 11:32

Quote:

Originally Posted by dixen (Post 497956)
Which files contans 2k & 4k video?

g00s001 has the PlayStation Studios logo. Search KB2j in Hex Editor.

abidnnt 14-08-2022 06:30

Can anyone share the methods for repacking Spider-Man Remastered?

ScOOt3r 14-08-2022 06:42

abidnnt there is an xtool plugin that Razor12911 (thanks Razor) did for that, its located in the xtool plug in section, im not sure if i can link that section here.
but you will find it easily on the main page, and it works just fine

cheers
ScOOter

KaktoR 15-08-2022 08:38

Elden Ring
v1.06

Code:

12:25:36 - Selected ARC/DS method for Data1a-01.bin was: oo2reck:8+srep_new:1+lzma2:d96m:fb273:lc4:mc1000000
 12:25:36 - Selected ARC/DS method for Data1b-01.bin was: srep_new:1+lzma2:d96m:fb273:lc4:mc1000000
-------------------------------------------------------------------------------------------------------------------------------------------
 17:38:00 - Overall input size: 48.75 GB
 17:38:00 - Overall output size: 39.73 GB (Ratio 81.49%)
 17:38:00 - Overall conversion time: 05:12:22


HunterEmrah 18-08-2022 12:22

Spider-Man Remastered - decompression fails (archive data corrupted). Compression method:

Code:

xtool:spidey+srep+lolz

[External compressor:xtool]
header    = 0
packcmd  = xtool2020\xtool.exe precomp { -moption} -c32mb -t100p - - <stdin> <stdout>
unpackcmd = xtool2020\xtool.exe decode -t100p - - <stdin> <stdout>

[External compressor:srep]
header = 0
packcmd = srep -m3f -a16 {options} InFile OutFile
datafile = InFile
packedfile = OutFile
unpackcmd = srep -d -mem2048mb - - <stdin> <stdout>

[External compressor:lolz]
header = 0
packcmd = lolz_x64.exe -mtt1 -mt6 -ldmf0 -d128m {options} $$arcdatafile$$.tmp $$arcpackedfile$$.tmp
unpackcmd = lolz_x64.exe $$arcpackedfile$$.tmp $$arcdatafile$$.tmp

using latest xtool and spidey.dll+liblz4.dll that comes with plugin.

files while decompressing:

xtool.exe
spidey.dll
liblz4.dll
hif2raw_dll.dll
raw2hif_dll.dll

cls-lolz.dll
cls-lolz_x64.exe
cls-srep.dll
cls-srep_x64.exe

kj911 19-08-2022 05:24

HunterEmrah: Unpacking error message? Used SREP versions from decompress? Its probably SREP error. Trying test to replace use one srep.exe file to older v3.2/3.92 or use ProFrager's cls-srep v0.3.3 or try modify srep decompression rows:

Code:

"srep" -d -ia- <stdin> <stdout>
Use these files from old ProFrager's SREP decomp methods: cls-srep.dll only, remove cls-srep*.exe and srep.exe files.

Use native srep.exe file from during decompression: remove all cls-srep*.dll/exe files!

Testing try again.

HunterEmrah 19-08-2022 11:53

error code -11 decompression fails (archive data corrupted)

SREP 3.93a beta - i'm using this version for a while and never got error like this.

i have tried spidey+srep+4x4 same error. so there is problem with xtool or srep

ScOOt3r 19-08-2022 12:07

This is what i did and it worked fine for me

bin\arc.exe a -ep1 -dses --dirs -s; -lc- -di -i2 -r -w_Temp -mxtool+srep+lolz Output\data1.bin "data1\*
packcmd = xtool\xtool.exe precomp -mspidey -c32mb -t100p --dbase - - <stdin> <stdout>

copied spidey.dll 210 kb and liblz4.dll 439kb into the xtool directory
my srep.exe is 498KB

not sure if this will help but it worked for me

Cheers
ScOOt3r

Masquerade 19-08-2022 13:23

Little League World Series Baseball 2022
  • *.pak
    Code:

    xkraken+srep+lolz
  • *.mp4
    Code:

    srep
  • Everything else:
    Code:

    srep+razor

3.6GB ---> 2GB

Thanks Haoose for files

kj911 20-08-2022 04:26

Quote:

Originally Posted by HunterEmrah (Post 498020)
SREP 3.93a beta - i'm using this version for a while and never got error like this.
....

Thanks more infos. (Use installer from decompress the archive and srep versions.)

Make three blank folder! (Naming in: Test1/2/3)

Copy these files to all three folder: Arc.exe, unarc.exe, all xtool files, correct arc.ini, correct cls.ini and all lolz files.

Copy the files to Test1 folder: SREP.exe (x86 or x64 versions) and replace the 3.93a executable to 3.92a (use official binaries) and testing now.* Not working? Replace the 3.92a to 3.2 and try again.

Copy the files to Test2 folder: Old v0.3.3 cls-srep.dll file only! (87552 byte) and testing now.

Copy the files to Test3 folder: Newest (any version)** cls-srep.dll, cls-srep_x86/64.exe files. And testing now.

See posting testing results. All methods, not working, try recompress again the data files use SREP 3.92 or 3.2 (From v3.92 my preference, -m3f -a0 -l512 -c512)

*Open CMD window with admin rights and execute the commands:
"Arc.exe t archivefile"
Its ok or not, try use unarc.exe again: "unarc.exe t archivefile" (There are cases when this is almost good, but not with Arc.exe. Rare case. Decompression its ok, with error message.)
**Note: Too many cls-srep versions available! That's the problem with him.

KaktoR 25-08-2022 02:34

V Rising
v0.5.43163

Code:

11:22:53 - Selected ARC/DS method for Data1a-01.bin was: xtool:munity,lz4+srep_new:1+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000
-------------------------------------------------------------------------------------------------------------------------------------------
 11:32:32 - Overall input size: 6.26 GB
 11:32:32 - Overall output size: 1.94 GB (Ratio 31.01%)
 11:32:32 - Overall conversion time: 00:09:36

Code:

Extracted 2,568 files, 2,085,180,612 => 6,724,656,770 bytes. Ratio 31.01%
Extraction time: cpu 169.67 sec/real 101.85 sec = 167%. Speed 66.03 mB/s
All OK

I know this is early access, but nevertheless I will post it for later purposes (or something else).

liblz4 v1.8.0 is used for this


All times are GMT -7. The time now is 08:56.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2026, vBulletin Solutions Inc.
FileForums @ https://fileforums.com