FileForums

FileForums (https://fileforums.com/index.php)
-   Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)
-   -   Best Compression Methods for 'Specific' Games. Q&A (https://fileforums.com/showthread.php?t=99554)

KaktoR 01-04-2020 01:41

Because of this I never use MSC with dds images.

Masquerade 01-04-2020 03:18

Quote:

Originally Posted by KaktoR (Post 485079)
Because of this I never use MSC with dds images.

Just SREP it then?

KaktoR 01-04-2020 04:20

Well I just use lolz for this

KaktoR 01-04-2020 04:38

Quote:

Originally Posted by Masquerade (Post 484926)
*.asset, *.(noextension) *.resS, *.resource =
Code:

xtool(2019):precomp:unity:c16m,lm,t100p

:confused:

Code:

Compressed 257 files, 7,876,979,476 => 7,876,983,244 bytes. Ratio 100.00%
Compression time: cpu 8.27 sec/real 204.47 sec = 4%. Speed 38.52 mB/s


Masquerade 01-04-2020 07:45

Quote:

Originally Posted by KaktoR (Post 485083)
:confused:

Code:

Compressed 257 files, 7,876,979,476 => 7,876,983,244 bytes. Ratio 100.00%
Compression time: cpu 8.27 sec/real 204.47 sec = 4%. Speed 38.52 mB/s


Today I learned it's a good idea to test precompressors on their own to see if they make a difference. :(

Carldric Clement 01-04-2020 09:36

Quote:

Originally Posted by KaktoR (Post 485083)
:confused:

Code:

Compressed 257 files, 7,876,979,476 => 7,876,983,244 bytes. Ratio 100.00%
Compression time: cpu 8.27 sec/real 204.47 sec = 4%. Speed 38.52 mB/s


I didn't expected that. XTool didnt have unity codec. They usually use with zlib + lz4 and oodle only. :p

JustFun 01-04-2020 10:31

@Masquerade thx a lot for a complete package of xTool, I am sure that it will be helpful to a lot of other users ;)

Masquerade 01-04-2020 10:48

Quote:

Originally Posted by Carldric Clement (Post 485088)
I didn't expected that. XTool didnt have unity codec. They usually use with zlib + lz4 and oodle only. :p

XTool 2019 has a Unity Codec:

https://fileforums.com/showthread.php?t=102832

https://fileforums.com/showthread.php?t=102833

Forgive me if I completely missed the point of your comment.

hwang4 01-04-2020 13:31

Hi! Is there a better method than "xZLib:max+srep:m3f+lolz:dtb1:d192m:mtt1:mt6:mc102 3/$binkpack=bpk" to compress
"DirT Rally 2.0 - GOTY Edition"?
When I used it , i Got 73,7Go (Unpacked 108Go)!

Thanks!

Kitsune1982 02-04-2020 02:55

Absent Mind (1.77 GB)
 
Absent Mind (1.77 GB)

Repack Name: Absent Mind
Compression Methods used:
srep:lolz:dtb1:d192m:tt1:mt3:ldmf0:mc1023
Media Ripped/Removed/Re-encoded: Nothing!
Original Installed Size: 4.39 GB
Original Repack Size: 2.60 GB
My Repacked Size: 1.77 GB
Compressed Output Size - 4.39 gb -> 1.77 gb

Output file names

Data1.bin
Data2.bin

Compression Time and specifications of hardware used:
Quad Core @ 3.2 ghz x4 - 14 GB RAM 4 GB Vram - 1 Hr 40 Min
Decompression TIme: 4-5 Min on 2 cores, 1-2 minutes on 4+
Repack made with UltraArc <c> Razor12911 and CIUv3 3.0.0.0.u8 by Cesar82/Kaktor.

Note, this conversion's file size is slightly bigger due to 2 reasons. first, i was missing some of the core files from the original game so when i added and compressed them, the file went up a bit. second, this conversion installs much faster than my old one. my old one took almost 5-6 minutes on a dual core with 8 gb ram but this one installs in under 1 1/2 minutes! Just thought i'd make a small mention as to why my first post about the compression methods for this game was smaller than my new one :) Also, the increased file size is due to me adding extra autorun/installer images and more backgrounds to my CIU 3.0.0.0u8 project so the installer exe was 10 mb in size which added to the increased file size. the original size was 1.69 gb but then i added in my extras so it got bigger :)

Masquerade 02-04-2020 11:44

Imperator Rome
 
Total Unpacked Size (game files only): 2.93GB

Total Unpacked Size (Art, Soundtrack and Wallpapers): 0.99GB

Total Unpacked Size: 3.92GB

Compression as follows:
  • The following passage is required in Arc.groups:
    Code:

    $jpg
    *.jpg

    $mp3
    *.mp3

    $bmp
    *.bmp

  • Game files =
    Code:

    srep+lolz/$bmp=msc:bmp=1:bmf=9+srep+lolz/$jpg=packjpg+srep
  • Art, Soundtrack and Wallpapers =
    Code:

    srep+lolz/$mp3=msc:mp3=1:tak=9+srep+lolz/$jpg=packjpg+srep

Total Packed Size using above methods (game files only): 912mb

Total Packed Size (Art, Soundtrack and Wallpapers): 950mb

Total Packed Size: 1.81GB

Kitsune1982 02-04-2020 15:51

A.I. Invasion 1 (922 MB) (Nothing ripped or removed)
 
Repack Name: A.I. Invasion
Compression Methods used:
srep:lolz:d192m:tt1:m43:ldmf0:mc1023
Media Ripped/Removed/Re-encoded: Nothing!
Original Installed Size: 956 MB
Original Repack Size: 923 MB
My Repacked Size: 922 MB
Compressed Output Size - 956 MB -> 922 MB

Output file names

Data1.bin


Compression Time and specifications of hardware used:
Quad Core @ 3.2 ghz x4 - 14 GB RAM 4 GB Vram -
Compression Time:15 Min
Decompression TIme: 2-3 Min on 2 cores, 1-2 minutes on 4+
Repack made with UltraArc <c> Razor12911 and CIUv3 3.0.0.0.u8 by Cesar82/Kaktor.

Prettyboy099 03-04-2020 07:20

Assassin's Creed Odyssey 1.5.3 (eng only)
 
Game size - 89.3gb
Method used - srep+4x4:lzma
Final size - 50.9gb

Masquerade 04-04-2020 05:48

LEGO Indiana Jones
 
In it's retail state (just like all other LEGO Games) this game is impossible to compress properly. In order to make a decent conversion, you first need to use QuickBMS on the DAT files to unpack them.

You can download QuickBMS here.

You will also need the TTGames BMS script which you can get here.

Drag the BMS script over the QuickBMS EXE file and select all the DAT files. I like to unpack to a separate folder then merge the files after unpacking, but it's perfectly fine to unpack to the game directory.

After unpacking, move or delete the DAT files since we no longer need them.

Test the game to make sure it still loads okay, it will load from the raw asssets just like it would from the DAT files.

Total Unpacked Size: 5.61GB

Compression as follows:
  • The following passage is required in Arc.groups:
    Code:

    $ogg
    *.ogg

    $wav
    *.wav

    $bink
    *.bik

    $dds
    *.dds

  • Compress using the folowing:
    Code:

    srep:m3f:l512:v2+lolz:dt:dtb1:dtw1:dtm1:dto1:dtd1:mtt1:mt4:d64m:fba4096/$wav=msc:wav=1:tak=9+srep:m3f:l512:v2+lolz:dt:dtb1:dtw1:dtm1:dto1:dtd1:mtt1:mt4:d64m:fba4096/$ogg=oggre/$bink=bpk+srep:m3f:l512:v2/$dds=msc:dxt=5:ddsraw=1+srep:m3f:l512:v2+lolz:dt:dtb1:dtw1:dtm1:dto1:dtd1:mtt1:mt4:d64m:fba4096

Total Packed Size using above methods: 1.77GB

Prettyboy099 05-04-2020 04:17

Tom Clancy's Ghost Recon Breakpoint - Standard Edition (eng only)
 
Compression method - srep+4x4:lzma
Compression time - 2 hours 57 min.
Input size - 62.015 GB
Output size - 43.563 GB
Installation time - 12 min. 32 sec. (SSD)

Masquerade 05-04-2020 08:25

Gaia (4th April 2020)
 
Total Unpacked Size: 5.74GB

Compression as follows:
  • Compress using the folowing:
    Code:

    srep:m3f:l512:v2+lolz:dt:dtb1:dtw1:dtm1:dto1:dtd1:mtt1:mt4:d64m:fba4096
Total Packed Size using above method: 1.74GB

HunterEmrah 06-04-2020 01:32

Quote:

Originally Posted by Masquerade (Post 485137)
  • Code:

    srep:m3f:l512:v2

what is v2 ?

KaktoR 06-04-2020 01:51

Quote:

Originally Posted by HunterEmrah (Post 485144)
what is v2 ?

Code:

-v[0..2]: verbosity level

Masquerade 06-04-2020 02:00

Quote:

Originally Posted by HunterEmrah (Post 485144)
what is v2 ?

Quote:

Originally Posted by KaktoR (Post 485145)
Code:

-v[0..2]: verbosity level

It's not needed, but I like it verbose :cool:

Masquerade 06-04-2020 02:05

Mana Spark
 
Total unpacked size: 411mb

Compression as follows:
  • *.assets, *.resS, *.(noextension) =
    Code:

    srep:m3f:l512+lolz:dt:dtb1:dtw1:dtm1:dto1:dtd1:mtt1:mt4:d64m:fba4096
  • Everything else =
    Code:

    srep:m3f:l512+razor
Total packed size using above methods: 75mb

panker1992 06-04-2020 16:53

Resident Evil 3 Repack
 
1 Attachment(s)
It's been a while since i last showed up so i decided to post a how to.

Positions

::Video
sfk partcopy re_chunk_000.pak -fromto 7464214027 7575275053 1.mov -yes
sfk partcopy re_chunk_000.pak -fromto 9622058441 9629337927 2.mov -yes
sfk partcopy re_chunk_000.pak -fromto 8145547141 8163076314 3.mov -yes
sfk partcopy re_chunk_000.pak -fromto 7599547555 7639073075 4.mov -yes
sfk partcopy re_chunk_000.pak -fromto 8049664957 8102519365 5.mov -yes


::Sound
sfk partcopy re_chunk_000.pak -fromto 19490788305 19526528621 1.dat -yes
sfk partcopy re_chunk_000.pak -fromto 20901797711 20919846410‬ 2.dat -yes
sfk partcopy re_chunk_000.pak -fromto 20831803983 20844636465 3.dat -yes
sfk partcopy re_chunk_000.pak -fromto 20926725234 20939511775 4.dat -yes
sfk partcopy re_chunk_000.pak -fromto 20889493542 20901797710 5.dat -yes
sfk partcopy re_chunk_000.pak -fromto 19589291883 19600926828 6.dat -yes


Precompression + Compression (optimal)

pzlib e -r9 -st1 -c256m -ct75p - <stdin> -o $$arcpackedfile$$.tmp
or
Ztool with pzlib
+
srep:m3f:a16:l256
+
lolzi_x64.exe d1023m -mt1 -ldmf1 -ldc0 -al1 -mc1023 -lm0 -rt0 -dt1 -mtb64m -dtb0 -x0 -tt12 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp

grab the files i uploaded.

first run SFK patch Making and move the extra files into a new area.

then

move the files you see into the file and run SFK patching.

then use the above instructions to repack main file games.
and video and audio files separetely.

after you do this, at the installation for the game you will need to run patching to patch the files back.

Game Final Size: 13.2GB

PS: Modders and Administrators, the files distributed below are dummies, if you examine them via hex editor you will see 0s.

dixen 07-04-2020 02:40

ANNO 1800 Complete Edition (RUS+ENG)

Data6.rda & Data8.rda - xtool(v0.12)+srep+lolz (fast mode)
Rest *.rda + other fles - srep+lolz (fast mode)

48 gb > 20.9 gb

Fast mode = mtt1 tt1 mc2 mt6

KaktoR 07-04-2020 03:07

Quote:

Originally Posted by dixen (Post 485166)
ANNO 1800 Complete Edition (RUS+ENG)

Data6.rda & Data8.rda - xtool(v0.12)+srep+lolz (fast mode)
Rest *.rda + other fles - srep+lolz (fast mode)

48 gb > 20.9 gb

Fast mode = mtt1 tt1 mc2 mt6


I had to use xtool x86, because x64 version stuck on precompression.

dixen 07-04-2020 03:24

Quote:

Originally Posted by KaktoR (Post 485169)
I had to use xtool x86, because x64 version stuck on precompression.

Wow! Thank for info

UPD..
Hmm. With precompressor (xtool 0.12 x86) total size on 2 gb bigger..

dixen 08-04-2020 03:55

Quote:

Originally Posted by panker1992 (Post 485160)
Ztool with pzlib

For what? Why not xtool?

Masquerade 08-04-2020 10:30

I'm taking a look at Terminator Resistance, and however I'm encountering an error.

It's an Unreal Engine game, so i'm using xtool(zlib)+srep+lolz, however I always recieve the Failed CRC check error during unpacking. This always seems to happen when I use XTool, because I can use srep+lolz on their own without issues.

Here's my Arc.ini:
Code:

[External compressor:xzlib]
header = 0
packcmd = xtool e:precomp:t100p:zlib - - <stdin> <stdout>

[External compressor:srep]
header = 0
packcmd = srep -m3f -l512 -v2 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp

[External compressor:lolz]
header = 0
packcmd = lolz_x64 -dt -dtb1 -dtw1 -dtm1 -dto1 -dtd1 -mtt1 -mt4 -d64m -fba4096 {options} $$arcdatafile$$.tmp $$arcpackedfile$$.tmp

I'm not sure what could be happening, because other games such as No Mans Sky where xzlib is also useful has the same unpacking error.

I am using XTool 0.12, lolz v22c4b and SREP 3.93 Beta.

Snake288 08-04-2020 11:05

Terminator Resistance
use this method
XTool 0.09
xZLib:c128mb,t8:zlib+srep:mem512m:l2k:m5f:a16+lolz :d256:mc1023:dtb1:tt4:mtt1:mt6:mtb256

Masquerade 08-04-2020 11:25

Quote:

Originally Posted by Snake288 (Post 485193)
Terminator Resistance
use this method
XTool 0.09
xZLib:c128mb,t8:zlib+srep:mem512m:l2k:m5f:a16+lolz :d256:mc1023:dtb1:tt4:mtt1:mt6:mtb256

I understand the method, no issues there. What I don't understand is why I'm getting CRC errors when I try to unpack the Archive.

Snake288 08-04-2020 11:51

mausschieber working we thank him

Terminator Resistance [?xDVD5]

I used it when installing it no problem

KaktoR 08-04-2020 12:34

@Masquerade

Try to pack with c128m,t75p and unpack with t2 or t50p. If this works try to increase thread usage

Masquerade 09-04-2020 03:21

Quote:

Originally Posted by KaktoR (Post 485196)
@Masquerade

Try to pack with c128m,t75p and unpack with t2 or t50p. If this works try to increase thread usage

Thanks for this, I tried a varitation of commands following your guide which was helpful. I settled on leaving out any extra options and just allowing XTool to do its thing.

Everything works now.

Also, do you know where I could find a complete ZTool config? Would it help in some games vs XTool, or should I always stick to XTool (since it succeeds ZTool)?

panker1992 09-04-2020 07:57

xtool has significant bugs.
unfortunately it doesnt work with RE3 and some new games.
Zee hasnt update the project in a while and from what i know so far he has lost part or parts of the code :D

i had to use ztool for it:D

devil777 09-04-2020 08:28

So incase of using zlib , dude you can use "PZlib".... it'll work... @masquerade

And I don't own Terminator Resistance

Masquerade 10-04-2020 09:46

The Stanley Parable (Latest Version)
 
Total Unpacked Size: 1.95GB

Compression as follows:
  • The following passage is required in Arc.groups:
    Code:

    $wav
    *.wav

    $jpg
    *.jpg

    $raw
    *.raw

  • All files:
    Code:

    srep+lolz/$jpg=packjpg+srep+lolz/$wav=msc:wav=1:tak=9+srep+lolz/$raw=msc:raw=1:tak=9+srep+lolz
Total packed size using above methods: 676mb

HunterEmrah 10-04-2020 18:22

Quote:

Originally Posted by Masquerade (Post 485146)
It's not needed, but I like it verbose :cool:

with v2 it says unsupported compression method.

Quote:

Originally Posted by Masquerade (Post 484926)
Cities Skylines (Sunset Harbor / 26th March 2020)

Total unpacked size (game files only): 12.2GB


Compression as follows:

*.asset, *.(noextension) *.resS, *.resource =
Code:

xtool(2019):precomp:unity:c16m,lm,t100p+srep:m3f:l512:v2+lolz:dt:dtb1:dtw1:dtm1:dto1:dtd1:mtt1:mt4:d64m:fba4096

also this method gives error when unpacking
https://thumbs2.imgbox.com/3d/3e/Rf6GxJdM_t.png

ZAZA4EVER 10-04-2020 21:33

ANNO 1800 Complete Edition
 
2 Attachment(s)
ANNO 1800 Complete Edition (Eng/Multi8)
Remove Sound track + Support
Repack Size From 47.7GB To 19.78GB


Method on **data0.rda ---> data14.rda (maindata folder)

Code:

-mPzlib(zlib)+srep:m3f:a16:d1g+lolz:d128m:mt6:fba0: mc1023:gm20
Method on **data05.rda (maindata folder)

Code:

-mPzlib(zlib)+srep:m3f:a16:d1g+lolz:d128m:mt6:fba0: mc1023:gm20
Other Files Langs+bin Folder

Code:

-msrep:m3f:a16:d1g+lolz:d128m:mt6:fba0:mc1023:gm20

Zlib Checker V2.00 Results
Code:

data0.rda:
Found streams: 333
Duration: 1 minute 19 seconds
218 MB > 238 MB

data1.rda:
Found streams: 0
Duration: 16 minutes 6 seconds
4.42 GB > 4.42 GB

data2.rda:
Found streams: 30
Duration: 3 minutes 19 seconds
677 MB > 708 MB

data3.rda:
Found streams: 1
Duration: 59 minutes 19 seconds
8.47 GB > 8.47 GB

data4.rda:
Found streams: 0
Duration: 17 seconds
43.4 MB > 43.4 MB

data5.rda:
Found streams: 71
Duration: 3 seconds
25.1 MB > 2.81 GB

data6.rda:
Found streams: 30082
Duration: 19 minutes 21 seconds
8.54 GB > 18.0 GB

data7.rda:
Found streams: 266
Duration: 23 seconds
140 MB > 273 MB

data8.rda:
Found streams: 32
Duration: 56 minutes 36 seconds
10.8 GB > 10.8 GB

data9.rda:
Found streams: 0
Duration: 2 minutes 53 seconds
464 MB > 464 MB

data10.rda:
Found streams: 137
Duration: 2 minutes 26 seconds
511 MB > 2.72 GB

data11.rda:
Found streams: 2792
Duration: 11 minutes 25 seconds
2.39 GB > 4.57 GB

data12.rda:
Found streams: 380
Duration: 10 minutes 8 seconds
2.25 GB > 4.77 GB

data13.rda:
Found streams: 4991
Duration: 29 minutes 15 seconds
5.80 GB > 7.48 GB

data14.rda:
Found streams: 30
Duration: 5 minutes 10 seconds
1.01 GB > 1.10 GB

E:\ZAZA Source\Anno 1800\1\maindata:
Found streams: 39145
Duration: 3 hours 38 minutes
45.7 GB > 66.8 GB

i want to thanks dixen , KaktoR , panker1992 for their comments of zlib problem . i learned from them
game stuck with xtool v0.9 , 0.12 x64 .. but works fine with xtool x86 v.12
but you dont get good ratio
Pzlib is stable and works fine with the game files
You must repack data05.rda sperated to unpack game files well

Masquerade 11-04-2020 02:50

Quote:

Originally Posted by HunterEmrah (Post 485222)
with v2 it says unsupported compression method.

also this method gives error when unpacking
https://thumbs2.imgbox.com/3d/3e/Rf6GxJdM_t.png

KaktoR showed me that you don't need XTool's Unity codec for this game, so there's no point in using it - that was my mistake. I've edited my post to remove that bit. Just use srep+lolz.

Masquerade 11-04-2020 11:17

Saints Row 2
 
Total unpacked size: 11.6GB

Compression as follows:
  • *.vpp_pc =
    Code:

    srep:m3f:l512+lolz:dt:dtb1:dtw1:dtm1:dto1:dtd1:mtt1:mt4:d64m:fba4096
  • *.bik =
    Code:

    bpk
  • Everything else:
    Code:

    razor+srep:m3f:l512
Total packed size using above methods: 3.51GB

Masquerade 12-04-2020 11:31

Saints Row IV (4)
 
Total unpacked size: 9.88GB

Compression as follows:
  • *.vpp_pc =
    Code:

    srep:m3f:l512+lolz:dt:dtb1:dtw1:dtm1:dto1:dtd1:mtt1:mt4:d64m:fba4096
  • *.bik =
    Code:

    bpk
  • Everything else:
    Code:

    razor+srep:m3f:l512
Total packed size using above methods: 6.15GB

Masquerade 13-04-2020 04:47

Saints Row The Third (3)
 
Total unpacked size: 9.22GB

Compression as follows:
  • *.vpp_pc =
    Code:

    srep:m3f:l512+lolz:dt:dtb1:dtw1:dtm1:dto1:dtd1:mtt1:mt4:d64m:fba4096
  • *.bik =
    Code:

    bpk
  • Everything else:
    Code:

    razor+srep:m3f:l512
Total packed size using above methods: 5.09GB


All times are GMT -7. The time now is 22:55.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2026, vBulletin Solutions Inc.
FileForums @ https://fileforums.com