FileForums

FileForums (https://fileforums.com/index.php)
-   Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)
-   -   Best Compression Methods for 'Specific' Games. Q&A (https://fileforums.com/showthread.php?t=99554)

drcool 26-03-2021 07:15

Quote:

Originally Posted by :( Sad8669 (Post 491108)
Please make sure that you are using the correct parameters.
I ain't got that much experience, but recently played with the tools and the results were good. And try using Knight Compressor since it has built in options for parameters.

I am using Knight Compressor...I am using the highest parameters for xtool and lolz/srep and I still get that.

:( Sad8669 26-03-2021 07:45

Quote:

Originally Posted by drcool (Post 491114)
I am using Knight Compressor...I am using the highest parameters for xtool and lolz/srep and I still get that.

Make sure that the version of your game matches with the one in the INDEX.
Parameters doesn't need to be high all the time, makes decompression slower. They should be the same as the one mentioned in the INDEX.

Masquerade 26-03-2021 08:44

Yakuza 6

srep+lolz on everything except *.usm and *.awb in which case use srep+lzma

Ripped vf5end.usm (credits video).

Final size: 28GB

You can use lolz on the audio/videos for size decrease but with this amount of data it will take hours for not much of a tradeoff.

hydefromt70s 26-03-2021 10:44

Paradise Lost
 
Srep: -m5f -a2

Code:

17:14:03 - Selected ARC method for setup.bin was: xtool_zlib+srep_new+4x4:b128mb:lzma:ultra:128m:bt4:fb273:lc8:mc1000000
 18:40:14 - Overall input size: 17,91 GB
 18:40:14 - Overall output size: 13,28 GB
 18:40:14 - Overall conversion time: 01:26:08


Hexagon123 26-03-2021 12:40

Quote:

Originally Posted by Masquerade (Post 491116)
Yakuza 6

srep+lolz on everything except *.usm and *.awb in which case use srep+lzma

Ripped vf5end.usm (credits video).

Final size: 28GB

You can use lolz on the audio/videos for size decrease but with this amount of data it will take hours for not much of a tradeoff.

SREP+LOLZ should've been used for .par files

Masquerade 26-03-2021 13:03

Quote:

Originally Posted by Hexagon123 (Post 491122)
SREP+LOLZ should've been used for .par files

That's... what I said.

KaktoR 27-03-2021 00:35

Assassin's Creed Valhalla
v1.2.0, 1 language

Code:

14:28:01 - Selected ARC method for Data1a.bin was: oo2recm7+srep_new+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000
------------------------------------------------------------------------------------
 21:45:22 - Overall input size: 72.15 GB
 21:45:22 - Overall output size: 30.56 GB
 21:45:22 - Overall conversion time: 07:17:19


Masquerade 28-03-2021 03:49

If you are compressing Crash Bandicoot 4, don't use any stdio.

STDIN for srep ---> crash at srep

STDIO for xtool ---> crash at xtool

After running both in normal $$arc$$ mode, it has precompressed and waiting to run through srep without issue.

KaktoR 28-03-2021 05:54

Quote:

Originally Posted by Masquerade (Post 491140)
If you are compressing Crash Bandicoot 4, don't use any stdio.

STDIN for srep ---> crash at srep

STDIO for xtool ---> crash at xtool

After running both in normal $$arc$$ mode, it has precompressed and waiting to run through srep without issue.

Nope, works fine for me with stdio (xtool).

dixen 28-03-2021 14:03

Quote:

Originally Posted by Masquerade (Post 491140)
If you are compressing Crash Bandicoot 4, don't use any stdio.

STDIN for srep ---> crash at srep

STDIO for xtool ---> crash at xtool

After running both in normal $$arc$$ mode, it has precompressed and waiting to run through srep without issue.

XTool_Preflate STDIN - No errors))

Quote:

XTSRARC = xt20+srep+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:m c1000000

[External compressor:xt20]
header = 0
packcmd = xt20\xt20 precomp -mpreflate -c64mb -t100p --dbase - - <stdin> <stdout>

Masquerade 29-03-2021 01:33

You see, with srep stdin I get:

Code:

  ERROR! Input file is larger than filesize spe  5.5%
  5.5%
Errorlevel=3

ERROR: write error (disk full?) in compression algorithm srep

Now, I did run last night , xtool+srep+lolz but the archive was broken so I am going again.

EDIT 2: After looking at srep command line, I'd need to use -s param to set the input size for stdin to something like 50GB in ordedr to not get this error, am surprised I didn't know this earlier.

KaktoR 29-03-2021 01:49

If I remember right srep has still bugs with stdin, so always use $$InFile$$ $$OutFile$$ with srep to be safe.

Code:

[External compressor:srep]
header    = 0
packcmd    = "srep.exe" {options} InFile OutFile
datafile  = InFile
packedfile = OutFile


samadrahman74 29-03-2021 06:56

What is the problem ?
Project CARS 3
Code:

pcars3_bdt+pcars3_oo2reck+srep_new+lolz
https://s16.picofile.com/file/8429106500/Untitled.png

KaktoR 29-03-2021 08:17

Quote:

Originally Posted by samadrahman74 (Post 491155)
What is the problem ?
Project CARS 3
Code:

pcars3_bdt+pcars3_oo2reck+srep_new+lolz
https://s16.picofile.com/file/8429106500/Untitled.png

https://fileforums.com/showpost.php?...19&postcount=2

Masquerade 29-03-2021 12:55

Crash Bandicoot 4: It's About Time
 
Ripped all files shown in this photo (unused videos) (25 vids + 3 folders = 380MB)

Start Size: 22.8GB

Compression method:

Code:

xtool-preflate+srep+lolz
  • on mp4 videos:
Code:

srep+LZMA2
Final Size: 14.0GB :D

Thanks KaktoR and dixen

KaktoR 29-03-2021 13:03

You can even rip out unused languages from videos files with ffmpeg. Unfortunatelly the audio streams are not named, therefore you have to figure out what audio stream is what language they speak.

Usage:
Code:

ffmpeg -i "C_GameEnd.mp4" -map 0:0 -map 0:X -acodec copy -vcodec copy "Processed\C_GameEnd.mp4"


-map 0:X
I don't know what number is for which language. Use MPC and change audio track to listen.

X = Audio stream ID

-map 0:0 is the main audio track (non-voice) I guess.

First -map 0:0 tells ffmpreg to not touch non-voice track (main audio like sounds and stuff)
Second -map 0:X tells ffmpeg to not touch selected voice audio track (X).
In the above example ffmpeg removes all audio tracks with the exception of 0 (main audio track) and X.
"-acodec copy" tells ffmpeg to copy audio source (it doesn't change stuff like bitrate, etc.). Just a 1-to-1 copy without changing anything.
"-vcodec copy" is the same for video source.

If you want to leave 2 voice languages, add a third -map option:
ffmpeg -i "C_GameEnd.mp4" -map 0:0 -map 0:X -map 0:Y -acodec copy -vcodec copy "Processed\C_GameEnd.mp4"


Masquerade 29-03-2021 15:35

Quote:

Originally Posted by KaktoR (Post 491164)
therefore you have to figure out what audio stream is what language they speak.

Audio track 1 is english, audio track 3 is German. Can't understand much more right now.

Masquerade 30-03-2021 05:14

PayDay 2 [Build 6412663 / v204.1 + All DLCs]
 
Start Size: 69GB

Compression Method:
  • Code:

    xtool+srep+lolz

Final Size: 20.3GB

Notes: This game does use bink video. I unpacked the bundle files but could not see any videos, not even a videos folder. I didn't take a deep look however.

If you can find the videos, you may be able to patch them out of the bundles then use bpk to lower the size.

Entai44 30-03-2021 11:06

Quote:

Originally Posted by Masquerade (Post 491170)
Start Size: 69GB

Compression Method:
  • Code:

    xtool+srep+lolz

Final Size: 20.3GB

Notes: This game does use bink video. I unpacked the bundle files but could not see any videos, not even a videos folder. I didn't take a deep look however.

If you can find the videos, you may be able to patch them out of the bundles then use bpk to lower the size.

I think you forget to add about 990 MB worth of .movie files (from The End achievement) and 2.9GB of soundtrack (flac and mp3), so the final size should be around 73GB.

Edit: I own the game and it is 73GB, so I just think that 69GB is just the bundle files with the dlls and exe :P

Hexagon123 30-03-2021 13:16

Quote:

Originally Posted by Masquerade (Post 491166)
Audio track 1 is english, audio track 3 is German. Can't understand much more right now.

unless you have to play throughout?

mr_gnar 01-04-2021 19:43

Kingdom Hearts HD 1.5 + 2.5 ReMIX
Main files - Xtool(xzlib)+srep+lzma (lolz takes 3 hours longer and there is barely a difference)
Video files - srep+lzma

Starting size: 45.84 GB
Output size: 34.4 GB

- Days/Recoded videos are ripped, since they're just compiled cutscenes turned into two extra movies.

Has anyone looked at the video files yet for the main games? They seem to be MP4s, but it's not possible to view their metadata/bitrate or even play them. I assume there would be no way to remove this encryption, right? Never heard of videos in a game having such a thing before.

:( Sad8669 03-04-2021 02:01

Dragon Ball Z : Kakarot [v1.0]
 
Code:

Method : xZLib+SREP+LOLZ
Method 2 : SREP+LZMA
Method 3 : SREP+4X4:LZMA
Initial Size : 33.6 GB
Final Size : 26.6 GB
Hints : *.pak - *.usm,*.uasset,*.awb - Other Files


:( Sad8669 03-04-2021 07:47

I was trying to decompress DBZ : Kakarot with the methods mentioned above. I was able to extract all the archives except the *.pak ones, which were compressed using xZLib+SREP+LOLZ.

While decompressing, i ran into an error saying :

ISDone.dll
Error code : -12
failed CRC check at pakchunk0-WindowsNoEditor.pak

If someone else faced this issue before or it was discussed in the past please refer me to that thread.

Edit : I managed to decompress the *.pak archive with LIS.

Hexagon123 03-04-2021 12:30

Quote:

Originally Posted by :( Sad8669 (Post 491207)
I was trying to decompress DBZ : Kakarot with the methods mentioned above. I was able to extract all the archives except the *.pak ones, which were compressed using xZLib+SREP+LOLZ.

While decompressing, i ran into an error saying :

ISDone.dll
Error code : -12
failed CRC check at pakchunk0-WindowsNoEditor.pak

If someone else faced this issue before or it was discussed in the past please refer me to that thread.

It might be the code itself?

:( Sad8669 03-04-2021 18:34

Quote:

Originally Posted by Hexagon123 (Post 491208)
It might be the code itself?

The installer code? maybe, i am going to try some other installer scripts and see if it works.

For this one i used WPI by Razor12911 - added some external compressors and most of the time decompression goes smooth.

Edit : Check above messages.

sajmon83 04-04-2021 05:35

3 Attachment(s)
Call of Duty World at War v1.7
Original size 8,05 GB One language >>> 3,85 GB nothing ripped

*.ff - zt_zlib+srep_new+mm+lolz:dtb1:d512m:mtt1:mc1023
Code:

Compressed 128 files, 2,415,695,168 => 799,966,928 bytes. Ratio 33.12%
Compression time: cpu 9.48 sec/real 4662.91 sec = 0%. Speed 0.52 mB/s

*.iwd - zt_zlib+srep_new+mm+lolz:dtb1:d512m:mtt1:mc1023
Code:

Compressed 35 files, 4,724,009,324 => 2,198,061,598 bytes. Ratio 46.53%
Compression time: cpu 13.05 sec/real 7282.30 sec = 0%. Speed 0.65 mB/s

*.bik - bpk
Code:

Compressed 23 files, 1,490,644,508 => 1,124,306,505 bytes. Ratio 75.42%
Compression time: cpu 0.89 sec/real 70.05 sec = 1%. Speed 21.28 mB/s

Rest of the files - precomp_mtx+razor_mtx
Code:

Compressed 47 files, 15,569,584 => 7,857,720 bytes. Ratio 50.47%
Compression time: cpu 0.00 sec/real 16.95 sec = 0%. Speed 0.92 mB/s

Test results in the excel attached file.
An example of an excel file:
https://i.imgur.com/ryI5FbZ.png

KaktoR 04-04-2021 09:42

Quote:

Originally Posted by mr_gnar (Post 491195)
Kingdom Hearts HD 1.5 + 2.5 ReMIX
Main files - Xtool(xzlib)+srep+lzma (lolz takes 3 hours longer and there is barely a difference)
Video files - srep+lzma

Starting size: 45.84 GB
Output size: 34.4 GB

- Days/Recoded videos are ripped, since they're just compiled cutscenes turned into two extra movies.

Has anyone looked at the video files yet for the main games? They seem to be MP4s, but it's not possible to view their metadata/bitrate or even play them. I assume there would be no way to remove this encryption, right? Never heard of videos in a game having such a thing before.

They are mp4 files for sure. Looked a bit with hex

Code:

meta�������Oilst���GŠtoo���?data�������TMPGEnc MPEG Smart Renderer 4 Version 4.0.15.50���"hdlr��������mdirappl�������������Āfree
But seems to crypted or somewhat, for whatever reason.

Masquerade 05-04-2021 06:11

Was there ever an LZMA precompressor? I was reading some ancient threads in my search. At the time, it was said to be a difficult task and to avoid large xdeltas the right params had to be used in recompression. Read a bit on Encode through the wayback machine, but nothing really related to what I was looking for.

Reason I am asking, Crash Bandicoot N Sane Trilogy uses LZMA compression on the archives:

https://i.ibb.co/1sLgDsJ/Capture.png

There's an Extract tool for this game which removes the compression from the pak, and saves it as an decompressed pak file, only thing is no way to re-compress said pak.

https://github.com/AdventureT/Nst-Ex...r/SevenZip/sdk

Update: https://github.com/pashok6798/Crash-.../master/CBNSTT

I found another tool with compression abilities, however it is a GUI with no command line unfortunately. Its says it can compress but it doesn't appear to do.

:( Sad8669 06-04-2021 21:47

FiFA 20
 
Code:

Method on *.cas : SREP+LZMA
Method on Other files : SREP+4X4:LZMA
Decompress Time : 52 ~ 53 minutes (4T)
Initial Size : 42.3 GB
Final Size : 39.2 GB

Code:

Method : fifa19_oo2reck+srep:m3f:l512+lolz:dtb1:d128m:mtt1:mt4:mc1023
Initial Size : 42.3 GB
Final Size : 32.3 GB
Compression Time : 38 hours ~ 40 hours
Decompression Time :  7 hours ~ 8 hours (4T)


KaktoR 07-04-2021 01:16

FINAL FANTASY XIII
JP videos ripped, modded

Code:

22:45:26 - Selected ARC method for Data1a.bin was: bpk+xt_zlib+srep_new+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000
--------------------------------------------------------------------------------------------------------------------------------------
 05:37:27 - Overall input size: 124.27 GB
 05:37:27 - Overall output size: 97.74 GB
 05:37:27 - Overall conversion time: 06:51:54

A whole different game with mods :D

Mods used:
FF XIII HD Full Motion Video - English
The FF XIII HD Project
HD Models Plus
Clean ReShade Preset
FF13Fix

Install time: ~1.5h

ashishkumarji 07-04-2021 01:54

Tell me, What about method compression for CoD Infinite Warfare? get-vidmateapp.com get-mobdroapk.com

Masquerade 07-04-2021 02:58

Hex Edit instructions for LEGO Star Wars: The Force Awakens, to make the game run unpacked.

http://i.imgur.com/vNXOIMG.png
http://i.imgur.com/ekjt82o.png

Taken from Zehax, however the game version this change works on isn't specified.

Anyone can test this?

I'm still searching for how to patch these games, but everyone I've talked to is hiding information and doesn't want to share how to do it.

KaktoR 07-04-2021 03:22

^
Because of reasons ... :D

kj911 08-04-2021 05:46

Hello!

My questions: How to repackaging very old MSTS game with highly compression ratio?

Its probably make likely, two different variants for:

1. Very old PCs (1-2 Core CPUs) users with XP x86 SP2/3 compatibility installers and compressions. (Like FreeArc+SREP+PreComp 0.38 and any more)

2. And newest modern PCs (Core-i3 and higher era) and OS (Vista to Win10) with x86/x64 compatiblity and better maximum compression (FreeArc+SREP+PreComp+RZ+MSC and much more) than older variants.

This very old my repack use classic CMD based installation methods. This game size 7.6GB size reduced to 2.2GB and v2 unfinished version, lower than 2GB file size. (Added more fan made maps.)

Any good idea from best compression?

UPDATE: Testing game compression, use Tiny Compressor v1.4

Used methods: 2 (method 3, this oldschool testing now)

Results:

Source: 19 027 732 495 byte (159 379 files)
Compression time (CPU): ~17700 sec
Compressed size: 4 166 060 315 byte

17.72GB -> 3.88GB

UPDATE2: Method 3 compression failed to 40+ hour long time its my noob errors. Not enough free space from finishing. Pff.

Tested with Method 1:

Compression time (CPU): ~7383 sec
Compressed size: 6 699 140 554 byte (6.24GB)

Any idea how to make setup with stdio (reduced disk i/o) methods from decompression?

hydefromt70s 09-04-2021 05:35

Quote:

Originally Posted by dixen (Post 482888)
The Walking Dead: The Telltale Definitive Series`(RUS/ENG)

*.ttarch2 (without prefixes _ms_ & _voice_) - decrypt with key and QBMS
XTool+srep+lzma2 (2019) = 35 > 13 gb
Sound and other files - srep+4x4:lzma

Install size - 45.6 gb
RePack size - 19.06 gb
Install time - 60 min.

Decrypt key

For QBMS

For ttarchext

I'm planning to make may own repack of this game version and have few questions.
Obviously game is encrypted so it won't compress well until being decrypted first. If I decrypt the game first I understand I will get decrypted and unpacked game files – does the game work properly with unpacked files? Is there anyway to compress game with decryption on the fly?

dixen 09-04-2021 05:54

Quote:

Originally Posted by hydefromt70s (Post 491259)
I'm planning to make may own repack of this game version and have few questions.
Obviously game is encrypted so it won't compress well until being decrypted first. If I decrypt the game first I understand I will get decrypted and unpacked game files – does the game work properly with unpacked files? Is there anyway to compress game with decryption on the fly?

Only decrypt in new *.ttarch file..Unpack no needed

hydefromt70s 09-04-2021 06:04

Quote:

Originally Posted by dixen (Post 491260)
Only decrypt in new *.ttarch file..Unpack no needed

Sorry for being pain in the ass but I'm afraid I don't understand what do you mean by that. Could you explain it more? How can I take encryption off those files?

I'm using 'ttarchext.exe' tool for unpacking from zenhax forum (it has built-in encryption keys). Do I have to unpack each .ttarch2 files and then rebuilt them to take that encryption off or what?

Masquerade 09-04-2021 07:45

Euro Truck Simulator
 
Euro Truck Simulator 2

12GB into 4.9GB

xtool+srep+lolz

Use a small chunk size if you're on a low amount of RAM otherwise your PC will crash.

On newer XTools use -lm param.

:( Sad8669 09-04-2021 08:16

Quote:

Originally Posted by Masquerade (Post 491264)
Euro Truck Simulator 2

12GB into 4.9GB

xtool+srep+lolz

Use a small chunk size if you're on a low amount of RAM otherwise your PC will crash.

On newer XTools use -lm param.

Does lm mean 'limited memory' ?
Its merely just a guess.

if not i would like to know what does it do.

Snake288 09-04-2021 08:20

Hello
Masquerade

Use a small chunk size if you're on a low amount of RAM otherwise your PC will crash.

On newer XTools use -lm param.

Informs by showing an example


;XTool 2020
[External compressor:xtool_zlib]
header = 0
packcmd = "PRE\XTool_2020\Win64\XTool.exe" precomp -mzlib -c64mb -t100p --dbase - - <stdin> <stdout>


All times are GMT -7. The time now is 09:33.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2026, vBulletin Solutions Inc.
FileForums @ https://fileforums.com