FileForums

FileForums (https://fileforums.com/index.php)
-   Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)
-   -   Best Compression Methods for 'Specific' Games. Q&A (https://fileforums.com/showthread.php?t=99554)

L33THAK0R 23-04-2021 22:07

Hi all,

Does anyone know if there are any better methods to use to compress Metro 2033 & Last Light redux other than srep+lolz?

darkwolves 23-04-2021 22:19

Quote:

Originally Posted by L33THAK0R (Post 491517)
Hi all,

Does anyone know if there are any better methods to use to compress Metro 2033 & Last Light redux other than srep+lolz?

not familiar with those do you know what kind of streams they contain?

L33THAK0R 24-04-2021 00:36

Quote:

Originally Posted by darkwolves (Post 491518)
not familiar with those do you know what kind of streams they contain?

Both contain zlib streams, however I don't have any other file scanning tools to determine what other streams may be present. There are two .dll files present in each of the data folders for the titles that I don't recognise: "oaremote_plugin.dll" & "BugTrap.dll". I'm not sure if these are relevant at all but I thought I'd mention it. The vast majority of zlib streams in both titles are level 9.

Newbie-repacker 24-04-2021 00:55

How to find out which streams the game files contain.......???

Dragonis40 24-04-2021 08:19

Quote:

Originally Posted by KaktoR (Post 491344)
I think you have to remove -a2 parameter for old forge engine games with v0.19.

You were right, once I've removed -a2 option, everything was OK! Thank you!

L33THAK0R 24-04-2021 09:45

Compressing "Mad Max" with xzlib+srep+lolz gave me an output of 21.8gb (from 31.8gb), instead of the reported under 4gb compressions others were able to achieve! My arc.ini headers look like the following, any help would be greatly appreciated!

Code:

[Compression Methods]
CUSTOM    = xzlib+srep+lolz
LOLZ          = lolz:dt:dtb1:dtw1:dtm1:dto1:dtd1:mtt1:mt4:d64m:fba4096:mc1023

[External compressor:xzlib]
header    = 0
packcmd  = "Resources\XT.exe" e:precomp:t50p,c128m:zlib $$arcdatafile$$.tmp $$arcpackedfile$$.tmp

[External compressor:srep]
header    = 0
default    = -m3f
packcmd    = "Resources\srep_x64" {options} $$arcdatafile$$.tmp $$arcpackedfile$$.tmp

[External compressor:lolz]
header    = 0
packcmd    = "Resources\lolz_x64" {options} $$arcdatafile$$.tmp $$arcpackedfile$$.tmp


Masquerade 24-04-2021 12:00

L33THAK0R
Since srep does the most work in removing dupes here, I think something maybe wrong there,

your xtool and lolz looks fine

Try this srep packcmd:

Code:

[External compressor:srep]
header = 0
packcmd = srep -a0 -m3f -l512 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp


L33THAK0R 24-04-2021 12:02

Quote:

Originally Posted by Masquerade (Post 491532)
L33THAK0R
Since srep does the most work in removing dupes here, I think something maybe wrong there,

your xtool and lolz looks fine

Try this srep packcmd:

Code:

[External compressor:srep]
header = 0
packcmd = srep -a0 -m3f -l512 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp


thanks man I'll give that a go

kj911 25-04-2021 07:21

Quote:

Originally Posted by kj911 (Post 491297)
Noob questions from compression via LOLZ: Compression rate better than TTA (FreeArc) methods use LOLZ compressing *.WAV files??

Or use double compression from WAV files?

My favorized methods (from MSTS or any older(?) games): The *.WAV files compressed to --> Archive.arc (with TTA comp.) this archive compressed again to --> Archive2.arc (with REP) method. Fast and good ratio. Second compression process replacing REP to RAZOR or MSC or/and any ones?

Some testing its best rate. Size reduction lower than 35-40% via original size.

Example: Compressed 5001 WAV files. Size: 2.09GB --> 923MB--> 632MB

Please testing and find better methods.

Any tests:

Test1

Compressing WAV files used UltraARC 2.9.0.0R6. Its programs created 4 dataX.arc files.

Methods=SREP+LOLZ
Total size: 811672kB

Test2

Methods=MSC+LZMA2
Total size: 736416kB (187533+199572+282870+66441kB's)
Compression time its badly long. (TAK compressor issue.)

Test3

A test: Its will compress 923MB's ARC file (compressed WAV files with TTA methods) with RAZOR 1.03.7 compressor use 256MB dictionary size.

Its results: 646908kB

B test: Its will compress 923MB's ARC file (compressed WAV files with TTA methods) with LOLZ x64 compressors.

Methods: dt:dtb1:dtw1:dtm1:dto1:dtd1:mtt1:mt4:d64m:fba4096: f1023
Its results: 647626kB

Its use MMC switches?

And any big test results with UltraARC2.9.0.0R6 it will compress the full games via 40+ hours process times. (with splitted arc files.)

Generated archive files list:

data1a.arc: 1047552kB
data1b.arc: 604611kB
data2a.arc: 443966kB
data2b.arc: 1048576kB
data2c.arc: 243390kB
data3a.arc: 805187kB
data3b.arc: 309900kB
data4.arc: 703894kB
----------------------
Total: 5207076kB = ~4.966GB. Hmm. Tiny Compressor v1.4 results will better than UARC versions. Smaller than ~1.08GB file size.

L33THAK0R 25-04-2021 07:29

Quote:

Originally Posted by Hexagon123 (Post 489634)
Start Sizes:
Total original size: 5.68GB
Total new size: 8.85GB

Note: This is not MD5 perfect, but completely lossless since it plays fine.

Compress with the following:
  • .resS, .assets, .(noextension): Note! For AssetBundles, use UABE to decompress them (Size for the AssetBundles: Old: 3.94GB, New: 7.12GB)
    Code:

    srep:m5f:a2:l512+lolz:dt:dtb1:dtw1:dtm1:dto1:dtd1:mtt1:mt4:d128m:fba4096
  • .bk2: Exclude the transparent and the credits video itself due to some CRC errors
    Code:

    bpk
  • Rest of the Files
    Code:

    srep:m5f:a2:l512+mcm:t9

Total packed size: 2.51GB


Hi there, what is "UABE"? If this tool/pre-compressor helps with getting .bundle files to decompress properly I'd be very interested in seeing if this tool helps!

Snake288 25-04-2021 07:35

Greetings To All

Hitman 3 do you know the method about compression

L33THAK0R 25-04-2021 07:42

Quote:

Originally Posted by K4miKaZe (Post 476189)
Hi, this one is not on the list. I made this with two methods (with and without xtool zlib). The gain with xtool is not worth, it has not much zlib...

Tools used: Ultraarc 2900 R5, Srep v3.9.2, Lolz, Xtool 0.9 (zlib)

Computer: As always, 4 cores at 3.9GHz (up to 4.3 on boost), 8GB Ram + SSD

Ripped Files:
Only languages, keeping only 1 Language (spanish in my case)

Title: Dying Light: The Following Enhanced Edition v1.16.0 (GOG with all DLCs)

Method 1:
Code:

Method:          binkpack on .bik/.bk2 files, rest of files srep+lolz
Original Size:  25.1 GB
Compressed Size: 13.2 GB
Compress Time:  6 Hours
Install Time:    12 minutes

Method 2:
Code:

Method:          binkpack on .bik/.bk2 files, rest of files xzlib+srep+lolz
Original Size:  25.1 GB
Compressed Size: 12.9 GB
Compress Time:  6 Hours 20 minutes
Install Time:    13-14 minutes

Final note: I did tried some more things but could not get this lower, using higher dictionaries like d256, d512, etc... on lolz should give you more gain but it takes double time on compression and probably you wont get more than 100-200mb (not worth to me). There are also some repeated bik files, i didnt delete them because i dont know if it works and im too busy to test them now. As always CRC checked and all is fine !!!

Just wanted to report with the duplicate .bik files comment, of which there are 4 (2 versions per video, total 8). By removing the LQ_ (low quality) at the start of the filename you can force the game to load that videofile. I'm not sure when the LQ_ is actually played in normal circumstances, but I would assume it would be when the user is playing at a 720p or lower resolution. Opting to use only the high quality (1080p) videos saves you 702mb, and when choosing the opposite (720p) you save yourself 1.27gb! Additionally the following videos can be removed to save an additional 1.21gb. These videos are just promotional or introductory videos for the "Be the Zombie" online-multiplayer gamemode, which isn't very active especially with cracked clients (mp is p2p).

Masquerade 25-04-2021 09:34

Quote:

Originally Posted by L33THAK0R (Post 491544)
Hi there, what is "UABE"? If this tool/pre-compressor helps with getting .bundle files to decompress properly I'd be very interested in seeing if this tool helps!

Literally 1 online search away: https://duckduckgo.com/?q=uabe :(

L33THAK0R 25-04-2021 10:16

Quote:

Originally Posted by Simorq (Post 468497)
Mad Max
pzlib+srep:m3f:l1024+lolz:d1024:al1:mc1023:tt16 (3.95 GB)

Did you have to decrypt or unpack the archives first? Try as I might, I can't seem to get the blasted thing under 21.8gb.

L33THAK0R 25-04-2021 10:18

Quote:

Originally Posted by Masquerade (Post 491551)
Literally 1 online search away: https://duckduckgo.com/?q=uabe :(

Argh the 1am brain strikes again, sorry about that.

Masquerade 25-04-2021 11:52

Quote:

Originally Posted by L33THAK0R (Post 491553)
Did you have to decrypt or unpack the archives first? Try as I might, I can't seem to get the blasted thing under 21.8gb.

There's no decryption involved, or unpacking, 100% lossless. Try using a different xtool version (e.g. xtool 2020 with -mzlib+reflate in xtool packcmd)

L33THAK0R 25-04-2021 19:01

Quote:

Originally Posted by Masquerade (Post 491558)
There's no decryption involved, or unpacking, 100% lossless. Try using a different xtool version (e.g. xtool 2020 with -mzlib+reflate in xtool packcmd)

that might be the issue, I'll give it a shot!

L33THAK0R 27-04-2021 11:20

Does the UE3 title "Batman Arkham Knight" need any specific special treatment to squash the file size down? Using uelr+srep+lolz+bpk gave me a final output of just over 38gb, but I'm fairly sure this is a very unimpressive final result, as I've seen other users detailing their results but I'm unable to replicate these.

Gehrman 27-04-2021 11:22

Forza Horizon 4: Ultimate Edition v.1.467.783.0
 
Code:

Method: xtool:mpreflate:c128mb+rep+srep:m3f:l512:m512+4x4:lzma:a1:mfbt4:32m:fb273:mc10000:lc8
Original Size:  87.4 GB
Compressed Size: 52.8 GB
Compress Time:  4 Hours
Install Time:    52 Minutes (6T)


L33THAK0R 28-04-2021 18:53

Quote:

Originally Posted by darkwolves (Post 469091)
batman arkham knight premium edition
repacked from
54 gigs (i removed foreign things i did not need wich is why it isn't over 60 gigs)
to 26.2 gigs
xprecomp+srep:m3f:a2+lolz:dt1:dtp1:dto1:dtm1:dtw1: dtd1:mtt1:mt4:d64m:tt10 h14:fba4096:mc1023:bc8:blr8:bm

-------note
the under development xtool used in this repack
Razor is definately outdoing himself with this tool
for anyone who hasn't seen it get it here

http://fileforums.com/showthread.php?t=101729

what exactly is xprecomp?

kj911 29-04-2021 10:38

WAVE compression again: Retested few switches, use TinyGamePack tools by Panker1992 and from searching best results.

In last time good results: 635 350kB

Its default versions with lzma switches its bigger 1-3MBs. This data package contains huge duplicated files. cca. 10-20% in full sizes and files. Deduping compression schemes its preferred than LZMA1/2 ones from data size reduction. UCC2.9.0.0R6 badly configured or any Win7 issues its previous badly results or switches? Used WinXP x86 from compression.

Use scripts and switches from Compress.bat file

Code:

arc a -lc1024 -ld1024 -ep1 -ed -r -w.\ datax.bin -mmsc+rep:256m  "Data\*"
and msc/srep switches:
Code:

[External compressor:msc]
header = 0
packcmd = MSC c -v -f -wav=1 -raw=1 -bmp=1 -ddsraw=1 -ddsdxt=1 -mp3=1 -bmf=9s -tak=9 -dxt=1  $$arcdatafile$$.tmp $$arcpackedfile$$.tmp

[External compressor:srep]
header = 0
packcmd  = srep -m5f -l64 $$arcdatafile$$.tmp $$arcpackedfile$$.tmp

SREP-ed versions its probably few kBs smaller. Without srep compressed archive its probably, minimally faster decompression.

Optimfrog compression testing now again in future time. (waiting smallest results from than TAK compressed ones?) This -tak=9 switch replaced to -frog=9 switches.

Mini archive sample: https://mixdrop.co/f/dq0knxpdu733pqd
Archive info: https://i.kek.sh/34voYtawVu7.jpg

Noob questions: The TAK or OptimFrog its better programs the stability issues these (De)compression progress?
TAK: v2.3.0 32bit OptimFrog: v4.910b

Any results, from my repack:

V2 separated files version summary in last time:

Data1: 2743279kB
Data2: 628021kB
Sound: 635350kB
Binaries: 3683kB
_________________
Total: 4010333kB = 3916.34MB = 3.825GB

Saved 58086kB's from v1 one BIG files versions. Binaries its probably minimally better compress use various lzma switches.

Any results2:

Its 'Data2' files set compress again use arc with -mx switches.

Results: 786646kB (xZLIB+SREP+ARC)

And compress again 'Data2' files set without LOLZ compression. (xZLib+SREP only)
Its precompressed archive size: 3711090kB.

The file compress and RAZOR 1.03.7 x64 use -d 256m switches. NanoZip v0.09 x64 compressing use: -cO -m4g -t4 switches and waiting in few hours from results.

RZ: 638301kB
NZ: 662708kB

Its winner? LOLZ!

:( Sad8669 29-04-2021 12:35

I am finally done testing FIFA 20.
Here is the link to the updated post : https://fileforums.com/showpost.php?...postcount=2232

BTW, you don't want to look at the time it took......xD

Gehrman 29-04-2021 12:50

Serious Sam 4 Deluxe Edition
 
Code:

Method: xtool:mzLib:c128mb+srep:m3f:l512:m512+4x4:lzma:a1:mfbt4:32m:fb273:mc10000:lc8
Original Size:  43.6 GB
Compressed Size: 30.7 GB
Compress Time:  5 Hours
Install Time:    22 Minutes (6T)

Note:
Due to the use of SREP 3.20, the compression speed is very low but has a better ratio.

Gehrman 29-04-2021 13:43

HITMAN 3 Deluxe Edition
 
Code:

Method: srep:m3f:l512:m512+4x4:lzma:a1:mfbt4:32m:fb273:mc10000:lc8
Original Size:  58.2 GB
Compressed Size: 49.3 GB
Compress Time:  90 Minutes
Install Time:    10 Minutes (6T)


:( Sad8669 29-04-2021 13:48

Quote:

Originally Posted by Gehrman (Post 491666)
Code:

Method: srep:m3f:l512:m512+4x4:lzma:a1:mfbt4:32m:fb273:mc10000:lc8
Original Size:  58.2 GB
Compressed Size: 49.3 GB
Compress Time:  90 Minutes
Install Time:    10 Minutes (6T)


What about the XTool Plugin that was made for HITMAN 3?
Have you tested it?

Gehrman 29-04-2021 13:55

Gears Tactics
 
Code:

Method 1: xtool:mkraken:c128mb+rep+srep:m3f:l512:m512+4x4:lzma:a1:mfbt4:32m:fb273:mc10000:lc8
Original Size:  28.6 GB
Compressed Size: 22.6 GB
Compress Time:  2.30 Hours
Install Time:    25 Minutes (6T)

Code:

Method 2: xtool:mkraken:c128mb+srep:m3f:l512:m512+lolz:dtb1:mtt1:d128m
Original Size:  28.6 GB
Compressed Size: 19.5 GB
Compress Time:  ? Hours
Install Time:    ? Minutes (6T)


:( Sad8669 29-04-2021 13:56

HITMAN 3 [v3.10.0]
 
Code:

Method : SREP+4X4:LZMA
Initial Size : 55.0 GB
Final Size : 47.4 GB


Gehrman 29-04-2021 13:59

Quote:

Originally Posted by :( Sad8669 (Post 491667)
What about the XTool Plugin that was made for HITMAN 3?
Have you tested it?

:(
https://fileforums.com/showpost.php?...7&postcount=94

:( Sad8669 29-04-2021 14:03

Quote:

Originally Posted by Gehrman (Post 491670)

:( Guess it can't be helped unless Razor does something.

kj911 30-04-2021 05:05

Small requests from members: Please testing my mini archive sample from testing decompression time comparisons and calculate in rough estimate decompression times.

This archive package: https://mixdrop.co/f/0vlope03akxpv8m
Decompressor packs: https://mixdrop.co/f/840v3zvdildgw7

The old Celeron 430 CPU 1.8Ghz with 1GB memory PC's extraction times in ~1 minutes. (cca. 54-57 seconds.) Please testing from newest high-era PC's users, from pre calculate my repacks extraction times. The full packages, extraction time cca. 60-90 minutes calculated in last time, the old PC. Core-i3/5 or higher capable in 2-3* or higher faster decomp. times.

And previous days uploaded "audio_testfile[tak].bin" properly how to extracting in via CMD mode?? ARC/TAK doesn't extract work. Use "FA.sfx" module in locate TinyGamePack tools by Panker1992 its working.

And mini SREP vs. LOLZ DeDup figthing results, from repacking again these 100k files sets the game.

Results:

Data2.arc with xZLib+SREP+LOLZ: 628021kB
Data2_new.arc with xZLib+LOLZ: 658117kB

Yes. SREP-ed compressed versions its better.

Data2.arc decomp. results (Data1.arc its 3hours! not enough available free RAM):
https://i.kek.sh/NVXNLD4mrBX.jpg

Sound.bin with "FA.sfx" in ~250-260 seconds. Binaries cca. ~5 sec.

Masquerade 30-04-2021 08:03

Steel Division 2 (Build 6613813) [+All DLCs]
 
Commander Deluxe Folder ripped.

Code:

xtool_reflate+srep+lolz
79GB -----> 29.2GB :D

KaktoR 30-04-2021 10:47

Total War: ROME Remastered
Intitial release version, including HighRes pack

Code:

17:18:16 - Selected ARC method for Data1a.bin was: srep_new+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000
--------------------------------------------------------------------------------------------------------------------------------------
 19:41:51 - Overall input size: 67.91 GB
 19:41:51 - Overall output size: 30.69 GB
 19:41:51 - Overall conversion time: 02:23:30

Probably you will get more out of it using lolz, because 90% of the game consists of tga/dds images..

Masquerade 01-05-2021 00:30

Total War Rome Remastered [NOT 4K] [v2.0] [ENGLISH ONLY]
 
Code:

xtool+srep+lolz
On *.webm and *.feral
Code:

srep+lzma2
*feral is ogg audio archives, so you could use oggre but no idea of the benefit (or any issue in decompression)

50.1GB ----> 17.3GB :D

panker1992 01-05-2021 08:29

1 Attachment(s)
Quote:

Originally Posted by Masquerade (Post 491701)
Code:

xtool+srep+lolz
On *.webm and *.feral
Code:

srep+lzma2
*feral is ogg audio archives, so you could use oggre but no idea of the benefit (or any issue in decompression)

50.1GB ----> 17.3GB :D

Optimal Packing

JPG files and use: -mbrunsli or -mlepton
Sound Folders = msc+srep+lzma
PNG+GIF = precomp+srep+lolz

everything else = reflate+srep+lolz_ldmf=d1023m:mt2


Lossy Section:
Run this script to all webm files, you get 25% of the original size
so for 1GB of raw video you get 250mb.

ffmpeg -i "webm_files" -c:v libvpx-vp9 -b:v 2000k -c:a opus -b:a 112k -y "output_webm_files"

I Attached Brunsli:

[External compressor:brunsli]
packcmd = brunsli_x64 c $$arcdatafile$$.jpg $$arcdatafile$$.brn
unpackcmd = brunsli_x64 d $$arcdatafile$$.brn $$arcdatafile$$.jpg
datafile = $$arcdatafile$$.jpg
packedfile = $$arcdatafile$$.brn
solid = 0

Visual Paradigm of trully eye lossless Compression:
Lossless: https://drive.google.com/file/d/18O_...ew?usp=sharing
Lossy: https://drive.google.com/file/d/1QmL...ew?usp=sharing

Masquerade 01-05-2021 11:06

panker1992
Just FYI, game sounds + eng only packing:

srep+LZMA2 = 927MB

msc+srep+LMA2 = 886MB

41mb space savings.

darkwolves 01-05-2021 11:42

Quote:

Originally Posted by L33THAK0R (Post 491622)
what exactly is xprecomp?

Xprecomp is an arc.ini command line introduced from the original version of xtools that razor released..
i had the command line tied into the zlib fuction at the time i did that
so pretty much i used the zlib function on xtools

Snake288 01-05-2021 14:47

Hello
panker1992

Total War ROME RemasteredTotal War ROME Remastered How To Reduce The Size Of Gaming Videos Without Disrupting Quality

Could it be possible to show it as a sample Video ?

Multiple *.reducing the size of WebM game videos in bulk

L33THAK0R 01-05-2021 20:44

Has anyone given "PC Building Simulator" a try? Tried compressing with msc + srep + lolz but unfortunately got a "-1" FreeArc archive corruption error at around 1.2% into attempted installation. Not sure if there's a trick to these Unity titles but it seems I'm yet to figure them out.

I'm not sure as to what is causing issues as I was able to get from 26gb to ~4gb so I'm certain the files can't already be compressed. Whats stranger is similar methods with the title "Outward" gave me similar final ratios, however "Outward" was able to decompress properly. My guess is that this might be an engine version issue, or with how Unity creates its packed files, but I'm honestly at a loss.

Masquerade 02-05-2021 00:49

PC Building Simulator is just srep+lolz. Not sure why you're using msc.

kj911 02-05-2021 01:32

L33THAK0R: This error message used Arc.exe from decompression?? Its probably used bad arc/cls.ini configuration or/and missing/wrong external exe/dll files these unpacking process.


All times are GMT -7. The time now is 08:02.

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2026, vBulletin Solutions Inc.
FileForums @ https://fileforums.com