FileForums
Page 54 of 119 « First 4445253 54 555664104 Last »
Show 40 post(s) from this thread on one page

FileForums (https://fileforums.com/index.php)
-   Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)
-   -   Best Compression Methods for 'Specific' Games. Q&A (https://fileforums.com/showthread.php?t=99554)

hydefromt70s 06-02-2021 06:02
1 Attachment(s)
The Medium

16:39:58 - Selected ARC method for Data1.bin was: xtool_zlib+srep_new+lolz_mtx
------------------------------------------------------------------------------------
13:43:41 - Overall input size: 37,83 GB
13:43:41 - Overall output size: 15,17 GB
13:43:41 - Overall conversion time: 21:03:41
------------------------------------------------------------------------------------

Tried with lolz mt - took 21h !!! Unpacking now to check if everything is right. Can someone explain me why does it take so much RAM ??

KaktoR 06-02-2021 11:53
Elite Dangerous Horizons
v2021-01-28

Code:
19:21:37 - Selected ARC method for Data1a.bin was: xtool_preflate+srep_new+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000
--------------------------------------------------------------------------------------------------------------------------------------
 20:38:50 - Overall input size: 20.98 GB
 20:38:50 - Overall output size: 14.73 GB
 20:38:50 - Data file creation time: 01:08:28
Code:
Extracted 38,187 files, 15,814,328,948 => 22,531,389,592 bytes. Ratio 70.19%
Extraction time: cpu 1189.64 sec/real 800.00 sec = 149%. Speed 28.16 mB/s
All OK

hydefromt70s 06-02-2021 14:23
The Medium - compression test and comparision
 
1 Attachment(s)
I did some more tests today on "The Medium". My goal was to find the most efficient algorithm for test file which contains Zlib streams. The winner is XT xZlib (old xtool) - managed to inflate file from 968MB to 1250MB and with srep and lzma compression, final size went down to 283MB. That meets my expectation.

Now is the part where I don't understand things. If file contains Zlib streams (I did check in "Drop and Scan for Zlib" app so I'm sure of that) I was betting on XT2020 Zlib/reflate/preflate that one of them will be the most efficient pre-compressor and they didn't work at all! I thought that new version of xtool 2020 will be better than old one? What I don't understand here is why XT xZlib worked and XT2020 Zlib didn't? Are they not the same algorithms to deal with deflated streams?

Would highly appreciate if someone could explain this to me.

Cesar82 06-02-2021 14:32
Quote:
Originally Posted by hydefromt70s (Post 490335)
I did some more tests today on "The Medium". My goal was to find the most efficient algorithm for test file which contains Zlib streams. The winner is XT xZlib (old xtool) - managed to inflate file from 968MB to 1250MB and with srep and lzma compression, final size went down to 283MB. That meets my expectation.

Now is the part where I don't understand things. If file contains Zlib streams (I did check in "Drop and Scan for Zlib" app so I'm sure of that) I was betting on XT2020 Zlib/reflate/preflate that one of them will be the most efficient pre-compressor and they didn't work at all! I thought that new version of xtool 2020 will be better than old one? What I don't understand here is why XT xZlib worked and XT2020 Zlib didn't? Are they not the same algorithms to deal with deflated streams?

Would highly appreciate if someone could explain this to me.
Maybe it's because old XTool includes reflate automatically if the reflate libraries (raw2hif_dll.dll and hif2raw_dll.dll) are next to xtool.exe
If they were with XTool, try removing raw2hif_dll.dll and hif2raw_dll.dll from the old Xtool, or use XTool 2020 with "zlib+reflate" methods to get an equivalent comparison.
If I reported wrong, please correct me.


Edit:
I did some tests:
sample: pakchunk0-WindowsNoEditor.pak (from game version GOG)
The results were practically the same.

Code:
XTool v0.3.9 (2020) (only zlibwapi.dll next to XTool.exe)
packcmd  = XTool.exe precomp -mzlib -c64mb -t100p --dbase - - <stdin> <stdout>
unpackcmd = XTool.exe decode -t100p - - <stdin> <stdout>

pack:
Compressed 1 file, 1,015,774,515 => 1,303,484,741 bytes. Ratio 128.32%
Compression time: cpu 1.03 sec/real 14.51 sec = 7%. Speed 70.00 mB/s

unpack:
Extracted 1 file, 1,303,484,741 => 1,015,774,515 bytes. Ratio 128.32%
Extraction time: cpu 1.84 sec/real 5.74 sec = 32%. Speed 177.02 mB/s

XTool v0.1.2 (OLD) (only zlibwapi.dll next to XTool.exe)
packcmd  = XTool.exe e:precomp:t75p,c128m:zlib - - <stdin> <stdout>
unpackcmd = XTool.exe d:precomp:t90p - - <stdin> <stdout>

pack:
Compressed 1 file, 1,015,774,515 => 1,303,386,164 bytes. Ratio 128.31%
Compression time: cpu 1.05 sec/real 28.62 sec = 4%. Speed 35.50 mB/s

unpack:
Extracted 1 file, 1,303,386,164 => 1,015,774,515 bytes. Ratio 128.31%
Extraction time: cpu 1.50 sec/real 5.83 sec = 26%. Speed 174.33 mB/s
https://i.imgur.com/6oxJUx2.png

KaktoR 06-02-2021 15:36
You can also combine xtool codecs like -mreflate+zlib

Razor12911 06-02-2021 19:18
Quote:
Originally Posted by Cesar82 (Post 490336)
Maybe it's because old XTool includes reflate automatically if the reflate libraries (raw2hif_dll.dll and hif2raw_dll.dll) are next to xtool.exe
If they were with XTool, try removing raw2hif_dll.dll and hif2raw_dll.dll from the old Xtool, or use XTool 2020 with "zlib+reflate" methods to get an equivalent comparison.
If I reported wrong, please correct me.
this is correct but if you look at the time xtool took, one would assume that the either the codec didn't initialize due to missing libraries, failure to load libraries or incorrect command line used. I mean to process 968 MB worth of data taking only 3 secs seems like that's what happened.

Quote:
Originally Posted by hydefromt70s (Post 490335)
Would highly appreciate if someone could explain this to me.
the arc.ini/command line you used for xtool would be helpful, that includes the libraries you placed near xtool 2020

KaktoR 07-02-2021 02:23
Age of Wonders Planetfall
v1.403, nothing ripped

Code:
01:49:01 - Selected ARC method for Data1a.bin was: srep_new+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000/$oggrepack=oggre/$mp3pack=mpz_mtx/$soundpack=srep_new+razor_mtx/$deflatepack=xtool_reflate+srep_new+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000
--------------------------------------------------------------------------------------------------------------------------------------
 02:25:35 - Overall input size: 16.24 GB
 02:25:35 - Overall output size: 5.27 GB
 02:25:35 - Data file creation time: 00:33:38

hydefromt70s 07-02-2021 07:39
1 Attachment(s)
Quote:
Originally Posted by Razor12911 (Post 490339)
this is correct but if you look at the time xtool took, one would assume that the either the codec didn't initialize due to missing libraries, failure to load libraries or incorrect command line used. I mean to process 968 MB worth of data taking only 3 secs seems like that's what happened.

the arc.ini/command line you used for xtool would be helpful, that includes the libraries you placed near xtool 2020
And that's exactly what happened. I was using old version of DS GUI with XTools 0.37 and despite problem with loading xtool, GUI was saying everything's fine.

Now with DS GUI 1.0.0.1 and XTool 0.38 everything works as it should.
Thank you all.

KaktoR 09-02-2021 06:51
Dragon Age Inquisition
v1.12, nothing ripped

Code:
13:30:11 - Selected ARC method for Data1a.bin was: xtool_reflate+srep_new+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000
--------------------------------------------------------------------------------------------------------------------------------------
 15:29:49 - Overall input size: 44.46 GB
 15:29:49 - Overall output size: 28.50 GB
 15:29:49 - Overall conversion time: 01:59:37

Masquerade 11-02-2021 12:04
Immortals Fenyx Rising quick test sample:

Compression used: selkie

Code:
FreeArc 0.67 (March 15 2014) Creating archive: __Test\output.MSQ using selkie
Memory for compression 0b, decompression 0b, cache 1mb
Compressing 1 file, 121,110,528 bytes
  Compressing DataPC_Orpheus_gi.forge
Compressed 1 file, 121,110,528 => 180,603,282 bytes. Ratio 149.12%
Compression time: cpu 0.13 sec/real 9.88 sec = 1%. Speed 12.26 mB/s
All OK
Use the oodle core I posted here.

You can also use the oo2core_8_win64.dll that comes with XTool 2020, but you'll need to rename it to oo2core_7_win64.dll in order for oo2recs to work with it!

dixen 14-02-2021 13:42
Immortals: Fenyx Rising (RUS+ENG Voice) v1.1.1

v1

*.forge - oo2recks+srep+4x4:lzma
30.1 gb > 50 gb > 24.4 gb > 13.1 gb
Rest files - srep+4x4:lzma
RePack size - 15.6 gb.
Install time - almost 1 hour

v2

*.forge - srep+4x4:lzma
30.1 gb > 18.8 gb > 14.6 gb
Rest files - srep+4x4:lzma
RePack size - 17 gb.
Install time - 8 min.

v3

*.forge - srep+lolz
30.1 gb > 18.8 gb > 14 gb
Rest files - srep+4x4:lzma
RePack size - 16.3 gb.
Install time - 7 min.

NOTE.
Thanks Masquerade for shared .dll))

Hexagon123 16-02-2021 18:11
Minecraft Dungeons
 
Note: This is the Codex release one. (build version 1.7.3.0_5135400)

Total start size: 3.23GB

Compress with the following:
  • .pak, .mp4
    Code:
    srep:m5f:a2:l512+lolz:dt:dtb1:dtw1:dtm1:dto1:dtd1:mtt1:mt4:d128m:fba4096

  • Rest of the Files
    Code:
    srep:m5f:a2:l512+mcm:x11

Total packed size: 1.14GB

biotec2k8 18-02-2021 12:13
Quote:
Originally Posted by Masquerade (Post 489906)
Start size: 2.7GB

Compress using following method:
  • *.saf
    Code:
    srep+lolz
  • Everything else:
    Code:
    srep+razorx
Final Size: 1GB
Game: Shady Part of Me

I would like to package this game effectively. With which attributes is stated above.

The problem is now:

I get 2 BIN files when I compress the SAF with LOLZ and the rest with razorx.

Is there a possibility to create a self-extracting file from the two bin files?

Or can someone give me a tip on how to create an installer from the game?

KaktoR 19-02-2021 08:25
Star Wars Jedi Fallen Order
v7th January 2021, 1 language

Code:
22:05:10 - Selected ARC method for Data1a.bin was: oo2recm7+srep_new+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000
--------------------------------------------------------------------------------------------------------------------------------------
 03:04:51 - Overall input size: 51.82 GB
 03:04:51 - Overall output size: 43.34 GB
 03:04:51 - Data files creation time: 04:26:14

KaktoR 20-02-2021 00:06
Assassin's Creed Valhalla
v1.1.2, nothing ripped

Code:
23:29:50 - Selected ARC method for Data1a.bin was: oo2recm7+srep_new+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000
--------------------------------------------------------------------------------------------------------------------------------------
 06:16:46 - Overall input size: 71.07 GB
 06:16:46 - Overall output size: 30.45 GB
 06:16:46 - Overall conversion time: 06:46:54

dixen 25-02-2021 01:00
Street Fighter V - Champion Edition (v6.000)

*.pak - Xtool v0.39(Zlib+UE4 codec)+SREP+LOLZ (Fast mode)
47 gb > 112 gb > 56 gb > 25.3 gb

Rest files - SREP+LZMA
460 mb > 403 mb

Test

Quote:
Tested 70 files, 27,240,582,676 => 50,300,374,185 bytes. Ratio 54.16%
Testing time: cpu 622.50 sec/real 4127.19 sec = 15%. Speed 12.19 mB/s
All OK
Quote:
[External compressor:xt20]
header = 0
packcmd = xt20\xt20 precomp -mzlib+preflate+ue4:m1:k5F6153346D665A4B384D3573354 B5743324C7A325673466E474B4937617A676C -d2 -c96mb -t100p-1 - - <stdin> <stdout>
Notes.
unreal.dll and option -d# is necessarily.
NO zlib+reflate because CRC errors

Masquerade 27-02-2021 02:57
Beyond A Steel Sky uses ZSTD compression on the pak, but neither zstdrec nor xtool can inflate it.

Xtool:zlib will only inflate +0.44% over original size.

Any clues?

KaktoR 27-02-2021 03:05
Did you use the correct zstdlib version?

Masquerade 27-02-2021 10:03
Quote:
Originally Posted by KaktoR (Post 490558)
Did you use the correct zstdlib version?
Good idea! Added new libzstd and it came to 109%. Good fun.

KaktoR 27-02-2021 10:08
Use a version near game release date (month / week) if possible and go downwards in libzstd versions until you reach a peak in precompression. I don't know if there is a proper way to detect what zstandard version the game was packed. But that's basically what I do (same with lz4).

Masquerade 28-02-2021 01:49
I talked to spiritiovod and he managed to help identify that the version used was v1.4.4 of libzstd. Added that to zstdrec and wowzers:

Code:
Compressed 1 file, 15,471,886,944 => 31,903,067,738 bytes. Ratio 206.20%
Compression time: cpu 18.23 sec/real 1843.42 sec = 1%. Speed 8.39 mB/s
All OK

KaktoR 28-02-2021 05:20
The Elder Scrolls Online

Code:
15:18:11 - Selected ARC method for Data1.bin was: bpk+srep_new+razor_mtx
 15:18:11 - Selected ARC method for Data2.bin was: oo2reck7+oo2recs7+srep_new+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000
------------------------------------------------------------------------------------
 02:01:02 - Overall input size: 95.05 GB
 02:01:02 - Overall output size: 74.66 GB
 02:01:02 - Overall conversion time: 10:42:48

KaktoR 01-03-2021 09:05
Anyone knows what xtool+ue4 settings to use (m#) for Everspace 2? Ratio is always 100%:confused:

Hexagon123 01-03-2021 13:14
Quote:
Originally Posted by KaktoR (Post 490663)
Anyone knows what xtool+ue4 settings to use (m#) for Everspace 2? Ratio is always 100%:confused:
Try finding the possibly?

Snake288 01-03-2021 13:32
Call of Duty Advanced Warfare sıkıştırma yöntemi?
I would like to know if there are tools and methods that I can use to compress this game

KaktoR 01-03-2021 14:10
Horizon Zero Dawn

Code:
18:42:27 - Selected ARC method for Data1.bin was: bpk
 18:42:27 - Selected ARC method for Data2.bin was: xtool_kraken4+srep_new+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000
--------------------------------------------------------------------------------------------------------------------------------------
 22:20:19 - Overall input size: 52.78 GB
 22:20:19 - Overall output size: 41.71 GB
 22:20:19 - Data files creation time: 02:51:31
Edit: Files to exclude from bpk

Hexagon123 01-03-2021 15:27
Quote:
Originally Posted by Snake288 (Post 490668)
Call of Duty Advanced Warfare sıkıştırma yöntemi?
I would like to know if there are tools and methods that I can use to compress this game
Use precomp?

sajmon83 03-03-2021 14:07
2 Attachment(s)
v3 20.03.2021
Call of Duty 2 v1.3
Original size 3,44 GB One language >>> 2,50 GB nothing ripped

*.iwd - precomp+srep_new+mm+lolz:dtb1:d512m:mtt1:mc1023
Code:
Compressed 28 files, 3,665,523,982 => 2,681,617,294 bytes. Ratio 73.16%
Compression time: cpu 20.73 sec/real 10284.07 sec = 0%. Speed 0.36 mB/s
Rest of the files - precomp_mtx+exe2+razor_mtx
Code:
Compressed 66 files, 15,786,867 => 4,244,062 bytes. Ratio 26.88%      100%
Compression time: cpu 0.13 sec/real 18.92 sec = 1%. Speed 0.83 mB/s
Test results in the attached file.

anime 04-03-2021 16:34
Has anyone tried Half-Life Alyx?

KaktoR 06-03-2021 10:05
Borderlands 3
v2021.01.21, 1 language

Code:
13:52:45 - Selected ARC method for Data1.bin was: xtool_ue4:m1:k0x115EE4F8C625C792F37A503308048E79726E512F0BF8D2AD7C4C87BC5947CBA7+xtool_reflate+srep_new+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000
--------------------------------------------------------------------------------------------------------------------------------------
 18:56:02 - Overall input size: 91.16 GB
 18:56:02 - Overall output size: 70.49 GB
 18:56:02 - Overall conversion time: 05:03:16
Languages: https://fileforums.com/showpost.php?...78&postcount=6

Note: You can rip other languages from video files with ffmpeg. This will save you some additional mb's.

Unpack time: 45 min

dixen 07-03-2021 01:45
Quote:
Originally Posted by KaktoR (Post 490716)
Borderlands 3
v2021.01.21, 1 language

Code:
13:52:45 - Selected ARC method for Data1.bin was: xtool_ue4:m1:k0x115EE4F8C625C792F37A503308048E79726E512F0BF8D2AD7C4C87BC5947CBA7+xtool_reflate+srep_new+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000
--------------------------------------------------------------------------------------------------------------------------------------
 18:56:02 - Overall input size: 91.16 GB
 18:56:02 - Overall output size: 70.49 GB
 18:56:02 - Overall conversion time: 05:03:16
Languages: https://fileforums.com/showpost.php?...78&postcount=6

Note: You can rip other languages from video files with ffmpeg. This will save you some additional mb's.

Unpack time: 45 min
Also you can rip non-english voices folders with QBMS and UnrealPak.exe in AdditionalContent *.pak (32 gb > 24 gb with -compress parameter)

Masquerade 07-03-2021 03:36
Amnesia Dark Descent & Amnesia A Machine For Pigs
 
Both games use same method:

Code:
xzlib+srep+lolz

ogg = oggre
Amnesia The Dark Descent Packed Size: 790MB

Amnesia A Machine For Pigs Packed Size: 1.6GB

Masquerade 07-03-2021 03:37
The Sims 4 v1.71.86.1020
 
Start Size: 44GB

Method:

Code:
xzlib+srep+lolz

mp3 soundtrack = mpz
Packed Size: 27.4GB

Masquerade 07-03-2021 03:39
Among Us v2021.3.5s
 
Start Size: 331MB

Method:

Code:
mcm:x11
Packed Size: 64.4MB

Masquerade 07-03-2021 03:41
Destroy All Humans v1.0.2550
 
*.pdb ripped

Start Size: 15.5GB

Method:

Code:
xzlib+srep+lolz

MP4 videos = srep+lzma
Packed Size: 10.1GB

Masquerade 07-03-2021 03:43
Horse Riding Deluxe 2
 
Start Size: 5.6GB

Method:

Code:
xzlib+srep+lolz
Packed Size: 3.2GB

dixen 07-03-2021 09:35
Far Cry Primal (+HD-Textures)

*.dat - bpk+XTool(FCP Plugin) + SREP + LOLZ
16.20 gb > 16 gb > 11 gb > 9 gb

*.dat (hd-txtrs) - XTool(FCP Plugin) + SREP + LOLZ
3.4 gb > 4.2 gb > 3.8 gb > 2.3 gb

rest files - SREP + LOLZ
190 mb > 150 mb

:( Sad8669 08-03-2021 20:21
Just Cause 4 [ All DLC's + Languages][Hi-res][Final Release]
 
Code:
Method [v1]
Method : SREP+4X4:LZMA
Initial Size : 69.58 GB.
Final Size: 53.68 GB.
Code:
Method [v2]
Method : oo2reck+oo2recs+oo2recm+srep:m3f:l512+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000
Compression Time : 9.6 Hours
Decompression Time : 1.7 Hours (4T)
Initial Size : 69.5 GB
Final Size: 35.2 GB

dixen 09-03-2021 07:37
Borderlands 2 Remastered (RUS+ENG lang)

*.tfc, *.upk - uelr+srep+lolz
16 gb > 27 gb > 17 gb > 6.1 gb

*.bik video - bpk
3.2 gb > 2.2 gb
(exclude VO_ because CRC Errors)

*.pck (2 lang voice) - srep+lzma
5.4 gb > 2.5 gb

Rest files - srep+lzma
250 mb > 89 mb

24.2 gb > 11.3 gb
Install time - 9 min.

Note.
4K video deleted because upscale

Thank ProFrager for uelr tool and guys from csrin for Clean Steam Files))

KaktoR 09-03-2021 20:35
Street Fighter V

Code:
18:32:22 - Selected ARC method for Data1a.bin was: xtool_ue4:m1:k0x5F6153346D665A4B384D3573354B5743324C7A325673466E474B4937617A676C+xtool_zlib+srep_new+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000
--------------------------------------------------------------------------------------------------------------------------------------
 21:40:18 - Overall input size: 52.77 GB
 21:40:18 - Overall output size: 32.88 GB
 21:40:18 - Overall conversion time: 03:07:51
--------------------------------------------------------------------------------------------------------------------------------------


All times are GMT -7. The time now is 18:53.
Page 54 of 119 « First 4445253 54 555664104 Last »
Show 40 post(s) from this thread on one page

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2026, vBulletin Solutions Inc.
FileForums @ https://fileforums.com