FileForums - Best Compression Methods for 'Specific' Games. Q&A

Page 96 of 119

« First

106

Last »

Show 40 post(s) from this thread on one page

FileForums (https://fileforums.com/index.php)

- Conversion Tutorials (https://fileforums.com/forumdisplay.php?f=55)

- - Best Compression Methods for 'Specific' Games. Q&A (https://fileforums.com/showthread.php?t=99554)

KaktoR

16-12-2023 11:49

Days Gone
For pak files use kraken with t400 setting and use oodle library v2.9.5.
I have tested alot and t400 is the peak here.

Code:

xtool:mkraken:core_2.9.5

Compressed 1 file, 1,434,557,804 => 2,372,372,647 bytes. Ratio 165.37%

Compression time: cpu 0.73 sec/real 625.16 sec = 0%. Speed 2.29 mB/s

All OK

Code:

xtool:mkraken,t400:core_2.9.5

Compressed 1 file, 1,434,557,804 => 3,436,433,230 bytes. Ratio 239.55%

Compression time: cpu 0.58 sec/real 22.03 sec = 3%. Speed 65.13 mB/s

All OK

Masquerade

16-12-2023 13:30

Compass of Destiny: Istanbul

Code:

*.pak/*.ucas



oo2reck+srep+lolz



Everything else:



razor

4.9GB -> 3.6GB

Astronimo

Code:

srep+lolz

5.1GB -> 1GB

KaktoR

19-12-2023 14:09

Baldurs Gate 3
v4.1.1.4216792

Code:

 17:44:59 - Selected ARC/DS method for Data1a-01.bin was: xtool:mbg3:1+xtool:mzlib:mpreflate+xtool:dd3+4x4:lzma

 17:44:59 - Selected ARC/DS method for Data1b-01.bin was: xtool:c1500mb:mbk2:df20p

 17:44:59 - Selected ARC/DS method for Data1c-01.bin was: xtool:dd3+4x4:lzma

 17:44:59 - Selected ARC/DS method for Data1d-01.bin was: xtool:c128mb:mpreflate:mpng:d1+xtool:dd3+4x4:lzma

 17:44:59 - Selected ARC/DS method for Data1e-01.bin was: 7zdll-pm:m0=mp3det:m1=packmp3c

-------------------------------------------------------------------------------------------------------------------------------------------

 22:12:08 - Overall input size: 139,17 GB

 22:12:08 - Overall output size: 95,69 GB (Ratio 68.76%)

 22:12:08 - Overall conversion time: 04:27:07

SharedSounds.pak was unpacked, 400kb+ wem's were processed and erased out to get additional ratio.

Dragonis40

28-12-2023 01:16

Good morning everybody! I'm trying to compress "Fallout: New Vegas Ultimate Edition" and my mask for *.bsa files and results are:

xt_zlib+srep_old:392+lolz:dtb1:d512m:mtb96:ldl5:mc 1023
21 files: 8,588,002,315 byte => 4,776,969,655 bytes. Ratio 55.62%

This site reports the following results:

*.bsa
xtool(zlib)+srep+lolz
(54.5% Ratio)

I suppose the compressed byte are the same bytes as I compressed, but the result is better. Which is the configuration of LOLZ? It seems it isn't reported...

Thanks for your reply!

Jahan1373

28-12-2023 22:56

Quote:

Originally Posted by Masquerade (Post 498969)

American Truck Simulator [v.1.45.3.26s + All DLCs]

Code:

xzlib+srep+lolz

Latest XTool used with -mzlib only (no reflate).
Use -lm due to incredibly huge streams being decompressed.
I have 64GB of memory so I used a 1024mb chunk size.

10.7GB ----> 6.1GB

Hello my friend how to download this compressor

mausschieber

29-12-2023 02:59

Quote:

Originally Posted by Jahan1373 (Post 503213)

Hello my friend how to download this compressor

Look in the DiskSpan GUI Thread

Dragonis40

29-12-2023 03:08

Quote:

Originally Posted by Dragonis40 (Post 503202)

Any suggestion?

Jahan1373

29-12-2023 04:30

Quote:

Originally Posted by Masquerade (Post 498969)

American Truck Simulator [v.1.45.3.26s + All DLCs]

Code:

xzlib+srep+lolz

Latest XTool used with -mzlib only (no reflate).
Use -lm due to incredibly huge streams being decompressed.
I have 64GB of memory so I used a 1024mb chunk size.

10.7GB ----> 6.1GB

Hello
Friends, you introduce a strong compressor
For high compression

KaktoR

01-01-2024 12:28

1 Attachment(s)

Alan Wake 2
Maybe someone may find this usefull.

I have attached xtl files to extract WEM sounds from both "base-generic-*.rmdblob" and "stream0-generic-*.rmdblob" files (only files which are atleast 100kb in size). The xtl files are compatible with game version 1.0.14.

Processing those WEM files will save you additional 200mb. I removed in total 5 of the WEM files which are not encodable, which leads either to infinite encoding the files (temp files will grow infinitely), or they are just finished instantly, which will lead to broken files if you decode them back to original.

Maybe you have to edit file paths.

Bad WEM files in question:

Code:

"base-generic\data_pc\audio\media\152227780.wem"

"base-generic\data_pc\audio\media\526830619.wem"

"base-generic\data_pc\audio\media\577391364.wem"

"base-generic\data_pc\audio\media\880082258.wem"

"base-generic\data_pc\audio\media\908534589.wem"

1. Command to extract WEM files with xtool

Code:

:WEM base-generic

if not exist "extracted_data\base-generic" (

mkdir "extracted_data\base-generic" )

xtool extract "aw2_base_generic.xtl" "base-generic-000.rmdblob" "extracted_data\base-generic"

xtool extract "aw2_base_generic.xtl" "base-generic-001.rmdblob" "extracted_data\base-generic"

xtool extract "aw2_base_generic.xtl" "base-generic-002.rmdblob" "extracted_data\base-generic"

xtool extract "aw2_base_generic.xtl" "base-generic-003.rmdblob" "extracted_data\base-generic"

xtool extract "aw2_base_generic.xtl" "base-generic-004.rmdblob" "extracted_data\base-generic"

xtool extract "aw2_base_generic.xtl" "base-generic-005.rmdblob" "extracted_data\base-generic"

xtool extract "aw2_base_generic.xtl" "base-generic-006.rmdblob" "extracted_data\base-generic"

xtool extract "aw2_base_generic.xtl" "base-generic-007.rmdblob" "extracted_data\base-generic"

xtool extract "aw2_base_generic.xtl" "base-generic-008.rmdblob" "extracted_data\base-generic"



:WEM stream0-generic

if not exist "extracted_data\stream0-generic" (

mkdir "extracted_data\stream0-generic" )

xtool extract "aw2_stream_generic.xtl" "stream0-generic-000.rmdblob" "extracted_data\stream0-generic"

xtool extract "aw2_stream_generic.xtl" "stream0-generic-001.rmdblob" "extracted_data\stream0-generic"

xtool extract "aw2_stream_generic.xtl" "stream0-generic-002.rmdblob" "extracted_data\stream0-generic"

xtool extract "aw2_stream_generic.xtl" "stream0-generic-003.rmdblob" "extracted_data\stream0-generic"

xtool extract "aw2_stream_generic.xtl" "stream0-generic-004.rmdblob" "extracted_data\stream0-generic"

xtool extract "aw2_stream_generic.xtl" "stream0-generic-005.rmdblob" "extracted_data\stream0-generic"

xtool extract "aw2_stream_generic.xtl" "stream0-generic-006.rmdblob" "extracted_data\stream0-generic"

xtool extract "aw2_stream_generic.xtl" "stream0-generic-007.rmdblob" "extracted_data\stream0-generic"

xtool extract "aw2_stream_generic.xtl" "stream0-generic-008.rmdblob" "extracted_data\stream0-generic"

xtool extract "aw2_stream_generic.xtl" "stream0-generic-009.rmdblob" "extracted_data\stream0-generic"

pause

2. Command to erase WEM files with xtool

Code:

:WEM base-generic

xtool erase "base-generic" "base-generic-000.rmdblob" "aw2_base_generic.xtl"

xtool erase "base-generic" "base-generic-001.rmdblob" "aw2_base_generic.xtl"

xtool erase "base-generic" "base-generic-002.rmdblob" "aw2_base_generic.xtl"

xtool erase "base-generic" "base-generic-003.rmdblob" "aw2_base_generic.xtl"

xtool erase "base-generic" "base-generic-004.rmdblob" "aw2_base_generic.xtl"

xtool erase "base-generic" "base-generic-005.rmdblob" "aw2_base_generic.xtl"

xtool erase "base-generic" "base-generic-006.rmdblob" "aw2_base_generic.xtl"

xtool erase "base-generic" "base-generic-007.rmdblob" "aw2_base_generic.xtl"

xtool erase "base-generic" "base-generic-008.rmdblob" "aw2_base_generic.xtl"



:WEM stream0-generic

xtool erase "stream0-generic" "stream0-generic-000.rmdblob" "aw2_stream_generic.xtl"

xtool erase "stream0-generic" "stream0-generic-001.rmdblob" "aw2_stream_generic.xtl"

xtool erase "stream0-generic" "stream0-generic-002.rmdblob" "aw2_stream_generic.xtl"

xtool erase "stream0-generic" "stream0-generic-003.rmdblob" "aw2_stream_generic.xtl"

xtool erase "stream0-generic" "stream0-generic-004.rmdblob" "aw2_stream_generic.xtl"

xtool erase "stream0-generic" "stream0-generic-005.rmdblob" "aw2_stream_generic.xtl"

xtool erase "stream0-generic" "stream0-generic-006.rmdblob" "aw2_stream_generic.xtl"

xtool erase "stream0-generic" "stream0-generic-007.rmdblob" "aw2_stream_generic.xtl"

xtool erase "stream0-generic" "stream0-generic-008.rmdblob" "aw2_stream_generic.xtl"

xtool erase "stream0-generic" "stream0-generic-009.rmdblob" "aw2_stream_generic.xtl"

pause

3. Encode with WemTool

Code:

WemTool.exe -e

4. Compress files

5. Decode with WemTool

Code:

WemTool.exe -d

6. Decode files with xtool (move them back into game files)

Code:

:WEM base-generic

xtool decode "aw2_base_generic.xtl" "base-generic" "base-generic-000.rmdblob"

xtool decode "aw2_base_generic.xtl" "base-generic" "base-generic-001.rmdblob"

xtool decode "aw2_base_generic.xtl" "base-generic" "base-generic-002.rmdblob"

xtool decode "aw2_base_generic.xtl" "base-generic" "base-generic-003.rmdblob"

xtool decode "aw2_base_generic.xtl" "base-generic" "base-generic-004.rmdblob"

xtool decode "aw2_base_generic.xtl" "base-generic" "base-generic-005.rmdblob"

xtool decode "aw2_base_generic.xtl" "base-generic" "base-generic-006.rmdblob"

xtool decode "aw2_base_generic.xtl" "base-generic" "base-generic-007.rmdblob"

xtool decode "aw2_base_generic.xtl" "base-generic" "base-generic-008.rmdblob"



:WEM stream0-generic

xtool decode "aw2_stream_generic.xtl" "stream0-generic" "stream0-generic-000.rmdblob"

xtool decode "aw2_stream_generic.xtl" "stream0-generic" "stream0-generic-001.rmdblob"

xtool decode "aw2_stream_generic.xtl" "stream0-generic" "stream0-generic-002.rmdblob"

xtool decode "aw2_stream_generic.xtl" "stream0-generic" "stream0-generic-003.rmdblob"

xtool decode "aw2_stream_generic.xtl" "stream0-generic" "stream0-generic-004.rmdblob"

xtool decode "aw2_stream_generic.xtl" "stream0-generic" "stream0-generic-005.rmdblob"

xtool decode "aw2_stream_generic.xtl" "stream0-generic" "stream0-generic-006.rmdblob"

xtool decode "aw2_stream_generic.xtl" "stream0-generic" "stream0-generic-007.rmdblob"

xtool decode "aw2_stream_generic.xtl" "stream0-generic" "stream0-generic-008.rmdblob"

xtool decode "aw2_stream_generic.xtl" "stream0-generic" "stream0-generic-009.rmdblob"

pause

7. Finished. All files are crc perfect at the end.

Jahan1373

01-01-2024 23:01

Quote:

Originally Posted by Masquerade (Post 498969)

American Truck Simulator [v.1.45.3.26s + All DLCs]

Code:

xzlib+srep+lolz

Latest XTool used with -mzlib only (no reflate).
Use -lm due to incredibly huge streams being decompressed.
I have 64GB of memory so I used a 1024mb chunk size.

10.7GB ----> 6.1GB

Help

KaktoR

02-01-2024 10:40

What help you need exactly? Try to be more precise!

Jahan1373

03-01-2024 12:55

://www.mediafire.com/file/gh4um1dsm92xbeg/Help.zip/file
Hello
Can someone help me, I have a problem with this installation
CPU will be 100% after installation
Second, after the installation is finished, the installation icons are opened, the problem is that they are white on the desktop, the game icon is not visible.
Friends, how to solve this problem?

KaktoR

06-01-2024 12:54

Portal Revolution

Code:

 21:39:48 - Selected ARC/DS method for Data1a-01.bin was: msc_tak+xtool:dd3+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000

 21:39:48 - Selected ARC/DS method for Data1b-01.bin was: xtool:dd3+4x4:b64mb:lzma:ultra:64m:bt4:fb273:lc8:mc1000000

-------------------------------------------------------------------------------------------------------------------------------------------

 21:53:29 - Overall input size: 10,84 GB

 21:53:29 - Overall output size: 4,23 GB (Ratio 39.07%)

 21:53:29 - Overall conversion time: 00:13:40

Dragonis40

08-01-2024 22:35

Quote:

Originally Posted by Dragonis40 (Post 503202)

Any ideas?

Masquerade

10-01-2024 07:07

Very likely high dictionary size, all detections on, fast bytes off etc.

Dragonis40

10-01-2024 07:50

Quote:

Originally Posted by Masquerade (Post 503352)

Very likely high dictionary size, all detections on, fast bytes off etc.

What does exactly mean? Can you post here an example of the parameters you're talking about please?

KaktoR

10-01-2024 08:28

lolz settings

Quote:

Data detection options:
-dt [0..1] - enables / disables the pos_ctx / dxt / raw detection. no headers, everything is detected based on the analysis of data statistics. Default: -dt1;
-dtp [0..1] - enables / disables transmission of statistics from previous blocks to subsequent blocks in the detector. Default: -dtp1;
-dtb [0..1] - enables / disables enumeration of all options regardless of heuristics. Default: dtb0;
-dto [0..1] - enables / disables detection of the best positional o1 context. Default: -dto1;
-dtm [0..1] - enables / disables raw graphics multimedia detection. Default: -dtm1;
-dtw [0..1] - enables / disables width detection for raw graphics and dxt textures;
-dtd [0..1] - enables / disables detection of dxt textures;

Multithreading options:
-mtt [0..1] - for multithreading, specifies the operating mode to be used. When set to 0, the dictionary size must be at least 2 times the block size. In this mode, data for each stream will be loaded alternately in block size. In this mode, in most cases, you can achieve better compression than in the second, however, decompression requires as many streams as in compression. When 1, each block is compressed separately, without dependencies on neighboring data, respectively, the compression here is usually worse than in the first mode, but you can specify any number of streams for unpacking. It is for this mode that the options from cls.ini MaxThreadsUsage and MaxMemoryUsage are used. Default: -mtt0;
-mt [1..16] - sets the number of threads to process. With -mt1 and -mtt0, normal sequential lossless compression is obtained by dividing the stream into blocks. Default: -mt1;
-mtb [2..512] - sets the block size in MB. When -mt1 -mtt0 also plays a role, but minimal. And more does not mean better. Usually, for -mtt0, the optimal value is about 32-64MB, respectively, the size of the dictionary should be more than 2 times larger. For mtt1 mode, the dictionary size must be no more than the block size;

Main options:
-d [16..2032] - dictionary size in MB. Default: -d64;
-tt [1..256] - the number of paths to consider in the optimal parser. It greatly affects the speed and compression ratio, but not unpacking. You shouldn't ask more than 16, I assure you it's not worth it. Default: -tt4;
-oh [8..14] - sets the maximum number of bytes that the optimal parser will process at a time (2 ^ X). Default: -oh12;
-os [0 .. -oh] - sets the minimum number of bytes that the parser will process at a time (2 ^ X). Default: -os8;
-fba [0..4096] - sets the size of the minimum match, at which the parser will not bother much in the calculations. Compression is decently accelerated (2 times) with a small loss of compression. At 0, these simplifications are disabled. Default: -fba256;
-fbb [0..4096] - THIS OPTION DOES NOT WORK AT THE MOMENT. Asked for even greater simplifications;
-al [0..1] - enables / disables the calculation of the literal price even if rep0 matches. Default: -al1;
-x [0..2] - turns on the slow modes of the parser with the calculation of (almost) all lengths of the found match, as well as the options match + lit + rep0match. Very slow and merciless. And the benefit is very small. It's easier to add -tt; Default: -x0;

Matchfinder options:
-rt [0..2] - THIS OPTION DOES NOT WORK AT THE MOMENT. I set the type of matchfinder - lz, rolz or hybrid mode, but rolz did not meet expectations and I made all the changes without taking it into account, so it does not work now; Default: -rt0;
-mc [2..1023] - Specifies the maximum number of traversals of the binary tree of matches, after which matches for the given position are no longer searched; Default: -mc128;

Model options:
-cm [0..1] - enables / disables a simple context mixer in some critical places, which mixes a couple of models in each place. When enabled, it improves compression, but slows down decompression. Default: -cm1;
-bc [0..8] - sets the level of influence of the previous byte on the mixer; Default: -bc4;
-lm [0..4] - THIS OPTION DOES NOT WORK AT THE MOMENT. Specifies the type of an "elementary" literal. Complex high-order models with cm did not perform well, so I abandoned them, as did rolz. Default: -lm0;
-blo [0..8] - sets the degree of influence of the previous byte on the encoding of the upper part of the literal. Default: -blo8;
-bll [0..8] - sets the degree of influence of the previous byte on the encoding of the lower part of the literal. Default: -bll8;
-blr [0..8] - sets the degree of influence of the rep0lit byte on the encoding of the upper part of the literal. Default: -blr4;
-bm [0..8] - sets the degree of influence of the rep0lit byte on the encoding of the match type flag. Default: -bm4;
-pc [0..4] - sets the positional context for all encoding operations. It is automatically ignored when the detector is on. Default: -pc2;
-dmXY(X [0..3], Y [0..4]) - sets the model for encoding color pairs (X) and alpha channel (Y) pairs. At the maximum value of each parameter, adaptive switching between models is used, while the decompression speed is reduced, but the compression is better in most cases. Default: -dm34;
-gmXY(X [0..2], Y [0..1]) - X - sets the model for encoding raw graphics. at the maximum value, adaptive switching between models is enabled. However, in this case, it is rare to see a gain over the adaptive mode. Generally, mode 0 is in the lead, but its unpacking is 2 times slower than mode 1. Y - enables updating of statistics of models when they were not used (for example, there was a long match). For X0 and X1, it usually gives a small gain in compression, but the speed drops by a factor of 2 (it all depends on the data). In general, the most appropriate is -gm00, he - and the default mode;

kj911

11-01-2024 04:40

Dragonis40: Try it use very slooow settings:

Code:

xZLib(old Xtool 0.12)+srep:m3f:a0(srep 3.92!)+lolz:dt:dtb1:dtw1:dtm1:dto1:dtd1:d512m:fba4096:mc1023:tt16:mt1:ldmf1:ldc0:ldl5

Questions: The *.bsa archives contain goodly compressibble raw/wav audio files? MP3's or any? Videos?

Code:

Alternative1: xZLib+srep+delta+lolz

Alternative2: xZLib+msc+srep+delta(used or not)+lolz

Dragonis40

11-01-2024 06:47

Quote:

Originally Posted by kj911 (Post 503360)

Dragonis40: Try it use very slooow settings:

Code:

xZLib(old Xtool 0.12)+srep:m3f:a0(srep 3.92!)+lolz:dt:dtb1:dtw1:dtm1:dto1:dtd1:d512m:fba4096:mc1023:tt16:mt1:ldmf1:ldc0:ldl5

Questions: The *.bsa archives contain goodly compressibble raw/wav audio files? MP3's or any? Videos?

Code:

Alternative1: xZLib+srep+delta+lolz

Alternative2: xZLib+msc+srep+delta(used or not)+lolz

Thanks for your reply! How can I find out what *.bsa files contain? Do I have to use any tool?

Masquerade

11-01-2024 10:20

@kj911
Don't use LDMF if you are using srep, no point.

@Dragonis40
https://www.nexusmods.com/skyrimspecialedition/mods/974

KaktoR

11-01-2024 11:04

Tales of Arise
v12.12.23

Code:

 17:25:12 - Selected ARC/DS method for Data1a-01.bin was: xtool:c128mb:mzlib+xtool:dd3+4x4:lzma

 17:25:12 - Selected ARC/DS method for Data1b-01.bin was: xtool:dd3+bcm

 17:25:12 - Selected ARC/DS method for Data1c-01.bin was: xtool:dd3+4x4:lzma

------------------------------------------------------------------------------------------

 19:54:25 - Overall input size: 52,19 GB

 19:54:25 - Overall output size: 34,37 GB (Ratio 65.86%)

 19:54:25 - Overall conversion time: 02:29:11

Japanese audio was ripped out and wem's are encoded. Gives you another 100mb size reduction (only 100kb+ files), but honestly not sure if the game is working without crashes with ripped files as this was just meant to be as a test with fixed xtool version (like I was doing since days for now) and I am not planning to play this game actually. So if you want to test this for yourself use "unreal_tournament_4_0.4.25e.bms" script with quickbms, then erase with xtool.

Code:

Key: 0x61FC71A1376A29598393E2BFB976B275705F460E29399CFD1F9C060C8CD6CE40

Just keep in mind that any patch for the game will fail, as long as "pakchunk0-WindowsNoEditor.pak" is modified.

Dragonis40

12-01-2024 23:43

Quote:

Originally Posted by kj911 (Post 503360)

Dragonis40: Try it use very slooow settings:

Code:

xZLib(old Xtool 0.12)+srep:m3f:a0(srep 3.92!)+lolz:dt:dtb1:dtw1:dtm1:dto1:dtd1:d512m:fba4096:mc1023:tt16:mt1:ldmf1:ldc0:ldl5

Questions: The *.bsa archives contain goodly compressibble raw/wav audio files? MP3's or any? Videos?

Code:

Alternative1: xZLib+srep+delta+lolz

Alternative2: xZLib+msc+srep+delta(used or not)+lolz

I've made some test:

xt_zlib+srep_old:392+lolz:dtb1:d512m:mtb96:ldl5:mc 1023
21 files: 8,588,002,315 byte => 4,776,969,655 bytes. Ratio 55.62% (17127.25 sec)

xt_zlib+srep_old:392+lolz:dt:dtb1:dtw1:dtm1:dto1:d td1:d512m:fba4096:mc1023:tt16:mt1:ldmf1:ldc0:ldl5
21 files: 8,588,002,315 => 4,764,178,217 bytes. Ratio 55.47% (35516.07 sec)

The last combination is NOT worth it!

Several .bsa files are plenty of .ogg file (hundreds), so I should extract these .ogg files, compress and before the end of installation make a backup of original .bsa file. I think it takes very long to install the videogame!!!

KaktoR

13-01-2024 08:02

I tested a bit.

Do not use msc for any of the files. msc_frog will work for encoding, but decoding will give you errors. So use xtool with flac plugin and l8 setting for now for all the WAV files.

For the OGG files you have to extract them first (with BAE tool), erase them out of all the files (with xtool erase command), compress them with oggre. After installation, you have to put all the OGG files back inside the original bsa files (with xtool decode command + XTL file).

BIK file use bpk.

Edit: You can also use bethesda.bms with quickbms instead of BAE if you want.

Edit 2:

Code:

 17:55:35 - Selected ARC/DS method for Data1a-01.bin was: xtool:mzlib:dd3+lolz:d81:mtt1:mt12:mc1023

 17:55:35 - Selected ARC/DS method for Data1b-01.bin was: xtool:mflac,l8

 17:55:35 - Selected ARC/DS method for Data1c-01.bin was: oggre

 17:55:35 - Selected ARC/DS method for Data1d-01.bin was: 7zdll-pm:m0=mp3det:m1=packmp3c

 17:55:35 - Selected ARC/DS method for Data1e-01.bin was: bpk+xtool:dd3+lolz:d81:mtt1:mt12:mc1023



 18:53:04 - Overall input size: 11,92 GB

 18:53:04 - Overall output size: 5,41 GB (Ratio 45.41%)

 18:53:04 - Overall conversion time: 00:57:28

Install time + decoding BSA files 10 minutes.

Edit 3: Keep in mind that the input size is a bit inaccurate due to extracted files, as the BSA files are still the original size but with zero'd data inside.

KaktoR

13-01-2024 12:28

Fallout 3 GOTY

Code:

 20:13:34 - Selected ARC/DS method for Data1a-01.bin was: xtool:mzlib:dd3+lolz:d84:mtt1:mt12:mc1023

 20:13:34 - Selected ARC/DS method for Data1b-01.bin was: msc_tak

 20:13:34 - Selected ARC/DS method for Data1c-01.bin was: oggre

 20:13:34 - Selected ARC/DS method for Data1d-01.bin was: 7zdll-pm:m0=mp3det:m1=packmp3c

 20:13:34 - Selected ARC/DS method for Data1e-01.bin was: bpk+xtool:dd3+lolz:d84:mtt1:mt12:mc1023

-------------------------------------------------------------------------------------------------------------------------------------------

 21:22:39 - Overall input size: 11,25 GB

 21:22:39 - Overall output size: 5,08 GB (Ratio 45.13%)

 21:22:39 - Overall conversion time: 01:00:31

Used quickbms + bethesda.bms to extract WAV, OGG and MP3 files out of all the BSA sound archives.

Example

Code:

for %%f in (*.bsa) do quickbms_4gb_files -d -o -Y -f "{}.wav,{}.ogg,{}.mp3" bethesda.bms "%%f"

Used xtool with erase command to erase all WAV, OGG and MP3 files out of the sound archives (zero data).

Example

Code:

"xtool" erase "Fallout - Sound.bsa_extract" "Fallout - Sound.bsa" "fallout3.xtl"

...

...

Data1a -> All BSA files
Data1b -> WAV
Data1c -> OGG
Data1d -> MP3
Data1e -> Rest of the files

After installation, use xtool again with decode command to copy all the extracted files back into their original places inside the BSA archives.

Example

Code:

"xtool" decode "fallout3.xtl" "Fallout - Sound.bsa_extract" "Fallout - Sound.bsa"

...

...

Remove extracted files from game dir.

PS: Original input size is about 8.70 GB, but due to extracted data this is a bit bigger. However all the sound BSA files will be compressable something like near 1%, so only the install step will require some additional space, but once xtool decode is finished, the size is the same as original but also the crc of all the files.

Edit: Install time + xtool decode about 10mins

Masquerade

14-01-2024 14:06

A Way Out

Code:

xtool_zlib_dd5+lolz

20.6GB ---> 14.4GB

KaktoR

15-01-2024 03:32

The Hopebringer

Code:

 23:48:33 - Selected ARC/DS method for Data1a-01.bin was: xtool:mmermaid:mhopebringer:dd3+4x4:lzma

 23:48:33 - Selected ARC/DS method for Data1b-01.bin was: xtool:dd3+4x4:lzma

-----------------------------------------------------------------------------------------

 00:20:16 - Overall input size: 9,73 GB

 00:20:16 - Overall output size: 7,78 GB (Ratio 79.95%)

 00:20:16 - Overall conversion time: 00:31:41

murphy78

18-01-2024 00:45

Some Diskspan GUI settings for The Sinking City - The Merciful Madness and Starfield
Feel free to alter the LOLZ_ST lines to whatever you feel comfortable with if you don't like to wait for unpacks.

Code:

The Sinking City - The Merciful Madness 47.5 GB -> 27.1 GB



[Game1Settings]

Name=The Sinking City

Exec=

MethodState=1

Data1.Name=Data1.bin

Data1.ExclText=

Data1.InclText=*.pak

Data1.Method1=xtool:c128mb:d1:mue5,k0xAAF9AF1F40225A1241C120533277804B09A56CE78CBF100C131423788DB0E34F:mkraken:core_2.9.5+xtool:dd3+LOLZ_ST|0

Data1.SplitMode=2

Data2.Name=Data2.bin

Data2.Method1=xtool:dd3+LOLZ_ST|0

Data2.InclText=

Data2.SplitMode=2

Data2.ExclText=*.pak



---- DSG_arc.ini addition for LOLZ_ST

LOLZ_ST  = lolz:dtb1:d1536:mtt0:mtb64:mc1023

----could just sub for other LOLZ methods for less compression and faster decompression



Also Starfieldv1888 eng only 118 GB -> 62.3 GB



[Game1Settings]

Name=Starfield

Exec=

MethodState=1

Data1.Name=Data1.bin

Data1.ExclText=*WwiseSounds*|*Voices*

Data1.InclText=*.ba2|*.esm

EGPath=C:\Program Files\Bethesda Softworks\Starfield

Data1.Method1=xtool:mstarfield+xtool:mzlib:mreflate:dd3+LOLZ_ST|0

Data1.SplitMode=2

Data2.Name=Data2.bin

Data2.Method1=msc+xtool:dd3+LOLZ_ST|0

Data2.InclText=*WwiseSounds*|*Voices*

Data2.SplitMode=2

Data3.Name=Data3.bin

Data3.Method1=xtool:c512mb:mbk2:df20p+LOLZ_ST|0

Data3.InclText=*.bk2

Data3.SplitMode=2

Data4.Name=Data4.bin

Data4.Method1=xtool:dd3+LOLZ_ST/$mp3pack=7zdll-pm:m0=mp3det:m1=packmp3c|0

Data4.ExclText=*.ba2|*.bk2|*.esm

Data4.SplitMode=1

Dragonis40

18-01-2024 10:11

Quote:

Originally Posted by KaktoR (Post 503389)

Code:

 17:55:35 - Selected ARC/DS method for Data1a-01.bin was: xtool:mzlib:dd3+lolz:d81:mtt1:mt12:mc1023

 17:55:35 - Selected ARC/DS method for Data1b-01.bin was: xtool:mflac,l8

 17:55:35 - Selected ARC/DS method for Data1c-01.bin was: oggre

 17:55:35 - Selected ARC/DS method for Data1d-01.bin was: 7zdll-pm:m0=mp3det:m1=packmp3c

 17:55:35 - Selected ARC/DS method for Data1e-01.bin was: bpk+xtool:dd3+lolz:d81:mtt1:mt12:mc1023



 18:53:04 - Overall input size: 11,92 GB

 18:53:04 - Overall output size: 5,41 GB (Ratio 45.41%)

 18:53:04 - Overall conversion time: 00:57:28

Good evening and thanks for your help! These are my results:

18:37:14 - Selected ARC/DS method for BPK.bin was: bpk
18:37:14 - Selected ARC/DS method for BSA.bin was: xt_zlib+srep_old:392+lolz:dtb1:d512m:mtb96:ldl5:mc 1023
18:37:14 - Selected ARC/DS method for ESM.bin was: xt_zlib+srep_old:392+lolz:dtb1:d512m:mtb96:ldl5:mc 1023
18:37:14 - Selected ARC/DS method for MP3.bin was: msc_tak:mp3=1:wav=1:raw=1:ddsraw=1:bmf=9:TAK=9:bmp =1:lzma,bt4,fb128,lc8+srep_old:392+lolz:dtb1:d512m :mtb96:ldl5:mc1023
18:37:14 - Selected ARC/DS method for OGG.bin was: oggre+srep_old:392+lolz:dtb1:d512m:mtb96:ldl5:mc10 23
18:37:14 - Selected ARC/DS method for WAV.bin was: xtool:mflac,l8+srep_old:392+lolz:dtb1:d512m:mtb96: ldl5:mc1023
18:37:14 - Selected ARC/DS method for OTHERS.bin was: srep_old:392+lolz:dtb1:d512m:mtb96:ldl5:mc1023

23:55:30 - Overall input size: 12,22 GB
23:55:30 - Overall output size: 5,27 GB (Ratio 43.18%)
23:55:30 - Overall conversion time: 05:18:12

One cons is decompression time: about 19 minutes! How can I reduce this time? How did you obtain 10 minutes of decompression time?

Masquerade

18-01-2024 23:58

^^
MSC's MP3 codec is quite slow. I'd recommend try mpzmt by BlackFire69 with srep after.

Don't use srep+lolz after oggre.

KaktoR

19-01-2024 07:37

There is a 7zip MP3 plugin which is super good, even better then mpz, and it will not be detected as virus unlike mpz (false positive)

shazzla

19-01-2024 08:12

Can it process stream ,or only for file ?

KaktoR

19-01-2024 08:38

Both.

Code:

7zdll

Compressed 1 file, 1,135,385,600 => 918,525,479 bytes. Ratio 80.90%

Compression time: cpu 0.73 sec/real 78.90 sec = 1%. Speed 14.39 mB/s

All OK



mpzmt

Compressed 1 file, 1,135,385,600 => 922,505,962 bytes. Ratio 81.25%

Compression time: cpu 0.44 sec/real 75.45 sec = 1%. Speed 15.05 mB/s

All OK

Dragonis40

20-01-2024 04:40

Quote:

Originally Posted by KaktoR (Post 503419)

There is a 7zip MP3 plugin which is super good, even better then mpz, and it will not be detected as virus unlike mpz (false positive)

Thanks so much for your help!!! Not only compression ratio is slightly better, but i've also saved at least 5 minutes of decompression time! Now, the bottleneck is .ogg file decompression, it still takes too long. Is there any compression that can allow me faster decompression time?
I'll try to extract only .mp3 and .wav files from .bsa files to understand if it worth it in terms of decompression time!

Masquerade

20-01-2024 05:46

^^
You cannot speed up oggre.

Dragonis40

24-01-2024 13:25

Quote:

Originally Posted by Dragonis40 (Post 503424)

Good evening! Concerning Fallout New Vegas, if you extract only .mp3 and .wav files, the final size is compressed videogame about 140 MB bigger than the version with .mp3, .ogg and .wav, but compression time is about half time!!!
So it can worth it!

infovs

25-01-2024 05:40

If some kind soul can reveal method used for Monster Hunter Rise .pak files? Can't figure this one out...only two .pak files should be encrypted and with zlib and zstd streams but there is no AES key for this anywhere and using xtool on this will produce only small gain...but I saw few repackers ratio was *much* better. Tried creating database also but without luck.
Thanks in advance, regards.

KaktoR

25-01-2024 06:41

^
zstd.

Use xtool 0.8.4 + libzstd 1.1.4

https://www.patreon.com/posts/97128362

Code:

Streams: 77311 / 77311

Time: 00:05:06 (CPU 00:01:06)



Size: 34.5 GB >> 48.2 GB



Done!!!

Actually there are even some zlib streams.

Code:

Streams: 20756 / 20756

Time: 00:03:52 (CPU 00:14:47)



Size: 34.5 GB >> 36.0 GB



Done!!!

infovs

25-01-2024 07:23

^you're right KaktoR,:) I was using most recent version of libzstd without any success but using 1.1.4 from 2017 worked like a charm! If I may ask, how can one pinpoint exact version of libzstd to use? I tried everything, yes I even tried different versions of libzstd but it never occurred to me that I should be using 7 years old dll version :eek:
Thanks again,
best regards

KaktoR

25-01-2024 07:56

xtool library checker :)

Masquerade

26-01-2024 01:24

Never seen a ReEngine game use anything else but libzstd.dll v1.1.4 when zstd is used by that engine.

All times are GMT -7. The time now is 02:57.

Page 96 of 119

« First

106

Last »

Show 40 post(s) from this thread on one page