DiskSpan GUI

[Dragonis40]

Originally Posted by Masquerade

^^
d128m fba4096
You will get lesser compression but better ram usage.

You mean lolz:d128m:fba4096? that's it?

[Masquerade]

Originally Posted by Dragonis40

You mean lolz:d128m:fba4096? that's it?

Well, all detections on too. Been a while since I last used lolz

[kj911]

From my "modest" compression experiments, I have concluded that with a given dictionary size "ex: d128m" and running on 1(!) thread, the occurrence of a given event, in this case RAM exhaustion, may also depend on how compressible and large the given dataset is.

For example, with TS12:

*.JA files, we are talking about nearly 9GB, with 16GB RAM, it can be compressed even with d1024. (~uses 12GB or more)

MSTS:
In the case of *.ACE files, if you get a total size of over 25GB even after the "xZLib+srep" phase, because we have so much starting data, then... (a lot of extra tracks, textures, compared to the 2CD release)

d128-d160, the safe limit, up to 25GB. Around 28-30GB, it's already a lot, here with d192, already at about the 80% stage, you'll run out of memory. (What's nice? After a 32-40 hour process.) If we have less data, the dictionary can be larger. This is probably due to too many matching results. After SREP, LDMF can find up to 1 million matches or even more in this example.

Example (test after xZLib+srep, With 8GB RAM from older times):

LOLZ v22c4b / LDMF1 / lolz:dt:dtb1:d160m:fba4096:mc1023:tt16:mtt0:mt1:ldmf1:ldc0:ldl5
100.00%  ldmf dec_mem=0m    37:45:22

thread 0 : w/o manager mem_usage = 239941 kb,  manager mem_usage = 49726 kb
alloc mem = 73728 kb,  stat compr_size = 2934533
matches num = 1322053,  additional mem overhead = 28329 kb

o1 model                 : 1'182'372 kb
raw graphic model 8 bit  : 7'706'602 kb
raw graphic model 16 bit : 52'900 kb
raw graphic model 24 bit : 16'728 kb
raw graphic model 32 bit : 68'208 kb
dxt1 model               : 1'275'819 kb
dxt3 model               : 292 kb
dxt5 model               : 302 kb
o1 model pos mod 2       : 99'564 kb
o1 model pos mod 4       : 61'374 kb
o1 model pos mod 8       : 562'345 kb
o1 model pos mod 16      : 76'373 kb
total size               : 11'102'884 kb

total decode mem usage = 183mb

100.0%
Errorlevel=0
Compressed 1 file, 11,369,353,897 => 3,074,609,947 bytes. Ratio 27.04%
Compression time: cpu 38.00 sec/real 136505.89 sec = 0%. Speed 0.08 mB/s
All OK

Tech infos: ~9.5GB *.ACE file -> xZlib out: ~28GB -> srep out: ~10.6GB.

In the above case, instead of breaking up the files separately, a much better solution is to use multiple solid blocks, as in FreeArc. For such 100+ GB monsters, let's use solid blocks of say 15-25GB during packaging, if we force the use of the given dictionary size/compression configuration, we do not make concessions. In the above MSTS, 25GB example, the solid block was fragmented precisely because of the 100k file limit, so I was able to avoid the possibility of running out of memory.

Has anyone seen "bc5" DDS texture info after packaging? And what could these be: "float0, float1....4" ??

[KaktoR]

The released xtool version has some bugs. Internal version 0.9.6 already exists and is in testing.

However even DSG has still some bugs which are adressed at the moment.

You can just replace xtool.exe in DSG with the new version if you like. No need to update DSG just for that.

[Cesar82]

DSG modules updated on first post.
Download and replace in DSG folder.

- Fixed bugs in DSG module.

[MartinezPL]

A little question.
How to create multiple small data.bins on ONE SINGLE DISK, instead one big one?
Maybe I doing something wrong but the end result always is multiple data.bins split to few DISKs intead all of the be on one DISK.

[Cesar82]

Originally Posted by MartinezPL

A little question.
How to create multiple small data.bins on ONE SINGLE DISK, instead one big one?
Maybe I doing something wrong but the end result always is multiple data.bins split to few DISKs intead all of the be on one DISK.

DiskSpan GUI splits the ARC files so they can be placed on DVD or BD.
It doesn't have a mode like SlicesPerDisk= using inno Setup's native compression.
The fewer parts the better the extraction will work, so putting more than one part on a disk is useless.

If you want you can choose the Custom size option from DiskSpan GUI and set the size you want and then after compressing move all files from all DISK_# folders to DISK_1 folder and change the Disk= key value of Records.ini to 1 in all sections.

[KaktoR]

Originally Posted by Gehrman

XTool_Bink:

bik.ini and bk2.ini files are also missing in this path "Resources\TOOLS\Bink" which causes compression error.

This files should be downloaded by DSG if you choose the bink video xtool plugin.

[Fak Eid]

Originally Posted by audiofeel

In my opinion, DiskSpanGui lacks a checkbox - "do everything for another unpacker and do not hide everything in setup.dll ". Then any user can compress the archive in this program and transfer all the necessary dll and ini files to his installer without incident. "FMXInno" is gaining popularity and there will be even more questions and problems with this program "DiskSpanGui".

I've something which is compatible with FMXInno and DSG, but not all methods of DSG are used in it till now. Always kept it as alpha for myself. Maybe we can work together for it.

Originally Posted by Gummy Bear

Hi, I don't understand what's wrong, I compress any file using xtool+srep+lolz, and it gives this error, I can't figure out how to fix it.

Bad compression. You need to compress the files again

[CrownRepack]

Originally Posted by Fak Eid

I've something which is compatible with FMXInno and DSG, but not all methods of DSG are used in it till now. Always kept it as alpha for myself. Maybe we can work together for it.



Bad compression. You need to compress the files again

Audiofeel and FakEid i think you two should do a colab, FMXInno is the future and DSG is best to ever exist.

[Cesar82]

Originally Posted by audiofeel

In my opinion, DiskSpanGui lacks a checkbox - "do everything for another unpacker and do not hide everything in setup.dll ". Then any user can compress the archive in this program and transfer all the necessary dll and ini files to his installer without incident. "FMXInno" is gaining popularity and there will be even more questions and problems with this program "DiskSpanGui".

Do you think DSG is hiding something from you, no, it stores the compressors in an ARC file with method "-m3" (Yes, it could be zip/7-zip/RAR, but the unpacking speed of the compressors would be harmed). This way you can replace your executable without needing to change anything else.
Example: I can take a conversion created in CIU and replace Setup.exe with one created by ASIS by other user that will continue to work, without me knowing which method was used to compress.
See for yourself...

How many times have you contributed compression information, more precisely configurations to be added to the DSG database?
If it weren't for @KakoR, who spends hours and hours carrying out tests, obtaining the best methods for each game and creating configurations, we wouldn't have any preconfiguration of methods in the DATABASE.
So be grateful for what you have, if you don't want to use DSG, don't use it.

Remembering that it is very important that you share information to the database, especially about games that are not there yet.
To share information to the new database is very simple.
After configuring your method for a game in DEVELOPER mode, just hold down the "CTRL+SHIFT" keys on your keyboard and click the "Save" button (without releasing the "CTRL+SHIFT" keys) and click OK in the dialog box that will be displayed.
The information from the database will be copied to the clipboard, so you just paste it into your collaboration post here on the DSG thead.
But only send information after testing and obtaining the best compression ratio, there is no point in sending srep+LOLZ_ULTRA for everything without test (Not everything is srep + lolz.. ).