![]() |
Quote:
|
Quote:
|
DiskSpan GUI v2.0.2.4
The DiskSpan GUI has been updated in the first post.
I ask everyone who downloaded the previous version to delete the DSG and download the new version again from the first post (the file version remains 2.0.2.4). Several errors related to methods such as those using BMS scripts, among others, have been fixed. Several internal bugs and some visual bugs in full-size mode have also been corrected. Edit: Fixed checksum module. Download in first post if replace if your use. |
Good evening, i'd like to know how I can compress big files (for example .ucas files in "Banishers Ghosts of New Eden") into several .bin files. I mean: if I try to compress "Banishers Ghosts of New Eden" with LOLZ parameters like ldl5, my RAM (32 GB) will overflow because ldl5 file uses an increasing amount of RAM till overflow! Can I split compressed .ucas files into ucas1.bin, ucas2.bin, ecc instead of a unique ucas.bin file?
Is it a matter of DiskSpan options or anything else? |
Natively you cannot do this. You can do the following:
Split the ucas files into several parts before compression and merge them back again after unpacking. Problem: You will probably miss alot of duplicates from srep. |
Quote:
Use a smaller dictionary size and be aware of the -tt parameter. Increase your fba to 4096 if needs be. Use fewer threads if needs be. https://krinkels.org/resources/lolz.264/<-- translate the page to learn all the parameters |
First of all, thanks to KaktoR and Masquerade for replying me. The combination of parameters I use is as follow:
lolz:dtb1:d512m:mtb96:ldl5:mc1023 It's a very compressing combination, but when I try to compress .archive files in Cyberpunk, .ucas files in Banishers, or .bdt files in Armored Core VI (for example), it overflows my RAM. So i'm forced to use the following combination: lolz:dtb1:d512m:mtt1:mc1023 It compresses less then the first, but the pro is no problem with RAM overflow. Is there a more compressing combination then the second with a constant use of RAM? |
^^
d128m fba4096 You will get lesser compression but better ram usage. |
Quote:
|
From my "modest" compression experiments, I have concluded that with a given dictionary size "ex: d128m" and running on 1(!) thread, the occurrence of a given event, in this case RAM exhaustion, may also depend on how compressible and large the given dataset is.
For example, with TS12: *.JA files, we are talking about nearly 9GB, with 16GB RAM, it can be compressed even with d1024. (~uses 12GB or more) MSTS: In the case of *.ACE files, if you get a total size of over 25GB even after the "xZLib+srep" phase, because we have so much starting data, then... (a lot of extra tracks, textures, compared to the 2CD release) d128-d160, the safe limit, up to 25GB. Around 28-30GB, it's already a lot, here with d192, already at about the 80% stage, you'll run out of memory. (What's nice? After a 32-40 hour process.) If we have less data, the dictionary can be larger. This is probably due to too many matching results. After SREP, LDMF can find up to 1 million matches or even more in this example. Example (test after xZLib+srep, With 8GB RAM from older times): Code:
LOLZ v22c4b / LDMF1 / lolz:dt:dtb1:d160m:fba4096:mc1023:tt16:mtt0:mt1:ldmf1:ldc0:ldl5In the above case, instead of breaking up the files separately, a much better solution is to use multiple solid blocks, as in FreeArc. For such 100+ GB monsters, let's use solid blocks of say 15-25GB during packaging, if we force the use of the given dictionary size/compression configuration, we do not make concessions. In the above MSTS, 25GB example, the solid block was fragmented precisely because of the 100k file limit, so I was able to avoid the possibility of running out of memory. Has anyone seen "bc5" DDS texture info after packaging? And what could these be: "float0, float1....4" ?? |
Quote:
|
Quote:
|
The released xtool version has some bugs. Internal version 0.9.6 already exists and is in testing.
However even DSG has still some bugs which are adressed at the moment. You can just replace xtool.exe in DSG with the new version if you like. No need to update DSG just for that. |
DiskSpan GUI v2.0.2.4
DSG modules updated on first post.
Download and replace in DSG folder. Code:
- Fixed bugs in DSG module. |
Hello the setting "File type *.bat|.exe* to run before compress this data file" is it possible to run the file in the directory that you select before you compress the data.
Thanks. |
| All times are GMT -7. The time now is 00:49. |
Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2026, vBulletin Solutions Inc.
FileForums @ https://fileforums.com