DiskSpan GUI

[pslove]

Originally Posted by Gehrman

Your RAM is not enough to try with -a2.

I have 24 gigabytes of ram but I'll try with -a2 and report back.

EDIT: nope same error

-------------------------------------------------------------------------------------------------------------------------------------------
 Creating Setup-1a.bin file, Please Wait...
-------------------------------------------------------------------------------------------------------------------------------------------
Executing external file (Call Before)...
FreeArc 0.67 (March 15 2014) creating archive: Conversion_Output\Setup-1a.bin
Compressing 33 files, 43,614,224,208 bytes. Processed  10.0%
Compressing 92,960,894,073 bytes with "OTHERS\SREP\srep.exe" -m3f -a2 -d1g  InFile OutFile
 10.0%SREP 3.2: huge-dictionary LZ77 preprocessor   (c) Bulat.Ziganshin********com
April 6, 2013    homepage: http://freearc.org/research/SREP.aspx
Usage: SREP [options] infile [outfile]
   infile/outfile may be specified as "-" that means stdin/stdout
   "SREP [options] infile" compress data to infile.srep
   "SREP [options] infile.srep" decompress data to infile
   "SREP [options]" compress or decompress data from stdin to stdout
Options are:
   -m1: check matches by SHA1 digest (compression memory = 7-10% of filesize)
   -m2: check matches by rereading old data (compression memory = 3-6% of filesize)
   -m3: byte-accurate matches and rereading (compression memory = 7-12% of filesize)
   -l: minimum LZ match length, default 512
   -c: size of hash chunk, by default as small as required to find all these LZ matches
   -a: compression accelerator (0/1/2/4/8/16/32/64). -a0 is slowest but requires least memory
   -m1f..-m3f: future-LZ - decompression dictionary will hold only future matches
   -memBYTES: amount of RAM used by future-LZ decompression (extra goes into VM file)
      -mem75% AKA -mem75p means "use no more than 75% of RAM" - that's by default
      -mem600mb means itself
      -mem75%-600mb means "use no more than 75% of RAM minus 600 mb"
   -mBYTES: don't store matches larger than BYTES on future-LZ decompression
   -d: decompression (for -m1..m3 requires only 24 mb of memory besides of OS I/O buffers)
   -delete: delete source file after successful (de)compression
   -sBYTES: explicitly specify filesize (for compression from stdin), default 25gb
   -index=FILENAME: read/write index of compressed data into separate file
   -temp=[FILENAME]: keep uncompressed data in the file in stdin-to-stdout mode, default srep-data.tmp
   -vmfile=FILENAME: temporary file used by Virtual Memory manager, default srep-virtual-memory.tmp
   -vmblock=BYTES: size of one block in VM temporary file, default 8mb
   -nomd5: don't store/check MD5 checksum of every block
   -mmap: use memory-mapped files for match checking
   -slp[+/-/]: force/disable/try(default) large pages support
   -pc[max_offset]: display performance counters [for matches closer than max_offset]
   -s: save printed stats from overwriting
   -v[0..2]: verbosity level
   -rem...: command-line remark
Errorlevel=2
 10.0%
ERROR: general (de)compression error in srep_new:m3f:a2:d1g

[Rainiu]

I have a question about unpacking.
When my compression of Cyberdimension Neptunia Four Goddess Online was successful and I tried the unpacking option right after merging the ini, this happened:

RYHÅDKåxu±+A¼cXvD¦'T¿84tveN**-ìo÷ÄaUVu¼QUKëb7q#K«[ìaJqFlI^òLiçL*¢tYû¬Q^óêaWpºbi^íæu+mêpm“V;#öq4„M `V)à¿„mriX]©YZ
Tvzì)„¡"QY<Rhe!ua˜“ÀYQ)%4Wa÷%ª`}LO´{dT(sß,klWb Câ4$Éd/7h';m’ nrSµHZ �µVü;yIÉd}ÄŒB`$¼ˆXߣ<1I$äXhüf3ÉxdhÆih>díÇ?{· ŒñsèSæd°<²Öµ)X-“QY9jçQ “!±gêfï/ÄHôQÜ"wb¥S*°<ëaG±˜lJßa)S0Uè&½H¿ˆ$q]@=DZG½*acÿ±Ç²µDåVW°ëv9wíˆßa?¥OSx´¥«¦h]!Je“ºQ ˜é*º¿
¦“gÑÆ+RµeU"U!ˆlPöjêé:Q2tä)¬Ä}»WL>ü=^£ DaA§ìa)í<í>C!ª[)dYf€ä*ˆ.=5mYî„ÄêG
zdiäÿ=TóZ!Cx±Ñ&ô0újë VFïå/|ÅJÑS
`fDÜß¿fYúF¦ŒéIª‘AíêtKÿuéöûá fë**+'ÅèÑCC4;ñe¼Å“68m2oìaëPg*±kgP .»,leí»2i),d¿"kQXó5W)ÿ~e‘Ñ|Æ"kg�
¦Xf,U¡d^ëê’EF\iª„|§ï!lG-0ü]*lA)Vg’§‘t9µ4î’4]’€??
ê^PaÅe�HÇ¢7»Z?±Oa€ Qvg^€,ÆQ7 R#ÜUnLÆÆ¢YPOk1ßµ2“wAH_UÃ÷Æ
K*vD_»?Ñ*'ax²²?Tx[HÄÆÄF#e«?“?%Æ8?„xso/<ì_)s€àÄÆ
Extracted 13,111 files, 6,387,943,920 => 13,004,248,648 bytes. Ratio 49.12%9´,jgÄ’¡ÇÀa2Æ[Öæa’£]´]?ePåC\DoˆK˜£sÄW]êX±ŒÄkq0cÄP
8B0 b a  Ç 
Ç 0 )@jP 
Extraction time: cpu 28.36 sec/real 246.28 sec = 12%. Speed 52.80 mB/s
All OK

I compared the size of the two files and it is smaller so I thought there was a problem with the compression. I tried unpacking with the dsg.exe and the unpacking was sucessful. Those gibberish words didn't show up and the files size were similar.

[Masquerade]

Originally Posted by Rainiu

I have a question about unpacking.
When my compression of Cyberdimension Neptunia Four Goddess Online was successful and I tried the unpacking option right after merging the ini, this happened:

There's a bug in FreeArc which redirects output from STDIO tools to console window. What you are seeing isn't gibberish, but actual data.

Razor12911 explains:

some bug where it incorrectly redirects stdout to stderr (cmd window)

[Cesar82]

Updated DiskSpan GUI to v2.0.1.0 in the first post.
Special thanks to @KaktoR for all the work testing compressions and creating the DiskSpan GUI database settings.

Changes:
- Updated XTool plugin Unity Engine (R2)".
- Updated XTool precompressor from v0.4.8 to v0.5.3
  >> XTool has removed support to grittibanzli codec.
- Updated database maker "Dunia/Disrupt Engine Database maker (R4)".
- Updated DiskSpan GUI database to version 1.0.4.6 (Thanks to KaktoR).
- Added new Edit in "Developer Mode" to support load independent DSG_External.ini file for each game from the DiskSpan GUI database. 
- Added new checkbox in "Developer Mode" to run CallBefore file before starting compression and CallAfter after file only after finishing compression.
- Added file list creation for each "Records.ini" file section allowing to create a custom uninstall or for other purposes (see ISDone script Example).
  >> The list files will be included in the "Setup DLL" file which will be in the installer's temporary folder during the installation process.
  >> The list is generated with all the filenames compressed in each "Data" file, allowing you to recreate the list depending on the selected language or task options during installation.
- Added "Apply XDelta Patch" checkbox on the unpacker tab of the tools group (Tools button).
  >> If you uncheck this option and it contains an XDelta patch it will not be applied during extraction.
- Added new XTool plugin "Age of Empires III Definitive Edition" created by Edison007.
- Fixed some bugs found and several parts of the DSG code was redone in order to correct errors.

READ: I ask everyone who uses DiskSpan GUI when using a plugin to download the plugin in DEVELOPER mode and share after downloading, this way the plugin will already be in the DiskSpan GUI folder avoiding version incompatibility errors if the plugin is updated in the future.
Currently the plugin "Ubisoft Dunia 2 Engine" and "Unity Engine" have been updated in the googledrive for compatibility with the current XTool included in DiskSpan GUI 2.0.1.0, so using DiskSpan GUI 2.0.0.9 and downloading these plugins will not work .

[abror]

Originally Posted by Cesar82

Updated DiskSpan GUI to v2.0.1.0 in the first post.
Special thanks to @KaktoR for all the work testing compressions and creating the DiskSpan GUI database settings.

Changes:
- Updated XTool plugin Unity Engine (R2)".
- Updated XTool precompressor from v0.4.8 to v0.5.3
  >> XTool has removed support to grittibanzli codec.
- Updated database maker "Dunia/Disrupt Engine Database maker (R4)".
- Updated DiskSpan GUI database to version 1.0.4.6 (Thanks to KaktoR).
- Added new Edit in "Developer Mode" to support load independent DSG_External.ini file for each game from the DiskSpan GUI database. 
- Added new checkbox in "Developer Mode" to run CallBefore file before starting compression and CallAfter after file only after finishing compression.
- Added file list creation for each "Records.ini" file section allowing to create a custom uninstall or for other purposes (see ISDone script Example).
  >> The list files will be included in the "Setup DLL" file which will be in the installer's temporary folder during the installation process.
  >> The list is generated with all the filenames compressed in each "Data" file, allowing you to recreate the list depending on the selected language or task options during installation.
- Added "Apply XDelta Patch" checkbox on the unpacker tab of the tools group (Tools button).
  >> If you uncheck this option and it contains an XDelta patch it will not be applied during extraction.
- Added new XTool plugin "Age of Empires III Definitive Edition" created by Edison007.
- Fixed some bugs found and several parts of the DSG code was redone in order to correct errors.

READ: I ask everyone who uses DiskSpan GUI when using a plugin to download the plugin in DEVELOPER mode and share after downloading, this way the plugin will already be in the DiskSpan GUI folder avoiding version incompatibility errors if the plugin is updated in the future.
Currently the plugin "Ubisoft Dunia 2 Engine" and "Unity Engine" have been updated in the googledrive for compatibility with the current XTool included in DiskSpan GUI 2.0.1.0, so using DiskSpan GUI 2.0.0.9 and downloading these plugins will not work .

Hi Cesar82, thanks for helping me yesterday...

I want to ask again what compression method is the best

So far I'm using [1] Srep + Lzma I'm compressing a 200 mb file to 60 mb but last night I compressed WWE 2K22 with 10 Hours only cut very little from 49.8 GB to 49.3 GB

I read on the internet every game has a different compression method.. Do you know what the difference is?

[KaktoR]

Originally Posted by abror

Hi Cesar82, thanks for helping me yesterday...

I want to ask again what compression method is the best

So far I'm using [1] Srep + Lzma I'm compressing a 200 mb file to 60 mb but last night I compressed WWE 2K22 with 10 Hours only cut very little from 49.8 GB to 49.3 GB

I read on the internet every game has a different compression method.. Do you know what the difference is?

Next time ask such questions about compression in this topic: https://fileforums.com/showthread.php?t=99554

I don't have the game nor any game file samples, but did a quick check on steamdb and found a "oo2core_9_win64.dll" file, so I assume this game use oodle compression. Check with oo2rec precompressor (select "k9", "m9" or "s9") and take a sample file from the game and see if the output will get bigger then the input.

[mausschieber]

Originally Posted by Cesar82

Updated DiskSpan GUI to v2.0.1.0 in the first post.
Special thanks to @KaktoR for all the work testing compressions and creating the DiskSpan GUI database settings.

Changes:
- Updated XTool plugin Unity Engine (R2)".
- Updated XTool precompressor from v0.4.8 to v0.5.3
  >> XTool has removed support to grittibanzli codec.
- Updated database maker "Dunia/Disrupt Engine Database maker (R4)".
- Updated DiskSpan GUI database to version 1.0.4.6 (Thanks to KaktoR).
- Added new Edit in "Developer Mode" to support load independent DSG_External.ini file for each game from the DiskSpan GUI database. 
- Added new checkbox in "Developer Mode" to run CallBefore file before starting compression and CallAfter after file only after finishing compression.
- Added file list creation for each "Records.ini" file section allowing to create a custom uninstall or for other purposes (see ISDone script Example).
  >> The list files will be included in the "Setup DLL" file which will be in the installer's temporary folder during the installation process.
  >> The list is generated with all the filenames compressed in each "Data" file, allowing you to recreate the list depending on the selected language or task options during installation.
- Added "Apply XDelta Patch" checkbox on the unpacker tab of the tools group (Tools button).
  >> If you uncheck this option and it contains an XDelta patch it will not be applied during extraction.
- Added new XTool plugin "Age of Empires III Definitive Edition" created by Edison007.
- Fixed some bugs found and several parts of the DSG code was redone in order to correct errors.

READ: I ask everyone who uses DiskSpan GUI when using a plugin to download the plugin in DEVELOPER mode and share after downloading, this way the plugin will already be in the DiskSpan GUI folder avoiding version incompatibility errors if the plugin is updated in the future.
Currently the plugin "Ubisoft Dunia 2 Engine" and "Unity Engine" have been updated in the googledrive for compatibility with the current XTool included in DiskSpan GUI 2.0.1.0, so using DiskSpan GUI 2.0.0.9 and downloading these plugins will not work .

I think I found a Bug today i Compress the Conversion Max Payne 3
and at Finishing it Looks like That

[fabrieunko]

hello, when I download the scripts I have an error when it happens to Middle earth shadow of war^

this seems to have been corrected, I redid an update later..

[KaktoR]

Originally Posted by fabrieunko

hello, when I download the scripts I have an error when it happens to Middle earth shadow of war^

this seems to have been corrected, I redid an update later..

What error?

[abror]

Earlier I tried to download version v2.0.1.0

and LOG results

Creating Data1a-01.bin file, Please Wait...
-------------------------------------------------------------------------------------------------------------------------------------------
Could Not Find D:\Repack\COMPRESSOR\Resources\OTHERS\SREP\srep.ex e
FreeArc 0.67 (March 15 2014) creating archive: Conversion_Output\Data1a-01.bin
Compressing 680 files, 4,204,551,596 bytes. Processed 11.1%
Compressing 4,204,551,596 bytes with "OTHERS\SREP\srep.exe" -m3f -a0/0 InFile OutFile
11.5%SREP 3.93a beta (October 11, 2014): input size 4009 mb, memory used 268 mb, -m3f -l512 -c512 -a0/0 -hash=vmac -b8mb
100%: 4,204,551,596 -> 4,069,328,212: 96.78%. Cpu 63 mb/s (64.109 sec), real 58 mb/s (69.110 sec) = 93%. Remains 00:00
Decompression memory is 17 mb. 79,000 matches = 1,264,000 bytes = 0.03% of file
Errorlevel=0
Compressed 680 files, 4,204,551,596 => 3,387,460,172 bytes. Ratio 80.57%
Compression time: cpu 4.73 sec/real 1327.95 sec = 0%. Speed 3.17 mB/s
All OK

************************************************** **********

"Could Not Find D:\Repack\COMPRESSOR\Resources\OTHERS\SREP\srep.ex e"


srep.exe was not found and I've checked myself, it's not there..

Is it really like that? because the previous version of srep.exe exists

[KaktoR]

Originally Posted by abror

Earlier I tried to download version v2.0.1.0

and LOG results

Creating Data1a-01.bin file, Please Wait...
-------------------------------------------------------------------------------------------------------------------------------------------
Could Not Find D:\Repack\COMPRESSOR\Resources\OTHERS\SREP\srep.ex e
FreeArc 0.67 (March 15 2014) creating archive: Conversion_Output\Data1a-01.bin
Compressing 680 files, 4,204,551,596 bytes. Processed 11.1%
Compressing 4,204,551,596 bytes with "OTHERS\SREP\srep.exe" -m3f -a0/0 InFile OutFile
11.5%SREP 3.93a beta (October 11, 2014): input size 4009 mb, memory used 268 mb, -m3f -l512 -c512 -a0/0 -hash=vmac -b8mb
100%: 4,204,551,596 -> 4,069,328,212: 96.78%. Cpu 63 mb/s (64.109 sec), real 58 mb/s (69.110 sec) = 93%. Remains 00:00
Decompression memory is 17 mb. 79,000 matches = 1,264,000 bytes = 0.03% of file
Errorlevel=0
Compressed 680 files, 4,204,551,596 => 3,387,460,172 bytes. Ratio 80.57%
Compression time: cpu 4.73 sec/real 1327.95 sec = 0%. Speed 3.17 mB/s
All OK

************************************************** **********

"Could Not Find D:\Repack\COMPRESSOR\Resources\OTHERS\SREP\srep.ex e"


srep.exe was not found and I've checked myself, it's not there..

Is it really like that? because the previous version of srep.exe exists

Srep.exe is there like it should be. Maybe you or your AV deleted it by accident

[fabrieunko]

Originally Posted by KaktoR

What error?

when I want to download the script (bms2plugin) for metal gear rising I have this message at the bottom that appears

An error occured while downloading one or more items ty again.
I try to do it again later, I have the same message

[Cesar82]

Originally Posted by fabrieunko

when I want to download the script (bms2plugin) for metal gear rising I have this message at the bottom that appears

An error occured while downloading one or more items ty again.
I try to do it again later, I have the same message

I noticed after I uploaded DSG that two of the plugins ("Metal Gear Rising: Revengeance" and "Sonic Forces") are offering download options, but they shouldn't.
I don't have this game to generate the .xtl file (From the complete folder with all files), so the option to generate the .xtl database during compression was added (It shouldn't show in the download list, because the download list is only for .xtl databases).

For Metal Gear Rising: Revengeance use: 
xtool:c32mb:mmgrr:1

For Sonic Forces use:
xtool:c32mb:msonicf:1

Originally Posted by mausschieber

I think I found a Bug today i Compress the Conversion Max Payne 3
and at Finishing it Looks like That

Thanks for reporting the bug.
Soon I will correct and update the attachment in the first post.

Originally Posted by abror

"Could Not Find D:\Repack\COMPRESSOR\Resources\OTHERS\SREP\srep.ex e"

srep.exe was not found and I've checked myself, it's not there..

Is it really like that? because the previous version of srep.exe exists

This error does not affect functionality. I'll fix it to not display this message.
It just indicates that srep.exe cannot be deleted (didn't exist) before copying one of the executables (srep_3.92.exe or srep_3.93a.exe) to srep.exe.

EDIT:
Updated DiskSpan GUI in the first post.
I think all reported bugs have been fixed.
I ask those who downloaded the previous version to delete it and download the updated version.

EDIT:
Attached HOTFIX 2 to DiskSpan GUI 2.0.1.0 (replace).
- Fix error when copying required libraries in some methods (Thanks to KaktoR for reporting the bug).
- Fix srep method error after first arc file when used in more than one arc file in the same game (Thanks to KaktoR for reporting the bug).

[abror]

100% cpu usage

Can CPU usage be limited?

[pcblizzard]

Is there a possibility to provide version 2.0.0.9? After updating to v2.0.1.0 I have problems compiling certain files that were not there before (so I just want to test if the problem is really due to the new version).


KR pcblizzard