DiskSpan GUI

[Tigru]

Originally Posted by :( Sad8669

Tigru

I am really sorry for replying this late, i was on holiday.

check it out here.

as for oodle scanner, check Razor's Oodle Precompressor (Side Project) Thread you will find it with the attachments.

No problem at all, I definitely appreciate the help, I will check it out as soon as I have some time available.

I hope you had a nice holiday!

[lolaya]

Cesar82 ♥

[Rainiu]

Hi, I need a little help editing DiskSpan Gui to include PMT to precomp/razor. I was able to get it to compress but always get a decompression error.
Razor mtx is included with the base DiskSpan Gui, but I always get a decompression error with using that and I am not sure if Precomp mtx also have that problem too. Thanks.

https://www41.zippyshare.com/v/OVLwOnFJ/file.html

[KaktoR]

Originally Posted by Rainiu

Hi, I need a little help editing DiskSpan Gui to include PMT to precomp/razor. I was able to get it to compress but always get a decompression error.
Razor mtx is included with the base DiskSpan Gui, but I always get a decompression error with using that and I am not sure if Precomp mtx also have that problem too. Thanks.

https://www41.zippyshare.com/v/OVLwOnFJ/file.html

Will be corrected in next version (will be shared soon).

[Rainiu]

Can I get a tutorial or example on how to use split mode and masks effectively on DiskSpan GUI?

[Cesar82]

Updated DiskSpan GUI to v2.0.0.9 in the first post.
Special thanks to @KaktoR for all the work testing compressions and creating the DiskSpan GUI database settings.

Changes:
- Updated XTool plugin Unreal Engine (R6)
- Updated XTool plugin EA Frostbite Engine (R7)
- Updated QuickBMS to XTool from v0.3.1 to v0.3.2
- Updated XTool precompressor from v0.4.2 to v0.4.8
- Updated key list for Unreal Engine games (with some news keys).
- Updated database maker "Unreal Engine ucas Database maker (R4)".
- Updated DiskSpan GUI database to version 1.0.4.1 (Thanks to KaktoR).
- Updated compressor/decompressor "ZStandard" (ZSTD) from v1.5.1 to v1.5.2
- Added some XTool Bms2xtl (QuickBMS) plugins for some others games.
- Added external file DSG_External.ini to include methods without having to change DSG_Methods.ini file.
- Added XMemCompress (R2), LZ4X (R1) and Crilayla (R1) tools to work together with Bms2xtl (QuickBMS to XTool).
- Added new checkbox to enable copying DiskSpan GUI executable to first media folder if setup files do not exist.
- Added support for 3 new configuration keys in Database.ini, but the content of these keys can only be manually edited in Database.ini.
  >> The key content supports the same equivalent commands supported in DSG_Methods.ini but cannot use the {compressor} and {parametter} constants, the new supported keys below:
  >> Data#.Backup#=<create backup of compressors library before compression, the file is restored after compression>
  >> Data#.CopyLib#=<copy library to compressor folder before compression, the file is delete after compression>
  >> Data#.Decomp#=<include library to creation of the Setup DLL file>
- Fixed some bugs found.

[pslove]

Hello, I tried to compress Death Stranding Director's Cut with the preset in DiskSpan GUI but I get error on srep.

ERROR: general (de)compression error in srep

Any help would be appreciated.

[KaktoR]

Originally Posted by pslove

Hello, I tried to compress Death Stranding Director's Cut with the preset in DiskSpan GUI but I get error on srep.

ERROR: general (de)compression error in srep

Any help would be appreciated.

Use this srep version
https://fileforums.com/showpost.php?...postcount=2927

replace srep.exe in "Resources\OTHERS\SREP" and try again.

If you follow the quotes you will find out that I had the same error and with provided srep version it worked.

[pslove]

Originally Posted by KaktoR

Use this srep version
https://fileforums.com/showpost.php?...postcount=2927

replace srep.exe in "Resources\OTHERS\SREP" and try again.

If you follow the quotes you will find out that I had the same error and with provided srep version it worked.

Thank you but I already did this before posting and got this error instead:

------------------------------------------------------------------------------------------------------------------------------------------
 Creating Setup-1a.bin file, Please Wait...
-------------------------------------------------------------------------------------------------------------------------------------------
Executing external file (Call Before)...
FreeArc 0.67 (March 15 2014) creating archive: Conversion_Output\Setup-1a.bin
Compressing 33 files, 43,614,224,208 bytes. Processed  10.0%
Compressing 92,960,894,073 bytes with "OTHERS\SREP\srep.exe" -m3f -a0/0 InFile OutFile
SREP 3.2: huge-dictionary LZ77 preprocessor   (c) Bulat.Ziganshin********com
April 6, 2013    homepage: http://freearc.org/research/SREP.aspx
Usage: SREP [options] infile [outfile]
   infile/outfile may be specified as "-" that means stdin/stdout
   "SREP [options] infile" compress data to infile.srep
   "SREP [options] infile.srep" decompress data to infile
   "SREP [options]" compress or decompress data from stdin to stdout
Options are:
   -m1: check matches by SHA1 digest (compression memory = 7-10% of filesize)
   -m2: check matches by rereading old data (compression memory = 3-6% of filesize)
   -m3: byte-accurate matches and rereading (compression memory = 7-12% of filesize)
   -l: minimum LZ match length, default 512
   -c: size of hash chunk, by default as small as required to find all these LZ matches
   -a: compression accelerator (0/1/2/4/8/16/32/64). -a0 is slowest but requires least memory
   -m1f..-m3f: future-LZ - decompression dictionary will hold only future matches
   -memBYTES: amount of RAM used by future-LZ decompression (extra goes into VM file)
      -mem75% AKA -mem75p means "use no more than 75% of RAM" - that's by default
      -mem600mb means itself
      -mem75%-600mb means "use no more than 75% of RAM minus 600 mb"
   -mBYTES: don't store matches larger than BYTES on future-LZ decompression
   -d: decompression (for -m1..m3 requires only 24 mb of memory besides of OS I/O buffers)
   -delete: delete source file after successful (de)compression
   -sBYTES: explicitly specify filesize (for compression from stdin), default 25gb
   -index=FILENAME: read/write index of compressed data into separate file
   -temp=[FILENAME]: keep uncompressed data in the file in stdin-to-stdout mode, default srep-data.tmp
   -vmfile=FILENAME: temporary file used by Virtual Memory manager, default srep-virtual-memory.tmp
   -vmblock=BYTES: size of one block in VM temporary file, default 8mb
   -nomd5: don't store/check MD5 checksum of every block
   -mmap: use memory-mapped files for match checking
   -slp[+/-/]: force/disable/try(default) large pages support
   -pc[max_offset]: display performance counters [for matches closer than max_offset]
   -s: save printed stats from overwriting
   -v[0..2]: verbosity level
   -rem...: command-line remark
Errorlevel=2
 10.0%
ERROR: general (de)compression error in srep_new

[KaktoR]

Use these parameters for srep (just copy&paste)

:m3f:a16:d1g

Example

[pslove]

Originally Posted by KaktoR

Use these parameters for srep (just copy&paste)

:m3f:a16:d1g

Example

Unfortunately same error I think:

-------------------------------------------------------------------------------------------------------------------------------------------
 Creating Setup-1a.bin file, Please Wait...
-------------------------------------------------------------------------------------------------------------------------------------------
Executing external file (Call Before)...
FreeArc 0.67 (March 15 2014) creating archive: Conversion_Output\Setup-1a.bin
Compressing 33 files, 43,614,224,208 bytes. Processed  10.0%
Compressing 92,960,894,073 bytes with "OTHERS\SREP\srep.exe" -m3f -a16 -d1g  InFile OutFile
SREP 3.2: huge-dictionary LZ77 preprocessor   (c) Bulat.Ziganshin********com
April 6, 2013    homepage: http://freearc.org/research/SREP.aspx
Usage: SREP [options] infile [outfile]
   infile/outfile may be specified as "-" that means stdin/stdout
   "SREP [options] infile" compress data to infile.srep
   "SREP [options] infile.srep" decompress data to infile
   "SREP [options]" compress or decompress data from stdin to stdout
Options are:
   -m1: check matches by SHA1 digest (compression memory = 7-10% of filesize)
   -m2: check matches by rereading old data (compression memory = 3-6% of filesize)
   -m3: byte-accurate matches and rereading (compression memory = 7-12% of filesize)
   -l: minimum LZ match length, default 512
   -c: size of hash chunk, by default as small as required to find all these LZ matches
   -a: compression accelerator (0/1/2/4/8/16/32/64). -a0 is slowest but requires least memory
   -m1f..-m3f: future-LZ - decompression dictionary will hold only future matches
   -memBYTES: amount of RAM used by future-LZ decompression (extra goes into VM file)
      -mem75% AKA -mem75p means "use no more than 75% of RAM" - that's by default
      -mem600mb means itself
      -mem75%-600mb means "use no more than 75% of RAM minus 600 mb"
   -mBYTES: don't store matches larger than BYTES on future-LZ decompression
   -d: decompression (for -m1..m3 requires only 24 mb of memory besides of OS I/O buffers)
   -delete: delete source file after successful (de)compression
   -sBYTES: explicitly specify filesize (for compression from stdin), default 25gb
   -index=FILENAME: read/write index of compressed data into separate file
   -temp=[FILENAME]: keep uncompressed data in the file in stdin-to-stdout mode, default srep-data.tmp
   -vmfile=FILENAME: temporary file used by Virtual Memory manager, default srep-virtual-memory.tmp
   -vmblock=BYTES: size of one block in VM temporary file, default 8mb
   -nomd5: don't store/check MD5 checksum of every block
   -mmap: use memory-mapped files for match checking
   -slp[+/-/]: force/disable/try(default) large pages support
   -pc[max_offset]: display performance counters [for matches closer than max_offset]
   -s: save printed stats from overwriting
   -v[0..2]: verbosity level
   -rem...: command-line remark
Errorlevel=2
 10.0%
ERROR: general (de)compression error in srep_new:m3f:a16:d1g

[Gehrman]

Your RAM is not enough to try with -a2.

[pslove]

Originally Posted by Gehrman

Your RAM is not enough to try with -a2.

I have 24 gigabytes of ram but I'll try with -a2 and report back.

EDIT: nope same error

-------------------------------------------------------------------------------------------------------------------------------------------
 Creating Setup-1a.bin file, Please Wait...
-------------------------------------------------------------------------------------------------------------------------------------------
Executing external file (Call Before)...
FreeArc 0.67 (March 15 2014) creating archive: Conversion_Output\Setup-1a.bin
Compressing 33 files, 43,614,224,208 bytes. Processed  10.0%
Compressing 92,960,894,073 bytes with "OTHERS\SREP\srep.exe" -m3f -a2 -d1g  InFile OutFile
 10.0%SREP 3.2: huge-dictionary LZ77 preprocessor   (c) Bulat.Ziganshin********com
April 6, 2013    homepage: http://freearc.org/research/SREP.aspx
Usage: SREP [options] infile [outfile]
   infile/outfile may be specified as "-" that means stdin/stdout
   "SREP [options] infile" compress data to infile.srep
   "SREP [options] infile.srep" decompress data to infile
   "SREP [options]" compress or decompress data from stdin to stdout
Options are:
   -m1: check matches by SHA1 digest (compression memory = 7-10% of filesize)
   -m2: check matches by rereading old data (compression memory = 3-6% of filesize)
   -m3: byte-accurate matches and rereading (compression memory = 7-12% of filesize)
   -l: minimum LZ match length, default 512
   -c: size of hash chunk, by default as small as required to find all these LZ matches
   -a: compression accelerator (0/1/2/4/8/16/32/64). -a0 is slowest but requires least memory
   -m1f..-m3f: future-LZ - decompression dictionary will hold only future matches
   -memBYTES: amount of RAM used by future-LZ decompression (extra goes into VM file)
      -mem75% AKA -mem75p means "use no more than 75% of RAM" - that's by default
      -mem600mb means itself
      -mem75%-600mb means "use no more than 75% of RAM minus 600 mb"
   -mBYTES: don't store matches larger than BYTES on future-LZ decompression
   -d: decompression (for -m1..m3 requires only 24 mb of memory besides of OS I/O buffers)
   -delete: delete source file after successful (de)compression
   -sBYTES: explicitly specify filesize (for compression from stdin), default 25gb
   -index=FILENAME: read/write index of compressed data into separate file
   -temp=[FILENAME]: keep uncompressed data in the file in stdin-to-stdout mode, default srep-data.tmp
   -vmfile=FILENAME: temporary file used by Virtual Memory manager, default srep-virtual-memory.tmp
   -vmblock=BYTES: size of one block in VM temporary file, default 8mb
   -nomd5: don't store/check MD5 checksum of every block
   -mmap: use memory-mapped files for match checking
   -slp[+/-/]: force/disable/try(default) large pages support
   -pc[max_offset]: display performance counters [for matches closer than max_offset]
   -s: save printed stats from overwriting
   -v[0..2]: verbosity level
   -rem...: command-line remark
Errorlevel=2
 10.0%
ERROR: general (de)compression error in srep_new:m3f:a2:d1g

[Rainiu]

I have a question about unpacking.
When my compression of Cyberdimension Neptunia Four Goddess Online was successful and I tried the unpacking option right after merging the ini, this happened:

RYHÅDKåxu±+A¼cXvD¦'T¿84tveN**-ìo÷ÄaUVu¼QUKëb7q#K«[ìaJqFlI^òLiçL*¢tYû¬Q^óêaWpºbi^íæu+mêpm“V;#öq4„M `V)à¿„mriX]©YZ
Tvzì)„¡"QY<Rhe!ua˜“ÀYQ)%4Wa÷%ª`}LO´{dT(sß,klWb Câ4$Éd/7h';m’ nrSµHZ �µVü;yIÉd}ÄŒB`$¼ˆXߣ<1I$äXhüf3ÉxdhÆih>díÇ?{· ŒñsèSæd°<²Öµ)X-“QY9jçQ “!±gêfï/ÄHôQÜ"wb¥S*°<ëaG±˜lJßa)S0Uè&½H¿ˆ$q]@=DZG½*acÿ±Ç²µDåVW°ëv9wíˆßa?¥OSx´¥«¦h]!Je“ºQ ˜é*º¿
¦“gÑÆ+RµeU"U!ˆlPöjêé:Q2tä)¬Ä}»WL>ü=^£ DaA§ìa)í<í>C!ª[)dYf€ä*ˆ.=5mYî„ÄêG
zdiäÿ=TóZ!Cx±Ñ&ô0újë VFïå/|ÅJÑS
`fDÜß¿fYúF¦ŒéIª‘AíêtKÿuéöûá fë**+'ÅèÑCC4;ñe¼Å“68m2oìaëPg*±kgP .»,leí»2i),d¿"kQXó5W)ÿ~e‘Ñ|Æ"kg�
¦Xf,U¡d^ëê’EF\iª„|§ï!lG-0ü]*lA)Vg’§‘t9µ4î’4]’€??
ê^PaÅe�HÇ¢7»Z?±Oa€ Qvg^€,ÆQ7 R#ÜUnLÆÆ¢YPOk1ßµ2“wAH_UÃ÷Æ
K*vD_»?Ñ*'ax²²?Tx[HÄÆÄF#e«?“?%Æ8?„xso/<ì_)s€àÄÆ
Extracted 13,111 files, 6,387,943,920 => 13,004,248,648 bytes. Ratio 49.12%9´,jgÄ’¡ÇÀa2Æ[Öæa’£]´]?ePåC\DoˆK˜£sÄW]êX±ŒÄkq0cÄP
8B0 b a  Ç 
Ç 0 )@jP 
Extraction time: cpu 28.36 sec/real 246.28 sec = 12%. Speed 52.80 mB/s
All OK

I compared the size of the two files and it is smaller so I thought there was a problem with the compression. I tried unpacking with the dsg.exe and the unpacking was sucessful. Those gibberish words didn't show up and the files size were similar.

[Masquerade]

Originally Posted by Rainiu

I have a question about unpacking.
When my compression of Cyberdimension Neptunia Four Goddess Online was successful and I tried the unpacking option right after merging the ini, this happened:

There's a bug in FreeArc which redirects output from STDIO tools to console window. What you are seeing isn't gibberish, but actual data.

Razor12911 explains:

some bug where it incorrectly redirects stdout to stderr (cmd window)