CLS-DiskSpan (UltraARC replacement)

[Razor12911]

I'll check it out

[BLACKFIRE69]

Originally Posted by Cesar82

Yes, I know that, but it's still great.
NORMAL: 1130 Kb (1.10 MB) >> UPX = 288 Kb
if compile with DELPHI 2010 (but not work) = NORMAL: 339 Kb >> UPX 119 Kb.
I will try with other dephi XE versions later.
But it would be useful to know what would be the minimum version of DELPHI that this source works for.
Thank you!

Cesar82,

I was able to reduce the size to 249kb with (Delphi Rio + UPX)


,

[Cesar82]

@BLACKFIRE69
I got smaller sizes using DELPHI 10 Seattle.
From XE8 all functions/classes are coded without any changes.
Using DELPHI XE4 through XE7 it is only necessary to include a variable to make the script compatible.
I tested the version compiled with DELPHI Seattle and it is working fine.
Only the same errors happen as in the original lib that Razor12911 shared.
I did not test these older versions XE4 to XE7, because as the Lib compiled with DELPHI Seattle was better than with previous versions it does not make sense to test.


Adaptation of DiskSpan R3 to work with old delphi (Delphi XE4...XE7)

Changhed this:

begin
  FormatSettings := TFormatSettings.Invariant;

Changed To:

var
  FormatSettings: TFormatSettings;
begin
  GetLocaleFormatSettings(LOCALE_SYSTEM_DEFAULT, FormatSettings);

I think that this would not bring errors in the use of the library.
But as the size of the library compiled with DELPHI Seatle is smaller, it is more viable to use it.
This other executable compressor used from the image "uPACK" I found the compression rate very good and super fast, but in some libs it brings false positives.

[BLACKFIRE69]

Originally Posted by Cesar82

@BLACKFIRE69
I got smaller sizes using DELPHI 10 Seattle.
From XE8 all functions/classes are coded without any changes.

Adaptation of DiskSpan R3 to work with old delphi (Delphi XE4...XE7)

Changhed this:

begin
  FormatSettings := TFormatSettings.Invariant;

Changed To:

var
  FormatSettings: TFormatSettings;
begin
  GetLocaleFormatSettings(LOCALE_SYSTEM_DEFAULT, FormatSettings);

Cesar,

I compiled it in Delphi Seattle without changing and changing the code, but it wasn't working.

[Cesar82]

Originally Posted by BLACKFIRE69

Cesar,
I compiled it in Delphi Seattle without changing and changing the code, but it wasn't working.

Thanks for the info.
There's no logic (I don't understand why Seattle isn't even that old).
Here it was working when I tested it before (I only tested the extraction using CIU).
I didn’t even remember to try to pack it with him because the idea was to include it in CIU to perform the extraction.
I'm going to do some tests in a few days.

[Cesar82]

@Razor12911
If possible, look at the attached file.
The native methods of freearc "2", "3", "4", "5" do not work if you include cls diskspan R2/R3.
Or am I setting the command line wrong?
P.S. If I remove the diskspan from the command line it compresses normal.
P.S. The other native methods "0", "1", "9x -ld192m", "x -ld800m", "x -ld1600m" work normally.
Thank you!

[Razor12911]

Originally Posted by Cesar82

@Razor12911
If possible, look at the attached file.
The native methods of freearc "2", "3", "4", "5" do not work if you include cls diskspan R2/R3.
Or am I setting the command line wrong?
P.S. If I remove the diskspan from the command line it compresses normal.
P.S. The other native methods "0", "1", "9x -ld192m", "x -ld800m", "x -ld1600m" work normally.
Thank you!

have you tried to use other cls other than diskspan to see how it reacts?
1.PNG

2.PNG

3.PNG

the cls by itself...

4.PNG

perhaps this is why I have never mixed default freearc methods with additional methods set by user (-m1..9,x that is)

[Cesar82]

Originally Posted by Razor12911

have you tried to use other cls other than diskspan to see how it reacts?

Do not.
I just tested how I sent the file and realized that from method 2..5 it showed the same error as the third image.
I found it strange to support method 0 and 1 and the others display error.
See the result in the VIDEO of my test using only CLS-DiskSpan.dll + Arc Method

Do you mean that FreeArc does not support combining methods 2 - 5 with CLS?

I wanted to enable users to use these native methods when using diskspan.bat that I changed.
I came to think that it could be some config of the arc.ini that I used so I took Arc.ini and arc.groups directly from the freearc folder and even so when using cls-diskspan it showed errors and without it it does not display an error.
Thanks.

I ran some tests and found that:
If you open a compressed file with FreeArc and go to Info, only with the native method 2..5 it is possible to obtain the combination of the method referring to preset 2..5.


If you include these methods as presets of 2..5 in Arc.ini, the methods are now supported by Cls DiskSpan


Just adding the Arc.ini presets 2..5 makes it possible to support methods using cls-diskspan.

[Compression methods]

2=rep:1kb:256+exe+4x4:tor:1kb:h4kb
3=rep:1kb+exe+delta+4x4:lzma:32kb:fast:32:mc4
4=rep:1kb+exe+delta+4x4:lzma:32kb:normal:16:mc8
5=rep:1kb+exe+delta+4x4:lzma:32kb:normal:32:mc32

After opening the compressed file with FreeArc it is possible to verify that the method was used correctly.

[Razor12911]

I was going to say, that even using these

Arc.exe a -cfgArc.ini -ep1 -r -ed -s; -w.\TEMP -m2+rep -dp".\files" ".\Data2.bin.001"
Arc.exe a -cfgArc.ini -ep1 -r -ed -s; -w.\TEMP -m3+rep -dp".\files" ".\Data3.bin.001"
Arc.exe a -cfgArc.ini -ep1 -r -ed -s; -w.\TEMP -m4+rep -dp".\files" ".\Data4.bin.001"
Arc.exe a -cfgArc.ini -ep1 -r -ed -s; -w.\TEMP -m5+rep -dp".\files" ".\Data5.bin.001"

causes the same errors so it's not just cls issue.
I had to look at Freearc source and to give proper aliases for m2-5, you should look at this

5.PNG

-m2 is actually -mrep:96m:256:c256+exe+tor:6

use this to fill up the rest

[Cesar82]

@Razor12911
Thanks for the info, but I don't understand this FreeArc souce language (I was even more confused).
You could share the native presets (0..5 and "9x -ld192m", "x -ld800m", "x -ld1600m" (UltrArc GUI predifinitions)) so that I can include them in the [Compression Methods] section of Arc.ini.
I don't know if I can use memos numbers as a preset (Only with DiskSpan using 2..5 in the presets it worked here).

Currently I used the "Arc.ini" file for DiskSpan.bat "(UltraARC 2900 R8 + DiskSpan.bat)".
Apparently it's working well (I don't know if it's the right way)
In this way the user can use an option such as "3" or "FA-FAST" (optional) that will have the same result.

[Compression Methods]
FA-STORE=0
FA-MIN=1
FA-LOW=2
FA-FAST=3
FA-NORMAL=4
FA-HIGHT=5
FA-BEST=9x -ld192m
FA-MAX=x -ld800m
FA-ULTRA=x -ld1600m

2=rep:1kb:256+exe+4x4:tor:1kb:h4kb
3=rep:1kb+exe+delta+4x4:lzma:32kb:fast:32:mc4
4=rep:1kb+exe+delta+4x4:lzma:32kb:normal:16:mc8
5=rep:1kb+exe+delta+4x4:lzma:32kb:normal:32:mc32

[Razor12911]

Originally Posted by Cesar82

Thanks for the info, but I don't understand this FreeArc souce language (I was even more confused).
You could share the native presets (0..5 and "9x -ld192m", "x -ld800m", "x -ld1600m" (UltrArc GUI predifinitions)) so that I can include them in the

don't be lazy bro

5.PNG

# is a number, like -m2
if one uses 2, it then triggers the next part #rep+exe+#xb
then you check what 2rep+exe+2xb is
2rep is rep:96m:256:c256
2xb is tor:6
then you have rep:96m:256:c256+exe+tor:6

is the same when you use 5, it's not that difficult to follow.

when # is 5, we have 5rep+exe+5xb
5rep is rep:96m
#xb for 5 is delta + #binary

we check what 5binary is
5binary is lzma:96m:normal:32:mc32

then you combine everything
rep:96m+exe+delta+lzma:96m:normal:32:mc32

m9x follows this #x
where #x = #xb/#xt
9x= 9xb =delta + 9binary = delta+lzma:254m:max
#xt is a mask for text ($text)
9xt=dict:128m:75% + 9binary = dict:128m:75%+delta+lzma:254m:max

not sure about the 75% part of dict but that is what the method says

I also don't know this programming language either but I understood it just by looking at it, it's not that hard to parse the other methods.

[Cesar82]

Originally Posted by Razor12911

# is a number, like -m2
if one uses 2, it then triggers the next part #rep+exe+#xb
then you check what 2rep+exe+2xb is
2rep is rep:96m:256:c256
2xb is tor:6
then you have rep:96m:256:c256+exe+tor:6

I understood the line of reasoning.
I just don't understand why the final file has different compression following this line of reasoning.

Where can I get this freearc src?

[Razor12911]

Originally Posted by Cesar82

I understood the line of reasoning.
I just don't understand why the final file has different compression following this line of reasoning.

looks like I missed this part, you can just go ahead and write xtor:5 instead of tor:6
7.PNG
8.PNG

Where can I get this freearc src?

it's part of this project
https://krinkels.org/resources/fa_protect.5/

[BLACKFIRE69]

CLS·DiskSpan v2.0
Reliability + Performance Overhaul for FreeArc Disk-Spanning
Windows · 32-bit plugin · built on Razor12911's original · modified by BLACKFIRE69

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ━━━━━━━━━━━━

▌ OVERVIEW
A big reliability + speed overhaul of Razor12911's original FreeArc disk-spanning plugin. The plugin splits a FreeArc archive across multiple volumes (discs) and reassembles them on extraction.

⚠ Matched-pair release: the on-disk volume format changed (clean break). Archives written by v2.0 are unpacked by v2.0 only — v2.0 will not read old volumes, and the old version will not read v2.0 volumes.

▌ TECHNICAL SPECIFICATIONS

• Version — v2.0
• Built with — RAD Studio 13.1
• Platform — Windows, 32-bit
• Targets — FreeArc 0.67 (March 15 2014)
• Disc format — self-describing v2 volumes: a "set GUID" shared by every disc of one archive, plus a CRC32 on every disc
• I/O buffer — 1 MB default; override with env var CLS_BUFKB (KB, clamped 64 KB .. 64 MB)
• Debug log — OFF by default; set env var CLS_DEBUG_LOG=1 to append diagnostics to cls-diskspan.log next to the DLL

▌ CREDITS / AUTHORS

• Original — Razor12911
• v2.0 overhaul — BLACKFIRE69

▌ WHAT'S NEW IN v2.0

✦ It just works for EVERY layout
Solid (-s;), non-solid (-s-), big/small solid blocks (-s<size>), and file-mask / grouped packs (arc.groups) all span and round-trip correctly. The old "must be fully solid" limitation — where anything else silently produced broken discs — is GONE.

✦ Mixed group sizes are flagged
With file-mask groups, all groups share ONE volume set, so only a single disc size is used (the first block's). A group asking for a different size is kept-and-warned once — the set still packs/unpacks fine. Use the same size on every group.

✦ Damage is caught
Every disc carries a CRC32 and shares a "set GUID". A truncated, byte-flipped, swapped, or foreign disc is detected and reported instead of silently producing a bad install.

✦ No more hangs or crashes
A missing disc in a silent/hidden run fails cleanly with a non-zero exit (never freezes on a hidden dialog), and no Delphi error can crash back into FreeArc — failures map to proper CLS error codes.

✦ Safe inside installers
The --sort / --makeiso / --version helpers only run under FreeArc's Arc.exe; embedded in an Inno Setup installer (unarc) the DLL never touches the host process, so it can't accidentally close the installer.

✦ You see the REAL reason a span failed
On a bad decode FreeArc otherwise prints only its own misleading line. diskspan now prints the true cause first, on its own line — e.g. "diskspan: missing volume game-3.bin" or "diskspan: CRC mismatch ...".

✦ An archive kept in a subfolder extracts correctly
Sibling discs are found next to the main archive, not in the current directory — so arc t out\game-1.bin works (the real conversion flow packs into a subfolder).

✦ The disc-tools are reliable
--sort lays the discs into burnable DISC_1, DISC_2 … folders (with autorun.inf + setup files) and exits cleanly. --makeiso turns those folders into .ISO images without freezing; a folder is deleted ONLY after its ISO is safely written, so a failing oscdimg.exe can never destroy your discs.

✦ Flexible disc naming (auto-detected — no new option)

game-1.bin      ->  game-2.bin,  game-3.bin       (classic digit step)
data.bin.001    ->  data.bin.002, data.bin.003    (numbered extension)
setup-1a.bin    ->  setup-1b.bin ... 1z, 1aa      (letters per disc, number fixed)

Put a number in the name. If you forget, it still works, but extra discs get plain names (game_002.bin …) and you get a reminder at pack time.

✦ Locate-a-missing-disc
In an interactive run (installer window OR plain console arc x), a Browse dialog lets the user point to a disc that isn't where expected, then the extract continues. Silent/hidden/redirected runs just fail cleanly.

✦ It's faster
The per-disc checksum was rewritten (slicing-by-8) and I/O runs in 1 MB chunks (tunable), cutting spanning overhead from ~+96% over a raw store down to ~+20%. At real multi-GB disc sizes the compressor dominates anyway, so spanning is effectively free.

✦ Version probe
arc.exe --version prints build, disc-format version and copyright; the DiskSpanInfo() export returns the same string to tools.

▌ ORIGINAL vs MODIFIED (v1 → v2.0)

╭───────────────────────────┬────────────────────────┬────────────────────────────────╮
│ Area                      │ Original (v1)          │ Modified (v2.0)                │
├───────────────────────────┼────────────────────────┼────────────────────────────────┤
│ Solid archives            │ OK                     │ OK                             │
│ Non-solid (-s-)           │ SILENT CORRUPTION      │ OK                             │
│ Small blocks (-s<size>)   │ SILENT CORRUPTION      │ OK                             │
│ File masks / arc.groups   │ unsupported / corrupt  │ OK                             │
│ Mixed group sizes         │ SILENT CORRUPTION      │ OK (first size wins + warns)   │
│ Archive in a subfolder    │ extract could fail     │ OK                             │
│ Per-volume integrity      │ none                   │ CRC32 (header + payload)       │
│ Wrong / foreign volume    │ used blindly           │ rejected (set GUID)            │
│ Missing volume (hidden)   │ could hang on dialog   │ clean nonzero exit             │
│ Missing volume (UI)       │ host prompt only       │ browse (UI or console)         │
│ Failure reason shown      │ FreeArc generic only   │ + diskspan's real reason       │
│ Disc-1 size budget        │ per solid block        │ shared across all blocks       │
│ Error across DLL boundary │ could crash FreeArc    │ mapped to CLS error codes      │
│ Volume naming             │ digit substitution     │ Type 1 / 2 / 3 (auto)          │
│ Non-numbered name         │ silent _NNN fallback   │ warns at pack time             │
│ --sort exit code          │ wrong ("unknown cmd")  │ clean exit 0                   │
│ --makeiso                 │ hung after first ISO   │ builds all, safe delete        │
│ CRC speed                 │ n/a (no CRC)           │ slicing-by-8 (~20% over)       │
│ Copy buffer               │ 64 KB                  │ 1 MB (tunable, CLS_BUFKB)      │
│ Version/copyright probe   │ none                   │ --version + DiskSpanInfo()     │
│ Build toolchain           │ older Delphi           │ RAD Studio 13.1, Win32         │
│ Compiler warnings         │ several                │ zero (clean build)             │
╰───────────────────────────┴────────────────────────┴────────────────────────────────╯

▌ EXPORTED API FUNCTIONS

• ClsInit — host hands over the source path + a parent-window handle (used by Inno Setup installers)
• ClsFunc1 (ClsSourcePath) — host sets/updates the folder to read sibling discs (2, 3 …) from
• ClsFunc2 (ClsDiskRequest) — host installs a callback diskspan can call to ask the user to "insert the next disc"
• DiskSpanInfo — extra (not called by FreeArc): returns build + format-version + copyright so any tool can identify the DLL

▌ QUICK START EXAMPLES
Run from the folder holding Arc.exe + cls-diskspan.dll. -w sets a temp folder; diskspan:<first>:<other> = first-disc size : every other disc's size. Put a NUMBER in the archive name.
Legend: program · command · method / diskspan · archive · input mask

Show the build + copyright

Arc.exe --version

Pack + split — one solid block, no compression (fast smoke test)

Arc.exe a -ep1 -r -ed -s; -w.\temp -m0+diskspan:2mb:2mb out\game-1.bin "_data\*"

Pack + split — REAL compression (put diskspan LAST in the chain)

Arc.exe a -ep1 -r -ed -s; -w.\temp -msrep+lzma+diskspan:4467mb:4474mb out\game-1.bin "MyFiles\*"

Pack + split — NON-solid (-s-, every file its own block — the old silent-corruption case, now correct)

Arc.exe a -ep1 -r -ed -s- -w.\temp -m0+diskspan:2mb:2mb out\game-1.bin "_data\*"

Pack + split — SMALL solid blocks (-s<size>, e.g. 8 MB blocks)

Arc.exe a -ep1 -r -ed -s8m -w.\temp -m0+diskspan:2mb:2mb out\game-1.bin "_data\*"

Pack + split — file-MASK groups (end each group's method with diskspan so every group spans)

Arc.exe a -ep1 -r -ed -s; -w.\temp -mtor+diskspan:2mb:2mb/$grpbin=lzma+diskspan:2mb:2mb out\game-1.bin "_data\*"

Test the disc set (reads every disc + checks each disc's CRC)

Arc.exe t out\game-1.bin

Extract

Arc.exe x -dpunpacked\ out\game-1.bin

Lay discs into burnable folders, then build one ISO per folder

Arc.exe --sort    OutFolder DISC_ setup.exe setup.ico out\game-1.bin
Arc.exe --makeiso OutFolder\DISC_

Pick a naming style just by how you name the first disc

Arc.exe a ... +diskspan:300mb:400mb data.bin.001 "MyFiles\*"   (-> .002, .003)
Arc.exe a ... +diskspan:300mb:400mb setup-1a.bin "MyFiles\*"   (-> 1b, 1c, ...)

Tuning / diagnostics (env vars, optional)

setlocal & set CLS_BUFKB=4096   & Arc.exe ...   (4 MB I/O buffer instead of 1 MB)
setlocal & set CLS_DEBUG_LOG=1  & Arc.exe ...   (write cls-diskspan.log next to the DLL)

Ready-to-run, double-clickable versions of all of the above are in bin\_examples\_demo\ — start with 00_version.bat.

▌ TESTS & BENCHMARKS
Both live in bin\_test\ and drive the same Arc.exe + cls-diskspan.dll in bin\.

run_tests.ps1 — round-trip regression suite. Packs 8 scenarios (solid, single-volume, store, groups, non-solid, small-block, groups+blocks, many-volume), extracts each, and SHA-256 compares against the originals. Plus 3 robustness checks: a missing middle disc must fail cleanly, a byte-flipped disc must be caught by the CRC, and a subfolder archive must still round-trip. Prints a PASS/FAIL matrix.

powershell -ExecutionPolicy Bypass -File run_tests.ps1 -DebugLog

bench.ps1 — throughput benchmark. Uses a STORE method (-m0) on purpose, so timing reflects the spanning engine + disk I/O, not a compressor. Packs/extracts at three I/O buffer sizes (CLS_BUFKB = 64 / 1024 / 4096 KB), verifies SHA-256 each time, and prints encode/decode MB/s per buffer size.

powershell -ExecutionPolicy Bypass -File bench.ps1 -SizeMB 120

▌ DISTRIBUTION FILES

• cls-diskspan.dll — REQUIRED. The plugin. Drop it next to Arc.exe.
• Arc.exe — REQUIRED. FreeArc's command-line driver (ships in bin\).
• oscdimg.exe — Optional. Only for --makeiso; must sit next to Arc.exe.
• arc.ini — Optional. FreeArc method config (example in bin\).
• arc.groups — Optional. File-grouping masks (example in bin\).

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ━━━━━━━━━━━━

▌ A NOTE FROM THE MODDER
I'm not quite a repack guy, so do some rough tests on this with your own data and feel free to report any bug / issue.


.