CLS-DiskSpan (UltraARC replacement)

[BLACKFIRE69]

CLS·DiskSpan v2.0
Reliability + Performance Overhaul for FreeArc Disk-Spanning
Windows · 32-bit plugin · built on Razor12911's original · modified by BLACKFIRE69

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ━━━━━━━━━━━━

▌ OVERVIEW
A big reliability + speed overhaul of Razor12911's original FreeArc disk-spanning plugin. The plugin splits a FreeArc archive across multiple volumes (discs) and reassembles them on extraction.

⚠ Matched-pair release: the on-disk volume format changed (clean break). Archives written by v2.0 are unpacked by v2.0 only — v2.0 will not read old volumes, and the old version will not read v2.0 volumes.

▌ TECHNICAL SPECIFICATIONS

• Version — v2.0
• Built with — RAD Studio 13.1
• Platform — Windows, 32-bit
• Targets — FreeArc 0.67 (March 15 2014)
• Disc format — self-describing v2 volumes: a "set GUID" shared by every disc of one archive, plus a CRC32 on every disc
• I/O buffer — 1 MB default; override with env var CLS_BUFKB (KB, clamped 64 KB .. 64 MB)
• Debug log — OFF by default; set env var CLS_DEBUG_LOG=1 to append diagnostics to cls-diskspan.log next to the DLL

▌ CREDITS / AUTHORS

• Original — Razor12911
• v2.0 overhaul — BLACKFIRE69

▌ WHAT'S NEW IN v2.0

✦ It just works for EVERY layout
Solid (-s;), non-solid (-s-), big/small solid blocks (-s<size>), and file-mask / grouped packs (arc.groups) all span and round-trip correctly. The old "must be fully solid" limitation — where anything else silently produced broken discs — is GONE.

✦ Mixed group sizes are flagged
With file-mask groups, all groups share ONE volume set, so only a single disc size is used (the first block's). A group asking for a different size is kept-and-warned once — the set still packs/unpacks fine. Use the same size on every group.

✦ Damage is caught
Every disc carries a CRC32 and shares a "set GUID". A truncated, byte-flipped, swapped, or foreign disc is detected and reported instead of silently producing a bad install.

✦ No more hangs or crashes
A missing disc in a silent/hidden run fails cleanly with a non-zero exit (never freezes on a hidden dialog), and no Delphi error can crash back into FreeArc — failures map to proper CLS error codes.

✦ Safe inside installers
The --sort / --makeiso / --version helpers only run under FreeArc's Arc.exe; embedded in an Inno Setup installer (unarc) the DLL never touches the host process, so it can't accidentally close the installer.

✦ You see the REAL reason a span failed
On a bad decode FreeArc otherwise prints only its own misleading line. diskspan now prints the true cause first, on its own line — e.g. "diskspan: missing volume game-3.bin" or "diskspan: CRC mismatch ...".

✦ An archive kept in a subfolder extracts correctly
Sibling discs are found next to the main archive, not in the current directory — so arc t out\game-1.bin works (the real conversion flow packs into a subfolder).

✦ The disc-tools are reliable
--sort lays the discs into burnable DISC_1, DISC_2 … folders (with autorun.inf + setup files) and exits cleanly. --makeiso turns those folders into .ISO images without freezing; a folder is deleted ONLY after its ISO is safely written, so a failing oscdimg.exe can never destroy your discs.

✦ Flexible disc naming (auto-detected — no new option)

game-1.bin      ->  game-2.bin,  game-3.bin       (classic digit step)
data.bin.001    ->  data.bin.002, data.bin.003    (numbered extension)
setup-1a.bin    ->  setup-1b.bin ... 1z, 1aa      (letters per disc, number fixed)

Put a number in the name. If you forget, it still works, but extra discs get plain names (game_002.bin …) and you get a reminder at pack time.

✦ Locate-a-missing-disc
In an interactive run (installer window OR plain console arc x), a Browse dialog lets the user point to a disc that isn't where expected, then the extract continues. Silent/hidden/redirected runs just fail cleanly.

✦ It's faster
The per-disc checksum was rewritten (slicing-by-8) and I/O runs in 1 MB chunks (tunable), cutting spanning overhead from ~+96% over a raw store down to ~+20%. At real multi-GB disc sizes the compressor dominates anyway, so spanning is effectively free.

✦ Version probe
arc.exe --version prints build, disc-format version and copyright; the DiskSpanInfo() export returns the same string to tools.

▌ ORIGINAL vs MODIFIED (v1 → v2.0)

╭───────────────────────────┬────────────────────────┬────────────────────────────────╮
│ Area                      │ Original (v1)          │ Modified (v2.0)                │
├───────────────────────────┼────────────────────────┼────────────────────────────────┤
│ Solid archives            │ OK                     │ OK                             │
│ Non-solid (-s-)           │ SILENT CORRUPTION      │ OK                             │
│ Small blocks (-s<size>)   │ SILENT CORRUPTION      │ OK                             │
│ File masks / arc.groups   │ unsupported / corrupt  │ OK                             │
│ Mixed group sizes         │ SILENT CORRUPTION      │ OK (first size wins + warns)   │
│ Archive in a subfolder    │ extract could fail     │ OK                             │
│ Per-volume integrity      │ none                   │ CRC32 (header + payload)       │
│ Wrong / foreign volume    │ used blindly           │ rejected (set GUID)            │
│ Missing volume (hidden)   │ could hang on dialog   │ clean nonzero exit             │
│ Missing volume (UI)       │ host prompt only       │ browse (UI or console)         │
│ Failure reason shown      │ FreeArc generic only   │ + diskspan's real reason       │
│ Disc-1 size budget        │ per solid block        │ shared across all blocks       │
│ Error across DLL boundary │ could crash FreeArc    │ mapped to CLS error codes      │
│ Volume naming             │ digit substitution     │ Type 1 / 2 / 3 (auto)          │
│ Non-numbered name         │ silent _NNN fallback   │ warns at pack time             │
│ --sort exit code          │ wrong ("unknown cmd")  │ clean exit 0                   │
│ --makeiso                 │ hung after first ISO   │ builds all, safe delete        │
│ CRC speed                 │ n/a (no CRC)           │ slicing-by-8 (~20% over)       │
│ Copy buffer               │ 64 KB                  │ 1 MB (tunable, CLS_BUFKB)      │
│ Version/copyright probe   │ none                   │ --version + DiskSpanInfo()     │
│ Build toolchain           │ older Delphi           │ RAD Studio 13.1, Win32         │
│ Compiler warnings         │ several                │ zero (clean build)             │
╰───────────────────────────┴────────────────────────┴────────────────────────────────╯

▌ EXPORTED API FUNCTIONS

• ClsInit — host hands over the source path + a parent-window handle (used by Inno Setup installers)
• ClsFunc1 (ClsSourcePath) — host sets/updates the folder to read sibling discs (2, 3 …) from
• ClsFunc2 (ClsDiskRequest) — host installs a callback diskspan can call to ask the user to "insert the next disc"
• DiskSpanInfo — extra (not called by FreeArc): returns build + format-version + copyright so any tool can identify the DLL

▌ QUICK START EXAMPLES
Run from the folder holding Arc.exe + cls-diskspan.dll. -w sets a temp folder; diskspan:<first>:<other> = first-disc size : every other disc's size. Put a NUMBER in the archive name.
Legend: program · command · method / diskspan · archive · input mask

Show the build + copyright

Arc.exe --version

Pack + split — one solid block, no compression (fast smoke test)

Arc.exe a -ep1 -r -ed -s; -w.\temp -m0+diskspan:2mb:2mb out\game-1.bin "_data\*"

Pack + split — REAL compression (put diskspan LAST in the chain)

Arc.exe a -ep1 -r -ed -s; -w.\temp -msrep+lzma+diskspan:4467mb:4474mb out\game-1.bin "MyFiles\*"

Pack + split — NON-solid (-s-, every file its own block — the old silent-corruption case, now correct)

Arc.exe a -ep1 -r -ed -s- -w.\temp -m0+diskspan:2mb:2mb out\game-1.bin "_data\*"

Pack + split — SMALL solid blocks (-s<size>, e.g. 8 MB blocks)

Arc.exe a -ep1 -r -ed -s8m -w.\temp -m0+diskspan:2mb:2mb out\game-1.bin "_data\*"

Pack + split — file-MASK groups (end each group's method with diskspan so every group spans)

Arc.exe a -ep1 -r -ed -s; -w.\temp -mtor+diskspan:2mb:2mb/$grpbin=lzma+diskspan:2mb:2mb out\game-1.bin "_data\*"

Test the disc set (reads every disc + checks each disc's CRC)

Arc.exe t out\game-1.bin

Extract

Arc.exe x -dpunpacked\ out\game-1.bin

Lay discs into burnable folders, then build one ISO per folder

Arc.exe --sort    OutFolder DISC_ setup.exe setup.ico out\game-1.bin
Arc.exe --makeiso OutFolder\DISC_

Pick a naming style just by how you name the first disc

Arc.exe a ... +diskspan:300mb:400mb data.bin.001 "MyFiles\*"   (-> .002, .003)
Arc.exe a ... +diskspan:300mb:400mb setup-1a.bin "MyFiles\*"   (-> 1b, 1c, ...)

Tuning / diagnostics (env vars, optional)

setlocal & set CLS_BUFKB=4096   & Arc.exe ...   (4 MB I/O buffer instead of 1 MB)
setlocal & set CLS_DEBUG_LOG=1  & Arc.exe ...   (write cls-diskspan.log next to the DLL)

Ready-to-run, double-clickable versions of all of the above are in bin\_examples\_demo\ — start with 00_version.bat.

▌ TESTS & BENCHMARKS
Both live in bin\_test\ and drive the same Arc.exe + cls-diskspan.dll in bin\.

run_tests.ps1 — round-trip regression suite. Packs 8 scenarios (solid, single-volume, store, groups, non-solid, small-block, groups+blocks, many-volume), extracts each, and SHA-256 compares against the originals. Plus 3 robustness checks: a missing middle disc must fail cleanly, a byte-flipped disc must be caught by the CRC, and a subfolder archive must still round-trip. Prints a PASS/FAIL matrix.

powershell -ExecutionPolicy Bypass -File run_tests.ps1 -DebugLog

bench.ps1 — throughput benchmark. Uses a STORE method (-m0) on purpose, so timing reflects the spanning engine + disk I/O, not a compressor. Packs/extracts at three I/O buffer sizes (CLS_BUFKB = 64 / 1024 / 4096 KB), verifies SHA-256 each time, and prints encode/decode MB/s per buffer size.

powershell -ExecutionPolicy Bypass -File bench.ps1 -SizeMB 120

▌ DISTRIBUTION FILES

• cls-diskspan.dll — REQUIRED. The plugin. Drop it next to Arc.exe.
• Arc.exe — REQUIRED. FreeArc's command-line driver (ships in bin\).
• oscdimg.exe — Optional. Only for --makeiso; must sit next to Arc.exe.
• arc.ini — Optional. FreeArc method config (example in bin\).
• arc.groups — Optional. File-grouping masks (example in bin\).

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ━━━━━━━━━━━━

▌ A NOTE FROM THE MODDER
I'm not quite a repack guy, so do some rough tests on this with your own data and feel free to report any bug / issue.


.