ISApexEx - Unified API for 7z / FreeArc / RAR Extraction and Xdelta3 Patching

[BLACKFIRE69]

ISApexEx Update
Unified Archive Extraction & Patching API for Inno Setup
by BLACKFIRE69

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ━━━━━━━━━━━━

▌ v0.1 Beta 3 (Build 6A406400)
Compared to: v0.1 Beta 2 (Build 6A3DC100)

✦ Added

• Archive testing (no extraction) — ISApexExTestDisk(n) and ISApexExTestAll test and CRC-check every member while writing nothing to disk. Real archives (7z / FreeArc / RAR) only; patch and zip-create jobs are skipped. Returns False on the first corrupt archive (fires APEX_EVENT_ERROR).
• Existing-file overwrite policy — ISApexExSetOverwritePolicy(n, policy): APEX_OVERWRITE (default) / APEX_SKIP / APEX_RENAME. Skip and overwrite work on all engines; rename ('name (N).ext') is honoured by 7z and RAR. FreeArc has no per-file rename, so rename falls back to overwrite there.
• Selective extraction — ISApexExSetExtractFilter(n, include, exclude): per-disk include/exclude masks using the scanner syntax — pipe-separated globs ('*.dll|data\*'), '@listfile', or regex wrapped as $"pattern" (e.g. '$"^setup.*\.exe$"' — the $ and quotes are required and protect any '|' inside). Honoured per-member by 7z and RAR; FreeArc maps globs to native filespecs / -x (the $"..." regex and @list forms are ignored for FreeArc).
• Interactive password prompt — ISApexExSetPasswordCallback(cb) for a custom prompt, plus a built-in fallback password box. An encrypted archive with no / empty / wrong stored password now prompts the user instead of failing. Covers all three real engines — 7z, RAR, and FreeArc. (Triggers only for genuinely encrypted archives.)
• ZIP creation from a folder — ISApexExAddZip(srcDir, outZip, include, exclude, method, level, password, deleteSource) builds a .zip with a selectable compression method (APEX_ZIP_STORE / DEFLATE / DEFLATE64 / BZIP2 / LZMA / PPMD) and level (0=store .. 9=ultra, default 5), plus optional AES-256 (non-empty password). deleteSource=True removes the source folder (incl. subdirs) after a successful zip — never if the job fails or is cancelled. Registered like the patch helpers and runs in the unified extraction loop with continuous progress; the folder is walked at extraction time (deferred). New constants APEX_ARC_ZIPCREATE = 5 and APEX_ZIP_* methods.
• New example scripts — one per feature, for individual testing: ISApexEx_Example_Test.iss, ISApexEx_Example_Overwrite.iss, ISApexEx_Example_Filter.iss, ISApexEx_Example_Password.iss, ISApexEx_Example_Zip.iss, ISApexEx_Example_Verify.iss.
• Batch signature verification — ISApexExVerifyFolder(folder, recursive, callback) walks a folder, finds every <file>.sig and verifies the file beside it in one call (instead of many ISApexExVerifyFile calls). Returns the number of files that failed (0 = all authentic); a missing signed file is caught as VFY_NO_FILE. The optional callback reports per-file status (path, index/total, VFY_* code, running fail count) and can cancel (VFY_CANCELLED). New VFY_CANCELLED = -10.
• Job counter — ISApexExGetDiskCount returns how many jobs are registered (archives, patches and zip-create jobs). The DLL is now the single source of truth, so scripts no longer need to track their own disk counter — just loop 1..ISApexExGetDiskCount.
• Encrypted password keystore — ISApexExOpenKeyStore(file) / ISApexExCloseKeyStore. Keep archive (and .ekey) passwords out of the installer script entirely — store them in an encrypted (.keydb) keystore file and reference each by name. Once a keystore is open, a password argument of the form '@key:<name>' passed to ISApexExAddDisks / AddDisksEx / AddZip / SetVerifyKey is replaced with the matching stored password; the clear-text never appears in the .iss or the compiled Setup.exe. <name> is a FIXED reserved name, validated per use-site: 'disk' (archive password shared by all archives), 'disk2'/'disk3'/... (per-archive overrides, only when one differs), and 'ekey' (the ISApexExSetVerifyKey password). So 10 archives sharing one password need a single 'disk' entry. A name invalid for the site, or absent, resolves to an empty password (and is logged), so a typo or misuse cannot fall back to literal text. Companion console tool ApexKeyTool builds keystores (guided 'gen', plus add / remove / list); it is deliberately write-only for passwords — 'list' shows entry NAMES only and there is no command to print stored passwords (the keystore's key is embedded in the shipped tool, so a dump command would expose every keystore).
• Full FreeArc error-message coverage — a failed FreeArc job now reports a specific, localizable reason (unknown method, output block too small, read/write error, not enough memory, unsupported feature, corrupted data, checksum mismatch, damaged headers, invalid password, ...) instead of a single generic "decompression error". New ERROR_ARC_* keys in English.ini / Russian.ini.
• ZIP-create progress label — creating a zip now shows a clear status ("<name>.zip - Creating ZIP archive...", key STATUS_ZIPPING) and the archive-test (ISApexExTestDisk / TestAll) now reports per-file progress + count like a real extraction.

✦ Changed

• The selective-extraction mask engine is now shared by both selective extraction and zip creation — one consistent include/exclude syntax.
• The 7z engine gained archive-creation support, used by ISApexExAddZip (requires the full 7z library that ISApexEx ships).
• FreeArc extraction now honours the per-disk overwrite policy when it would otherwise prompt to overwrite (overwrite-all / skip-all).
• The 7z engine now opens real .zip archives (the format is detected from the file itself). Previously a .zip could fail to open and be reported as "not a valid 7z archive"; this also affected the Zip-Slip security test.
• The log now records WHY a run ended early instead of a bare "finished with errors". On stop, ISApexEx writes the specific reason to ISApexEx.log: user cancellation from the progress callback, a declined password or disk-span volume prompt, the engine's own error text (e.g. a FreeArc "checksum mismatch"), or a host-initiated stop while a job was still running. (Needs a log file set with ISApexExSetLogFile.)
• Version v0.1 Beta 2 → v0.1 Beta 3; build 6A3DC100 → 6A406400.

✦ Fixed

• Interactive password on encrypted .7z archives whose file list is itself encrypted — a wrong or missing password now shows the prompt and retries (just like RAR and FreeArc) instead of failing with "not a valid 7z archive".
• RAR wrong-password retry now also covers archives whose encrypted header is reported as generic data damage — but only when a password is actually involved, so genuinely corrupt archives are still reported as damaged.
• A password-prompt callback that raises an error is now treated as a clean cancel instead of being misreported as a corrupt archive.
• The crash-recovery (resume) checkpoint now saves reliably on all supported Windows versions (an internal integrity step could fail on some systems).
• Optional component signature verification now loads the key and verifies files reliably on all supported Windows versions (same underlying fix). Files signed with the SignTool Suite verify unchanged.

✦ Security

• In-memory password hygiene — archive passwords (per-disk, the session "last good password" cache, the interactive-prompt hand-off, and the FreeArc engine's working copies) are now overwritten with zeros as soon as they are no longer needed — not merely released — so a memory/crash dump cannot recover them from freed heap. Passwords are also wiped on ISApexExStop and on library unload.
• Encrypted password keystore (AES-256, OFF by default) — see Added above. Lets the installer reference passwords by name instead of embedding clear text, and the stored passwords are wiped from memory on close / unload.
• Optional component signature verification (RSA-2048 / SHA-256), OFF by default — ISApexExSetVerifyKey(ekey, password) turns on a check of every native engine component ISApexEx loads: each must carry a valid <name>.sig (made with the SignTool Suite) or the load is refused and the run fails (fail-closed). Plus ISApexExVerifyFile(path) to verify arbitrary shipped files (external compressors, helper tools, ISApexEx.dll's own image) and ISApexExGetVerifyStatus for diagnostics. Uses the OS crypto provider; no extra DLL shipped. One-time cost only; zero when unused. New result codes VFY_OK / VFY_TAMPERED / ...
• Crash-recovery (resume) hardening — the resume file is now written atomically (temp file + flush-to-disk + atomic replace), so an instant crash mid-save can no longer truncate or lose the checkpoint.
• Path-traversal protection on extraction (anti "Zip-Slip") — a malicious archive member that tries to write OUTSIDE the chosen destination — via '..' segments, an absolute path, a drive letter or a UNC path — is now blocked. ISApexEx fails closed: it stops the extraction, reports the blocked path, fires APEX_EVENT_ERROR and returns failure (no partial or dangerous install). Enforced for the 7z and RAR engines.
• Hardened native-DLL loading — the engine DLLs and their dependencies now resolve only from System32 and ISApexEx's own folder, never the installer's working directory — closing a DLL-planting vector on end-user machines.
• Password-handling robustness — the RAR password path now respects the engine's buffer size, removing a potential overflow with very long passwords.
• New language string ERROR_UNSAFE_PATH (English.ini / Russian.ini).

✦ Notes / Known Limitations

• Interactive password covers the three real archive engines (7z, RAR, FreeArc), including .7z archives whose file list is itself encrypted. On a wrong password all three now re-prompt: FreeArc retries until the password is correct or the user cancels; 7z and RAR retry up to 3 times, then fail that archive (re-run to try again). Cancelling the prompt aborts the archive cleanly (no corrupt output). Xdelta/HDiff patch jobs are unencrypted and never prompt.
• Rename overwrite policy works on 7z and RAR. Only FreeArc lacks a rename-on-collision option and falls back to overwrite.
• Selective extraction on FreeArc uses native globs only — quoted-regex and @listfile entries in the mask are ignored for the FreeArc engine (7z and RAR honour the full scanner syntax). Listing-time size totals are computed before filtering, so a heavily filtered job may show a slightly high byte total.
• ZIP creation requires the full 7-Zip library that ISApexEx ships (it includes the zip writer). A stripped extract-only build cannot create archives and ISApexExAddZip fails. RAR creation is not possible; use ZIP for in-installer repacks.
• ZIP methods STORE/DEFLATE/DEFLATE64 open in any unzip tool (incl. Windows Explorer); BZIP2/LZMA/PPMD compress better but need a modern tool (7-Zip, WinRAR). DEFLATE (the default) is the safe choice for end-user output.
• Path-traversal protection is enforced for 7z and RAR. FreeArc (.arc/.bf) relies on its own engine to keep output inside the destination folder.

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ━━━━━━━━━━━━

▌ v0.1 Beta 3 — Inno Setup Snippets

Runnable single-feature snippets — each also ships as its own example .iss (see the Added list above).

Example 1 — Archive Testing (No Extraction)

{ one shot - test every registered archive (writes nothing): }
if ISApexExTestAll then
  MsgBox('All archives are valid.', mbInformation, MB_OK)
else
  MsgBox('An archive is corrupt or the password is wrong.', mbError, MB_OK);

{ or per-disk, to pinpoint WHICH archive is bad: }
for i := 1 to ISApexExGetDiskCount do
  if not ISApexExTestDisk(i) then
    MsgBox(Format('Archive #%d failed the test.', [i]), mbError, MB_OK);

Example 2 — Existing-File Overwrite Policy

{ call AFTER AddDisks, once per disk. Policy is one of:
    APEX_OVERWRITE (0, default) | APEX_SKIP (1) | APEX_RENAME (2) }
ISApexExSetOverwritePolicy(ISApexExGetDiskCount, APEX_RENAME);
{ Overwrite + Skip work on all engines; Rename works on 7z and RAR
  (FreeArc has no per-file rename and falls back to overwrite). }

Example 3 — Selective Extraction (Include / Exclude Masks)

{ AFTER AddDisks, once per disk. '|' separates masks; exclude wins.
  Empty include = everything. }
ISApexExSetExtractFilter(ISApexExGetDiskCount, '*.dll|*.exe|data\*', '*.log|*.tmp');

{ regex form - wrap each pattern as $"..." (single-quoted Pascal string, so the
  inner double quotes are literal); a '|' inside is then protected: }
ISApexExSetExtractFilter(ISApexExGetDiskCount, '$".*\.(exe|dll)$"', '');
{ @listfile and the $"..." regex are honoured by 7z and RAR;
  FreeArc uses native globs only. }

Example 4 — Interactive Password Prompt (Custom Callback)

• Way 1 — Native: do nothing extra; ISApexEx shows its own built-in password box automatically.
• Way 2 — Custom callback (shown below): register your own prompt to replace the built-in box.

{ Custom callback prompt: register your own custom prompt to REPLACE
  the built-in box. Return the password, or '' to cancel (clean abort).
  Attempt is 1-based for retry messages. }
function OnPasswordRequest(const Filename: WideString; Attempt: Integer): WideString;
begin
  if Attempt > 1 then
    Result := InputBox('Wrong password - try again for ' + Filename, '', '')
  else
    Result := InputBox('Password for ' + Filename, '', '');
end;

{ register it BEFORE extraction (omit this call to use the native box): }
ISApexExSetPasswordCallback(@OnPasswordRequest);

{ Either way: triggers only for genuinely encrypted archives with no/empty/wrong
  password. The first entered password is cached and reused for the rest of the
  batch. Cancelling at the prompt writes the reason to ISApexEx.log (when a log
  file is set with ISApexExSetLogFile). }

Example 5 — Build a ZIP From a Folder

{ ISApexExAddZip(SrcDir, OutZip, Include, Exclude, Method, Level, Password, DeleteSource).
  Method : APEX_ZIP_STORE | DEFLATE | DEFLATE64 | BZIP2 | LZMA | PPMD
           (Store/Deflate/Deflate64 open anywhere; BZip2/LZMA/PPMd need a modern tool)
  Level  : 0=store .. 9=ultra (default 5; ignored for STORE)
  Password: '' = plain zip; non-empty = AES-256
  DeleteSource: True removes the source folder after a SUCCESSFUL zip. }
if not ISApexExAddZip(ExpandConstant('{app}\unpacked'),
                      ExpandConstant('{app}\bundle.zip'),
                      '*', '', APEX_ZIP_DEFLATE, 5, '', True) then
  MsgBox('Could not register the zip job.', mbError, MB_OK);
{ Runs in the same extraction loop as the archives, with unified progress. }

Example 6 — Job Counter (No Manual Disk Tracking)

{ No manual disk counter needed - loop 1..ISApexExGetDiskCount: }
for i := 1 to ISApexExGetDiskCount do
begin
  if not ISApexExExtract(i) then Break;
end;

Example 7 — Component Signature Verification
SetVerifyKey turns on fail-closed loading; VerifyFile checks one file, VerifyFolder batch-checks a whole tree.

{ Turn on verification: every engine DLL loaded after this must carry a valid
  '<name>.sig' beside it or its load is refused (fail-closed). }
if not ISApexExSetVerifyKey(ExpandConstant('{tmp}\public.ekey'), VerifyPassword) then
begin
  MsgBox('Verification could not be enabled: ' + ISApexExGetVerifyStatus,
         mbCriticalError, MB_OK);
  Exit;
end;

{ verify one file yourself (returns a VFY_* code; VFY_OK = authentic): }
if ISApexExVerifyFile(ExpandConstant('{tmp}\7z_32.dll')) <> VFY_OK then
  MsgBox('7z_32.dll is not trusted: ' + ISApexExGetVerifyStatus,
         mbCriticalError, MB_OK);

{ batch-verify every signed file under a folder. The callback (main thread)
  reports each file; return non-zero to cancel. The result is the number that
  FAILED (0 = all authentic; a negative value is an operational error). }
function VerifyProgress(Info: TISApexExVerifyProgress): LongWord;
begin
  Log(Format('Verify %d/%d: %s  code=%d',
      [Info.Index, Info.Total, Info.FilePath, Info.ResultCode]));
  Result := 0;
end;

Failed := ISApexExVerifyFolder(ExpandConstant('{tmp}'), True, @VerifyProgress);
if Failed <> 0 then
  MsgBox('Folder verification reported ' + IntToStr(Failed) + ' problem(s).',
         mbCriticalError, MB_OK);

Example 8 — Encrypted Password Keystore
Reserved names — 'disk' (shared password), 'disk2'/'disk3'... (per-archive overrides), 'ekey' (the SetVerifyKey password).

{ Keep the archive password out of the script: store it in an encrypted .keydb
  (built with the ApexKeyTool console utility) and reference it by NAME.
  Reserved names: 'disk' (shared archive password), 'disk2'/'disk3'.. (per-archive
  overrides), 'ekey' (the ISApexExSetVerifyKey password). }
if not ISApexExOpenKeyStore(ExpandConstant('{tmp}\ApexKeys.keydb')) then
begin
  MsgBox('Could not open the password keystore.', mbCriticalError, MB_OK);
  Exit;
end;

{ pass '@key:<name>' instead of a clear-text password - it is substituted from
  the keystore. A wrong/missing name resolves to an empty password (and is logged). }
if not ISApexExAddDisks(ExpandConstant('{src}\data1.7z'),
                        '@key:disk', ExpandConstant('{app}')) then
  MsgBox('Could not open the archive.', mbCriticalError, MB_OK);

{ close when done registering - the stored passwords are wiped from memory. }
ISApexExCloseKeyStore;

✦ Download

• - Check the first post.

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ━━━━━━━━━━━━

Feedback, bug reports, and edge cases are all welcome.