SharpSFV - A Spiritual Sucessor to QuickSFV

[L33THAK0R]

I'm not sure if this is *really* appropriate for this forum, but I did design this around features I'd like to have as a compression hobbyist, so I thought I'd share it here, on the off-chance someone else finds it useful/interesting.

A while ago I read a forum post about xxhash3 and it's improvements over md5 in terms of speed and reliability. I meant to use it as motivation to create a GUI with support for the algorithm, as I've had my own gripes with QuickSFV, but have yet to find a suitable alternative, save for QuickDash, which was developed by a good friend of mine, but lacked a GUI, and given my complete lack of experience with Rust, I didn't feel confident trying to.

Anywho, over the last few days I've actually put in a bit of work and gotten a decent prototype working, that I'll be implementing in my own workloads/projects and replacing QuickSFV. It's definitely not a perfect product, I've no experience with C++, and it's challenges, and opted instead for C# which was much easier to get my head around, but relies on the .NET 8.0 framework. In the future I would like to eventually port this to C++ to ensure longer term compatibility with Windows 10/11, but I've currently got no experience with C++ so I'll need to do a bit of learning in the interim. The other downside is the size of the compiled binary, I wasn't able to get trimming working, the final size clocks in at 287 KiB, to QuickSFV's 114 KiB. Again I'm sure if I port this to C++ or Rust even the size will be smaller, but that's a project for another day.

SharpSFV has feature parity with QuickSFV, in addition it's key features include primarily the addition of the ultra-fast xxHash-3 algorithm and native recursive folder scanning. Unlike QuickSFV, it offers robust handling of Unicode characters and long file paths to prevent errors verifying files on modern filesystems, and introduces productivity enhancements such as real-time list filtering, granular dual-progress tracking for individual files, clipboard integration for batch processing, and a simpler portable configuration file.

Anyone interested in checking out the project can find it at the following repo:


https://github.com/Wishwanderer/SharpSFV

[luxuse]

i cant find any bat cammand in the readme any auto check or other type ?

[KaktoR]

You mean command line?

Doesn't look like this tool has one. But would be good imo.

Somehow I dislike tools that don't have any command line arguments.

[L33THAK0R]

Originally Posted by luxuse

i cant find any bat cammand in the readme any auto check or other type ?

I'll update the readme, apologies.

the usage in the command line is simply

sharpsfv.exe <CHECKSUM_FILE>

supported extensions include

*.sfv
*.md5
*.xxh3
*.sha1
*.sha256

I'm unsure what you're referring to by "auto check" / "other type". Again apologies for the lack of information.

[L33THAK0R]

Originally Posted by KaktoR

You mean command line?

Doesn't look like this tool has one. But would be good imo.

Somehow I dislike tools that don't have any command line arguments.

I made it more as an extension of QuickSFV, with support for xxhash3 and some QoL features I've always wanted, since I use QuickSFV for a lot of my testing, if there are any arguments you think would be useful I'd be happy to add them in. I've always used a Shell Extension tool for generating checksums rather than using the command line, so I didn't think to add any initially.

[KaktoR]

Does not work here.

SharpSFV.exe "test.sfv"

Even absolute paths doesn't work.

[L33THAK0R]

Originally Posted by audiofeel

Hello.

Could you please let me know if it is possible to move the core logic of SharpSFV into a DLL (class library)?

I would like to use the SFV/hash calculation and verification functionality from another application, without running the exe or GUI.

For example:

move the core logic into a separate library project...callback
keep the current application as a wrapper over it
I understand if this is not planned — I just wanted to ask.

Thank you for the project.

I think this would be an entirely different project, since SharpSFV relies heavily on the System.IO.Hashing package for checksum validation/generation. Unfortunately I've got no experience with creating DLLs, I'll have a look into it over the weekend.

[L33THAK0R]

Originally Posted by KaktoR

Does not work here.

SharpSFV.exe "test.sfv"

Even absolute paths doesn't work.

I've updated the latest build and binary to add some basic debugging if it's failing to launch, I tested this on a couple systems and was satisfied with the validation results, do md5/xxh3 hashes work? Additionally if building from source (I updated this readme accordingly) the command

dotnet publish -c Release -r win-x64 --no-self-contained -p:PublishSingleFile=true

will ensure the external hashing library will be included. Apologies you've had this subpar experience so far.

[KaktoR]

Other hash algorythms working fine, just sfv/crc32 (I believe) doesn't work here.

Edit: I found out why it don't work. If you create SFV with SharpSFV, I was asuming the hashes are stored as crc32 hashsum, but it looks like md5 or something else.
I am creating hash files with other applications, like hashcheck shell extension.

So basically If I want to do hashchecking with SharpSFV + command line + SFV extension, I have to change options "Default Hash algorithm" in SharpSFV to "CRC-32 (SFV)".

Edit 2: All other applications I worked with stored CRC32 hash if you export them as SFV file format as default. If you use other programs to create sfv hash files and want to check them with sharpsfv, then this attempt will fail.

Example:

[L33THAK0R]

Originally Posted by KaktoR

Other hash algorythms working fine, just sfv/crc32 (I believe) doesn't work here.

Edit: I found out why it don't work. If you create SFV with SharpSFV, I was asuming the hashes are stored as crc32 hashsum, but it looks like md5 or something else.
I am creating hash files with other applications, like hashcheck shell extension.

So basically If I want to do hashchecking with SharpSFV + command line + SFV extension, I have to change options "Default Hash algorithm" in SharpSFV to "CRC-32 (SFV)".

Edit 2: All other applications I worked with stored CRC32 hash if you export them as SFV file format as default. If you use other programs to create sfv hash files and want to check them with sharpsfv, then this attempt will fail.

Example:

The latest release should have resolved these issues. The issue was with Big/Little Endian state of the checksum, as well as with the formatting of the checksums, I hadn't tested legacy sfvs and wrongly assumed compatibility. Release notes are in the GitHub repo, for those who aren't as interested, said release notes are as follows:

• Fixed issues with legacy *.sfv support (Big/Little Endian conflicts)
• Added Multi-threading support (Automatically determined by whether the target drive is a HDD/SSD)
• For files larger than 1 GiB, these files will be pinned at the top of the list, with progress tracked.
• Re-worked numerous aspects of the code-base for greater efficiency and stability.
• Large data sets via drag & drop have been stabilised and confirmed (test sample contained 500.000 files of varying sizes)
• Added sound cues to play when verification has finished.
• Some additional edge cases have been resolved.
• Upgraded from .NET 8 to .NET 10 for some minor performance improvements, this has resulted in a slightly larger binary.
• Re-wrote source code comments for clarity.
• Minor memory improvements.

[KaktoR]

Thanks. The issues are fixed with the update.

[luxuse]

Great update and thanks for updating the README! Everything works really well, though I hope in the future there might be a version without the .NET dependency. Really appreciate your work!

[L33THAK0R]

Patch Notes:

- Significantly overhauled logic for HDD/SSD identification, added a debug info window to catch future edge cases.
- Abstracted monolithic class into partial classes, code is still a bit spaghetti but easier to manage.
- When the time elapsed tab is enabled, total time is also tracked and appears upon completion, this was mainly to benchmark the actual difference between HDD and SSD mode. On a 7 GiB sample containing 5 files of varying size, using HDD mode saved ~ 5 seconds (~27 using SSD/Parallel, ~22 using HDD/Sequential mode). These improvements were mirrored over other tests, however I didn't record the results at the time. Just included this note to demonstrate that parallel operations aren't always optimal.

Additional Notes:

The SSD/HDD detection feature has turned out to be a real bugger of an issue, the issue was not with standard drives but encrypted drives, where previous volume IOCTLs checks weren't returning any valid values, at least with VeraCrypt encrypted drives. New checks return to checking the "friendly name" and analysing it for key words, i.e. SSD, NVMe, Flash, M.2. Under the help menu in the toolbar, is a disk info debug box, with a clipboard friendly result reading. If anyone has a drive which is wrongly identified and can submit the debug info, I should be able to improve these detection layers.

[L33THAK0R]

Originally Posted by luxuse

Great update and thanks for updating the README! Everything works really well, though I hope in the future there might be a version without the .NET dependency. Really appreciate your work!

If/when I learn C/Rust down the road, I'll give it a crack, unfortunately, at least currently, I'm not equipped to make a more portable application.

[L33THAK0R]

Patch Notes:
- Replaced LockWindowUpdate with WM_SETREDRAW to eliminate desktop flickering and visual artifacts during bulk file updates.
- Replaced string-based status tracking with a lightweight ItemStatus byte enum to significantly reduce memory usage and accelerate sorting operations.
- Switched worker threads to synchronous I/O and hashing to remove Async State Machine overhead, maximizing throughput on high-speed NVMe drives.
- Implemented modulo-based throttling in the UI update loop to drastically reduce CPU cycles spent on system time checks during high-speed processing.
- Split the monolithic Form1 and Helpers files into specialized partial classes and namespaces to improve organization.
- Centralized P/Invoke definitions into a dedicated Win32Storage class to isolate native code and resolve compiler ambiguity errors.
- Optimized memory usage by refactoring the data model to store hashes as raw byte arrays instead of strings.
- Reduced garbage collection pressure by implementing ArrayPool for buffer reuse within the hashing engine.
- Corrected the project build configuration to disable incompatible trimming flags while maintaining a lightweight single-file executable.
- Implemented a new "View" menu to optionally toggle Hash/Expected Hash columns and lock column ordering.
- Added a "Show Full File Paths" option that dynamically adjusts the File Name column's maximum width limit to the longest text length plus a 25px buffer.
- Integrated a "Show Duplicates" filter that groups files by hash and enables conditional "Copy Duplicates" and "Delete Duplicates" batch script generation.
- Added a "Delete BAD Files" script generator that automatically enables only when verification errors are detected.
- Updated all save dialogs to default to the root directory of the scanned files rather than the application path.
- Removed "Script Saved" message boxes to allow for silent, uninterrupted script generation.
- Fixed column resizing logic to enforce "Hard Stop" limits (0.75x–1.5x) using event cancellation, eliminating visual snap-back artifacts.
- Enabled horizontal scrolling on the main list view to ensure long file paths remain viewable.
- Replaced the hashing engine with a Producer/Consumer pattern using System.Threading.Channels to decouple high-speed file scanning from processing logic.
- Implemented System.Buffers.ArrayPool<byte> to recycle memory buffers, eliminating massive Garbage Collection overhead during large batch operations.
- Optimized the file scanner to batch UI updates in groups of 2,000, allowing raw disk read speeds to saturate without freezing the interface.
- Replaced the performance-heavy "Pin-to-Top" mechanism with a scalable SplitContainer and secondary ListView for tracking active large files.
- Added GDI+ resource caching to the Virtual Mode rendering pipeline to prevent system handle exhaustion and crashes during extensive scrolling.
- Fixed critical thread-safety race conditions by ensuring list counts and progress limits are only resolved on the UI thread.
- Enhanced the user interface with multi-colored status statistics, synchronized column alignment, and a debounced search filter for responsive interaction.
- Replaced the object-based data model with a Structure of Arrays (SoA) architecture to eliminate per-file object overhead and maximize CPU cache locality.
- Switched from FileStream to System.IO.RandomAccess for zero-allocation file reading, creating a direct "disk-to-memory" pipeline.
- Hoisted hashing buffers and algorithm instances to thread-local storage to prevent Garbage Collection churn during batch processing.
- Implemented a custom String Pool utilizing .NET 10's AlternateLookup to deduplicate repetitive path strings and significantly reduce memory usage.
- Enabled Dynamic Profile-Guided Optimization (PGO) in the project settings to let the JIT compiler optimize hot loops based on runtime execution data.
- Refactored the producer-consumer pipeline to pass lightweight structs (FileJob) between threads instead of heap-allocated class objects.
- Updated the Virtual ListView to render directly from primitive arrays and cache hex string conversions, eliminating UI scroll lag.

Additional Notes:

Processing is 1.61x faster than v2.30, memory usage has been reduced by 84%. SharpSFV is now 16.53x faster than QuickSFV (when verifying using pre-generated MD5 hashes), with QuickSFV only being able to process around 50% of the provided benchmark MD5 file, before crashing.