DskDitto: Ultra-fast, parallel duplicate-file detector
The project, authored by jdefrancesco, ships as a single binary that can be installed directly from source with Go 1.22 or later. Its core engine hashes files in parallel, defaulting to SHA‑256 but allowing the faster BLAKE3 algorithm, and presents results in an interactive TUI. Users can also compile an experimental Raylib GUI by enabling the gui build tag, provided they have CGo and Raylib installed. dskDitto’s command‑line flags let operators filter by size, depth, hidden status, or filesystem boundaries, and it respects hard links to avoid double‑counting. De‑duplication actions are gated behind explicit --remove and --link options, with confirmation prompts to prevent accidental data loss.
The tool arrives amid a crowded space of deduplication utilities such as fdupes, rdfind, and dupeGuru, many of which rely on single‑threaded scans or lack interactive interfaces. By leveraging Go’s concurrency model and modern hash functions, dskDitto promises “instant” scans on multi‑terabyte disks—a claim that could appeal to sysadmins managing backup repositories or media libraries. Its TUI, built on the Bubble Tea framework, follows the recent surge in terminal‑first applications that prioritize keyboard‑driven workflows while still offering a graphical fallback for less‑technical users. The inclusion of fuzzy matching and name‑only modes broadens its applicability beyond strict byte‑for‑byte duplicates, aligning with emerging needs for near‑duplicate detection in large media collections.
While the feature set is impressive, the experimental GUI and the reliance on Go’s --no-cache option to bypass OS caching may limit adoption in production environments where stability is paramount. Users must be comfortable with Go toolchains to build the GUI variant, and the extensive flag matrix introduces a learning curve that could deter casual users. Future development should focus on stabilizing the GUI, adding automated testing for edge cases like virtual filesystems, and perhaps integrating with existing backup solutions to provide seamless deduplication pipelines.
Key Takeaways
dskDitto delivers parallel, Go‑native duplicate detection with both a Bubble Tea TUI and an optional Raylib GUI.
It supports SHA‑256 and BLAKE3 hashing, offering speed advantages for large‑scale scans.
Safe removal and symlink conversion require explicit flags and confirmation dialogs, reducing accidental data loss.
Adoption may hinge on the maturity of the GUI build and the ease with which users can navigate its extensive command‑line options.
About the Source
This analysis is based on reporting by Hacker News. Here is a short excerpt for context:
CommentsRead the original at Hacker News