日本語English

Duplicate Files

Load a whole folder and see groups of files whose content is byte-for-byte identical, with the space they waste. Nothing is uploaded and nothing is deleted: save the keep/duplicate verdicts as CSV or JSON and clean up with your own file manager.

On-device・No upload

Your files and input are processed only inside your browser on this device and are never uploaded to our servers.

Drop folders or files to find duplicates

or

Files are compared on this device and never uploadedAccepts: Any format (whole folders OK)Paste (Ctrl+V) also works

Limits: 100,000 files and 100 GB in total (only references are kept; contents are read one file at a time)

Finds images that look alike (resized, re-saved, lightly adjusted) with a dHash. Images are grouped only when both the horizontal and the vertical hash are close. Shown apart from exact matches.

Ignore patterns

One per line; * and ? work. A trailing / matches folders only; a pattern with / matches part of a path. Lines starting with # are comments. After removing a pattern, load the files again.

How to use

  1. Drop the folder to check, or click “Choose folder”. You can also pick several files or add folders in multiple rounds.

  2. For photos, turn on “Also find similar images” to catch resized or re-saved copies.

  3. Review the ignore patterns if needed (.DS_Store, Thumbs.db, node_modules and more are excluded by default).

  4. Press “Find duplicates”. Files are narrowed by size, then by head and tail, then by a full hash, with progress shown and a Cancel button.

  5. Pick the default rule for the file to keep (oldest, newest or shortest path); change a single group with its radio buttons.

  6. Save the verdicts as CSV or JSON: group ID, verdict, path, size, modified date and hash. Do the actual clean-up in your file manager.

What it does

Three stages keep reading to a minimum

Only files of equal size are candidates; then the first and last 64 KB are compared; only the survivors are hashed in full with BLAKE3. Files of 128 KB or less are completely covered by the head and tail, so they need no second read. Files that share a size but differ inside are separated reliably by the full hash.

Tens of thousands of files, constant memory

Files are read 8 MB at a time and hashed incrementally in a Web Worker, so the page stays responsive, with progress and a Cancel button. Only file references and hash values are kept in memory.

Similar images with dHash

Each image is shrunk to a 9×9 grayscale grid and hashed twice — brighter than the right neighbour, and brighter than the neighbour below — as two 64-bit dHashes; images are grouped only when both are close. EXIF orientation is applied. The distance (number of differing bits) threshold runs from 0 to 20 and can be changed after the search.

No chaining

Similar images are grouped by distance to the first image of the group. If A~B and B~C but A and C are far apart, C is not pulled into A’s group, so burst shots or numbered sequences cannot snowball into one group of unrelated photos.

Keep/duplicate verdicts you can export

Choose a default rule — oldest, newest or shortest path — and override single groups by hand. The CSV is UTF-8 with BOM, and paths starting with =, +, - or @ are neutralised so a spreadsheet cannot run them as formulas.

Ignore patterns and filters

.DS_Store, Thumbs.db, desktop.ini, node_modules and .git are skipped by default. Edit the patterns freely (* and ? work), and filter the results by path text or minimum size.

Use cases

  • Find identical files piling up in backups or photo folders and see how much space you could free
  • Tidy a Downloads folder or project archive full of copies of copies
  • Review resized or re-saved JPEGs that look the same as the originals
  • Find files with identical content under different names — duplicates a name search cannot see
  • Produce a keep/remove list as CSV to check with colleagues before cleaning up

Supported formats

Files

Any format (raw content is compared)Dropping or choosing folders (relative paths kept)Choosing and adding several files

Similar-image formats

JPEGPNGWebPGIF (first frame)BMPAVIF (if your browser can open it)

Output

CSV (UTF-8 with BOM)JSON

Capability matrix

FeatureSupportNotes
Finding files with identical contentSupportedSize → first/last 64 KB → full BLAKE3
Loading folders (drop or choose)SupportedRelative paths are kept; browsers do not reveal absolute paths
Tens of thousands of files and large foldersSupportedUp to 100,000 files and 100 GB in total, read 8 MB at a time
Finding visually similar imagesPartialHandles resizing, re-saving and light colour changes; cannot catch cropping, large rotations or composites. Up to 20,000 images
Saving keep/duplicate verdicts (CSV, JSON)SupportedDefault rule plus manual per-group changes
Deleting, moving or trashing filesNot supportedA browser cannot delete files safely, so only the verdicts are exported
Replacing duplicates with hard links or shortcutsNot supportedYour files are never modified
“Similar” matching for video, audio or documentsNot supportedOnly byte-identical content is detected
Similarity for images the browser cannot open (e.g. HEIC)Not supportedExact-match detection still works
Offline useSupportedWorks without a connection once loaded

Limits & notes

  • Up to 100,000 files and 100 GB in total can be loaded; anything beyond is not loaded and the screen says so.
  • Similar-image comparison covers at most 20,000 images (in path order); identical images count once.
  • Empty (0-byte) files are not treated as duplicates.
  • Nearly single-colour images, images with a brightness change in one direction only, and plain backgrounds with a few small shapes cannot be told apart by a dHash; they are left out of the similar-image check and counted on screen.
  • If a file is modified after it was loaded, the result for it is not guaranteed.
  • After removing an ignore pattern, load the files again.

Privacy & safety

Files are never uploaded: your browser reads and compares them directly on your device. File names, paths, contents and hashes are not sent, stored or included in analytics. The CSV and JSON you save are also created on your device.

FAQ

Does it find duplicates with different file names?
Yes. It compares file content (a BLAKE3 hash), not names or dates. Same name with different content stays separate; different names with identical content end up in one group.
Can this tool delete the duplicates?
No. A browser has no safe way to delete your files. Save the keep/duplicate verdicts as CSV or JSON and review them in Explorer or Finder.
Could files of the same size but slightly different content be marked as duplicates?
No. Same-size files are compared by head and tail, and if those match the whole file is hashed with BLAKE3 to confirm. A file that differs by a single byte in the middle is separated by the full hash.
How similar must images be?
Images whose dHash differs by no more than the chosen number of bits (the distance, default 10) are grouped. Lower is stricter, higher is looser, and you can change it after the search.
How many files or GB can it check?
Up to 100,000 files and 100 GB in total. Files are read 8 MB at a time so memory stays flat; time depends on your storage speed.
Does a different EXIF orientation make images different?
Orientation is applied before comparing. An image that was physically rotated and re-saved is still found as similar if it looks the same.

Related tools

Duplicate Similar Images

Load a photo folder and identical images and near-identical photos are grouped, with one image per group recommended to keep and the reason shown. Nothing is uploaded, and your original files are never deleted or changed.

File Hash — Checksum Calculator & Verifier

Drop a file to get its SHA-256 (and more); paste the published hash to see match or mismatch instantly. Even multi-gigabyte ISOs are read in chunks on your own device — nothing is uploaded.

Batch Rename

Combine rules to rename files in bulk. Check old → new names live, catch duplicates and invalid names, then save renamed copies as a ZIP or into a folder. Files are never uploaded.

Disk Usage Analyzer

Load a folder and see where the space goes as a treemap. Click a rectangle to drill down, then export tables by extension, kind, largest files, old files and duplicate candidates to Excel or CSV. Neither file names nor contents leave your device.

File Manifest

Load a folder and export a list of relative paths, sizes, modified times and types to Excel or CSV. Neither file names nor contents are uploaded. Use it for delivery lists, inventories, audits and backup records.

File Split & Join

Cut a file to a size you choose (around 18 MB suits a 25 MB mail limit), save it as name.ext.001, .002 … and put it back together on the same page. Multi-GB files are read in small pieces on your device, so memory stays flat. Missing or mixed-in parts are reported before joining, and a SHA-256 record confirms the result.

Last updated: