Short answer
Duplicate detection identifies identical or near-identical files in a DAM using file hashes or visual similarity. Hash matching catches exact copies, and visual similarity catches resized, cropped or re-exported versions. Duplicate detection pays off most during migration and bulk ingest, when legacy libraries hold the same image under several names and folders.
How duplicate detection works in a DAM
Duplicate detection compares each new file with the existing library on ingest, and flags matches before they become separate assets. DAMs use two methods, often together.
| Method | Detects | Misses or confuses |
|---|---|---|
| Cryptographic hash, such as SHA-256 | Byte-identical copies, regardless of file name | Any re-save, resize or metadata change |
| Visual similarity or perceptual hash | Resized, recompressed and lightly cropped copies | Heavy edits, and it can confuse colour variants of one product |
Fact: SHA-256 is one of the hash functions defined in NIST FIPS 180-4, the Secure Hash Standard. Any change to the file produces a different hash, which makes the method exact and narrow.
When the DAM finds a match, it applies a configured action. Typical actions are to block the upload, warn the user, link the file as a new version, or queue the pair for a librarian.
Example of duplicate detection
Worked example: a company migrates 120,000 files from shared drives. Assume a hash pass finds 8,000 byte-identical copies, which the team removes after keeping the best metadata from each copy.
A similarity pass then flags further near-duplicates, such as web exports of print masters. The librarian keeps the master, links the exports as renditions, and rejects false matches between colour variants of the same shoe.
Duplicate detection in a DAM selection
Duplicate detection belongs at checkpoint 5, Test, with a scripted scenario on the buyer's own files. The content audit guide estimates the duplicate problem before the demo.
Test these four behaviours in the demo:
- Upload a file already in the library under a new name and show the exact-match warning.
- Upload a resized, recompressed copy and show the similarity match.
- Upload two colour variants of one product and confirm the DAM does not merge them.
- Show the review queue and how metadata from both copies is merged.
Run duplicate detection before migration, using the migration plan. The demo script generator includes a duplicate scenario.
Common mistakes with duplicate detection
Duplicate detection goes wrong through four common mistakes.
- Deleting duplicates automatically, which loses metadata and usage history.
- Setting the similarity threshold too loose, so colour variants and sequential shots merge.
- Running detection only after migration, when duplicates already carry separate links.
- Keeping the newest copy by default, when an older copy holds the richer metadata.
Related terms: Auto-tagging.
Frequently asked
How does a DAM detect duplicate images?
A DAM detects duplicate images with file hashes for exact copies and visual similarity for near copies. Hashes match identical bytes. Similarity models compare image content, so they catch resized or recompressed versions. Many DAMs run both on ingest.
Should a DAM delete duplicates automatically?
A DAM should not delete duplicates automatically. Send matches to a review queue, merge the best metadata, keep the usage history and then archive or remove the extra copy. Automatic blocking of exact copies at upload is safe; automatic deletion of existing assets carries risk.