Silent Sifter 2.5 Preview – MD5 Checksums and Duplicate Detection

In Silent Sifter 2.5, we’ve added support for MD5 checksums, both as a method for deduplication and as a filename / folder structure token.

MD5 checksum support

If you are not familiar with checksums, they are essentially algorithms for creating unique a ‘fingerprint’ for data. In the case of Silent Sifter, the data is an entire image, photo, or video file, and we create an MD5 checksum identifier for the file.

MD5 checksums look like this:

42b7bfa9a2d727a8e6068157bb56760c

The benefit of using checksums is that they are nearly bulletproof as a method of deduplication.

Be mindful that using MD5 checksums as either a token or a deduplication method will cause sifting to take significantly longer, because in order to create an MD5 fingerprint, the entire file needs to be analyzed byte by byte. Because of this, network attached drives, usb drives, etc will be especially sensitive to the additional sift time because every byte of the file will need to be sent to your Mac to create the md5 fingerprint.

MD5 as a Token

MD5 Checksum Filenames

Several users requested the ability to name their files based on md5 checksums, which is why we created the ability to use md5 checksum as a standard token. For example:

42b7bfa9a2d727a8e6068157bb56760c.png

All of the standard token options apply, including appending other tokens, text strings, or applying token options.

[fusion_builder_container hundred_percent=”yes” overflow=”visible”][fusion_builder_row][fusion_builder_column type=”1_1″ background_position=”left top” background_color=”” border_size=”” border_color=”” border_style=”solid” spacing=”yes” background_image=”” background_repeat=”no-repeat” padding=”” margin_top=”0px” margin_bottom=”0px” class=”” id=”” animation_type=”” animation_speed=”0.3″ animation_direction=”left” hide_on_mobile=”no” center_content=”no” min_height=”none”][fusion_toggle title=”The MD5 Token”]

The MD5 checksum of the file

Examples:

Usage Input Result
MD5 IMG_1234.JPG 55bff5d42dc80e87c23e669d7a83af87

[/fusion_toggle]

MD5 As Deduplication Option

You can use MD5 as a deduplication method, if you choose, by opening the Preferences screen, and selecting ‘Best’ as your deduplication method.

MD5 Photo Deduplication

Once you do this, all future sift operations will use MD5 checksums as deduplication, which is a substantially better deduplication option that the medium quality fingerprinting we have done in the past.

Customers who are upgrading from previous versions of Silent Sifter will need to go change this setting, because the default is ‘Medium’, which refers to the same deduplication method used in Silent Sifter 2.0 and earlier versions. Customers who install Silent Sifter 2.5 as their first version will have MD5 as the default setting, until they change it.

In Summary

MD5 checksums are a fantastic addition to Silent Sifter 2.5, and you will find that duplicate prevention with MD5/Best Quality is fabulous. You can expect to see Silent Sifter 2.5 in the App Store in May.

How do you plan to use MD5 checksums? Let us know in the comments! [/fusion_builder_column][/fusion_builder_row][/fusion_builder_container]