FORUMS: list search recent posts

How to validate and dedupe tons of video (R3D) ?

COW Forums : NAS - Network Attached Storage

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
Will Luker
How to validate and dedupe tons of video (R3D) ?
on Jan 5, 2017 at 8:09:51 pm

We are in the process of consolidating all our data from multiple servers down to one. We have about 250 TB of data, and we’ve moved about 200 TB of stuff already, and I’m seeing we have tons of duplicate files and folders.

Our data is a mixture of RAW video (.R3D), pictures and general business data. Some of our data has alternate data streams. I need to make sure all our data transferred correctly and all the metadata and ADS is intact.

What’s the best way to easily verify all the files are the same and then deduplicate? I’m looking for those who’ve been there, done that for some advice on tools and methods to make sure everything is good.

Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2019 All Rights Reserved