FORUMS: list search recent posts

ShotPut Pro and Checksums

COW Forums : Archiving and Back-Up

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
Catherine Robbins
ShotPut Pro and Checksums
on Jun 4, 2012 at 10:01:20 pm

I am starting a new position as a DIT (Digital Imaging Technician). I will be in charge of wrangling camera files and making sure they get to the editors. I will be using ShotPut Pro on set to verify the camera files as I transfer them from CF cards to an OWC Mercury Elite-Al Pro hard drive. The production is mostly using a Canon EOS C 300. The show is fairly effects heavy/ will go through a lot of post/color correction etc. I am looking for information on "File Verification" and checksums. ShotPut Pro has several different options for file verification: File Size Comparison, 32-bit CRC Checksum, MD5 Checksum, and byte verification. I am wondering what the pros and cons are to each of these options and which might best fit the parameters of the project I have mentioned above. Thanks.


Return to posts index

Malcom Sanders
Re: ShotPut Pro and Checksums
on Jun 5, 2012 at 9:05:28 pm

I work with dailies, preparing the footage for editors (I'm the first person that works with the footage you offloaded :). Experience has shown us that file size verification is good enough about 98% of the time. It's fast - ShotPut just checks the file sizes of the two files and if they match, gives it a green light.

However, there's always that 2% where some byte got swapped somewhere along the transfer (or something similar). The file sizes are the same, but the data is not (I've seen this happen at least 3 times). That's where the MD5 checksum is your friend. ShotPut will read the full file from CF card and generate a checksum for each file (string of garbage-looking characters). That checksum is unique to that file. After the file gets offloaded, ShotPut runs a checksum on the newly created file. If the checksums match, then the offloaded file is a 100% perfect copy.

The downside of using MD5 Checksums is it takes a while to re-read the files from hard disk after offloading. Depending on your connection (eSATA? FW800?) it could take a while to generate the checksums. But in the end, MD5 checksums are well worth the extra time. (All it takes is one corrupt file to get blackballed as a DIT.)

Also, you should have a 2nd hard drive to offload to. A drive that you will keep near your rig/cart that will not be traveling back/forth to editorial. In this world, it's C.Y.A.


Return to posts index

Catherine Robbins
Re: ShotPut Pro and Checksums
on Jun 22, 2012 at 3:40:36 pm

Thanks so much this is really helpful. I appreciate it!


Return to posts index


Angelo Lorenzo
Re: ShotPut Pro and Checksums
on Jun 19, 2012 at 5:23:18 am

I don't trust file size verification for very large files. While MD5 and CRC32 hashes take more time, you're also saving time in general because Shotput Pro writes to multiple drives at once.

In terms of MD5 vs CRC32, MD5 is more of an industry standard. CRC32 is quicker but it also suffers from more collisions, aka you can change a file a certain way and still produce the same checksum as the original; with MD5, this issue is greatly reduced.

Byte level verification is overkill, and should be avoided unless you're troubleshooting an issue like bad camera media.

tl;dr version: Use MD5 and save the log files to the drive as well.

Angelo Lorenzo
Fallen Empire - Digital Production Services


Return to posts index

Catherine Robbins
Re: ShotPut Pro and Checksums
on Jun 22, 2012 at 3:42:47 pm

I've been using the MD5 and saving the log files for the editor and it's been working like a dream so far. I did some more of my own research and think what you have to say about time and veracity of files is quite true. Thanks for your advice I appreciate it!


Return to posts index

Rasch Dè rascher
Re: ShotPut Pro and Checksums
on Nov 10, 2012 at 5:06:46 pm

I've got some problems using shotput pro for the c300 files.
I've tried both, filesize and md5 and it always give me an error log.
In the end I've tried whit a simple grag and drop and whit the canon utility, check size and open all clips by hand.everithing was fine. I've tried whit different footages, from different cards, on different hard drives destination and has been the same, offload error.
Maybe is just a stupid bug on shotput pro, i'm lookin' over the net, but if someone here knows something...please post!

This town needs an enema! _The Joker

Mac Pro 3,1 2008 GeForce GTX 570 4 Gb RAM


Return to posts index


Joe Trimmer
Re: ShotPut Pro and Checksums
on Jul 14, 2014 at 1:53:49 pm

Hey Rasch,

I've been experiencing the same offload log errors saying that I don't have permission, but it's only with one of the two hard drives. I'm using Lacie Rugged USB3/Thunderbolt 2TB and it just started doing it the other day. I've tried formatting the drive twice using Disk Utility. No luck... I'm using the new Mac Pro and my connection is Thunderbolt. I've purged the log history to see if that would help. Nothing. It seems the copying process is working, but then it seems to error during the verifying...

Have you had any success problem solving this?

Thanks.

Joe T.


Return to posts index

Alex Dzialo
Re: ShotPut Pro and Checksums
on Jul 5, 2017 at 7:18:01 am

Hi all,

I'm media managing for a shoot using C300 MkII's, recording to 64 GB Lexar 3500x CFast 2.0 cards. I'm using Shotput Pro 5 for offloading, and I've had errors spring up with the following message:

"The backup completed but __ additional files were detected at the source that were not there at the start of the backup This means any new source files have not been offloaded. This may be caused by a virtual file system not yet having loaded all of its contents before the offload began."

Any ideas what's causing SPP 5 to miss these files in its initial sweep? I've never gotten this error before. When I try a second transfer it goes through without a problem, but obviously I can't be making multiple attempts. I suspect it might be the outdated version of SPP, or the Lexar cards not playing nice with the MkII's. Any help would be appreciated, thanks!

-Alex


Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2017 CreativeCOW.net All Rights Reserved
[TOP]