Import Error from SXS card
Just got back from a shoot, when I started to open XDCAM transferred, pointed it to my Sony card reader that had a 16 gig sony SXS card in it, I got this error message on my Quad Core intel.
"The Clip thumbnail could not be saved to the cache as it was an unrecognized format.
Cancel all task OK
I was able to import the clips to my media drive, except for the last clip. It only imported the first 2 seconds of it. The whole clip, about 30 seconds, can play in the XDCAM TRANSFER program. I am off to anther shoot. I dragged the BPAV folder onto my drive. Does anybody know how I can get the last clip to import, and does this mean my 16 gig SXS stick is corrupted? Also I have not yet tried the clips in FCP.
Even though you haven't given me the information about either platform or OS my crystal ball says you're using Mac with Snow Leopard. Known Bug since Snow Leopard came out in September. Sony's crack team of coders should have this fixed sometime between December and March of next year.
Always include all system specs and version numbers since my crystal ball can be cloudy at times.
[ron darby] ". I dragged the BPAV folder onto my drive. "
ONLY use ClipBrowser with CRC checking on to copy clips. EVER!
If you make a corrupt copy by drag and drop and erase the SxS card you're toasted.
Yes I am on SL on a Mac, sorry about that.
I only dragged the BPAV folder on the drive as a back up because on off to a shoot and need to wipe out my card. I use xdcam transfer for importing.
So dose this mean I SOL until they come up with the fix.
I am leaving for about 5 hours thanks for the reply.
[ron darby] "I only dragged the BPAV folder on the drive as a back up because on off to a shoot and need to wipe out my card. I use xdcam transfer for importing. "
Exactly the worst reason to do that. Drag and drop and the risk of corruption on a hard drive without any other backup followed by erasing your card.
ALWAYS use Clip Browser for copying. It's the best to ensure the copy is good. Then backup even if only temporarily to something else.
ClipBrowser to copy, Then backup BPAV, Then XDCAM Transfer to wrap to MOV. Anything else is like driving without a seat belt. Accidents can be fatal to your masters, your business, your client.
Don't confuse copy with import. Import after PROPER copying.
When you say clip browser do you mean the software xdcam transfer or is there a program for transfer would not import the clip and I needed the SXS stick for my next shoot. So my only way of getting the files off the stick was to drag and drop.
Now is there a way to get that last clip imported or do I have to wait for the up grade or us and older OS?
Mac Quad Intel 14 gigs ram
3 TB Caldigit
Ron, you've apparently been using the camera for nearly a year. Clip Browser came on the CD with the camera and has been updated several times. Clip Browser hasn't had any issues with Snow Leopard that I know of. XDCAM Transfer has issues though.
Your right I have been using this camera for a year and I used xdcam transfer the whole time, I totally forget about clip browser, thanks for the reminder and getting me back on track.
I'm having the very same issues with these cards, and one thing I've had to do is use FCP, use the camera as your playback deck, insert the cafd back into the camera, play the card off the camera, use an SDI-out of the camera, SDI-in to your Mac, via a Blackmagic card with SDI-in, and capture the video using the capture settings "Apple Pro Res 422 HQ 8-bit 59.94." In this way, you can recover your video.
Now, if the video file is corrupted, there will be a stop during the playback, and a large blue screen "Halt: Media Error, Halted during Playback"- this means you are screwed. What I then found was that it was a small portion of corrupted video, so you might go to the playback in the card, find the thumbnail of the video clip that has the corruption, then hit "Expand the Clip", then go into the area of where your captuirng stopped and the camera playback stopped, then you go to the next thumbnail and try to play it, a minute or two after, and I was able to salvage many clips doing this. Granted, if you shot multiple takes, of let's say, an interview, you have other takes to grab the sound bites from, and re-insert that into the edit, using the audio from another take, covering you on the corrupted 1 or 2 minutes of video - that would not transfer over or playback.
One of my editor's tried converting directly from the card, and sometimes have had no problems. But in this case, with non-recoverable video, unless you can play it back, it may be gone for good. You can off-load cards and come back to them and put them back on to cards (that have also been wiped) and you can put the video back on the cards and capture them later, but you also need to make sure that the ENTIRE file transfers over. During BPAV File transfer, regardless if it's Sony software or not, you must right click on the copy of the BPAV, and see that those exact numbers match, example: BPAV Folder says 31GB on disk, but when you "right click" on the details, it says, (31,987,654,311 bytes). This (31,987,654,311 bytes) must match the original or the transfer is off, and most of the time it is, but just by a few numbers, so it appears that the data all transferred but in reality, it didn't. Double-check everything, and don't rely on a 1-2-3 step process, as everyone's macs and softwares are different.
Salt Lake City, UT.
I've been dragging and dropping BPAV's since I got the camera 6 months ago, no issues so far.
Not sure I understood the wording of your post; you use Clip Browser to import footage from the SxS to your external HD? Details regarding your workflow are appreciated.
Canon 5d mark 2
[Chandler Kauffman] "I've been dragging and dropping BPAV's since I got the camera 6 months ago, no issues so far. "
You can drive for years without wearing a set belt and have no issue. Avoiding serious injury or death in one accidents makes it worthwhile and why it's a law in the USA.
More to come.
Use ClipBrowser with CRC to copy to hard drive of your choice.
Copy to backup device of your choice. I use ClipBrowser to properly split BPAV to DL-DVD but Blu-ray certainly would be good too. Personally I think a backup should go to something that doesn't have the risk of being accidentally erased/deleted or might crash.
Then use XDCAM Transfer to re-wrap to MOV making sure all folders are selected so all split and spanned clips are joined.
Gotcha. What's CRC?
I have a file that somehow is out of it's BPAV folder (it's from a 2nd camera on a shoot, someone else did the download) that I can't get Transfer to process. I know it's there, I've used VLC player to watch the footage. Any reco's on how to restore it?
The card is long since erased.
Canon 5d mark 2
[Chandler Kauffman] "Gotcha. What's CRC? "
There's a pref setting to turn this on in ClipBrowser. It basically checks to make sure the data matches the source. Here's the explanation in Wiki.
[Chandler Kauffman] "I have a file that somehow is out of it's BPAV folder (it's from a 2nd camera on a shoot, someone else did the download) that I can't get Transfer to process."
Import the raw MP4 into ClipBrowser and it'll create a new BPAV. Use File/Import.
BTW I picked up a 4TB Drobo to back everything up. So far so good.
Canon 5d mark 2
Ok, so Craig, questions, here's my problem, for a while I have several BPAV folders either not import, or it takes multiple attempts to make sure they copy all the way. Ones that do import, I've got several that are not giving me clean video. I get all sorts of green pixel lines and artifacts in my video. What can I do to take these out? I've tried converting from the card, no luck. And, with other forum info I've tried, using Clip Browser, it shows up in Clip Browser playback. I've copied the original BPAV to a card, from the archived external hd, and using Clip Browser, made a copy of the BPAV to see if it fixes the CRC, but the copy is a copy of the corrupted BPAV, it doens't affect the quality, or am I doing it wrong? I'm running the latest version of Clip Browser, XDCam Log & Transfer won't work, and the video is not clean when testing it through a mac with just leopard, no snow leopard.? What do think I can do?
Salt Lake City, UT
FCS3/Sony EX-3/Mac Intel
[Tom Laughlin] "using Clip Browser, made a copy of the BPAV to see if it fixes the CRC, but the copy is a copy of the corrupted BPAV,"
CRC only ensures it matches the source. You need to use ClipBrowser for your initial copy from the cards. Once a bad copy is made, all things stemming from those copies will continue to be bad.
Play the source clips in ClipBrowser. That's not using Final Cut Pro's EX codec. If it's bad in Clip Browser then the files themselves are corrupted. If it's good there post back and see if we can figure out where the trouble spot might be.
It appears that out of about 300 BPAV folders or more, we have about 77 with corrupted video. We are using a few things that I'm not sure if they are the source of the problem or not. We are often hard-lined SDI into a switcher for events, so I'm not sure if it is a bad SDI-in port in the camera or not, so you have experience with this and if quality suffered?
In the field, we are shooting all over and often inside offices, and we use wireless video transmitters and the IDX battery mounts with IDX batteries powering the wireless system and the camera all at once. I'm not sure if airports and their security systems are 'frying' the cards and the footage on them, but many times, we play the video, about 95% of the video is playable with no artifacts, when played inside the camera. Maybe the artifacts are too small to see in the viewfinder window, but we don't see them.
I don't see them when the video is played and video is captured back to FCP. It is when the video card hits the computer. I know we had once, it happen where we converted from the card and the artifacts went away, but weren't able to consistently replicate that fix, it fixed a few cards, but most of the artifacts went away, not all, on the cards where that fix worked.
How can a Sony XDcam record hours and all the video end up corrupted? Can Transcend cards corrupt the media recording capabilities of a XD camera? Again, this is a significant amount of cards that are or had recorded and corrupted. We have used the same cards and backed them up over and over, is it possible for cards to go bad after many uses?
Do you know of anyone that has had or is currently having this problem? We are also having some editors having to actually edit around these artifacts, in FCP, they are editing out frames, and some of the video is not as bad as others, but I'm in a real pickle about this, as we have hundreds of hours that are being logged and so far, into this particular project with over 1TB of video, about 70% is somewhat artifact"y".
Maybe it is the camera, but we have A and B camera both having issues with this artifact problem. This one project was shot on and off over a period of three months, so, there's no consistent way to know how the cameras were reacting in the field, when in playback, the video looked fine.
I read a forum about CRC, I think you posted it, in which you talked about not using Mac's "Finder", and I'm not sure if that is part of the problem, as most of our work with off-loading was drag and drop and verify files by exact size, we assumed it was a clean transfer and the rest was history. I'm thinking more and more that it is camera recording, because we have always had either clean video, and Finder transferring happened, in every way. I don't think we've ever used Clip Browser, and the concept of CRC checking wasn't emphasized with the guys we got the cameras from.
I know many others who have had no problems with drag and drop using Finder, and this CRC, how crucially important has it been for you, in terms of what is actually does? And would this cause or contribute to media corruption? Is there any suggestions you have and can I email you some screen-grabs on Monday?
Salt Lake City, UT
FCS3/Sony EX-3/Mac Intel
I have no clue what your workflow is from your post. It's all over the place.
Damn I feel like a FRIGIN BROKEN RECORD.
Whether from SxS or SDHC
ClipBrowser with CRC - NO EXCEPTIONS EVER!
Beginning and end of it!
You have no idea where the weak point is in your workflow because you seem to have no system beyond creating multiple weak points.
SDI out is another story but that has NOTHING TO DO WITH BPAV.
If you're using SDI in the field you better have the field techs to QC ON LOCATION.
Certainly SDI to devices like nanoFlash is sound but you still have to check connections and test.
Hard Drives are a danger zone. Files can corrupt even if drives don't crash. BACK UP IS IMPERATIVE and IMMEDIATE. I prefer Optical Disk (Flavor is you choice), others use well tested tape drive technology.
When you've done ALL THE ABOVE then you might consider the camera at issue. Currently you have NO WAY OF KNOWING.
Yes cards can develop bad sectors but you have so many holes in your workflow you have no idea where the source of your problem. I can't imagine the cameras themselves to be at issue. Of course if the slot contacts (or card contacts) get dirty that can be an issue.
Sorry to stir you up, it's not my intention to upset anyone.
This is our workflow:
1. We shoot using the (2) Sony EX-3 using 32GB Transcend Class 6 Cards. (1080p/24fps SP-mode). We always stop and power off the cameras before we eject the cards. We never let the camera keep rolling and "roll into a new card".
2. We transfer the cards BPAV Folders to 500GB Rugged Orange drives using Mac's Finder.
3. We verify the transfer by opening the file's info and seeing that the exact file size matches the original file on the card.
4. We play the original card and verify no video is corrupted.
1. We try using Clip Browser to play the BPAV Folder from off the hard-drives, and also importing them using XD Log and Transfer.
2. The clips playback with several green artifacts that look like static pixel lines, or lines of video that are pixeled. Some lines are green and some are lines that are thicker than others. Some are smaller and longer, etc, etc. The audio is fine, just the video.
3. We have had some projects where there is no artifacts and we have not altered any part of our workflow. We shot on brand new cards, and off-load them once the files are transferred to a HDD.
4. We try using FCP for the conversion process, as well as the XDCAM Log & Transfer to edit the video. The artifacts are not seen in the playback on the camera. They show up after the transfer or the conversion to .mov.
In reference to our workflow, we haven't changed one thing, as mentioned in your response, we are simply trying multiple ways of trouble-shooting this artifacts problem. Interesting that it is happening with two separate cameras both having the same issues.
Can cards go bad only after 2-3 re-uses? Can a Sony camera's recording and internal recording mechanisms go bad? Could the hard-drives, as you mentioned, be corrupting the media? Would e-Sata be better than a Firewire 800 Transfer? Do you think that a card can be fixed using CRC checking even after the transfers? How did you learn about CRC checking and has this been something other people have experienced, in terms of artifacting? How can video not be corrupted on the cards, yet with using 3 different transfer test methods, the cards still getting artifacts? You don't have to respond to this post, I know this is over-kill.
Thanks a ton,
Salt Lake City, UT
FCS3/Sony EX-3/Mac Intel
[Tom Laughlin] "We always stop and power off the cameras before we eject the cards. We never let the camera keep rolling and "roll into a new card". "
There's no reason to do this and if you power down while still writing to the card you will corrupt the file.
[Tom Laughlin] "We transfer the cards BPAV Folders to 500GB Rugged Orange drives using Mac's Finder. "
Wrong way to do it. ALWAYS use ClipBrowser with CRC ON.
[Tom Laughlin] "We verify the transfer by opening the file's info and seeing that the exact file size matches the original file on the card. "
That does NOT verify that data was corrupted. That's the point of CRC checking.
[Tom Laughlin] "We play the original card and verify no video is corrupted. "
Short of playing the video from end to end you can't possibly verify everything. The card can be fine. The copy can be bad.
[Tom Laughlin] "We try using Clip Browser to play the BPAV Folder from off the hard-drives, and also importing them using XD Log and Transfer. "
Too late. You need to use ClipBrowser when copying from the card. If the copy is corrupted you're only maintaining the corruption at this point.
[Tom Laughlin] "The clips playback with several green artifacts that look like static pixel lines, or lines of video that are pixeled. Some lines are green and some are lines that are thicker than others. "
Proves my point. Obviously you haven't verified your copy in any technically meaningful way.
[Tom Laughlin] "We have had some projects where there is no artifacts and we have not altered any part of our workflow. We shot on brand new cards, and off-load them once the files are transferred to a HDD. "
See my past comments about driving without seat belts. You may have just had a car accident without wearing a set belt.
[Tom Laughlin] "We try using FCP for the conversion process, as well as the XDCAM Log & Transfer to edit the video. The artifacts are not seen in the playback on the camera. They show up after the transfer or the conversion to .mov. "
Sorry but this makes no technical sense. You can't play the .mov in the camera. Are you copying the BPAV back to the camera? What are you playing in the camera? Are your copied BPAV playing fine in ClipBrowser?
Tom, I'm not trying to be unkind. Professionally speaking your workflow is flawed. It's high risk. It should not be done that way. It's an accident waiting to happen and have just happened.
Check COPIED BPAV are flawless in ClipBrowser with clips displayed at full 1920x1080 (Option key + Return) to avoid scaling. This isn't using the Quicktime EX codec so it eliminates any issues relating to that codec and/or bad re-wrap to mov. Please check a known bad file.
If you have a minute: firstname.lastname@example.org. I'll email you my cell number.
Thanks for your timely responses.
Salt Lake City, UT
FCS3/Sony EX-3/Mac Intel
Thanks for answering my questions. No, what I meant was, when we playback the BPAVs on cards and using the cameras to playback the video on the cards, the video is clean from what I see. But, yes, it doesn't mean the entire card is clean. I know the cameras don't playback .mov's, I meant when 'we play cards with the camera'. But yea, we'll be doing CRC-checking from now on. This CRC-info was never emphasized by the Sony work-flow dudes at NAB this year, the B&H guys, or anyone else up to this point, before hitting the forums to trouble-shoot this. This is a major issue, that needs to be emphasized more. I guess we're part of the small minority of whom have not been so lucky as others - who've gone way longer and have never had these problems, with CRC-checking. It's just so frustrating that we've been doing this for a few months now, and are just now seeing this flaw in workflow.
Craig, thanks again for your posts. I really appreciate your patience and time in answering. Have a great week. Look forward to your future posts.
Salt Lake City, UT
FCS3/Sony EX-3/Mac Intel
[Tom Laughlin] " This CRC-info was never emphasized by the Sony work-flow dudes at NAB this year, the B&H guys, or anyone else up to this point, before hitting the forums to trouble-shoot this. "
Sony doesn't even mention ClipBrowser in their XDCAM EX Final Cut Pro guide on their website. The original Clip Browser 2.0 didn't have it as an option. Apparently they learned better and added to a following update. They never added any details about it on their site though.
In the ClipBrowser 2.6 manual (Mac Page 74)
Copy error check option
For greater copy reliability, you can enable the CRC (cyclic redundancy check) error check option.
and page 90
Perform CRC check after copy: When this check box is checked, the application performs a CRC (cyclic redundancy check) error check after copying a file, and displays a message if a copy error is detected.
[Tom Laughlin] " I guess we're part of the small minority of whom have not been so lucky as others - who've gone way longer and have never had these problems, with CRC-checking. It's just so frustrating that we've been doing this for a few months now, and are just now seeing this flaw in workflow. "
Actually I think over time a fair number of people bump into issues that may be the result of bad copies. It isn't common but it's a numbers game. Shoot day in day out and the odds increase.