Massive Amounts of XD Cam files

	Massive Amounts of XD Cam files by Tom Daigon on Aug 18, 2008 at 12:04:14 am The facility I work at just switched over to FCP. We have 2 bays with 8 tb for each. A series of 13 1 hour shows will be editied simultaneously (segments). Theres a good chance it will be shot on XD Cam and mastered as DVCProHD. We are about to be buried in media and I am wondering if there is any way the XD Cam can be injested as a low res proxy then conformed back to the Hi res clips on the blu laser disk. I checked out MXR4MAC and FLIPFORMAC and neither seem to offer this workflow. Yes, I know storage is cheap, but I find it hard to believe that this new technology with all the info that is embedded in the clips is unable to function in the "low res/high res" work flow that tape did so well. In some situations it would be nice to have the option. Respond to this post   •   Return to posts index

	Re: Massive Amounts of XD Cam files by David Roth Weiss on Aug 18, 2008 at 1:03:43 am Tom, XDCAM HD at 35mbps is only 15.31Gb per hour and at 25mbps it's only 11Gb per hour. That's great looking HD but at DV file sizes, and you can fit about 100 hours per terrabyte. That's nothing. I don't know why you would even consider trading in your time for hard drive space at that kind of miniscule data rate? David Roth Weiss Director/Editor David Weiss Productions, Inc. Los Angeles POST-PRODUCTION WITHOUT THE USUAL INSANITY ™ A forum host of Creative COW's Apple Final Cut Pro, Business & Marketing, and Indie Film & Documentary forums. Respond to this post   •   Return to posts index

	Re: Massive Amounts of XD Cam files by Tom Daigon on Aug 18, 2008 at 1:10:50 am I think the data rate they will be shooting at is 50mbps, but your point is well taken. I just want to know what contingencies I have if other large projects materialize. After buying both bays the owner will definatley not spring for more storage so its good to know what options I do and dont have with the files. Respond to this post   •   Return to posts index

	Re: Massive Amounts of XD Cam files by walter biscardi on Aug 18, 2008 at 3:16:10 am One big issue we just found with XDCAM is that the camera simply repeats file names at the start of each disc. Hopefully your camera operators will be better about setting up the file structure, but we got 14 discs that all started with clip C0001 so even if we did want to work in some sort of off line / online workflow, it wouldn't work with those discs since we have 14 C0001 clips. That's one place where I find that P2 works very nicely in that even with the raw file names, it never repeats on the same camera. As for your archiving / backup, there's a new replacement for DLT tape that's much faster. We're going to be starting a new series with either the EX-1 or EX-3 and before we even start the edit, all material will be backed up to this format. By the way, the XDCAM discs were shot at DV50 (SD) and we put all 14 discs on our system (8TB) and it's just 185GB total. Walter Biscardi, Jr. Biscardi Creative Media HD and SD Production for Broadcast and Independent Productions. STOP STARING AND START GRADING WITH APPLE COLOR Apple Color Training DVD available now! Read my Blog! Respond to this post   •   Return to posts index

	Re: Massive Amounts of XD Cam files by Shane Ross on Aug 18, 2008 at 5:13:36 am Yeah, this is one issue with the tapeless formats. No way to import as low res. Gotta bring it in full resolution. We have an odd struggle with this too. Complicated by the fact that we are mixing with HDCAM and MULTICLIPPING the files. Don't ask. Shane GETTING ORGANIZED WITH FINAL CUT PRO DVD...don't miss it. Read my blog, Little Frog in High Def Respond to this post   •   Return to posts index

	Re: Massive Amounts of XD Cam files by Wayne Carey on Aug 18, 2008 at 3:21:21 pm Oh... But, there is a way to do all that you are asking. Its very different from P2. Working with the low-res proxies are a chore BUT can be done through the Sony Transfer Software, under the menu - Clip - Export Proxy Movie to File. You'll have to select which you want and place them into a directory of your choice in the program prefs. As for the file naming, name your discs with the Sony Transfer Software and choose an export directory and all of those files will go into that folder under the disc name. And if you don't like the C0001 naming convention, just log your clips with the same software under the tab - Logging and give your clips a name. It will then be able to save it to the disc forever. Anytime you use this disc, all of the clip names and disc names will follow your forever. Not putting down P2, but Panasonic can't do this... _______________________________ Wayne Carey Schazam Productions http://web.mac.com/schazamproductions schazamproductions@mac.com Respond to this post   •   Return to posts index

	Re: Massive Amounts of XD Cam files by walter biscardi on Aug 18, 2008 at 4:17:01 pm [Wayne Carey] "As for the file naming, name your discs with the Sony Transfer Software and choose an export directory and all of those files will go into that folder under the disc name. And if you don't like the C0001 naming convention, just log your clips with the same software under the tab " That's nice when you have time. In our case we had 14 discs to transfer and only had access to a camera for 10 hours. Barely enough time to actually transfer all the material so we did not re-name anything. We're doing it now in FCP. That's one place where I really like the P2 workflow and it's something we'll be working on with videographers when we start our new series. Walter Biscardi, Jr. Biscardi Creative Media HD and SD Production for Broadcast and Independent Productions. STOP STARING AND START GRADING WITH APPLE COLOR Apple Color Training DVD available now! Read my Blog! Respond to this post   •   Return to posts index

	Re: Massive Amounts of XD Cam files by Wayne Carey on Aug 18, 2008 at 4:34:00 pm Very understandable, Walter. It does take time, but I'm sure that you could rent a Sony PDW-U1 portable deck. Its has a USB connection and is real cheap - about $3200, so I'm sure that rent has to be reasonable for this deck. Just something to think about for the future... _______________________________ Wayne Carey Schazam Productions http://web.mac.com/schazamproductions schazamproductions@mac.com Respond to this post   •   Return to posts index

	Re: Massive Amounts of XD Cam files by walter biscardi on Aug 18, 2008 at 5:12:16 pm [Wayne Carey] "Very understandable, Walter. It does take time, but I'm sure that you could rent a Sony PDW-U1 portable deck. Its has a USB connection and is real cheap - about $3200, so I'm sure that rent has to be reasonable for this deck. " Actually no, we tried for three days to find a deck to rent in Atlanta, they're not here. Bexel, the largest rental house I know of, has two of them in inventory but they're not in town. The shooter tried to three days to locate a deck for us and found one in Florida, but it was too expensive. So last ditch, we rented his camera but could only have it for a very short time as he needed it. I'm not purchasing any XDCAM equipment as this was the first request we've had for this format and the Producer has already said she's not using it again because of all the problems we had trying to find something to play the discs back on. That's why I say, we're moving forward with the EX-1 cameras as it's easier to find something to transfer the sticks than the discs. Walter Biscardi, Jr. Biscardi Creative Media HD and SD Production for Broadcast and Independent Productions. STOP STARING AND START GRADING WITH APPLE COLOR Apple Color Training DVD available now! Read my Blog! Respond to this post   •   Return to posts index

	Re: Massive Amounts of XD Cam files by Wayne Carey on Aug 18, 2008 at 5:53:35 pm Bummer... Having a hard time finding equipment to rent only means this stuff is so viable that keep XDCAM in stock is a challenge. I don't blame you for not purchasing any XDCAM equipment but if anything comes up just ask for a PDW-U1 deck and you will find it much easier to deal with. Being a P2 shop, I could understand fully your reasoning in this. You had a bad time using XDCAM and we had a bad experience with P2 in ours... so its all in our regular workflows that makes the difference. Just to give you a heads on the PDW-U1 deck... Sometime in the very near future, Sony is supposed to be sending out a firmware update to make that deck be a data compatible deck - meaning that you could write P2 files on the 50 gig Sony discs. Yes, it is in the near future and not now, but it is something worth considering for data storage. _______________________________ Wayne Carey Schazam Productions http://web.mac.com/schazamproductions schazamproductions@mac.com Respond to this post   •   Return to posts index

	Re: Massive Amounts of XD Cam files by walter biscardi on Aug 19, 2008 at 3:27:10 pm [Wayne Carey] "Being a P2 shop, I could understand fully your reasoning in this. You had a bad time using XDCAM and we had a bad experience with P2 in ours... so its all in our regular workflows that makes the difference. " We're not a P2 shop, we're a DVCPro HD shop with all tape based acquisition. We've done precisely one P2 project in four years of HD. I said I like the P2 method of never recording the same file twice, period. Sony should look at that. As I've already said several times on this thread, we're moving to the EX-1 / EX-3 for a new series we're doing. That's XDCAM, but without the large BluRay disc cassettes that caused all the issues with this other shoot. I absolutely will not purchase any XDCAM / BluRay equipment for this shop because I see no reason to. I will, however, support the EX1 / EX3 workflow. Not sure what else to say. if you've invested money in the BluRay Cassette deal, fine, but I'm not wasting my money on something like that. Seems already outdated. I'll stick to the solid state stuff and move on from there. Walter Biscardi, Jr. Biscardi Creative Media HD and SD Production for Broadcast and Independent Productions. STOP STARING AND START GRADING WITH APPLE COLOR Apple Color Training DVD available now! Read my Blog! Respond to this post   •   Return to posts index

	Re: Massive Amounts of XD Cam files by Jeremy Garchow on Aug 18, 2008 at 6:12:14 pm [Wayne Carey] "Anytime you use this disc, all of the clip names and disc names will follow your forever. Not putting down P2, but Panasonic can't do this... " Au Contraire. It's FCP that can't do this. P2 has some very robust metadata structure, FCP is just not smart enough to see it. Although Quicktime, Motion, and Soundtrack Pro are. Go figure. Jeremy Respond to this post   •   Return to posts index

	Re: Massive Amounts of XD Cam files by Wayne Carey on Aug 18, 2008 at 6:51:56 pm You're right, Jeremy... FCP can't do this BUT the Sony Transfer Software program can. Anyone who is using FCP with XDCAM needs this software to fully utilize it properly. I'll give you that... why is it that FCP can't handle it and Quicktime and the other supporting programs can. Once again, we are held captive by a large company deciding on how we are to use their program, instead of using our input to better ourselves and them. _______________________________ Wayne Carey Schazam Productions http://web.mac.com/schazamproductions schazamproductions@mac.com Respond to this post   •   Return to posts index

	Re: Massive Amounts of XD Cam files by Jeremy Garchow on Aug 18, 2008 at 9:09:18 pm [Wayne Carey] "I'll give you that... why is it that FCP can't handle it and Quicktime and the other supporting programs can." Edited for clarity and posterity: The only thought I have about that is that the other programs in the Suite were written later and Quicktime has been updated A LOT compared to FCP. I don't know exactly why, but it does hinder the workflow a bunch. Respond to this post   •   Return to posts index

	Re: Massive Amounts of XD Cam files by Rafael Amador on Aug 18, 2008 at 5:14:07 am Hi Tom, I guess that may be a possibility to capture the clips at 18Mbps and then recapture at 50Mbps. But, as David says, about starage is the same that if you would be working with DV50. [walter biscardi] "One big issue we just found with XDCAM is that the camera simply repeats file names at the start of each disc. Hopefully your camera operators will be better about setting up the file structure, but we got 14 discs that all started with clip C0001 so even if we did want to work in some sort of off line / online workflow, it wouldn't work with those discs since we have 14 C0001 clips. " With the EX-1 this don't happens. You start with number like that: XYZ_0000 so you have to shoot 9.999 clips to go again to XYZ_0000. Then you can change the values of the three first figures. You can shoot 9.999.999 clips without repeating clip number. rafael www.nagavideo.com Respond to this post   •   Return to posts index

	Re: Massive Amounts of XD Cam files by Tom Daigon on Aug 18, 2008 at 2:13:05 pm Thanks alot everybody. I guess I've stumbled into the Achilles heal of video file technology. I am rather surprised and dismayed that such an effective method (low ress/ high res) cant be implimented when needed with the XDCAM format , although it sounds like P2 is doable. Respond to this post   •   Return to posts index

	Re: Massive Amounts of XD Cam files by Andy Mees on Aug 18, 2008 at 2:46:22 pm its not that it can't be done effectively Tom, its just that FCP doesn't offer free and easy access to such a workflow. this is a choice by Apple, who promote instead the full resolution online only workflow that said, there are options (i posted a couple on your same question on another board) if you're not tied to FCP but are tied to the format and workflow then the online/offline workflow using XDCAM proxies to edit with final conform to full res at a later stage works an absolute treat with other NLE's ... GVG EDIUS and Sony Vegas both offer exemplary XDCAM proxy to full res workflows Respond to this post   •   Return to posts index

	Re: Massive Amounts of XD Cam files by Tom Daigon on Aug 18, 2008 at 3:17:12 pm Thanks Andy. Unfortunately after purchaseing 2 FCP bays recently, we are tied to FCP. Sad that Apple has chosen a path that short changes the options we sometimes need to use. Thanks for the info. Respond to this post   •   Return to posts index

	Re: Massive Amounts of XD Cam files by Mark Raudonis on Aug 19, 2008 at 4:08:57 am Tom, Don't despair! There are still possibilities and work flow that may be right for you. Example. We're shooting the current season of MTV's "The Real World" using Sony's XD-CAM. A typical season of this show will generate approximately 6,500 HOURS of media. Because of this massive amount of media, we employ the classic "Off-line to on-line" workflow. Here's how it goes: Disc is brought in house. BOTH hi and low res media is copied onto our server using the Sony PDW-D1. As Wayne has mentioned, this is basically just a blu Ray player and it's economically priced. We use five at once. The low res proxies are split off into a separate folder and immediately made available to our loggers. They access the X-SAN via AFP using "PilotWare" for logging. The high res media is then recompressed using "Compressor" to "Off-line RT". Using a the Q-master cluster approach, this step is done much faster than real time. We now have three files on our server: Full res media, Sony proxy, and Apple "Off-line RT". The loggers can start immediately, the story department will request "selects" and we will then "recompress" selected media. Because we can't keep all of the full res media on line forever, we purge it after approx two weeks. When picture is locked, we have to media manage the sequence and redigitize the ful res media. This may seem like a lot of steps, but trust me, this is much easier than our old taped based workflow. Anyone who says that XD-Cam doesn't work for you, hasn't really tried hard enough. We've been using the format for years now (Back when it was taped based IMX), and I can tell you that it works. I can also say that I wished FCP would play nicer with the Sony proxies, but fundamentally, there's the problem of "off-line quality" audio. We take the steps that we do specifically to preserve the original audio quality. As for DRW's comment about why anyone would still use "off-line", all I can say is 6,500 hours x 5 shows and you'll be thinking about low rez too! mark Respond to this post   •   Return to posts index

	Re: Massive Amounts of XD Cam files by Wayne Carey on Aug 19, 2008 at 1:38:56 pm [Mark Raudonis] "Anyone who says that XD-Cam doesn't work for you, hasn't really tried hard enough. We've been using the format for years now (Back when it was taped based IMX), and I can tell you that it works. " Here, Here!!! This was the point that I was trying to convey. With FCP, there is more than one way to do ANYTHING. Yes, I said ANYTHING. We, too, have been using XDCAM since its earliest days when it was called BetaSX. And I can tell you that it is the best format that we have used. Keep in mind that every workflow is mostly a personal preference and company preference, its not for everyone. We use XDCAM HD (35 mbit) and HDV (25 mbit) for our programs and for compatibility reasons and speed of editing, all of the footage is captured the old way using Log and Capture. Now, when a program comes along that is all XDCAM HD, we edit entirely in XDCAM HD. Our final product is then written back to XDCAM HD for program masters. But like I said, this is OUR workflow. Anything can be done if you just experiment and give it some thought. _______________________________ Wayne Carey Schazam Productions http://web.mac.com/schazamproductions schazamproductions@mac.com Respond to this post   •   Return to posts index

	Re: Massive Amounts of XD Cam files by Tom Daigon on Aug 19, 2008 at 2:28:41 pm Thanks Mark. Could you clarify somethings for me? When you say.. "we employ the classic "Off-line to on-line" workflow" does that mean you log and capture XDCAM material using it like a tape playback unit (which would allow a unique library number to be assigned to each disk for later redigitization at high res) or do you use the Sony software to injest clips . The part Im not clear on is how you can edit in low res then redigitize in high res. Do the clips metadata allow you to assign reel numbers to each disk allowing you then to access the unique timecodes for each clip when its time to go high res? Its this match back process that is so easy with the tape work flow that is confusing me with XDCAM. Tom Respond to this post   •   Return to posts index

	Re: Massive Amounts of XD Cam files by Mark Raudonis on Aug 20, 2008 at 6:49:36 am Tom, We do NOT use log and capture. Rather, we use the Sony software in FAM (File Access Mode) to COPY the files from the disc to our X-SAN shared storage. This part never even touches Final cut pro. We have a "tried and true" numbering system so that each disc gets it's own unique number. I've written about it before, so quite simply it's SHOW INTITAL/Date/Camera/load: RW815A01 The "classic off-line to on-line" works like this. These Full resolution clips are then "recompressed" in compressor to create "Off-line RT" media. This is what we edit with. Once picture is locked, we then take the locked sequence and media manage it to create "off-line clips" with handles. Then, using a deck like the sony FDW-F75, we batch digitize the off-line sequence. FCP will ask for a reel number, but it will then find the correct clip based on timecode. This process is a "play video" workflow, and unfortunately not file based. However, the cueing of the XD-CAM is much faster than a tape, so it does go quicker. I hope I answered your question. mark Respond to this post   •   Return to posts index