New to Smoke, hoping for some advice on an Archive Strategy.
I'm generally using it for TV spot finishing, so the quantity of data should not be too overwhelming, I hope. I'm still wrapping my head around the best workflow during the process - when to use History vs. saving all setups, etc.
For now - I'm thinking I should just archive entire projects, just to be safe - (there's not tons of raw footage, as I'm just doing the finish). My first job was two :30 spots - raw footage DPX scans, pretty moderate Color & VFX work on both, and all of the HD/SD deliverables. Archiving the entire job was 200 GB.
I do have access to an LTO-5 machine (Cache-A), so even though S-Mac only does file based archiving, I'm thinking I could set up a "SMOKE_JOBS_ARCH" folder on my RAID, and whenever than gets to 1.4 TB or so, I'll just copy the contents to LTO tape and clear it out. So, if 200 GB is typical, I could fit 6-7 of those per $100 LTO tape. Not bad, eh?
I was planning to just estimate the size for the entire archive, then "Format" it to be slightly larger. I understand how the format capacity would be useful if archiving to multiple discs, but is there any reason to do that with my LTO plan? I suppose for a large project, each "bucket" would have to be 1400 GB to fit on my LTO-5, tape - but is there any advantage to splitting a 200 GB archive into 20 GB chunks, or whatever? Faster to retrieve a particular shot, if you don't need the whole project, etc.?
Thanks for any help!
We do something similar here. I break my archive in to 20GB chunks. It was a recommended practice by Autodesk, at one point, but that might have changed. Since I'm running flame, I use Batch, but in many ways History is kind of like a "batch lite". So I would archive those and your setups along with your data. One thing you can do to save storage, is to unrender all versions of a spot, except your final version. If you ever have to restore a project, you will most likely only be restoring the final version anyways.
I too do mainly spot work. Most of your archive size is in the source footage. My archives are normally anywhere from under 1 gig to 100 gigs. Most are around 50gig.
We shoot on P2 and that footage is kept on USB hardrives which is where I import them from. So I never "store local" my HD footage. I run it from the USB drive. So when I archive, I archive as LINKED. This will save any material I made in the system in modules like Action and then just archive the link to the footage on USB. This keeps my archives small. I always use History, so my setups are saved with the clips. I never archive setups separately, but that's just me. I also just archive my final edit sequence. (which contains History)
If you want all of your footage stored as well then you can archive as STONE, but then the archive will be large. Things that you can do to help trim the fat a bit is to do a Consolidate on your edit to lessen the handles, and then unrender the clip. This will still save all of the soft effects, but not the rendendered files. You can re-render them when you bring the archive back.
Hope that helps. Everyone has their own needs. I always know that the footage on the USBs will be there, so when I unarchive, I juts remount the drive and the project with footage all comes back.
Senior Editor - Autodesk Smoke
WTHR-TV Indianapolis,IN, USA
To me 200 GB is way too large for any archive. It would take too long to retreieve and restore. Time is money.
Archive only the final spots, unless you have a good reason to save other elements as well.
Build as much into the timeline as possible. Without BatchFX you are limited in this regard, so yes, use History for everything. But then ditch the intermediates. You should use setups only when necessary (I never use them at all except for deliverables, which you don't have).
Save textless as a layer or a seperate timeline. The most requested element from a client will be a textless pass. Never fails.
Then it's a decision of linked or unlinked. Are you soft-importing your elements and plan on saving the sources seperately? Then do a linked archive. Did you capture everything from tape? Then unlink the VTR based media (use filters in the timeline and unlink them) since you can always recapture this media. If the media will be backup up somewhere else, there is no reason to back it up again.
If you try to archive entire projects you will soon discover you will be overwhelmed with data management tasks. Those don't earn you money and are incredibly boring.
Besides, how much is the client paying you to store thier crap? What are the expecations for how long you will keep thier crap? This needs to be spelled out in the contract between you and them. If this is a service you wish to offer, then charge the client for it.
Good to hear another perspective. Is there any trouble with using history for everything? I keep hearing in tutorials that is "could" bog down your system. I haven't noticed it yet, but I also haven't been using it that much.
Any idea when it will "bog down"? - on a feature length project, with every shot carrying Color Warper and Action history? Most of my work is TV spots, so for short projects, maybe I'd be ok?
We're actually in-house at an ad agency, so we're quite often doing revisions to previously finished work - we always save textless versions of everything, so I'm sure I'll be covered. And while the expectations are ridiculous, (people honestly think we save every frame that ever passes through our computers), we have often saved people's asses by over-archiving (especially with tapeless media), even if the agency's client doesn't pay extra for it. It helps us keep getting the work, to be honest, because producers know we'll have their back if something comes up down the road... You never know when a Portuguese version will pop up, and I have to re-comp in different product labels, etc...
I have a two :30 spots project coming in next week - I'll try use History/Consolidate/Unrender/Archive and see how it goes. Thanks for your help.
I can't speak to using History as I haven't used it very often. I do everything in the timeline as I have BFX at my disposal. When I have used History (on a Mac) I haven't noticed any performance hit. With the ability to choose to use history or not everyime you go to process in a module, you should be able to keep it to only what's needed.
If you have BFX there is no need for history at all. Build everything in the timeline.
I wanted to express that your time is vaulable. If you are spending time and money storing your clients projects... you need to bill for that. This is a cost of doing business and is billable.