| Migrating primary representations from one device to another
• | | | |
 | Migrating primary representations from one device to another
by Jonathan Bierman on Mar 7, 2012 at 11:03:44 pm |
I'm in a tight spot - I hope someone can advise.
The XSAN where we are storing media for a new documentary (our "XSAN2") is rapidly filling up, and there is still more footage to log and transfer, and add to the EIP device. We are going to run out of space, and we cannot clear any more off this SAN.
Luckily, we do have another XSAN ("XSAN 1")... although currently full, it will soon be nuked/paved to make way for this doc.
My question is, how difficult/easy is it to "migrate" the primary representations of the media but have FCSrv's catalogue connect to the media which is currently on the XSAN2? Will it be possible to do this? Or will I need to rebuild the database from scratch?
Again, any assistance greatly appreciated.
Jon
| | | | |
• | | | |  | Re: Migrating primary representations from one device to another
by Andrew Richards on Mar 8, 2012 at 1:22:52 am |
First, let me get the obvious out of the way: do you need to clear off XSAN1 for use elsewhere? Because you can just define XSAN2 as another device and leave everything on XSAN1 where it is.
If you do need to clear XSAN1, it is pretty straightforward to do what you want to do.
1. Run a manual backup of FCSvr and then stop services.
2. Copy the entire contents of your XSAN1 devIce over to XSAN2, maintaining the directory structure.
3. Unmount XSAN1 from the FCsvr server.
4. Start FCSvr services and go into the Java client's admin panel and modify the Device that was XSAN1 to have the same path except with XSAN2 in place of XSAN1.
5. Stop and restart FCsvr services.
6. Check some assets to make sure FCsvr sees the Primary Reps.
7. If all is good, you're done. Run another backup. Clear XSAN1 before you remount it to the FCSvr server to avoid confusing the database (though it really shouldn't matter).
If you get to step 6 and Primary Reps are coming up offline, you can always revert to XSAN1 by restoring your backup from step 1. For step 2 you should look at using cvcp on the command line for a much faster multithreaded copy on Xsan. Test a copy and make sure it maintains your filesystem permissions on the destination.
Best,
Andy
| | | | |
• | | | | | Re: Migrating primary representations from one device to another
by Jonathan Bierman on Mar 8, 2012 at 2:07:51 am |
I didn't explain the scenario very well... mainly because it is stupid and complicated. I'll try to be brief.
I started by creating the EIP device on the XSAN2 because it had more free space than the XSAN1. I thought we'd have enough space to do what we needed to do. Due to poor Log and Transfer techniques by the students, (a lot of L&T was done in the field, I told them to do so instead of just copying the CF card data, big mistake on my part) the media came back waaaay bigger than it should have been (5D footage encoded ProRes4444 because they didn't check L&T preferences, for example)
I am currently transcoding all the 5D footage using a watcher back to ProRes422, so this will free up some space, but it'll take a while for the transcoding to finish.
As it stands, the EIP device (on XSAN2) contains about 1.6TB of transcoded footage, which is catalogued by FCS, and the students are currently logging/tagging/describing the assets... but there is a bunch more footage which has NOT been L&T'd. I anticipate at least another TB worth will be added to the EIP device (but at least now the codecs will be lighter, because I will be doing the L&Ting) There is 1.7TB remaining on the XSAN2... therefore I anticipate we'd be getting dangerously low on space, which could cause the whole thing to just fall over. So, we need to migrate. Phew.
We have an older XSAN, XSAN1. Its capacity is smaller, 3.6TB, and is currently full of another, much older project. The folder hierarchy is total junk. It's a disgraceful mess, and basically the director of the school has given the order to back up essential files, and wipe it clean so we can use it for this new, properly managed project... therefore, once cleared, I will be creating a folder on XSAN1, and copying the primary reps from XSAN2 into this folder, and then deleting the primary reps from XSAN2...can I then just go into FCSrv administration and just change the path of the EIP device to this new folder on XSAN1, and everything will remain the same?
Sorry for length.
| | | | |
• | | | | | Re: Migrating primary representations from one device to another
by Andrew Richards on Mar 8, 2012 at 2:39:36 am |
Oops, my directions had your XSAN1 and XSAN2 reversed. But the process is the same but in the other direction.
But...
It sounds like you'd be trading one getting-too-full-for-comfort volume for another. So why not just draw a line in the SAN (har-har) and stop adding to XSAN2, wipe XSAN1 and start using XSAN1 alongside it as another EIP device?
Best,
Andy
| | | | |
• | | | | | Re: Migrating primary representations from one device to another
by Jonathan Bierman on Mar 8, 2012 at 7:14:26 am |
"draw a line in the SAN"
Arf... I bet you've been waiting years to drop that one ;)
re spreading the assets across two devices on both XSANs: that had certainly occurred to me. the problem is that I use search filters based on a "stored on device" lookup to filter assets to only show media from this particular project, in this case the device is called AMAZON Master EIP. the footage is all from Brazil, and none of the other media managed by FCS is from Brazil, so I could filter it that way, I suppose...
If I had a second device (say, "AMAZON Master EIP 2"), would it be possible to include some sort of "and" modifier in the asset filter, to show assets stored on the devices AMAZON Master EIP 1 AND 2?
and for the sake of tidiness and organisation, I would really like to have ALL the media residing in one folder on one storage device.
I think with judicial deleting of junk or unnecessary footage, I should be able to keep the size of the media under 3TB... which would be just about acceptable on an XSAN of that size. obviously not ideal though.
| | | | |
| |
|
Reply
Like
