FORUMS: list search recent posts

Cat DV - Slooooooow

COW Forums : Square Box CatDV

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
Aaron Rock
Cat DV - Slooooooow
on Jan 3, 2014 at 6:34:33 pm

So, we have been working in CatDV for a few years now. One issue becoming increasingly problematic is trying to transfer media in and out of catdv through the actual program.
Recently I tried to update the location of a few files we had transferred onto the server. I selected the unlinked files in catdv and chose "Update media location".

Unfortunately, the folder we house all our media in will not open in CatDV.

I tried to select the "media" folder (where we keep all video files) and waited for 25 minutes while he program was "thinking". A list of files in this folder never appeared.

Thus to copy files onto our server I have had to resort to doing everything through finder (copy the media files, delete the originals, renaming et.al.) then dragging them to CatDV. It just seems to me that this is wildly inefficient for this program, is there some particular reason why CatDV would fail to function in this respect so completely? In other words, am I doing something wrong? or does the program just not work?


Return to posts index

Robb Harriss
Re: Cat DV - Slooooooow
on Jan 3, 2014 at 7:20:16 pm

The obvious question:
How big is the database?

Non-linear: all the time and nothing but.


Return to posts index

Aaron Rock
Re: Cat DV - Slooooooow
on Jan 3, 2014 at 8:10:32 pm

Massive (15k clips+), which is part of the issue, but the database has only recently slowed down to this extent, and we haven't added too much to the initial catalog.

Over the past year probably 1500 clips, when we started with CatDV 2 years ago, the Catalogs already had roughly 12k clips and the speed was markedly faster.

I guess thats why I'm wondering about the sudden slowdown. Could it be anything else or are we just finally getting to the point where we'll need to sub-folder some of our footage?


Return to posts index


Tim Jones
Re: Cat DV - Slooooooow
on Jan 3, 2014 at 9:29:46 pm
Last Edited By Tim Jones on Jan 3, 2014 at 9:30:07 pm

Robb beat me to it, but catalog size (file count) is definitely your issue here. Also, using subfolders won't fix the issue because the file count still remains high. Your only solution is to create smaller catalogs that separate the files into smaller (recommended sub 8K to 10K) sections.

As a stop-gap work around, what happens if you modify the advanced properties to use 2048K of memory (Preferences -> Advanced Functionality)?



Tim
--
Tim Jones
CTO - TOLIS Group, Inc.
http://www.productionbackup.com
BRU ... because it's the RESTORE that matters!


Return to posts index

Robb Harriss
Re: Cat DV - Slooooooow
on Jan 3, 2014 at 9:44:09 pm

Wait, you have a single catalog with 15000 clips in it? Ok, that's going to cause a problem no matter what you do.

I have a LOT more clips than that, but they're cataloged by shoot location. That's just for convenience. Many of our projects are created by pulling clips from many different locations, as if it were stock footage. Well, I guess it is because it's repurposed as well. We're our own stock agency in that sense. So I have a long list of catalogs filled with lots of clips, as I hope all the CatDV users do. But then how to find anything? Well, that's why we went from a couple of copies of Pro to the Workgroup server. The server version allowed us to search across all the catalogs. So now that we have a boatload of shots and clips in many catalogs we can treat them all as one when we have to. We have one drive on the network that holds all the proxies all the time. That way I don't have to worry about having full-fez footage online for logging and searching. Plus I'm pretty much the only one who needs to go near the full-fez clips.

Non-linear: all the time and nothing but.


Return to posts index

Aaron Rock
Re: Cat DV - Slooooooow
on Jan 3, 2014 at 9:44:44 pm

The Memory allocation helps somewhat, thanks, I think there's a communication mixup on the "database" question.

The server (database) as in, the raid where we house all video files is massive.
The catalogs are relatively small (at least the ones I'm working with) the particular catalog I'm in only has (had) about 75 clips, however I was trying to update the media location to the files on the server as opposed to those on my computer.

The folder holding these files contains about 15000 items. These are nested in various catalogs on CatDV, but housed in one location.

No matter how small or large the catalog I'm in is, I still deal with the program freezing as soon as I try and access this folder location through CatDV. Does that make sense? I'm wondering about shrinking the size of this folder by adding subfolders, but don't know if this will get to the core issue, at the same time I know the core issue can't be catalog size because it happens in any size catalog


Return to posts index


Robb Harriss
Re: Cat DV - Slooooooow
on Jan 3, 2014 at 9:46:06 pm

ok, that makes me feel better. I was scared for a second there.
:-)

Non-linear: all the time and nothing but.


Return to posts index

bryson jones
Re: Cat DV - Slooooooow
on Jan 4, 2014 at 2:44:38 am

If you're talking about using the tree to browse the file system, yes, that will be slow.

Just use your finder and drag and drop, or better yet, use your CatDV filters to gather groups of files and move them into folders to ease that issue.

When you browse in CatDV a lot is going on as the system is gathering info as it goes and that can create a lot of overhead.

Is there a reason that you are using the tree or is that just the way that you came to use the app? I tend to not use the file browsing much so I'm curious to hear what the advantage or use case is?

bryson

bryson "at" northshoreautomation.com

northshoreautomation.com


Return to posts index

Aaron Rock
Re: Cat DV - Slooooooow
on Jan 6, 2014 at 5:37:35 pm

Thanks Bryson, I see what you're saying. When we move files into or out of catdv, we often just use drag and drop.

To archive files into catdv and onto our server, we run into issues.

When we add a file to a catalog from one of our computers, we drag and drop the file into the catdv window. Then copy the media files onto the raid we use as a server (using "Copy Media Files" and being sure to include the checkbox "update media so they refer to new copy").

However, if files are accidentally copied to the wrong location, or their media reference point isn't updated, then we have to update the connection to the base file in the tree, Whether I select the files individually and search them out, or grab a cluster and choose "update media location", once it tries to access the server and populate the list of media files in our server, it freezes up.

Generally we use the tee to reference catalogs and nothing else. It seems like the slowdown comes when the program tries to populate a list of all the media files housed in our RAID. These files are easy for it to reference in smaller catalogs, but not when it is trying to open a window which accesses our entire media folder. There is a smaller folder of files we kept separate in the database. Accessing these files takes a matter of seconds, where the main database can take 5 mins. plus.
Below I'll try and illustrate what I mean.


This is our media folder, right around 27,000 clips. Copying media to this folder through catdv is next to impossible thanks to the size of the folder.


Conversely, this folder only holds a limited base of clips, this updating items in this folder either through catdv or finder is a breeze.

So my question is, would the best way to fix the problem be to bring files into catdv only from the server (as in copy the files to our server raid, then bring them into catdv or would it be better to trim this folder down to a more manageable level with sub-folders and keep on using it the way we have? Or is there a better option than either of these?

Thanks for all the feedback. I definitely appreciate the help.


Return to posts index


bryson jones
Re: Cat DV - Slooooooow
on Jan 7, 2014 at 4:29:19 pm

If it were mine to manage, I'd say get the folder size more manageable.

You don't mention a reason for not using folders. Is there a reason? If not, then split it up and many problems will disappear.

As this is an AFP (I'm assuming 1Gb ethernet) I believe that this indexing will be a problem for many apps, not only CatDV. Additionally, it will only get worse as the file count increases.

Also note that if you want to work with this structure you could also look at moving to a 10Gb infrastructure and faster storage. (Not sure what the raid speed is like on it's own.) But I'd imagine that opening this folder in the OS X Finder is probably pretty slow/impossible if the previews are turned on.

This situation was fairly common on Final Cut Server rigs. We used to joke about saying "do not open that folder" as it would take forever to open and would lock your system until it was done or the rig would crash. This was just in the finder, not even an app.

bryson

bryson "at" northshoreautomation.com

northshoreautomation.com


Return to posts index

Dougal Matthews
Re: Cat DV - Slooooooow
on Jan 7, 2014 at 4:25:52 pm

hi Aaron

From the sound of things most of the slowdown is due to trying to browse the files on your storage if you can separate them into sub-folders then that will certainly speed catdv up

I'd also recommend splitting any large CatDV catalogs up into a number of smaller catalogs all your server searches will run across all of your catalogs but individually they'll be much more responsive and easier to visually search in the client

I hope this helps you to improve the performance

thank you Dougal

Improvise Adapt and Overcome


Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2017 CreativeCOW.net All Rights Reserved
[TOP]