Watchers not watching
by Drew Pearce
on
Jun 18, 2009 at 8:11:30 pm
Has anybody run across this: I have several watchers set up that trigger copy responses with transcoding. Yesterday, they all stopped working. When an event happens that should be seen by one of the watchers, nothing happens, and nothing shows up in the log either. I've rebooted twice. Anybody have ideas?
Re: Watchers not watching by Andy Gallagher on Jun 26, 2009 at 11:04:33 am
Have you checked to see if the job queue is full? We had a similar problem a while back, turned out that the job queue had filled up. My guess is that with 20,000+ entries in the queue it was taking longer to scan the queue than the interval between the scans so nothing ever happened.
We ended up having to go into the db and manually flush the queue - re-post if you've still got the issue and I'll try to find the SQL code.
Re: Watchers not watching by Drew Pearce on Jul 7, 2009 at 11:50:26 am
Still having this problem. The watchers are watching XSAN volumes if that makes a difference. If you could post the SQL code to clear the queue, that would be great. Thanks.
Re: Watchers not watching by Andy Gallagher on Aug 4, 2009 at 9:19:01 am
Hi Drew,
The following have worked for me on v1.1, but there's no guarantees I'm afraid. This should only be used as a last resort. Always remember to back your database up first!
WARNING: This almost certainly won't work on v1.5 (but I haven't tried it). Apple say that they've completely redesigned the database, so I would expect you to get SQL errors.
The first thing you need to do is to create symlinks so that the console installation of PostGres that comes with Final Cut Server actually works.
Log into your Final Cut Server machine, and in a Terminal window type:
cd "/Library/Application Support/Final Cut Server/Final Cut Server.bundle/Contents/PostgreSQL/lib" [this is all one line]
for x in `ls`; do ln -s $x /usr/lib; done
This will create symlinks (Unix-speak for Aliases, kind of) to all of the libraries that the Postgres commandline client uses.
This runs the Postgres database client (psql) and connects you to the Final Cut Server database, which is called px. You will see a prompt which looks something like
px=#
Type in:
update pxeventresponse set eventstatusid=3;
\q
You should see a message saying something like "20000 rows affected".
The first line is the key one. This sets the column "eventstatusid" in the table "pxeventresponse" to 3 (which means "job complete"), for every entry.
The second line exits the Postgres commandline and brings you back to a Terminal prompt. Note the ; on the end of the line - you need this to make it run.
For me, this made the queues un-stick and things start running again. However, a few days later it jammed up again.... closer inspection showed that Final Cut Server hadn't flushed the rows from the table as I expected it to. If that happens, then what I did was:
cd "/Library/Application Support/Final Cut Server/Final Cut Server.bundle/Contents/PostgreSQL/" [all on one line]
bin/psql -h localhost -p 5433 -d px -U postgres
delete from pxeventresponse;
\q
This completely clears out the table pxeventresponse, and our installation has been running fine ever since.
Re: Watchers not watching by Drew Pearce on Aug 4, 2009 at 11:34:23 am
Thanks for your help Andy. I'm having a bit of trouble though. When I attempt to connect to the database (bin/psql -h localhost -p 5433 -d px -U postgres) I receive the following error:
dyld: Library not loaded: /var/tmp/Pacifica/Pacifica-1.3.36~1/vendor/lib/libpq.5.dylib
Referenced from: /Library/Application Support/Final Cut Server/Final Cut Server.bundle/Contents/PostgreSQL/ bin/psql
Reason: no suitable image found. Did find:
/usr/lib/libpq.5.dylib: stat() failed with errno=62
Trace/BPT trap
Looks like I'm missing a library? Anybody know how I can fix this? I don't really have any experience with PostgreSQL
Re: Watchers not watching by Drew Pearce on Aug 4, 2009 at 11:54:15 am
That was the first thing I tried. Also tried running the whole thing as SU. It looks like it's finding the lib file in the symlink, but can't load it for some reason?
Re: Watchers not watching by Andy Gallagher on Aug 4, 2009 at 12:30:56 pm
Yes - I'm confused, as I don't recognise the error message; errno=62 corresponds to a "stream error" on my Linux system, so I don't think that's the problem here. Can you use ls -l to get information on the file concerned (/usr/lib/libpq.5.dylib) and post it back? Maybe it's trying to symlink somewhere that doesn't exist or something?
Re: Watchers not watching by Andy Gallagher on Aug 4, 2009 at 3:50:19 pm
Hi Drew,
Any joy with getting an ls -l on the library and posting it back? It might have got symlinked to the wrong place - it should reference /Library/blah/Final Cut Server/blah/lib.
Re: Watchers not watching by Andy Gallagher on Aug 5, 2009 at 8:29:07 pm
Hi Drew,
Sorry, I should have been a bit more clear! I'd like you to type in
ls -lh /usr/lib/libpq.5.dylib and to paste the result into the reply. This should show you the path that the library is linked to - it's possible it's somehow not linked to the actual location properly.
After you've done that, it would be a good idea to run
cp /usr/lib/libpq.5.dylib /Users/{your-username}, replacing {your-username} with your actual username of course! See if this gives an error, or not. If it doesn't, you can delete the libpq.5.dylib file that will have appeared in your home folder. If it does give an error it may be a bit more forthcoming as to what the problem is - post the result of that command here as well.
Re: Watchers not watching by Andy Gallagher on Aug 6, 2009 at 9:17:49 am
Hi Drew,
Yes it does! I think I made a mistake in the first line of code I sent to you.
Try the following (as root, obviously):
cd "/Library/Application Support/Final Cut Server/Final Cut Server.bundle/Contents/PostgreSQL/lib" [this is all one line]
for x in `ls`; do rm /usr/lib/$x; done
This will delete the old, non-working symlinks from /usr/lib. CAUTION - make sure you type it right!
Then (while still in the Postgres directory):
for x in `ls`; do ln -s "$PWD/$x" /usr/lib; done
This will re-create the symlinks, but will include the specific path to them ($PWD means "present working directory"). Then try running psql again - let me know if you have any luck!
Re: Watchers not watching by Drew Pearce on Aug 6, 2009 at 2:06:51 pm
Ok, I fixed the symlinks and was able to run psql.
The commands to modify the database were successful, but I'm still experiencing the problem of the watchers not doing anything.
