I have FCS running on an Xserve and am trying without success to use another MacPro as a QuickCluster. It simply refuses to show up in the Compressor panel in FCS. Reason being I would like the XServe to be dedicated to search and browsing tasks and let the MacPro dedicated to transcoding, if it is possible.
The 8-core MacPro is set to QuickCluster with services under QMaster with 4 instances running.
Setting up a managed cluster, and qmaster problems by francois stark on Jul 15, 2008 at 4:06:07 pm
I found that it is easier to set up a managed cluster. I have done so many times.
In the qmaster control panel, stop qmaster, select "managed" in the compressor line, and start it again.
Then run Apple Qadministrator. You should see the mac pro's cluster but when you select it, there should be no services in the services tab.
There should be services listed at the bottom in the qmaster service browser. Select the mac pro and drag it up into the services tab. Open the triangle and make sure all the compressor tickboxes (one for each instance on the mac pro) are selected. Then click Apply, which updates the managed cluster in the mac pro to use all the instances.
Right-o.
Next, start up compressor and set up a compression test job. When you submit the job, you should be able to select a cluster: either "this computer" or the mac pro cluster.
The same in Final Cut Server - you should be able to choose the mac pro cluster in the preferences of the client's admin pane.
However, This is how it is supposed to work. I have found quite a bit of instability:
One system does not create a cluster controller. I have tried everything. It is a G5 quadcore. I have given up. It just does not show up in qadmin.
Some client computers don't appear to be "managed" in the qadministrator program, no matter what you try. I have 10 macs, three of them just won't be managed - they can only share qmaster jobs in an unmanaged fashion.
Others always only allow one less instance to be set up in qadministrator than you set up. Always. Set up 4 instances, you get three. Set up 8 in a mac pro, you only get 7 in qadmin.
AND then for the cicker. Be prepared to re-install compressor every few days after it's been working hard. Like doing a FCServer SAN scan of 6 TB footage. I have found that I have to run the ProApplicationsUpdate2008-01 every few days to re-install compressor after it stops working.
If you have as many problems with compressor, please tell apple about it here so they can fix it!
Re: Setting up a managed cluster, and qmaster problems by Zhang Junhao on Jul 16, 2008 at 4:53:36 am
Hi, Francois
For some strange reason, FCS v1.1 will only see QuickCluster and not managed ones. And even so, QuickClusters will sometimes disappear from the Compressor panel in FCS for no reason at all.
I have even experienced my QuickCluster (another MacPro) disappeared halfway through an FCS transcoding process causing loss of proxies and thumbnails as a result. :(
I have already reinstall/delete FCS and Compressor nodes countless times with no success.
Have you succeeded in having managed clusters showing up in FCS ? I would definitely like to know how you did that.
Re: Setting up a managed cluster, and qmaster problems by Francois Stark on Jul 16, 2008 at 3:47:04 pm
I don't know how! I just set up the managed cluster, made sure it works in compressor, and then set it up in FCServer. It just works. And fails. If you know what I mean.
I remember when I first started FCServer, I also tried a quickcluster. But I could not tell it which systems to use and which not, so it would grab all systems for the compression jobs. That turned out to be a problem when the systems were doing edit to tape. They would stutter and lose sync when asked to compress in the background... Besides that, it worked very much the same as managed clusters. When I created my first cluster, it just appeared as an option in FCServer. No problem.
Overall, the whole compressor-quemaster software group seems very unstable to me. When it works, it is very fast, but it bombs out in the middle of jobs and requires re-installation before it works again. Bad. Very bad.
Error handling is not good. If you submit a video compression job to a cluster it divides it among many render servers. If one of those servers freezes or it's compressord stops running, the complete job hangs, waiting for the system to keep on working. There is no timeout to re-submit that "stuck" segment to a working render server - at all. The whole job hangs, and so does all other jobs submitted after it. The only way around is to reboot the cluster controller and re-submit the job(s) from scratch. And the issue becomes much more complicated when FCServer resides on the same system, as in my case. At least you want to split the two. I don't have that luxury.
Sooo. We are in for a long, rocky ride. FCServer is only in version 1.1, and compressor also needs a lot of work.
Re: qmaster problems by Shane Sokolosky on Jul 17, 2008 at 10:00:29 pm
We've been able to setup a managed cluster of 32 cores with FCP server ver. 1.1 and things have worked while encoding 20TB of media to proxys, thumbnails, and webclips.
Do you guys have DNS setup for all the machines?
Shane Sokolosky
Consultant / Systems Engineer
XSAN for Video Apple Certified Technician
Apple Consultants Network - Storage Area Networks
Apple Developer Connection
Re: qmaster problems by Francois Stark on Jul 18, 2008 at 4:02:37 pm
Hi Shane.
You must have helped half the pro's in the world by now - I know you helped me before; I also met you at NAB in 2005 and 2008.
Yes. We have DNS set up. I can access E8-G5-2, the FCServer's webserver from another system by typing http://E8-G5-2.local/. Which means DNS resolves the name right.
Here's a specific problem: I have a G5 quad 2.5 that I would like to add to the managed cluster - it is named E1 G5. The cluster controller is the FCServer system: E8-G5-2. It runs Apple Qmaster as "Services and cluster controller". E8's cluster only has its own services included.
Then, on the E1 system, I run Apple Qadministrator. I want to add E1 to the E8 cluster. I also open the Qmaster control panel. Stop Sharing services. In the compressor line, I untick "Managed". Start sharing. And immediately E1 pops into the QMaster Service browser window in QAdministrator. However, the services are all unmanaged.
Back to QMaster control panel. Stop Sharing. E1 immediately vanishes from the QMaster service browser. Tick "Managed" in the compressor line. Start sharing. Nothing. It just does not appear in the Qmaster service browser when "managed" is on.
By the second time of trying to get this to work, QMaster freezes on starting up. Now I have to restart this system. To be continued.
qmaster problem by Francois Stark on Jul 18, 2008 at 4:13:59 pm
After a reboot, the QMaster shows the system is sharing compressor unmanaged. The I stopped sharing, and when I tried to start it up again, it freezes on "starting up".
Next step is to re-install the latest compressor update by running ProApplicationsUpdate2008-01.dmg. Repair disk permissions and reboot. To be continued...
Re: qmaster problem by Francois Stark on Jul 18, 2008 at 4:45:10 pm
Ok now qmaster has stopped the "starting up" freeze. I can stop and start it without problems. But this is repeatable - the only way E1 shows up in QAdministrator is if I share its compressor service as unmanaged. Whenever I share it as managed, it does not show up in QAdmin.
This is essentially the same system and the same installation as E8 G5. Both are quadcore G5's with the latest leopard and FCP installation. E1 has a AJA Kona LSe card, while E8 has no capture card, and it has FCServer installed on top.
Non of these differences should have any influence on Compressor and Qmaster / QAdmin.
Please tell me more about the successful compressor setup you run? What hardware? DNS? How many systems? How many cores each?
Re: qmaster problem by Shane Sokolosky on Jul 18, 2008 at 6:50:14 pm
Well I'll tell you guys, the only way I was able to get it to work was using the managed cluster, unmanaged had issues of not showing up, also the ones I've setup are on clean installs of leopard (OS X 5.4) and latest FCP and quicktime. I also made the FCP Server Server the controller. It sounds like there maybe something up with Qmaster on that one machine, the only thing I can think of is to reinstall Qmaster see if that helps.
Shane Sokolosky
Consultant / Systems Engineer
XSAN for Video Apple Certified Technician
Apple Consultants Network - Storage Area Networks
Apple Developer Connection
Re: qmaster problem by Francois Stark on Jul 21, 2008 at 10:02:13 pm
Hi Shane
Well I re-installed fresh 10.5.4, qt 7.5 and FCSrv 1.1 on our mac pro 8 core. And it is a thing of beauty. I'm compressing our 11 tb SAN's capture scratch folders through the night.
The mac pro is running a managed cluster with just 4 of its own cores. Pulling in media from five systems on our SANMP san over ethernet at up to 80MB/sec while compressing four streams to h.264. I found that if I give it 6 cores, the ethernet chokes, the most efficient for this setup is 4 cores. And it seems quite stable.
Update... QMaster/Compressor failed miserably by Francois Stark on Jul 22, 2008 at 6:00:29 am
So after one night I come back to work and it seems that the compression is done - the CPU's are not working... Then I open up system preferences and look at Qmaster.
Shock! Horror! Qmaster is not running. In fact, the "services" window is empty! If I hit "start sharing", I get the infamous "qmasterd not running" message. That means a full reinstall of compressor is needed. The "restart every 24 hrs" setting would not help one bit in this situation. FCServer, using compressor created 32GB of proxy files in about 6 hours before compressor failed.
So I stick to my first statement - QMaster/Compressor is not stable yet. I did a clean installation / upgrade on a brand new, otherwise completely stable mac pro, and this is what happens.
Re: Update... QMaster/Compressor failed miserably by Brent Bye on Jul 23, 2008 at 5:05:16 pm
In the interest of this thread, check you Compressor version.
It is known that the Compressor 3.0.3 update is breaking the auto Proxy generation feature of FCSvr 1.0 and 1.1.
Perhaps the QMaster issues reported here are related, perhaps not. Would be interesting to hear if anyone has reported success with Compressor 3.0.2 and not 3.0.3.
Re: Update... QMaster/Compressor failed miserably by Francois Stark on Jul 23, 2008 at 6:59:24 pm
Hi Brent
I would also like to hear of someone had success with compressor 3.02. Because I had problems with compressor before I downloaded the proapplicationsupdate 2. If I remember correctly proapplicationsupdate 1 took compressor up to 3.02 level, and it crashed in the same way the new compressor crashes.
Where did you hear that 3.03 caused the problems? Do you know of anybody with 3.02 success?
I re-installed FCP this morning after 3.03 crashed last night. Also did the update to 3.03 before I read this post. Tonight I started FCServer, and the update-scan caused compressor to break within 15 minutes. Again.
FCStudio 2 does not allow you to install compressor 3 by itself. It gets installed automatically with DVDStudioPro, FCPro, and other software. Not by itself. I suppose the easiest would be to re-install DVDSP without any of the presets or templates - this would install compressor as well.
