My Experience: DIY #5 Dual Xeon Dual-Core
I'm building a DIY #5 machine, and I thought I would report on it for other DIYers' benefit. This will be somewhat of a play-by-play, as I haven't finished building it yet. But I'll report back here as I feel it will help people.
Introduction and Background
This is my first time building a CPU from scratch, although I've tinkered with CPU innards and replaced RAM and drives several times before. I'm building the (mostly) Dual Xeon Dual-Core model described at http://www.videoguys.com/DIY5updateNAB07.html. This is the Top-of-the-Line "ultimate" model with 4 Gigs of RAM and a total of 4 processor cores, as you can see from this page. Pretty sweet!!!
We're actually building a hybrid between the Ultimate and the Best models. We're only doing SD, but we may upgrade to HD in the next few years, so we didn't feel the extra money for the lightning-fast Quadro graphics card or the Blue-Ray DVD burner was necessary at this time. So, we went with a Quadro FX 1500 recommended elsewhere by VideoGuys and for half the cost, and a Pioneer dual-layer DVD burner for 1/10 the cost. We put some of the money back into disk space, going for double the capacity on the RAID storage array.
Parts List and Pricing
Here is our parts list and the prices from Newegg.com:
1 x TYAN S2696A2NRF (SATA) Dual Socket 771 Intel 5000X Extended ATX Server Motherboard ($480)
2 x Intel Xeon 5130 Woodcrest 2.0GHz Socket 771 Active or 1U Processor Model BX805565130A ($342 each; $684 total)
4 x Kingston 1GB 240-Pin DDR2 FB-DIMM ECC Fully Buffered DDR2 667 (PC2 5300) ($80 each; $320 total)
1 x Western Digital Caviar SE16 WD2500KS 250GB 7200 RPM SATA 3.0Gb/s Hard Drive - OEM ($70)
2 x Western Digital Caviar RE2 WD5000YS 500GB 7200 RPM 16MB Cache SATA 3.0Gb/s Hard Drive - OEM ($150 each; $300 total)
1 x Thermaltake Eureka VC8000BWA Black Chassis : 1.0mm all aluminum Front bezel : Aluminum ATX Full Tower Computer Case ($150)
1 x Thermaltake Purepower RX W0142RU ATX12V 500W Cable Management / Active PFC / 14CM Fan / Triple +12V Rails Power Supply 115/230 V CE, FCC, UL, CUL, TUV, and BSMI certification ($100)
1 x PNY VCQFX1500-PCIE-PB-V Quadro FX1500 256MB 256-bit GDDR3 PCI Express x16 Video Card ($530)
1 x Pioneer 18X DVD±R DVD Burner Black E-IDE/ATAPI Model DVR-112D - OEM ($32)
1 x Microsoft Windows XP Professional With SP2B 1 Pack - OEM ($140)
Total (less tax and S&H): $2806
When developing our parts list, the first thing we asked was, "Where do VideoGuys find such low prices!?!" Long story short, get as much of it from Newegg.com as you can. Almost everything there is 10-20% cheaper than anywhere else. They had all of the components we needed. (Well, sort of. Keep reading.) And some of them even arrived the next day!
Note that when you search Newegg.com for the requisite Tyan motherboard, for some reason it never comes up. But when you search for Tyan motherboards without specifying the model number, and scroll through the list of results, it's there.
Trouble Getting the Right Parts
I've made some VERY IMPORTANT discoveries: The parts list for the Ultimate machine on the DIY #5 page has some inaccuracies. The Blue-Ray burner (Pioneer BDR-101A) actually costs about 10x as much as listed (so their total price for this machine should read something like $3400 not $2939), and the RAM is the INCORRECT MODEL NUMBER for that Tyan motherboard. It needs the FB-DIMM version, such as Kingston KVR667D2D8F5, rather than plain ol' DIMM as is listed.
Also, we couldn't find the DVD burner from the DIY #5 "Best" model that we were looking for. But we found another dual-layer burner from the same manufacturer for slightly cheaper, the Pioneer DVR-112D.
The DIY #5 Ultimate parts list also has some omissions that would have made things much easier for us and saved us on some restocking and return shipping fees. I ended up ordering the Xeon processors with Passive cooling because 1) the DIY parts list didn't specify Active vs. Passive, 2) I didn't realize the difference, and 3) Newegg.com was out of the Active-cooled ones. But when they arrived and I read the manuals, I discovered that Passive cooling is for rack-mount server installations with direct ducted airflow and that the Active-cooling model is for pedestal-type CPU installations like we're doing. We chose not to return these units, however, thinking that if things get too hot, active heat sinks are fairly cheap and we can go buy whatever we need. And I'm told the Xeon 5130's run fairly cool anyway, and the case is pretty spacious inside allowing for good airflow. (Yes, I could have researched more and figured out which one was needed, but if the DIY list had said to get the Active model, it also would have prevented this problem.)
We also weren't sure which "Thermaltake 500W" power supply to order, and ended up ordering the wrong one. The Tyan motherboard needs a supplemental 8-pin power connector, and the original one we bought had a 6-pin. I'm not sure how we could have figured this out without reading both the motherboard's and the power supply's instruction manuals before ordering, which would have required digging through the manufacturer's websites and hoping they had online versions of their manuals.
My initial reaction when the Thermaltake case arrived was, "This thing is HUGE!!!!!!!" It measures a whopping 25" deep, 22" tall, and 7.5" wide. Definitely the SUV of CPU cases. It's even wide enough that the hard drives are installed pointing toward the side of the case, rather than the long axis like I'm accustomed to.
Thus far, I've assembled everything I could, which includes the second try on the power supply. The second try on the RAM is on its way, so I unfortunately can't install any software yet. I may try powering it up just to see what happens, but I doubt I'll get very far without any RAM.
Some installation notes thus far: The metal I/O plate that comes with the case doesn't have holes in the proper places for this motherboard. Thankfully, the motherboard came with one that's specially made for it, although the manual didn't say anything about this except for having a picture of it in the parts list. It was a little tricky to replace because the original plate was wedged very tightly in the back panel of the case. I got it out with some careful prying with a flat-head screwdriver, and a good application of elbow grease. I'm not really sure how I managed to remove the original one intact and avoid mangling it.
Also, the brass-colored standoffs for the case's mobo-mounting tray needed some serious rearranging to fit this mobo. ("mobo" = motherboard) It was no problem, although I had to use the extra mounting plate that came with the case, and several additional standoffs than what came pre-installed on the tray.
Also, the case came with tons of extra screws of 4 different varieties, but there was no way to tell which screws to use for which purpose. The manual didn't say anything about this, and the pictures are so small it's hard to tell. I ended up using the small short screws with rounded heads to mount the mobo, the small short screws with hex-shaped heads to mount the DVD drive, and the long screws with rounded heads to mount the hard drives. (The purpose of the latter screws I'm sure of, but I still don't know if I got it "right" for the other screws. But I'm not sure if it matters 'cause the threads are the same.)
Conclusion (for now)
I think that's about it for now. Feel free to ask questions. I'm not sure what people will want to know. I still have some lingering questions, but I'm sure I'll figure things out soon enough when the RAM arrives and I power it up. For example, I'm not sure how to configure the storage drives into a RAID configuration, but I suspect it's in the BIOS. I'm also not sure if the storage drives require being plugged in in a special configuration on the mobo to achieve this. I'm also not completely sure if I did the processor and heat sink installation properly. Everything visually looks okay, but I really hope I didn't zap either of the processors and that the heat sinks are making proper contact with them.
To be continued....
Now for Part II of my DIY 5 Dual Xeon Dual-Core experience.
The second try on the RAM arrived and worked perfectly. I finished installing the hardware and booted it up for the first time. The fans spun, the lights came on, and nothing blew up or started smoking, so I must have done something right.
WinXP install and SATA
Then I reviewed the BIOS configuration options and attempted to install XP. I came a cross a hitch here. It turns out that the Windows XP Pro SP2 install routine doesn't recognize SATA drives. Some quick Googling revealed that I would have to manually load some drivers to make it work. I was shocked to find out that either I had to make a floppy disk with drivers (YES, a floppy! In this day and age!?!) or patch the CD itself. My mobo didn't come with any floppy disks, but the manual describes a way to make one using the driver CD. In light of this, I figured it would be much quicker to go the floppy route, even though it meant I had to go buy a floppy drive.
I must confess I'm not a big fan of Microsoft, and this ordeal is another reason. Just how long has SATA been around? I know SP2 is supposed to be only a patch release, but they sure gave Windows plenty of new features anyway. So why didn't Microsoft bother with updating the drivers on their installation disk to include SATA? Did they somehow think that people still use floppy drives these days? Did they think that nobody in their right mind would ever have a SATA drive as their system drive? Helooooo Microsoft. Which century are we in, again?
Note that I didn't include the floppy drive with the parts list in my previous posting. I don't use floppies normally, and I have no desire to keep the drive in this machine once it's finished. I'm planning to keep the drive in my closet "just in case", and for when I run into this problem with other machines I may build in the near future.
Once I had made the SATA driver floppy, the Windows installation worked just fine.
I haven't been able to benchmark things yet, but I want to share a little about its speed. The boot time takes a lot longer than I feel that it should. It's about 1 minute and 10 seconds, and that's before installing all the other software. When first powered up, it takes about 15 seconds before it sends a signal to the monitor. This is an eternity compared to all other machines I've used. And it takes another 30 seconds after that before Windows starts booting. And this after adjusting the BIOS settings for decreased boot time (such as disabling unused hardware controllers, the memory test, etc.) I'm not sure what gives, but my only guess is that this mobo is made for servers which aren't rebooted that often, and thus it isn't optimized for boot speed. Anyway, once Windows starts booting, it seems really fast and snappy.
Noise and Temperature
I could say a word about noise and temperature, too. The Quadro FX 1500 is extremely noisy when its fan is on high, which is the default until its drivers are loaded. At that point, the fan speed slows way down and appears to be based on the GPU's temperature. I'm very glad this is the case, because if it was that loud all the time, I was gonna have to return it and get something quieter.
The case's fans are very quiet. When the GPU fan isn't on full bore, this machine is hardly noticeable. At least, that is, when the windows in my room are open and it has to compete with chirping birds and city noises. It would be louder except I ended up with the Xeon's with passive cooling rather than active, so there are two fewer fans in the case than it would ordinarily have. I checked in the BIOS after it had been running for a while, and the processor temperatures were hovering between 5C and 10C below the maximum allowed for these processors. I'm not sure where it really should be, but that doesn't seem like a lot of headroom. I am strongly considering supplemental cooling options, especially because a hot Southern California summer is coming on real quick.
Ram Utilization Issue
I discovered that Windows only seems to be using 3 Gigs out of the 4 Gigs of RAM installed. I know it's a Windows thing because the BIOS sees all 4 Gigs just fine. At this point I still don't know what's going on, but Google told me a variety of things, some of it conflicting. I discovered the 4 Gig limit on memory addressing space for 32-bit OS's including Windows, and that the /3GB and/or /PAE options for C:/boot.ini might help, as well as reducing the size of my page file. I did these things, but nothing seems to have changed. I'm not sure if what Windows tells me in My Computer => Properties is the amount of RAM allocated for applications, or if it's for applications + the system. If the former, then there isn't any problem after all. I want to try unplugging 1 or 2 of the RAM chips just to see how this number changes.
Software Installation and Updates, and OS Tweaks
To squeeze out the maximum performance, I then applied the VideoGuys' XP performance tweaks at http://www.videoguys.com/WinXP.html, http://www.videoguys.com/TweaksWINXPVE.html, and http://www.videoguys.com/WinXP2.html. No problems here.
I also applied whatever driver updates I could find online, which is to say not much. Tyan has an updated BIOS available, but it's still in Beta so I decided not to go for it. I found and installed an updated driver for the Quadro. The only thing I've noticed since then is that there's no longer a warning about the second SLI port being unplugged every time it boots. Nice.
Then it was on to installing Adobe Production Bundle and my other software. I'm big into open-source and Free Software, so I installed OpenOffice, FireFox, GIMP, an (S)FTP client, dbPowerAmp, and some others. I left 50 Gigs on the system drive clear, so I could install Linux and make it a dual-boot. I may do this later today or tomorrow. Linux has some sweet (and free!) video transcoding tools that I haven't found good rivals for in Windows. I'm also curious just to see how a Linux installation fares on this class of machine. I'm thinking of installing Ubuntu or Kubuntu, although Gentoo is my most favorite.
That's all for now. For my next post, I hope to offer some benchmark information and share my first experiences running Premiere with this machine.
Are you ready for Part 3? Here goes...
RAID and Reinstall
When I first installed Windows, I went for the AHCI driver rather than the RAID one. I hadn't yet decided if I wanted to do RAID on the storage drives or not, but I knew for sure I wasn't going to RAID the system drive. So I selected the AHCI driver via the boot floppy I had had to make. But after the installation, when I decided to RAID the storage drives, I ran into a brick wall: Windows kept BSoD'ing early in the boot process whenever I had the SATA fake-RAID activated in the BIOS settings.
A note about RAID implementations: There are basically 2 kinds, hardware-based and software-based. Hardware-based is the only "real" RAID, and it's also very expensive. And I learned that just because a Mobo says it has RAID onboard, it doesn't mean it's hardware-based. There is such a thing as Fake RAID, which is actually software RAID that's based in the BIOS rather than in the OS, which is what my Mobo has.
Anyway, back to my situation. After a few hours of Googling and experimenting, I was unable to find a way to activate RAID without having to reinstall Windows and everything else. (I even tried a repair installation but it failed.) I think the crux of the issue is that activating RAID in the BIOS settings caused Windows to need the RAID driver during boot time instead of the AHCI one I had installed with, even though the system drive wasn't going to be part of a RAID array anyway. But I found it impossible to change that driver, either before or after changing the BIOS setting.
If I knew more of what I was doing in the Recovery Console and/or had more patience to create a driver floppy disk, maybe I could have accomplished it. But I was anxious to finish the installation and get working, so I decided a reload was quicker. Another 3-4 hours later and I had a working OS with all of my software, plus RAID. Cool!
I was surprised that Adobe Video Bundle didn't give me any troubles upon reinstalling and registering. However, with Windows I had to call Microsoft to convince them that I wasn't a software pirate and was rather just reloading my machine. (Why is is that honest people of the world are the ones who end up being inconvenienced or worse because others choose to be dishonest?) It was relatively painless, and I didn't even have to talk to a real live person. But while I understand why Microsoft feels the need to do this, I resent the whole "guilty until proven innocent" idea. I thought it was supposed to be the other way around.
Initial benchmark: disk write speed
Then I began to get my data situated. I had about 1.5 terabytes of Premiere projects and captured video scattered between 4 external drives plus a computer I was borrowing from a friend. About 175 Gigs were on that other computer, and they happened to be on a SATA drive. "Great!" I thought, "now's my chance to see how fast SATA really is." So I parked the other box next to mine and connected the drive directly to my machine using spare SATA Power and data cables.
It transferred all 175 Gig's in less than 40 minutes. I was expecting half of a day! Suh-weet!!!!!!
This led me to think about transfer speeds and where the bottlenecks are in computer systems. This led me to replace my PATA optical drive with a SATA one. I'm not sure how much it would help due to the speeds of optical drives. But at least it meant no ribbon cables inside my machine, which is good for ventilation.
But I ended up with ribbon cables after all, because I decided to install two 750 Gig PATA drives I had been intending to get enclosures for and to use as external drives.
I tried to RAID these drives too, but I found that RAID-1 (mirroring) is disabled in Windows XP. It requires Windows 2003 / Server edition to be activated. So fake-RAID is a good thing when running Windows XP: it seems to be the only way to do software-based RAID mirroring. But on my particular Mobo, this is only possible with SATA drives, not PATA.
So, as XP sees it, I now have 2.25 Tb of disk space: 250 Gig's for C:, 500 Gigs for the RAID-1 storage array, and two 750 Gig drives.
It turns out that this machine does have cooling issues. I'm pretty sure it's because of the heat sinks being passive rather than active. When things get hot, it gets quirky. It severely scuffed a DVD I was ripping one night. It had trouble finding a keyboard a while back. It spontaneously turned off once. But all of these happened when the room temperature was in the low- to mid-90's coupled with the processor load being high for a sustained period of time.
I've been running it with the side of the case removed, and with a small cooling fan sitting next to the machine pointed at the processors such that the air would deflect and leave the case easier. I've had no problems since then, even with the same room temperatures, and even with the external fan's speed set to 'low'. (It doesn't help that my office gets the most direct sunlight of any other room in the house, the house is very poorly insulated, and we're in a particularly hot Southern California summer.)
I am not sure how best to fix this problem, especially because we're already over budget on this machine. I see four options: 1) Buy some plain ol' active heat sinks. But I'm concerned that it might make the machine a lot noisier. 2) Get some active heat sinks that are intended for rack-mounted installations, where the fans are on the side of the heat sink, pointing parallel to the plane of the Mobo toward the exhaust fan on the back of the cause, hopefully augmenting its flow. 3) Do the second option and add homemade ducts, in effect isolating the cooling of the processors from the cooling of the rest of the case. 4) Go liquid-cooled. But with this, I'd be worried about things leaking. Plus, it's the most expensive option of the four.
I'm most in favor of option #3, but I haven't made a final decision yet.
Now for the part that you've been patiently waiting for: benchmarking.
I couldn't find any standardized benchmarking tests, so I just used a stopwatch and timed what I felt would give a good idea of real-time performance. All of these numbers are after installing XP and my preferred minimalistic software manifest, and after a dozen or two boot-ups and shutdowns and several days of getting real work done and finishing tweaking the settings. No Microsoft system updates have been applied (running OEM Service Pack 2b). All of the VideoGuys performance tweaks mentioned earlier in this thread have been applied.
Cold boot: 1:25 (from button-press to when XP busy cursor disappears)
0:22 - power up and Firmware boot (?)
0:13 - System BIOS boot (Phoenix splash screen to when AHCI/RAID controller starts booting)
0:18 - AHCI/RAID controller boot
0:32 - WinXP boot
Premiere Pro 2.0:
Startup: 0:16 (from pressing the icon until the Open/New Project dialog) This is really cool because my P4 laptop takes about 1:40 to do this.
The first video I chose was not too complex: a pre-prepared 4x3 AVI that I imported into a 16x9 project to add the black vertical bars. It took just over 8 minutes to do a combined render + export, so the speed was about 1.8x. A previous render+export did it in just over 7 minutes, about 2.0x. I'm not sure what caused the difference or why it was quicker to render plus export rather than just to render. The video had *not* been rendered before I started the export.
I tried with a much more complicated video too. It's meant to be a half-hour (actually 28:30) television program, and we didn't have a video switcher or use multi-camera mode to edit it. So it has 3 video tracks with heavy switching between them. It also has 2 audio tracks. It has serious color correction for probably 90% of it. A few transitions, mostly at the beginning and end. At 5% through the rendering process, the Estimated Time Left was hovering around 5.5 hours. I didn't have the patience to let it go to completion. However, this is about half of the time it took my Hyperthreaded Pentium 4 laptop with 1 Gig of RAM to render it. Definitely an improvement, although I still would have to let it run overnight.
When doing a pure export of the 14:30 video as previously rendered, it took 5:35 or about 2.6x.
A 6.3 Gig AVI took 1:43 to copy from the SATA-II RAID-1 array to the SATA-II system drive, so about 61 Megs/sec. As I mentioned earlier, I had previously copied about 175 Gigs in about 40 minutes, which comes to about 72 Megs/sec. Pretty fast! Transfers to/from the 750G PATA drives are miserably slow in comparison, just like on the other machines I use.
This machine is great! It's enabled me to actually get organized and do my video work a LOT faster than on my previous machine. It's generally tons faster than anything else I've had access to. I just wish 64-bit was mature enough so I could eek out its full capabilities. In time, I expect.
I do have some suggestions and updates for the VideoGuys' DIY5 parts lists, as described earlier in this thread. In particular, on the Dual-Core Dual Xeon machine they *really* need to update the pricing for that BlueRay burner and the model number for the RAM. The latter cost me $20 in restocking and return shipping fees, and the correct model number cost $10 more per stick than the incorrect model. It would also be really nice if they listed specific model numbers for the mobo, power supply, and processors. Lacking these also caused me some headaches.
I'd be happy to answer further questions, so lemme know. If/when I address the cooling issue, I'll try to report on it as well.
[Update: Fixed typo in disk speed section. 175 Gigs, not 200]
thanx for the heads up. I have updated the page to include the Ram you purchased and the BDR202 drive.
Cooling is always an issue with dual processor machines, especially with that big video card as well. Make sure you take the time to tie your cables and run them neatly. You will be amazed how much improved airflow you can get by this little tweak. You may also want to invest in active heat sinks.