FORUMS: list search recent posts

Good test speeds, low real-world speeds

COW Forums : RAID Set-Up

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
Charles Taylor
Good test speeds, low real-world speeds
on May 4, 2010 at 11:00:15 pm

Hi there,
Running OS X 10.6.2 on a Rev 1 Mac Pro. Just installed a 3-disk software RAID 0 for video editing and compositing. AJA disk utility shows it averaging about 300MB/s, and never dropping below about 150MB/s.

That's great for my purposes, and jibes with what I was expecting.

The problem arose when doing some compositing on 4k frames (~40MB TIFFs) last night. Except when disk flip-booking in FrameCycler, Activity Monitor never reported disk read speeds peaking above ~25MB/s.

Any thoughts as to what is up/what I could do to get better speeds when actually reading big frames off of the array?

Thanks in advance,
Charles


Return to posts index

Jordan Woods
Re: Good test speeds, low real-world speeds
on May 5, 2010 at 4:05:30 pm

4k frames on an internal 3disk RAID0? That's crazy talk. Download the AJA data rate calculator and look at what it takes to run uncompressed 4k.

answer: 500MB/s to 1.2GB/s depending on bit depth and misc settings.


4k is the new flavor of the month, but to work with it correctly you need some serious spinning disk, at least in our current technological state.

For compositing you can work with what you've got, just don't expect ANY playback, ever. Also important to note, if you lose a drive or cough a drive temporarily, all your mind numbing hour killing compositing is gone. When I used to do rotoscoping I never worked outside of RAID 3 or 5. I'm pretty sure if I lost a weeks worth of compositing because my raid decided to drop a drive I'd be on a ledge somewhere :)



Return to posts index

Bob Zelin
Re: Good test speeds, low real-world speeds
on May 6, 2010 at 12:16:37 am

and why do you expect to be able to playback 4K media with an array that peaks at 300MB/sec ?

And let me tell you - if you think that you are going to do the highest end compositing work, for a 4K finishing job, and do this without spending serious money, (even if you find a way), I will lead a mob to your home, and beat you senseless. A 3 disk RAID 0 is nice for doing ProRes422 work (and 422HQ, and 444) - no, it's not good enough for uncompressed HD, it's not good enough for 2K, and it is CERTAINLY not good enough for 4K work -

who are you - you are certainly not an amateur doing a 4K job ? You are certainly not a student, being asked to work with 4K files ? Where on earth do you come up with the thought that you can do a hi end 4K job without the required hardware ?

I would be more respectful, if you were doing a "charity job", but no one is working for charity in 4K.

bob Zelin




Return to posts index


Charles Taylor
Re: Good test speeds, low real-world speeds
on May 6, 2010 at 4:23:16 am

Hmm, I made an earlier post that outlined more exactly what I'm doing, but it seems not to have gone through.

In that post, I stated explicitly that I'm NOT trying to play back 4K files. I am well aware of the data rates needed to play back 4k, but all I'm doing is rendering. Disk speed is much less relevant when reading and writing the files is a small fraction of the render time.

Not to be snippy, let's try and assume we all know what we're doing here, and that the underlying workflow makes sense, and let's deal with the actual question, OK?

The question is why I'm never seeing read speeds above 20MB/s off of the RAID when Nuke has to pull in ~20 4K files to render a frame. I feel like I should be seeing a spike to 200 or 300 MB/s as it reads in the frames, but I'm not.

If anyone knows why that is, I would love to hear it.


Return to posts index

Jordan Woods
Re: Good test speeds, low real-world speeds
on May 6, 2010 at 4:44:23 pm

well... if your RAID is performing perfectly every time you benchmark it, or if you digitize to it and it nails the data rates perfectly... it's time to look at Nuke.

I know some applications depending on Quicktime as a "render to" option choke out and top at speeds of only 70MB/s even though they are sitting on a SAN capable of 4+GB/s

I don't know Nuke that well, but from what you are saying I would place my money that is your issue, not the RAID0 array. (granted you feel the RAID is performing optimal outside of this)

What tests have to done with Nuke to know that it absolutely should be performing above and beyond this?


-jw


Return to posts index

Jordan Woods
Re: Good test speeds, low real-world speeds
on May 7, 2010 at 4:45:23 am

I was going to post a block size question but then figured otherwise since with the AJA system you would have seen issues with large frame sizes if your block sizes are off.


Return to posts index


Bob Zelin
Re: Good test speeds, low real-world speeds
on May 6, 2010 at 10:10:51 pm

The question is why I'm never seeing read speeds above 20MB/s off of the RAID when Nuke has to pull in ~20 4K files to render a frame.

REPLY -
I too blame Nuke, without further info. You can easily download AJA System Test (http://www.aja.com) and test your 3 drive internal SATA array, and I bet you money that you are doing over 200MB/sec (and probably more). So why are you only seeing 20MB/sec - I can only blame the application, but you can answer this question yourself, once you run AJA System Test, and test your internal drive array.

Bob Zelin




Return to posts index

Charles Taylor
Re: Good test speeds, low real-world speeds
on May 6, 2010 at 11:57:23 pm

Yes, I completely agree that it is application-related - some apps (FrameCycler, AJA System Test) will get occasional drops to 150 MB/s, but usually around 300 MB/s with spikes to 400-500 MB/s.

Shake and Nuke both can only get the array up to 20-30 MB/s, even if they are not doing anything with the footage (ie. read it in, and display it).

Why is that? What can I do about it? I will post this in the Nuke forum, as well, but I'm wondering if different block sizes would help? I think I'm at 64 or 128, but will check when I'm at that machine again.


Return to posts index

Bob Zelin
Re: Good test speeds, low real-world speeds
on May 7, 2010 at 12:18:01 pm

Charles writes -
Why is that? What can I do about it?


REPLY - aah ! this is the ultimate question. I am aware of specific problems with Final Cut Pro in Log and Transfer that only occurs over an Apple AFP network, yet the Log and Transfer problem does not happen anywhere else - nor does Apple AFP have any issues with data transfer ANYWHERE ELSE except in Log and Transfer - SO, what can I do about it?

At least you have the ability to call Nuke. As for Apple owned Shake (and my issue with Apple) - you know what will happen - ABSOLUTELY NOTHING.

Bob Zelin




Return to posts index


James Wilson
Re: Good test speeds, low real-world speeds
on May 12, 2010 at 11:20:24 pm

Is it possible to try a couple of different block sizes?

That would sort out whether that was the issue pretty easily...


Return to posts index

Simon Blackledge
Re: Good test speeds, low real-world speeds
on May 20, 2010 at 7:31:39 pm

Try DPX and see what happens.

Nuke reading 20 files.. what compression is on the Tiffs?

Your raid may say 20MB's , but thats probably all Nuke can/needs to pull... as its loading 20 tiffs.


s



Return to posts index

Charles Taylor
Re: Good test speeds, low real-world speeds
on May 20, 2010 at 7:43:44 pm

The TIFFs were about 40 MB each. Compression - I'm not sure, and don't have the files any more.

---
Director of Photography
http://www.CharlesAngusTaylor.com


Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2017 CreativeCOW.net All Rights Reserved
[TOP]