FORUMS: list search recent posts

iMacs and Fusion Drives

COW Forums : Apple Final Cut Pro X Debates

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
Howie Young
iMacs and Fusion Drives
on Aug 27, 2013 at 12:59:27 am

For those editors using a 2012-2013 iMac with a fusion drive, have you had any issues or problems with the fusion drive while editing with FCP X?

Is there a way to partition the SSD portion of the drive? Why would the SSD have to be partitioned?

Thanks.



Return to posts index

Marcus Moore
Re: iMacs and Fusion Drives
on Aug 27, 2013 at 1:14:16 am

The way a Fusion drive is set up is to give SSD speeds to your most used files and applications. The spinning disk portion is essentially there as an overflow for those files and applications that need to live on your MacHD, but are accessed so infrequently that they'd just be eating up space unnecessarily on your SSD.

In another year or two as SSD prices come down these will become redundant, but I figure Apple will get 3-4 years out of the technology.

Just like always, never run projects/events/media on this drive if you're looking for good performance.



Return to posts index

Howie Young
Re: iMacs and Fusion Drives
on Aug 27, 2013 at 1:24:04 am

Marcus,

Thanks for responding to my post. Is it worth it to spend the extra money and get the SSD drive even though it is smaller than the fusion drives?



Return to posts index


Marcus Moore
Re: iMacs and Fusion Drives
on Aug 27, 2013 at 1:32:28 am

That depends on how good you are about file organization. When I bought the final-generation 17"MBP, I ponied up $1,100 for the SSD drive. Very worth it.

That said, you should look at realistically what you need to keep on your system drive and buy accordingly.

I'll probably be able to save a bit of money on the new MacPro by NOT investing in a massive internal SSD. I doubt I'd buy anything bigger than a 750GB even if it was offered. As long as you have enough space for the OS, your applications, and the usual advised headroom, you don't need a ton of space. I keep everything else on external drives- Dropbox, iTunes Library, etc. If it's a work machine, try to keep the MacHD as clean and uncluttered as possible.



Return to posts index

Erik Lindahl
Re: iMacs and Fusion Drives
on Aug 27, 2013 at 5:37:01 am

I think as a concept the fusion drive could live for a long time.

2012-2013: 128GB SSD + 2-3TB HDD
2013-2014: 256GB SSD + 2-4TB HDD
2014-2015: 512GB SSD + 2-4TB HDD

And so forth. The larger the SSD-portion the more the HDD-portion becomes pure "archival" storeage. I think a few tech-savvy sites noted the 128 GB portion Apple uses today is a little on the low side.


Return to posts index

Rick Lang
Re: iMacs and Fusion Drives
on Aug 27, 2013 at 6:55:45 pm

Marcus:
"I'll probably be able to save a bit of money on the new MacPro by NOT investing in a massive internal SSD."

SSDs with conventional SATA 6G mounted externally as in a RAID connected to the host via Thunderbolt are going to become commonplace if and when prices fall and most people are working in greater than HD video.

In terms of the new Mac Pro and some other current computers, it would see that the option of having PCI-attached flash memory with even greater throughput than 6G SSDs, will be very attractive in some situations and put pressure on these computer manufactures to provide ever greater volumes. Hybrid SSD/HDD seem like a good idea now but will be a step toward completely removing internal spinning hard disks for high-end configurations shortly using PCI flash storage.

I think the trend to PCI flash will take off with the release of the Mac Pro if the machine is capable of satisfying high-end users. To do that requires a lit of third-party software and hardware vendors to work on the assumption that they will either provide software that runs very well on OpenCL/GL or provide hardware that can function well over Thunderbolt at some point. It is important for Intel and Apple to advance Thunderbolt quickly to get to 100 Gb/s to at least approach enabling PCI design specs which I understand can reach 128Gb/s at least in theory.

Edit: corrected references to Thunderbolt and PCIe GB/s to Gb/s.

Rick Lang

iMac 27” 2.8GHz i7 16GB


Return to posts index


Christian Schumacher
Re: iMacs and Fusion Drives
on Aug 27, 2013 at 8:43:29 pm

[Rick Lang] "SSDs with conventional SATA 6G mounted externally as in a RAID connected to the host via Thunderbolt are going to become commonplace if and when prices fall and most people are working in greater than HD video."

Thunderbolt is currently capped at 1000MB/s, so even if you pony up and go for the SSDs you're loosing money there as HDDs perform the same on TB RAIDs. Bad news is that apparently TB2 won't be much better than that.

http://www.barefeats.com/hard167.html

"Like all Thunderbolt devices, the transfer speed of 10 gigabit per second translates to something less than 1000MB/s -- even if you load it up with eight 6Gbps SSDs capable of a combined speed of 4000MB/s. And, as you can see, even with eight 6Gbps HDDs capable of a combined speed exceeding 1200MB/s, they don't come close."
(...)
"SSDs generally handle small random transactions much faster than HDDs. However, in the case of the ARC-8050, the aggregate random transfers per second is so close, one questions the need for solid state drives."


Return to posts index

Erik Lindahl
Re: iMacs and Fusion Drives
on Aug 27, 2013 at 9:21:28 pm

HDD vs SSD depends on what you're doing also. Random r/w will be vastly supperior on SSD and Tb2 has 2X the speed of Tb 1. At 1600-1800 Mb/s I have a hard time seeing that being a limit for storeage. Getting to 2GB/s requires a lot of HDD's


Return to posts index

Howie Young
Re: iMacs and Fusion Drives
on Aug 28, 2013 at 12:37:56 am

Hi Erik,

I will be editing a web series which will be shot partially on green screen. Segments will range from 3-7 minutes, some may run longer 10-15 minutes. I will also be editing some docs and short films.



Return to posts index


Christian Schumacher
Re: iMacs and Fusion Drives
on Aug 28, 2013 at 3:07:46 am

[Erik Lindahl] "HDD vs SSD depends on what you're doing also. Random r/w will be vastly supperior on SSD"

Sure, but in that testing when using 8 bays over Thunderbolt SSDs don't have any advantages over HDDs, did you see it? It's a waste of money.


Return to posts index

Erik Lindahl
Re: iMacs and Fusion Drives
on Aug 28, 2013 at 12:32:56 pm

True, but you could probably do a 2-3 disk SSD that has similar speed and fits in your pocket. 8 HDD's take a lot of space and use a lot of energy. We're talking some time into the future of course but SSD's will hit hard at some point.


Return to posts index

Christian Schumacher
Re: iMacs and Fusion Drives
on Aug 28, 2013 at 2:18:34 pm

Yes, in the context of using 1080 video on a pair of SSDs, Thunderbolt is definitely a good choice then. But I was originally referring to Rick's comments on "future" handling bigger than HD video and the use of RAIDs and SSDs over thunderbolt. Then I presented a recent test of a 8-bay RAID inclosure that shows Thunderbolt clipping at 1000MB/s. That should be relevant when discussing "bigger than HD", no? Fact is there's a lot of confusion regarding 4K and the new Mac Pro. It will support 4K panels for computer monitoring and UHD over TB as well, but I'm not sure if the new workstation will be the best choice for all purposes when handling 4K, as one will have to cope with its constrains, due to the lack of standard PCIe slots, which can reach much higher speeds on its turn. And the thing is, Thunderbolt 2 isn't capable of port multiplying either. So, I think Apple will have to prop up their software technologies in order to stay relevant in the content creation arena. Apparently that's what they're doing with Open CL and hopefully Macintoshes will stay around for a while. That was the part Rick has mentioned and I agree with. Gotta wait for new OS, new MP, and also, new Pro Apps.


Return to posts index


Jeremy Garchow
Re: iMacs and Fusion Drives
on Aug 28, 2013 at 5:18:57 pm

[Christian Schumacher] "Then I presented a recent test of a 8-bay RAID inclosure that shows Thunderbolt clipping at 1000MB/s. That should be relevant when discussing "bigger than HD", no?"

Again, it depends. If you're working on uncompressed 4k, perhaps. But who is doing that? Not many.

4k ProRes 4444 is 250MB/sec. 1GB/sec is plenty of overhead. it all depends on what you need to do. Thunderbolt is plenty for "bigger than HD" in most cases.

[Christian Schumacher] "And the thing is, Thunderbolt 2 isn't capable of port multiplying either. So, I think Apple will have to prop up their software technologies in order to stay relevant in the content creation arena."

No, but you can daisy chain, why would you need port-multiplication? I think that you have to take a really good look at Thunderbolt, realize that it will fit most content creation needs really really well at least for the foreseeable future. 4k broadcast is a long way off, 4k film editing is not super necessary. 4k uncompressed finishing is about the only practical reason I can think of where TBolt might be a constraint, as well as giant media pools.

The barefeats article mentions the top speed of 1,350MB/sec. I don't know anyone, personally, who would shy away from that and call it not good enough.


Return to posts index

Christian Schumacher
Re: iMacs and Fusion Drives
on Aug 28, 2013 at 6:03:33 pm

[Jeremy Garchow] "4k ProRes 4444 is 250MB/sec. 1GB/sec is plenty of overhead. it all depends on what you need to do. Thunderbolt is plenty for "bigger than HD" in most cases."

OK, but if anybody needs something like more than 6 streams and...But, wait! Who in hell does that, huh? Shoot a few hours of footage? None that I know of. Apply effects? That's only in Hollywood, sure! /end sarcasm/

[Jeremy Garchow] " why would you need port-multiplication?"

When the new workstation from Apple arrives, Thunderbolt is going to be the only way in MacOS to connect all of your computer monitors, video/audio gear and storage... And I don't think there's plenty of overhead there. That's why you're going to have a dual firepro. And that's also why Apple will step up their game in the software side.

[Jeremy Garchow] "
The barefeats article mentions the top speed of 1,350MB/sec. I don't know anyone, personally, who would shy away from that and call it not good enough."


I think that is when using two devices on two TB ports, which is out of the same TB controller, you can get to that but is shared with each device (meaning 675 for each of them) If you want a storage device in maximum speed over TB, is going to be clipped at less than 1000MB/s. That's why using an 8-bay enclosure filled with SSDs won't benefit TB users. On the other hand, 4-year technology from Apple can handle that, it is called a PCIe Mac Pro, and that was surely good enough to keep Apple at the spotlight in content creation. But it remains to be seen if the same will be applied to the new paradigm, software wise and hardware wise.


Return to posts index

Jeremy Garchow
Re: iMacs and Fusion Drives
on Aug 28, 2013 at 6:39:04 pm

[Christian Schumacher] "OK, but if anybody needs something like more than 6 streams and...But, wait! Who in hell does that, huh? Shoot a few hours of footage? None that I know of. Apply effects? That's only in Hollywood, sure! /end sarcasm/"

If you are talking about FCPX, it has an awesome Proxy system. There's even a multicam pref to create proxies as soon as you make a multicam clip.

And when you play six streams of 4k, it's not playing the full bandwidth 4k. You can check this on any computer with Activity Monitor. No sarcasm needed. Also, I don't know why you'd realistically edit a 6 stream multicam in 4k resolution.

[Christian Schumacher] "When the new workstation from Apple arrives, Thunderbolt is going to be the only way in MacOS to connect all of your computer monitors, video/audio gear and storage... And I don't think there's plenty of overhead there. That's why you're going to have a dual firepro. And that's also why Apple will step up their game in the software side."

And Thunderbolt will be plenty...I promise.

[Christian Schumacher] "I think that is when using two devices on two TB ports, which is out of the same TB controller, you can get to that but is shared with each device (meaning 675 for each of them) If you want a storage device in maximum speed over TB, is going to be clipped at less than 1000MB/s. That's why using an 8-bay enclosure filled with SSDs won't benefit TB users. On the other hand, 4-year technology from Apple can handle that, it is called a PCIe Mac Pro, and that was surely good enough to keep Apple at the spotlight in content creation. But it remains to be seen if the same will be applied to the new paradigm, software wise and hardware wise."

We are starting to go in circles.


Return to posts index


Erik Lindahl
Re: iMacs and Fusion Drives
on Aug 28, 2013 at 6:44:05 pm

Also all the figures mentioned here are reffering to Thunderbolt 1, the MacPro will have six ports of Thunderbolt 2. We know that each port will have at least 2X the bandwidth. We however don't know how / if the ports share bandwidth between one another in some way (I'd think not). So, connecting:

- 2X video displays to 2 ports
- 1X disk array running at 1500 MB/s or so to 1 port
- 1X video i/o to 1 port

…mot sure what else is needed? You still have 2 ports to spare on top of the native HDMI output.

The only thing TB2 will be limited in some way is when dealing with extreme processing cards and external GPU's. But there is a big might be there.

Not sure what all this has to do with iMacs and Fusion Drives either… :)


Return to posts index

Jeremy Garchow
Re: iMacs and Fusion Drives
on Aug 28, 2013 at 6:46:20 pm

[Erik Lindahl] "Not sure what all this has to do with iMacs and Fusion Drives either… :)"

As usual, it's my fault! :-D


Return to posts index

Walter Soyka
Re: iMacs and Fusion Drives
on Aug 28, 2013 at 6:55:59 pm

[Jeremy Garchow] "If you are talking about FCPX, it has an awesome Proxy system. There's even a multicam pref to create proxies as soon as you make a multicam clip. And when you play six streams of 4k, it's not playing the full bandwidth 4k. You can check this on any computer with Activity Monitor. No sarcasm needed. Also, I don't know why you'd realistically edit a 6 stream multicam in 4k resolution."

Multicam isn't the only scenario where you need multiple streams. A dissolve requires two streams. Any compositing operation requires at least two streams.

That said, I think six streams is probably plenty for most editorial needs.

I guess we have to wait for the hardware to ship, but TB2 looks promising in getting Thunderbolt RAIDs up to where the 8x PCIe RAID controllers are today.

In response to the larger point, I don't see HDDs going away any time soon. SDD has HDD beat on performance, but it has a long way to go before it catches up on price per capacity. Apple's Fusion drive is a very clever approach to gaining the advantages of both.

Walter Soyka
Principal & Designer at Keen Live
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
RenderBreak Blog - What I'm thinking when my workstation's thinking
Creative Cow Forum Host: Live & Stage Events


Return to posts index

Jeremy Garchow
Re: iMacs and Fusion Drives
on Aug 28, 2013 at 7:11:25 pm

[Walter Soyka] "Multicam isn't the only scenario where you need multiple streams. A dissolve requires two streams. Any compositing operation requires at least two streams."

It depends on the system, but all of this is really easy to track if you use Mac computers. You can open Activity Monitor and watch the Disk Activity. It will give you a real time graph of how much bandwidth is being pushed through the computer. Two streams of video does not necessarily require double the bandwidth. 6 streams of multicam certainly does not require 6 times the bandwidth. It requires more bandwidth than one stream, yes, but it is not 1:1.

Still, complaining about 1GB/sec being a hard limit seems a little far fetched.

1 streams of regular ProRes at 4k is 65 ish MB/sec. So even if full bandwidth is required, 390 MB/sec is the needed bandwidth. You could do 2 simultaneous 4k 6 stream ProRes multicam edits on one thunderbolt Raid that runs at 1GB/sec.


Return to posts index

Walter Soyka
Re: iMacs and Fusion Drives
on Aug 28, 2013 at 7:21:35 pm

[Jeremy Garchow] "Two streams of video does not necessarily require double the bandwidth. 6 streams of multicam certainly does not require 6 times the bandwidth. It requires more bandwidth than one stream, yes, but it is not 1:1."

Voodoo bandwithonomics?

Caching, proxies and render files lower bandwidth requirements, and IO may be bursty, throwing off sampled measurements, but math is a cruel mistress. How can you move two streams around a system within less bandwidth than the sum of their bandwidths?

There are loads of other factors like IOPS and latency, but that's left as an exercise to the reader.

Walter Soyka
Principal & Designer at Keen Live
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
RenderBreak Blog - What I'm thinking when my workstation's thinking
Creative Cow Forum Host: Live & Stage Events


Return to posts index

Jeremy Garchow
Re: iMacs and Fusion Drives
on Aug 28, 2013 at 7:27:58 pm

As mentioned, it depends on the system.

In FCP7, you have a choice of how you want things to playback (Dynamic RT, etc). FCPX has a portion of this.

When you playback 6 angles of multicam, it is not 6 full resolution streams of video. Other NLE's might work differently.

It's really easy to watch what is happening in Activity Monitor, but that requires a Mac. :-P

:)

Jeremy


Return to posts index

Erik Lindahl
Re: iMacs and Fusion Drives
on Aug 28, 2013 at 7:30:45 pm

One note here is that the i/o meter in the Activity Monitor isn't entirely accurate. I've measured faster than theoretically possible i/o over the network when judging from that app.

But I think if you are right. ProRes is a very scalable / dynamic format that will raster down when needed. In multicam you might only read every 4th pixel or something i.e. lowering the bandwidth a lot.


Return to posts index

Walter Soyka
Re: iMacs and Fusion Drives
on Aug 28, 2013 at 7:58:45 pm

[Jeremy Garchow] "As mentioned, it depends on the system. In FCP7, you have a choice of how you want things to playback (Dynamic RT, etc). FCPX has a portion of this. When you playback 6 angles of multicam, it is not 6 full resolution streams of video. Other NLE's might work differently."

We're not talking about the same thing.


[Jeremy Garchow] "It's really easy to watch what is happening in Activity Monitor, but that requires a Mac. :-P"

I've got a few lying around here somewhere... :)


[Erik Lindahl] "But I think if you are right. ProRes is a very scalable / dynamic format that will raster down when needed. In multicam you might only read every 4th pixel or something i.e. lowering the bandwidth a lot."

Working with scaled rasters can certainly lower bandwidth to/from the GPU, but resolution-on-decode scalability is generally a feature of wavelet-based compression, not DCT-based compression. I am really struggling to think of a way that this could work, but it's an interesting proposition.

Aren't we probably talking about automatically-created proxies?

Walter Soyka
Principal & Designer at Keen Live
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
RenderBreak Blog - What I'm thinking when my workstation's thinking
Creative Cow Forum Host: Live & Stage Events


Return to posts index

Walter Soyka
Re: iMacs and Fusion Drives
on Aug 28, 2013 at 7:22:49 pm

[Jeremy Garchow] "Still, complaining about 1GB/sec being a hard limit seems a little far fetched."

I should add that yes, 1 GB/s is a lot of bandwidth.

Walter Soyka
Principal & Designer at Keen Live
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
RenderBreak Blog - What I'm thinking when my workstation's thinking
Creative Cow Forum Host: Live & Stage Events


Return to posts index

Christian Schumacher
Re: iMacs and Fusion Drives
on Aug 28, 2013 at 7:51:49 pm

[Walter Soyka] " I should add that yes, 1 GB/s is a lot of bandwidth."

Nobody can say that Thunderbolt is not crazy fast bandwidth, it is. But no one should plain ignore that even faster performance is currently available on 4-5 year-old tech via PCIe. And all that isn't crazy expensive, elitist or unthinkable for current workflows - and these don't have to be comprised of 4K either. Yes Thunderbolt is more user friendly, with laptop proxy editing collaboration and all, but there was a compromise that was made for that to happen. Let's see what they're really giving us back later on. Stay tuned.


Return to posts index

Walter Soyka
Re: iMacs and Fusion Drives
on Aug 28, 2013 at 8:02:45 pm

Sorry, Howie, I know this all has nothing to do with any of your questions. Welcome to FCPX or Not: The Debate. If you haven't seen it yet, the FCPX Techniques forum [link] is a great place to discuss actually using the application!

[Christian Schumacher] "Nobody can say that Thunderbolt is not crazy fast bandwidth, it is. But no one should plain ignore that even faster performance is currently available on 4-5 year-old tech via PCIe. And all that isn't crazy expensive, elitist or unthinkable for current workflows - and these don't have to be comprised of 4K either. Yes Thunderbolt is more user friendly, with laptop proxy editing collaboration and all, but there was a compromise that was made for that to happen. Let's see what they're really giving us back later on. Stay tuned."

In other news, the keyboard port (USB) is all grown up, and USB 3.1 will be 10Gbps.

http://www.tomshardware.com/news/SuperSpeed-USB-3.0-Thunderbolt-10Gbps-Prom...

A couple good quotes:

"We're really focusing on USB 3.0 -- it's an excellent alternative to Thunderbolt," said Acer spokeswoman Ruth Rosene. "It's less expensive, offers comparable bandwidth, charging for devices such as mobile phones, and has a large installed base of accessories and peripherals."

Meanwhile, Intel seems to be supporting the new USB 3.1 spec despite its Thunderbolt efforts. "The industry has affirmed the strong demand for higher through-put, for user-connected peripherals and docks, by coming together to produce a quality SuperSpeed USB 10 Gbps specification," said Alex Peleg, Vice President, Intel Architecture Group. "Intel is fully committed to deliver on this request."

Walter Soyka
Principal & Designer at Keen Live
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
RenderBreak Blog - What I'm thinking when my workstation's thinking
Creative Cow Forum Host: Live & Stage Events


Return to posts index

Howie Young
Re: iMacs and Fusion Drives - FCPX Techniques forum
on Aug 29, 2013 at 11:16:48 pm

Hi Walter,

Thanks for partaking in this discussion.

I'd like to get some feedback on an iMac editing system in addition to the fusion drives. Do you suggest I post in the "FCPX Techniques forum?"



Return to posts index

Walter Soyka
Re: iMacs and Fusion Drives - FCPX Techniques forum
on Aug 30, 2013 at 1:45:34 pm

[Howie Young] "Hi Walter, Thanks for partaking in this discussion. I'd like to get some feedback on an iMac editing system in addition to the fusion drives. Do you suggest I post in the "FCPX Techniques forum?""

Yes, I do. That's a great forum for specific "how to best use FCPX" questions.

I like this forum for its broader perspective, but that's not necessarily best for a more specific question like yours. You asked about iMacs and fusion drives, and we ended up talking about Thunderbolt and Ae vs. Motion. Thanks for the jumping-off point, though!

Walter Soyka
Principal & Designer at Keen Live
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
RenderBreak Blog - What I'm thinking when my workstation's thinking
Creative Cow Forum Host: Live & Stage Events


Return to posts index

Howie Young
Re: iMacs and Fusion Drives - forum query
on Aug 30, 2013 at 1:58:56 pm

Walter,

Which would be the best Cow forum to post in to inquire about a new iMac editing system?

Thanks.



Return to posts index

Walter Soyka
Re: iMacs and Fusion Drives - forum query
on Aug 30, 2013 at 2:16:20 pm

[Howie Young] "Which would be the best Cow forum to post in to inquire about a new iMac editing system?"

I'd say it depends on the software you want to use, but most of the answers will be the same:

Buy the biggest, fastest iMac you can afford. Go for the i7 CPU if you can.

Load it up on RAM (which you can buy elsewhere like from Other World Computing to save money).

Get the beefiest GPU you can. (A 512 MB GPU will not accelerate Premiere Pro. More VRAM, more better.)

Some kind of fast internal storage (Fusion Drive or flash) will make the computer boot faster and make applications run faster. It will not necessarily improve render/export times.

You can edit off internal storage, and it will work fine, but you will probably want some kind of external Thunderbolt storage sooner or later.

Walter Soyka
Principal & Designer at Keen Live
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
RenderBreak Blog - What I'm thinking when my workstation's thinking
Creative Cow Forum Host: Live & Stage Events


Return to posts index

Howie Young
Re: iMacs and Fusion Drives - edit systems
on Aug 30, 2013 at 3:29:18 pm

Walter,

The software I will most likely edit with will be FCP X. Does that change the forum selection?

I thought the internal iMac system drive couldn't be used to edit from and store media footage. Doesn't media footage need to be kept on a separate external drive? Please explain.

What brand stand alone and RAID external Thunderbolt hard drives do you recommend?

Thanks!



Return to posts index

Walter Soyka
Re: iMacs and Fusion Drives - edit systems
on Aug 30, 2013 at 4:12:11 pm

[Howie Young] "I thought the internal iMac system drive couldn't be used to edit from and store media footage. Doesn't media footage need to be kept on a separate external drive? Please explain."

In the olden days of yore, you had to keep your media files on separate drives for several very good technical reasons (capacity, throughput, fragmentation, simultaneous access to media and system/application files, etc.). If you used your system disk, you were sure to suffer terrible performance, with dropped frames left and right.

A lot of these reasons have gone away by 2013, and it's not as verboten to use your system disk as it used to be.

That said, I still think it's a best practice to store media externally. It's too easy to run low on system disk space when you throw media & render files on the same drive. I also think there's an organizational benefit to external storage.

I am not the best person to advise you on Thunderbolt storage. If you were looking at workstations and wanted to talk about PCIe-based storage, I could be a lot more helpful.

I know the Promise R4/R6 series was the only RAID on the scene for a while. There are other Thunderbolt enclosures on the market, but as far as I know (again, that's not that far), the R6 is the one to beat.

Of course, you may not even NEED a RAID. ProRes uses only a reasonable amount of bandwidth and should work fine from a single spindle.

Walter Soyka
Principal & Designer at Keen Live
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
RenderBreak Blog - What I'm thinking when my workstation's thinking
Creative Cow Forum Host: Live & Stage Events


Return to posts index

Howie Young
Re: iMacs and Fusion Drives - edit systems
on Aug 30, 2013 at 4:18:59 pm

Walter,

Thanks again.



Return to posts index

Jeremy Garchow
Re: iMacs and Fusion Drives
on Aug 28, 2013 at 8:16:05 pm

[Christian Schumacher] "But no one should plain ignore that even faster performance is currently available on 4-5 year-old tech via PCIe."

I'm not ignoring it, I am weighing the pros and cons. Do you, personally, saturate an 8x PCIe bandwidth? Is it even possible?


Return to posts index

Christian Schumacher
Re: iMacs and Fusion Drives
on Aug 28, 2013 at 8:19:06 pm

[Jeremy Garchow] "Do you, personally, saturate an 8x PCIe bandwidth? Is it even possible?"

Apple seems to think so. TB2 is under way now. And Intel and all have plans to go optic and so on, why is that? Isn't 1,0GB enough?


Return to posts index

Jeremy Garchow
Re: iMacs and Fusion Drives
on Aug 28, 2013 at 8:37:46 pm

[Christian Schumacher] "Apple seems to think so. TB2 is under way now. And Intel and all have plans to go optic and so on, why is that? Isn't 1,0GB enough?"

For data and for now, I'm saying it is.

TBolt2 still has the same overall bandwidth as TB1, it is just aggregated differently.


Return to posts index

Walter Soyka
Re: iMacs and Fusion Drives
on Aug 28, 2013 at 8:57:20 pm

[Jeremy Garchow] "Do you, personally, saturate an 8x PCIe bandwidth? Is it even possible?"

The worst disk-bound render I have personally done was a pretty simple composite of a multipass 3D render, but it was a large format 1-minute long animation that weighed in at 972 GB. That's about 16 GB/s, or about 550 MB per frame. In my case, the main challenge was probably the image sequences, not a pure bandwidth issue, and if it hadn't been disk-bound, it would have been CPU-bound, but I'm not sure where that cross-over would have occurred. No big deal, because that's not an every day occurrence, but it's the worst real production bandwidth story I've got.

I can also totally saturate my PCIe SSD RAID thingie for Ae's disk cache with large raster projects in floating point, but I think that's only 4x PCIe and the bottleneck may be the RAID controller or the disks. This does kind of happen every day. I should switch it over to full-on flash storage and see if I can fix that.

Editors can comfortably ignore me, because 1 GB/s is a whole lot of data, but for corner cases like these... trust no external bus. The bandwidth is out there.

Walter Soyka
Principal & Designer at Keen Live
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
RenderBreak Blog - What I'm thinking when my workstation's thinking
Creative Cow Forum Host: Live & Stage Events


Return to posts index

Jeremy Garchow
Re: iMacs and Fusion Drives
on Aug 28, 2013 at 9:18:09 pm

[Walter Soyka] "The worst disk-bound render I have personally done was a pretty simple composite of a multipass 3D render, but it was a large format 1-minute long animation that weighed in at 972 GB. That's about 16 GB/s, or about 550 MB per frame. "

In what format? What on Earth needs 16GB/s and how would you realistically play that back?

[Walter Soyka] "I can also totally saturate my PCIe SSD RAID thingie for Ae's disk cache with large raster projects in floating point, but I think that's only 4x PCIe and the bottleneck may be the RAID controller or the disks. This does kind of happen every day. I should switch it over to full-on flash storage and see if I can fix that."

Meaning you need to playback 40Gb/sec or Ae is such a dog that it can't playback anything in realtime without writing it to cache? See what I did there?


Return to posts index

Erik Lindahl
Re: iMacs and Fusion Drives
on Aug 28, 2013 at 9:38:19 pm

When working in heavy 2D or 3D compositions you'll have multiple passes of each scene. This means you're working in, normally, 16 or 32bpp and X passes. It's not hard reaching the insane figures noted above since you're working lossless compressed at best.

1080p25 @ 16-bpp RGB we're talking over 300MB/s per pass. Jumping to 4K we're talking over 1.3 GB/s per pass. A normal scene can easily have 4-5 passes - i.e. saturating a few GB/s isn't all that hard even at HD resolution.

However, most often other things end up being the bottleneck. And these are quite extreme cases.


Return to posts index

Jeremy Garchow
Re: iMacs and Fusion Drives
on Aug 28, 2013 at 9:46:14 pm

[Erik Lindahl] "When working in heavy 2D or 3D compositions you'll have multiple passes of each scene. This means you're working in, normally, 16 or 32bpp and X passes. It's not hard reaching the insane figures noted above since you're working lossless compressed at best."

Sure, but are these played back in real time, typically?

[Erik Lindahl] "1080p25 @ 16-bpp RGB we're talking over 300MB/s per pass. Jumping to 4K we're talking over 1.3 GB/s per pass. A normal scene can easily have 4-5 passes - i.e. saturating a few GB/s isn't all that hard even at HD resolution. "

Right. As mentioned earlier, when working uncompressed, you will need the bandwidth, SSD will make a difference, and you most likely aren't working on a Mac, anyway.


Return to posts index

Erik Lindahl
Re: iMacs and Fusion Drives
on Aug 28, 2013 at 9:55:09 pm

I don't think you normally work in real time when doing these kinds of work no. Most comp-apps aren't built for it. However, i/o can be a limiting factor for sure even if you're not working in real time. AE for example has very aggressive caching to speed rendering / workflow up which frees the CPU but strains the I/O. If TB2 would be "limiting factor" I'm not sure about though. But seeing you have to potentially load X GB of frames before the CPU or GPU can process anything and some features requires temporal data... It can for sure be a limiting factor.

Going from internal HDDs to SSDs and a new external raid made a HUGE difference on my old 2008 MacPro even with limiting buss- and PCI-speeds. I/O matters but it depends on what you're doing and how much.


Return to posts index

Walter Soyka
Re: iMacs and Fusion Drives
on Aug 29, 2013 at 2:28:08 pm

[Jeremy Garchow] "In what format? What on Earth needs 16GB/s and how would you realistically play that back?"

What Erik said. It's not for realtime playback. By outputting multiple passes from a 3D app (lights separate from shadows separate from diffuse materials, depth maps, object buffers for isolating specific elements, etc.), you can exert an incredible degree of control over the look of the shot in post in a couple minutes or hours instead of hours, days, or weeks to re-render in 3D.

As I said, it was disk-bound -- CPU was sitting idle, waiting on frames to get read in from disk for processing. (I am sure this would not have been real-time even if I did have the bandwidth, but it was slowed by not having the bandwidth.)

It's a corner case, but I just wanted to illustrate that big bandwidth needs are not that far off.


[Jeremy Garchow] "Meaning you need to playback 40Gb/sec or Ae is such a dog that it can't playback anything in realtime without writing it to cache? See what I did there?"

You're a clever one!

Actually, the cache system is pretty brilliant. Reads from the cache are relatively fast, it's writes that are slow.

Consider a 4K comp, RGBA, floating point (32 bits per channel). That's 194 MB per frame. The Ae cache saves not only the final composite of all the layers in a comp, it actually saves out sub-renders of each layer. It'll write a frame to cache for anything that's faster to read than it is to calculate again, and it retrieves from the cache according to footage/frame and effects/properties. In other words, once cached, Ae can retrieve "frame 101 of clip A.mov with Blur applied with 1.5 radius" anywhere it's used in any project on the system.

When a frame is 194 MB, 1 GB/s is not enough to keep up. If you have a 10-layer 4K floating-point comp, that's about 2 GB per frame, or about 60 GB/s.

Walter Soyka
Principal & Designer at Keen Live
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
RenderBreak Blog - What I'm thinking when my workstation's thinking
Creative Cow Forum Host: Live & Stage Events


Return to posts index

Jeremy Garchow
Re: iMacs and Fusion Drives
on Aug 29, 2013 at 7:02:32 pm

[Walter Soyka] "What Erik said. It's not for realtime playback. By outputting multiple passes from a 3D app (lights separate from shadows separate from diffuse materials, depth maps, object buffers for isolating specific elements, etc.), you can exert an incredible degree of control over the look of the shot in post in a couple minutes or hours instead of hours, days, or weeks to re-render in 3D.

As I said, it was disk-bound -- CPU was sitting idle, waiting on frames to get read in from disk for processing. (I am sure this would not have been real-time even if I did have the bandwidth, but it was slowed by not having the bandwidth.)

It's a corner case, but I just wanted to illustrate that big bandwidth needs are not that far off."




I hear you, but we are speaking about completely separate processes. Sure, you need things to go as fast as they can go, but overall, it's still a slow process. Thunderbolt, perhaps, may drag you down a little, but it's not going to throw a wrench in the system, and I'm sure, for you, SSD would be worth the speed/money if you could can manage the decreased capacity.



[Walter Soyka] "Actually, the cache system is pretty brilliant. Reads from the cache are relatively fast, it's writes that are slow."

So why can't I play a simple "ken burns" in real time in Ae without burning it to cache? :-D

I know, I know, it's too smart, and my needs are too little. If I need 194MB/frame, I'm covered!

No really, I kid Ae. The global cache system is very good and seems to only get better. It is time for a performance enhancement to Ae, though.

Supposedly, it's in the works.


Return to posts index

Erik Lindahl
Re: iMacs and Fusion Drives
on Aug 29, 2013 at 7:15:12 pm

It's true that AE could use an enhanced engine but on the other hand I'd prefer it remains a useful tool in any circumstance and doesn't go from amazing real time goodness to unusable after you reach some breaking-point (yes, looking at you Motion!). And the mentioned frame-sizes would be the same in any app being relatively intelligent in it's rendering.


Return to posts index

Jeremy Garchow
Re: iMacs and Fusion Drives
on Aug 29, 2013 at 8:01:09 pm

[Erik Lindahl] "It's true that AE could use an enhanced engine but on the other hand I'd prefer it remains a useful tool in any circumstance and doesn't go from amazing real time goodness to unusable after you reach some breaking-point (yes, looking at you Motion!). And the mentioned frame-sizes would be the same in any app being relatively intelligent in it's rendering."

There's no question that Ae is the proverbial Jobsian truck.

Ae is the software that I have used the longest during my time in the video industry, even tough I am not a motion graphics designer.

There is no need for it to be Motion (I mean, there's Motion for that), very true, but there are days when it feels really creaky, despite all of the enhancements. See here:

http://forums.creativecow.net/readpost/378/2618


Return to posts index

Jeremy Garchow
Re: iMacs and Fusion Drives
on Aug 28, 2013 at 1:59:23 pm

[Christian Schumacher] "
Sure, but in that testing when using 8 bays over Thunderbolt SSDs don't have any advantages over HDDs, did you see it? It's a waste of money."


This is totally dependent on what you need to do, and you can skew the data to fit your particular needs.

There are very few people on this forum and around the world that need one computer to have a hard drive system that pulls 1GB/sec or more. ProRes at 1080 is some,thing like 20MB/sec, 4k is less than 70MB/sec. 4k ProRes 4444 is 225 MB/sec. If I'm editing 1080 ProRes, why do I need more than 1 GB/sec?

Where one would need 1+GB/sec would be in a shared storage situation where you need to be split the theoretical bandwidth.

With a theoretical 1GB/sec, you could thetertically have 4 people editing 4k 4444 ProRes off of one small and efficient raid.

For some, it's not a waste of money when you start looking at leveraging and scaling storage for more than one machine.

It is my feeling that looking and using fcpx in a shared environment, Apple is gearing fcpx for working in a shared environment, we just can't see it quite yet. People clown fcpx for its shared capabilities, I see a lot of potential.


Return to posts index

Christian Schumacher
Re: iMacs and Fusion Drives
on Aug 28, 2013 at 2:22:54 pm

[Jeremy Garchow] "For some, it's not a waste of money when you start looking at leveraging and scaling storage for more than one machine."

J, I was referencing the use of 8 SSDs in a RAID inclosure over TB. It IS a waste of money. This is the second enclosure I believe they tested and had the same conclusions. Is it some extreme corner case? An 8-bay RAID?


Return to posts index

Jeremy Garchow
Re: iMacs and Fusion Drives
on Aug 28, 2013 at 5:22:36 pm

[Christian Schumacher] "J, I was referencing the use of 8 SSDs in a RAID inclosure over TB. It IS a waste of money. This is the second enclosure I believe they tested and had the same conclusions. Is it some extreme corner case? An 8-bay RAID?"

You have to go Raid 0 HDD in order to get the same read (playback) speed as Raid5 SSD.

It's not necessarily a waste of money to everyone, is my point, especially when you start to look at connecting more than one computer to that very same storage and factor in Raid5 protection. You will want that raid to go as fast as possible and have the most protection when multiple people start drawing from it.


Return to posts index

Rick Lang
Re: iMacs and Fusion Drives
on Aug 27, 2013 at 10:27:32 pm

Thanks for the benchmark, Christian. Certainly interesting in that scenario there isn't a good benefit to justify the expense of SSDs. I think that test appears to be using 4K transfers over TB1 which may not represent the demands of working with video as I would think longer transfers could be used and does not show the performance using TB2, however that may not be a huge improvement. That's why I suggested the 100 Gb/s Thunderbolt may be necessary to take more advantage of the fastest external devices.

Rick Lang

iMac 27” 2.8GHz i7 16GB


Return to posts index

Howie Young
Re: iMacs and Fusion Drives
on Aug 28, 2013 at 12:35:03 am

Hi Christian,

Thanks for contributing to my post.

Would you advise against getting the SSD as the system drive in an iMac and opt for a fusion drive?



Return to posts index

Christian Schumacher
Re: iMacs and Fusion Drives
on Aug 28, 2013 at 3:22:03 am

[Howie Young] " Would you advise against getting the SSD as the system drive in an iMac and opt for a fusion drive?"

No, I would use the SSD as a boot, where you'd load the OS, apps and plugins and the HDD for Projects, Events and general media. I haven't done this, but I know it is possible to detach the fusion drive so they'll work as independent drives. Additionally, you have to have a RAID of some sort to house your video files (Thunderbolt being your best choice) and at least another big cheap USB one for Time Machine back ups. But if you can manage to fit the entirety of your video files into the HDD project drive, it's OK as long as you decouple the Fusion Drive as previously recommended. Remember that HDDs perform noticeably bad when they are more than 80% full.


Return to posts index

Howie Young
Re: iMacs and Fusion Drives
on Aug 28, 2013 at 12:35:53 am

What is the recommended headroom for an iMac system drive?

What type of applications do you have on your system drive?



Return to posts index

Mark Dobson
Re: iMacs and Fusion Drives
on Aug 27, 2013 at 8:05:25 am

[Howie Young] "For those editors using a 2012-2013 iMac with a fusion drive, have you had any issues or problems with the fusion drive while editing with FCP X?"

I've got the 1TB Fusion Drive and have had absolutely no problems since I bought it 9 months ago. I predominately use my iMac for editing with FCPX and the combination of the 3.4 GHz Intel Core i7 with it's fusion drive running in conjunction with with a GTech 8TB Thunderbolt Drive has given me a superb editing platform.

I'm editing full 1080P and rendering as I go with no major problems. Whilst I still have the occasional spinning ball crashe and use preference manager on a regular basis I put this down to the large number of Plug-ins I've got installed on the system.

It's hard to separate out what is giving me his great performance, the processor, the thunderbolt drive or the fusion drive?

My only concern is the size of the Fusion Drive, does one really need such a large Boot drive? I've been used to 320GB core drive and to keeping all my media on external drives.

After 9 months I'm up to almost 400 GBs on the drive. I've found that the more space I have , the less efficient I am at housekeeping.


Return to posts index

Howie Young
Re: iMacs and Fusion Drives
on Aug 28, 2013 at 12:41:39 am

Hi Mark,

Thanks for responding to my post.

How much RAM does your iMac have?

What kind of projects do you edit?

Are you using a G-Tech RAID or single hard drive for your media / scratch?

What kind of plug-ins do you use?

Are the plug-ins kept on the fusion drive or your external drive?

Do you plug-ins all run at the same time?



Return to posts index

Mark Dobson
Re: iMacs and Fusion Drives
on Aug 28, 2013 at 8:17:00 am

[Howie Young] "How much RAM does your iMac have? What kind of projects do you edit? Are you using a G-Tech RAID or single hard drive for your media / scratch? What kind of plug-ins do you use? Are the plug-ins kept on the fusion drive or your external drive? Do you plug-ins all run at the same time?"

I put 32Gb in, the max amount, and the 8GB memory the iMac came with are sitting on a shelf by my desk as the memory I bought came as 2 x 16GB. The specs for my iMac are: Processor 3.4 GHz Intel Core i7: Memory 32 GB 1600 MHz DDR3: Graphics NVIDIA GeForce GTX 680MX 2048 MB: Software OS X 10.8.4

I edit documentary style programmes shot with a Canon C300. I use FCP7's Log and transfer to transfer the files onto my scratch drive. I transfer them as native files rather than turning them into Pro-Res 422.

I don't then import them into FCPX, but simply refer to them. In fact apart from graphics and music I don't import anything into my Events.

I bought the Gtech 8GB unit. It is not a raid and I have read that one can get much faster speeds using a raid. I also use my older Gtech esata drives via a relatively cheap LaCie eSata Hub. These older Gteceh drives work really well and I will still be buying them for archival / back up purposes.

The other connecter I brought was the Matrox MXO2 Thunderbolt Adapter so that I can use an 32"external monitor. I have to say though that I've hardly used this at all as the screen on the iMac is so good. I have an additional 27" samsung computer monitor for the iMac so that when I am editing I can display my Event library on it's own screen.

I've got too many plugins to list them all. A lot of them came from this forum in the early days. But the main Plugins I use regularly are Core Melts Slice X and Lock & Load. I bought Magic Bullet Looks for FCPX but have hardly touched it as it is a pretty awkward program to use and not key frameable.

I actually think that the FCPX color adjustment tools are really good once you have got used to them and the lack of color wheels. Other Plugins I have got are White Balancer by Cineflare, Hawaiki Autograde, all of the excellent idustrial plugins and transitions. I've also got Irudis and Mflare and quite a few from Ripple.

Since shifting over to FCPX when it was launched I think I have probably spent well over $2,500 on various plugins etc. Some you can only use once because they are so identifiable.

But to be honest, having played around with a lot of plugins I often strip them out for the final edit and just do the best grade possible. Plugins are one way to enjoyably waste a lot of time!

All the plugins are kept on the Fusion drive in the Movies folder under Motion Templates apart from a couple which are actually installed as software programmes in the applications folder.

It's certainly possible to stack up a load of plugins and run them at the same time. The computer can easily cope with that although I tend to render as I go to actually see the finished result running smoothly at full resolution.

I bought my iMac because I was having such a miserable time with my 2008 MacPro which despite installing extra memory and a new Graphics card just didn't cope with full HD FCPX editing. I plan to buy the new MacPro when it is released this Autumn (maybe next month?) and there are also rumors of a new iMac coming out soon.

But I would highly recommend the current 27" IMac.


Return to posts index

John Davidson
Re: iMacs and Fusion Drives
on Aug 28, 2013 at 6:02:00 pm

If you go the SSD route, and maybe others can help me with my memory, but the SSD in the current iMac is not the same as the SSD in the new Macbook Air. I've never seen this kind of SSD performance like the air gives. I would want that performance in my iMac - but if the iMac doesn't offer that version of SSD yet, then I'd wait till it did.

John Davidson | President / Creative Director | Magic Feather Inc.


Return to posts index

Chris Murphy
Re: iMacs and Fusion Drives
on Aug 28, 2013 at 7:39:57 pm

"Fusion Drive" is a term like "Bootcamp" that encompases several different things, it's not a single thing or product. A Fusion Drive is made by leveraging CoreStorage, which is sorta like the logical volume manager (LVM2) in linux. CoreStorage is also used to implement full disk encryption.

Either a whole drive, or partition can be designated as a physical volume (PV), and its "space" added to a logical volume group (LVG) to aggregate storage from multiple PVs. From that LVG pool, one or more logical volumes (LV) can be created. So what Apple is doing at the factory is adding one large partition on each of the included SSD and HDDs, as PVs, into a LVG, and then creating a single LV. The LV is formatted HFSJ, so the file system thus extends across two physical devices. So can you do this yourself with your own SSD and HDD if you don't buy a "Fusion Drive" from Apple? Yes you can. And it can be done with non-Apple branded SSDs and HDDs. But it's non-obvious how to do this in Disk Utility, and honestly at this point I've found enough bugs in Disk Utility once going "off the rails" (it'll do simple formatting, partitioning, etc of a single disk, OK). But as soon as you start repartitioning, resizing, doing RAID setups, the utility is one of the worst UIs to ever come out of Apple, in my opinion. So I've resorted to using the CLI for this.

Much of this is in the coreStorage section of 'man diskutil' in Terminal.

The gotcha, of course, is if either the SSD or HDD dies, you lose the whole file system. In effect it's like the linear or concatenated RAID type. Don't count on being able to extract any information from the surviving drive. The other thing is that the migration of data from one PV to another, once it's successfully migrated, is removed from the other PV. Some may wonder what could happen if there were a kernel panic or power failure were to occur during this migration. Only upon successful copy and commit to the PV, is data then removed (technically deallocated) from the other PV.

Also, I'm pretty sure the migration is at an extent level, not a file level. I haven't read much about this one way or the other, but coreStorage is ignorant of the file system, it's simply a layer that maps a logical block device to one or more physical block devices, and at least with linux LVM this is done with extents (which are variable size, but default is a 4MB extent).

So if you wanted to intentionally over provision your SSD for wear leveling reasons, or you want a volume exclusively on the SSD or HDD, and a separate volume that's "fusion" between both drives, yes you can do that. But thus far I haven't figured out a way to do in the GUI and for other reasons (Bootcamp experiences) I don't trust Disk Utility aside from really simple tasks on single disks.


Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2017 CreativeCOW.net All Rights Reserved
[TOP]