FORUMS: list search recent posts

RAM Cache

COW Forums : SAN - Storage Area Networks

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
marcus lyall
RAM Cache
on Jan 18, 2013 at 2:41:29 pm

Here's a hypothetical situation.
You have an ethernet-based shared storage system based on a RAID 6 chassis. Unix box. XFS.

Someone hits the hard reset button, initiating a less than graceful shutdown. So you lose the contents of the RAM cache, because it never got the chance to write the cache to disk.

How much data would you expect to be lost?
In other words, how often would you expect the RAM cache to be written to disk under it's own initiative?

Do most systems write the RAM cache back to disk on a schedule, or when it's reached a certain amount of data.

Just trying to understand for my hypothetical situation.

Thoughts from the hive?


Return to posts index

Josh Thomason
Re: RAM Cache
on Jan 18, 2013 at 5:31:42 pm

The best protection for you in a scenario like this. Is to use a RAID Controller Card with a battery back-up.

This is not a guaranteed saving-grace.

But, it will add some level of protection.

Josh Thomason
ProMAX Systems
949-861-2721 -direct


Return to posts index

Alex Gerulaitis
Re: RAM Cache
on Jan 18, 2013 at 7:58:06 pm

[marcus lyall] "How much data would you expect to be lost?"

With write-back caching:

Worst case scenario (somewhat unlikely): all of it, e.g. corrupted file system; the OS and FS didn't have a 2nd copy of the allocation table.

Best case: none.

On the average: no way to tell but most files systems do have a 2nd copy of the allocation table and if there's something iffy with the primary one, it'll get replaced; data loss will be contained but may still be significant as the contents of the cache may be spread over a large number of blocks, corrupting all of them.

With write-through caching: no data loss except in pending I/Os (same as with a directly-attached hard drive).

[marcus lyall] "In other words, how often would you expect the RAM cache to be written to disk under it's own initiative?"

I think it's load-dependent (i.e. "flush when there're no I/Os") although there still may be timers telling the cache to get flushed no matter what within a certain number of milliseconds.

(All this is from ancient researches years ago - most of it should still be applicable today, some may not.)

Alex Gerulaitis
Systems Engineer
DV411 - Los Angeles, CA


Return to posts index


Eric Hansen
Re: RAM Cache
on Jan 18, 2013 at 9:55:02 pm

hey Marcus

this doesn't answer your question directly, but i avoid this situation buy using battery backup units on all RAIDs and servers that are large enough to allow time to be shut down correctly. all the RAIDs have power switches on their backs, so it's pretty difficult for someone to turn one off accidentally.

if say, we lost power overnight while computers were rendering files to the main SAN, the battery backups are distributed in a way that the edit systems will lose power (thus killing their renders) before the SAN loses power. so the possibility of corrupted files is lessened (because all file writes to the SAN will have stopped).

i use ATTO cards that have "CacheAssure" which is ATTO's version of battery-less cache backup for the RAID controller.

e

Eric Hansen
Production Workflow Designer / Consultant / Colorist / DIT
http://www.erichansen.tv


Return to posts index

marcus lyall
Re: RAM Cache
on Jan 19, 2013 at 1:32:37 pm

So hypothetically, there is a UPS, but someone has decided to hit the reset button on the front.

And when the storage device comes back up, the last 48 hours worth of work is corrupted. Because, as it is explained, all of the allocation information for this data (or indeed the data itself) was stored on the RAM cache, and hasn't been written to disk for the last 48 hours.

So with these systems it's all about whether it doing write thru or write back? Imagining that one of these settings is more optimised for video playback. ie the one doing less write-backs.

But it seems like having a RAM cache that doesn't write itself to disk more than, say once a day, would be a fairly fragile scheme?



Return to posts index

Eric Hansen
Re: RAM Cache
on Jan 19, 2013 at 6:08:06 pm

I've mostly been installing Ethernet SANs based on SAS RAIDs using ATTO cards, so i haven't had to think about this for awhile. it would be great to hear from Steve over at Small Tree about what their Titanium does with caching.

The last Xsan system I installed used Promise Vtrak RAIDs, which had WriteThru and WriteBack. i grabbed the manual to refresh my memory.

from the Promise Manual:

WriteBack - Data is written first to the cache, then to the logical drive. Better Performance. VTrak as a cache backup battery to protect data in the cache from sudden power failure.
WriteThru - Data is written to the cache and logical drive at the same time. Safer.


once a month the cache batteries will recondition themselves. this forces the system into WriteThru mode until the batteries come back online. the Vtraks have dual controllers, and the caches will mirror each other in each controller in case one controller fails.

you can also specify the interval that the caches flush to the drives. between 1-12 seconds. i'm not in front of the RAID, so i'm not sure what the default is or what it's currently set to. so i guess the most would be 12 seconds, but the batteries in the cache last a few days.

e

Eric Hansen
Production Workflow Designer / Consultant / Colorist / DIT
http://www.erichansen.tv


Return to posts index


Eric Hansen
Re: RAM Cache
on Jan 19, 2013 at 6:13:22 pm

like Alex, i believe that the flushing would be load dependent. Promise is probably asking for the maximum interval between forced flushes. 12 seconds at 400MB/s would be 4.8GB. there's no way the caches are that big. at full tilt, it's probably flushing almost instantaneously. whereas when there's nothing going on, it will flush every 12 seconds no matter what.

e

Eric Hansen
Production Workflow Designer / Consultant / Colorist / DIT
http://www.erichansen.tv


Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2017 CreativeCOW.net All Rights Reserved
[TOP]