FORUMS: list search recent posts

What are the problems with archiving on external hard drives?

COW Forums : Archiving and Back-Up

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
tim gallaher
What are the problems with archiving on external hard drives?
on Sep 3, 2015 at 1:26:56 am
Last Edited By tim gallaher on Oct 14, 2015 at 9:07:54 pm

I am in charge of archiving uncompressed audio files (and a few video files) for a small yet international school. I archive off site using stored, duplicate 2TB external hard drives, plus a third set on site. I add new data to the hard drives three times a year and the drives just sit unused the rest of the time in our off-site data storage facility. We haven't had any problems with this in 7 years, yet I've read that external hard drives are dangerous for archiving and that data can just disappear. I'm not understanding this warning because 1) no data (that I know of) has disappeared on us so far, 2) we are archiving in triplicate so the chances that all the same data would disappear are low and 3) hard drives were, just a few years ago, considered the wisest choice for archiving.

My question is this: in the experience of those on this site, is archiving on external hard drives actually risky or are these risks trumped up by folks trying to sell LTO tape options?

I don't see this question in my searches on here, but if this is a duplicate, my apologies.

Thanks much



Return to posts index

Nathan Walters
Re: What are the problems of with archiving on external hard drives?
on Sep 3, 2015 at 2:37:57 am

I have personally never had any real issues with archiving to external hard drives. LTO tapes I'm sure are the much safer solution. With technology changing so often, LTO tapes could likely be used in a much longer time, while, with rapid changing technology, those external drives could become outdated and unusable on newer systems.

Nathan Walters
Halo Union Productions
http://www.HaloUnion.com


Return to posts index

Tim Jones
Re: What are the problems of with archiving on external hard drives?
on Sep 3, 2015 at 11:16:49 pm

I'm one of those who promote Tape (not just LTO) for long term archival. I've been in the storage industry for more than 30 years and I have a stack of disks that suffered failures within one or two years and tapes that we still perform test restores from dating back to 1988.

You are correct that disk was previously marketed as a safe archival technology for many years - by guess who - the disk drive manufacturers. I've been a disk drive manufacturer engineer and product manager (Connor Peripherals) and even then I knew that disk was a technology that was prone to failure simply because of the densities and mechanical operations involved. Back in 1993, I was involved in the development of a new technology named "RAID", we were making a box that contained 6 512MB 3.5" disks and a 2GB DAT tape dive. When we presented the box to upper management, we were instructed to "remove the DAT tape drive because it implies that the new RAID technology is unreliable." This was a very odd position since one side of Connor was promoting backup and archival solutions while the other was promoting disk storage. The marketing messages back then were already starting to contradict each other within the same company. It's not that you can't support both messages, just that the messages should be adjusted to allow for the pros and cons of both technologies.

I do agree that your use of 3 disk copies is very smart if you are planning to only use disk. However, if you are truly interested in long term retention (5 years+), you really should invest in a tape solution.

Tim
--
Tim Jones
CTO - TOLIS Group, Inc.
http://www.tolisgroup.com
BRU ... because it's the RESTORE that matters!


Return to posts index


tim gallaher
Re: What are the problems of with archiving on external hard drives?
on Sep 4, 2015 at 2:35:54 am

Thanks for your help, guys. This is useful information.

Cheers



Return to posts index

Tom Goldberg
Re: What are the problems of with archiving on external hard drives?
on Sep 4, 2015 at 4:44:05 pm

Several years ago, my former company (Cache-A) commissioned a white paper on recommendations for long term storage from a well known storage engineer. In particular it describes the potential hazards of long term storage on hard disk drives. The paper is now 4 years old, but the information is still valid. I thought I would post a link to it here for anyone on the Cow's information: Protecting Digital Assets for the Long Term.

I agree with Tim that your triple redundancy approach is very smart and should keep you covered if you remain diligent. That said, we had many customers over the years who encountered a much higher level of disk drive failures than it sounds like you are seeing, and many of them were not so careful to keep multiple copies.

Tom Goldberg
TGCS
30201 Rainbow Hill Rd.
Evergreen, CO 80439
mailto:tomgoldberg@gmail.com
http://tomgoldberg.net




Return to posts index

tim gallaher
Re: What are the problems of with archiving on external hard drives?
on Sep 8, 2015 at 9:52:13 pm

Thanks for this addition, Tom, especially the paper. I've started reading it.

As an aside to all of you: Do any of you have opinions about using the cloud as a primary archive? Security isn't a huge consideration here, btw.



Return to posts index


Tim Jones
Re: What are the problems of with archiving on external hard drives?
on Sep 9, 2015 at 1:19:37 am

[tim gallaher] "Do any of you have opinions about using the cloud as a primary archive?"

Simply put - not for my data.

The cloud is "okay" for data that you can afford to lose access to when the cloud provider folds or is hacked (DDoS on DropBox last week is an example). Also, for small data. Remember that even if you have 100Mb cable networking, that's still only 10MB/sec, or 36GB/hr. If you have a T1, your speed is only 1.44Mb/sec or 51.8MB/hr. Performing Cloud-based saving of data beyond a 100GB or so is really a tough situation unless you're talking about Amazon Glacier and a 10Gb pipeline.

If you do go that route, ask what the cloud vendor's network downtime recovery policy is. Do they FedEx or courier disks/DVD's with your data if you need to recover?

Tim
--
Tim Jones
CTO - TOLIS Group, Inc.
http://www.tolisgroup.com
BRU ... because it's the RESTORE that matters!


Return to posts index

Tom Goldberg
Re: What are the problems of with archiving on external hard drives?
on Sep 9, 2015 at 4:59:06 pm

Tim's points about archiving to the cloud and how long it takes to transfer large amounts of data are very valid, but maybe you have the time and/or bandwidth. Not only are transfer speeds an issue, cost is exorbitant as soon as you start to consider how much data you'd want to keep.

Take Amazon Glacier, which sounds cheap at only 1ยข/GB. Well, that's per month, so look at the cost of storing just one LTO-6 tape's worth of data for just one year: 2.5TB = 2500GB x .01 x 12 = $300 (that's almost 10x the cost of a tape that's good for 30 years)

If you really have much data to store for any length of time, these numbers really argue in favor of investing in an LTO based solution.

Tom Goldberg
TGCS
30201 Rainbow Hill Rd.
Evergreen, CO 80439
mailto:tomgoldberg@gmail.com
http://tomgoldberg.net




Return to posts index

tim gallaher
Re: What are the problems of with archiving on external hard drives?
on Sep 13, 2015 at 6:56:48 am

Thanks Guys. This is very helpful information. Much appreciated!



Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2017 CreativeCOW.net All Rights Reserved
[TOP]