shelf life of hard drives

[peter_nelson1]

I used to do all my backup and archiving on writable DVD's but they're taking

up too much space. I've already had to jump one size in my bank safe-deposit

box and my current one is nearly full. Last year I started doing HD video

and my still-photos are also getting bigger as I buy newer cameras. I'll

probably be buying a D300 soon - 12MP at 14 bits/pixel. <br><br>

 

My current frequent backup scheme involves external USB hard drives - a pair

of 320 GB Maxtor One-Touch drives that I swap between work and home, so one is

always offsite, and a pair of 250 GB WD Passports that I swap between my safe-

deposit box and home so one is always offsite. But I'd like to extend this so

I keep older backups for a deeper recovery - e.g., keep some HD backups for

several years or send one to my sister for safekeeping. I realize this is

going from "backup" to "archive", though it's a matter of opinion where the

line is drawn. But there's no denying that HD's are way more convenient than

DVD's because mine run automatically overnight. <br><br>

 

So my question is, is there any <b>NON-ANECDOTAL</b> data out there about

the "shelf life" of hard drives? There's a zillion posting on Photo.net or

the web with anecdotal experiences - "I stored _this or that brand_ of hard

drive for 6 months and it wouldn't spin-up/couldn't read the data/made a

burning smell/etc. Any consideration of "shelf life" should consider both

the mechanics as well as the data integrity.

 

Thanks in advance.

[bill_clark___minnetonka_mi]

<a href="http://www.pro-face.com/support/technical/00apr3.htm">Hard drive Life Info</A>

[bruce_watson1]

The main problem with storing data on a hard drive on a shelf is that it's still magnetic storage. Magnetic dipoles have a half-life. They make the half-life longer with improved magnetic coatings and head designs. They make it shorter by increasing bit density. So in the end the half-life stays about the same. Somewhere in the two - ten year range usually.

 

This in turn can be increased in software with extensive use of ECC. It's been 15 or 20 years since I studied this, but IIRC a block of data recorded on a DAT tape used to include something like 768 bits of data and 1024 bits of ECC. Enough to find and correct a 17 bit error, which made DAT tape storage quite reliable. Still, for backups, not archives.

 

Modern file systems for computers use little to no ECC, because the hard drives are not intended for either backup or archive use. They are considered on-line high-speed fairly short-term storage devices.

 

And therein lies your problem.

[Uhooru]

Also I've read that regular DVD' aren't really "long term" either. So what is a long term storage solution?

[peter_nelson1]

<i>OK<br>

 

I'll try again, gotta click html!<br>

 

Hard drive Life Info<br><br></i>

 

Thanks for the link, but I'm not sure how it addresses my shelf-life question.

[peter_nelson1]

<i>And therein lies your problem.</i>

<p>

Yes, I'm familiar with the theoretical problems involving decaying dipoles and lack of adequate error correction. And to those I would add that many anecdotal reports suggest the bearings and other drive mechanics degrade even faster than the data.<br><br>

What I'm looking for is something more concrete than anecdotal and theoretical discussions. Has anyone actually attempted to study the shelf life of hard drives?

In other words, if I had 10 WD 250G Passports and put them all in my safe-deposit box, how many would be readable in 6 months? All 10? Two? How about after a year?

[kelly_flanigan1]

Re <i>Has anyone actually attempted to study the shelf life of hard drives?</i><BR><BR>Yah; I worked in the disc drive industry in core; disc pack; mag tape; floppy; optical and the hda industry for over two decades; and was a memeber of the IEEE mag society.<BR><BR>You are asking how long a new widget will last; with no actual product history for the item. You might as well ask whos going to be Britneys next boyfriend next fall.<BR><BR>HDA's have lifes like people; some live along time; some die as infants; some die do to accidents; some lead sheltered lives and die early; others lead risky lives and life a long time.<BR><BR>

[Brad_]

You can't expect an answer that one can rely on from anecdotal internet postings.

[kelly_flanigan1]

Some head to media interfaces get stuck with time. Thus the heads tend to stick to the media when as time grows. A drive thats stored too long can have its heads stuck. Either the flexures get damaged; the spindle motor ownt start; or the heads rip off. Or they get tweaked abit add the heads fly abit wonky. This problem is typically increased with a worn disc coating; ones thats abit wrong; or just bad luck of the draw.

[peter_nelson1]

<i>You can't expect an answer that one can rely on from anecdotal internet postings.</i><br><br>

 

No kidding. But maybe someone knows about some actual research. Ask about printing paper and someone will quote from Wilhelm Imaging Research, for example. <br><br>

 

 

<i>Some head to media interfaces get stuck with time. Thus the heads tend to stick to the media when as time grows.</i><br><br>

 

I was under the impression that the heads never touch the media. Could someone clarify this?

[douglas_white1]

The problem with hard drives for archival is that if you don't spin them up every so often they will fail much quicker than if they are used every day. Movie studios are paying a fortune to keep their digitally scanned films on HDD backup systems and the hard drives have to be spun up once every week to keep them in working order.

 

DVDs are rated for around 20-40 years if kept properly stored, but they are subject to failing as well.

 

As always with this kind of media, a cool dry, dark place will help preserve the longevity, but there are no guarantees.

 

Your best bet is to stick with DVD for right now and burn new copies once every 3 years and destroy the old disks.

[kelly_flanigan1]

The heads touch the disc when the drive is not turning; ie off. When the disc rotates they drag along until the velocity is high enough that they "fly" a few microinches above the actual disc. <BR><BR>The disc has a protective carbon or other overcoat. The heads have typically two rails and a leading edge chamfer that allows forces air under the rails; and the slider/head floats above the disc. The head/slider is afixed to a gimbled flexure that is typically swaged to the rotary positioners arm. A tiny teflon tube runs from the IC head preamp along the positioner arm; to the flexure; to the slider. the slider has the read-write gap thats the business end of the recording circuit. <BR><BR>The sliders/heads do not touch the media when the discs are fully up to speed; ie typical operation. The same goes for the main journal bearing in your automobile; the journal doesnt contact the bearings once the engine is running. With an old worn out car or disc drive the RPM at which the journal/slider starts to fully float increases. This is because the bearing gets rougher. One can afix an ultrasonic probe and vary the rpm of a journal or hda disc; and hear the noise drop as the bearing full floats. With the disc drive is an air bearing. The preload is fixed; the "fly height" in microinches drops with thinner air; ie a hotter temp or a higher altitude. <BR><BR>In HDA's used in satelites and upper atmosphere stuff the HDA is inside a pressurised box or sphere; so the fly height doesnt vary with altitude. <BR><BR>

[douglas_white1]

The problem with hard drives for archival is that if you don't spin them up every so often they will fail much quicker than if they are used every day. Movie studios are paying a fortune to keep their digitally scanned films on HDD backup systems and the hard drives have to be spun up once every week to keep them in working order.

 

DVDs are rated for around 20-40 years if kept properly stored, but they are subject to failing as well.

 

As always with this kind of media, a cool dry, dark place will help preserve the longevity, but there are no guarantees.

 

Your best bet is to stick with DVD for right now and burn new copies once every 3 years and destroy the old disks.

 

If you want to study the best archival methods, research what the movie studios are doing. (Though admittedly in their case they tend to believe that making new copies of the films every so often is the most economical form of archival. In your case that would translate to Dual Layer DVDs (8.5 gigs) or Blu-ray disks (50 gigabytes).

[kelly_flanigan1]

Here I just leave all my HDAs always on. The oldest one still on 24/7 is a 850meg unit built in 1994; 13 to 14 years old. With some historical old 286's that have a 20meg HDA's on a long card built in the 1980's; I have to bump them abit to unstick the heads to get the disc pack to start. Bumping unsticks the stuck sliders; but risks tearing up the media too. <BR><BR>Todays disc drives typically all have fluid spindle bearings; not actual ball bearings. Like a car it probably not to wise to let HDA's off for too many months. Like a car engine each "start stop cycle" causes a very tiny bit of wear to the journal bearing. With contact lubrication before the journal/slider float; the *mystery eye of newt* boundary layer lubes come in place; this is where the pickle is; the mystery lubes; the finger crossing. <BR><BR> Asking the question if its good to store ones outboard or HDA for many years then starting it and going to full rpm without a hitch; without any bearing wear is abit of no mans land. <BR><BR> With time lubes dissappear; they squish out; they outgas; they evaporate; they wander; they corrode too. Parts tend to go metal to metal ; item A to item B with super long term storage. When in contact items often fuse locally; in small regions; or they corrode more. When started theses local things must give; leaving crap on the bearing surfaces; divots.<BR><BR>In weird cases many secondary failures result in long term storage of items with bearings; stuff that one normally doesnt model or test. <BR><BR>In one disc drive disaster long ago there was a mess of heads that got thru all QA; installed in drives; passing all tests. The sliders had a tad of contaminates in the slider fab/wafer process; and the slider's mag poles slowing grew worse due to corrosion. Read Write margins dropped; PW50 increased; output voltages in microvolts dropped on the sliders. This time bomb in the field took about 6 to 12 months to notice in the field; then the MTBF of these drives dropped like a stone and we got caught in a massive oh shit warranty recall; and alot of backpeddling and finger pointing and eating crow.<BR><BR>The MTBF of a population in the field drops with time as the group ages. At some point secondary failures can occur and the MTBF drops like a rock. Or if you are lucky they might go a decade plus. <BR><BR>

[kelly_flanigan1]

Leaving stuff off for a very long time and expecting a perfect non wearing start with no issues maybe a nice wish. Its often not the best thing for an fueled outboard; car; diesel engine; hda, camera shutter, focal plane shutter; lawnmower; TV; radio etc. Bearings dry out; lubes wander; cloth focal plane shutters get stuck together on their rollers leaving pinholes. Heads tend to get wrung on discs; valves tend to get stuck in old car transmissions; bearings tend to grab and spin; internal batteries leak; car valves get stuck; pistons rings corrode to cylinder walls; plastics creep and develop a set; some types of capicitors dry out. In a car AC the seal in the compressor may leak when its not used in along time. Better yet you may forget how to turn these devices on ; or how to use them !:)

[marc_rochkind]

What you want is storage, not a storage device. Devices, whether magnetic or optical, have limited theoretical lifetimes and are subjected to physical hazards.

 

To extend the life requires management which includes, among other things, backup and replacement of devices.

 

So, what you want is managed storage, such as is provided by Amazon's S3. True, you don't know that Amazon and its service will be around forever, but, in principle, this is the kind of managed storage (not storage device) that's the long term answer.

 

You could also manage your own storage, with a system of duplication and periodic verification and replacement. (E.g., verify DVDs against duplicate copies every 3 - 5 years and then re-write them.) But, is this something you want to do and are likely to continue doing?

 

(I recently saw an interview of Jay Maisel on a Luminous Landscape video journal in which he said that his Kodachromes are OK, but his old Ektachromes are mostly gone. Faded away. So, limited lifetime is not unique to digital. At least with digital it is possible to design a system with an infinite lifetime. Not so with film, because, unlike digital, there is only one original.)

[jim_horton]

Kelley,

Thanks for some very informative info on the HDA subject. I too have considered swapping out my 500gig drives and shelving them as archive, and it's interesting to know the pitfalls.

 

I can "anectdotaly" say that I recently set two new 500 gig drives to sleep mode, and within a month both failed--I now wonder if it has to do with your comments about the heads getting too comfortable parked on the media.

 

 

They were both Seagates (and each 'refurbished' drive I recieved from seagate under warranty also failed) so I'm also (partly toungue-in-cheek) wondering if the company you mention with the qc issues was Seagate/Maxtor?

 

 

To get back on topic, I currently use a separate machine, now with 4 sata 500 gigs (now Western Digital) constantly spinning as mid-term storage...with many fans for cooling and it sits in the basement to keep the noise away.

 

When time permits I've been burning to DVD. I may have to delegate this chore to one of my kids...perhaps trade for the lawn mowing duties...

--Jim

[don_e]

There may not be shelf-life data on storage devices, but there is on human beings. Check your sex and dob in the proper tables to find yours. After that date 'archiving' will only matter to your heirs if your work produces income for them. Otherwise they are landfill.

 

I'm a 'Stalinist' when it comes to purging digital images. If I can't imagine paying to have a good print made of an image , I see no reason to care much if the file gets corrupted and fails someday. The exceptions are documentary and personal photos whether I'd want a print or not. The rest might have got corrupted last night and I would not mourn for long.

 

This attitude saves me time, money, energy, and fretting. It also makes me a better critic of my own work.

[don_e]

I buy a new photo editing box every 3 or 4 years. Right now I'm copying the photo archive from the 2004 machine to the new one. If the bigger, faster new drives are flawed the odds are it will show during the days of copying. The old drives become a dated backup. So, my digital files get transferred to new hard drives every 3 or 4 years.

 

The Lightroom library containing all the good stuff gets redundant and incremental backup to several outputs including dvd and two off-LAN machines in two other states. I don't do more, figuring if those machines get fried at the same time, there are probably way more important issues at hand for us all than the condition of my photo backups.

[Brad_]

>>>> I can "anectdotaly" say that I recently set two new 500 gig drives to sleep mode,

and within a month both failed--I now wonder if it has to do with your comments about

the heads getting too comfortable parked on the media.

 

Yes it is anecdotal. I recently powered up a Mac IIci from being underneath my house in

the crawl space. I used that machine to do software development in 1991 - hasn't been

turned on since 1992. Was a computer I had forgotten about.

 

The machine and it's Quantum 100 MB hard disk came up instantly. It was fun seeing a

snapshot in time 16 years with all my files and application software intact. Though that's

an extreme, I've yet to have a computer/disk in storage not come to life fine after many

years. Similar stories with other colleagues.

 

I also have an IMSAI 8800 computer (S100 based - Intel 8080 cpu) sitting down there. For

grins I might power that one up someday to see if it's add-on 80 kb floppy drive (and 16

kb ram) still works and boots up...

 

But again, like other personal experiences, it's anecdotal at best. One can theorize on all

sorts of potentials...

[kelly_flanigan1]

In field service sometimes items with disc drives that are "stuck" get "fixed" during shipping due to UPS; or when folks carry the computer to a repair shop; or they just remove the drive from the computers case. A buddy of mine helps this old lady in his neighborhood with her computer woes. We carried her computer across street to his lab/gargage and it booted up just fine.

[kelly_flanigan1]

About a decade ago we had this old 286 unit that was always left on. I really didnt know what all was kept on this unit. It was suppose to be just a demo unit with no data.<BR><BR> It turns out that it had unbackedup data like inventory; phone lists; masters for printing forms. The employee had backed very little to 5 1/4 floppy; but had filled out the checklists that that had backedup all of it.<BR><BR> When I was on the other side of the country on business another employee turned off the old 286 to "save money". This 286 was primary used for demo-ing how to call into our ancient hard line BBS system; ie how to use terminal. <BR><BR>When I got back the 286 was off and the drive would not start; ie heads locked. After many attempts with slight taps I got the HDA to spin up; but now not all the DOS file structure could be seen. I had to manually using DOS remove what I could with the compromised HDA that was crashing; ie with sliders not fully flying. To make a long story short the "save money" event cost 2 man days of futzing; doing DOS command over and over; and not all data was recovered. The prior years inventory list on lotus 123 was lost; we had to use the year befores and add a mess of stuff to recreate the document. One would try the same command say move/copy over and over and it would fail; then by luck one would grab a few files. The backup discs were blank.<BR><BR> The computer was marked "dont turn off"; the warning ignored "to save money". If you have others doing any backups make sure they are really doing them; see if the backups are readable.

[bill_clark___minnetonka_mi]

If you'd like, peruse this site as they have a vital part they make for hard drives. It's

interesting. They are located in a community surrounded by farms in Western Minnesota.

Matter of fact their first building was a chicken coop!

 

Enjoy!

 

http://www.htch.com/

[peter_nelson1]

<i>So, what you want is managed storage, such as is provided by Amazon's S3. </i><br><br>

 

I don't have enough bandwidth for that. I do a lot of video, audio, and Photoshop editing. Currently I'm getting rid of the last of my film gear so I'm scanning in my favorite slides and negatives and this generates big TIFF files. So a typical day's work for me is in the gigabyte range, which would take too long to upload to S3 over my DSL connection on a daily basis.<br><br>

 

I run a backup automatically every night. My current backup set is about 140GB. This is just data - I don't back up OS or application sw. Granted, not all of that represents changes, but I want to be able to do a complete restore from every backup set so I don't have to worry about keeping two copies of everything - an original reference and a delta. This is also why DVD's don't represent a good option - it would take dozens of them and there's no easy way to automate backups onto multiple DVD's.<br><br>

 

Tape would be a possibility but tape drives are not readily available or well-standardized, whereas any portable USB HD can be read right out of the box by any PC without special drivers or software. With an offsite portable HD, in the event of a house fire or other disaster, I could plug it into any PC and immediately be data-recovered. The only way to do this with tape would be to offsite the media <b>AND</b> the drive (and whatever software the drive needed), since I can't just walk in to the average Best Buy or Staples and be sure of buying a tape drive that can read my tape media. But offsiting the tape drive introduces the same mechanical shelf life issues as offsiting a HD.(not to mention that I can't fit a tape drive in my safe-deposit box) <br><br>

 

So I don't see any practical alternative to HD's for both recent and deep backup ("deep backup" = going back several rev's, e.g., from several months ago). I've heard a lot of comments here along the same lines as I've heard elsewhere about drive hardware freezing up on the shelf, etc, "I had two drives freeze in a month" "I had a drive on the shelf for 10 years that didn't freeze" and all kinds of FOAF stories and personal experiences. But there's no way to guess how representative these are. For example, is the Hollywood practice of spinning up the drives every week an extreme precaution or a reasonable one? And does it work or are they still have extensive failures? <br><br>

 

<b>So</b> ... From this am I correct in assuming that even though companies all over the world use HD's for backups, including deep backups, of their valuable data, no one, as far as we know, has attempted to study the reliability of this systematically?

[Brad_]

>>> From this am I correct in assuming that even though companies all over the world use HD's

for backups, including deep backups, of their valuable data, no one, as far as we know, has

attempted to study the reliability of this systematically?

 

 

Don't know. You'll have to search. And then employ a great deal of skepticism on how the study was

done.

 

I've seen all the "net" chatter on potential HD lubricity issues, sticktion, bearing seisure, etc. As an

engineer you nod your head and say uh-huh, makes sense. But so far it hasn't come from a trusted

source but rather propagates through the net anecdotally. Many drive manufacturers post MTTF

numbers - perhaps useful for analyzing your own backup scenarios.

 

If you find a thorough definitive study that passes engineering muster, please post it here. Perhaps

NIST or Library of Congress has something interesting to say on the matter?

 

Personally, I use multiple large external FW800 disks. So far so good - that's all I can say for sure...