Archiving images - external hard drive or DVD-R, or use both?

[david_jones18]

I am concerned about putting all my eggs in one basket...

 

I am tempted to use two 250 GB drives (one as master and one as a

backup) in which to archive my digital images.

 

I am concerned though as the media is magnetic which is prone to

changes over time. The time scale before errors become noticeable is

10 years. Is this a true and a valid concern?

 

As an alternative I could archive to DVD-R in addition, which seems to

be the way to go according to other posts.

 

The external hard drive is one the one hand very accessible and I'd

imagine that USB will be around for some time and if not then an

adaptor would be.

 

DVD-R might not be in the future.

 

Yours thoughts on this are appreciated,

 

Thanks

[ron_jonson]

I currently backup my files (digital images as well as other stuff) onto a couple external hard drives. It's probably the most convenient method and secure enough. I keep my working files on a drive in my computer, and dump everything to an external drive when I remember (which is not often enough). I'm upgrading to setup a RAID (redundant array of disks) on my system to add more data security for when I forget to backup for long stretches of time. I tried the DVD-R thing, but it was way too time consuming and used too many disks. Hard drives are cheap enough, where you can just have a few and cycle through them as they fill up or die.

[Ed_Ingold]

It's hard to justify the term "archive" in connection with hard drives. Hard drives can and do fail, and consequently lose a large number of images at one time. The magnetic images can be erased, accidently or otherwise, and tend to degrade with time and disuse.

 

The advantage of CD/DVD technology for archiving is that the risk of loss is distributed throughout many discs, the discs cannot be erased (RW discs should not be used) or the images altered, and the durability of the chemistry is well established. The greatest danger is physical damage (and obsolescence).

 

The cost of DVD's per gigabyte is about 1/3 that of hard drives. Use of a RAID improves reliability, but multiplies the cost of storage. I burned over 1800 Gb of images onto DVD's last year. Even with redundancy, the cost is far less than for hard drives, and discs are easier to catalog physically (i.e., sorted by project).

 

Perhaps you use the term DVD-R generically, but DVD+R discs are somewhat superior to DVD-R for data storage. In my experience, they are faster to record, and have a substantially lower error rate at the same recording speeds.

 

If you use recordable CD's or DVD's, it is important to establish a QA program - do a file compare after each save (part of the recording software), and check the quality of the recordings from time to time using a utility for that purpose (e.g., PlexTools from Plextor).

 

The greatest threat to long-term archiving is the evolution of technology which renders media obsolete in a surprisingly short length of time, along with the means to recover the data therein. Witness the demise of 8 and 5 inch floppy discs, that few new computers include a 5 inch floppy disc drive, it's rare to find cassette tapes, Beta video is dead, and CD's are becoming threatened in the music world.

 

The best, most durable archive of all may be prints rather than digital images. Scrap books have been around a long time, and as long as the images last (which can be very long indeed), there will be eyes to comprehend their content.

[working camera]

The best way to archive digital files is a tricky one.

 

DVD and CD technologies are not considered archival. It is recommended

that they be rewritten every 5 to 12 years depending on storage conditions.

 

My understanding is that hard drive storage is preferable although fallible.

 

Best advice is to use a combination both hard drive and DVD technologies.

 

C.

[swenson]

I use hard external hard drives. If you're using them for backup then you have at least two copies. I got my first hard drive back in '87, If my memory serves me correctly. A whooping 10 meg on 11 meg 286. I've never had a serious problem. The reliabilty has been getting better with each passing year. The drive on my current machine is 6 years old and haven't had any problems. So, I think that this talk that the hard drive will fail is way overstated.

 

Every 5-10 years just copy over to the newer, faster, bigger, cheaper drives. Machines will be able to read the older hard drives longer than the cd/dvds.

 

What I would suggest is to keep the back-up in a different location, to protect from fire, flood, etc.

[emre]

If you plan to migrate platform every 5-10 years, the most important archival consideration is to use lossless compression. OF course, hedge your bets by making copies and store them in different locations (e.g., home & work).

[patricklavoie]

both. good hard drive AND good DVD like the Matsui (i think) that use gold something for

100+ archival time. Separate location too, indeed : )

[mendel_leisk]

Consider _internal_ hard drives, too. Not without drawbacks, like any media, but cheaper than external, and (imho) less trouble prone.

[stephen_bay]

Use both. I try to have at least 3 copies of everything and at least one offsite. Also make sure that you turn on verification whenever you are making a backup. Using verification will take longer but it is well worth it.

[Ed_Ingold]

There is nothing to be gained by rewriting CD/DVD discs periodically. The rate of signal degradation is very low and well documented. The greatest danger for CD's is damage to the silver coating (the top side). In DVD's, the dye/reflective layer is sandwiched between two plastic discs, and is very well protected. With caveats regarding storage conditions, many CD/DVD discs are generally recognized as archival - some brands and formulations are better than others.

 

CD/DVD burners and software have greatly improved over the years. I have been burning CD's since they cost over $10 each (c1994). I have experienced two problems with older discs - the reflective layer has been scratched due to poor storage (e.g., paper sleeves rather than jewel cases or archival pages), but mostly due to poor recording quality. Using Plextor drives (the best in the industry, IMO) and their software, I have identified problems with recordings, interface cards (USB and FireWire) and media by various manufacturers (especially DVD's).

 

There is, however, a good reason to rewrite hard drives periodically. The magnetic domains gradually interract with time, degrading the s/n ratio. This bleed-through is readily heard on magnetic audio tapes, where the spacing (about 1 mil) is at least 10x the spacing between domains in an hard drive. When the drive is in use, data is continually refreshed to counteract this tendency. This restoration does not occur if the disk is placed on a shelf.

[stephen_bay]

Edward,

 

Do you know what is a reasonable time frame for rewriting hard drives? Is this something that should be done yearly? more or less frequently? does it make a difference if you read the file as opposed to write it?

 

I ask because have many back-up drives and files that I don't open or modify very frequently.

[__jon__]

I have two 300GB hard drives for storage. I have a small application on my G5 that checks

daily and syncs the two. I also have a small portable FireWire drive--I irredularly shuttle my

archive to a spare drive that I have in my G4 at work. I'm also burning DVDs.

[jbq]

The first rule is to have backups in two different physical locations (flood, fire, theft).

 

The two options you're looking at are not mutually exclusive (HD on-site plus DVDs off-site).

[Ed_Ingold]

Stephen, I don't have an answer for how to refresh disk drives other than to copy the files between disks. Most have some sort of maintenance cycle, but it's not published. Perhaps the manufacturer of your drive(s) has a recommendation. Using Norton Utilities to do a disk check (with repairs) might also do the trick.

 

I have over 20 drives, most in removeable carts which I use to store various audio, imaging and video projects until completion. This is short-term (under 1 year), and I usually back up to CD/DVD within a week, then follow up by archiving any new work after that point. With that many drives, I have a catastrophic failure every 18 months or so. Ya' wonder why I don't consider hard drives archival?

[alan_chan4]

I think 2 hard drives are okay. I would be surprised if you still use them in 5 years time, let alone 10. Just fill them up, then buy a pair of bigger ones and transfer the files to the new pairs when needed. No hard drives are expected to be useful after 10 years. I am certain 250GB would be considered "tiny" and useless in 2015.

[pbizarro]

I have read that some people are actually creating slides from digital files. Slides and negatives are well known for their archival properties, if properly stored.

 

When I plan on taking "important" photos, I shoot slide film. Then I scan and save the TIFs twice on different DVDs. If something goes wrong, I can always scan the slide again...

 

Thankfully, I do not need DLSRs to enjoy my photography, or to make money.

[mattb1]

Skip the DVDs, they are too prone to error. They have a much higher rate of failure than a good hard drive. Wouldn't you rather be printing or taking photos than trying to get a DVD to write correctly? And then hope you can actually read it in a few years?

 

Also skip prints and film as a way to archive, first they are second generation and second prints have a much smaller color and contast gamuts than your files. Also they are much more prone to damage than a good hard drive. All digital media will have a relatively short life span when talking about archiving. The key is to have a medium that is reliable over that life span and will allow you to quickly move the data to a new media. IMO Hard drives are the media that fullfill these qualifications.

 

Note that storage will be come cheaper per gigabyte quickly. The 300 gig drives will easily be replaced in a few years with drives that are two or three times the storage capacity and hopefully much faster.

[andy_piper2]

I work for a newspaper which has to archive a huge number of images digitally - our own pictures, wire-service pictures, etc.

 

I have had several long talks with our "technology editor" who is in charge of figuring out how to do this, in order to figure out how best to handle my own images. We use a triple-terabyte server (1 terabyte (1000 gigs) backed up twice, once off-site)

 

Her main points (some of which have been touched on here):

 

1. You MUST assume that you will have to migrate your files to new media every 5 years or so. Doesn't matter what you use for storage - you will need to do this anyway. Media formats become obsolescent (seen any 5.25" floppies recently?) - media capacity becomes obsolescent (used any 1.4 Mb floppies for pictures recently?) - media connectivity becomes obsolescent (Used any SCSI drives recently?). Physically, DVD, CD and magnetic disks will all have a pretty good 5-year survival rate, so that really isn't the issue.

 

As Eric(?) put it: "The greatest threat to long-term archiving is the evolution of technology which renders media obsolete in a surprisingly short length of time, along with the means to recover the data therein."

 

2. So - assume you will take, say, 250 Gigabytes of images over the next 5 years: 50 Gigs a year, which is 20,000 2.5Mb jpegs, but only 5000 10Mb RAW files or 2500 20Mb RAW files (minus whatever space is needed for "finished" full-res RGB images (10 megapixels x RGB x 16-bit = 60 Mb each).

 

3. Five years from now you will need to transfer that 250 Gigs (or more) to new media. Would you rather sit around feeding 60 DVDs for reading, and 5 new Super-DuperDVDs (50Mb capacity) for writing, into your computer?

 

Or would you rather point your brand-new 1 Terabyte hard drive at your old 250 Gig hard drive, say "copy all that" once, and then go have coffee or a nice Bordeaux or whatever while the single big transfer takes place?

 

In the meantime, you'll be working with those picture for 5 years. If you want a picture in 2010 from, say, Feb 16, 2007, is it easier to go find, pull out, load, and read a DVD from Feb 2007, or just open a Feb 2007 folder already on your hard drive (along with the other 60 months)?

 

For me (and I have 18 months worth of pictures archived on CDs) it's no contest.

 

I'm switching to hard drives (with two backups) so that I have every picture I've ever made immediately available at my fingertips; so that the backup can be automated ("Retrospect, copy everything on Drive 1 onto Drive 2 every other day"); and so that the inevitable migration to media will involve a single point-and-click operation.

 

As my technology editor put it "Think of removeable media of any kind as Post-it notes: great for communicating with other people, but not the medium you'd choose for your marriage license or anything else you want to keep safe and secure over the years."

[andy_piper2]

The "Eric" in the post above should, of course, be Edward (Ingold). S'cuse!

 

And I might add (everyone's experience will vary, naturally) that I have never had a sealed

(non-removeable media) hard drive lose data in 18 years of using them, and that includes

the one on my previous computer which has gone 7 years without a hiccup (4 Gigs in a

Mac G3, to show its age).

 

I HAVE had some CD-ROMs turn up as "unreadable" after just 4 years. Fortunately this

appears to be a reader problem rather than a media problem - but it still points up the

weak link in removeable media - by their nature they are unsealed and subject to - stuff -

getting inside onto the laser lens or drive heads and causing reading problems or head

crashes, even if the media itself is OK.

[alan_chan4]

I am with Andy. I have never understood why so many people are into CD/DVD burning. Piles and piles of CD is just too time consuming and expensive. Can anyone imagine of managing boxes of floppies now? Scary idea.

[rob_sronce]

For what its worth here is my backup workflow:

1. transfer to PC's main internal HD

2. burn to DVD (CD's are too small)

3. copy to backup HD

4. perform incremental back up of all HDs

 

I use an external HD (USB) to backup for two main reasons:

1. I can take (almost) all my images with me.

2. If your disk controller craps out you can still get to your images.

 

Yes I have lots of DVD's in binders with contact sheets but thats nothing compared with the countless bins of proofs and negatives!

[noreen]

On this subject, what external hard drives are folks using with good results? I need to

acquire one very soon (tomorrow, if possible) for temporary storage, before documents

(photos & text) are archived as hardcopies and on CDs. This will be augmenting an iBook

G4. It's much harder, I've discovered, to find reviews of HDs than digital cameras. :)