Best (safest) digital storage

I have recently discovered that a number of images I have scanned

since 2000 are corrupted on my hard drive. The original scans (early

files mostly on Photo CD's) still appear to be intact so I can restore

them on my hard drive (and back up external drives); but the basic

issue remains: what's the safest way to keep digital files?

 

I am not sure when or how the files became corrupted, but I did have a

technician move files when he installed a larger hard drive. I suspect

he did not use a backup program that checks the integrity of copies.

That may have introduced the errors; or I may have inadvertently

introduced them some way although I routinely use a back up program

(currently Retrospect). I assume the backups I make are identical to

the originals, although I don't know for sure.

 

So, I have several questions:

 

1. What's the safest backup routine?

 

2. What's the longest-lasting medium? (I assume high quality CD's last

about ten years)

 

3. Are hard drives a good medium to use as storage (with a good backup

program)?

 

Thank you,

 

--Gib

Gib, I believe the safest system is in multiples.

I backup to CD,DVD and firewire external drives.

Nothing in single form is fail proof that is known for sure.

 

Take Care, Pete

I back up to an external Firewire drive, and I periodically burn my photos to DVD, which

get sent offsite.

my hard-drive crashes on a regular basis, so I wouldn't recomend archiving any important photos on hard-drives alone. Singapore National Library asked the same question a few years back and the safest (and least likely to become obsolete) storage media for them was on film (microfiche) and not any of the digital format!

Multiple hard drives + MAM-E gold CD-R or DVD

<< what's the safest way to keep digital files? >>

 

The short answer is no one knows.

 

The longer answer is: redundancy redundancy redundancy.

 

External HD's that are disconnected and unpowered between backups, CDs or DVDs, off-site internet-based storage, and maybe even tape. (I do all of the above except tape.)

redundancy redundancy redundancy = RAID 5

(1) The safest backup routine is anything that double-checks whether the original files and the copied files are identical.

 

(2) I feel it's the wrong question to ask. This is digital world -- copies are perfect. Thus you should not think about putting your stuff somewhere forever, but rather putting your stuff somewhere where it can be easily copied on as-needed basis.

 

I would strongly argue that hard-drive storage (with multiple backups, some offsite) is the best way to go about it, provided once in ten years or so you just copy all your files to a new set of hard drives.

 

(3) Yes, hard drives (note: multiple, redundant hard drives) are a good storage medium. No, RAID is not enough. RAID and backup are different.

Second most responses here:

 

1. A good backup system is one that uses as many different media in as many different locations as possible.

 

2. Any one system *will* fail -- CD, DVD, HD, Tape -- when you need it most.

 

3. A good fire (not so much flood) will destroy your backups if they're all in the same place. I keep some backup DVDs from my home system at the office and was thinking of posting a pack to my parents in Canada. Since that's a different continent I figure it's the most remote I can get short of worry about nuclear war. :P

 

4. Don't use re-writable CDs and DVDs -- not only are they more expensive, but they're more error-prone than the write-once disks. Besides, it never hurts to have multiple copies of the same file.

 

5. Don't confuse RAID5 with backup. It *will* help to prevent errors but it's not the same as actual backup.

 

HTH,

 

jon

I have triple redundancy on my computer at home. I make copies of all of my files to my C,E, and F hard drives. My E hard drive is a RAID5 array. I then copy them all again periodically (once each month or so) to multiple DVD's and take offsite and load them to another computer. As mentioned above, ALWAYS have an offsite backup in case of fire. There are also many companys that offer network storage for a fee. Virtual Tape drive comes to mind.

> redundancy

 

> once in ten years or so you just copy all your files to a new set of hard drives

 

Won't protect you from having redundant copies of a corrupt master file. You need a check for unwanted file changes, too.

 

> redundancy = RAID 5

 

One fried power-supply = everything gone

 

One dying RAID-controller = contents corrupt = everything gone

 

...

 

Andreas

So far only the tapes considered to be safe for long term data storage. Hoever for the better safety every backup should be doubled. In our practice our backup ends a "backup copy" session which copies and checks the actual backup tapes. The copy ones then moved offsite storage and kept for years just like the local ones.

I don't know if tapes checked reguraly but should. A simple read check would suffice I think.

once in ten years or so you just copy all your files to a new set of hard drives - is that how long an external hard drive should last? Im in the process of copying cd/dvd to one and was wondering about life span

Babette, the lifespan of hard drives is a matter of statistics. On average they may last well over ten years, but you can always win the lottery with your first try...

 

Out of three external hard drives I own, one died about eight months after purchase (mechanical failure).

 

Andreas

Thank you all. This thread has been very helpful.

 

Some points I will surely keep in mind:

 

1. FILE CHECKING SOFTWARE

 

> Won't protect you from having redundant copies of a corrupt master file. You need a check for unwanted file changes, too.<

 

Error checking. It's crucial. I HAVE redundant copies of some corrupt files. I think the corruptions were introduced because a technician, in the process of changing out a hard drive, made copies without using an error checking program. I DO ASSUME THAT USING A RELIABLE BACKUP PROGRAM (LIKE RETROSPECT) WILL GUARANTEE IDENTICAL COPIES.

 

2. MULTIPLE MEDIA.

This has been my salvation so far. I have the original CD's which have lasted 5 years without apparent corruption. I will now add DVD copies.

 

 

3. MULTIPLE COPIES

This is part of my current routine. I keep three copies (one off site) on hard drives in addition to the original CD's. I will continue that practice.

 

4. DON'T ASSUME EVERYTHING IS OK.

I did not check through my original files carefully for two years. I will now figure out a routine for checking on the viability of image files. Till now I have not checked all of the files. I have usually just checked the thumbnails. Thumbnails may be viable while the complete file may be corrupt.

 

 

5. NO GUARANTEES. NOTHING'S PERFECT. DO EVERYTHING YOU CAN.

 

<< what's the safest way to keep digital files? >>.

> The short answer is no one knows.<

 

External HD's that are disconnected and unpowered between backups, CDs or DVDs, off-site internet-based storage, and maybe even tape. (I do all of the above except tape.)

My Routine

 

1) Copy CF Card to HDD

 

2) Tag Images with id keywords (EventName/SubjectName/InterestingInfo) [EX: SDAirShow-BlueAngels-InFlight]

 

3) Copy files to 2nd HDD on main machine

 

4) Copy files to all 3 HDDs on server (Poor mans RAID, ALT-C,ALT-V on each HDD.)

 

4) 2 Copies to CDR, Written at 1/2 media max speed or 1/2 drive maxspeed, wichever is slower. Also included on disk are PARity (PAR) files using 10% of the disk space for recovery should a portion of the disk be unreadable. Of course the disks are verified at time of writing.

 

5) 1 set of CDRs into the emergency bag (Earthquake,Flood,fire,boogieman, grab this bag on the way out the front door.)

 

6) 2nd set of CDRs go offsite.

 

7) Thumbnail process.

 

8) Start working on fullsize images (RAW Conversion, Color, Curves, pre-sharpening,etc.)

 

9) Finishing work for a session gets any files changed archived to 2nd hdd, server HDDs, and two sets of cdr.

 

I'm covered against user errors [Delete key is so close to Insert and End] hdd crashes [5 HDD copies] computer crashes [2 computers, 2 non hdd backups] disasters on site [off site backup and the "emergency bag"] and CD problems [two sets of CDs made]

 

What I don't have right now is a process to maintain my CD archive beyond jewel cases. I suppose when DVDR matures more I'll re-archive my CDRs to DVD+-R. I also don't have a prodecure to confirm the viability of my CDR backups once stored in the E-BAG or offsite. When I do switch from CDR to DVDR this will be simplified in that I can just burn the entire dataset to fewer DVDRs than I'd be burning CDRs for incremental backups.

 

Winkflash.com offers unlimitedphoto hosting with no limits. They also make decent prints. If you've got a super-fast internet conection you might use their photoalbums as a 2nd form of offsite backup.

 

Sounds like I do alot, but, from the point I pull the CF card from the camera till I get the 1st set of CDRs into the E-Bag takes me about 30-40 minutes per 700MB.

 

With the bottleneck being the CDR burning process.

 

Copies from HDD1 to HDD2, Copies from Workstation to Server, and Copies from Server HDD1 to HDD2 and HDD3 go very quickly relative to the time it takes for the CDRs to burn and be validated(this is the biggest time consumer.)

 

I've yet to loose a photo that's gone through this process. My offsite backup is only 14 miles from my home. I suppose a region-wide catastrophe could take out my entire archive, but I imagine I'd have bigger things to worry about at that point.