Backing up photo data

[greg_lisi]

<p>Hi all..<br>

Just wondering what media/function y'all (I'm from the south) use to backup your treasured photo files?<br>

Recently my neighbor lost quite a bit of his family NEF's/JPEGS to a "reliable" external drive without off-loading to some form of media. He told me this nightmare scenario made him feel as if he experienced a death in the family (God forbid!). I also have a decent external drive with about 200GB of family related stuff. I'm not about to re-live my neighbor's titanic so I've been doing some research.<br>

1. Online backup services-(Mozy.com high recommend) My take:Good, fairly cheap. Recovery not so cut and dry.<br>

2. DVD- Unless you have dual layer functionality and even with that, how can you store large capacity backups?<br>

3. Use another external hard drive-Practical idea, however, it's still a mechanical device prone to failure.<br>

4. Tape (DAT)- Old technology, initial expense, BUT....large capacity storage. HP claims their DAT 72 media can take long term storage.<br>

These are just a few of my unqualified opinions.</p>

[hans_janssen]

<p>I tried a online back-up, but it was not possible to get it all backed-up in a reasonable time(more than a week for 350GB).<br>

Now I use a a 1TB HDD and a second one with a TrueImage back-up on it. I have tried to store this HDD at a friends house and than upload the files to there, but that wasn't reliable. Now I store the 2nd HDD on a other address(in case of fire, burgelary).</p>

<p>DVDs are not reliable and too small. You have to check an renew the DVDs every 2 years or so and you have the chance that there a some corrupt ones.</p>

[david_mccormick]

<p>Great question. I'm sure everyone here has wrestled and agonized over this stuff. <br>

I won't speak to 2-4, but I use Mozy. One issue that I have faced repeatedly with Mozy is that if you try to back up an external hard disk, if you remove it and then re-attach it, Mozy often seems to think this is a new volume and spends eons trying to re-check and backup the drive. I have about 200 GB nominally on Mozy and one of these incidents can mean days to a few weeks for the backup to complete. And I have Verizon Fios with good bandwidth. If you have everything on an internal hard disk it works much, much better. So I'm kind of ambivalent on Mozy at this point. And you have to think about the likelihood that broadband providers will very likely go to volume-tiered service in the next few years, which if you shoot GB's of photos might become a more expensive proposition. Having said that, it's really easy to set up and use. <br>

Another option that I'm considering is Drobo: a very simple multi-disk RAID system. You buy any type of SATA drives of any capacity, stick them in the enclosure, and plug into your computer. This is an on-site storage solution. www.drobo.com<br>

I'm sure you will get lots of good suggestions on this. </p>

[david_mccormick]

<p>And I remembered that Scott Kelby had a long blog piece on his backup strategy last year (includes Drobo as part of his): http://www.scottkelby.com/blog/2008/archives/1410</p>

[will_daniel1]

<p>Greg,</p>

<p>Use your No. 3 idea. Yes, hard drives are mechanical devices. What is the likelihood that they'll both fail at the same time? Near zero, I guess. I can think of only a very rare occurence that could make that happen -- an extremely large power surge, as occurs in a lightening strike, with both backup drives operating at the same time.</p>

<p>I have my photos and other important data on three hard drives, one of which resides off-site and comes home for weekly updates, then right back off site again. Also, every six months or so, I burn DVDs of all the family photos for my two grown children. This gives me two more off-site locations to retrieve most of the files in case of a catastrophe.</p>

<p>Will</p>

[John Crowe]

<p>I am sending in my dropped 1 TB external this week for recovery. I know the files are okay but I am keeping my fingers crossed for my pocketbook!</p>

<p>I will now buy two 1 TB externals and keep one in a permanent safe place. I will back up the second drive once a month or more depending on images created. I will buy a DVD burner and back up my portfolio images. I will create a second set of DVDs and store them at a relative's house for fire/flood/wrath-of-god scenarios. I will create additional DVDs on a yearly basis or as required. </p>

<p>As an aside the CDs that I back-upped to 5 years ago are fine. Now I just have to get the last 5 years back! </p>

[ShunCheung]

<p>Just like there is no human being who will never die, there is no such thing as a "reliable" hard drive. All hard drives will eventually fail; it is merely a matter of time, but the better ones will last longer than the others.</p>

<p>However, since they are getting cheaper and cheaper, the simple solution is to copy your images (and other important digital files) onto multiple hard drives. When one of them fails, get another new drive and copy the files to the new one from one of the surviving copies.</p>

<p>I am more to the extreme. I routinely have my images on 3, 4 different hard drives stored at different locations. Once or twice a year I take a backup to my parents' place, as they live in a different city. You need a lot of dicipline to keep the various copies updated, though.</p>

[User_3804048]

<p>When I started to become concerned about backing up my photos, I added a 200 GB external HDD and copied all of my photos to it. However, continually manually transferring new photos requires a certain amount of discipline, so I also added a 2 x 1 TB RAID system to automatically back up my photos and a few other items. Some of the better stuff is also backed up to DVD or CD. My old computer recently developed problems unrelated to the HDD, so when I recycle it I will remove and retain the HDD from it. The 200 GB hard drive was very useful, by the way, for transferring files from my old computer to the new one.</p>

[fernando_scherer]

<p>i use a 320gb hard disk on time machine. will buy another one soon.its firewire 800 for faster transfer.<br>

And the more duplicated the better.. good idea to put on a different address in case of fire, burglary and dropping..<br>

only dont use dvd because in my early trials, they seem unreadable after some months..</p>

[jyoungman]

<p>Personally, I store my photos in my desktop computer. It has a RAID-1 array in it, meaning that it contains internally two identical copies of the data, and so if one disk fails the computer just asks me to replace it - no data loss at all.</p>

<p>However - and this is <strong>really important</strong> - this is NOT ENOUGH. For example if someone stole my computer I'd lose both disks. In the case of flood or lighning strike the same (not likely for me, I live on the 3rd floor, out of flood danger, and in Ireland, no significant danger from ligtning strikes). So a proper backup is still absolutely necessary, as you said.</p>

<p>I do this with a pair of external drives. I synchronise all my (valuable) data to the external drive attached to my computer, and leave the other one in the office at work. I work far enough away from my home to make it very unlikely that a natural disaster will wipe out both locations. Every couple of weeks I swap the external drives over.</p>

<p>So there are 4 physical copies of my photos, spanning two locations.</p>

<p>As you point out, hard disks are mechanical devices and therefore may fail. But the same is true to some extent of any physical object. They can fail, be stolen, get damaged or become lost. The solution is redundancy.</p>

<p>The main remaining risk to my data is data corruption due to software or hardware error. For example, some piece of the system scribbling over my data. There is no real protection from the event itself (of course using quality software and hardware probably mitigates this). However, it is certainly possible to compute checksums of the data that should never change. For example, my NEF files never change at all. If I recompute the checksums of the dtaa regularly, I can spot when one of them has changed and investigate (to recover, I can restore a known-good copy of the file from e.g. the backup in my office, but in reality I would be more likely to do something more extreme, depending on what seemed to be the original cause of the problem).</p>

<p>Regarding your 4 candidate methods, I have used DAT tape for backup in the past. Most of my DAT backups were made in the mid-1990s. About 5 months ago, I transferred all the data off my DAT tapes back to the RAID array, since data on DAT (more correctly, DDS) tapes has a finite life. One likely probem with DAT tapes is backup software compatibility. You indicate you may rely on DAT as long term storage. For that to work, you need to be certain that you can get software to run and understand your backups in 10 or 15 years. Many programs written for Windows 95 won't work on Windows Vista, so this is a potentially difficult problem (unless your vendor never goes bust, is not taken over and doesn't change its product range). I avoided this problem by using Linux, which has a small number (about 3) of standardised backup formats, meaning it will read backups made any time since some point in the 1980s and they'll continue to be readable for another, oh, at least 20-30 years).</p>

<p>Personally I discarded DVD as an option since 4.5GB per volume is too small; I currently have about 80GB of photo images.</p>

<p>That leaves offsite backup services. I implemented this for myself by having the computer in my office connected to the external drive I keep there syncronising with my home computer using rsync. Unfortunately, this does not work satisfactorily, because the bandwidth I get _out_ of my apartment is very very much lower than the bandwidth I get _into_ the apartment. This is very common with consumer broadband services. If that bottleneck wasn't such a big problem I'd also check out Amazon's storage offerings (they have 2 - an online solution plus a facility where you courier them a USB hard disk).</p>

 

[ken_i_h]

<p>Um, everything has possibility to fail. CD-R, DVD disc, Tape, all the same as time goes by. There is no 100% guaranteed method, I guess.<br>

I just use TWO identical HDDs (Seagate FreeAgent Pro) and use FreeAgent to mirror the data to another one. I store most of my images in NEF file, FWIW.<br>

Sorry for my short answer, but hope this helps.</p>

[pete_s.]

<p>You'll be hard tasked to hook up a usb or sata harddrive to anything in even 20 years so for long term backup print the images you care about and store them in a safe deposit box. Perhaps make a few photobooks as well. These will outlive any digital attempts of long term storage.</p>

<p>To minimize the effect of a harddrive failure run mirrored disks (RAID1) on your computer or external network attached storage (NAS).</p>

<p>Any offline service or storage attempt will fail in a couple of years. I have files from 1989 that I kept on a raid server which has been upgraded and replaced every 2 years. I also have stuff on tape and other storage medias but no way of reading it because there is nothing compatible with it anymore.</p>

<p> </p>

[jyoungman]

<p>Prints fade with time. A digital record does not degrade as such. Yes, you will need to copy the data to fresh media every 10 years or so, but the copy created is an exact copy of the original.<br>

On Ken's point, I'd be cautious about trusting my data to two devices of identical manufacture. Better I think to use two dissimilar devices in order to avoid getting a batch with a defect.</p>

[bob_flood1]

<p>Every method of storage has failure modes - every one of them. No single method can be expected to be bullet-proof, so multiple copies seem the best solution. I like the notebook USB drives - 500 GB drives are cheap, 1 TB drives are getting there. I use 3 500 GB drives and have a batch file (remember batch files?) that uses the old xcopy command. Once I load a new set of files onto the first drive, the batch routine copies only the new or changed files to the other 2 drives. The batch files is an icon on my desktop - easy to fire it off whenever I want.</p>

[steve_wagner1]

<p>External ESATA or Firewire800. Avoid DVDs like the plague</p>

[ArthurRichardson]

<p>External Raid 1 esata 2TB drive for storage. Disks will fail, but I trus not both at the same time.<br />I heard stories of recent online backup gone dead so bye bye pictures.....</p>

[buffdr_rasouliyan]

<p>I now use 3 Rosewill SATA enclosures that hold 2x1 TB each. That's 6 TB for my use and I back them up weekly. I hate to say this but I don't backup RAW to DVD. After my work flow, I backup the JPEGs on DVD. I also have 3 AMS enclosure with fans that I use to move files to and from. I also try not to transfer large (200 gig) to or from a hard drive. It seems like all the action tends to run them hot and fail. For the most part, I have been lucky. The answer is don't keep your eggs in one basket! Use both hard drives and DVD. Next up for me is a Blue Ray burner. Good luck v/r Buffdr</p>

[stephen_asprey2]

<p>The real elephant in the room is what file types and media will be the go in 5,10, 20 years. Computer manufacturers of large systems have faced this problem for decades.<br>

The only answer is to back up to physical media that is the preferred choice at regular points of time. And to continually move your files as these systems become obsolete.<br>

Its inevitable that in 10 years Jpeg and 5 1/4 disks won't exist. You need to keep it all moving. But one this is true, as media and file types change, vendors also produce conversion software to suit...but only for a time.<br>

We are on the cusp of ordinary DVDs and CDs being obsolete and Blue Ray replacing them. So when BR takes over, move your files. And keep doing that regularly.<br>

The really crazy part is that there are now some firms who will move your Jpegs to film for you for long term archiving. How about that? Its going full circle.</p>

[pete_s.]

<blockquote>

<p>Prints fade with time.</p>

</blockquote>

<p>Because of all the faded images from the 50s, 60s and 70s research was initiated how to make lasting prints. Regular fujicolor crystal archive that minilab uses has 40 years of display permanence and 100 years if you keep them in the dark, like in an album. That's what Wilhelm Research says, not the manufacturer.</p>

<p>If you want to up that can use a printer like my z3100 that uses HP vivera ink. It has >250 years display permanence (under glass) and more than 300 years of album/dark storage. http://www.wilhelm-research.com/hp/WIR_HP_Z3100_2007_12_28.pdf</p>

<p>Of course try to keep your digital backup running. I have kept my digital backup for 20 years and have all my digital images (from 2000 and forward). I wonder how many years all the posters here have kept their backups.</p>

<p> </p>

[rgerraty]

<p>Minimum of three external HDs but I agree with Pete S. — print and store in an archive box the precious ones. Of course, if you still use film then you have another source of relatively stable archive material.</p>

[walter_strong5]

<p>Whatever you use for backup consider storing it in a safe deposit box at your bank. Somehow, leaving my backups at a "friends" house just does not seem at all wise. </p>

[David_Cavan]

<p>I am in IT - I've been around IT operations for over 20 years. That's taught me a few things that I've put in place for my family photo history and personal archives. First, keep whatever you do simple so its easy to keep track of what-is -where; second, NEVER trust tape- so many things can go wrong; and third, assume the worst when you put your plan together - whatever you assume, its likely something even worse will happen. I use two external HDD's on a monthly rotating schedule, kept in different places outside our home. For those I just do a complete backup for the 24 hours or so one of them is in the house - nothing fancy, just copy everything onto the HDD and then move it back to its home elsewhere. In between those backups I've invested in a $125, 750GB pocket-size Passport device that runs an incremental backup using it own software - usually in less than 30 minutes. That goes back to the office between uses. The outcome is that I can always get back to the last incremental backup easily, using a combination of the last good HDD (that allows one of them to be bad and still have a fall-back) plus the incremental pocket-device. Also, all of our photos (about 300GB) are on the latest technology just by keeping the desktop and HDD's up to date. So far, so good.</p>

[george_peng1]

<p>I've said this before, but I think the most reasonable answer is some sort of network-based RAID array. It provides you with the mechanical redundancy necessary as well separating the disk array from any sort of PC-based risk (crash, BSOD, etc.). Data transfer rates are obviously compromised slightly, although not enough that you'd probably notice with gigabit ethernet, but a minor tradeoff.<br>

You could compound it with periodic offsite backup as well, if you wanted belt and suspenders treatment as well. Either way, its far superior to manually offloading to an external HD or DVD solution.<br>

The cost can vary dramatically, but the one I use is pretty cheap and works well - Dlink. Works for me.</p>

[kevin_beretta]

 

<p>I'm in IT as well. Here's what I do:<br>

1 RAID connected to the WiFi network at home. I back up my laptop every day, wirelessly, takes 2 mins, sitting anywhere in the house. Product is D-LINK DNS-323 with 2 X 250GB drives. Takes 2 X 1TB drive as well if required.<br>

1 Vantec Nexstar drive docking station. Takes 3.5 inch as well as 2.5 inch drives. You can pop them in and out like the old floppy drives: <a href="http://ncix.com/products/?sku=29676&vpn=NST-D100SU&manufacture=Vantec">http://ncix.com/products/?sku=29676&vpn=NST-D100SU&manufacture=Vantec</a><br>

I use the RAID daily, the Nexstar weekly or so, with 3 drives. One in a safe at the bank (every 6 months), one docked in the Nexstar, one at another house which get swapped with the docked one every month or so.<br>

In addition, I have a 320 GB USB pocket drive for when I travel.<br>

The key to making this simple consists of two things:<br>

1) Streamline your workflow. My backup discipline involves copying/adding as follows <br>

Computers > RAID on WiFi network > Vantec (swap Vantec drives on rotation).<br>

Or<br>

Computers > 320 GB USB pocket > RAID on WiFi network > Vantec (swap Vantec drives on rotation).<br>

That way, my computers are always the "master" of data, with copies on other drives. I never copy "backwards"<br>

2) I don't use backup software because I've never found something that really works and that I can trust and restore if needed. I use a product called Beyond Compare from Scooter Software. It allows you to "see" the different/incremental versions of files between locations on your computer or between drives, similar to browsing through folders. Basically, it highlights newer/updated/removed files and you select what you want to move over to the next drive. I usually "select all" and it will only copy what is not existent on the target drive or newer on the source drive. If I prefer to not keep xyz file on my laptop, I simply remove it from the laptop and doing the above it does not get removed from the backup drive and propagated to all the other drives. <br>

So I basically have 6 places where my data lives ... computer, 320 GB pocket, RAID, Vantec (times 3). <br>

I also go through and format drives once in a blue moon, as that is the only true way to clean them up, do a disk check etc. Drives are dirt cheap.. There is no way you can shoot enough to ever catch up with the falling price of storage. The media will change, as always, so as long as your workflow is ok, you should be fine. Formats like RAW and JPEG I hope will be accessible in the future. The more pictures exist in the world, the more formats will be embedded in the future. In the earlier part of the computer century, fewer of use used stuff like RJB formats (still the best compression software available), but as things get more widespread it will be baked into future products more.</p>

 

In addition to all of the above, I FTP my pictures to a protected folder on my Godaddy hosting server. You can get domain hosting for $77 for 2 years and it gives you 150GB of space.

[chris_murray1]

<p>Lots of good suggestions to this question. Let me add my 2 cents.<br /> <br /> For my situation, I'm worried about mechanical failures (disk drives going bad), but also disk corruption ( <a href="http://en.wikipedia.org/wiki/Bit_rot">bit rot</a> ). I don't want my files to become silently corrupt over time, and never know it till I try and access the file. That's why I choose the <a href="http://en.wikipedia.org/wiki/ZFS">ZFS filesystem</a> for my file server, running under <a href="http://www.opensolaris.com/"> Open Solaris</a> . It performs checksumming on the files anytime they are accessed. I've setup a raid-5 configuration (Sun calls it RAIDZ). To further check the validity of the data, I've setup a weekly "scrub" to check the consistency of all the on-disk data. If any issues arise, ZFS will correct them with the metadata from the existing, valid drives. <br /> <br /> So my file server contains the "live" data. I've got a nightly rsync script to copy the data over to a network attached Western Digital MyBook World Edition (pair of mirrored 1 TB drives). I'm now setting up another ZFS file server to install off site, and rsync the data nightly to it for extra backups across the Internet. Plus, there's another 500 gb external drive for backup I store in the home safe. As well, I rsync from the file server to my local workstation nightly. I'm a bit paranoid about losing data! I think I've got my data in enough spots. Someone mentioned simplicity. Once you've setup your sync scripts, they give you piece of mind. <br /> <br /> I'm surprised Mac OS didn't go ahead and put ZFS in Snow Leopard. I thought that was a no brainer. <br /> <br /> Good luck with whatever solution you go with. Hope it lets you sleep well at night!<br /> <br /> Chris</p>