Is blu ray the way to go yet?

<p><em>...I'm pretty confident a removable 1TB HDD disconnected in a case in my office is just as "survivable" as an optical disk which has it's own potential demons. The problem with optical is mostly the limited capacity.</em></p>

<p>David, mechnical hdd's and optical disks are apples and oranges. You can not favour one based on it's pro's and dismiss the other becasue of its con's.</p>

<p>The problem with optical is NOT mostly the limited capacity, it's the laziness of people and how little they value their data.</p>

<p>I now use 1.5T to 2T hard drives. While each one hard drive is vulnerable, as long as you have 3 or more coies, it is very safe. All of my digital images are copied to hard drives at home and a separate copy at work. I store another copy at my parents' place a few hundred miles away. I visit them 2, 3 times a year and each time I swap an updated hard driver over there. I also carry around a 500G portable drive with my recent work.</p>

<p>Effectively I have like 3 to 4 copies of all of my images. I have yet to lose any image in 8 years of shooting digital.</p>

<p>Blu-Ray discs are simply way too expensive and their capacity too low. If you need to make a copy of all of your images, it'll cost you a lot of money and you need to keep swapping discs from the burner. For that same $79, you can now get a 1T hard drive. With one click on the mouse I can copy the entire content from a 2T drive to another empty 2T drive. It may take hours (like close to a full day) to copy, but I don't need to baby sit it; I could be sleeping while the computer is busy copying.</p>

<p>I think Marc (the OP) is way ahead of the camp that just use hdd's. The point of this thread, I thought, was that Marc already had his data on hdd's and has those bases covered and is now looking for an optical solution for extra redundancy? Good on him and not sure why some are trying to convince him out of that extra step and that his hdd's are good enough?</p>

<p><em>Effectively I have like 3 to 4 copies of all of my images. I have yet to lose any image in 8 years of shooting digital.</em></p>

<p>You mean you have 3 or 4 <em>versions</em> of your images. 3 or 4 copies would be on 3 or 4 different types of medium.</p>

<p>I haven't lost an image in a week. Or in 14 years of saving and archiving. It doesn't mean what I am doing is correct or bomb proof. But the first $22 blank cd I bought in 1996 still reads and I can open those tif files. Yet the jazz drive and the hdd that the data was in that PPC is unusable or a damn pain in the wallet to access.</p>

<p>There's problems with every medium. That's why you are not backed up until you are on at least types.</p>

<p>To be reasonably safe you need multiple backups stored in at least two different locations. By two different locations, I do not mean two different rooms in the same building. It needs to be two different locations, preferably separated by miles.</p>

<p>As an example - last year we had a house fire. The fire started outside and blew into the attic where it was confined. The fire department arrived and quickly extinguished it before it could spread to the living area. We were lucky. Even though I did not need them, I was very glad I keep a second set of backups in my safe deposit box.</p>

<p>It is not just photos that I would have lost. Financial data, household inventory, and correspondence for the past 12 years are also on the computer - and on my off site backups. Be sure that your other critical records are backuped multiple times and at least one copy is stored off site, not just your photos. </p>

<p>If you decide to use HDD for off site backup, I suggest you use notebook type 2.5 inch disk drives. Althought they are slower, they are built to withstand more rough handling than the 3.5 inch drives. That does not mean you can throw them about, but they are designed to hold up better to transport.</p>

<p>I use redundant copies on hard drives for immediate storage of new files, and after moderate selection and editing, I burn the files to blu-ray discs within about six months of capture. Then I delete the redundant hard drive copy. Later on, I will go through the files again and delete everything but the very best pics, which I then burn again on optical. I have lost controllers to hard drives due to power spikes and the data were unrecoverable (at least within reasonable cost). It's not that difficult to loose both hard drive copies if you're unlucky. Because of their large storage capacity, the loss would be catastrophic. I have never lost any data I burned on CD, DVD, or blu-ray disc yet (provided that the verify process gives green after burning). I have lost many hard drives over the years. IMO they're the most unreliable storage. Fast, yes, large capacity, yup, but very unreliable over long term. Optical disks have no moving parts and thermal changes are what will eventually kill them but I'll be six foot under long before I lose files to that I suspect. Of course with optical I also need to keep multiple copies like with any media, but at least it has a chance of being reasonably stable over long term, unlike hard drives. When you've got your primary storage and backup connected to the computer at one time, all it takes is one spike and it could all be gone. </p>

<p>By the way, my LG blu-ray drive has turned out to be very reliable in burning all three main types of optical discs. I've lost one burn only, and that was a user error, when burning from internal drives and having enough free space on the boot drive, there are no problems. Unlike with DVD drives which I've had many over the years, they all routinely failed to burn discs correctly from time to time, especially with dual layer 8 GB DVDs. The blu-ray drive seems to be far better designed in terms of reliability.</p>

<p>I'm curious why no one has mentioned off-site storage or back-up. I've been using Mozy (which is only back-up... it will only keep back-ups of the files on your hard drives... even if you have a bunch of them linked together) I have an unlimited size account. I dont recall the cost but I would not have done it if it wasn't reasonable, competitive with owning my own equipment, and offered what I thought was a better solution to everything in one place. There are plenty of other similar services out there.</p>

<p>and I forgot to mention... it's automatic. it backs up often throughout the day. it's search-able... how easy can it get?</p>

<p>Tom, how long does it take to upload, let's say 100 GB of files on the backup system? This could be one major event. </p>

<p>Ilkka - the initial back up will take a lot longer than the rest. how long, of course, will depend on how much you are backing up. so - of course the first one takes longer... but, it works in the back ground and will not hog your computer. I never even notice it anymore. just a little window pops up and tells me how long ago the last back-up was.</p>

<p>The problem with cloud backups - as distinct from physically moving the data off-site - is that the data is no longer under your control. Too many of these companies have gone out of business. Then where are you? More important, where is your data? Who has it? Is it really encrypted? If so, who now has the crypto keys? Even if the company is still in business, how secure is your data? This may not matter for photos, but it certainly does for much of the other data we keep on our computers.</p>

<p>By the way, Tom, have you ever tried a full restore from your cloud backup to simulate disaster recovery? I would be interested to know how well and how fast the restore performes.</p>

 

Tom, I casually mentioned off-site (or online) storage in my post Apr 20, 2010; 04:11 p.m. My main beef is that it depends heavily on upload speeds (which are considerably slower than download speeds). Granted, if you just let your system do its thing automatically, before too many weeks have passed you are completely synced up with your online service. But actually, with a typical upload speed of 1.5 Mbps for high speed users, the 100 GB that Ilkka asks about would take around 150 hours, or just under a week if uninterrupted. In reality, it would be hard to maintain maximum load nonstop, so extend this out to 2 or 3 weeks for a 1-time upload. After that, you only have to worry about your periodic changes to the archive. Say you shoot 8GB one weekend. You delete 2/3 of the pictures, but still have to sync almost 3GB of data. That's around 24 hours of downloading before you're synced up. That puts an idea on the real limitation. 3GB per day. No more. That's around 1TB per year. If you use any more than that, online storage won't work for you. And as we all know, if you are using that much bandwidth, you can say goodbye to any other services. Don't expect, for example, to be able to check your email in less than 30 minutes while you are syncing at max-speed. Forget casual browsing, Facebook, Youtube, news, weather, or anything else. You will have to (as I do) pause all other internet activities while you are using the PC. I believe, however, that in the next generation of internet, the fiber-optic powered era of 2-3 years hence, this solution will become a no-brainer. For now, I think it only works for people with more modest needs, or professionals who are only selectively uploading the most important data to the service, like the final-cut customer images, the ones for sale.

<p>Brooks... "simulated disaster recovery"? they do offer a limited size free account... you can go simulate all you want.<br>

but... I looked--> I paid $209 for 2 years of unlimited file back up. (not storage. if you delete a file from your hard drive it gets deleted from the back up. [that's why I have a series of hard drives])they have 3 modes of restoring your system: </p>

<p >1. re install Mozy on new system. and then do a small backup. After that, click on the Mozy icon and select Restore. Select the old date and restore. This the recommended way.</p>

<p >2. you can order DVD's, they deliver them to you.</p>

<p >3. (they don't recommend this way) is Web Restore you have to log on to your account and then select the files there. You will get the links to download your data. In web restore you dont have to install Mozy on your system. Log on to your account and click on the link Restore files to get your data.</p>

<p > </p>

<p >as for comfort levels and cloud back ups... that is all up to the individual. some people wont even vaccinate their kids... so... I'm not going to try to convince anybody of anything. I just thought the off site option should be included in the discussion.</p>

<p>Craig, good points. I'm not a "heavy" user by any means. I do put up about a dozen photos a day along with all the other stuff I do during a day. with that I have not noticed any depletion of service levels.</p>

<p>IMO using multiple types of media is not what I would do, IMO it is just increasing the likely hood of you loosing one whole set of backups. Be it from accident, damage, or obsolescence. And optical media is almost the worst option IMO. In less than 5 yrs I believe you will see a 25-30% loss of your data with an optical solution. Do not trust your data to manufacture claims. </p>

<p>Use the HD tech that is current and migrate your data every 3-5 years and use two (if possible) back up drives. For me, the last two system upgrades have resulted in a singe image lost out of 750gb of photos. </p>

<p>If you are a light user, then use one of the on-line services. They will regularly back up your images for you.</p>

<p>"The problem with cloud backups - as distinct from physically moving the data off-site - is that the data is no longer under your control. Too many of these companies have gone out of business. Then where are you?"</p>

<p>Mozy is owned by EMC, a Fortune 500 company. So I wouldn't worry that they'll go out of business!</p>

<p>I use Mozy, too, but backups are slow because the $5/month plan is bandwidth limited. SOme backups run for days, but that's fine with me because i's in the background. And I only plan to use it if my primary backups fail.</p>

<p>From my experience, I don't plan to rely on HDs as they fail at a frightening rate. And, things like RAID are just too complicated. You do a BIOS update or a drivers update or a full OS refresh and suddenly you PC isn't sure what's going on with the HDs any longer and its time to reload RAID drivers and try to figure out what's where. Same but worse when one of your HDs dies.</p>

<p>I hear people dissing optical disks, and I understand that there are reasons to be concerned. But the simplicity of optical disks remains a powerful attraction for me. In addition, consider hard encoded DVDs like commercially sold movies. Mine are all good going back 10 years. So, what's the problem again? Doesn't it really come down to identifying the right kind of DVD-Rs for archival purposes? My research suggests blu ray has indeed taken this to another level, so I plan to start there. </p>

<p>Indeed, hard drives get messed up all the time and files are frequently lost in daily usage. At work I make daily backups of my documents and code because of this.</p>

<p>In the 12 years I've been storing my photographs on CDs, DVDs, and blu-ray discs, I haven't had a single instance of a read error so far. I estimate I have stored about 100000 images on optical discs so far - according to Matt Brost I should now have tens of thousands of unreadable images by now - wow, quite a lot of luck that I haven't run into one of them yet.</p>

<p>I can also play my old music CDs from early 90s and later on my blu-ray drive - audio CDs have existed as a format for thirty years and they read just fine unless you physically scratch them (I managed to do that to a few audio CDs a long time ago when I played them in a portable player on my bed, have since learned to handle them properly.) Since every single standard CD, DVD, blu-ray, and HD-DVD drive can read them all, where is this obsolescence that is supposed to make the files unreadable in 5 years?</p>

<p>Hard Drives I believe do much better job that disks when it comes to the archival properties.Just about all data gathered by TV stations ends up on hard drives or a magnetic tapes.</p>

<p>@Ilkka There is a big difference between commercial CDs and DVDs and the ones we make on our computers. Commercial CD/DVD are literally pressed - the "bumps are physically pressed into the plastic. CD/DVD made on computers use the heat of a laser to cause changes in dyes. Here is an article that describes the processes: <a href="http://en.wikipedia.org/wiki/CD#Manufacture">http://en.wikipedia.org/wiki/CD#Manufacture</a> .</p>

<p>The fact that your commercial CD/DVD has a long life does not mean a data CD/DVD you make on your computer will have the same life</p>

In the CD era (granted, the technology was in its infancy), my early failure rate (coasters) was around 50% for 1st generation media. Later, I discovered more reliable media (and burners. My first burner was 2x). Eventually, I settled into a pattern with nearly 100% reliability on my CD burns, but I've still had failures on those discs after around 6-8 years of climate-controlled storage. Also, I remember one specific experience when I bought an entire spindle of 50 discs, which happened to be a bad batch. I had burned all 50 discs before discovering the problem. I later learned that nearly 2/3 of those discs were unreadable. My DVD experience was better, as I have burned maybe 2 or 3 coasters out of around 500 total. I have only had, perhaps, 5 or 6 failures. Sometimes the failures show up after 6 months, sometimes it's 2 years. I had 1 give out after about 4 years. If you aren't extremely careful to keep them in a low-humidity environment, you can even lose discs due to mold in the dye layer. This is exactly the same with Blu-Ray recordables. The only discs that don't use an organic dye layer are special archival discs. I don't use BD-R, because they aren't financially viable yet even at $3 a disc. Forget it at $15-20 a disc. Maybe at $1.00 per disc, it would be worth it to back everything up to BD-R, but at the bargain archival rate of $16 per disc, you're paying out $640 per TB. For redundancy, that's $1,280 per TB. In hard drives, I can accomplish this for as little as $140 per TB with full redundancy. No, the hard drives aren't archival. But dang it, they're cheap and fast. And in a world where technologies are out-moded faster than I can wear out a pair of shoes, it's important for my archives to be nimble and easily replicable. And as to this argument about 30-year-old CDs: first of all, those are commercial CDs. CD-R's are only about 15-years-old. Secondly, that's a historical anomoly. No other media has lasted so long in a continuously back-wards compatible environment. CDs are already 2 generations behind, and the only reason you can still access them is because Blu-Ray players are on the same physical platform with full compatibility. If Blu-Rays had been, for example, 100mm instead of 120mm, CDs would be not only obsolete, but inaccessible. Completely unreadable. I would look at my future archival options with the completely open-minded outlook that 120mm discs might become non-existent within 5 years, and modern readers aren't designed to last longer than about 5 years. There's no way that your current machine will still be functioning well enough to read your 200 year discs even 20 years from now.

<p><em>In hard drives, I can accomplish this for as little a...</em></p>

<p>Again, Craig, this isn't about which medium is best and make a choice between optical or hdd. There is no "best" and is why the wise use more than medium to duplicate.</p>

Check again: "Is blu ray the way to go yet?" See, it IS about which medium is best. I contend that there is NO ARCHIVAL SOLUTION for digital data. This is the problem. A suitable archival solution would require that a civilation far removed from ours could recover data from our archives, without the use of our computers. I'm talking a post-apocalyptic civilization. We dug up ancient Greek and Hebrew writings from caves surrounding the Dead Sea. If the code used was any more complicated than deciphering an ancient language, they would have been unreadable. The very concept of an archive is at the root of the problem. For viable archives, 500 years is sort of a minimum of readibility, as it means that every 500 years the entire archive has to be re-copied. Optical solutions are no better than magnetic ones, as they will both become obsolete or null in a shockingly short amount of time. If a disc doen't become unreadable within 20 years, then the machines necessary to read it will at least disappear. The only solution our current system gives us is continuous, unbroken maintenance.

 

I strongly disagree that using multiple storage formats is any better than using multiple copies of the same storage format. Either way, you have to maintain the technology to read it. And just to be clear, neither optical discs nor magnetic discs qualify as a "medium". Since either one requires a decoding machine to interpret the data and present it. Digital photographs are at once a medium, and not a medium, at the same time. They exist, and they don't exist. The only archived photograph is one that has been printed out on 500 year paper with 500 year ink and stored in a 500 year locker.

 

Now given all that context, I will unabashed declare that redundant storage on hard drives is just as reliable as any other disc format, and infinitely more convenient, cheaper, and affords the only method for keeping a single, contiguous archive. It also provides the simplest and best means for transferring your library forward onto the next format whenever it arrives. You don't see large companies with millions of dollars in digital assets backing everything up onto a myriad of BD-R's. Only small-time, rinky-dink operations with negligible quantities of data bother to use optical media for library replication. That's because it's too darn expensive to use in any large number. If you could back up everything using less than 10 BD-Rs, maybe it would be viable. But that's a joke in my book. It would take me closer to 30 for what I have on file today, and that doesn't account for what I will produce tomorrow.

<p>No, sorry mate, you to need to check again. Marc is already doing hdd back-up's and is looking for an additional method and is wondering if blu is the way to go. That's the way I read it, anyways. Maybe I'm reading it that way becasue that's how I do it? I don't know. I don't infer that he is looking to replace his hdd back ups with optical.</p>

<p><em>I contend that there is NO ARCHIVAL SOLUTION for digital data.</em></p>

<p>Exactly. So why do you keeping carrying on about using only one medium?</p>

It's quite simple: I've been there, done that, learned my lessons, accepted the truth, and moved on. I simply can't afford to use archival BD-Rs in the quantity that I would need them. Recordable discs are no longer a financially viable tool. If you have your own money-tree, then by all means drop $16 a disc on 200 or 500 year media. It's pointless, no more secure than other measures, and cost prohibitive. I had hoped I could dissuade someone from wasting hundreds, perhaps thousands, of dollars on unnecessary, fruitless, and time-consuming duplicature in the face of logical alternatives. Is Blu-Ray the way to go yet? Straight answer: no. All told, there aren't that many choices. 1) HDD. 2) BD-R. 3) Online storage. 4) Tape Drives. 5) DVD-Rs. Did I miss something? These all suck, but there's only 1 that makes fiscal sense in every case. There are not 2 good cost-effective options. There is only 1. Not two. One. For certain isolated, infrequent, and low-capacity applications, there might be 1 or 2 of the other options that honestly make sense to a dwindling minority of the populace. But this is certainly not the case by-and-large. There's just one. For the record, the words "fiscal", "financial", and "cost-effective", among others, deal strictly with the issue of money, dollars, dinero, plata, pounds, lira, yen, kronas, $$$, green backs, frog-skins. See where I'm coming from?

<p>I guess some people can base backup strictly around cost. For some of us, that makes no sense, the images are more important than marginal dollars in the bank account. I archive with a similar strategy to Garrison and can't imagine worrying more about the cost than the images.</p>