digital image storage

[knowlesy]

as some of you may have gathered from my post yesterday I am purchasing a

K100D, which seems to be a very capable camera, and although I did consider

alternatives from Canon/Nikon my excellent experiences with Pentax in the past

plus the compatibility of my existing lenses tipped the balance.

 

My question is though something a little more boring but essential all the same

(you may wish to know this will be my first DSLR but I am fairly computer

literate). From what I can see it see shooting RAW is the way to go for image

quality, I realise these must then be converted but to what format? In terms

of image quality and future safeness.

[mohammed_abidally]

RAW can be converted to JPG.

 

For better quality you first convert to a 16bit TIFF which is then further processed in PS etc and then converted to 8 bit JPG for final output/storage. However it is good to also archive the RAW files.

 

I was also an old Pentax user howver converted to canon last year when I went DSLR. Still have several old M42 lenses...

[nickwhite]

Firstly, always keep the RAW file - you can then always return to it in the future.

 

For 'working' images use a lossless format such as TIFF, and if you want to return to that 'final' image, without loss of detail, save as a TIFF also (preferably in addition to the original RAW).

 

If you save in a lossy format such as JPEG you are throwing out some of the image detail (in which case you may as well have shot in JPEG to start with).

 

The only reason to save in JPEG is to reduce the file size eg for Web postings, emailing etc.

[mr. sullen]

"From what I can see it see shooting RAW is the way to go for image quality" Well from what I have learned so far, shooting in RAW will produce the highest quality image you camera can produce. I have the Nikon D80 which is 10.2 mega pixels, however I have not downloaded a single photo that has actually said it was that size, close sometimes to like 9.29 Mbyte's or something but really so what unless you plan on cropping to a really small area or getting a poster size blow up which I have done on less size photos and have turned out good. Converting is converting, the lower the conversion the lower the quality. It all depends on the application of the photo. If it will just be your myspace photo then who cares how low the quality is, it won't be noticeable in such a small format but if you want to show your photo in a gallery than obviously the better quality the better. What the hell is "future safeness" Forgive my ignorance I'm still new to photography. In the computer age you can copy a file a million million times and it will still have the same quality as the original, aside from hard drive problems or something, is this the "safeness" to which you speak?

[andrei_serbanescu]

Hi, Thomas!

Analyze this: http://secure.serverlab.net/shop/merchant.mvc?Screen=PROD&Product_Code=9060&Category_Code=MVP&Store_Code=T00107.

It's good to store and see RAW files also.

Keep in touch!

Andrei

[les_lammers]

Thomas,

 

I just got a K10D am going to put the keepers on archival DVD's on a timely basis in the DNG format (opinions welcome) just in case the computer dies. *I think* I am going to use ACDsee Pro 2 as it seems very easy to do this with that program. http://www.acdseepro.com/

 

I have other photo editors and they all have +'s and -'s but the ACDsee looks like the ticket for easy organizing and burning to DVD.

[mountainvisions]

Save RAW, Convert to DNG (using either PBL or DNG converter), convert to 16bit tiff for workable files, convert to 8bit jpeg for web files.

 

Download FastStone Image Viewer to manage your images. Excellent program and free. Much better than the overpriced ACDSee.

 

DVD's are not archival, and archival DVDs are just an expensive way to not have archival images.

 

Buy several external HD's. Backup constantly to one attached to the PC, backup a second to a non attached which if you are fearful of fire or natural disaster should be kept offsite. You can make two of these remote backups and rotate them to the remote location.

 

Finally, for a mere $5 a month you can do an online backup. Mozy, is a service with unlimited storage and bandwith that allows you to backup safely and remotely.

 

Or for $25 a year, you can buy a flickr pro account (non pro accounts are free but limited in bandwidth per month and a maximum of 200 images at a time) and upload ALL your keepers in high res JPEG (highest quality, highest pixel count) for a fairly secure storage option, plus your images can be viewed.

 

Finally, DVD is not archival. Everytime I see this I cringe. CD and DVD have a 5-10 year real world shelf life. The disk may still be readable in 5 years (probably will) but you will lose some files. Perhaps it will be your favorite image.

 

Hard drives are so cheap (500GB-750GB for $150-250 very often) that it just makes more sense to buy a bunch and mirror them.

[les_lammers]

Hi Justin,

 

I have music CD that are 20+ years old and sound as new but they stay inside. I agree that nothing lasts forever. Hard drives crash too. If you have a lot of images the online storage may be the best. I downloaded FasStone and will give it a spin. The 'new' ACDsee is not bad at all and may be all most people ever need for an image editor. It is much more user friendly than Adobe. I also use Elements and LightZone, which I like a lot, but may get ACDsee 'cause it can be had for $90 if you signed up as a Beta user. Competition is good and *most* of us do not need CS3. There are a lot of choices today. It almost gives me a headache!

[godfrey]

What to save: <br><br>

- Save all RAW image files. <br>

- Save full resolution TIFF or PSD rendered files for work you have rendered from the

original RAW files. <br>

- Optionally save lower resolution, compressed JPEG renderings as well.<br><br>

Where/how to save them:<br><br>

- A pair of external hard drives, duplicates of one another, is the most cost efficient and

usable storage facility available today. Use a backup program that you can create simple

scripts to do the archiving efficiently and consistently, on a regular basis. (For photo files, I

use ChronoSync from Econ Technologies on Mac OS X. There are many options.)<br>

- Optionally, make multiple CD-R or DVD-R backups of sets of files as well. <br><br>

The important things to remember when it comes to digital archiving are duplication and

maintenance. <br><br>

- Maintenance: Storage will change over time, you simply need to move your

data to new storage means as needs and technologies develop. <br>

- Duplication: Any individual

storage device ... CD or DVD or hard drive ... can fail and take its media with it. Two

backup copies (making three copies in toto) of important data, on *separate* hardware,

reduces risk of loss to very very small numbers. <br><br>

Don't worry too much about RAW, TIFF, PSD or JPEG formats. RAW formats for most

cameras are well known enough that multiple sources produce RAW converters for them.

TIFF and JPEG is well documented and will be around for as long as I care to think about.

PSD, Adobe Photoshop file format, is proprietary but is a derivative of a TIFF format

(Adobe owns TIFF ...), and it's well enough understood that many other applications can

read it. Translating RAW files into DNG (Digital Negative) format also puts them into a

published, standard format (Adobe also owns DNG ...).

<br><br>

Godfrey

[young_y]

Thanks for the great info. Any drives you can recommend? Also, what would you prefer, two bigger hard drives or 4 smaller drives (say, 2 500G hard drives or 4 250G hard drives)? Thanks once again in advance.

[Jochen_S]

Depending on your computers' speed you can also safe the RAWs and view them with Google Picasa (freeware). - My old boxes prefer JPGs for flipping through the files. I'm tempted to store the RAWs anyhow, since they are smaller than TIFs and I'm still hoping my digital darkroom skills might magically improve some day. For serious in depth work done in multiple shifts I 'd save as tiff or PSD usual stuff to be printed I save as JPG. To me color tweaking seems easier during RAW conversion than in other programs.

 

Storage media: I love external drives. They aren't that expensive, rather compact and convenient to handle. Keep in mind that Ds or DVDs would need pricy covers too. Here I haven't even got any DVD writer, since I can't warm up with that format. 400 something RAW files are a pretty huge lot of images, so it might take a while until you have filled one DVD and I promise your computer will crash during that moment, before you'll burn. CDs suck IMHO. Too much hassle burning and storing them and maybe even more to find the one with the image you 're looking for. Dumb as I am I can't understand how a DVD could be more reliable than CDS which are technically less sophisticated but already not really entirely practical, since here I have 3 writers among which at least one produces hardly legible results, according to some customers of mine. - I can't imagine how much copying a bunch of disks might suck 10 years from now. If I feel a urge to backup a huge HDD, I just plug the pair of them in and fall asleep. Rewriting DVDs might demand frequent disk changes, writing titles and storing the disks + probably even worse to restrain from using the computer for something else while a disk is written. - I don't mind writing a CD while I take a shower or breakfast but writing about 50 DVDs to replace a HDD? - No way.

 

A big advantage of external drives is you can fill them as soon as you come home and weed them out later.

 

Personally I'm semi paranoid about backup; I store more important stuff at least on 2 different drives. Since I own another PC for serious editing I usually download important cards to both computers.

[mendel_leisk]

Well, since the word "internal" doesn't appear once: consider internal drives. A typical pc case can has the bays, cables, power, fans and motherboard capacity for 3-4 internal drives. They are one heck of a lot faster, and less prone to heat induced failure.

[mountainvisions]

Les,

 

You have music CDs professionally made, not burned on your burner. They also are WAV files which are probably more tolerant to errors. I have many MP3 home burns and some have lasted 8 years (when I first started buring MP3) and others have lasted as little as 1 year.

 

I had a backup Photoshop disk that I needed because my original is in a box in the garage but the backups are up here. I loaded the disk and it couldn't be read. less than 3 years old.

 

Don't believe me if you choose not too, I'm just a blowhard, but the data can be found all over the web from people who make a living selling photos and who cannot afford to lose even a few images.

 

And a great example, I'm in B&H buying a Domke insert and a guy ask a sales rep if he thought "Archival" DVD were a good idea, he smiled and said, a good idea to spend money. Then he followed by saying he didn't think they were any better than regular DVD and didn't recommend them.

 

In one test I read a professional photographer openned up 10 disk at random that were 5 years old, he found at least 1 file errror on each disk.

 

Like I said, you'll be able to use the disk for quite some time, the file error might just be your favorite photo or it might be a bracket that you really don't need anyway. If this is something you are willing to live with, and you don't mind spending a fortune on small and virtually useless DVD (8 GB of double sided dvd doesn't really store much in 2007 image sizes and 4.7 GB single sided barely stores a single card) don't mind the time required to back up individual DVDs rather than have an automated program copy to a disk while you sleep, than I say DVD's are the way to go. For everyone else, I'm a fan of daisy chaining portable hard drives for the most economical backup.

[michael_kuhne]

That is bad news to me regarding DVD's being vulnerable. I have numerous music CD's burned some years back on an outboard home burner, as well as on my computer's burner. They still seem to sound fine. They are regular full size CD; I don't do MP3.

 

I have a digital video recorder with its HDD, which also has a DVD burner, and I am in the process of copying my video tapes onto DVD format for more compact storage. I was also thinking the DVD copy would be LESS subject to deterioration. Perhaps the old analogue tapes were a better storage medium after all, as well as 35mm negatives and transparencies. Ah, convenience and progress...

[mc2imaging]

I used to build million dollar RAID systems as a contractor for IBM. We used Seagate

drives.

They have the best warranty for a reason. Backing up yer stuff is not a time to go cheep.

Get

yourself 3 or 4 FreeAgent drives (I'm using 500's). Mirror 2 of them and keep everything

backed up there. On a regular basis (weekly, bi-weekly, or monthly) copy your master

files to

one of the extras and put it in a safe deposit box at your bank. Rotating two spares will

save

you trips if you have one to drop off when you pick one up. If you had a fire, tornado,

earthquake, burglary, etc., a mirror wouldn't be enough. Get copies off site.

 

I use Aperture on my Macs to handle most of my post-production (including RAW

processing). I use the 'Vault' feature for maintaining backup files, and I use a free little

scriptable utility called Carbon Copy Cloner to clone my files. Apple's Disk Utility makes it

easy to set up a RAID.

 

If you must rely on optical discs for back-up, burn at the SLOWEST speed possible, this

will produce crisper 'dots' on your disk and more tolerance as the dyes shift and fade.

 

FYI, shooting RAW will also slow your camera down (more to transfer to and from the

card), so if speed is essential, you may want to shoot jpeg from time to time. As long as

your metering is spot on, you will still get beautiful jpegs.

[mc2imaging]

I used to build million dollar RAID systems as a contractor for IBM. We used Seagate

drives.

They have the best warranty for a reason. Backing up yer stuff is not a time to go cheep.

Get

yourself 3 or 4 FreeAgent drives (I'm using 500's). Mirror 2 of them and keep everything

backed up there. On a regular basis (weekly, bi-weekly, or monthly) copy your master

files to

one of the extras and put it in a safe deposit box at your bank. Rotating two spares will

save

you trips if you have one to drop off when you pick one up. If you had a fire, tornado,

earthquake, burglary, etc., a mirror wouldn't be enough. Get copies off site.

 

I use Aperture on my Macs to handle most of my post-production (including RAW

processing). I use the 'Vault' feature for maintaining backup files, and I use a free little

scriptable utility called Carbon Copy Cloner to clone my files. Apple's Disk Utility makes it

easy to set up a RAID.

 

If you must rely on optical discs for back-up, burn at the SLOWEST speed possible, this

will produce crisper 'dots' on your disk and more tolerance as the dyes shift and fade.

 

FYI, shooting RAW will also slow your camera down (more to transfer to and from the

card), so if speed is essential, you may want to shoot jpeg from time to time. As long as

your metering is spot on, you will still get beautiful jpegs.

[les_lammers]

Justin,

 

Lighten up bud. Nothing personal intended. I'm 55 and digitally challenged and have no MP3 music. I still play LP records too. When you realy think about it off site back up is best.

 

Check this out:

 

http://www.conservationresources.com/Main/section_6/section6_12.htm

[andrewg_ny]

I am generally in agreement with using additional hard drives to keep data backups. If you feel you must use optical drives as well, <A href=http://andrew.andrewandangela.com/photography/2007/01/04/how-to-choose-cddvd-archival-media/>there was an article I found and posted about</A> several months ago. In addition to the other recommendations, prefer write-once media to rewritable media.

 

<P>Also consider that you can buy 500GB external hard drives now for <$150. That cost is half what it costs for the 106 DVD-R media @2.85 it takes to match that capacity. Furthermore, the DVD's are considerably less convenient in use. For better safety, get two of the hard drives.

[Jochen_S]

Mendel, I'm baffled. Maybe my PCs are too old to pass as typical? - Yes, I build the 2nd HDD into everyone. But already trying out a 3rd SCSI disk, (since I only had IDE data cables for 2) was too much for my power supply. And also: many people use labtops these days... I kept the internal disks on my workstation PC small, to force myself into backup routines.

 

Matthew, thanks for sharing. - I wish I understood how to mirror on PCs with inexpensive gear, since there are still gaps in my lens line. - I agree totally on your experts POV about which HDDS to buy, just the RAIDs seem somehow too pricy compared to simple disks.

 

Just to mention it on the side: Some supermarket sold a batch of decent HDDs in bad external cases a few years ago, so I ended building disks into cases I got seperately.

 

I can agree too burning as slow as possible from personal experience, my worst writer is readable then. OTOH I have a 1-2x speed writer too and forcing a weekend's harvest through it is no fun, but it's capable of copying cards on the fly, which the faster ones aren't.

 

For those without bank vaults: Lockers at work or drawers at friends' or relatives' are a alternative.

[young_y]

I have been using a couple of older external hard drives of smaller capacity connected to a laptop. Also, I have heavily relied on DVD-R. Thanks to the posters here, I ordered a 500G drive a couple of days ago. I should have read this thread again before I made the order, though, as I got a Western Digital instead of the Seagate recommended here. :o) I am planning to add one more big one once I ran out of space in the smaller external drives. Hopefully, I am better protected. Thanks once again.