CR2 File Format Will Go Away, How To Save Files

[eddie g]

All our eggs are in one basket by relying/assuming the CR2 file format

will always be accessible by software applications 40 years from now.

I realize that we can't predict the future with a crystal ball,

but...what is the best way to protect ourselves?

 

How do we preserve the "rawness" of CR2 while still saving our files

in a format that has highest probability of being a dominant format in

40 years (or 120 years for our grandkids)? I want to preserve the

"rawness" of CR2 because software in the future will be able to do

even more amazing things with image processing. Also, if my prints

get damaged or lost or faded and/or future printing technology is 100

times better, then I'd like the option to revert back to the pure

unadulterated "rawness" of what my camera originally captured.

 

I'm thinking that maybe the solution is to do a batch conversion of my

20,000 files into the latest file format each time a "new" promising

format is introduced (and also keep the CR2). So I could end up with

5 versions of each file. But my two priorities are in question each time.

 

1) Use a file format that will preserve the "rawness"

2) Use a file format with highest probability of being dominant in the

long run

 

If you are in the software/computer industry, please identify yourself

as such because your thoughts on this matter are deeply appreciated.

[chiswick_john]

Adobe have such a format which they are trying to convince the industry to adopt. Who knows how long it will be around and what the future holds in terms of anything. What is certain is that potentialy digital information has an infinite life and there will be better RAW processing apps in the future that will extract the information in improved ways. The only certainty with film is that it will decay, fade and be lost in the distant future.

[gavinbell]

You also need to think about migration of the backup medium. In the 20-40 year timeframe DVDs and compact flash will become obselete, in the nearterm your average DVD-R will slowly decay.

 

Stewart Brand's Clock of the Long Now is a good read on this kind of stuff, see www.longnow.org

 

cheers

 

Gavin

[jim_larson1]

I suppose you would convert each RAW file to a TIFF. . . and the backup media issues are very, very real.

 

Also. . .keeping an old computer or two with the old software couldn't hurt either.

[cfimages]

The best bet, for the moment, is probably to do a batch conversion into Adobe's DNG format. AFAIK, one of their aims in creating it was to have a universal RAW format, so that would suggest that it'll be around for the foreseeable future, at least.

[cenelsonfoto]

Film.

[chiswick_john]

"Film" - please expand on this - do you mean writing digital images to film? if so what are the archival qualities of film? If you mean abandon digital in favour of film how long do you think you are going to be able to buy film and get it processed. Or is this just a glib and pointless one word 'contribution' to the debate?

[christopher_bibbs]

One thing to note about Adobe's DNG format is that it is an open standard. Nothing hidden, encrypted, patented, or otherwise restricted. In a practical sense that means you aren't locked into Adobe software and even if Adobe drops the format, it is feasible that other image editors could continue to support it in the future.<p>

There are already <a href=http://avondale.typepad.com/rawformat/2005/04/dng_support_sta.html>quite a few</a> programs that support it. I'm making it part of my practice to convert all my old RAW files from various cameras into DNG. It just makes sense given the state of things.

[cenelsonfoto]

Don't get me wrong - I shoot both formats and enjoy each for its own benefits, perceived or otherwise.

 

As far as digital is concerned, I shoot JPG and convert to TIFF for archiving. I gave up on fiddling with RAW. It takes too much room on my XD media. 256 meg card yields a piddling 37 images in RAW, just doesn't make sense to me.

 

TIFF, for now, is the highest quality, easily transferrable format, IMHO, and will likely remain that way until the day comes that they develop a universal standard.

[cenelsonfoto]

Oh, and as for the point about getting film and processing it... B&W film will be around for many years to come, though the vendors may change hands a few times. Fine by me. And development is handled by me, either through proprietary forumlae or homebrew soup.

[dave_nelson___atlanta__ga]

Film is not a realistic answer for most because even if the image is initially captured on film it will be scanned in, manipulated and then printed. The manipulated image is what needs to be archived along with the original.

 

I personally do not trust the DNG format. TIF is a better choice in my opinion. JPG is a close second, although you do give up a lot.

 

I prefer to save my best images in a couple different formats (CR, CR2, TIF, PSD, JPG) and save them on a couple different mediums and store them in at least two locations, currently they are all stored in three counting my web site. I can't afford to do that for all of my pictures, but I do it for the 2000 or so "best" each year. As my skills increase I expect that number to grow exponentially, but as that happens, the cost of storage will decrease.

[ozone42]

I sincerely doubt *ANY* existing file formats will be around 40 years from now. Maybe not even 20. I wouldn't worry about formats, just keep in touch with what software is available and convert as new formats are brought into play, or keep a copy of something you know handles the format.

 

I'd worry a lot more about archiving the data than what format it's in. A good quality DVD kept in a cool dry and dark place will probably last 20-40 years but if it's not stored carefully you'd be lucky with 10. A lot of people are buying spare hard drives to store data to and then stow away, or resorting to high quality tape backup.

[awindsor]

I am far more worried by the problem of back-up media than file format. If simultaneously all vendors decide to stop supporting CR2 I might be tempted to take the converters I already own and convert all my raw files to DNG or whatever is the flavour of the day but it strikes me as a complete waste of time until then.

 

Whilst it is true that CR2 is not an open format there are paradoxically open source converters (dcraw) the code for which has been used in many other converters.

 

Writable CD/DVDs are not good archival media and the best bet seems to be using external hard-drives. Keeping good back-ups then requires multiple hard-drives (mirrored and stored in different locations). Single RAID arrays are very stable until your house burns down. No guarantee there will be hardware to connect devices to in 40 years. Better archive a computer with your hard-drives.

 

Of course these are vulnerable to electromagnetic pulses and I may not have my files after a global nuclear war (about the only thing I can think of that might simultaneously destroy all my converters, all your converters, and convince all the manufacturers to simulataneously stop supporting CR2).

[dogbert]

In 40 years time there will be so many digital images available that are so much better than what we are doing today that no one other than our grandkids will be interested in reprinting them. Seriously what percentage of the images taken 40 years ago are being reworked today? Probably less than one in a million. The main people that may be interested are your decendants. Leave them a good looking print in an album. They won't be going through thousands of images on computer file.

[cenelsonfoto]

This is beginning to sound like folks going mad over toilet paper during the Y2K panic... look, archive in the best possible (read: highest quality, most accessible format) that suits your means and needs. When the option to convert to newer or better technology appears, get it converted, makign sure, of course, that the new tech provides significant leap in storage cap./longevity/etc.... if nothing appears to significantly improve over the current scheme, stay put.

 

That simple. I would suggest at least one level of redundancy to avoid catastrophic loss, ie. floods, etc. Burn to DVDs, keep one set in your safedesposit, or secured elsewhere miles away.

[PuppyDigs]

"Also. . .keeping an old computer or two with the old software couldn't hurt either."

 

I recently dug through Mac graveyards and found a working 1989 Mac SE with 40MB hard

drive. I had thousands of graphics files I produced from 1984 to 1992 using now defunk

software. I eventually was able to convert most of them to PDFs but only after struggling

endlessly through a maze of old software, storage mediums and computers (3 Macs: OS 6,

8 and 10x!). It was ficken hell! I can only imagine what a bitch RAW, TIFF, etc. will present

a few decades from now.

[miles_feigenbaum___dallas_]

There sure is alot to think about here. I for one am going to pop an old 8-track tape in and listen to some tunes while I sort through some of the Kodachrome 25's from my childhood that I need to dupe for the rest of the family.

[jeff medkeff anchorage, a]

<p><i>I personally do not trust the DNG format. TIF is a better choice in my opinion.</i>

 

<p>The problem is that you "give up the rawness" (as the original poster put it) with TIFF as it is standardly implemented. It buries the raw capture data under an avalanche of interpolated values, which is what you do not want to do if you are taking the trouble to shoot raw in the first place.

 

<p>Eddie, a small part of my job is writing imaging-related software for scientific applications. I don't know that this should cause you to respect my opinion any more than others'. My take is this: don't do anything until your files begin to show <b>some</b> obsolescence. Once they have begun to show some obsolescence, but before they are completely unsupported, convert them to the dominant alternative available at that time, and archive both formats.

 

<p>We are currently doing this with D30 and D60 images. The D30 raw format was unsupported by a recently released version of Canon's converters and SDK; however, we can still run the older versions of the raw converters on current hardware and operating systems, and there are still third-party converters that support these files in their current versions. To my way of thinking this constitutes "partial obsolescence." So without throwing away the originals, we are converting them to DNG and archiving those as well.

 

<p>Why did we choose DNG? Because it is the dominant alternative format for storing mosaicked (that is, unconverted) raw camera output <b>at the present time</b>. No common implementation of TIFF does this, but <a href="http://avondale.typepad.com/rawformat/2005/04/dng_support_sta.html">almost everyone of significance in the RAW industry</a> supports DNG. We can't predict what will happen in ten years, so we go with what works for us this year.

 

<p>If in the future DNG fades away, and something better replaces it, we will convert to that. OTOH, if DNG lives long and prospers, then we'll just ride with it.

 

<p>Whatever happens, we're in good shape <i>right now</i>. If we did nothing about this issue during the next year or two, and instead spent our time fretting over DNG's flash in the pan potential, or speculating whether DNG constitutes an Adobe plot to take over the world, the chances are that our images would be at risk right now or at least in the near future.

 

<p>It is a nice bonus that the DNG format is fully documented. That means even dummies like us can write our own software to access the data if the format somehow fades away without being supplanted by some other format in the future. But this is distantly secondary to widespread vendor support in our format selection criteria. As the industry moves, we will have to move as well, and as things currently stand, the DNG skeptics have not provided a credible alternative format for our use.

[sjg]

Get hold of the source to a converter - like dcraw, and save the _source_ along with your .cr2 files. Computer h/w comes and goes as

do proprietary file formats and applications - but source is forever ;-)

 

Same goes for your OS too of course ;-)

 

Note dcraw may not be the ideal choice for use today - converts .cr2's to .png's (I think) which are huge, but as a safety net its a good option. Same holds for your OS - use windows or whatever if that

helps your productivity - but like .cr2 it may not last (who run's dos 2.x apps on NT?), have a backup plan.

[brian_mielke]

>>Get hold of the source to a converter - like dcraw, and save the

>>_source_ along with your .cr2 files. Computer h/w comes and goes as do

>>proprietary file formats and applications - but source is forever ;-)

 

Not exactly. Standards for programming languages change over time as well. If you're going to store the source for the converter, to be safe, then you also need to store the compiler/intrepreter which can run that source. But then in order to run that compiler to turn the source into something you can run, you need a developement environment. But operating systems also evolve, thus you'll need a copy of the OS which supports this compiler. Hardware also changes, thus you'll need to keep an old system around as well....

 

Try compling C code that was written 15 years ago with a modern day compiler. I do things like this at work, and can guarentee you that it won't go smoothly.

 

Personally I don't see storing the source for the converter helping that much if we're worried about 40+ years in the future. However, the spec for the file format, so a converter could be implemented in the development tools of the future, could be useful.

[jeff medkeff anchorage, a]

<p><i>However, the spec for the file format, so a converter could be implemented in the development tools of the future, could be useful.</i>

 

<p>I agree completely.

 

<p>There are a lot of undocumented bytes in my Canon raw files. Actually, I think <i>all</i> the bytes are undocumented; as far as I know, Canon has said nothing about the file format and everything we know about Canon raw files comes from third parties. But there are still a lot of words of unknown function, that people haven't been able to figure out. So it is currently not really possible to archive the file format specification; you can only archive some talented hacker's best guess about the format, assuming you can get them to talk.

 

<p>The camera manufacturers suck. They are creating a lot of unnecessary problems for people with important images that deserve a long lifetime. Canon D30 images of 9-11 (of which there are many thousands) and all sorts of technical and documentary photographs come to mind as prime examples. Know-it-all photographers, self-appointed to determine my image capture and archiving needs, haven't helped in the least. They alternately suggest utopian blind trust of allegedly benevolent camera manufacturers (who have <i>already</i> taken away file format support from their tools or gone <i>completely out of business</i>), advance paranoid hatred of the open-source and royalty-free DNG format, or conduct insane blathering that RAW is overkill and if I'd only "get it right in-camera" I could use JPEG for for every single image, past, present, and future. I've grown sick of nonsense like this and thrown in my lot with <a href="http://openraw.org/">OpenRaw</a>. Meantime, as mentioned above, I'm converting some stuff to DNG as a hedge.

[zhi_da_zhong]

<p>Think of the raw converter source as a form of documentation. It'll be useful as long

as people can

still read and understand the program. So all we need is a copy of the language spec in

addition to the source. Then some future programmer can re-implement the conversion

algorithm on whatever software/hardware platform available at that time. Having the

actual spec for the format would make things easier, but in the absence of one, storing the

source is a good idea.

</p>