Layered files Archiving(for the long term) ?

Wtih the very real expectation that I'll be moving away from software I've been using for quite a while I am questioning how to archive many many many past projects. Yes all the raw files are kept but a significate portion of my digital work is saved in layered files with the psd suffix. I guess my question is a little technical and a little philiosical. I could (over the course of the winter) go back through the folders flatten the files and save them as tiff or I could leave my archives alone and in the off chance I need an image from the past start over processing the raw file in whatever software I'm using.

What are other's thoughts on the subject of archiving layered files in an ever changing software landscape?

On the web, batch PSD -> TIFF converters are advertised for about $30 so it needn't take a whole winter. This 2007 PN thread describes setting up a Photoshop action which - via Bridge - can do the same job.

I think it depends a lot on the extent of the differences between the RAW and (flattened) PSD files and how often (if ever) you expect to use them.

You could copy the masters as layered TIFF files. The downside is the files are larger and not compatible with some programs. You may also lose the ability to edit adjustment layers, except in Photoshop.

I presume you are leaving the world of Photoshop for uncharted regions. I suggest that you stop and consider what you have invested in Photoshop in terms of work product and learning. The cost of "renting" Photoshop and Lightroom is only $10/mo, and you will be continually up to date.

Thanks, @Ed_Ingold for pointing out that TIFF does (in principle) support layered files and images can be exported from PS to TIFF maintaining layers. I don't use TIFF much and exclusively as a lossless flat image format.

I agree 100% that preserving layers between software packages either directly (export/import)) or via TIFF is unlikely to be 'seamless'. So the OP shouldn't count on layered image formats (including layers) as an archival formaT.

As I understood the OP, she/he was considering the choice between a) archiving exported flat (non-layered) TIFF files or b) relying on archived RAW files to re-create images in the future.

Personally, most of my 'archived' photo's are in (yes, lossy) JPEG format. I know I'm not going to go back and re-edit these so for me it makes more sense to archive high quality JPEGS for my 'keepers' and - after a year or two - delete the corresponding RAW and PSD files which just take up space.

Every time I get some kind of project archived, the "default", "archival" format becomes obsolete.

I still have some data on actual, physical IBM punch cards.

It was once described as consigning the project

"to the gnawing criticism of the mice"

It's nature's way of housecleaning.

As an amateur, I don't foresee any negative effects. But I personally don't consider any digital file formats (even TIFF) to be absolute and unchanging forever. As far as archiving goes, it's good practice to refresh your archives every 3-5 years.

Really did not want this to become an Adobe subscribition debate. So allow me ask a technical question. I also have been working digitally since the mid 90's and have many hard drives in my archive. If I go through all the drives opening up all the layered files, flatten them then saving the 16bit tiff and throw away the layered file. Will this have any unforeseen negative effect on the hard drive's health? What would the best practice be for making sure the hard drive is in good health. Yes I have redundant copies of all the drives.

The Library of Congress has specified TIFF as their archival standard. See: TIFF, Revision 6.0

So, the TIFF format should be around for a while.

I'm just going to make a general comment: there is not a file format in use today that can't be read in the future.

If you want a batch converter, try Automator first. It's included with OS X. I'm not sure if OS X can read PSD files, as I don't use PS. In any case, it will take you two minutes to create a workflow with Automator that converts your PSD files to TIFF. If you want me to show you what to do, feel free to ask.

I'm just going to make a general comment: there is not a file format in use today that can't be read in the future.

That should be true as long as someone remembers the file format.

Maybe you are thinking about a few years, but some people think in hundreds of years.

More than the file format, is the storage media. Will there be USB readers or SATA readers 100 years from now?

The usual system is to copy to new technology every so many years.

Hard drives continually refresh their data as long as they are running. Once they stop, the magnetic domains begin to weaken and interact with adjacent domains, causing errors. Eventually there is not enough redundancy to recover the data. I have heard the lifetime while idle is about 6 years. The mean time between failures for running drives is about 20,000 hours. However they don't all fail at the same time, and disks in redundant arrays (e.g., RAID drive) can be replaced as they fail without loss of data. Tending these archives sounds a bit tedious. Perhaps there's a future for political science and art history majors looking for work. We could also hire them to transfer digital records to clay tablets ;)

I have CDRs still readable after 25 years. Any longer and who will care? I visited an antique store a few years back, with hundreds of mounted prints once entered in photography contests. Shades of Ozymandias.

Hard drives continually refresh their data as long as they are running. Once they stop, the magnetic domains begin to weaken and interact with adjacent domains, causing errors. Eventually there is not enough redundancy to recover the data. I have heard the lifetime while idle is about 6 years. The mean time between failures for running drives is about 20,000 hours. However they don't all fail at the same time, and disks in redundant arrays (e.g., RAID drive) can be replaced as they fail without loss of data. Tending these archives sounds a bit tedious. Perhaps there's a future for political science and art history majors looking for work. We could also hire them to transfer digital records to clay tablets ;)

 

I have CDRs still readable after 25 years. Any longer and who will care? I visited an antique store a few years back, with hundreds of mounted prints once entered in photography contests. Shades of Ozymandias.

Hard drives don't refresh the data, unless specifically done by the computer. Even more, sector headers and servo data are never rewritten.

(Some drives you can low-level format to rewrite the sector headers. Servo data is only written in the factory.)

Powered down, they tend to fail for reasons similar to cameras. The bearing oils settle, or the disk surface lubrication gets sticky.

A common problem some years ago, maybe fixed by now, was the heads getting stuck to the disk when stopped.

The fix was to bang the drive on a table. The failure modes are mostly mechanical, not electrical or magnetic.

Yes in a RAID array they should fail independently, but there is some probability of a second drive failing during the time it takes to fill the replacement drive. Similar drive models have similar failure modes, increasing the likelihood of such failure.

RAID 6 gives you two redundant drives, reducing the probability of data loss.