Why sidecar files?

I've been looking at replacements for my current photo management software, and one thing that has me puzzled is why sidecar files are still in wide use? Embedded XMP metadata has been around for many years, and yet I still see Capture One, On1 Raw, and Darktable all using sidecar files. Sidecar files complicate file management since they can easily be lost during copy or rename operations (outside the application), and they don't provide the performance advantage of a database approach.

 

I've been in IT for a long time, so it's not as if I haven't seen obsolete practices hang around for years just because 'that's how we've always done it', but is anyone aware of actual advantages to sidecars?

At least for Adobe products, sidecar files are in use because they will not write that metadata into proprietary raw files. It’s not that they cannot, they choose (some would say wisely) not to. So with ACR, that’s why. For Lightroom, you can write this to the catalog. Or even better, just write this into a DNG.

I use Lightroom now, so catalog/in-DNG is what I'm used too. I was surprised to look around at alternatives and find sidecars in such wide use. I'm leaning towards On1 at the moment, but sidecars feel like a real step back.

The sidecars are great. I don’t want any random raw editor making changes to my raw files - suppose it commits an error and corrupts the file? Better to have a sidecar, and if I want to change from, say ON1 to Capture One, leave the sidecar there in case I ever need it. ON1 has a particularly nice system for anybody using centralized, cloud or synced storage and more than one computer. If you make edits on your laptop in the field and later open the file on your home computer, you get the edits.

I'll be going Lightroom 6 -> On 1 (most likely). I've looked at:

• Capture One 20 - I just can't stand the UI
   
  
• On1 Raw 2020 - current lead candidate
   
  
• Aftershot Pro 3 - wouldn't open DNG files from VueScan, so mostly useless to me
   
  
• Darktable 2.6 & 3.0 - powerful, but unstable and weak collection management tools
  

I'll be looking at DxO PhotoLab 3 over the weekend before I decide.

I'm using Capture One, and the interface is different but once you get used to it it's great. E.g. you can make tool panels for different workflows, so I have one for working with camera-scanned negatives that has all the tools I use for that, in order. It does actually use sidecar files, which are best used when working in Sessions - they're saved in the Session folder hierarchy and can be found and copied with the file, and since they're in the same folder set as the images and the rest of the support files it makes taking sessions from one computer to another easy. But the main reason I like it is that it does the best job on Fuji raws. If you don't use Fuji it's not necessary.

 

On1 is great if you like the tools in the Effects panel. There is some excellent stuff in there. One thing I especially like is that each effect has an opacity slider, so you can tone down an effect with one adjustment. You can also do some tricks like add a B&W effect, tweak it and then make it semi transparent, for finely customized low saturation. It's also perfectly happy with you leaving your photos organized in folders and just browsing to the folders - no import needed.

Check out DXO Photo Lab 3 with the NIK Collection. I am in the process of evaluating it as a replacement for my free Nikon software. ON1 now supports dual monitors if that feature is important to you. Within the Adobe family I prefer ACR and Bridge and not Lightroom. Photoshop is just too complicated to use.

 

I think another reason why side car files are still supported is that many use more than one program and sidecar files can be used by different programs.

I've narrowed down to DxO or On1, each of which has it's quirks, not surprisingly, but I could be happy with either (I think). Just for fun, here is an uncorrected crop (CR2 opened in Lightroom, cropped, and exported with no corrections) along with quick (and a bit heavy-handed) edits in DxO and On1. They're not exactly the same, but close enough that I'm focusing more on UI and support for my workflow that on features.

 

 

 

Before I retired, I worked for Apple, mostly on stuff related to PDF. We had a big internal debate about using sidecars vs. embedding for dealing with user annotations. I favoured sidecars for various reasons, and I still do. Although PDF is very well specified, the mechanism for modifying existing PDFs can lead to the document constantly growing in size. For example, if you delete an existing annotation and replace it with a new one, it is likely that the old one will not actually be deleted. From a programming point of view, you can delete the old annotation only by completely rewriting the document, and this can be an expensive operation. So you might decide to use PDF's internal update stuff, but then the client apps probably have to offer its users a way to rewrite the file when bloat starts becoming a problem.

 

In the case of image files, there is a benefit to leaving the original RAW file pristine. There are always risks when a file is modified, including the possibility that some future app of yours does not understand the changes made by the old app. Of course, if the developers are careful to follow the XMP spec precisely, you are probably OK, but for me (with lots of experience with programmers), I would much rather not take the chance.

 

Of course, sidecars come with there own risks, but I would rather loose my edits than my image.

At least for Adobe products, sidecar files are in use because they will not write that metadata into proprietary raw files.

For Adobe products, you can use either sidecar files or one database file. Personally, I don't understand why one would you many sidecar files vs. one database file.

For Adobe products, you can use either sidecar files or one database file. Personally, I don't understand why one would you many sidecar files vs. one database file.

It makes many workflows easier. Working on more than one computer, migrating to a new computer, sharing work with colleagues, archiving old projects, bringing the most recent projects with you on a laptop with limited storage, loading travel photos onto a laptop and bringing them to a desktop when you get home...

For Adobe products, you can use either sidecar files or one database file. Personally, I don't understand why one would you many sidecar files vs. one database file.

Simple. For some, sidecar files backup quickly as they are tiny. But you alter ONE image in a database, the ENTIRE database has to be backed up. Not a deal breaker for those of us who’s backups take place automatically, unattended but a consideration for those perhaps with databases that are very, very large, updating to the cloud and not wanting to update so often when one tiny sidecar could be backed up instead. Something to consider understanding..... ;)

Edited January 12, 2020 by digitaldog

Simple. For some, sidecar files backup quickly as they are tiny. But you alter ONE image in a database, the ENTIRE database has to be backed up. Not a deal breaker for those of us who’s backups take place automatically, unintended but a consideration for those perhaps with databases that are very, very large, updating to the cloud and not wanting to update so often when one tiny sidecar could be backed up instead. Something to consider.....

That’s a good point. And having the sidecar and raw file back up in the same place is way easier to restore from.