Why is Apple Aperture not compatible with DNG file.

Aperture will only read DNG files from some camera but not DNG converted

files. I systematically convert all my RAW files into DNG, first as an

insurance (I thought) that these files are manufacturer independent, second

because the lossless compression means smaller files.

 

If Aperture would accept DNG files, it would de facto be compatible with all

digital camera from most manufacturers.

 

Anyway, in the absence of a DNG, I will stick to my old good iViewPro 3!

DNG is an Adobe format, and Adobe is a competitor for Apple. So there isn't really a big surprise that Aperture doesn't support DNG.

 

Does Aperture have curves?

Yes, DNG is an Adobe Proprietary format. But Adobe and Apple are not exactly competitors. Aperture supports import and export of Photoshop images as well as allows for opening files in photoshop for more advanced editing of individual images. They are great companion programs. Aperture is, however, a competitor with lightroom.

According to Apple: <I>DNG files must be generated by the Adobe DNG Converter, with the "Convert to Linear Image" option turned off, and created from RAW formats that are otherwise already supported by Mac OS X 10.4.3 or later.</I><BR>Don't know if that helps but it is from the horse's mouth.</BR>

Ace is right. The first thing to question is the process (or tool) you are using for DNG conversion. Although it is a standard, not all DNGs are identical. Like TIFF, JPG, etc. there are many options that make them incompatible.

One of the reasons I like aperture so much is that I can convert my 10MB E-1 raw files into

4.5MB DNG files and use those.

 

So it is probably the settings you are using to generate the DNG files, as suggested above.

Bas: Are you converting your RAW files into DNG within Aperture?

 

I was about to get Aperture when I saw on Apple website that Aperture does not recognize DNG format, unless it is generated by the camera (like the Leica DMR). A Canon 5D RAW converted into DNG will not be recognized.

 

I am still confused why Apple cannot read these files, other non-Adobe RAW converter read them without any problem.

Aperture only supports .DNG files of RAW files for those cameras that it supports RAW. Does that make sense? No. But, if your camera's RAW isn't supported then a .DNG of that RAW won't be supported either. This has been discussed ad naseum on the Apple discussion pages for Aperture.

 

It does support supported .DNG files that are converted using the option mentioned above.

 

I suggest heading to the Aperture discussion pages to learn the rest of the story.

Bredd, This is very clear, thanks. But if I did not use that option, I am stuck with a stock of DNG file I won t be able to use in Aperture.

" systematically convert all my RAW files into DNG, first as an insurance ..." What insurance? So they will be read in various programs?

 

Why not just convert them to Tiff's as that format is pretty much universal.

Yes, I like the idea DNG files can be read in any programs, that it provides a standard for all RAW format.

 

I do most of my editing on the RAW file, and if need be I can benefit from improvement in RAW conversion (something that would be lost with a TIFF file), also the size of TIFF files are huge compared to DNG.

As Brett said, Aperture's implementation of DNG format compatibility is limited to only the

RAW format files they already support in native form.

 

Personally, I think this is just crappy DNG support on Apple's part. I don't like the way they

are supporting RAW format processing at all, to be honest. That's why I've not been

interested in Aperture at all, despite many excellent looking features.

 

Godfrey

Samir, Godfrey, and Brett, you all fault Apple's support of DuNG. Hard as this is to imagine, it's not Apple's fault, it's just how DuNG works.

 

Brett pondered "Aperture only supports .DNG files of RAW files for those cameras that it supports RAW. Does that make sense? No."

 

Actually, it makes perfect sense. DuNG is just a wrapper around raw data. The program opening the raw file needs to know what to do with that raw data. This can be simple, like needing to know the how to convert a particular sensor's raw color to a usable color space like sRGB or Adobe RGB. Any really good raw file processor does this in a way that goes beyond the color information in a DuNG file. Phase 1 Capture 1, for example, uses a pair of space to space interpolators representing extremes of the color temperature curve, and interpolates between the two interpolators. A camera's sensor actually has to be measured, and its spectral curves incorporated into Phase One's database before this will work. So, unless Phase One has actually had a particular camera in their lab to measure the sensor, they can't convert its raw file. So, if the DuNG came from a camera unknown to Phase One, Phase One can't do a thing with it, unless the DuNG actually contained color measurements done Phase One's way.

 

That's a simple aspect of the problem. There are more complex parts. Imagine a bunch of camera's raw data as a stack of books, written in different languages. You've got books written in French, English, Polish, German, Italian, Russian and Chinese. You go out and buy a bunch of Hello Kitty book covers, and cover each book so they all look alike. But now you try to open one up and read it. Can you read French, English, Polish, German, Italian, Russian and Chinese? If not, then just having the same cover on the books doesn't do you a DaNG bit of good.

 

Raw files are the same way. You can't process a Fuji S1, S2, or S3 raw unless you have an algorithm in your raw converter to deal with the way the Fuji Super CCD rotates the Bayer matrix 45 degrees relative to other sensors. It's written in Russian.

 

You can't open a Fuji S3 raw unless you also know how to combine the outputs of the S "highlight" and R "shadow detail" pixels into one coherent image. Parts of it are written in French.

 

You can't open a Leica Modul-R or M8 file unless you have an algorithm for removing color moire, because the Kodak sensor used in those cameras has no anti-aliasing filter. It's written in Polish.

 

You can't open a Nikon D2X or D200 raw file unless you know how to deal with a sensor that does white balance by having two independent multipliers for red and blue channel gains (one hardware and one software). It's written in British English instead of American English, and they spell color "colour".

 

You can't open a Sigma SD10 or SD14 raw unless you have a routine to deal with simultaneous loss of all channels if one channel clips, and can work the color math with a space to space interpolator because the sensor isn't sufficiently colormetric for a matrix multiplication to work well. And you need to know about things like sharpening only the red channel because of diffusion in that layer. Written in Mandarin Chinese.

 

It's not just Appleture that works this way: most programs that support DuNG do the same thing: unless they can already read the raw file, they can't read it if it's wearing a DuNG cover. In other words, that's just the Hulk with a fin on his head.

Joseph,

 

Thanks for the very useful explanation. My understanding was that DNG would translate all

these books into a common language, so the Polish, English, French books be translated

into Esperanto. If what you say is correct, then what's the point of DNG? Why has it been

recommended by the guru of digital capture, in particular Bruce Fraser in his excellent

Camera Raw with CS2.

 

Now, I take your point, my files are all from Canon 10D, 1D Mark II, Canon 5D, and

Panasonic Lumix, all recognized by Aperture. However, some have said above that to be

recognize by Aperture the Convert to Linear Image option must be checked. Why?

Checking that option results in much larger file, defeating one of the benefit of the DNG?

And again Raw Developer (from Iridient Digital) does open all my DNG without any

problem, so why can' t aperture do it as well?

One more element. I tried to import one of my DNG file into iPhoto and it worked well. The

file was converted into a JPG. Since the RAW engine in iPhoto and Aperture are within the OS,

I guess Aperture should be able to read them and work normally. Right?

Sorry, but Joseph is plainly wrong. This is a legend that lives hard with Apple zealots at

this moment. I use Aperture every day, I like it but the lack of real DNG support is a

problem.

 

DNG is not just a wrapper. If it was, there would be no need to have an option to imbed

the original raw in it.

 

Also, if you have a camera not supported by LightZone, convert its raw in DNG and

LightZone will make a nice job of it. Silkypix as well.

 

DNG does include precise camera profiling and the conversion process is far from a trivial

copy. The DNG convertor from Adobe get updated as ACR gets updated because it

requires good knowledge of the original format to produce the proper DNG.

 

Apple could implement complete DNG support and it would make life easier for many of

their customers. It would also turn many more people into potential customers.

 

The decision is political, not technical.

Samir, goad you found the discussion useful. As to your question of "If what you say is correct, then what's the point of DNG?"

 

It was one of several attempts by Adobe to try to divert attention from some very strong negative press that they have received lately, mostly concerning their predatory intellectual property practices in the arrest of Dmitry Sklyarov. "See, we're not people who prosecute someone for fair use, we're really the 'good guys', promoting open standards."

 

Anyway, contrary to the claims of their (most likely paid) zealot, DNG files converted from raw by Adobe's DNG converter don't contain any "profiling" information beyond what's already in the camera maker's raw file. That's typically 11 numbers, (9 in a 3x3 matrix, and 2 white balance scalars for red and blue). So, if you want to do color the much more accurate way Phase One, Bibble, (and presumably, Aperture) do, then you have to have access to actual spectral response data, and the only way to get that is to take a digital camera, put it on the optical bench, and feed 31 different colors of light into it with a device called a monochromator. This is then typically used to create a space to space interpolator (the method used in the ICC profile), most commonly with 4913 values. That takes a lot of math, so it's typically only done once, by the person writing the raw processing software, not on the fly from the 31 spectral values.

 

So, unless Adobe has gathered up every single camera known to man, done this lab testing, generated the profiles, and built them all into the single executable of the DNG converter, it most certainly does not contain all the information that you need to actually process a raw file.

 

That's also why iPhoto worked and Aperture didn't. iPhoto doesn't do any color math beyond what's in the OS, and that works by simple 3x3 matricies. Aperture does more sophisticated math. Time them both, you'll see quite a difference.

Joseph, again I still don't get it: RAW Developer works perfectly well with all my DNG-converted RAW files. If the people at Iridient can do it (and as I said they do a pretty good job), why can't apple do it?

Thx

Not only Raw Developper, but Silkypix and LightEone too.

 

What Joseph seems to forget is that Adobe has to be able to interpret the raw files quite well

in order to make ACR. It is that knowledge that is used in DNG Converter.

Samir,

 

Sensors don't produce values that are exactly equivalent to the way the human eye senses color. There are all sorts of ways to attempt to mathematically transform the values from the sensor into "colormetric" (or "human eye") values. The simplest of these is the linear technique that has been in use for decades:

 

1) Correct the color temperature by multiplying the sensor's red value by one gain value, and the blue value by another gain value. All raw files contain these scaling values, so the DNG converter can also put them into the DNG file.

 

2) Transform the color temperature corrected red, blue, and green from step 1 into a more "eye like" value by multiplying it by a 3x3 matrix, determined by the camera manufacturer. These values are written into the raw files by some camera, but mot by others. Since there aren't many of them (a 3x3 matrix is only 9 values) they can be worked out by trial and error to produce acceptable looking results. This is the technique that Dave Coffin used in dcraw, the popular open source raw file converter. (Adobe acknowledges that they use some of Dave's work in their own raw converters). Dave didn't have to work out the values for hundreds of cameras by himself, many volunteers tweaked values for their cameras and sent them to Dave for incorporation into dcraw. It's interesting that even for some cameras that do put the 3x3 matrix into their raw files, Dave and the volunteers worked out values that work even better. Since the values are in many raw files, or available for camera that don't put them into raw files from Dave's efforts, or possible to work out with rather simple experiments for raw files not covered either by the original manufacturer or Dave and friends, they're easy for the DNG converter to place in the DNG file. And the data is small, 9 values per camera, so even the vales for 200 cameras is only 1800 numbers, 8 bytes each for floating point, for 14,400 bytes. That's just 0.3% of the 4,788,224 byte DNG converter version 3.4. And the math for this simple technique is pretty fast and efficient, so if you want a raw converter that runs quickly, it's the way to go.

 

Unfortunately, this simple "two gain, then 3x3 matrix" linear technique has its limits. Most sensors are not sufficiently "eye like" that a simple 3x3 matrix multiplication gives you accurate colors. The breakdown is most severe in the crossover regions between blue and green and between green and red (where skin tones fall) or at the very ends of the spectrum in the deep reds, deep blues or violets, and in purples purples (blends of red and blue), all important colors to fashion, product, and nature photographers. You end up in a situation where you tweak the color for a neutral white, and the skin tones go pink. Get the skin tones right and blue clothing is too purple or the red lipstick too orange. The two scale value for color temperature is only an approximate technique, and increases the errors in the crossover regions and the extremes.

 

The writers of the best raw converters have long understood this. That is why top end raw converters like Phase One Capture One, Bibble, and now, apparently Aperture, go to great lengths to do color by more sophisticated non-linear math. Unfortunately, that kind of math needs more data than just a 3x3 matrix. It can be done with curve fitting, splines, or "3 space to 3 space interpolators".

 

The space to space interpolator is the most popular technique, because it's the model used by the ICC (International Color Consortium) for profiling everything from cameras to scanners to printers. It consists of lookup table containing thousands of sensor color estimates and an interpolator to map from any sensor values in between these estimates to the proper value in a human eye compensated color space like LAB, sRGB, or Adobe RGB. Some camera makers also do this type of math in their own raw converters, ignoring the 3x3 matrix values in their own raw files in favor of an interpolator.

 

Such a raw converter identifies the camera from the raw file (there's always some "magic numbers" somewhere in a raw file that lets you identify the camera that produced it, look at the camera ID part of dcraw for an example) and then ignores the 3x3 data in the raw and uses the interpolator values in the raw converter's library.

 

Generating the values for a color interpolator is not easy, and therefore, it's not free. I have a small lab with an optical bench setup capable of doing a sensor spectral sensitivity sweep and software I've written in C and Matlab that matches spectral data for about 4000 plant and animal samples, commercial dyes and pigments, cosmetics, etc. to a sensor spectrum and generates interpolator tables for different color temperatures. I have, in the past, measured cameras under contract to one of the raw converter manufacturers (an NDA prevents me from saying which one) and generated their interpolator tables.

 

Aside from being expensive, these tables are huge. Even with data compression, they're a bit big for Adobe's free download DNG converter. So, the Adobe raw converter doesn't contain a giant library of such things. That means if Phase One wants Capture One to only do raw conversion with interpolator tables, then Capture One can only do a raw conversion for a camera that Phase One has measured themselves or farmed out to a lab like mine. So, if they have the values for a Nikon D200, then they can process a D200 file, whether it's a "native" Nikon NEF or it's been converted to DNG. If Phase One doesn't have values for the D80, then converting the D80 NEF to DNG doesn't make the needed data magically appear.

 

Now, I have no idea how Irident is handling color. They may only use 3x3 matrices, and those will always be in a raw file, whether native or DNG. They may have an interpolator library for some cameras, and fall back to 3x3 for the ones that aren't in their library. This would make DNG work for any camera, but it would work better for cameras that are in the interpolator library.

 

But other manufacturers drew a hard line, and would rather not do a raw conversion at all, if they can't do it their way. No fall back to 3x3, and converting to DNG won't allow the raw processor to process a file that isn't already supported in its native form.

 

It's interesting to note that Adobe's own "Camera Raw" raw converter apparently uses the 3x3 approach, but admits to the limitations of this approach and allows you to perform a calibration step where you adjust another seven non-linear hue, saturation, and tint parameters to improve color accuracy.

 

Color is one reason a program might only support DNG only for cameras that it supports natively. I went into the other reason earlier in this thread: many cameras require some exotic algorithms to deal with their particular sensors:

 

45 degree rotation for any Fuji DLSR or P&S with raw files.

 

Combining the S & R pixels of a Fuji S3 or 700.

 

Reducing moire in Leica Modul-R and M8 which have no AA filters.

 

Several unique algorithms for Foveon sensors in Sigma SD9, SD10, and SD14: blown highlight handling, non linear color, red channel sharpening ("memory colors" don't hurt, either)

 

Non-rectangular pixel handling for Nikon D1X.

 

Horizontal color shift ("Italian flag effect") compensation for Kodak 14n, SLR/n, and SLR/c.

 

And that's why the DNG Zealots don't tell you the whole story. Yes, converting to DNG can let a raw converter support a raw file that it doesn't support as a native file, provided you define "support" as using just two decade old linear color, not removing the moire in a Leica file, not utilizing the high dynamic range of a Fuji sensor, having the left half of your Kodak image green and the right half magenta, etc.

 

Personally, I call that kind of support...

 

DuNG

Thanks for that explanation. I stand corrected. And I've learned a lot.

 

But now, it still appears to me that when Silkypix was not supporting the Sony DSC-R1 it

did a great job out of DNG files produced by Adobe DNG converter from the Sony SR2

files. So does now LightZone. Maybe it 95 or even only 90% correct. But since when 90% is

better than 0%?

 

I think many customers would appreciate it if Apple supported DNG as well as Silkypix and

LightZone, it would help a lot, especially for cameras they will never support natively.

Thanks, Joseph. It's a great public service to get this information out into the open! Much appreciated, and has certainly given me a much better understanding of DNG, to enable better evaluation of whether or not to use it.

Joseph, thanks a lot...I better understand that process. The bottom line is that the Aperture

RAW converter might be of better quality and more color accurate than the Lightroom/Adobe

Raw converter?

Stephane, you're quite welcome. And I agree with you. 90% is better than zero percent. If I were Phase One or Apple, I'd opt to support as many cameras as I could, as best as I could. I'd probably publish a list of cameras that could be supported 100%, and a note that anything else could be supported at the 90% level, without fully calibrated color, and possibly without optimal results because some algorithms might be missing or insufficiently tuned. I'd even try to do this both through DNG, and through reading as much of the raw as possible via something like Dave Coffin's library (which he says supports the raw files from 231 different cameras).

David and Samir, you're welcome.

 

Samir, yes, that's the bottom line. The approach Apple is taking has the potential to outperform the approach Adobe is taking. I'm not saying they've achieved that, just that the potential for higher performance is there.