Thoughts about Foveon

<p >I’ve been watching the Foveon story on and off since they made their first announcement early in the last decade. Only recently, with the introduction of the SD and DP series have I started to look more closely at the technology. It appears to be capable of producing stunning images, better to my eye than anything I have seen with traditional Bayer-pattern sensors. Some people say they can’t perceive the often-described 3-D effect, but regardless of pixel count, another controversial area, there is an unmistakeable organic quality that might be considered film-like. Delving into how the technology works, it makes sense from an engineering perspective. Three layers of pixels, one for each colour, so the image is recorded in a similar way to three-layered film. When you compare it to the way a Bayer-pattern sensor works, the latter seems a flawed concept. An image comes from a non-foveon camera via a complex processing cycle designed to interpolate colour in adjacent pixels and then blur the image with an anti-aliasing filter to reduce moiré. I’m surprised they work as well as they do. I’m not an engineer but I love what I see as ‘simple’ and ‘elegant’ designs. If you have to add further levels of technology to fix an initial weakness in the design, then there is something wrong. My concern now is that the Foveon technology will never get the attention it deserves and may one day become a curious footnote in the history of digital photography. I see nothing on the web to indicate that research and development is taking the sensor beyond its original design, and although Sigma has implemented it in their cameras, current and past reviews suggest these mechanical platforms leave a lot to be desired. I’m sure there is a lot of corporate politics in the behind-the-scenes story here; perhaps long-term contractual commitments between manufacturers that shut out Foveon from access to the big players such as Canon and Nikon. I hope the technology survives, and I hope Sigma gets the platform right. If they can’t, then I hope another company will take up the baton and run with it. It will be interesting to watch what happens next.</p>

<p>I don't think any other manufacturer has any chance if Sigma don't make anything of it. Sigma wholly own Foveon.</p>

<p>I wonder what it is they own though? Just the patent and manufacturing rights, or does Foveon still exist as a company with continuing R&D.</p>

Ironically, if you look at the code for a RAW processor such as dcraw, you'll find that the Foveon sensor requires much more complex processing to produce reasonable images than a Bayer sensor. The Foveon approach looks simple and elegant in a brochure, and it has some good properties, but the sensor has poor color separation. That is, a lot of the "red" and "green" photons that the sensor captures are actually absorbed by the "blue" layer, etc. The RAW processor has to tease all of that apart, and it's not very easy, leading to a lot of color noise in low-light situations.

 

I suspect that cameras will eventually use something other than Bayer sensors. But I don't think that the "something" will be a Foveon sensor.

<p>Thanks for the information David. I'm thinking that whatever supercedes Bayer-type sensors will have some issues if it approaches anything resembling multiple single sensor layers. It will be deemed too close to the Foveon technology and probably get tied up in patent litigation for years. Perhaps we should be hoping for a multi-layered photo-sensitive medium which could be chemically post-processed to produce a RAW image... hmm that sounds familiar.....</p>

<p>Foveon images can be spectacular, but the technology needs to be improved in order to atract more users. With the latest DSLR, the SD14, you always need to shot RAW and make corrections in SPP ( Sigmas softwere) then export Tiff files to PS or LR for further processing for best results.</p>

<p>ISO 800 is not good with respect to color and noise. Its very sensitive to color temperature, which often results in images with a green or magenta cast. It can be corrected in SPP, but you have to do the job.<br>

I find the SD14 quite unreliable with its autoexposure ( my Canon EOS 400D isnt great either, but its better with a large margin).<br>

SD14 is slooow. Six shots in a fast pace and you can take a cup of coffe and then its ready for another six.<br>

Coming from shoting Kodacrome 64 and Velvia with Canon F1 and Hasselblad 500C, I find SD14 a very nice camera to work with despite its flaws. Used within its limits and with careful exposure and low ISO (50-200), It simply shines in picture quality over the canon (same lenses) in prints up to A3 ( I dont print larger).</p>

<p>It is sad that Sigma dont make the SD14 with Canon EF mount. I think they should sell more cameras that way thou somewhat less lenses.</p>

<p><em>My concern now is that the Foveon technology will never get the attention it deserves and may one day become a curious footnote in the history of digital photography. </em></p>

<p>Everybody knows about them, but no one besides Sigma (and a noisy bunch of fanboys)seems to care. The premise is interesting but largely ineffective due to the low cell density, resulting in mediocre resolution, an low intrinsic sensitivity due to serial filtration.</p>

<blockquote>

<p>When you compare it to the way a Bayer-pattern sensor works, the latter seems a flawed concept.</p>

</blockquote>

<p>A Bayer pattern sensor works pretty much just the way a human eye does, with different numbers of pixels allocated to different colors. If anything, the Bayer sensor doesn't take this concept far enough, only having twice as many green as red or blue pixels. The difference on a human eye is more like 15:1.</p>

<blockquote>

<p>An image comes from a non-foveon camera via a complex processing cycle designed to interpolate colour in adjacent pixels</p>

</blockquote>

<p>As David pointed out, the color math for a Foveon sensor is much more complex than a Bayer sensor. It also requires more work for graceful clipping of blown highlights (when one channel blows on a Bayer sensor, that's all that blows. Because a Foveon sensor puts three photodiodes "in series", when one channel blows, you can't read the other two. That's why the first two years of Foveon software saw the really weird "all blowouts go gray" behavior, and the current software uses "inpainting" to paint blowouts in a solid color averaged from the colors of the region bordering the blowout. And then there's red sharpening...</p>

<blockquote>

<p>and then blur the image with an anti-aliasing filter to reduce moiré.</p>

</blockquote>

<p>Technically, an AA filter is necessary, even for a Foveon image. Just pecause you're less likely to see aliasing in a sensor that has luminance at the same pitch as chroma doesn't mean that it's not there. I've seen Foveon images positively mutilate hair.</p>

<blockquote>

<p>I’m surprised they work as well as they do. I’m not an engineer but I love what I see as ‘simple’ and ‘elegant’ designs.</p>

</blockquote>

<p>I am an engineer. I've designed industrial cameras using both Bayer and Foveon sensors (two Foveon sensors to one camera, no less). The Bayer sensor is a more "data efficient". Printed at the same size, an image from a 10mp Bayer looks sharper and more detailed than an image from a 3.4x3mp Foveon. The Foveon only looks better when you're "pixel peeping" at 100%.</p>

<p>So, why did I use it? Not for the "elegance" or "simplicity" or the "organic" quality that it certainly does not have...</p>

<ol>

<li>It was the only sensor available at the time in small quantities on a COTS (commercial off-the-shelf) basis having liveview.</li>

<li>It was about the right size to operate with a leaf shutter behind the rear element of a Nikon lens, then an optical glass cube beam splitter, then the filter stacks for each of the two sensors.</li>

</ol>

<p>Now that APS liveview sensors outperforming the Foveon are easily avaialble, it's essentially dead in industrial applications.</p>

<blockquote>

<p>I'm thinking that whatever supercedes Bayer-type sensors will have some issues if it approaches anything resembling multiple single sensor layers. It will be deemed too close to the Foveon technology and probably get tied up in patent litigation for years.</p>

</blockquote>

<p>Run a patent search or two. You'll see that there's prior art on multiple layer sensors going back two decades. The Foveon patents all pretain to unique architecture relating to the triple stacked PNPN (or was it NPNP) photodiode with the middle diode reversed, and the differential sensing necessary to read the diode stack.</p>

<blockquote>

<p>I’m sure there is a lot of corporate politics in the behind-the-scenes story here; perhaps long-term contractual commitments between manufacturers that shut out Foveon from access to the big players such as Canon and Nikon.</p>

</blockquote>

<p>Nikon has bough sensors from Sony, designed sensors that they've had fabricated by Toshiba and IBM, supplied camera bodies to Kodak and Fuji for the integration of Kodak, FillFactory, and Fuji sensors, and even did the actual integration for the Fuji S5.</p>

<p>There's a principle called "Ocham's Razor" that says (in Latin terms of formal logic) that the explanation that requires the least assumptions is generally the right one. Your "conspiracy theory" requites two assumptions, that the Foveon sensor outperforms Bayer sensors, and that some sort of conspiracy actually exists to block its use. The theory that Canon and Nikon completely ignored Foveon because Foveon offers insufficient performance advantages requires no assumptions at all. Foveon's insufficient performance is proveable, not an assumption.</p>

<ul>

<li>The color computations are complex, resulting in large amounts of "mathematical noise". </li>

<li>The complex color computations make it more difficult to derive an adequate in-camera JPEG.</li>

<li>The poor color accuracy due to the lack of a colorimetric response on the sensor (human eyes, film, and Bayer sensors all use similar organic filters, Foveon uses an inorganic color separation process that leads to observer metamerism. That's why color processing can drive a Foveon shooter up the wall. There's a problem with what is called "observer metamerism". For example, shoot a portrait outdoors. Leaves and human skin that have essentially the same hue to a human eye (or film, or a Bayer sensor) may acquire two different hues to a Foveon sensor. If you try to fix the color so that the skin is right, the leaves go magenta. Try to get the leaves right, and the skin goes green...</li>

<li>Date efficiency is terrrible. Whether you call the current (DP1, DP2, SD14) sensor 14mp, or 4.7x3mp, it produces 14 million samples of data, but produces an image that prints as well as a 10 or 12mp Bayer image. More data to transfer, process, and store means a more expensive and/or slower camera and puts greater demands on the computer you use for raw processing.</li>

</ul>

<blockquote>

<p>an low intrinsic sensitivity due to serial filtration.</p>

</blockquote>

<p>Actually, serial filtration has a high intrinsic sensitivity, about twice that of a CFA (such as Bayer) sensor. The low sensitiyivity of the Foveon is due to the low spectral separation of the silicon absorbtion filter concept and the noise doubling differential sensing scheme required by the "cost effective" Foveon four layer three diode structure.</p>

<p>Other serial filtration (layered sensor) concepts, such as the Fuji stacked organic filter sensor, get around both of these problems.</p>

<p>Joseph, great stuff! Many thanks for answering my questions and explaining my misconceptions. That's what I like about this site, it's full of trained professionals ready to give insight into the most complex technical issues. I still think the images beat anything else I have seen even though on paper perhaps they shouldn't. I am reminded of the initial debate between analog and digital audio. Technically digital seemed perfect, no distortion, a perfect waveform reproduction, no extraneous noise. Yet most audiophiles still preferred their turntables, vinyl albums and vintage tube amps, many still do. It's the final product that counts no matter how it gets here.</p>

<p>Milo, you're quite welcome.</p>

<p>Anything involving perceptual sciences is fun. Humans tend to not like accurate colors in photographs or accurate frequency responses in audio. Part of the experience of a landscape is the sounds, smells, little movements in the scene, the feel of the breeze on your face, etc. Take all that away, and an accurate visual representation isn't enough anymore, so we "punch it up" a bit. Audio is the same way, without real musicians, a venu, and a crowd, it's an abstract experience. The sound needs to be made into "more than real" to compensate for the loss of the rest of the experience.</p>

<p>Well written, Joseph, and informative. I'm sure Occam will forgive your mispelling.</p>

<p>Edward, thank you.</p>

<p>p.s. it's not a misspelling, it's just an older one. Back in William of Ockham's time (early 14th century), they spelled it, well, "Ockham". Then they began omitting the k, and "Ocham" makes sense with modern usage, although the little town is still spelled "Ockham". If you want to impress people with your pronunciation, hit the "Och" hard, and shorten the "a", more like "Och'm".</p>

<p>The "Occam" spelling is French frippery in which I will not participate. It encourages the "oc-cum" pronunciation. As far as "Bockham", "Bocheham", or "Hockham", well, let's just not go there. ;)</p>

<p>p.p.s. I know you probably meant it as a pun, but you misspelled "mispelling" ;)</p>

<p>In favor of Foveon it has the same resolution for all colors, bayer hasnt. The output data from each pixel from foveon is independent of data from other pixels. No risk of guessing wrong. Pixel by pixel the accuracy are on a much higher level than from a Bayer sensor.</p>

<p>The lack of a AA filter keep the contrast almost up to the resolution limit. With Bayer it starts to degrade far from its limit ( Bayer with AA, that is). This is why medium format sensors and Leica M8,2 and M9 dont have AA filters. The absense of AA filters sometimes couse problems, but Sigma, Leica, Mamia, Hasselblad, Phase one et.al decide that the benefits are greater than the disadvantage to not use AA filters.</p>

<p>Dark hair with strong reflections is something that Foveon fail to manage. Many camera reviewers makes photographs of B/W resolution charts and when they think about the small number of pixel locations of only 4,7 Mpix, they just have to confirm what they already "know" by looking for how small branches they can see against the white sky in a real photograph ( which is essentially the same as locking at the B/W chart). The verdict is clear, Foveon 14 Mpix is not quite as good as a 8 Mpix Bayer. Few actually look at the hole prints with no prejudice mind.</p>

<p>It would be interesting if someone made a photograph of a 4,7 Mpix chart of a Bayer color pattern with Bayer sensor camera of 10-12 Mpix and compared the results with Foveon 14Mpix. What I mean is that one can always use test methods that benefits either type of technology.</p>

<p>I have to strongly agree with Milo, that it is what you see whith your eyes looking on a print that matters. I remember when the CD players was intruduced around 76-77, most people was listning to the absense of noise and forgot listning for factors as precence, staging and harmonics.</p>

<p> </p>

It is clear why Sigma cameras are slow, given what Joseph said about computational complexity in his very fine post. What is not clear is why Foveon images look so good, especially on a high-res LCD monitor. They just seem more lifelike, more 3D, than comparable Bayer images.

 

Bengt, when you say "same lenses" (in your Sept.28 post), do you mean two samples of the same lens design, one for Canon mount and one for Sigma mount?

<p>Bill, Same lenses I mean two identical lenses with different mount. Since Canon EF and Sigma mount have identical flange distance they could be as close to each other as two samples of the same brand. I used Sigma 17-70 2,8-4,5 DC Macro on both cameras.</p>

<p>SD 14 show better resolution of low contrast texture than EOS 400D. Its the clarity, contrast and smoth color transitions that strikes me with SD14.</p>