subtracting dark frame

I've heard of this technique for helping to reduce noise on long

exposures using DSLR's. Can someone explain how I would do this in

photoshop, and whether it is possible with several combined exposures?

If you take an exposure of equal length with the lens cap on, then put that dark layer as the top layer in Photoshop, and set it for Difference mode and 100% Opacity, it appears this should give you the results you're looking for.

 

I just tried this and it seems to work. Someone might have a different way of doing it though.

 

I also found if there aren't too many "hot spots" that you can use the History Brush with the Dust and Scratches filter to manually paint them out.

Interesting, I haven't tried this (first time I'm hearing about it, in fact), but I have a question... why would the lens cap exposure need to be the same duration as the noisy photo?

wait, when you say equal length, are you talking about focal length or duration of open shutter?

<i>Interesting, I haven't tried this (first time I'm hearing about it, in fact), but I have a question... why would the lens cap exposure need to be the same duration as the noisy photo?</i><p>

 

Someone correct me if I'm wrong, but you'd want the exposures to be the same length as the longer the exposure, the greater the number and intensity of "hot spots."<p>

 

Aperture shouldn't make a difference, but I'd stop it down all the way anyway to reduce the likelihood of light getting in.

...I've never tried any of this... but I've read that you should also block the eyepiece as well

during the long exposure. This should be done on any long exposure capture including

the dark-frame capture. When you don't have your eye up to the eyepiece, some stray

light can get in there and cloud the captured image.

 

The neck strap that came with my Canon 10D has a clip built in designed to cover the

eyepiece for long exposures.

 

Malcolm

<p>In lieu of repeating a bunch of stuff here, you might want to read an explanation of how to take the darks, and one way¹ to subtract them using Photoshop, <a href="http://www.photo.net/bboard/q-and-a-fetch-msg?msg_id=00ARAm">which I posted here</a> in the Canon EOS forum. The method I describe is not Canon-specific.

 

<p>Malcolm, that is a good point about covering the eyepiece during the exposure.

 

<p>As anyone who pays attention to my blatherings here probably knows, I'm a technical photographer and need to know a lot about sources of noise in CCD and CMOS sensors and how to calibrate (loosely meaning "remove") it from digital images for my work. The contributions I've made in the forums on this topic have been hasty and brief, not thorough or even necessarily understandable. The topic of noise and how to deal with it comes up <b>an awful lot</b> in the forums. Does anyone think that I ought to write a proper article on the topic for photo.net, and if so, can someone tell me, to whom do I send the copy?

 

<p><small>¹ That's one way out of approximately a billion, I guess.</small>

<i>why would the lens cap exposure need to be the same duration as the noisy photo?

</i><p>

Dark frames are daily (nightly) fodder for digital astrophotography. Noise is an accumulation of random signals over time. It increases linearly: twice the time, twice the noise. So when your photo is 60 seconds, it has accumlated 60 seconds of noise, which you want to remove; thus a 60 second dark frame. Use a longer dark frame, and you'll be removing more data than what's just noise (i.e. stuff you want to keep). <p>

There is another factor influencing the amount of noise though, and like duration, you also need to shoot your dark frame in the same conditions as your light frame: temperature. If you are out shooting pre-dawn landscapes in the winter, your camera will be running at the ambient temp of, say 0C. The next day, in the comfort of your kitchen, you shoot the darkframes at 20C and you will have vastly more noise than your actual photos contain. <p>

Duration must be the same, temperature must be the same.

<p><i>Noise is an accumulation of random signals over time.</i>

 

<p>I know this may sound like nit-picking, but dark and bias noise is not actually random¹. The manifestations of this noise are best described statistically, but the dark noise is decidedly unrandom and in fact is highly reproducible from frame to frame on the same sensor. That is why subtracting a dark frame works - if dark noise were random, you'd be subtracting random values from your image, and that simply wouldn't do the image any good.

 

<p><i>Duration must be the same, temperature must be the same.</i>

 

<p>This is also incorrect. Although my examples specify the same exposure time and the taking of the dark as soon as possible before or after taking the actual image, this is unnecessary.

 

<p>Because dark noise scales predictably with both time and temperature, it is possible to make a dark frame at both a different temperature and a different integration time, and still use it to calibrate images. For example, in my work I commonly take an exposure of (say, for the sake of argument) three minutes. It is not uncommon for me to then take a dark frame of six minutes exposure duration. This dark frame would be divided by a constant (two, in this case) before being subtracted. The advantage here is that dark noise is a greater proportion of electrons compared to read noise in such a dark; you get a better sample of the noise this way for some purposes.

 

<p>Similarly, it is possible to characterize a sensor's dark current behaviour with temperature, and you could take data frames at a temperature ten degrees C different from your darks, and still use them for subtraction. To do this right you would probably derive a function to apply a constant to the pixel values in the dark frame. But most people aren't going to measure fluxes in their digital SLR images - they can just scale by the seat of their pants until it 'looks right' and get decent results. I've rescued plenty of long-exposure 'snapshots' in just this way, and at least eight of the asteroids I've discovered were found in CCD frames calibrated with darks that were taken at both different temerpatures and for different exposure times.

 

<p>In any case, "noise is random" and "duration and temperature must be the same" are not strictly correct, and I think that if this topic were better understood, people interested in doing quality long exposure (or high ISO) work with digital SLRs would have a higher success rate. I'm not presenting this as a gratuitous correction to a minor technical error, but because I honestly believe that a correct understanding of the circumstances can lead photographers to better results.

 

<p><small>¹ Ok - it might be randomly varying from pixel to pixel and/or from sensor to sensor, but this isn't the camera user's concern.</small>

Your understanding is much more extensive for sure Jeff. I poorly chose one word: "must". Allow me to rephrase, using the same duration and same temperature <i>is advisable</i>. Better? Anything other than that is an further extrapolation and thus by definition less precise. You could argue that a longer dark frame could more accurately model the sensor, but realistically and practically, you'd be displaying good discipline to just match duration. <p>

So, if you want to settle for approximations, you could also just set the camera onto "noise reduction" and let it do the work, or just use your one dark frame for any duration/temp.<p>

It would be nice of you to also go an explain to the astro community at large that the hours spent building up a library of dark frames at each temp and each duration was futile and incorrect.

<p><i>Allow me to rephrase, using the same duration and same temperature is advisable. Better?</i>

 

<p>Certainly easier, I agree with you there. And definitely better in the sense that you are <b>certain</b> to have a dark that applies to the shooting conditions you actually encountered.

 

<p>Your point about discipline is a good one, as well.

 

<p><i>It would be nice of you to also go an explain to the astro community at large that the hours spent building up a library of dark frames at each temp and each duration was futile and incorrect.</i>

 

<p>Some of my former colleagues at <i>Sky & Telescope</i> magazine believe or believed that this is true and have said as much in print. I don't agree with them, though. My experience has been that <i>most</i> cameras dark noise varies in some simple manner - e.g., by a factor of two for every <i>n</i> degrees C increase in temperature and in lockstep with exposure duration. But not every sensor is like this. Multi-pinned phase sensors behave a little bit differently than non-MPP, and at higher temperatures and shorter exposures some of them have nonlinearly varying noise characteristics. Anti-blooming gated sensors also tend to have complex, nonlinear variations in dark noise by temperature. These would be two cases in which a dark library taken at a variety of exposures temperatures would be a good thing, if only because doing so is easier and less time consuming than characterizing the sensor mathematically. Another case would be when observing time on the telescope is so expensive that calibration frames are best taken during the day or when cloudy. In these cases, software such as IRAF or Maxim DL can rescale the darks to compensate for the small remaining amount of temperature or exposure duration difference, and a big old calibration library works really well.

Ok, this is starting to go waaaaay off the OP's topic of how to cut down noise on a few minute-long exposure with a dSLR. Just finding out whether the CMOS in the D60 is multipinned (no idea what that is by the way) would probably take longer than shooting the dark frames ad hoc. <p>

On the other hand, and very selfishly... forget the OP. I'm realizing that my library of dark frames I used during the summer months (-10 C) is well below the potential temperatures I can reach now in the winter, and I'm faced with recreating a whole new set of dark frames for, say -25 C. <p>

I do use an ABG SBIG ST8, but before I go off to expose 8 frames each at one temp, two durations (5 and 10 minutes) and two binning modes: four hours of exposure, your thread suggests perhaps there is a better/easier way? I have used scaling for temp and duration in CCDSoft (not MaximDL) but in my experience, the results are far from satisfactory with artefacts/noise all over the place because seemingly the image reduction was imprecise.

<p><i>Ok, this is starting to go waaaaay off the OP's topic of how to cut down noise on a few minute-long exposure with a dSLR.</i>

 

<p>Yeah, well, we are at least on-topic for the forum.

 

<p><i>Just finding out whether the CMOS in the D60 is multipinned (no idea what that is by the way)</i>

 

<p>Multi-pinned phase operation is a way of giving dark current (which is the source of dark noise) a path of lesser resistance than through the sensor's photosite. It is a way of draining off that current before the electrons end up in the CCD wells and add noise to the image. This is mainly a design issue, not an end-user issue, but its presence or absence does affect dark subtraction in some cases.

 

<p>The D60 is a CMOS sensor camera, right? So it wouldn't be MPP. But on-sensor processing could make it behave non-linearly anyway.

 

<p><i>your thread suggests perhaps there is a better/easier way?</i>

 

<p>Yikes. Sounds tedious to me. Could you write a script to do this dirty work while you sleep? In any case, ABG CCDs are one of the case scenarios above in which I said a dark library makes a lot of sense.

 

<p>The ABGs (and related design issues) have a lot of in-use side effects, including non-linearizing the noise profile. When this happens, e.g., a very noisy pixel is suppressed more than the not-as-noisy pixel next to it. So if you scale by a factor, the scaling gets all screwed up since the factor necessary for each pixel is different.

 

<p>If you were using a non-ABG camera, scaling would not be as troublesome. One of my cameras is a non-ABG ST-8E and it scales pretty nicely as long as you don't encounter a 30 degree C or greater temperature delta. It is a lot better if you use a hot dark to calibrate a cold light frame, though.

 

<p><i>I have used scaling for temp and duration in CCDSoft (not MaximDL) but in my experience, the results are far from satisfactory with artefacts/noise all over the place</i>

 

<p>If you want, you can send me a couple sample frames and I can do a scaled calibration in Maxim to see how things come out. I don't know why the scaling routines in the two packages would be any different, but you never know. Anyway, I wouldn't hope for much - with ABG I think you will have an uphill battle trying to scale calibration frames no matter what software you are using.

Thanks for the offer Jeff. Very kind. I should simply put to use the back-to-back cloudy nights right now (of course, they're cloudy, it's close to new moon!) to start building up that library! The Xmas holidays (which will be perfectly clear of course: full moon) are also an opportunity to grind through the dark frames. <p>

Thanks for the tips and season's greetings.

<p>Anyone still following this discussion? If so, I've written up a first draft of an <a href="http://medkeff.com/jeff/photo/noise/">article about digital SLRs and dark frame subtraction</a>. I intend to offer the article to the powers that be for use on photo.net, as soon as I can figure out who the powers-that-be are....