Film Negative Invert and Processing in L* Gamma FAQ

Discussion in 'Digital Darkroom' started by dmitry_shijan, Mar 23, 2021.

  1. Here is also an idea how to emulate scanner multisampling with DSLR.
    There is a well known very effective lossless temporal noise reduction method. It is based on multiple similar images blended with "Mean", "Median" or some other special blending modes. It is well described in this article Pat David: Noise Removal in Photos with Median Stacks (GIMP/G'MIC & Imagemagick)

    This stacking option also available in Photoshop, but as usual designed in rather hidden and confused way:
    File -> Scripts -> Load Files Into Stack -> check "Create Smart Object"
    Layer -> Smart Objects -> Stack Mode -> Mean

    This stacking option also available in Affinity Photo, but currently not available PhotoLine. Hope PhotoLine developers will add it in future.

    Mean — averages pixel content across the stack of images. Good for long exposure simulation and noise reduction.
    Median — removes pixel content that is not consistent in each image. Suitable for object removal and noise reduction.
    Outlier — exposes pixel content that differs in each image: great for sequence composites.
    Maximum — uses the maximum pixel values from each image. Can be used for creative exposure blending where the subject is lighter than the background.
    Minimum — uses the minimum pixel values from each image. Suitable for exposure blending where the subject is darker than the background.
    Range — indicates areas that change across the image stack. Good for analyzing what has changed between each image.
    Mid-Range — uses the middle pixel values from each image. Can be used to increase tonal range if used with bracketed exposures.
    Total — produces the total value of pixels from each image. Usually results in overexposure, but can be used to lighten very underexposed imagery.
    Standard Deviation — analytical: measures the distribution of information between the images. Useful for object removal as it clearly indicates areas that will be averaged out with a Median operator.
    Variance — analytical: as Standard Deviation, indicates how pixel values are spread between images. More intense distributions are shown very clearly.
    Skewness — analytical: highlights edge detail and indicates the intensity of pixel value distribution. Can be used to determine tonal and spatial differences between images.
    Kurtosis — analytical: detects the peakedness of an image. A brighter result represents low noise levels and a tonal uniformity (most pixels at dominant gray level). Darker results represent greater noise and less tonal uniformity (more pixels further away from dominant gray level).
    Entropy — analytical: represents the number of bits required to encode information in the stack. Could be used with stacked video frames (within the same scene or shot).


    So in short:
    1. You just need to quickly shoot 5-10 similar copies of the same film. Continuous Shooting (Burst Mode) will do the trick. Make sure your setup is stable and you don't move camera or film during shooting.
    2. Process RAW files to TIFFs as it was described earlier, but don't do invert and don't do other processing yet.
    3. Stack images into one single file with "Mean" blending mode and save as single TIFF.
    4. Process negative with workflow described earlier in my posts.

    This will clean up all possible digital noise without touching film grain structure and will make your source file more dense at pixel level.

    Also this method probably should be way less risky than HDR stacking, because it will not change original tonal relations taken from linear sensor data.

    And a quick test to proof my theory. Here is crop of inverted and processed film negative. This film negative sample was scanned with camera and was underexposed more than usual to amplify the noise and see the camera limits. This is 400% scaled crop, but i can see the difference at 100% as well.

    [​IMG]
    [​IMG]
    [​IMG]
    [​IMG]
     
  2. Looks great if you want slightly more blurry 'scans'.

    The digital noise isn't an issue even with a single frame, since it's at a level of about 1/10th of the grain/dye-cloud 'noise' that multi-shot scanning does nothing to reduce. - Unless, as in the first example above, there's some obvious image shift between shots.

    Why not just increase the illumination level to get noise down? That's if it's even an issue to start with.
     
  3. Since my last post I've processed nearly 1000 negatives using the "NegativeLabPro" plugin for Lightroom. About 1/3 of my time is spent "scanning" film strips with a Sony A7Riv camera and Nikon 55/2.8 Macro, and 2/3 if the time straightening and cropping the images before processing. I used a Lumecube LED panel, set to 5500K with a CIE rating of 95%, and a sampled WB to match in the camera. The negatives are a mix of Fuji Reala and Fuji Superia 400.

    The results were surprisingly consistent, regardless of the emulsion, lighting, exposure level and environment. There were none I had to reject based on color balance. That said the colors are not as bold as you would get from a minilab, actually quite close to Sony colors, that is to say "accurate". I spent roughly 4 hours on this project, averaging about 250 image/hour scanning, and about the same time in Lightroom. If better processing software comes along, I still have the negatives in RAW format.
     
    Last edited: Apr 6, 2021
    digitaldog likes this.
  4. Temporal noise reduction don't blur any details, until you stack 100-1000 of images.
    From my observation Median produce slightly higher micro contrast than Mean. But the difference is near invisible.
    Increased illumination level won't help a lot. I inspect IT7.8 test slide frame and i can see that very small amount of digital noise always started somewhere from the middle grey patches. Sure it is near invisible but it is always there. To get rid of all noise with illumination you need to increase exposure a lot and so clip huge amount of dynamic range.
    And this is not only about noise. It is about pixels density and pixels quality. With stacked images from multiple 14 bit "relaxed" bayer pattern-based images you get solid and dense real 16 bit color data.
    From my personal tests 5 images stack is OK, but some tiny amount of noise in deepest shadows is still there. 10 stacked images look perfect, but take more time and process. Stack more than 10-20 images probably useless for film scans. But for sure is may depend of camera sensor. Lower quality sens0d -> more noise -> need more stacking.
    Also don't forget that stacking is highly exponental. Dfference between 0 and 5 images always look huge. Difference between 10 and 100 stacked images may be near invisible.
     
  5. And here are some interesting resolution and artifacts tests.
    In the past i was lucky (actually not lucky) to use MINOLTA DiMAGE SCAN 5400II. It is a great scanner, but it had a global problem with non uniform backlight stripes, described here Backlight stripes in MINOLTA DiMAGE Scan 5400 II or what? (too bad uploaded photos are gone over time)
    After many attempts to modify LEDs or diffuse that defect i gave up with it. I decide to take off the lens form that scanner and build a camera scan system.
    Same time i had access to smaller MINOLTA DiMAGE SCAN Elite II model, so i was able to take some shots from it for compare.

    For this test i build input ICC profiles for camera and both scanners based on Kodak IT 8.7 Scanner Calibration Target from Affordable IT 8.7 (ISO 12641) Scanner Color Calibration Targets
    Profile Type: Single gamma + Matrix

    Scan from camera debayered in Iridient Developer with Anti Aliasing setting: 2.
    Custom contrast camera curve removed. No noise reduction, no sharpening, no any other adjustments applied to RAW file. 5 frames stacked in Median mode.
    Poor quality consumer furniture LED panel used as backlight.

    Scans from scanners are in linear gamma with disabled color management.

    All images processed with my workflow described earlier:
    Transform from Camera input ICC profile to ProPhotoRGB with L* gamma ICC profile -> Invert -> Apply RGB AutoLevels -> Recover back clipped data from RGB AutoLevels -> Contrast.
    No custom White point picker.

    This is rather complicated frame i use specially for tests. Is is very scratched and it have a lot of extreme saturated colors.

    [​IMG]
    [​IMG]
    [​IMG]
     
  6. And some Anti Aliasing tests. If turn Anti Aliasing test OFF image looks very close to source form scanner. Huge amount of rainbow patterns and dots over film grain structure and aliasing artifacts around contrast white dust and scratches. This probably means that scanners use very basic internal debayering and don't do any anti aliasing reduction.
    Too many anti aliasing filtration also looks not good. Film negatives are very sensitive to small changes in source image, so even small amount of extra filtration may decrease global saturation in final processed image.
    So after some tests i can suggest:
    Anti Aliasing: 2 (instead of default 3)
    No Luma/Chroma noise reduction.
    No sharpening.
    And of course it is all depends of camera and sensor technology. Sensors with OLPF filters need less AA filtration. Normal Bayer sensors need more filtration that Fuji X-Trans sensors.

    [​IMG]
    [​IMG]
     
  7. Scanners almost invariably use a tri-linear sensor. They don't need de-Bayering because they don't use a Bayer CFA to start with.

    The exceptions are some of Nikon's scanners that use an unfiltered CCD linear sensor and rapidly switch red, green, blue and IR LEDs to get the colour and defect-map channels.

    Scanners are also prone to aliasing. It's an interaction between the regular digital sampling spatial frequency and the irregular 'grain' pattern.

    Do some homework on how this stuff works.
    So, that would be the brightest parts of the negative that show most noise?
    How does that work?
     
    Last edited: Apr 7, 2021
  8. Yep, probably "dabyter" is not a correct world for single or triple row linear scanners, but anyway they do some sensor-to-image pixels processing and usually have pretty hard aliasing.
    And yep, brightest parts of the negative will show most digital noise because negatives are inverted. And the most digital noise in blue channel, because orange color cast in negatives.
     
  9. Here is better underexposed example crop. Not a sharpest example because i still don't have stable setup. But it is enough to see that digital noise may be very easy messed up with film grain :)
    [​IMG]
    [​IMG]
    [​IMG]
     
  10. LOL.
    Where does this noise originate then?
    If it's in the camera or scanner, then that's very strange behaviour; to have noise that's super-proportional to the signal level.

    Use your eyes! That multi 'scan' composite is just blurred compared to the single shot version. To the point that even the grain isn't visible.

    And why would you ever underexpose a camera copy to start with?
     
    Last edited: Apr 7, 2021
  11. Add fake noise to that image again and it will look subjectively not blurred :) It is a known - visual effect more noise add fake feeling of more detailed image.
    I just do some tests here and try to amplify some effects. Hope somewhere i'll be able to get more focused and grainy camera scan.

    Seems you confused with everything in this world. I really tied to explain every basic thing to you here.
     
  12. Yes, well somebody is.
    Here's what happens when you invert digital noise:

    A Kodak greyscale and colour swatch shot at 25,600 ISO with all noise reduction turned off -
    Frame.jpg
    Can you see much noise in the two lightest patches?

    A tighter crop of the greyscale -
    Noise-strip.jpg
    And a negative inversion of the above -
    Inverted-Noise.jpg
    Note that what was the brightest step is now practically noise-free.

    How it works is: Noise is indeed more visible in darker tones. However, noise also reduces with exposure. The nett result is that, after inversion, the darker tones get less noisy.

    The noise is in fact more visible in mid tones, where the two effects of noise perception and real percentage noise cross over.

    But nobody in their right mind would use a digital camera set at 25,600 ISO and with its noise reduction turned off to copy negatives!

    Incidentally, the 2.1D 'brightness range' of the above greyscale is almost exactly the same as the density range found in a colour negative. The cyan, yellow and magenta dye images each only have a brightness range of just over 100:1, which won't tax the ability of any digital camera worthy of the name.
     
    Last edited: Apr 7, 2021
  13. In multi sampling, you seem to lose a lot of edge detail. IMO, it's more important to retain sharpness than avoid noise/grain. I'm not convinced you are seeing noise, and not grain, in your illustrations, with the exception of Rodeo's shots of a color chart.
     
    Last edited: Apr 7, 2021
  14. Once again - my examples are not 100% sharp because i don't have a proper film holder yet to share some decent frames. Currently working on quality film holder project that may be useful for many people in future.

    Digital noise in digital sensors - in shadows.
    Film negative - Inverted.
    Digital noise in film negative became visible - in lights.
    All my current examples are with NEGATIVES.

    Diffused light - softer film grain look, but also near invisible dust and scratches.
    Collimated light (usually used in film scanners) - sharper film grain look, but all dust and scratches became extremely visible.

    Defocused or soft lens - Softer film grain, but same amount of digital noise.

    26MP is not a enough resolution to capture sharpest possible film grain details.

    My examples only about 2 stops underexposed from native camera ISO 160.
     
  15. This is a recent example of negative conversion. It was done with a Sony A7Riv (61 MP), Nikon 55/2.8 Macro lens, and Nikon ES-2 film holder. The software was Lightroom, with a NegativeLabPro plugin. You can see dye clouds In the magnified sample, but no signs of noise. You will have to enlarge the sample panel at least 4x to see individual pixels. The film, as I recall, is Fuji Reala. The scene is the Charles Bridge in Prague.

    Full Image
    _7R41446.jpg

    Pixel = Pixel Sample
    Sample-1.jpg
     
    Last edited: Apr 7, 2021
  16. There is no compelling reason to use a Log gamma profile when scanning slides or negatives. The dynamic range (on film) for negatives is very low. Even the densest parts are transparent compared to overexposed B&W film, and the Dmax of properly exposed B&W is well within the range of a modern digital camera. The dynamic range (on film) of certain brands of slide film can be very high due to the high potential density of blacks. However little detail remains in this region, and no detail remains at all in areas overexposed.

    Log gamma profiles are at their best in flattening the curve in mid to high tone regions, and in avoiding overexposure in the highlights by reducing the median exposure by 2-3 stops. This is done at the expense of quality in dense areas, which tend to show excessive noise when expanded. While these properties have some benefit when copying slides, they are a distinct disadvange with negatives.

    I use log gamma profiles regularly with video, where the bit depth is limited. You must be very careful to expose typical 8-bit clips correctly. Log gamma comes into its own with 10-12 bit video, or 16-bit for some raw outputs, especially in mixed sunlight.

    Why make life more complicated than necessary?
     
  17. You will notice that film grain us much less noticeable where there is a lot of detail. That's because the dye clouds tend to coalesce on edge boundaries. Since the A7Riv has more than 4x the resolution of film, it's contribution to uncertainty can be neglected (< 1%).

    Detail-2.jpg
     
  18. How can you sure that it is not some amount of digital noise partially overlapped above film grain structure?

    To see where digital noise starts in your camera just do stacking tests with any regular sharp and fine detailed digital photo (not a film scan). Or just shoot few frames in burst mode and inspect them one by one to see where moving digital noise structure starts to became visible. With same basic test you can confirm or deny your "details loss" theory.

    Digital noise is not a individual pixels. It is a dynamic structure from some point of view similar to film grain.

    I never suggested to use Log gamma here. L* gamma is not a log gamma.

    I provided side by side tests showing 42MP vs 26 MP vs 11 MP. There is a difference in fine details visibility between 42MP in 26 MP. And it was sample shoot taken with basic consumer film and rather soft 50mm portrait lens.
    Can we see your measurements where A7Riv has more than 4x the resolution of film?
     
  19. Individual pixels cannot be distinguished in Lightroom when magnified 16x. At that magnification, the finest detail in the image are the dye clouds. For normal contrast, Reala and Velvia barely deliver 80 lpi, whereas the A7Riv delivers 249 ppi. 3x might be a better number, but that's a ratio which can be neglected with regard with 96% confidence.

    The premise of this thread was the scanning and conversion of color negatives, not noise only detected in solid color targets. Besides, I'm using enough light to keep the ISO at or below 150 at 1/30 second @ f/5.6. If there is noise, it is less significnt than the dye clouds, and far less than the finest details recorded.

    In short, I think you're blowing smoke.
     
    digitaldog likes this.
  20. I don't insist that stacking is a must have option for every scan. Median Stacking probably the last thing you need to do in camera scanning and negative invert process. You never see difference between all those extreme resolutions and stacking on 10x15 prints or when downscaled to HD or 4K monitor resolution.

    But once again:
    Film grain is a static texture in multiple scans of same frame.
    Digital noise is dynamic texture and it changes randomly in every frame.
    Median Stacking removes moving noise structure and don't touch static film grain structure.

    I have no idea where those "Reala and Velvia barely deliver 80 lpi" numbers came from and how and when was it tested and why should i believe that? I did my own tests and i see that 42MP provide more details in cloth textures that 26 MP camera scan of crappy old faded scratched film negative shoot on basic consumer film roll with basic 50mm lens.

    Also don't confuse specs. 2x 4x are rather abstract defenitions. x4 of what? 2x megapixels is not same as 2x pixel resolution.
    Measurement in megapixels is rather stupid, but we are live in this reality. For consumer 42MP sounds like about 2x more than 26MP WOW!!! But in reality 42MP vs 26MP is just a tiny increase in resolution. To increase resolution 2x from 26MP you need 103MP

    11 MP = 4032 x 2688
    22 MP = 5646 x 3964
    26MP = 6240 x 4160
    42MP =7920 x 5328
    61MP = 9504 x 6336
    103 MP = 12480 x 8320
     
    Last edited: Apr 8, 2021

Share This Page