Converting to DNG: What are we losing?

Discussion in 'Digital Darkroom' started by proy, Feb 2, 2013.

  1. Hi.
    So, I've been debating whether or not to convert my CR2 files into Adobe's DNG. I have so many good use for DNGs. I personally think the idea is great! Being able to search by keywords on all my photos, from all drives, etc directly from Windows or Mac's search. Being able to save all my mods directly in the DNG (and sending these overs across countries without the need of sidecars xmp or my LR catalog), etc. etc.
    There's just one thing I'd like to know: do we lose image quality when converting to DNG? I mean, Just look at the file sizes of a converted image! No, I ain't doing any lossy compression or anything. ALL my converted CR2 becomes smaller in filesize after converting.
    What am I losing here!? If anything, shouldn't my file be bigger... having the Embed Fast Load Data on also?
    Thanks. Patrick
     
  2. The one thing you lose with the conversion to DNG is the option to work with these files with the manufacturers software. Some proprietary metadata may or may not be preserved/accessible (active AF sensor ...)
    Regarding the file size: If your original files have no compression at all you'll already get significantly smaller files with lossless compression (as used in standard DNG). You can try to zip one of your original files and see how much smaller it gets.
     
  3. Andreas, but this is what I am saying: I do not use lossless compression. The "Use Lossy Compression" remains unchecked in the DNG converter. That's true for both, my settings using Adobe's DNG converter, and Adobe Lightroom. So, I should have NO compression done.
    Surely, I am not the only one wondering what's missing here seeing my file size decrease.
    I personnaly know of some photographers that WONT convert to DNG just because of this "unclear" thing going on.
     
    mike_sea likes this.
  4. And if we talk about the lost of proprietary metadata.... Anyone has any examples of this?? Like what? Should I care? How can this affect us?
     
  5. > Andreas, but this is what I am saying
    I read what you said; please do the same.
    > I do not use lossless compression.
    Yes you do.
    > The "Use Lossy Compression" remains unchecked in the DNG converter.
    That's why the compression is lossless, as in "not lossy".
     
  6. Okay. Sorry.. I'm french and I guess I've mistakenly mixed the words "lossy / lossless". Apologize for my misuse.
    So, can you clarify...
    DNG, no matter what, has a minimum compression being done. This being called "lossless"? And there is no data lost possible with this technique? (as it appears by googling).
    IF I were to check that checkbox "Use Lossy Compression", THEN I would potentially lose some quality.

    So, my files bit count would be smaller because of a) there IS a minimum of lossless compression being done in DNG, and b) because some proprietary metadata is lost.

    Is this correct? I'm getting this right? I'd still like to understand what is this proprietary metadata lost... as in should I care about that?

    For anyone else reading this post and confused with lossless vs Lossy:
    http://www.maximumcompression.com/lossless_vs_lossy.php
     
    mike_sea likes this.
  7. From where I stand, concers here mentioned are purely theoretical. As long as one preserves the original file, either on the memory card or as a copy, there's nothing to worry about. When in doubt I process the same image several times; once in the camera maker's sw, once in LR and/or in PhShp. The latter could be DXO or Capture One. There may be others like SilkyPix.
     
  8. EricM

    EricM Planet Eric

    "What am I losing here!?"

    Lots of info and choices. You can't possibly know what software you will want to use in the future. DNG conversion has lost its popularity as it never became the industry standard we hoped it to be.
     
    mike_sea likes this.
  9. You loose time and harddisk space. Advantages are insignificant since I'm using Lightroom for organising pictures. All in all I don't see many advantages for DNG. Perhaps for storing Canon CHDK raw's -if possible but in that case TIFF's will do as well.
     
  10. How am I losing HD space? My files are smaller in size after converting to DNG, thus my hole questioning about what I am losing. Time however perhaps. The time to convert them...
    @Eric: well there's something I don't get with people regarding this. You are talking about some kind of dependancy to Adobe, right? By all means do counter-ague me afterwards but... people say things like they dont want to be stuck with a software like Adobe PS or something to edit, having proprietary file formats and all. Isn't just keeping your Camera RAW files the same!?? You'll STILL need to open them with your camera manufacturer's proprietary software, or wait intul another application starts supporting that file format....
    I think whether you use CR2s or DNGs, the dependency problem will remain the same. Really. That's what I think.
    My apologize if I got it all wrong and didn't properly understand what you were saying lol :)
    Pat
     
  11. Well, perhaps you want to save the original files as well. In that case it takes extra disk space. As far as I know Adobe's
    DNG converter will let you save the original RAW's as well, inside the DNG file.
    Dependency on Canon or Adobe - i don't think it matters much. Either may decide to drop support of their proprietary
    format some time in the (far?) future. Not too much of a problem now. If it happens you can choose the next format , be it
    TIFF, JPG, DNG, CR2 or another.
     
  12. Converting to DNG doesn't lock you into an Adobe workflow - it does lock you out of the manufacturer's software, however. Lightroom and pretty much all other converters not provided by a camera maker will take pretty much all file formats, including DNG and proprietary formats. As of today the camera makers' software typically only supports their own proprietary format, and not DNG (with the exception of those who use DNG in their cameras, of course).
     
  13. There are lots of strong opinions on the DNG vs RAW question and I wrestle with it fairly regularly. Here are my thoughts:
    - Incompatible with Camera manufacturer's software - insignificant to me. I use Lightroom and I thought Canon's DPP was pretty goofy.
    - Format may not be supported in the future - insignificant to me. I can't see how anyone can predict that DNG will die before CR2 or NEF dies. If it does, there will be a way to convert DNG to whatever format takes its place. And if there is no conversion software, I'll write it and get rich.
    - Faster, lighter, more powerful - kinda significant. XMP files are not an elegant solution to the metadata problem, but DNG is.
    - Photo competitions - kinda significant. I don't compete, but if I start and have a great photo I would hate to be disqualified because I converted it to DNG. I'm not sure how big of a problem that is, but somebody here said it was an issue.
    I still haven't converted to DNG, but I think about doing it all the time. I also pay attention to the expertise here on Photo.net (and post things like this to see who disagrees with me).
    Fire away!
     
    mike_sea likes this.
  14. And if we talk about the lost of proprietary metadata.... Anyone has any examples of this?? Like what? Should I care? How can this affect us?​
    For instance - if you're a Nikon and NX2 user, the software can read the sets you made via menus, e.g., WB, contrast adjustment, etc., while Adobe converters will not read them all and uses Adobe's converter defaults instead.
     
  15. I still have not found a compelling reason to convert. And glad that I haven't looking back since DNG was
    introduced.
     
  16. So, I've been debating whether or not to convert my CR2 files into Adobe's DNG. I have so many good use for DNGs.​
    Do a search on these forums, it's been discussed a lot.
    do we lose image quality when converting to DNG?​
    No. The only data that isn't usable is the proprietary metadata the manufacturer's raw converter could use (which some could argue could produce better quality than "fill in the blank for raw converter"). DNG is raw data assuming you don't set the conversion preferences incorrectly.
    shouldn't my file be bigger... having the Embed Fast Load Data on also?​
    Not necessarily, depends on the compression used. Embed Fast Load Data speeds up the preview's you see Lightroom (Develop) and ACR. It also doesn't require you use the ACR cache system which has a size limit (it's a rolling cache). That cached data may not be in the same location as the raw therefore not accessible, the Fast Load Data is part of the DNG.
     
  17. The time to convert them...​
    If you use Lightroom (as was admitted), the difference is tiny compared to the entire import process. Plus you can ask LR to write the proprietary raws to another disk at the same time. I doubt many LR users sit around looking at LR while it imports any way.
     
  18. See these threads for some considerations regarding converting to DNG:
    If you do choose to convert to DNG, it's still a good idea to archive the original camera raw, just in case you choose to submit a photo to a contest or news outlet that may require the original camera raw for verification.

    My Ricoh GX100 uses DNG as the raw format, so that's what I use. Works okay, although Lightroom doesn't seem to recognize Ricoh's in-camera JPEG settings well enough to recreate the same looks from DNG in LR4.
    My Nikon digicams use NEF and I don't convert to DNG. No reason to. Lightroom and most editing programs handle NEFs well enough. Nikon's own software (while annoyingly designed, to me) handles the few tricks other programs don't, such as applying in-camera JPEG settings to NEFs if desired.
     
  19. EricM

    EricM Planet Eric

    "How am I losing HD space? My files are smaller in size after converting to DNG, thus my hole questioning about what I am losing. Time however perhaps. The time to convert them..."

    Pat, everyone, including Adobe and it's apostles, suggests you keep your original raw when converting to dng. Assuming you are, you're going to need additional hdd space as you are literally doubling the amount of your files. If for example you have 2Tb of nef's and wanted to convert to dng, you'd need, at best guess, an additional 1.5Tb's of space for those dng's. Now you have 3.5TB's. I back up to more than one medium type. I burn everything to optical disk (twice) and on top that, I make two copies to external hdd's. And because hdd's that haven't been powered up for a long time have the possibility to lock up, I'm diligent and duplicate those again every couple years just to be safe. I have 12 high capacity hdd's at the moment holding my works. These 12 drives do not include the drives along the way that have have been consolidated into them. That sounds confusing. The point is, I don't even want to think back at how much time I would have saved if i didn't do the dng nonsense in the first place. Anyone that says dng speeds up your workflow, doesn't shoot very much.

    Or the money. I just looked at an old ide 120gb drive of mine and it has a $320 price tag lol. Also, today with cloud storage and back up, services like Crash Plan and Carbonite are able to identify the files on your system that have changed since it's last back up, and only upload those changed files to their remote server. When you open or revisit a nef, only its small xmp side car file is altered and only it is uploaded to Crashplan. This is not the case with dng as the whole dng is resent.

    "@Eric: well there's something I don't get with people regarding this. You are talking about some kind of dependancy to Adobe, right? By all means do counter-ague me afterwards but... people say things like they dont want to be stuck with a software like Adobe PS or something to edit, having proprietary file formats and all. Isn't just keeping your Camera RAW files the same!??"
    Dependency? No. I'm talking about choices or options. For example, today my native raw nef opens in Adobe, Nx2, and Dx0. Out of those three, my dng only opens in Adobe. Less software choices is one thing, but maybe more important is that there's a terrific amount of info in that native raw that gets thrown away.
    "I still have not found a compelling reason to convert. And glad that I haven't looking back since DNG was introduced."
    I wish I listened to you and others sooner. Instead, I swallowed the pill and started when dng was released a decade ago. I've concluded it's a complete waste of my resources.
     
  20. Instead of "Converting to DNG: What are we losing?"... "Converting to DNG: What are we gaining?".
    I still cannot find a reason in which DNG gives me something my NEFs do not have. DNG never became the standard it was hoped to be, and while the current situation with proprietary files is far from ideal, betting on a standard with very lacklustre support from the industry it is intended for, isn't a safe bet either. The argument that somebody might pull support for the RAW file format of an older model... so far, it simply did not happen yet, as nobody has much to gain with doing so. And there is plenty open-source code already to deal with those files, so there are and will be solutions.
    DNG would be an excellent idea if the camera-making industry supported it properly. As it is, NEFs (and CR2 possibly as well) simply do contain more metadata than DNG, and I see no compelling reason at all to switch.
     
  21. Pat, everyone, including Adobe and it's apostles, suggests you keep your original raw when converting to dng​
    I don't (and don't keep them). I see no reason. I have piles of CD's filled with proprietary files, raw and otherwise that are inaccessible unless I want to find a 12-15 year old OS and hardware that can read those files. Proprietary file formats are never a guarantee you can access your data. I've been shooting with DLSR's since one of the first (Kodak DCS-1) which at the time only spit out a TIFF. But onward, I have nearly every model of DCS camera that today can't be accessed without really old hardware and software (I did render the images I want and now they live as TIFFs, but the raws are mostly stored on drink coasters). Ditto with Live Picture iView or X-Rez proprietary file formats.
    "Converting to DNG: What are we gaining?".
    I still cannot find a reason in which DNG gives me something my NEFs do not have.​
    Thank you, that is an excellent question!
    In my case there are several items I gain:
    1. Smaller file on disk.
    2. Ability to embed multiple DNG profiles. I build them and use them and want them part of the DNG container, not stored deep in my system folder. Ditto with all the metadata that has to be separate with a proprietary raw (with Adobe converters). If I move from desktop to laptop to another location machine, ALL the necessary data is in the container along with the raw. Try this: Use a custom DNG profile and make the image look as you desire. Now move the NEF or CRW to another machine. You better also move that DNG profile with the original (and side car) or you're in for a big visual surprise.
    3. Embedded JPEG that shows the current rendering (if you set LR/ACR correctly). Extract it if you had an emergency. Or you just need something bigger than a tiny thumbnail preview.
    4. Fast Edit Preview. I already discussed this. On my machine, the differences between initial building then loading the ACR cache for Develop vs. Fast Load is significant (nearly 1.5X faster). Of course I never have to worry about losing the ACR rolling cache data, let along if it's stored on another machine. It's part of the DNG.
    5. Work with Virtual Copies in LR? Well that's again just a set of instructions that are proprietary within LR. But if you wanted a self contained raw with all the work you just did, export as DNG. That's the only option to save off a raw file but of course if you're sure you are done editing forever, save out a TIFF.
    ANY of the above items for ME make the chose of DNG simple. It doesn't take any more time in the grand scheme of things to convert to DNG while importing than not converting to DNG when converting. I'm just not sitting there watching the import.
    The bigger issue is that proprietary raw (proprietary anything) is never as good for the customer rather than an open format. Kodak DCS, Kodak Photo CD image pack, iVIEW files, etc. The JPEG generated by the newest camera to hit the market can be opened in my very, very, very old copy of Photoshop 1.0.7! The newest raw from Canon may not be accessible by the last version of Photoshop CS5. How is that good for photographers? You're either part of the solution (demanding an open raw format) or you're part of the problem. Not complaining to the camera manufacture's about this, dismissing DNG instead doesn't help in solving this problem. And it's a problem.
    DNG never became the standard it was hoped to be​
    Standards: This is a straw man, forget it. Is NEF or CRW a 'standard'? By what body? What does it mean? Your NEF is a standard because your camera club recommends it or more people use it than DNG? Doesn't make it a standard. FWIW, DNG IS working on becoming a standard in terms of recognitions by a Standards Body (ISO). It's actually just a TIFF (which Adobe owns, maybe that's why it's so scary). But if it makes it or doesn't is not important. NEF or CRW is not a standard as there are dozen's upon dozen's of different NEF's and CRWs. If they were the same, the poster here who just purchased a new 6D wouldn't have any difficulty using ACR 6. Bit that CRW is different from the last CRW Canon. It should really be CRW-6d, CRW-5DMII, CRW-5D etc. Same with NEF. They are all different. So much for standards.
    IF you are only going to use the manufacturer's raw converter, you DO NOT want to convert to DNG! All you posters who only use the manufacturer's converter, raise your hand. The rest of you using say Adobe raw converters, keep the original NEF's or CRW's if you want. I've been doing this long enough to have been burned and don't see how the original proprietary format does me any good unless I suspect I'll switch to (in my case) Canon's software. About as likely as my going back to shooting film.
    Bottom line. Like Gay marriage, if you're against it, don't marry a gay person: If you don't like DNG, don't use it. But dismissing the preferences of others, based on what they feel are the advantages of either is a personal choice most make with at least some thought. This isn't directed at any one person posting, only people who dismiss a format and workflow I and others find VERY useful. There's no reason to dismiss it, don't use it and move on. In terms of DNG, dismissal doesn't do much to help fellow photographers in controlling the data the camera they purchased provides. There is nothing, nothing stopping either Nikon or Canon from putting a 3rd option on the camera to write a DNG along with the JPEG or Proprietary raw. It's totally political. If everyone who purchased a DSLR demand an open raw format, we'd have one. I have no problem who marries who or what format you set your camera at. I do have an issue when I'm told I can't do something that harms no one and helps a lot of others. DNG is just such an option.
     
  22. Andrew, thanks for balancing the discussion with clear, proper reasons to choose for DNG. While I do not share your concerns on the proprietary file formats nor your issues with my use of the word "standard" (nowhere ever did I claim NEF was a standard?), I see that in your workflow (and hence, for quite some others) there are clear advantages (and I did not mean to argue with that) I am glad the discussion shows both sides of the coin this way - it'll help other make their informed decision in what suits them best.
     
  23. EricM

    EricM Planet Eric

    It's a shame that most of the info from the openraw.org movement is now gone and removed from the net. There's still a few little bread crumbs around though. It clearly spells out how dng isn't the "open source for profit" solution that the consumer or vendors are needing.

    FWIW, DNG IS working on becoming a standard in terms of recognitions by a Standards Body (ISO)."

    They been "working on it" since 2003. It's been so long, it's not even iso9000 anymore.

    "IF you are only going to use the manufacturer's raw converter, you DO NOT want to convert to DNG! All you posters who only use the manufacturer's converter, raise your hand."

    How wonderful it would be if the 20 digital darkroom contributors here at PN accurately represented the real world. Most people that own dslrs simply use the included software that is bundled with their $800 kit from Costco. The reality is that only a small percentage of dslrs owners take it to the next level and purchase Adobe products. Because of this small %, there is no incentive for Nikon and Canon et al to output a dng.
     
  24. They been "working on it" since 2003. It's been so long, it's not even iso9000 anymore.​
    As I said, it's a questionable and useful goal in any event. Point is, Adobe has tried to make it as open to everyone as possible. This also points out to the "it's not a standard" crowed how difficult and how long it takes for standards bodies to do anything. As a member of such an organization, I know too well they are usually represented by differing interests (some political), and getting anything done takes forever.
    How wonderful it would be if the 20 digital darkroom contributors here at PN accurately represented the real world.​
    If every DD contributors here at PN agreed on something, it wouldn't represent the so called, undefined 'real world'. The interesting thing about using such a term (real world) is it's so ambiguous, and anyone can say an opinion represents the real world or it doesn't. So why use some language?
    That said you could head over to Luminous Landscape which gets 1.1 million unique readers each month; 3.5 million page views from some 50,000 people a day, you'll certainly have the possibility to gauge a larger number of contributors. And if 25% agreed upon using DNG while 75% didn't, that would mean what in terms of the points I've outlined above (having choices, getting the manufacturer's to build something for me?). Do you have a point about the qualities of a workflow option that some prefer that in some way is just wrong and should be dismissed and avoided?
    Most people that own dslrs simply use the included software that is bundled with their $800 kit from Costco.

    The reality is that only a small percentage of dslrs owners take it to the next level and purchase Adobe products. Because of this small %, there is no incentive for Nikon and Canon et al to output a dng.​
    You can backup such metrics where? Just want to see if this is a real data point or an opinion.
    Let's say however this is true. The fix is education such that users WILL ask for DNG or an open raw format that fixes all this mess with proprietary raw's. Again, you're either part of the solution or part of the problem. Responsible for answering questions (the OP asked what do I lose converting to DNG) or you're part of the problem (continuing to accept the options forced on you). There is absolutely nothing to be lost by adding a 3rd option for data format on the camera and lots for some to be gained. If you don't want that, step out of the way please.
     
  25. I have piles of CD's filled with proprietary files, raw and otherwise that are inaccessible unless I want to find a 12-15 year old OS and hardware that can read those files.​
    Andrew, check with Isobuster if they can extract those files for you. They don't use windows to do so, as far as I know. It is worth sending them an email. The program managed to extract the data from my old UDF files.
     
  26. EricM

    EricM Planet Eric

    "The fix is education such that users WILL ask for DNG or an open raw format that fixes all this mess with proprietary raw's. Again, you're either part of the solution or part of the problem."


    There obviously isn't a mess with proprietary raw's as the world is getting along just fine without dng. The only mess is a software company getting into the mix of it all and wanting control of the imaging industry. Heck, I don't blame them, it's great business. But to the actual photographers that shoot, the sky isn't falling nor are the majority asking for dng.

    "There is absolutely nothing to be lost by adding a 3rd option for data format on the camera and lots for some to be gained. If you don't want that, step out of the way please."

    There sure is something to be lost. You've said so yourself and admitted it's happened to you and the evidence is the stacks of your cd's and a file format(s) you can't open. Yet here you are doing it again and throwing away your native raws after you convert to dng. Bizarre.
     
  27. There obviously isn't a mess with proprietary raw's as the world is getting along just fine without dng​
    Well there are people here (as recently as this week) that MUST convert to DNG because they have a newer camera and an older version of their (fill in the blank raw converter). They HAVE to use the manufacturer's raw converter, or maybe upgrade their software when neither would be necessary IF the new camera just spit out an open raw file like a JPEG. Since this obviously doesn't affect you, you don't care it affects others:

    http://www.photo.net/digital-darkroom-forum/00bIag

    This must be the real world you refer to, where people get even fewer options with their new camera system thanks solely to the refusal of the camera maker to provide just one simple option.
    But to the actual photographers that shoot, the sky isn't falling nor are the majority asking for dng.​
    I'm not sure if you are or I am an actual photographer*, again such language is used to confuse the discussion. But since you refer again to some group of people other than yourself, add that please to my request you provide some metrics for this and the former statements. As I said, I'm trying to decided if what you write warrants my change of opinion on the topic based on actual facts.
    You've said so yourself and admitted it's happened to you and the evidence is the stacks of your cd's and a file format(s) you can't open.​
    Yup, thanks to a proprietary format. Nothing more. I don't care if the new camera switch gives me a DNG or anything Nikon/Canon and the rest world to call their (new) open raw format that behaves exactly like the JPEG they also write. I simply want a raw format that doesn't behave like the current proprietary raw's that require everyone to recode their converters to open my data.
    I suppose in your real world, such a request is unreasonable. Why isn't clear.
    *Being yet another anonymous poster, I don't know if you even exist <g>
     
  28. The point is, I don't even want to think back at how much time I would have saved if i didn't do the dng nonsense in the first place. Anyone that says dng speeds up your workflow, doesn't shoot very much.​
    It's these kinds of statements that make me wonder if you're someone with an agenda. First, adding 'nonsense' in the sentence is again more editorial language that isn't necessary. I can easily prove in my workflow that DNG does speed up my processing even when I have to import many gigs of images. I do believe I'm someone. I do think others share my workflow.
    Did a DNG kill one of your parents? What's the cause of the bias? You have actual data to share to convince me I'm missing something, I'm all ears.
     
  29. Guys, it's a file format, come on. There is choice. Good. Andrew takes DNG, Eric takes CR2/NEF/... (as will I, btw). The only thing that really matters is finding a workflow that works for you. The choice for DNG can be part of finding that, as can be the choice for sticking to the manufacterers original. There is no right and wrong here.
     
  30. The only thing that really matters is finding a workflow that works for you
    Agreed!
    Getting back OT, the question was legitimate and the specific answer in the 2nd post were correct. No editorial non-fact based posts prior to that thankfully. That should aid someone new to the topic hoping for well-grounded education on the differences. Then they can make up their own minds.
     
  31. EricM

    EricM Planet Eric

    "Guys, it's a file format, come on."
    You're right. Good call. I can't believe I fell for it.

    "It's these kinds of statements that make me wonder if you're someone with an agenda. First, adding 'nonsense' in the sentence is again more editorial language that isn't necessary."

    Mr. Pot, meet Mr. Kettle.
     
  32. I absolutely have an agenda, that should be obvious. I want an open raw format.
    The differences are I don't have to insult anyone or make up stuff to express my POV. I'm happy to answer any questions as to why I do what I do. Big difference.
     
  33. I have a DNG workflow (scripted with a Cygwin import script and DNG converter running in command line mode) since I have an older Photoshop that does not do 5DMKIII RAW. No, I do not like the Canon workflow. I do not like "sidecar XMP" files to track the metadata that changed in my CR2 file. They get lost.
    I would like to have a standard file format which I hoped DNG would be. But, since I have been associated with standard bodies like 3GPP and others I know that they suck and can never make a decision and so I am not holding my breath. I really do not want to rely on proprietary formats like NEF and CR2. Incidentally the DNG file can embed the original file but this blows up the file size.
    On a similar front I need to convert all those DAT tapes I made for the videos I shot into plain ISO images on my jukeboxed hard drives so I don't lose them. Proprietary formats suck. We really need a good standard that will be around in the future and I will continue to use DNG - that is until I have to convert everything to who knows what down the line before DNG goes obsolete.
     
  34. I have read the DNG specification.
    • The modifications saved to the DNG are application specific and may not work across different products.
    • The DNG specification comes in different versions (from 1.0 to 1.4) and is not backwards compatible.
    • How much of the original data is preserved in one form or the other depends on the DNG converters ability to understand the raw file.
    • The raw converter used to render the DNG into a tiff or jpeg may still need to support a specific camera to be able to render a high quality image.
    • Other sensor technologies like Foveon X3 are not supported by the DNG format.
    My take on it would be that DNG is not as universal as it might seem. But it would work nicely in a Adobe centric workflow because Adobe would make sure of that as long as you have relatively recent software. I would consider it an intermediate "almost"-raw format.
    For pure archival use it would be better to render the images into white balanced, neutral, unprocessed 16 bit prophoto rgb tiff as a high resolution image, a jpeg as a compressed rendering lower quality image and keep the original raw file as the Original Image.
    Storage space may be an issue but since I have all my digital images since 2000 stored I'd say it's just a matter of spending enough cash on storage. As storage capacity is increased raw files size from example a D800 will appear insignificant in size.
    Anyway I would not delete the original raw file if the image is worth keeping at all.
     
    • The modifications saved to the DNG are application specific and may not work across different products.
    • The DNG specification comes in different versions (from 1.0 to 1.4) and is not backwards compatible.
    • How much of the original data is preserved in one form or the other depends on the DNG converters ability to understand the raw file.
    This is from the ASMP under Best Practices! Read the entire article by Peter or just the backwards compatibly section.
    http://dpbestflow.org/DNG#backwards-compatibility
    Backwards compatibility

    The DNG format makes backwards compatibility possible in several different ways. Sometimes this is done without any loss of quality, and sometimes it requires “baking in” some of the changes to the raw file, which does remove some flexibility to re-edit your files. Let’s look at the issues.
    Enhance camera compatibility

    Sometimes DNG backwards compatibility is used to make a newer camera compatible with older software. Because software must be engineered to decode each particular flavor of raw file, a new camera’s raw file may be incompatible with older software. DNG provides an orderly restructuring of the file data, and a color profile for the camera model, so that even older software may be able to open a newer file.
    This is often used by people who own older versions of Photoshop and want to open files from new cameras. It’s also a common work-around for Aperture users with this same issue. In most cases, there is no loss of quality, particularly if it’s possible to use the most recent compatibility setting when you do the DNG conversion.
    Remove incompatible features

    Sometimes the backwards compatibility must be done by linearizing the DNG because some new features are not supported by older software. For instance, the Panasonic LX5 camera requires some image de-warping in the raw conversion in order to remove lens distortion. Older Camera Raw versions did not support this function. In order to open these new files with old software, the DNG must be created with the de-warping baked in to the DNG, which is done by selecting a Compatibility prior to the latest version, as shown in Figure 5. This is a “lossy” operation, since you’ll be making changes that can’t be undone.
    [​IMG]
    FIGURE 6 In order to make a DNG compatible with an earlier software version, you may need to choose a compatibility setting that is from a previous specification. Only use these when your testing shows that current compatibility is not supported with your combination of camera and software.
    Only choose backwards compatibility because you need to

    So how do you know if the DNG you are making is baking in some kind of changes, or if you even need to? Here’s the simple rule. Only choose the backwards compatibility because you need to – and you only need to because the most recent compatibility version does not work. That is, the file won’t open in the older software when the DNG was made with a more recent compatibility.
    Sometimes you may want to choose an older compatibility version because you need to send the file to someone who does not own the latest software. Sometimes you might need to do this because you don’t own the latest software.
    Consider embedding the original whenever you choose earlier compatibility

    Whenever you are forced to use an older compatibility, you are baking in some kind of changes. This is a time to consider embedding the raw file in the DNG, particularly if you need to set backwards compatibility to your own copy of the DNG.
    Let’s take the case of the LX5 raw files outlined above. If you are opening these with Photoshop CS3, and therefore need to choose Camera Raw 4.6 as the compatibility version, we strongly suggest that you choose to embed the raw file in the DNG, as shown in Figure 7. This will provide you with the full rawness of the image when you eventually get software that supports the LX5 natively. 
     
  35. Another useful read from the ASMP. Apparently they too feel this is a mess Mr. Eric (not that the ASMP is part of your real world of photography):
    http://dpbestflow.org/node/634
    Proprietary raw considerations

    Raw image technology is a fast-moving area, and a major focus of development for camera companies. This has led to a bit of a mess with respect to raw file standardization. As camera companies implement new sensors in cameras, or develop new ways to render color or reduce noise, the structure of the raw file can change. Let’s take a look at some of the characteristics and ramifications of proprietary raw formats.
     
  36. EricM

    EricM Planet Eric

    "The DNG format makes backwards compatibility possible in several different ways. Sometimes this is done without any loss of quality, and sometimes it requires “baking in” some of the changes to the raw file, which does remove some flexibility to re-edit your files. Let’s look at the issues."

    Is this dng "baking in some of the changes" a non issue with nef files? In Nx2 or Adobe?
    Mr. Eric
     
  37. Is this dng "baking in some of the changes" a non issue with nef files? In Nx2 or Adobe?​
    It is a non issue yes. Read the example again. Let’s take the case of the LX5 raw files outlined above. A NEF loses none of it's rawness.
     
  38. EricM

    EricM Planet Eric

    An Lx5 that is baked into the dng, is not what I asked.
     
  39. An Lx5 that is baked into the dng, is not what I asked​
    Reread the entire article again OR just what I posted to your first question:
    It is a NON ISSUE, a NEF loses none of it's rawness when converted to DNG (if converted properly, eg not lossy). This was discussed clearly by several posters a dozen posts back in answer to the OP (what do I lose by converting?). Again, for about the 4th time, all that is lost is proprietary metadata only the Nikon converter could use anyway. IF you are NOT using the Nikon converter, you're not missing nor need this metadata one bit (literally).
     
  40. EricM

    EricM Planet Eric

    ""The DNG format makes backwards compatibility possible in several different ways. Sometimes this is done without any loss of quality, and sometimes it requires “baking in” some of the changes to the raw file, which does remove some flexibility to re-edit your files. Let’s look at the issues."
    I'll just start over and re-phrase my question. What if you never converted your nef in the first place and just used your nef in Adobe products or Nx2 all these years, do any changes or previous edits get "baked in" into the nef and remove flexibility to re-edit?
     
  41. if you never converted your nef in the first place and just used your nef in Adobe products or Nx2 all these years, do any changes or previous edits get "baked in" into the nef and remove flexibility to re-edit?​
    Nx2 no. Whatever XMP it writes is proprietary to it. Adobe, it could be everything you've done from day one. Adobe treats proprietary raw as read only (the single exception, since I want to be precise as an example, is that you can write date/time changes if you turn on a preference). If you had XMP and the NEF then converted to DNG, you'd have a raw file with all your edits from the Adobe converter.
     
  42. EricM

    EricM Planet Eric

    So, just to be clear, if I never used dng, and instead used nef and xmp from day one (in Adobe products), I wouldn't have the dng problem of “baking in” some of the changes to the raw file, which does remove some flexibility to re-edit your files"?
     
  43. I wouldn't have the dng problem of“baking in” some of the changes to the raw file, which does remove some flexibility to re-edit your files"?

    DNG is containing the same raw data as the NEF. It doesn't bake/change anything unless you ask for either a linear DNG or lossy DNG.
     
  44. EricM

    EricM Planet Eric

    Just a yes or no, to my question please
     
  45. I've answered all your questions while you've answered none of mine. Should I keep communicating with you based on your attitude in this series of posts?
    Examples of hyperboles and non substantiated data (opinions with nothing to back it up):
    Pat, everyone, including Adobe and it's apostles, suggests you keep your original raw when converting to dng.
    DNG conversion has lost its popularity as it never became the industry standard we hoped it to be.
    The point is, I don't even want to think back at how much time I would have saved if i didn't do the dng nonsense in the first place. Anyone that says dng speeds up your workflow, doesn't shoot very much.
    Instead, I swallowed the pill and started when dng was released a decade ago. I've concluded it's a complete waste of my resources. (note: DNG was released September 2004)
    Most people that own dslrs simply use the included software that is bundled with their $800 kit from Costco. The reality is that only a small percentage of dslrs owners take it to the next level and purchase Adobe products. Because of this small %, there is no incentive for Nikon and Canon et al to output a dng.
    The only mess is a software company getting into the mix of it all and wanting control of the imaging industry.
    I can't believe I fell for it.​
    Show us some facts to back up the above. Otherwise you're wasting some people's time.
     
  46. EricM

    EricM Planet Eric

    I guess we'll just take your distractions as a resounding "no". No, when using nef/xmp files, we do not have the same dng problem of “baking in” some of the changes to the raw file, which does remove some flexibility to re-edit your files"
     
  47. You could always convert your stored CR2 files in 20 years (with DNG or so many other free raw converters), if needed. I mean, just look at a program like Irfanview how many types of files it can read. Old operating systems and software can be run in virtual machines. I'd be more worried about long-term data integrity of storage media.
     
  48. I'd be more worried about long-term data integrity of storage media.​
    Absolutely and here again, DNG has an advantage as Peter discusses in the ASMP article.
    There's also this:
    http://dpbestflow.org/data-validation/dng-validation
     
  49. EricM

    EricM Planet Eric

    Anyone else remember Adobe dropping the .jpg2000 format? I have a few of those. That was another Adobe "solution" as well.
    I feel the same, Oscar. The sky is not falling and feel there is no rush to dng convert. As just pointed out, you bake in your edits with dng and is not future proof with even its own dng converter down the road. That's not something I want or is remotely close for those that started the OpenRaw project.
    I use linux distro's a fair amount and feel that instead of open-source (for profit) raw format like a corporate controlled dng, we'll instead have true community open-source (not for profit) raw converters like dcraw that will read anything. DNG was invented by a company that has an interest in locking in your business and was an answer to a question that no one was asking in the first place.
     
  50. Anyone else remember Adobe dropping the .jpg2000 format?​
    Nope, because they just didn't install, by default the JPEG2000 plug-in, you have to do it. Or download another of your choice. The original works just fine in my copy of CS6 and this one does as well:
    http://www.fnordware.com/j2k/
    As just pointed out, you bake in your edits with dng​
    In the raw/DNG? No you don't. You're making either XMP metadata instructions or you're saving a rendered JPEG of considerable size of the current rendering within the DNG (along with other useful data). The raw data is still raw. READ the articles by Peter Krogh from the ASMP!
    Get the facts straight, enough made up stuff please.
     
  51. In the other real world, even Save for Web supports JPEG 2000 within CS6:
    [​IMG]
     
  52. EricM

    EricM Planet Eric

    "Made up stuff", that's funny after dodging direct questions. Here's more, where do you get this stuff?
    "IF you are NOT using the Nikon converter, you're not missing nor need this metadata one bit (literally)."

    More (made up) biased opinions yet wrapped up as a fact. Perhaps it's best to tell us what is in this info that is being discarded and people can make up there own minds. There is all sorts of camera model-specific info and data used for lens corrections etc that gets discarded.
     
  53. There is all sorts of camera model-specific info and data used for lens corrections etc that gets discarded.​
    OK, and just what raw processor(s) can use it? The answer has been correctly stated several times now. Again, proprietary info, only one converter can use can't be used by any other converter. More so, even though it is proprietary, it can still be embedded into the DNG.
    For the last time, the degree of rawness of a converted NEF or CRW is exactly as raw as it was before the conversion. Understood?
    Made up stuff? Like your continuing incorrect statement about JPEG 2000 and Adobe, made up or just pure ignorance.
     
  54. Here's more, where do you get this stuff?​
    I read up on the subjects, written by experts in the industry who support peer review. You should try it some time. You might be less inclined to write things that are completely incorrect and untrue. I've provided several links, just three today from the ASMP. You have a beef with what they wrote? I suppose we can be kind and call you a peer, so please, peer review the text and as I know Peter well, I'll pass along the corrections to him so the ASMP can review your findings.

    You've made up a number of metrics about the market with zero reference. You were dead wrong about JPEG 2000. You seem to still misunderstand the effect on raw data before and after conversion to DNG despite my attempt to explain it to you several times. Other poster's have said the same things about proprietary metadata.

    You're not to be taken seriously based on your track record here!
     
  55. EricM

    EricM Planet Eric

    "OK, and just what raw processor(s) can use it? The answer has been correctly stated several times now. Again, proprietary info, only one converter can use can't be used by any other converter."

    I understand Dx0 uses it. As well as Nx2 of course.

    "For the last time, the degree of rawness of a converted NEF or CRW is exactly as raw as it was before the conversion. Understood?"

    I understand alright. That you wont answer direct questions that gets to the truth of the matter. I'm glad to be re-assured that when using nef/xmp files, I wont have the same dng problem of “baking in” some of the changes and in which decreases my flexibility to re-edit.
     
  56. I understand Dx0 uses it​
    You understand or you know? If it's proprietary, Dx0 can't understand it. If it isn't proprietary, most converters which have the capabilities to use it will use it if they find it useful. LR/ACR didn't originally have any lens correction. So whatever was written, proprietary or open metadata (like the lens type, aperture) was seen. LR/ACR got lens correction. EVEN if the lens data isn’t' proprietary, it might not use it. It's a totally different technology.
    IF you understand what the term proprietary (in this case metadata) means, then you should have zero problems understanding what is and isn't lost. And even if this NEF metadata is gone, you're using a different converter which may do a much better job at lens correction among other features.
    That you wont answer direct questions that gets to the truth of the matter.​
    You should start trying to answer questions, as thus far you're way, way behind.
    I'm glad to be re-assured that when using nef/xmp files, I wont have the same dng problem of “baking in” some of the changes and in which decreases my flexibility to re-edit.​
    That was never an issue from day one although for some reason, you found it necessary to take us all down that rabbit hole.
     
  57. EricM

    EricM Planet Eric

    It's a simple yes or no. If I never used dng, and instead used nef and xmp from day one (in Adobe products), I wouldn't have the dng problem of “baking in” some of the changes to the raw file, which does remove some flexibility to re-edit your files"?
    Rabbit holes. Like this one?
    http://www.photo.net/digital-darkroom-forum/00bCAi
    Have a good day.
     
  58. Just for the fun of it I ran some D600 files through the latest DNG converter 7.2 from Adobe.
    D600 and all Nikons AFAIK embeds two jpeg images into their raw files, one low resolution preview and one full resolution jpeg.
    If you setup the DNG converter to JPEG Preview: Full Size the size of the DNG is a few hundreds KB larger than the original raw file. So when the DNG files are smaller it's just because the embedded JPEG is smaller. Otherwise DNG files are roughly the same size.
    What I did find however was that the DNG converter actually does a raw conversion to make a preview image instead of taking the one the camera put in the raw file. The DNG converter by default also makes a very saturated preview image so it doesn't look like the original in the camera.
    The EXIF info in the DNG seems to be identical to the EXIF in the raw file, except that I couldn't find the user comment field in the DNG. That is a information field, for instance a copyright notice, that you can enter in the camera and it will end up in raw files and jpeg images.
    One use I found for DNG while playing with this is to take full resolution raw and turn them into lower resolution raw. That could be valuable for instance for D800 users that want a fast raw workflow but doesn't really need all 36 megapixels for 99% of the shots. For instance wedding shooters, where the bulk of the images never will be used bigger than for 8x10s (about 8 megapixels required).
     
  59. D600 and all Nikons AFAIK embeds two jpeg images into their raw files, one low resolution preview and one full resolution jpeg.​
    I don't believe that's the case with Canon raws. I believe there is only a small JPEG that matches what was shown on the LCD of the camera. As LR imports this (either raw or DNG), it shows for a second, then it is updated since ACR/LR have to replace it with it's own rendering of a preview. IOW, that initial preview from the embedded JPEG isn't useful and gets replaced. This causes a lot of questions on the UtoU forums as people think something is wrong and want the original preview back. But except for the manufacturer's converter, all other raw converters have to build their own preview using their own engines.
    If you setup the DNG converter to JPEG Preview: Full Size the size of the DNG is a few hundreds KB larger than the original raw file.​
    In this case, the current rendering of the raw converter is what is built. You can update this at any time (in LR, you have to use the update DNG metadata and preview command). Even if Auto Update XMP is on, you do not get an updated preview without using this command. Which makes sense, you would not want to have LR update the JPEG automatically.
    What I did find however was that the DNG converter actually does a raw conversion to make a preview image instead of taking the one the camera put in the raw file.​
    Correct, it has to for the reasons I outlined above. This would be true if you didn't have a DNG, the preview has to be created (and you can control the size in preferences) as the original preview isn't correct, it was built by the camera from the raw. Instead of the preview being embedded in the DNG, it's part of the LR database. Note too there are more than one preview in LR (one set is used in all modules expect Develop, the other is partially rendered and is either the ACR cache OR the DNG Fast Load data).
    For instance wedding shooters, where the bulk of the images never will be used bigger than for 8x10s (about 8 megapixels required).​
    The other option would be lossy DNG. The two are quite different. One is the full rez but partially processed (Lossy), the option you speak of I believe is still fully raw but smaller?
     

Share This Page