RX100 .ARW to DNG--crazy big file sizes

Discussion in 'Digital Darkroom' started by catcher, Jul 5, 2014.

  1. All,
    I just purchased an RX100 (first version). I have an old version of Photoshop (CS3), so I'm using .dng converter to convert the proprietary Sony files to .dng. Upgrading Photoshop is not an option; nor is getting the newest LR as I have an old Vista 32 bit system.
    Anyway, here's the question: when I convert the .ARW to .DNG the file sizes jump from about 20mb (ARW) to anywhere from 45-85mb (DNG). I've never seen anything like that from my Canon RAW files. I couldn't find much online, but I did find something that suggested Sony used codecs that were not compatible with DNG?
    Anybody else have any experience with this? I'm using the same settings as when I convert my Canon 6D files and they're very close in size to the Canon Raw. Are these large files just something to live with or am I missing something?
    Thanks.
     
  2. in the DNG converter, there is an option to 'embed' the original raw image. That will immediately double+ the size of the .dng. Check to see if you have that turned on.
     
  3. Yes. I just tried it with LR5 and got a .DNG slightly smaller than the .ARW file at about 20Mb.
     
  4. I do not have it set to embed the original raw image. So, that is not it.
    Does LR5 use a different converter? Will, is this with an RX100 (first version)?
     
  5. Aaron, I'm using an RX100-2 but I just tried LR5 on a RX100-1 .ARW file from someone else. The difference was larger but still not drastically so.
    00cggF-549547784.jpg
     
  6. Update: Because I'm using CS3, the last version of Camera Raw I can use is 4.6. When I got the huge .DNG files after conversion, I had DNG Converter set to be compatible with CR 4.1. I tried doing the conversion with compatibility set to 7.1. That produced an reasonable file size--just about the same size as the .ARW. But, of course, I can't open it because my version of Camera Raw is too old.
    So, I guess that's the problem. Anybody know why that would be the case? What changed in recent versions of Camera Raw that would make it possible to have a .DNG file one half to one third the size of a .DNG file for an older version of Camera Raw? Anybody got any idea of a workaround?
    I suppose it's not a huge deal. I can still edit the files. And storage is pretty cheap, all things considered.
     
  7. It really sounds like in the original conversion, you had the converter set to embed the original raw, hence the larger size. Maybe when you switched to a different DNG version, that was undone. You can test this yourself of course by toggling the settings for version and making sure no other settings differ and examining the size. I don't recall the older vesions having a big impact on size by YMMV. Double check by running the test again OR see if the bigger image you have allows the DNG converter to extract the original proprietary raw (if it's in there, it will).
     
  8. EricM

    EricM Planet Eric

    Anybody got any idea of a workaround? I suppose it's not a huge deal. I can still edit the files. And storage is pretty cheap, all things considered.​
    Instead of using CS3, how about paying $10/mnth for the newest PS and LR versions and benefiting from the modern/improved raw conversion engine? At the same time, the bonus is not having to worry about the extra and useless steps of dng conversion.
     
  9. EricM

    EricM Planet Eric

    Upgrading Photoshop is not an option; nor is getting the newest LR as I have an old Vista 32 bit system.​
    Opps, I should have read your post, Aaron!
     
  10. Opps, I should have read your post, Aaron!​
    A common fault of yours. But for once, we agree, upgrading to the subscription for the better raw processing is an excellent idea in general!
     
  11. Thanks for the responses.
    Just to confirm, I did NOT have it set to embed the original raw files.
     
  12. Be useful if you could upload one camera raw so we could test different options with newer versions of the DNG converter. Could be a bug in your older product I suppose.
     
  13. EricM

    EricM Planet Eric

    Aaron, I have that wonderful camera as well. I use the latest Adobe software and you're correct, the dng is much larger.
    [​IMG]
     
  14. Eric, Aaron isn't talking about a 3 Mb jump in size, he's upset by "the file sizes jump from about 20mb (ARW) to anywhere from 45-85mb (DNG)". Your size increases are about what I'm getting and no big deal.
     
  15. EricM

    EricM Planet Eric

    Thanks Will, well aware of the discrepancy after coffee and reading properly the second time, cheers. I've been using DNG since it came out and never noticed it make a larger file...
    I just delivered 2,410 dng files from a mixture of D800, D3s, and D700 cameras and the folder was 72.1 GB. The folder with the nefs was 72.6 GB...that's 0.5gb savings for 2400 files...so much for the "reduced storage with dng" theory.
     
  16. ted_marcus|1

    ted_marcus|1 Ted R. Marcus

    I'm in a similar situation, using DNG to let me process raw files from a Canon SL1 with Photoshop CS5. DNG files from the converter average 18% smaller than the original CR2 files, which helps with archiving the files to DVD with the finished images (I archive the original CR2 files separately). The conversion step is reasonably quick and painless. The converter even includes an option to read the subdirectories of an SD card and output all the DNG files to a single directory, which I otherwise would have to do manually.
    For me, using DNG is a cost-effective choice, at least for now. I see no compelling reason to upgrade CS5, and I particularly don't see the supposed benefits of renting software over a "traditional" perpetual license. (Actually, I understand how the new model benefits professionals whose livelihood depends on having the latest versions of multiple Adobe tools, and also how it benefits Adobe's shareholders. I just don't see how it benefits me.) I may go to Lightroom in the future, but I'm not yet willing to spend the money for it, or to change to the database-centric workflow it demands. I'm not entirely convinced that DNG is the future, or that it it is any more "archival" than the native formats of popular cameras, so I can't say that I'm an enthusiastic evangelist for DNG. But it's the best option I have at the moment, and it seems to be working well for me.
    I suspect the OP's problem of large file size is related to making the files compatible with CS3. It might need to load camera profiles (or some other data) that the CS3 Camera Raw doesn't provide natively. If that's the case, there might not be any way around the large files.
     
  17. All,
    Thanks for the continued input. I've attached (I think) an .ARW file from my RX100. When I convert this to .DNG I get a file around 79MB. I'm using Adobe Converter 8.3. I have it set to make it compatible with Camera Raw 4.6 (I'm not going to update to a newer version anytime soon). To be clear, I do NOT have it set to embed the original raw file.
    If anyone has a hankering to give it a try, here's the file: DSC02255.ARW
     
  18. EricM

    EricM Planet Eric

    With LR 5.4 exporting as dng, I converted with 7.1 Camera Raw and it was 24.1MB dng, then converted with 4.1 Camera Raw like you, and it was, wow, 77.3MB dng
     
  19. Converted DSC02255.ARW with latest DNG converter. Was 21.4MBb, DNG is 25.2MB.
     
  20. EricM

    EricM Planet Eric

    The exercise is to convert it with 4.6, like Aarron, and report ones findings
     
  21. It must have something to do with converting so that the .DNG is compatible with Camera Raw 4.6 or earlier. I wonder what the difference could be, though? I convert Canon 6D files with no problem. What would be different about Sony .ARW files from the RX100?
    Oh well.
     
  22. The exercise is to convert it with 4.6, like Aarron, and report ones findings.​
    No it's not. You've failed again to properly read or comprehend posts as the test that follows is quite clear (I wrote it):
    Be useful if you could upload one camera raw so we could test different options with newer versions of the DNG converter.
    Newer versions of the converter do not produce the large increase in size the OP reported so the solution is simple based on a proper test as I've conducted: Upgrade such a newer version of the DNG converter can be utilized or stick with the current workflow, understanding the size issue is due to an older versions routine. Further, using the most current version of the DNG converter, IF one selects the oldest format (2.4 and later), the same NEF becomes 81MB on disk! So it's clear what the issue is here and how to 'fix it'.
    The exercise is to convert it with 4.6, like Aarron, and report ones finding​
    Insanity: doing the same thing over and over again and expecting different results . Albert Einstein. Your exercise fits that to a T.
    It must have something to do with converting so that the .DNG is compatible with Camera Raw 4.6 or earlier. I wonder what the difference could be, though?​
    The update to the DNG processing. There are 6 options and for legacy workflows, Adobe updates but continues to support the older processing versions.
    You don't have to upgrade Photoshop, you don't have to use LR, you just need either a newer version of the DNG converter or examine the settings and use the latest (7.1). Then .ARW's are a few meg's larger than the original.
     
  23. Eric, Aaron isn't talking about a 3 Mb jump in size, he's upset by "the file sizes jump from about 20mb (ARW) to anywhere from 45-85mb (DNG)". Your size increases are about what I'm getting and no big deal.​
    Exactly. So with a modern version of the converter, the size increase is only 4-5mb, tiny and well worth the benefits.
     
  24. EricM

    EricM Planet Eric

    Aaron, I'm not sure and it's the first I've heard. It happens with 2.4 as well but stops the huge conversion when using 5.4 and beyond. Too bad there isn't an Adobe expert around to help us out with it.
    Andrew, downloading his file and being the forth to use 7.1 and returning here to report you also have a slightly bigger dng just like Aaron, Will and myself have, has nothing to do with solving the mystery of getting a huge dng file from using 4.1 or 4.6. Perhaps re-reading Aaron's third post at 8:11pm and then replicate his procedure of using 4.1 and 4.6 and see what you find
    Because I'm using CS3, the last version of Camera Raw I can use is 4.6. When I got the huge .DNG files after conversion, I had DNG Converter set to be compatible with CR 4.1. I tried doing the conversion with compatibility set to 7.1. That produced an reasonable file size--just about the same size as the .ARW. But, of course, I can't open it because my version of Camera Raw is too old. So, I guess that's the problem. Anybody know why that would be the case?​
     
  25. Andrew, downloading his file and being the forth to use 7.1 and returning here to report you also have a slightly bigger dng just like Aaron, Will and myself have, has nothing to do with solving the mystery of getting a huge dng file from using 4.1 or 4.6.​
    7.1 is the process, 8.4 is the converter version I tested with and the results are clear. The two version value absolutely do not represent the same things! Further, as Will pointed out TWICE, the issue is not a few meg's increase it's a huge (2x) size increase and no matter how many times you run the same tests, you'll get the same results hence my request for the raw to test it without the insanity! With a newer version and process, the file is only about 4MB bigger. It's pointless to continue to do anything other than:
    1. Upgrade the DNG converter and settings as I've illustrated.
    2. Continue with the original workflow and end up with a bigger DNG.
    It was I who asked for a sample and it was I who showed that the issue is based on an older encoding and no matter how many times you run the same tests, you'll get the same answers which is insanity as defiend by Dr. E.
    What part of the OP's original comment isn't clear: when I convert the .ARW to .DNG the file sizes jump from about 20mb (ARW) to anywhere from 45-85mb (DNG).

    That's not a few meg's if you check your math! Using a modern version of the converter and processing, it becomes the case.
     
  26. EricM

    EricM Planet Eric

    The mental gymnastics are insane. No once cares if you're the fourth here to use 7.1 and come back with a slightly larger dng. Arron has gone to the trouble of uploading the file so how about just playing along nicely and using Camera Raw 4.6 like Aaron and I have, and see if you can also achieve a massive 77.3MB dng like we both have?
     
  27. No once cares if you're the fourth here to use 7.1 and come back with a slightly larger dng.​
    Speaking for everyone again.

    I came back with a slightly bigger and much bigger DNG depending on how I set the converter. So we've all varified it works as it does. Arron's trouble of uploading was well served and I'm glad I asked for it. If you or Arron is unhappy with how an older version of software operates you can upgrade as you yourself advised (and I agreed with) or stick to the same behavior, now knowing what an upgrade path could provide.
     
  28. I'm not sure what the point of beating dead pixels is, but I get a really big file with 4.6 and one around 20Mb bigger if I include the original RAW in the DNG. Adobe changed the compression algorithm, that happens.
    00ch9z-549631984.jpg
     
  29. EricM

    EricM Planet Eric

    I came back with a slightly bigger and much bigger DNG depending on how I set the converter.​
    Where? I only see one figure...
    Converted DSC02255.ARW with latest DNG converter. Was 21.4MBb, DNG is 25.2MB.​
     
  30. Where? I only see one figure...​
    Andrew Rodney [​IMG][​IMG], Jul 09, 2014; 09:50 a.m.

    IF one selects the oldest format (2.4 and later), the same NEF becomes 81MB on disk!
     
  31. EricM

    EricM Planet Eric

    a nef? what a waste of time
     
  32. a nef? what a waste of time​
    Pointless post...
    You're not following the dots again. NEF produces three sizes (if not more) of DNG depending on converter version and setting. You can run the tests over and over again if it makes you happy, the results are the same.
     
  33. I use a Sony nex-7. My .awr file size is about 20MB.
    Converted to .dng is about 90MB.
    I can't figure it out, either.
     
  34. I ran Aaron's test on DSC02255.ARW ARW - 21.4 Mb DNG - 81.1 MB
    I'll try upgrading DNG Converter, and try again.
     
  35. Tried Aaron's file DSC02255.ARW with DNG 7.1 (About DNG said 8.3)
    The DNG file was 25.5MB for all except 4.6 compatibility. which was 81.3 MB
    Alas!
     
  36. The DNG file was 25.5MB for all except 4.6 compatibility. which was 81.3 MB
    Alas!​
    Exactly, it's the DNG version built, not the converter per se that causes the size differences. So it's pointless to use version 4.6 instead of 7.1 with a modern version of a raw converter that understands DNG.
     
  37. I think I'll probably just give up on DM G conversion.
    Who the hell cares of my files don't last 50 years.
     
  38. EricM

    EricM Planet Eric

    That's a great thought, Lionel. It's time consuming, assures nothing in the archival scheme of things, and now it's not even saving of hard drive space.
     
  39. Since Adobe states that DNG is open software, (anyone can alter it), why hasn’t anyone fixed this problem?
    I believe Adobe is trying to crowbar us into buying the latest software, which means more unnecessary expense.
    I won't use DNG to store files, just convert my raw files (AWR) so I can open them in ACR, edit them and save as TIF.
    Then delete the DNG files and keep only the AWR files and the TIFs
     
  40. I believe Adobe is trying to crowbar us into buying the latest software, which means more unnecessary expense.​
    How so? You don't want to buy new software, convert to DNG. Use the proper version of the spec too. You don't want to use DNG, update your software. OR how about complaining to the company making the raw file different each time such new software is needed to read that raw?
    I won't use DNG to store files, just convert my raw files (AWR) so I can open them in ACR, edit them and save as TIF.​
    Fine. Your choice, DNG fixed the issue (temporarily).
     
  41. How so? Which version of DNG Converter?
    Thanks
     
  42. Which version of DNG Converter? Thanks​
    See above post(s). Try converting using the 7.1 process.
     
  43. Thanks,
    Yeah, I tried that.
     
  44. Yeah, I tried that.​
    On which raw? Aaron's file DSC02255.ARW or your Sony nex-7? On Aaron's file, using version 7.1, the resulting DNG was only 4-5MB larger than the original. Don't have yours so I have no idea what the differences might be.
     
  45. I used Aaron's file and the converted file for ACR 4.6 was 81.3 MB
    My file from the Nex-7, AWR, 20MB converted to 90MB DNG
    It's the ACR 4.6 that's the problem. I wish someone would fix the DNG software.
    (I know nothing about writing code.)
     
  46. I used Aaron's file and the converted file for ACR 4.6 was 81.3 MB My file from the Nex-7, AWR, 20MB converted to 90MB DNG It's the ACR 4.6 that's the problem. I wish someone would fix the DNG software. (I know nothing about writing code.)​
    Sorry, I'm confused. When I did the test using DNG converter 8.4 with DNG version 7.1, I didn't get an 81MB file, the DNG was about 4MB larger than the original. Are you saying you can't use version 8.4 for some reason? What needs to be fixed and in what version of the DNG converter?
     
  47. Thank you for your persistence!
    Got 'er done!
    Downloaded DNG Converter 8.3.
    Converted Nex-7 file 23.5 MB to DNG 24.1 MB
     
  48. Oooops!
    When I set DNG 8.3 to support ACR 4.6, the file size was 90+MB
    Oh, well!
     
  49. When I set DNG 8.3 to support ACR 4.6, the file size was 90+MB​
    I'm confused again. What's ACR 4.6 (which is really, really old) got to do with converting the raws to DNG using DNG converter 8.3? Are you saying ACR (which came with CS3) can't deal with a DNG using 7.1 spec so you moved back to an older DNG version?
     
  50. No. I used DNG Converter 8.3 as well as 4 other versions. Using any DNG Converter and making the conversions
    compatible with ACR 4.6, the file size goes from 23MB to 90MB. Go figger!
    I use CS3, and it works just fine. I won't upgrade to CC. I must keep my fixed costs, (overhead), down.
    (Retired professional photographer.) I'd like someone to fix that software, but I can live with it, knowing its weakness.
     
  51. I won't upgrade to CC. I must keep my fixed costs, (overhead), down. (Retired professional photographer.)​
    Might consider buying CS6 which is perpetual, moving up 3 versions, much improved ACR functionality.
    I'd like someone to fix that software, but I can live with it, knowing its weakness.​
    If fix you mean ACR 4.6, that isn't going to happen. With the version in CS6, it is 'fixed' in terms of your ability to convert to DNG without huge size increase, your camera might have native support, not sure. Or stick with CS3 and a big DNG. I don't see other alternatives if you want to stay within the Adobe ecosystem.
     

Share This Page