Please advise - Corrupted image files (RAW CR2) upon transfer from the SD Card to the HDD

<p>Hi,<br /><br />For quite some time by now, I've been experiencing a very weird and annoying problem, which I hope someone at this wonderful community will be able to help me resolve:<br /><br />I have a Canon 60D with a few Kingston SD Cards.<br />I use a Sandisk imagemate SDDR-199 in order to transfer the images to the HDD.<br />Now, on about ~1% of the image files on any bulk transfer, the below happens:<br /><a href="https://goo.gl/photos/WGtcT6hUm2boDGDp9" rel="nofollow" target="_blank">https://goo.gl/photos/WGtcT6hUm2boDGDp9</a><br />[exported for web]<br /><br />It happens on the local CR2 copy only (on the HDD; not on the SD, which is inserted in the SDDR-199 interface).<br>

Below is a summary of a complete test exemplification scenario:<br />a. An original corrupted CR2 image file on the HDD: <a href="https://drive.google.com/file/d/0B3KKDxpU3w7reTdJZWhyeWJRbmc/view?usp=sharing" rel="nofollow" target="_blank">IMG_4881.CR2</a><br />b. 7-zip (latest build - 15.14) checksums results:<br />SD CRC32: size 21957648 bytes CRC32: 6950EDEF<br /><a href="https://goo.gl/photos/91ihTiPedp3HjgZQ8" rel="nofollow" target="_blank">7-zip PrtSc</a><br />HDD CRC32: size 21957648 bytes CRC32: 2786C016<br /><a href="https://goo.gl/photos/91ihTiPedp3HjgZQ8" rel="nofollow" target="_blank">7-zip PrtSc</a><br /><br />As you may see - the checksums differ - which leads me to believe that somehow some errors during the transfer occur.<br />Important information:<br />1. If I copy-paste the single image file again, it's copied from the SD card in order, without any corruptions. <br />2. This seems to happen only when I transfer a bulk-load of images, and only on a minute number of files from that stash (though statistically this has not been proven, since I did not copy-paste a single image file numerous times, in order to account as a verifiable statistical measurement)<br>

<br />For your convenience - summary of interfaces used:<br />A. Kingston XC 120 GB<br />B. Sandisk imagemate SDDR-199<br />C. Seagate FreeAgent GO 500 GB (in good health) / HITACHI 150 GB HTS722016K9SA00 (in good health, seldom used in bulk transfers due to available space limitation).<br /><br />Honestly, I'm quite bewildered by this quirk <img title=":??:" src="http://m.bestofmedia.com/sfp/images/design/usr/smilies/confused.gif" alt=":??:" data-src="http://m.bestofmedia.com/sfp/images/design/usr/smilies/confused.gif" /> <br />Any ideas / advice / suggestion etc. will be of great service <img title=":D" src="http://m.bestofmedia.com/sfp/images/design/usr/smilies/biggrin.gif" alt=":D" data-src="http://m.bestofmedia.com/sfp/images/design/usr/smilies/biggrin.gif" /> <br>

Thank you!<br /><br />Assaf</p>

I have seen this happen with some 5DS .CR2 files from a Lexar CompactFlash and another brand of reader. My solution

was to eject the CF and then try downloading again, which solved the problem .

<p>Hi Ellis,<br>

Thank you for your suggestion - thing is I only know what are the corrupted photos after finishing the entire transfer,<br>

so ejecting and inserting again doesn't really do the trick.<br>

Basically, I just can't understand the root cause of the problem - especially considering that the used interfaces are all genuine top quality.</p>

<p>Clues? anyone?</p>

<p>Perform an exhaustive memory test on the computer in use with a piece of diagnostic software - recent versions of Windows and MacOS have them built-in.<br /><br />The OS allocates RAM to use as a file system buffer, and during large transfers you could be hitting a defective area of memory. Depending on where it is, and what kind of problem, it may not be the sort of thing that would crash a computer regularly, but it could be enough to flip a few bits in a file during a copy.<br>

<br />I was seeing something similar a couple of years back, and when I tested it turned out that my second (the higher address) memory module had developed a fault. I replaced the RAM and it fixed it.<br /><br />As a side note, this is why business-class servers and workstation-class microcomputers usually require more expensive ECC RAM. It allows the computer to detect and report - and if the problem isn't too severe correct on the fly - certain memory areas. </p>

i import directly into Lightroom CC2015 so i saw immediately which files were not imported. After the import process was

finished and the previews built, I backed up the catalog,quit Lightroom, ejected the CF media, and restarted the computer,

and then Launched Lightroom again - keeping all other programs off - and was able to succesly finish the download/impirt

process.

This was likely overkill but what I have done since then is turn off all other programs prior to starting the download/import

process. I suspect the problem occured because during the first attempt I had the computer doing other work as well.

My raw files are stored on an drive array (a Drobo 5D populated with 3 and 4TB WD red drives).

<p>Ellis, Paul - Thank you for your kind informative responses.<br>

With the above in mind, a couple of questions/assertions if you may:<br>

a. Paul - I have recently executed Prime95 with various benchmarks: No - errors / crashes / blue-screen<br>

May you recommend a suitable memory benchmark test s/w ?<br />(freeware is preferable of course) <br>

b. Ellis - Your workflow and backup is super-rigid, I do not get to that levels.<br /> After considering your advice, I have decided that I probably should also perform bulk transfers on a resource-free Windows session. Nonetheless, I think it's more time efficient for me to first perform the bulk transfer as is (taking some corruptions into account), and straight away let Lightroom build the smart previews (as this is the most time consuming task in my system). Afterwards I shall look for the corrupted images and replace them manually.<br>

BTW - I think it's worth mentioning that I (still) use my beloved Lenovo T500 with 4 GB of RAM (which might account for Paul's plausible theory for the source of the problem).<br>

Assaf</p>

<p>Assuming Windows 7 or later, use the Windows memory diagnostic tool - just click Start and type "memory" - it should come up.<br>

It'll restart your Thinkpad with a diag boot of windows, start the memory testing tool, and start systematically going through the RAM with different operations and patterns. If there's an issue, it should trip over it.<br>

http://windows.microsoft.com/en-us/windows7/diagnosing-memory-problems-on-your-computer</p>

<p>Hi<br>

From what you say, I suggest trying a different card reader. You may also try to connect the camera directly to the computer with a suitable cable if available.</p>

<p>Cheers,</p>

<p>Did you determine the differences bvetween the original file and the corrupted copy ? I.e, is ir random bits; single blocks (or collection of blocks); specific bits (high/low); ....<br>

-<br>

The issue could be the card reader; the card; the hard disk; memory; .... <br>

-<br>

Enough data is not provided in this thread to determine the source of the error. While you cannot always determine the source; there are certains that can be done to narrow in on the cause.</p>

<p>Until you've sorted out the hardware issue, it might be worth using something like Teracopy that does CRC checks on the fly:<br>

http://www.codesector.com/teracopy</p>

<p>Alan / Frode - thank you for your advice.<br>

Alan - to be as specific as possible:<br>

a. I can tell with a high degree of certainty that the errors are random - this is by examining different corrupted images (some are almost completely unintelligible, others are only stroked on certain parts (you may see the 2 image links supplied at the top of the thread for comparison, Though I do agree that for absolute verification an actual bit-2-bit comparison is needed on several different corruption occasion).<br>

b. I use CrystalDiskInfo to attest to the health of my drive - it indicates it's healthy, so I tend to rule this option out.<br>

c. I have multiple SD cards - all Kingston / Sandisk, and all present this problem upon bulk transfer - so I also tend to rule this option out.<br>

d. I'm not sure about the card reader itself - I should try to perform back-2-back tests (i.e - different reader on my system and different system with this reader).<br>

e. I have yet to fully perform memory benchmark tests (so far I've only run various Prime95 benchmarks successfully).<br>

All in all, I'm somewhat limited in the means available at my disposal (what can I do - I'm not an actual lab)<br />So I do understand that it might not be possible to pinpoint the problem with absolute certainty, considering the current circumstances. Though by posting here I did hope that some solution which is beyond my reasoning is available.</p>

<p>As I understand the problem occurs across different cards, but you are able to restart the process and recover the picture. Is that basically correct?<br>

If yes, this suggests that it is something downstream from the cards. <br>

You can take the card reader and cable out of the equation by testing transfer directly from your camera. For that all you need is a charged battery and the appropriate cable which should have come with the camera. And do NOT use the same cable as used with the card reader.<br>

If you have problems with direct transfer from the camera, that would suggest something other than the card reader and cable in other words something on the computer. <br>

My bias is the most likely problem is the card reader/cable, but it could also be corrupt RAM or HD.<br>

Report back when you figure it out....)) </p>

<p>When I read your post, my first thought was the same as Paul Coen's, because I've had this problem. Very few computers these days use memory with error detection, which is a real shame. I would also replace the card reader and cable, as others suggested. And, do the copying with a utility that does error checking, not with Windows Explorer. You might try FastCopy:</p>

<p>http://ipmsg.org/tools/fastcopy.html.en</p>

<p>You don't want to detect corruption by looking at the images. You want the copy utility to just tell you if there's a problem.</p>

<p>Hi,<br>

Steven / Marc - thank you for your advice.<br>

I think I made some progress - I've tried performing a bulk copy (~200 files) by directly connecting my 60D to the laptop (different cable) - 0% errors.<br>

With this in mind, may I declare this to be a defected reader?<br />Is there a way to determine this assumption to be the exact cause of this, with absolute certainty (assuming the cable is insignificant in this equation, in a sense that it's most likely valid)?<br /><br />Thanks again!<br>

BTW - in any case I will make sure to use FastCopy for these kind of operations in future attempts.</p>