matt_t_butler Posted March 8, 2022 Share Posted March 8, 2022 It is 'good practice' not to completely fill up a computer hard drive to its maximum capacity. The suggested practice is to leave at least 10% to 20% of the capacity of the card empty of data so as not to affect the speed of data retrieval and usage. Is this the case for both HDDs and SSDs? Does the same apply to camera memory cards? Matt B Link to comment Share on other sites More sharing options...
steve_gallimore1 Posted March 8, 2022 Share Posted March 8, 2022 Does the same apply to camera memory cards? No, cameras write sequentially, from the beginning of the card to the end. The 10-20% rule is for computers with operating systems that need space for temporary files, defragmentation and such. (It's also, arguably, rubbish, and highly operating system dependent) 2 Link to comment Share on other sites More sharing options...
matt_t_butler Posted March 8, 2022 Author Share Posted March 8, 2022 [uSER=8475649]@steve_gallimore|1[/uSER] ... Thanks! Matt B Link to comment Share on other sites More sharing options...
Jochen_S Posted March 9, 2022 Share Posted March 9, 2022 Memory cards might generate an unwriteable last image or burst in a wrong moment, otherwise fill them up. (System)SSDs might benefit from a bit of vacant space, to distribute the read and write cycles evenly between their clusters? But I 'd fill an SSD that is only meant to sit around, storing images. 1 Link to comment Share on other sites More sharing options...
rodeo_joe1 Posted March 9, 2022 Share Posted March 9, 2022 Hmmm. Not so sure. I recently almost filled the removable 256gig micro-SD card in my phone (with about 5 gig to spare) and the read/write speeds became slower to the point of almost freezing the phone. There was still plenty of internal storage (>100GB) for the OS to work with. Transferring those surplus files to a backup HDD restored the read/write speed to full usability again. Not sure if the issue was fragmentation, with the card 'hunting' for spare storage space and file fragments, or what. But lesson learned; those bulky videos will get offloaded before the card gets near-full in future. 1 Link to comment Share on other sites More sharing options...
matt_t_butler Posted March 9, 2022 Author Share Posted March 9, 2022 I have had occasional issues when I saw the 'CARD FULL' message, where I could not immediately view picture preview files on the camera's LCD. Turning the camera OFF and re-inserting the card solved the problem. Thanks again for the answers. Matt B Link to comment Share on other sites More sharing options...
glen_h Posted March 17, 2022 Share Posted March 17, 2022 It is 'good practice' not to completely fill up a computer hard drive to its maximum capacity. The suggested practice is to leave at least 10% to 20% of the capacity of the card empty of data so as not to affect the speed of data retrieval and usage. Is this the case for both HDDs and SSDs? Does the same apply to camera memory cards? Unix, and unix-like systems, set the "full" value to 90% of capacity, except for root user. That mostly allows files to be in more contiguous disk blocks, especially as files are written and deleted, and new files written. And for magnetic disks, contiguous blocks are read faster. As well as I know, systems haven't changed the way they write file systems on SSD, though internally SSD usually have some system to distribute writes, such that the same blocks aren't written many times. (This is especially true for systems with a virtual memory swap file, which can be written over very often.) Since SSD don't have seek and latency delays, those reasons shouldn't apply. -- glen Link to comment Share on other sites More sharing options...
Ed_Ingold Posted March 28, 2022 Share Posted March 28, 2022 Data does not have to be stored in contiguous blocks, but are faster if they are. That's why it's important to reformat cards in the camera rather than deleting files, which causes fragmentation (a mix of filled and open blocks). The camera will store new files end to end on a reformatted disk or memory card, if possible, but break the file into smaller chunks if the disk is fragmented, and store the pieces where space is available. If the spaces aren't large enough, the disk may register insufficient space, even if overall it isn't completely filled. Link to comment Share on other sites More sharing options...
matt_t_butler Posted March 30, 2022 Author Share Posted March 30, 2022 "That's why it's important to reformat cards in the camera rather than deleting files, which causes fragmentation (a mix of filled and open blocks)." Does this also apply to deleting single unwanted photos in camera folders using the camera's own ERASE or DELETE function? Or rather deleting some files when an external card reader is connected to a computer before reinserting the same card back into the camera? Matt B Link to comment Share on other sites More sharing options...
glen_h Posted March 31, 2022 Share Posted March 31, 2022 Fragmentation is important for moving head (magnetic) disk drives, as it requires more seeks to read all of a file. Flash memory is a little complicated, but seek time is one that it doesn't have. The main complication that flash has, is a limited number of write cycles. Mostly that isn't a problem, except for files that get written a very large number of times. For most systems, that is only virtual memory swap files. More writes increases the probability of a bad block, where the data stored isn't what it is supposed to be. Systems have a way to spread writes around, to reduce problems. It seems that the number is about 10,000 for some older ones, up to 35,000 to 100,000 or even 1,000,000 for newer ones. One other effect of fragmenting is that it makes recovery of data accidentally erased, or otherwise needing to be recovered, harder. -- glen Link to comment Share on other sites More sharing options...
Tony Parsons Posted April 4, 2022 Share Posted April 4, 2022 Must admit that my standard practice is never to completely fill an SD card, just in case. With a 64 GB card, that usually means between 1600 and 1800 RAW images before it is retired and replaced. I NEVER re-use SD cards - they are so cheap I prefer to use a new one each time, and it also means that I have the original images (as well as a working copy and backups) - with 35mm film negatives they were always kept and filed, after all. Link to comment Share on other sites More sharing options...
q.g._de_bakker Posted April 4, 2022 Share Posted April 4, 2022 Must admit that my standard practice is never to completely fill an SD card, just in case. With a 64 GB card, that usually means between 1600 and 1800 RAW images before it is retired and replaced. I NEVER re-use SD cards - they are so cheap I prefer to use a new one each time, and it also means that I have the original images (as well as a working copy and backups) - with 35mm film negatives they were always kept and filed, after all. Is there a good reason to keep your files on the memory card you first recorded them on? I don't know of any, really. There is no analogy with negatives, since you can't have exact copies, let alone multiple exact copies. And you can't transfer a negative from the film it is on to another medium either. With digital files, you can. On a much cheaper medium too. Multiple exact copies. Negatives must be processed and stored well, and will then keep for a very long time. Memory cardshave a limited life span, and you can't tell by looking at them that they are no longer what they are supposed to be. So why would you keep them on expensive memory card, and never reuse those cards? Link to comment Share on other sites More sharing options...
glen_h Posted April 4, 2022 Share Posted April 4, 2022 (snip) So why would you keep them on expensive memory card, and never reuse those cards? A 64GB SDXC card costs less than one roll of film, not including processing, and as noted holds over 1600 raw images. I mostly shoot JPEG, though at the highest quality mode for the camera, so maybe 6MB each. So maybe 10,000 picture for a 64GB card. (Though I more often use smaller cards.) You can get 1TB SDXC for about $26, maybe a roll of Ektachrome + processing. Now, if it is some brand new card type, not readily available yet, then they might be more expensive. But the quote mentions SD. (Likely SDXC.) When I was young, the best data storage was 40 MB on a 12 inch reel of magnetic tape. I had a few reels for some of my files. -- glen Link to comment Share on other sites More sharing options...
glen_h Posted April 5, 2022 Share Posted April 5, 2022 (snip) There is no analogy with negatives, since you can't have exact copies, let alone multiple exact copies. And you can't transfer a negative from the film it is on to another medium either. (snip) This reminds me that in the early days of photography, with glass plate negatives, it was usual to reuse plates. Many negatives that probably should have been kept, were instead reused. In the case of digital data, there is a (hopefully small) chance of errors in any copy operations. More likely it is a defect in one card or the other. As well as I know, though, a bigger problem is losing cards. Having a good file method for storing them probably helps. Not requiring time to copy, and for verifying that the right things were copied, saves time. -- glen Link to comment Share on other sites More sharing options...
q.g._de_bakker Posted April 5, 2022 Share Posted April 5, 2022 The price per MB of memory cards is high. Ye olde spinning discs are much cheaper (allows for redundant storage) and hold much more. The time it takes to copy from card to other medium is short, and not time you need to spend watching and waiting. The chance of errors during copying is real, small and taken care of by just about all software you can use to copy files. And unless you accept the bigger risk of corrupt or lost memory cards, you will have to copy anyway. Link to comment Share on other sites More sharing options...
Tony Parsons Posted April 5, 2022 Share Posted April 5, 2022 I have never stated that I do not copy the cards. As I stated, I keep the cards, creating one working copy of each day's shooting on my data drive as soon as I arrive home, and have two further drives, to which I backup weekly the original images and those that have been processed. This is the way which suits me - others may wish to do differently, up to them. I can recall friends when having colour negative film processed, and being supplied with en-prints, or 6x4 enlargements, throwing away the negs - then wanting further copies, or having the original prints damaged, lost or destroyed, and wishing they had kept the negatives. Furthermore, although I have seen references to 'Memory cards have a limited life span', I cannot recall this ever being backed up with data showing the projected life span of an SD or SDXC card, or even anecdotal evidence of this phenomenon occurring. Perhaps someone can point me in the direction of any such published information, please ? And yes, my completed SD cards are kept filed in order, along with DVD copies of them. Current price of 64GB SDXC card on Ebay - £7.99. Hardly likely to break the bank, even using three per year. And, should it so happen that a memory card becomes corrupted, software is available that will recover data. I do not have any evidence to back up my views, but I feel it is more likely that a card will become corrupted through frequent formatting and re-use than in my own system of 'Write once. read once'. Finally, may I enquire why it is so important to you that I relinquish a system that has worked well for me since the days of my Pentax K10D (whose cards are still able to be read, by the way) to adopt a system which to me is less secure, and will save me less than the price of a pizza per year ? Thank you for your suggestions - they have been securely filed. Link to comment Share on other sites More sharing options...
Tony Parsons Posted April 5, 2022 Share Posted April 5, 2022 I have stated my point - my contribution to this thread is now ended. Link to comment Share on other sites More sharing options...
q.g._de_bakker Posted April 5, 2022 Share Posted April 5, 2022 [...] Finally, may I enquire why it is so important to you that I relinquish a system that has worked well for me since the days of my Pentax K10D (whose cards are still able to be read, by the way) to adopt a system which to me is less secure, and will save me less than the price of a pizza per year ? Thank you for your suggestions - they have been securely filed. The question i asked was what reason there would be (besides personal preference, sure) to keep files on memory cards. The emphasis you put on NEVER certainly suggests that there is some urgency to at least not reusing them. And yes, increasing the number of write cycles will not help keep a card good. But is it that problem that BIG that we should take note of your NEVER? You raised the question. Why is it so important to you that we take note of your habits? And i'll contest again that keeping them on cards is more secure (re your "less secure"). It certainly is not, so no good reason. Link to comment Share on other sites More sharing options...
glen_h Posted April 5, 2022 Share Posted April 5, 2022 There is much written on the number of write cycles for flash memory, with numbers around 10,000, and more recent ones maybe 100,000. That is mostly a problem when they are used for virtual memory swap space. The MacBook Air that I write this on has flash for its internal storage, but I mostly keep data on an NFS mounted network disk drive (at home). There is less written about data retention time when not being accessed. Storage is based on charge on a SiO2 capacitor, which does stay for an amazingly long time. Write and erase cycles damage the SiO2, which is the reason it fails after some number of write cycles. There are, then, physics reasons that repeated write cycles reduce the data retention time for data. For many forms of data storage media, the data lasts longer than the read/write devices. I have film negatives of mine from 55 years ago, and others much older than that. Much computer data storage 55 years ago was on 7 track magnetic tape, for which it is very difficult to find drives to read. Not so hard for 9 track tape, but not easy, either. I suspect CD-R and DVD-R will be not so easy to read in 55 years, even if the data is still just fine. -- glen Link to comment Share on other sites More sharing options...
glen_h Posted April 5, 2022 Share Posted April 5, 2022 More than you wanted to know: https://www.ni.com/en-us/support/documentation/supplemental/12/understanding-life-expectancy-of-flash-storage.html -- glen Link to comment Share on other sites More sharing options...
matt_t_butler Posted April 6, 2022 Author Share Posted April 6, 2022 After reading all about spinning disks and write cycles my head is spinning .... ;) I have CF camera cards from the last century - circa the late nineties - that are still readable. Do any forum readers have any anecdotes about CF card failure when used for long term storage? Matt B Link to comment Share on other sites More sharing options...
glen_h Posted April 6, 2022 Share Posted April 6, 2022 After reading all about spinning disks and write cycles my head is spinning .... ;) I have CF camera cards from the last century - circa the late nineties - that are still readable. Do any forum readers have any anecdotes about CF card failure when used for long term storage? We are used to keeping film cold, but note that keeping flash cold, also helps. The graphs I saw seem to indicate that the mentioned 10 years is at 40C. The longer times they state at 25C or so. Note that the chips warm up in use, but for long term storage, that isn't so important. But also they have changed a lot over the years, so older ones might not be representative of ones today. The early ones store one bit per cell, that is per transistor. They store two different charge levels. Newer ones store two or three bits, with four or eight charge levels. That makes them more sensitive to changes in stored charge. In any case, they mostly follow Arrhenius, so storage at higher temperature allows for speeding up testing. In any case, the oldest I have is 2006, and I haven't actually tried reading them recently to test. -- glen Link to comment Share on other sites More sharing options...
AlanKlein Posted December 12, 2022 Share Posted December 12, 2022 11 hours ago, EllinorWilliam said: Once your disk(s) are 80% full, you should consider them full, and you should immediately be either deleting things or upgrading. SSDs and SMR disks without enough free space will show decreased throughput and - even worse - increased latency (sometimes by many orders of magnitude). SSDs with a Sandforce controller can fall back in free-block erasure on high-write workloads, leading to write latencies going up by thousands of percent. We usually leave 25% free on the partition level, then observe a fill rate of below 80%. What does that mean? Flickr gallery: https://www.flickr.com/photos/alanklein2000/albums Link to comment Share on other sites More sharing options...
rodeo_joe1 Posted December 13, 2022 Share Posted December 13, 2022 On 12/12/2022 at 5:54 AM, EllinorWilliam said: SSDs with a Sandforce controller can fall back in free-block erasure on high-write workloads, leading to write latencies going up by thousands of percent. Any way of finding out what type of controller is used in a particular SSD? I tend to only use SSD/NVME for storing an OS and programs. So far I haven't let any 'C' drive get much more than 2/3rds full, so haven't experienced noticeable slowing. But overfilling is definitely something I'll keep an eye on. If it's a known issue, then why isn't a fast volatile RAM cache incorporated in the design? Because I see no point in supplying a 1TB drive that has 200GB of effectively unusable space. Link to comment Share on other sites More sharing options...
glen_h Posted January 24, 2023 Share Posted January 24, 2023 Unix tradition is to fill drives only 90% full. Unless you are root, it won't let you write more. In the case of rotating drives, you want to keep files on consecutive blocks, for faster access. As disks get fragmented, that is harder. But SSD don't have the rotation latency that magnetic drives have. -- glen Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now