Mass storage

[paul_ingram]

<p>I have a decent Dell desktop for my architect's practice and event photo hobby. I have added three external hard drives between 500G and 1T for thousands of photo files, organized with Lightroom. I think I need single massive huge storage because my PC is slowing down with all these external drives plugged in. Where do I start looking for big ole file storage options that will last me a while. It is a requirement that I access my photo archives often. Or should I try my first Apple for the photo side of my activities and in the process get a ton of storage? - Old and confused Paul</p>

[tony_davis6]

<p>One item that may help you out, which took a load off my system's need to track everything was to use an older computer and setup a basic file server. There are many ways to do from installing expensive software packages to handle it, or open source solutions, or just using share files on the system to connect to.</p>

<p>Setting a second/extra computer for file share means that the inherent networking can handle the file handling, and you have a fast connection between the computer. Unless you are gigabit it may be a little slower in transfers than from the drive you have, but at the best it will reduce the system load of devices.</p>

[mendel_leisk]

<p>Internal drives are a lot faster/cheaper. There are disadvantages: less flexible, not portable, but something to consider. Your desktop is likely pre-wired with available bays for 2~4 additional internal drives.</p>

[john_deerfield]

<p>I am not sure about setting up a file server as reducing overhead. And I am not sure Lightroom will work off a server? I know Aperture won't. I would also want to nail down the exact cause of the slow down. I don't know why multiple drives in and of itself would cause a slow down. But then again, I am on a Mac and I much prefer Macs approach to mounting different drives/storage devices however I see fit as opposed to some random letter assignment. In terms of actual connections I would prefer eSata first, followed by FW800, FW400, finally last and least preferred: USB2. In all cases, I would prefer that my drives have their own bus which simply means a dedicated PCI card with the appropriate port. Failing that, if I had let's say a FW800 or 400 drive(s) and would try to make sure I didn't have any other devices using the 800/400 bus. IE: everything else using the USB ports. To manage a growing database of images, nothing beats removable drive bays. If I had to do it all over again, I would get something like this:</p>

<p>http://www.macgurus.com/productpages/sata/BurlyPortMultiEncl.php</p>

<p>The trayless version (I have the units with trays now). This just treats a hard drive like a giant floppy. Fill one up, put in another. Put in one to act as a back up. Put in one for "other" files and so on. With a case like that, all you need is the bare bones hard drive and you are good. No need to constantly purchase external hard drive enclosures every-time you need more storage.</p>

[JAPster]

<p>There are a couple of things you can do before investing in another external HD.</p>

<p>First, I would recemmend "maxxing out" the internal RAM on the desktop. What kind of OS are you running? If you are running Windows XP Pro, I would suggest you have a minimum of 3GB RAM. You could add more, like say 4 GB Ram, but some apps under Windows XP Pro will only access 3GB no matter if you physically have more. Adding more RAM may help both your architect design apps run faster as well as photo mgt and editing apps.</p>

<p>Next, I would suggest you perform some repeated DEFRAG runs on both your primary hard drive, any 'secondary' internal drives, and lastly, those large external drives. By running about 10 passes of DEFRAG on each hard drive, you consolidate all the scattered file fragments and rewrite them to new disk spots. Then when the OS needs to read them it only has to go to that one spot instead of 6 or 8 spots. This one maintenance item alone can make your file system response time noticeably quicker, and that improvement makes any functions and apps that access those file run that much faster. I would suggest you do this once or twice a month to keep all files as packed as possible.</p>

<p>Another performance improvement suggestion is to resize the Windows Page File and make it about 2.5X the amt of RAM you have. The Windows Page File is a dedicated file on the boot drive that serves as a "cache" for data that is removed from RAM by the OS in order to make room for apps you are running. If you have 1GB of RAM, try setting the Windows Page File to 2.5GB (2.5 X 1024MB). If you have 2GB RAM, set the page file to 5GB (5 X 1024MB). This will give the OS plenty of "page file space" to use as it manages the RAM memory space. This can help the system be more reliable as you work with complex graphics file programs like Lightroom and load up large hi-res picture files.</p>

<p>To reset the Page File Size, right-click on My Computer, and select Properties. Click the Advanced Tab, then the Settings button under Performance. Under Performance Options, click the Advanced Tab. At the bottom, under Virtual Memory, click the Change button. This brings up the virtual memory GUI. At the top, select the C drive, in the middle select Custom Size, and in the Initial Size and Maximum Size fields, plug in the size of the new page file that you want to activate. Again, I suggest you use a size (in Megabytes) that is 2.5X the amount of RAM currently in your system. GB Ram X 2.5 X 1,024 = Pg File Sz in MB<br>

After plugging in the #, click SET, then click OK, then click OK in the remaining windows to exit out. You will need to logout, shutdown, and restart the system to make the new page file size take effect.</p>

<p>Making these basic changes to your system might improve the performance enough so that you no longer need another large external hard drive.</p>

<p>Western Digital makes some nice external "Network File Share" hard drives. These come in 1TB sizes and higher. They have a builtin Ethernet port that allows them to be connected to your local router, switch, or hub. You can even connect them directly to your PC's Ethernet port which makes them act like a connected external hard drive. You can use the built-in software to make accounts to access the network file share. Or you can just map the network file share drive to an unused drive letter on the PC and that makes it a Public access share drive. You can then transfer old files off your connected external hard drives onto folders on the network share drive. If you have other computers in the house on the same LAN, they too can map to the network file share hard drive. The only draw back is that the Ethernet port I/O is a bit slower than a USB 2 port. So it will take a bit longer to transfer files to/from this type drive than to/from a connected USB 2 type drive. You could even use this type drive to make extra copies of project files, as online backup file sets. They sell these type drives at places like Best Buy, back in the Hard Drive section.</p>

<p>Finally, I would suggest visting the vendor website that makes LightRoom, perusing their Support page, Q & A's, etc, and seeing if you can find suggestions that would improve the performance of Lightroom. I'm not familiar with that app, but that's where I would start.</p>

<p>I hope you find these suggestions helpful...</p>

 

[eric friedemann]

<p>Interesting LL article on topic:</p>

<p><a href="http://www.luminous-landscape.com/tutorials/backup.shtml">http://www.luminous-landscape.com/tutorials/backup.shtml</a></p>

[garrison_k.]

<p>Paul, can you open the Dell and install another hard drive? Like a 2TB Western Digital? Then take the contents of your externals and copy over to your internal.</p>

<p><em>Or should I try my first Apple for the photo side of my activities and in the process get a ton of storage?</em></p>

<p>Platform has nothing to do with the task at hand. But it's even harder and more expensive trying to get rid of externals with Mac's.</p>

[garrison_k.]

<p>Interesting article, Eric. I whole heartedly agree with the conclusion: put your data in the clouds as a backup.</p>

<p>It's only a matter of time before we rent applications like Photoshop. We'll open our account, point PS at our cloud storage, and work away faster than we ever have. There will be no need for desktop computers and us always racing for faster cpu's/ram, and worrying about our hard drive content.</p>

[paul_ingram]

<p>I may have overstated the slow-down issue. I have a lot of RAM and the PC works very well. The only thing that's a bit slow is the communication among external drives via all the USB connections. I added a Seagate "Free Agent" external drive and dragged a lot of photo files into it (via Lightroom) and within a few weeks it started making a grinding sound and is now very slow and unreliable. I just want to clean up the clutter and speed things up. I may look into a second internal HD. You guys are great.</p>

[szrimaging]

<p>Paul, how old is that free agent? That sound less like speed issues and more like a dying drive. That said, the more files you have on a given drive, the longer your seek times are.</p>

[john_deerfield]

<p>And USB is the slowest drive configuration, especially with multiple I/O requests on the USB bus.</p>

<p>And how is it harder and more expensive trying to get rid of externals on a Mac???</p>

[wpahnelas]

<p>if you are running external drives w/USB interfaces, that's your main problem. it'll sound like a pain, but put a firewire card in the PC, and replace your external drives with models that support firewire. this is how i manage, anyway, and data transfer is much faster.<br>

i understand that eSATA is available and is even faster. that's just as viable as firewire, if you can find the gear.<br>

finally, if your case can accomodate additional HDDs, that's another option, but one which i find somewhat less desirable. if i max out my present external drives, i can archive the entire device and swap in another in a couple of minutes. that's not practical with internal drives, even though they might give faster performance.</p>

[paul_ingram]

<p>>Update< I called Dell and thier man in Calcutta told me the only HD they could add to my 4-year-old desktop was a rebuilt 400Gb. Nuts.</p>

[tomwatt]

<p>Paul, unfortunately the individual in Calcutta was probably looking at what was available to him locally.<br>

:)<br>

Pop your case open and check your cable/belt that attaches the hard drive to the motherboard. There should be another slot on that ribbon cable for a hard drive. Also check your motherboard to see if it accepts a second ribbon cable... you could potentially put 3 more hard drives in there... if nothing is available for whatever reason, consider getting one of the newer drive controller cards and adding that, plus cables plus drives to the hilt. Then ditch your externals except for backups.<br>

I loaded my internals up and it helped a lot. Neither USB nor Firewire were fast enough to keep from chugging my system, which is also pretty old... my last-ditch holding-off-disaster will be to drop in a fast SCSI drive for temp/swap files and let this system coast its way into oblivion over the next couple of years.</p>

[mendel_leisk]

<p>If the bays are available, likely the cables and power plugs are as well, at least in my experience. Also, the newer SATA and SATAII are not ribbon cables, they're red, sort-of flattened pencil thickness. There are two varieties of power plugs as well. If your system has the older version there are adapters readily available.</p>

[paul_ingram]

<p >I have had an amazing day exchanging emails with Brian Adams of MacGurus & Burly Storage (from John Deerfield’s post above). VERY helpful with their devices, his analysis of my system, his experience in massive photo storage, etc. Many options. Makes me feel I can do this! </p>