Photoshop and photos on external hard drive

[mo_da1]

<p>My internal hard drive is about 300 GB, with only about 100 GB free.<br />My photos are starting to grow fast in size and quantity due to<br />purchase a 15 megapixel camera, so this drive is going to run out of<br />room fast.</p>

<p>I'm thinking about starting to store all of my photos on an external<br />drive. I'm considering some 2 TB Western Digital drives that provide<br />RAID 1 drives for data mirroring. This will provide some data security<br />and peace of mind should one of the two external drives fail.</p>

<p>Are there good reasons for not storing all my stuff on an external<br />drive. Is this something I will regret? Will Photoshop slow down when<br />working with these files (or just when opening and saving them)?</p>

<p>Thanks in advance.</p>

[William Kahn]

<p>An external hard drive is an option, but I think external HDs serve better as backup drives rather than primary file storage. You might consider adding a second internal drive and dedicate it to photo file storage, then use an EHD to back up those files. One advantage is that access to an internal drive is faster than a USB drive. Also, you can leave the EHD powered off except when backing up files, which save wear and tear on it.</p>

<p>And, I don't think you can run a program file like PS from an external drive since it's not the primary drive for your system.</p>

[michaelbrochstein]

<p>Assuming you have a relatively modern PC then the internal hard drives probably use a SATA bus for communucations. This SATA bus is much faster than USB 2.0. Given the relatively low cost these days for bare internal hard drives, I would echo what William Kahn wrote and suggest that you install a large SATA drive as your second internal hard drive and store your images there. Use the external USB hard drive for backups.</p>

[ellis_vener_photography]

<blockquote>

<p><em>An external hard drive is an option, but I think external HDs serve better as backup drives rather than primary file storage.</em></p>

</blockquote>

<p>I disagree. I keep all of my photos ( a few terabytes worth) on external disk drives. By doing this I do not have to access the main disk for the Operating System, Photoshop, and the photos. This means I do not inflict a performance hit on my system. I also have 20GB partions set up on external drives and set them as the primary scratch disks in Photoshop's preferences window., as this also improves performance of the system.</p>

[steven_clark]

<p>I've found that fragmentation is pretty much unavoidable on any drive that you run a browser on, or otherwise shuffle around lots of tiny files. This effect is such that I've seen files load and save faster over a 100Mbit connection to a linux box than to local ATA disks. There could easily be more than enough free space fragmentation, even after defragmenting, to make a USB disk faster than the internal drive the OS runs on. That's why I have a seperate partition for files I won't be frequently deleting, speed.</p>

[mike_bisom]

<p>In general, an internal drive will be faster than an external drive. In an external drive, eSata would be the fastest but would require an eSata port (more than likely you would need to add a PCI eSata card). Firewire 800 would be the next fastest, followed by FW400 and finally USB- which would be the slowest and least desirable. Keeping all of your media on a 2nd drive dedicated to storage is generally much faster than keeping everything on the same drive. With two drives, one is free to operate the OS and programs while the other read/writes you media files. Finally, a RAID 1 is typically a waste of money. The ONLY thing a Raid 1 protects you from is ONE hard drive failure. Any good disaster recovery plan means you will have a back disc of your files. Since you already have a back up of your files (correct?), a RAID 1 does nothing except cost you more $$$ for the 2nd drive.</p>

[raymond_ocampo]

<p>There is nothing to regret with storing your data - photo, video, documents - on an external drive or any other drive that is separate from your operating system and software applications hard drive. Also I have to commend on considering mirroring a drive in order to further protect your files should one drive fail (not both at the same time).</p>

<p>Although hard drives via USB connection is obviously not faster than internal drives, and while RAID 1 does not increase performance write/read speed (as long as you get a hardware-based RAID), I would still choose mirroring over striping. Better if you can do both.</p>

<p>If you're concerned with PS slowing down when reading files from an external drive, you may notice it a little bit or not depending on how large a file you are working with:</p>

<p>USB 2.0 has a theoretical maximum transfer rate of 480 Megabits/second (also 60 Megabytes/s or rather 60 MB/s). So you can compute your file from there. If you have a 60 MB file, then it will give or take a second to completely read that file from your external hard drive to your computer and then a few split seconds to have it processed and interpretted by PS/CPU and eventually displayed onto your monitor for viewing. Anytime there are changes or file save, then that's how long it takes to write that same file back to your hard drive. If you're working with a smaller file, then it won't matter. If you decide to work on larger files then it will affect you more.</p>

<p>The same speed can be expected with a FireWire 400 (400 Megabits/second) connection. While a FireWire 800 (800 Megabits/second) connection will practically half the read/write speed. Finally an SATA/eSATA drive will supposedly perform at 3,000 Megabits/second.</p>

<p>BUT all of this won't matter because they are theoretical and based on buffer to host connection which means actual transfer speed will be determined by your hard drive's access speed to spin fast enough and eventually go thru all the platters. That's why those enterprise drives are also so expensive.</p>

<p>But no matter what drive you choose, you still have to back-up your RAW data to yet another medium like a CD/DVD (or off-site/on-line) for those just in case scenarios.</p>

[William Kahn]

<p>Okay, as long as Raymond opened <em>that</em> can of worms, maybe you should take a look at this thread on RAID backup storage:</p>

<p><a href="http://www.photo.net/digital-darkroom-forum/00SfiF">http://www.photo.net/digital-darkroom-forum/00SfiF</a></p>

<p>I do think Raymond is right: RAID 1 (mirroring) is better than RAID 0 (striping) for data security. The downside is that data transfer rate is slower with RAID 1. Or, as Robert Heinlein put it, TANSTAAFL (there ain't no such thing as a free lunch).</p>

[garydem]

<p >i have 4 ext usb hdr drvs. 2 of which i use for photography only. 1 is the primary and 1 is the backup. my current working pics are on the pc's c drv, once i am done with them i move them to the K drv(primary photo).</p>

<p > </p>

<p >when i backup the primary, every 2 weeks, i simply delete the backup(drive L) and right click and drag the 1 and only folder in the primary drv to the backup and select copy here. for my 120gb of pics on primary drv, the copy takes about 2 1/2 hrs. i do that when i am doing something else. when the backup is being made i have no backup that is why i have 3rd copy of the photo drv on a 3rd hrd drv(drive O), which i also right click and drag immediately after the first backup.</p>

<p > </p>

<p >there is no software involved i simply use windows xp pro sp3's right click and drag feature. i do backup my c drv every saturday morning at 2am automatically with Norton Ghost 14.0, but i do not use it for the other backup process that is all right click and drag. it works fine and with no other software involved. i do right click and drag certain folders(biggest being about 40gb) from my c drv to my O drv, these are for backup purposes. This way I can backup and restore a folder without restoring the whole drv.</p>

<p > </p>

<p >1 item about using software to make a backup, that uses incremental backups after the first one is that the backup will grow and grow and grow because you are always adding to the the backup never replacing it. in time it will be much larger than the item you are trying to backup. the only alternative is to wipe the backup out and startover. with my right click and drag system the backup is exactly the same size as the original. Not all incremental backup programs do this, you have to check.</p>

<p > </p>

<p >my 4 ext hrd drvs are a 1.5tb, a duo of 500gb, and a 750gb.</p>

<p > </p>

<p >Note-my main primary ext hdr drv includes all format file types. they are raw jpeg tiff and several movie versions including mov. i have never had a problem with the files remaining as they are or being corrupted in any way. but also note that i am not keeping them on the c drv. the only image files that are on the c drv are the ones that i am currently working on or just put there from the camera.</p>

<p >note-I shoot jpeg all the time. When I am done with the original jpegs(these are always unaltered in any way) they go into a jpeg holdall folder. When that folder reaches 4.1gb I burn it to 2 dvds. 1 stay in my home while the second is put at m mother’s home which is 40miles from me.</p>

[derek_kennedy]

<p>Here is a horror story I hope no one else has to live through...</p>

<p>Ok, I've been putting my photos on a external HDD for years, starting with a small laptop drive, now a 1T drive (only half full but room to grow!).</p>

<p>I use PS elements 7 (well, I do now since its the latest) to tag my 40,000 digital files going back over ten years. I made a backup of the catalogue but when I went to put the files on a new hard drive in the new computer (so I would have two copies now), I guess I didnt understand what the program did.</p>

<p>Sure I had my original folder structure I first set up, but when PSE7 made the backup, it put all folders in the main directory, and also made copies of the photos in the main directory - so in essence I had 3 copies now.</p>

<p>I never was able to have the program create the directory (and ALL tags, each photo having at the very least 4 tags each) because the external HDD failed. A the snap of the fingers - it failed.</p>

<p>I got the original photos in the original folder structure copied over to the new computer no problem but when I saw what the problem was with the backed up catalogue, I went to copy over those othe files and I *zapped* ? the external drive when I had to move it. Not sure if it was static electricity, or a short but the drive was now completely dead. I even removed it from the external encloser and put it into the new compter - nothing. Dead as a doornail.</p>

<p>The good news is that I was able to copy over all 40,000 photos (but not the backed up catalogue stuff). Bad news? now I have 40,000 photos to retag. Thats a solid week of work, 8 hours a day. I know. This is the 4th time Ive had to retag those photos.</p>

<p>This is the first time Ive ever lost an external HDD. I lost everything else on the drive. Inluding home movies of my then 1 y/o son.</p>

<p>Now I need to replace the drive and the enclosure just incase it was the enclosers fault.</p>

[twmeyer]

<p>http://thedambook.com/smf/index.php?PHPSESSID=2b84c9c89ab2776c8a462227b6e3e327&board=4.0</p>

<p>Make an hour or two and read through that forum.</p>

<p>I use two drives in my computer case. The primary © is 250G of that 500G disk and the other partition (D) is 250G. Well more like 180G each, but who's counting?.</p>

<p>C has naught but OS and software, D has all my text files (about 12G) and nothin else. It's the primary scratch disk.</p>

<p>The second drive is a 500G disk that is for current commercial work and it is backed up to JBOD (Just A Box Of Disks) that has 4 500G drives in it. It's an SATA drive and the card for it costs $12.</p>

<p>One external is personal work that is backed up to another disk in the JBOD and then on my desktop is one of these</p>

<p>http://thermaltakeusa.com/Product.aspx?S=1268&ID=1731</p>

<p>It has a 1T 32mb buffer internal SATA drive that auto backs up my personal and commercial work using Mirror Folder software. That software also backs up automatically all new files that are written to the personal and commercial work drives. The 1T drive is pulled every night or when I leave the premises. When all of those drives get filled, it's time to reinvest.</p>

<p>I just bought the 1T WD Caviar Green drive for $100. That's 10 cents per gigabyte... t</p>

<p>(any raid is a headache that is just not necessary 'cause if something toasts one drive, it will probably toast the other. Better to have a backup that can be taken off line easily. With the Thermal Take dock, you can pull a 1T drive off line as easily as a thumb drive)</p>

[garydem]

<p>derek k-your expeience is precisely why i have a primary external hard drv and a a identical backup hard drv. i also have a 3rd copy on another ext drv. i also have the original pics burned to dvds in 2 copies, only 1 of which is kept at home. no matter what i can reproduce the images. i have had 2 drvs fail in 4yrs. in no case did it make any difference to me. i simply switched to a newly bought hdr drv and copied the data over. all i lost was some time.<br>

if you buy 1 ext hdr or a internal hrd drv then you should buy 2. one is the primary and the second is the backup. and that backup is touched except to make the backup copy. the only possibity of something going wrong is when you make the backup to tghe primary and you loose both the primary and the backup. and that is why i have a 3rd copy.</p>

 

[Tom Wiggins Photography]

<p>I use a Western Digital 1Tb for external storage/back up and 250GB for image storage along with my 500GB internal drive. Tom</p>

[Tom Wiggins Photography]

<p>I use a Western Digital 1Tb for external storage/back up and 250GB for image storage along with my 500GB internal drive. Tom</p>

[garrison_k.]

<p>Derek K, lokk into buying Norton Ghost v14 and set it up to back everything up. Not just the folders, but a clone of your drives.</p>

[nsfbr]

<p>For what it's worth, I use a 2TB WD external for primary storage of my images AND I access it wirelessly. I'm a Capture NX2 user so naturally speed is not my highest priority. When I want to work on some images without worrying about the speed slow down, I just copy them to my computer, work on them, and then copy them back. The WD doesn't get much wear and tear and I don't worry about it failing. At some point, when HD space is about $50/TB (next year probably), I'll clone the drive to a mirrored version or some such thing. If I was a professional and using the drive as a primary I'd mirror I'd mirror it now or use a higher level RAID array.</p>

[daniel flather]

<p>I don't shhot as much as many members here do, but I copy to an external and burn to DVD. The DVDs are stored off site. You must burn to DVD as soon as you have 4gb of files since you last burn; make that a habit.</p>

[philfx]

<p>Nothing wrong with external. If your computer can do eSATA, get a Sans Digital dual drive storage and do the RAID/0.<br>

If Not, it is worth getting a card for eSATA/ fast. I would make sure you keep the drives cool, as heat is the #1 killer (drops the life of the drive).... hence, San Digital has one with temp alarm. There are other makers for such devices.<br>

Fujitsu Makes very fast large drives 10K rpm. more pricey than getting each 1TB for $99. Which is a really great deal.<br>

I run 3 dual ext. enclosures on eSATA with 1TB drives and they are rather quick. 2 of the enclosures were the older style with no temp alaram, and I had a problem with one of those drives early. I wasn't sure if it was San Digital issue or bad luck, but I swapped to the new external cases with temp alarm, so far so good.<br>

I wouldnt bother with externals that come with a harddrive, as that limits your choice and size, and for the most part, if you got it for a low price the drive it came with is not the best you are able to get. I think getting the 2 sep is the way to go (if you can install a drive).</p>

[sasvata__shash__chatterjee]

<p>Well, looks like you'll get as many opinions as there are posters on this thread :-). Here's another opinion.</p>

<p>I think it is best to store your entire set/collection/library of photos on an external drive, simply because it gives you the freedom to access the photos from any other machine. Your data would be independent of the OS revs, virus-infection status, in-use state by other people, etc. etc. of that one machine. As you and others have said, you should use RAID-1 mirroring, it is a relatively inexpensive option but dramatically increases reliability. You should still consider some sort of infrequent off-site backup even after the RAID-1 protection, since although the probability is much lower than a single drive failing (in the non-RAID case), there are scenarios where you can loose data/access from the RAID-protected drive.</p>

<p>Access from RAID-1 is slower than the equivalent non-RAID, particularly for write operations. Access from external drive is slower than access from a drive on a local bus. I have two drives on the local bus, one for OS and applications (Photoshop), and the other for Photoshop scratch disk. In that case, the penalty from external+RAID is only when you open and save files. I find that I can live with this delay just fine, it is small compared to the time spent editing the file. You can reduce this delay if you want, if you consider that most edits are done around the time of first import from camera. In that case, import into local disk and make a copy into external RAID. Work on edits from local drive, overwrite on RAID drive when done on that import, and then delete from local drive. After that you only work on selected images, and I would live with the delay of a external RAID because of what you gain in reliability, access and transportability, not to mention sanity and peace of mind.</p>

<p>I personally find the management of multiple non-RAID backup drives/folders to be very tedious and error-prone, particularly when done manually without the aid of software.</p>

<p>EDIT: By the way, when I mention "external", I am using an old PC as a server running Linux and accessing files across a 100Mb/s Ethernet home network.</p>

[b_reagan]

<p>Good advice from all, particularly on drive failover strategies (RAID options). However as an IT pro designing mission-critical data storage solutions for my clients, I'd be leery of betting the farm (your entire photo archive) on PC-based drive technology, i.e. SATA or any of the earlier protocols. PC-based (as opposed to server-based hardware designed to run 24x7 for years) is by and large disposable junk with a very low MTBF and data transfer rate when compared to current SCSI technology. </p>

<p>For less than the cost of a Canon 300mm F2.8 IS lens you can equip your workstation with 2 fast (15k rpm) mirrored pairs (1 pair for O/S and software, 2nd pair for work-in-progress) AND build an NAS server with a 1Gb connection to your workstation that is almost infinitely extensible using slower 300Gb drives in a RAID5 +1 configuration, which gives you a hot-failover drive should any of your primaries fail. This means the volume will automatically recover and you can replace the failed drive at your leisure without any data loss. Let's say you start with a chassis that has 8 slots, your available near-line backup is ~1.8Tb. as a single volume (8 drives -1 for striping and 1 as hot spare = 6*300 = 1.8Tb). Of course you can increase the drive capacity as its available and add as many NAS boxes (volumrs) as you like.</p>

<p>I admit this is definitely a belt + suspenders solution, but it makes for long-term data security with no ongoing hassles. Myself, I do digital only as a 'contact sheet' for my developed film, but my workstation is configured with dual-mirrored SCSI3 pairs. The keepers (which are dwarfed by the number of throw-aways) get a high-res scan and the film goes off-site, so my long-term data storage needs are minimal as I can always get another scan. But for you digital-only folks, lose your hardware and you lose ALL your work. Makes belt + suspenders attractive, yes?<br>

cheers<br>

bernard</p>

[b_reagan]

<p>Good advice from all, particularly on drive failover strategies (RAID options). However as an IT pro designing mission-critical data storage solutions for my clients, I'd be leery of betting the farm (your entire photo archive) on PC-based drive technology, i.e. SATA or any of the earlier protocols. PC-based (as opposed to server-based hardware designed to run 24x7 for years) is by and large disposable junk with a very low MTBF and data transfer rate when compared to current SCSI technology.</p>

<p>For less than the cost of a Canon 300mm F2.8 IS lens you can equip your workstation with 2 fast (15k rpm) mirrored pairs (1 pair for O/S and software, 2nd pair for work-in-progress) AND build an NAS server with a 1Gb connection to your workstation that is almost infinitely extensible using slower 300Gb drives in a RAID5 +1 configuration, which gives you a hot-failover drive should any of your primaries fail. This means the volume will automatically recover and you can replace the failed drive at your leisure without any data loss. Let's say you start with a chassis that has 8 slots, your available near-line backup is ~1.8Tb. as a single volume (8 drives -1 for striping and 1 as hot spare = 6*300 = 1.8Tb). Of course you can increase the drive capacity as its available and add as many NAS boxes (volumrs) as you like.</p>

<p>I admit this is definitely a belt + suspenders solution, but it makes for long-term data security with no ongoing hassles. Myself, I do digital only as a 'contact sheet' for my developed film. The keepers (which are dwarfed by the number of throw-aways) get a high-res scan and the film goes off-site, so my long-term data storage needs are minimal as I can always get another scan. But for you digital-only folks, lose your hardware and you lose ALL your work. Makes belt + suspenders attractive, yes?<br />cheers<br />bernard</p>

[mirek_elsner1]

<blockquote>

<p>Are there good reasons for not storing all my stuff on an external<br /> drive. Is this something I will regret?</p>

</blockquote>

<p>I store my working copy internally and a backup externally.<br>

If you use consumer technologies like USB or FireWire, your external disk will be about 2x slower than internal (consumer grade) HD. External drives (at least those that I use) are not smart enough to switch off when I put my computer to sleep mode.</p>

 

<blockquote>

<p>Will Photoshop slow down when<br /> working with these files (or just when opening and saving them)?</p>

 

</blockquote>

<p>It will slow down just when opening and saving the files - as long as you don't use this disk for operating system paging/swap file and as Photoshop scratch disk.</p>

 

<blockquote>

<p>I'm considering some 2 TB Western Digital drives that provide<br /> RAID 1 drives for data mirroring. This will provide some data security<br /> and peace of mind should one of the two external drives fail.</p>

 

</blockquote>

<p>RAID 1 is better than single disk, but you should have independent backup. For example, with RAID, if you rewrite your image by mistake, it will be rewritten on both disks and your original data will be gone...</p>

[philfx]

<p>One key thing that I think was mentioned and perhaps someone has a sollution...<br>

External hard drives DO NOT spin down or turn off when your system goes into standby, or shuts down. Is there some sw that manages external drives? I know one of my SeaGate external drives does actually turn off or sleeps when not active, and it is connected via Firewire, but this is a enclosure that came with the drive. Or perhaps a few suggested enclosures that have chipsets that are smart enough to manage the drive (Like the SeaGate).</p>