USB Hard Drives -- fast enough?

You need USB2.0 at a minimum. Firewire 400 or 800 are even better choices. USB 2.0 has a rated speed of 480Mbps, compared to Firewire 400's 400mbps. However, real world experience shows firewire to be faster because firewire devices are more intelligent and does not require the computer CPU to do arbitration between the end devices.

 

Firewire 800 PCI card only costs about $70. Firewire 400 costs $30. Both are very cheap investments.

Also the backup utility that comes with Windows XP uses a single large file and FAT32 has a 4GB or so file size limit that causes the backup program to choke partway through.

Well, it happened sooner rather than later. My primary hard drive is filling

up quickly with medium format scans and I need more storage. Rather than just

toss in a new hard drive, I've decided to look at my storage needs in general,

backups (umm... what backups?)

 

My initial thought was to get three 500GB external USB hard drives. Perhaps,

Western Digital My Book, since it's a low-cost known brand. One would be my

main data storage. I'd no longer store files in my PC, which could make

upgrading easier. I could do a weekly backups to the other two drives

alternately. Does that sound reasonable?

 

But my primary concern, are external USB drives fast enough for saving large

images in Photoshop? Some of my files get rather large... I often do 20x24"

posters, which end up around 250mb while editing (300 dpi, 16 bit, you do the

math).

 

Thanks for helping with this!

I bought the 500 GB My Book Premium Edition with USB 2 and Firewire and Firewire ports. The My Book included both the USB and Firewire cords, which was nice. I use the Firewire connection and its pretty fast moving 450 MB files from medium format scans:

 

 

http://www.wdc.com/en/products/Products.asp?DriveID=224

I also bought a 500GB MyBook Premium recently. It is my first experience with Firewire I/O, and it seems noticeably faster than USB 2, although USB 2 is also "fast enough," in my estimation.

 

One tip: the MyBook external HDDs come preformatted in FAT32 format, for maximum compatibility (i.e., with Win9x systems). If you're using Win2K, XP or Vista, copy the folders that come preinstalled on the MyBook (the backup software, etc.) onto your internal HDD, then reformat the MyBook with NTFS. You'll experience much better performance -- particularly writes -- than with FAT32.

It's more than fast enough. Remember, you still would want to backup your stuff via DVD and/or another drive.

 

NEVER, EVER store ANY important files onto your system drive!

I think you will find USB (even 2.0) too slow for saving large Photoshop images in real time. I certainly did when I tried it. It would be OK as a backup solution for running overnight backups (provided you don't take an approach that requires backing up every file every night).

 

Far better would be to use a Serial ATA (SATA) interface. You can buy external drive enclosures that support a SATA connection. If your PC is an older model, you can buy a SATA card (i.e., PCI SATA controller), and attach both an internal (SATA) drive and a SATA drive housed in an external enclosure to the same SATA controller. If you have a newer PC, it might already support SATA natively (internally, at least, but you might still need to buy a separate card to support the external enclosure).

 

With cheaper SATA controllers, your external drive will not be hot-swappable (unlike USB). I use exactly the setup that I have described above, and I have to shut down my computer in order to switch backup drives (at all times I keep one of my external backup drives at a friend's house, as an offsite backup, and I rotate the two every week or two). This is not a big problem, but if you would find that inconvenient, you could always make sure that you purchase a SATA controller that supports hot-swapping drives.

<p>

<a href="http://www.kingwin.com/tl35cs.asp">This is the external SATA drive enclosure</a> that I use. I have a pair of them, each with a Western Digital 500GB drive in it. I've found the enclosures to be quiet (i.e., fan is very quiet) and reliable. I have <b>no</b> connection with the manufacturer (Kingwin). This drive enclosure also supports USB 2.0, by the way. It has annoyingly bright blue lights on the outside of the enclosure, but I've covered them with black tape.

</p>

<p>

My Western Digital drives (WD5000KS) are also very quiet, and the performance is good. I have three of them in total (one in the computer; two in external enclosures). I don't know if WD was going through some quality problems when I purchased mine 4-5 months ago, but I had to return four drives before I ended up with three good ones. On the other hand, I can't speak highly enough of WD's excellent customer service (replacements arrived very quickly).

</p>

It looks like eSATA contoller cards and enclosures can be had for under $25 if you know where to look.

Actually the same drive is used in internal and external. The "bottleneck" is in the USB/external controller area; not the hda. Remove the hda and place it in your box and it will read write at full tilt.

Yes, <a href="http://www.newegg.com/Product/Product.asp?Item=N82E16816132006">here</a>, for example.

Firewire if at all possible. The difference between 800 and USB2 is truly painful.

Les, a firewire 800 drive is comparabe in speed to an ATA133 or SATA drive, maybe a bit

slower, but not a quarter. Firewire 400 is about 70% of that in my experience. USB, nowhere

near.

 

Tom, USB is fine for backup, but not as a main drive. If you want an external main drive,

consider only e-SATA and Firewire 800.

<p><i>"All USB/FW external hard drives are about a quarter as fast as your internal IDE drives."</i></p>

 

<p>Sorry, but that's just not true. Well, maybe it is if you're buying cheap hardware, or plugging five USB devices into the same controller, or something like that, but good-quality hardware - implemented correctly - is pretty fast.</p>

 

<p>As an example, I have a few 500-gig Maxtor drives with Firewire B, and they can sustain 60+ MB/s *writes*. Admittely, that's with a PCI-X Firewire B controller, but again, that's where using good hardware and implementing it correctly come into play. If 60 MB/S is a "quarter as fast" as an internal IDE drive, let me know what kind of drives you're using that write at 240 MB/s...</p>

 

<p>(I do have one "drive" that can write at over 240 MB/s, but that's a RAID 0 array with 15K RPM SCSI disks, with a dedicated PCI-X bus just for the U320 controller)</p>

Les, as per your observations, sustained transfer rates of large files is where Firewire truly shines over USB. (That was why Nikon chose a Firewire interface for the Coolscan 4000). Smaller files not so much of a difference between them though.

eSATA is certainly the external interface of choice if performance is the issue. However...

 

Just be careful when employing an external SATA drive. The designation is eSATA, and it is more than an SATA drive that resides outside your computer. The HD drive itself is the same, but the host and client specifications have been reworked to ensure data integrity. Since this is a relatively new interface, it is still in the process of being implemented by hardware vendors and tends to be somewhat rare in the real world.

 

I have seen external enclosures advertised as supporting eSATA when in fact they did not. An eSATA connector is no guarantee of anything. At a minimum you should see the eSATA logo on the box or on the product itself. When in doubt, contact the vendor. Just be aware that they may not know either, or may not be completely honest. Rosewill certainly did NOT send me factual info regarding their eSATA enclosure.

 

Finally, older motherboards do not support eSATA and installing an add-in card is the only way to achieve a reliable connection with the external enclosure.

The main perceptible differences between SATA and eSATA (the 'e' stands for "external") are that 1) the cable connecting the external drive to the computer can be 2m long (with eSATA) instead of just 1m (with SATA); and that 2) the eSATA connector is more robust (is designed to better withstand frequent insertion and removal). These are the data integrity improvements that Howard is referring to. As Howard also pointed out, the drive itself is still a SATA drive -- it's just the enclosure that is either SATA or eSATA-compliant. Both SATA and eSATA theoretically support hot-swapping hard drives but (as noted earlier) not all SATA controllers actually support this.

 

My approach (several months ago) was to not sweat the SATA vs eSATA question and to buy a regular SATA controller (for the computer) and enclosure (for the hard drive). This means that my cable can only be 1m long (not a big deal for me) and is less robust than eSATA cables. As Howard noted, finding a good eSATA enclosure has not always been easy. I decided that if the robustness ever becomes a problem, I will upgrade the controller and enclosure, both of which are relatively cheap items. However, I should emphasize that I only swap my backup drives approximately once every 1-2 weeks, and treat the connector gently.

 

The eSATA market has probably moved on since I implemented my backup strategy, so if anyone has found a really good eSATA enclosure, I'm sure we'd all be very interested to hear about it.

The debate about eSATA, though interesting, does not apply here. For 300MB photo files, the firewire interfaces are plenty fast enough. I've been using firewiree 400 and 800 interfaces to process 300MB files and streaming DV video files with no problem. Also remember the PC bus has to be fast enough to process the data also, it's not just a matter of buying an interface card.

<BR><BR>With a 1996 PPro motherboard with a 66Mhz buss, a 133 class UDMA IDE-PCI controller here read writes at full tilt to a modern 80Gig 7200rpm 133 class HDA. One gets a nice 65megs/second at the OD and it drops to about the 30's to 40's at the ID. This is one 333Mhz Pentium II overdrive processor. <BR><BR>If place an 200Mhz PPro CPU in the box; the top end drops to about 50. If I declock the board from 3x to 2X and drop the buss down to 60 I can run the CPU at 120Mhz and drop the transfer rate down to the teens in Megabytes/second. <BR><BR>With a modern CPU:) of 333Mhz and the plain jane 33Mhz PCI buss and a modern 133 class controller; the HDA is still the limit if you box is settup correctly. <BR><BR>In most systems whether a decade old or todays the controller is several times quicker than the HDA's INTERNAL transfer rate. This number is NOT the one printed anywhere on the box.<BR><BR> The latest twin 300 gig maxtor IDE's I bought two weeks ago clock in at 72megs/second at the ID; whether on my latest 3Ghz box; or 10 year old 333Mhz PPro overdrive box with modern PCI IDE card. IF one places these new drives on the 10 year olds IDE port on the 1996 motherboard; one gets a flatline transfer rate of about 14megs/second; with the contoller whussing out the drives transfer rate. <BR><BR>If one places a modern HDA on a typical early Pentium III 450Mhz box or Pii one usually has an old 33 class IDE controller; and the transfer rate is usually about 25 to 28 with a modern HDA. A later Pentium III say a 550Mhz or higher ofen has a 66 class IDE controller; the transfer rate of a modern HDA is clipped again; usually to say 40 to 50 max speed.<BR><BR> You really want the controller to be double what your HDA is for a full tilt transfer; and to allow future upgrades too.<BR><BR>Several of the external USB 2 drives I have tested have been quick; one mystery brand one ALWAYS runs at snail slow USB 1.0 speeds of about 0.8 megs/second; no matter what computer its connected two. On several boxes with USB 2 adapters or USB 2 ports the drive runs; but the OS seems to think its a high speed USB 2 device plugged into a USB 1 port. Yet another mystery brand USB HDA of a friend runs faster than USB 1; but flatlines at the ancient 12 megs/second bussmaster speed of a decade ago. One can pull the hda out of the dumb enclosure and connect it to a IDE cable and it transfers at full tilt; 60 at the OD; 40 at the ID. Thus those Ebay specials might be cheap for a reason; goober controllers! :)

You really should run some HDA transfer rate benchmarks using your own hardware. Everyone has different USB and firewire ports; different HDA's'; it only takes an ill combo or weird problem for an HDA to back down in transfer rate. From a technical standpoint both usb2 and modern firewire or the contollerr should NOT be the limits of an HDA's transfer rate. Something is amock with the settup of these external boxes if one has transfer problems.

A real transfer rate program will also transfer BIG files; so the internal buffer of 1 to 16megs doesnt "goose" the transfer rate numbers up when dinky files are written.