Cheap RAID

[shawngibson]

<p>Hey guys,</p>

<p>I never bought into the whole RAID idea (I bought lots of Raptors with my last PC, good enough at the time usually), but right now, I'm working on 32bit images in PhotoShop, stitches, often with 40 images stitched, and it's taking 35-50 minutes to save a file with layers. I have the new iMac 27" quad, 4Gb ram (I don't think it's a ram issue, though 8-12Gb more ram will help with processing for sure).</p>

<p>If I buy some kind of 4-drive system for USB or firewire, can I <strong>dramatically</strong> decrease my save times? I mean Raid with no backup, merely a speed setup, on cheap drives (then pass the file over to my server when I go to bed for backup)...if I buy 4 drives does that cut my time saving files by 3/4 or thereabouts on iMac USB or Firewire?</p>

<p>Any recommendations?</p>

<p>Would a scratch drive make a big difference? I suspect not as that's a memory thing, not a write-to-disk thing...</p>

[brad_smith8]

<p>The USB or Firewire transfer rates aren't even fast enough to allow one drive to run at its rated speed. It would have to be internal using a RAID PCI card if possible on an iMAC.</p>

[shawngibson]

<p>I don't think that's possible:(</p>

<p>I think, if that's true, that I'll have to save ALL my files to the server (Supermicro AMD dual proc quad with 6 drives...that are all being used). God help me if I use the microwave oven, which cuts the breaker and kills my save. Note to self: dont eat, ever. Another note to self: get landlord to upgrade electrical stuff from 1940 when the house was built...</p>

<p> </p>

[garrison_k.]

<p>RAID O always makes a huge difference. It's one of the single biggest improvements you can do after topping out your ram. Unfortunately, you'll have to sell the laptop and get a real workstation.</p>

[shawngibson]

<p>I always forget the 0s and 1s with raid. Its not a laptop, it has a decent processor but its an all-in-one unit that cost $2500. My AMD box, OTOH, cost me $10,000. Maybe I should stick to the AMD box. You cant even add a vertical monitor to the iMac, which drives me bananas...</p>

<p>Unfortunately my AMD box is low profile (2U) and the video card (Matrox) cacked out; Ive never been able to find another lo-pro card to replace it:( Which is why I bought the iMac, which I must say I dont like. Why artists are so adamant on Apple, I dont know...</p>

[garrison_k.]

<blockquote>

<p>Its not a laptop, it has a decent processor but its an all-in-one unit that cost $2500.</p>

</blockquote>

<p>Sorry, I call them laptops because they are glued shut and you can't do f*k all with them. I call them laptops because they are about as fast as one. I call them laptops because in order to use one seriously, you end up with cables and power cords everywhere because the second you need a card reader or an external drive, there's a birds nest on your desk.</p>

 

<blockquote>

<p>Which is why I bought the iMac, which I must say I dont like. Why artists are so adamant on Apple, I dont know...</p>

</blockquote>

<p>Designers and directors are the only ones I know that are adamant and stuck in their ways. But I'm not sure why we, image-makers, buy it. The monitor is junk, and as most digital dark room users need, there's no room for upgrades. I realize there is no "in between" with Apple's product line and this is the entry level "desktop" but nonetheless, it's a horrible machine for digital dark room in terms of flexibility. And without a doubt, the most expensive route as well. You can buy the guts of an iMac at newegg for $900 then a $600 NEC monitor and build a hackintosh. Then at least, you'd be able to build your RAID system :)</p>

[shawngibson]

<p>Which is why I said its a POS. LOL</p>

<p>The monitor is CRAP.</p>

<p>I can open my AMD box and change anything. My mistake was making it so short...</p>

 

[brad_smith8]

<p>Boy, $2500 could buy an ultimate PC with 4 drive RAID0, 4+4 core i7 processor, etc. I don't know why Apple charges so much...wait, of course I do.</p>

[brad_smith8]

<p>Can you swap out the internal hard drive of the iMac with an SSD? I know that they're expensive though, but the ones based on the SandForce controller are faster than my four-drive (74GB Raptors) RAID0 array (250MB/s vs. ~220MB/s).</p>

[mpd]

<p>If you are looking for disk performance you should be looking at Raid 0 or 5. Both split the data transfer between the disks. Raid 5 adds an extra level of security. If one disk fails in raid 5, your whole array does not fail. You can replace the failed disk and no data will be lost. For a rule of thumb consider the more disk spindles, the faster disk performance will be (there are lots of other factors to consider but that is beyond this post). You can get a external raid 5 array pretty cheap these days. The more disks you put in it the better the performance will be. Use your firewire connection and not USB.<br>

You might also want to consider splitting up the disks where activity is taking place. You could have one disk for the operating system, another for Photoshop scratch disk and a third for storage (when I say disk I mean a raid array or a solid state disk). How many Firewire connections do you have?<br>

I have yet to see a laptop that outperforms a desktop. As a reference point, my desktop which does not have a particularly fast disk subsystem can still write 5 GB/min. That is more than enough for my needs.</p>

[garrison_k.]

<blockquote>

<p>Use your firewire connection and not USB.</p>

</blockquote>

<p>What would be the point of building a RAID system and then bottleneck it with a FW800 connection?</p>

 

<blockquote>

<p>Which is why I said its a POS. LOL</p>

</blockquote>

<p>lol, true.</p>

[brad_smith8]

<p>The fastest Firewire is rated at 800 MegaBITS (Mb) per second (~780 actual). To relate that to hard drive MegaBYTES (MB) per second you divide by eight. That gives you less than 100MB per second, yet the latest 2TB hard drives top out at 120MB/s. So putting anything that fast or faster (with RAID0 or RAID5) would just be a waste.</p>

<p>I don't know at the iMac's rated drive speed.</p>

[szrimaging]

<p>Do either of the machines have an eSata connection? You can get an external raid case and run it through an eSata connection to have it run about as fast as internal. You could run RAID6 or RAID1+0 to get some redundancy and speed.</p>

[brad_smith8]

<p>Interesting upgrades for the 27" iMac <a href="http://eshop.macsales.com/shop/turnkey/iMac_2010_27">here</a> including eSATA and SSD's.</p>

[Uhooru]

<p>New iMac 27 quad core i7 is no laptop in terms of performance though it's as hard to open up. It is a full desktop computer. I don't know why people spread that kind of disinformation. The thing performs as well as any computer with a comparable chip. The older model iMacs did use lap top components. No longer true in the higher end iMacs.<br>

Other World Computing (OWC) will now do a turn key installation of a eSata Port on the iMac 27. The eSata is much faster than FW 800. If you have the cash they can also put in SSD's as well.<br>

USB 2.0 is much slower than FW 800. The eSata conversion I think is around 200 USD or so. On your FW 800 you chain together drives as well. I believe you can buy a raid cabinet for esata as well as FW 800. Look at OWC, Newegg, Powermax etc. I wouldn't waste mony using the USB port for HDs.</p>

[garrison_k.]

<blockquote>

<p>New iMac 27 quad core i7 is no laptop in terms of performance though it's as hard to open up. It is a full desktop computer. I don't know why people spread that kind of disinformation.</p>

</blockquote>

<p>The iMac's have more in common with a laptop than a desktop. They have no upgrade path. The same guts are available in a laptop in some shape or form. You are stuck with a monitor. And just like laptops, with an iMac you are paying more than the equivalent power in a desktop. I'm not sure how that's confusing?</p>

 

[Wouter Willemse]

<p>eSATA would be the way to go, and apart from RAIDo and RAID5 as performance enhancement options, there is RAID0+1, meaning you make two RAID0 setups (striped drives, for performance) and mirror those in a RAID1. For a non-server, that would be my choice, but it does demand 4 drives. The advantage is, it can be done on normal hardware (with OS support, I am not sure for OS X), while RAID5 requires a dedicated RAID controller.</p>

<p>Low profile videocards are relatively common these days; many lower end videocards come with conversion brackets to make the full-profile into half-height. Think cards as Radeon HD5450, Geforce G315 etc. They are all PCI-Express, but if your AMD machine is recent enough, it should be no issue.</p>

[waltflanagan]

<p>35-50 <em><strong>minutes</strong></em>? Did you mean seconds? What is the final size of the file when saved? 1GB? 2GB? You should be able to write something that size in seconds. If it is taking minutes then you are probably swapping and need more memory. Check the memory usage of your apps first before blaming the hard drive.</p>

[shawngibson]

<p>Thanks guys. OK it might not be 35-40 minutes, but it's definitely in the half-hour range. The file (current one, I'm expecting they'll all be similar in the coming weeks/months) is 9.27GB on disk (that's not a typo...9.27Gb) and once I convert back to 16 bit (32 now) I'm sure it will get a lot smaller. It's a 3' x 4' print at 240dpi approx, though I think I'll print it at 360 dpi for detail, so it will obviously be a smaller print.</p>

<p>Near as I can tell based on the info you guys have provided, I should buy the eSata adaptor and get say 4 drives in a raid 5 configuration? Maybe from there a USB drive for scratch? I can't afford solid state at this point in time. This is on the assumption that I will backup my work nightly on the AMD box.</p>

<p>I'm sorry for bashing the Apple, but when I bought it I thought it LOOKED similar to my AMD box (AMD is 2 Opteron 285s - 2.8Ghz duals x 2=quad), 8Gb ram, but it's server ram, ie ECC and in theory slow, and I have 6 hot-swap drives - all are Raptors save a couple of 7200rpm 1Tb drives. The APPLE is a 2.8 Quad with 4Gb that I can upgrade I believe to 16, and I had no idea the upgrade path for hard drives would be so challenging LOL.</p>

<p>I'm at this point willing to hear someone chime in and say make the AMD box work. Anyone know of a decent 2-unit high video card that can run a pair of Dell 2408s, with one in portrait mode?</p>

<p>In fact, at this point, I guess the question is, what will be cheaper to upgrade, the Apple or the AMD? BOTH computers perform well in the processing area; both are equally quirky (they both have a few operational things I don't like); but on the saving-to-disk issue, that's what I need to upgrade. In fact, my ass-dyno says the Mac is a bit faster than the AMD for most computations in photoshop. But I've not tried to save-to-dis a big file to the AMD because of my videocard issue - not using the AMD for Photoshop at all these days.</p>

<p>I'm almost thinking buying/swapping over a couple of big slow drives for the Mac (external cases are cheap), use it as my do everything (minus Photoshop) box, and turn my AMD into a Photoshop box. It has so much more potential even though it's an old 940 socket box. If I can find a good video card. Heck maybe I can find a card that's not low profile and just take the top off the server LOL (it's a Supermicro 822 rackmount unit).</p>

<p>I'm so confused...lol</p>

<p>Shawn</p>

[Brad_]

>>> The iMac's have more in common with a laptop than a desktop. They have no upgrade path. The

same guts are available in a laptop in some shape or form. You are stuck with a monitor. And just like

laptops, with an iMac you are paying more than the equivalent power in a desktop. I'm not sure how that's

confusing?

 

That is confusing. A laptop is portable with 2.5" drives (and lately SSD), run on battery power, and can be

operated sitting on top of your lap (ie, laptop). And offer less computing power that desktops. That's not confusing. I could

go on, but that should be enough.

[yuri_yupiter3]

<p>You have something fundamentally wrong with your settup if a file takes 30 minutes to be saved. Even with 10 year old computers and 10 year old hard drives, times like this are like saving 5 gigs across a Network.</p>

 

[Wouter Willemse]

<p>Shawn, does your board have an AGP slot or PCI-Express? I think certainly for PCI-Express, finding a half-height videocard should be no problem (and any card nowadays has enough grunt for 2 large displays). For AGP, it will be more difficult.</p>

<p>For example a card like this one: http://www.newegg.com/Product/Product.aspx?Item=N82E16814134121 should fit, as far as I can tell; you will just need to replace the default full-size ATX bracket with the included half-height one, and probably add a HDMI-DVI converter to hook up the second monitor.</p>

[waltflanagan]

<blockquote>Thanks guys. OK it might not be 35-40 minutes, but it's definitely in the half-hour range. The file (current one, I'm expecting they'll all be similar in the coming weeks/months) is 9.27GB on disk (that's not a typo...9.27Gb)</blockquote>

<p>I just copied 8GB from one drive to itself and it took 3 minutes and 17 seconds. As Yuri said something is fundamentally wrong with your setup and I don't think your hard drive has anything to do with it.<br /> <br />You said that the machine only has 4GB RAM and the output file is 9GB. That 9GB will take even more memory when stored in the Photoshop data structure. Simple math shows that you are at least 5GB into swap. Run whatever MacOS memory usage program there is and find out how much Photoshop is using. I wouldn't be surprised if it was using 20-30GB of RAM + swap.<br /> <br />If you still want to "upgrade" your hard drive.... So this 35-40 minutes is saving to the internal iMac hard drive? The current iMacs already have 7200 RPM SATA drives. I'd bet $5 that going to a RAID 0 setup over a slower interface like USB 2, FW 800, or Gigabit Ethernet will be even slower than your internal drive. eSATA would be as fast as the internal SATA drive but it's not a trivial addition like in a PC. Read this about the different hacks to add it.<br /> <br />http://blog.fosketts.net/2010/09/07/howto-add-esata-imac/<br /><br /></p>

[brad_smith8]

<p>Yes likely RAID <em>and</em> RAM is what you'll need to get those times down where you want them.</p>

<p>I'm not sure if half-height = 2U height. Cases aren't that expensive; have you thought of putting THE AMD in a standard case to use standard PCI/e cards and perhaps more drives?</p>

[shawngibson]

Thanks guys. I'm glad I asked this question, something is obviously wrong.

 

Mac: single 7200rpm drive. No external drive for swap or data. It's brand new and has the installed OS plus Photoshop

and Nik silver plugin that's it lol. Would 4gb ram plus a 2.5" USB portable drive for saving and 7200 rpm powered 1tb

USB drive for scratch help? I have those two drives available right now and can save for ram.

 

Amd: I built it. Board is supermicro h8dce-hti iirc and it's a server board, larger than normal desktop case I believe. It

has dual pci-e or pci-express I forget the proper terms. Only time I ever studied components was to build this box so

I'm a bit ignorant. I researched and bought what seemed best at the time. The 2 unit rack mount case was a stupid,

stupid idea. Lol. A case plus a graphic card might be the simplest and cheapest way to go.

 

I'm sure the iMac would make a great movie watching Internet surfing machine:)