PC for working with 3gb files?

[don_v.]

What kind of setup should I look at for working with 3gb files? These are 3000 dpi drum scans of LF (sheet) film

for mural-sized output; I'm starting to get the scans back and now I need more computer power to handle them.

 

I've done as much reading as I can take and am just starting to get a handle on RAID options, multiple hard

drives, video

cards, and RAM needs. (I've always heard the "RAM should be 4-5x file size" rule of thumb, and I have to say that

1gb CS3 files run pretty leisurely on my current 4gb machine although there are other factors there too. But I

also don't know that current software can make use of more than 4gb and don't know how much other factors can

compensate for this.)

 

I'm thinking of buying from a semi-custom assembler (suggestions of specific vendors are welcome). For me at

least, that seems a good compromise between complete d-i-y (I'm willing to do whatever research is necessary, but

I'm not a "natural" with computer innards and thus want someone else to put the parts together) and buying a

one-size-fits-all package from Dell/HP/Sony etc.

 

Finally, I'm certainly open to a Mac - I've been an Apple user since the early 90s and have a 15" MacBook Pro

running Leopard as my travel computer - but only if a Mac solution is fiscally competitive

with a comparably performing PC. My overall workflow is split almost exactly 50/50 between Mac and Windows: Both

platforms are fine with me, I run CS3 on both with no problem, my parents and their senior-citizen friends

depend on me to be moderately conversant in both OS's, and I'll never go 100% to one or the other, so the choice

probably will come down to performance vs. cost. Fwiw, I'd prefer to NOT have to replace the computer when CS4

comes out. (No Mac/PC name-calling, please; we're all adults here.)

 

Budget is flexible, but I'd like to stay under $5K or so for the box (i.e., monitor and pointing devices additional).

[Stock-Photos]

The right RAID configuration will be the biggest factor on speed and your wait times. I use RAID 0 and it's super fast but the risk of disk failure over a single drive is doubled, a risk I'm willing to take.(I back -up frequently)

 

There are RAID configurations for performance that are mirrored AND striped. (more than 2 drives)

[kinell]

Some general thoughts:

 

Hard drive space is no issue here. Of course you need enough RAM to load a complete file into memory. Adding the

operating system and photoshop it seems you need at least 4 GB.

 

Now, what determines the speed of the process is not so much the amount of RAM (as long as you have enough to

handle the file), but its reaction time and throughput. Also, on startup the file has to be loaded from the hard

drive into the memory. The bottleneck here is clearly the hard drive, which is some orders of magnitude slower

than the memory.

In the end, it depends on what you want to optimize.

 

Conclusions:

 

- Once you have enough RAM to handle the file, more RAM will not increase speed

 

- To boost loading time, make sure to get a fast hard drive

 

- To speed up processing, get fast RAM (clock cycle measured in nanoseconds and throughput measured in Mhz)

[don_v.]

OK, that's helpful, esp. the RAID reminder and the part about "fast RAM"; one often hears about quantity of RAM but not speed.

 

A followup question: Are gaming computers a good starting point, or is one paying too much for things that are of no benefit to photographers? Whenever I check out the super-capable machines, from any manufacturer, it seems like the gaming PCs are always the most loaded. I would imagine that a lot of the components requiring speed for gamers would be useful with Photoshop as well?

[steve_dunn2]

<p>I have no idea what software you can use to edit files that size with reasonable performance. To date, no version of Photoshop runs at 64 bits, so Photoshop is limited to using less than 4 gigs of RAM (and in Windows, it's limited to either 2 or 3 gigs, depending on the version of Windows and how it's configured; I don't know about Macs). CS4 is supposed to be available in a 64-bit version, at least for Windows, which will be very useful for files such as yours; it would let you hold the whole file (including undo versions etc.) in RAM at once, provided you have enough RAM, and would avoid the performance hit of using swap and/or scratch disk.</p>

 

<p>As for gaming PCs, some of their features are good for Photoshop, and others not so much*. A high-end video card is wasted on Photoshop; the point of a high-end video card is stuff like 3D operations, which Photoshop doesn't use. Having tons of RAM on your graphics card isn't going to do much for Photoshop, either; two hypothetical graphics cards which are identical except that one has a gig of RAM while the other has 128 megs will likely show significant performance differences for gaming but not for Photoshop. But gaming PCs also usually have high-end multi-core CPUs, motherboards, and RAM, and all of those things will be useful.</p>

 

<p>*: graphics cards are, at their heart, extremely powerful CPUs, but they're not general-purpose CPUs; they're optimized for certain types of operations. <a href="http://www.tgdaily.com/content/view/37643/128/" target="_blank">Adobe has admitted that they're working on being able to offload some operations onto graphics cards to accelerate them, but will not state whether this will make it into CS4 or not</a>.</p>

[kelly_flanigan1]

Current Photoshop CS3 on a mac or Pc doesnt use even 4 gigs directly since its a 32 bit program, Thus you 3 gig image is like a ping pong table inside a VW bug that might be rotated by cutting it ie pieces at the corners; and reglueing the table back as one rotates it inside the car. Thats what happens when one doent have enough ram; it PUKES into the HDA. Thus the schemes of RAID; faster ram; faster RPM hds's; scratch disc placement are a kludge to reduce the pukeing time;when photoshop barfs into the scratch area. Your 3 gig file is like an elephant inside a Yugo that wants to turn around inside the car. You have to wait. Thus with CS4 on a PC with 64 bits one will be able to directly address more ram in photoshop; you elephant now can have a whole 2 car garage. A faster CPU, faster ram; a faster hda with the elephant in the YUGO to be rotated helps somewhat; but a pittance compared to having a bigger YUGO to work inside.

[kelly_flanigan1]

Here is a test with a 1 gig file on an obsolete Pentium III with 1 gig of ram; a 1 ghz cpu; with a 16 meg number nine graphics card; windows2000; only a 66 class udma controller; modern 7200 rpm 160 gig hda; cs2; Photoshop available ram 908 megs; slider set to 75 percent; max ram used by photoshop thus 681 megs; computer on the internet; 5 firefox windows open:<BR><BR>A 1 gig tif file was used; a 6.24 x9.983" 2400ppi image for the 1 gig image.<BR><BR><b>File size - Seconds to rotate 90 degrees</b><BR><BR>1 gig 256 seconds<br><br>713 megs 127 seconds<BR><BR>500.1 megs 101 seconds<BR><BR>350.1 megs 75 seconds<BR><BR>300.1 megs 20.7 seconds *<BR><BR>256.7megs 11.6seconds**<BR><BR>178.3 megs 3.3 seconds **<BR><BR>100.3 megs 1.8 seconds **<BR><BR>50.1 megs 0.9 seconds **<BR><BR>25.1megs 0.4 seconds **<BR><BR>12.5 megs 0.2 seconds **<BR><BR><BR>** CPU at 100 percent in task manager<BR>* CPU at 100 percent for most of the rotation; here the image is starting to puke into the 7200 rpm hda. <BR><BR>300 megs is the max before pukeing for this old box with 1 gig of ram; so figure about 900 megs for a modern box that CS3 can see 3 gigs of ram with photoshop. Since XP and Vista are more bloaded; the bog will happen at less than the 900 meg number. As one runs out of ram the time radically increases; one is venturing into the arena of the slow poke hda; radically slower than ram.<BR><BR><BR>Below the bog area on this example of 300 megs; the rotation time drops directly with cpu speed increases; thus a 3 Ghz cpu would cut these roughly to 1/3; thus the 100 meg file might rotate less than 1 second. <BR><BR>Above the bog area of 300 megs the bog is the hda thru put; the Pentium III CPU is loafing when the file is 1 gig; with the bog being most all the hda. Thus the knudge of raid; faster rpm discs helps in this bog region; but not much compared to real ram that photoshop can directly use.<BR><BR>For a 3 gig file one really needs a 64 bit OS and CS4 with say 8 + gigs of ram to make things move. <BR><BR>Here we have a dual core XP 64 bit unit with 8 gigs of ram; 8 is somewhat faster than 4 gigs; but not much. Photoshop CS3 and CS2 still only sees less than 4 gigs; say 2.5 to 3.3. The extra 4 gigs is sort of like a fast scratch area before the puking happens; but not so much as to be great,

[kelly_flanigan1]

Plot time versus file size and one has a "hockey stick" change in slope where one runs out of ram. The puck end of the stick has a slope of 50 to 60 megs per second; the mid and top of the handle is at 4 to 5 megs per second; and order of magnitude worse. More ram makes the puck end longer!

[william_john_smith1]

"On Macintosh computers, Photoshop can directly access up to approximately 3.5 GB of data. When there is more than

3.5 GB of document data, Photoshop writes data to its scratch files as necessary. On a computer with 4 GB or less of

RAM, the data is transferred directly between the scratch files on disk and the Photoshop RAM. On a computer with more

than 4 GB of RAM, Photoshop instructs the operating system to use the extra RAM as a buffer for the Photoshop scratch

file. In this case, when document data no longer fits in the 3.5 GB of Photoshop RAM and is written to the scratch file, the

operating system stores it in the extra RAM and can retrieve it faster than it could be read from disk. <B>This operation

allows Photoshop to take advantage of more than 4 GB of RAM to significantly increase performance with very large

documents."</B>

[kelly_flanigan1]

William; Your paragraph about Mac's using above 4 gigs of ram is what my 64bit XP systems with 8 gigs of ram do with a large giant file. The extra ram between 4 and 8 gigs on my units is used as a faster stratch area than the hda. Thats what my print shop uses with giant files; we have two PC's with this config. This scheme is not unique to the Mac; its what some of PC users do too. One of these units we tested both with 2, 3, 4 and 8 gigs of ram.<BR><BR>Having CS4 on a PC or CS5 on a Mac that can directly address 8 gigs instead of the hokey OS using the 4 to 8 ram area as a scratch will be alot better way. With all the whining on the net by Mac and PC users wanting Photoshop to be 64bit; the "significant increase" of the ram used above 4 gigs is optimistic at best.

[don_v.]

Kelly, after your first post (the one about the VW Bug and the Yugo and the elephant), I was under the impression

that we only need wait for Adobe to build a bigger 2-car, 64-bit "garage."

 

But your later posts seem to say that 64-bit Photoshop won't make handling multi-gig files significantly easier

or faster; improvement won't really come until we have an OS that can directly address 8 or more gigs. Is this

an expectation of the next Mac OS?

 

In other words, are you saying that everyone's whining about Adobe not catching up to the OS when in fact it's

the OS's that have to move ahead?

[kelly_flanigan1]

Don; the Mac OS is already 64 bit and can address more than 4 gigs. .<BR><BR>With a PC a version of Windows server added this I think with version 2003 ??; its in XP with the 64 bit version; in Vistas 64 bit version..<BR><BR> With a 32 bit OS one can directly address 4 gigs or less. With 64 bits this barrier is broken for the OS ONLY. .<BR><BR>Its the programs that are now the problem; they lag beyond the OS! .<BR><BR>Thus most all ones programs today run as 32 bit applications on a 64 bit OS; since most all programs are still 32 bits..<BR><BR> Its like when new houses were required in 1947 to be wired for 230 volts in the USA; and my dad was living in an older apartment wired only for 115 volts and had a 115 volt dryer. Or its like having digital TV broadcast now and you only have an old analog one; with no convertor box yet. .<BR><BR>A 64 bit OS can address alot of ram; way more than 4 gigs; like another universe., Also a building wired for 230 volts can power alot more stuff than an old WW2 one that might be still 115 volt service..<BR><BR> Just becuase one "upgrades" to a 64bit OS or a 230 volt house/barn doesnt mean ones programs and clothes dryers work better; they often work the same since they are still running as a 32bit program or a 115 volt device; like our old 1946 Westinghouse 115 volt dryer; it heated at about the same amount as a hair dryer today ie about 1400 watts..<BR><BR> There are already CPU's that work on 64 bits on Walmart value computers; there is already 64 bit OS's around in mac and PC too. Its the programs that are still stuck in a 32 bit world; like my dads 1946 westinghouse dryer was stuck in its 115 volt world. It has not been mentioned whether CS4 will run on a XP 64 bit OS; some of us fear they might do a lockout; like Adobe did with CS3 and Windows2000.<BR><BR>Its the current Adobe CS3 that makes the "garage" or "Yugo" be small; since its a 32 bit program. A 64 bit OS with alot of ram say 24 gigs is like a giant aircraft hanger; with the 32 bit Yugo or garage inside the hanger. Your giant 3 gig file/elephant is inside the YUGO trying to rotate; and is boxed in by the dinky space. The above 4 gig mac/pc 64 bit ram usage allows the elephant to put his bum and snout outside the YUGO; but alot of him is still inside mired in the 32 bit CS3 cramped space. Thus the scratch usage of using a 32bit CS3 with 8 gigs of ram occurs some int he 4 to 8 gig area; like the elephant "cheating" with part of him outside the YUGO during the rotation. Having the elephant rotate inside the hanger would be quicker.