Need External Hard Drive

This week I transferred from Lightroom CC, which included 1TB of storage space to Lightroom Classic, without storage. I like the Classic edition much better for many reasons, but I now have some 60 days to offload my approx. 30gb photo load from their server.

I've looked at several methods, but first had to accept that an external hard drive is just that, storage, not a plugged in box to load the wonders of today's outing from an SD card for later RAW format editing, since they've become jpegs upon transferring and essentially worthless except for minor adjustments.

So am I correct that any RAW editing one wants to do must be done immediately in LR Classic, before transfer to the hard drive? As I understand if transferred correctly, not just from the Lightroom server, but those from daily editing, the prior edits are saved and can be seen easily from the external hard drive?

Thanks!

I think you're confused. There is no reason that you can't copy raw files to an external hard drive and store them there. It's of no relevance whether the hard drive you use is internal or external, except that internal will generally be faster. You can store your raw files wherever you want and just tell Lightroom classic where they are.

I have all of my raw files mirrored, with one copy on an external drive and the other on an internal drive. I tell LR to look at the internal drive because it's faster.

Normally, I use a card reader and simply copy my raw files to the location where I want LR to find them. Because of speed, I use my internal drive, but I then immediately copy them to an external drive.

Nothing in this process converts files to JPEG. In fact, I have virtually no JPEGs on either of my drives. I create JPEGs when I need, them, primarily to post to the web, but I don't bother storing them since LR can re-create them whenever I need them.

There's no doubt confusion plays a large role here. As an experiment, I have a recently erased/reformatted SSD hard drive, from a dead Mac in a USB enclosure connected to my current Mac.

I exported a RAW photo from Lightroom directly to the SSD drive, open it and it reads jpg. Is there something I should be doing differently when exporting from Lightroom?

There's no doubt confusion plays a large role here. As an experiment, I have a recently erased/reformatted SSD hard drive, from a dead Mac in a USB enclosure connected to my current Mac.

 

I exported a RAW photo from Lightroom directly to the SSD drive, open it and it reads jpg. Is there something I should be doing differently when exporting from Lightroom?

Check the export presets. I bet it's set to export as Jpeg. If I were better at this stuff, I could attach a screen shot.

Look at the export window. In the file settings, there's a drop down menu for image format (on a PC, anyway).

1 TB portable USB HDs are cheap, and I use them to back of specific files and groups of files from my 6TB and 5TB (backup) solid state drives.

Unless the import/export settings are set "funny" there should be no conversion to different formats. Especially not from mere copying.

At the least convenient, you can just drag the files/folders from one 'drive' to another. You don't have to load them into any program

You should be keeping RAW files where you have them. They are the 'real pictures' rather than the jpgs, etc.

Check the export presets. I bet it's set to export as Jpeg. If I were better at this stuff, I could attach a screen shot.

Look at the export window. In the file settings, there's a drop down menu for image format (on a PC, anyway).

I bet you're right, but I'm getting older just trying to see where to tick the controlling box! Haven't found it yet, maybe dinner and a drink will help - Thanks!

1 TB portable USB HDs are cheap, and I use them to back of specific files and groups of files from my 6TB and 5TB (backup) solid state drives.

 

Unless the import/export settings are set "funny" there should be no conversion to different formats. Especially not from mere copying.

At the least convenient, you can just drag the files/folders from one 'drive' to another. You don't have to load them into any program

 

You should be keeping RAW files where you have them. They are the 'real pictures' rather than the jpgs, etc.

I've clicked on 'reset all settings' to make certain an inadvertent click didn't happen. I'll try some 'dragging' tomorrow and check results. The RAW photos I have are on the Adobe Creative Cloud server, a function of the previous LR CC I'd been using, some on Apple Photos also of course.

And yes, for the scant files I have and will likely generate in the future might mean a USB thumb drive makes sense - Thanks

Adobe Creative Cloud server

I don't think I am clinically paranoid, but I don't trust remote, connected sites as much as I do the machines sitting on my own table.

When you use "cloud" storage, I always suspect that someday I would wake up and find my files "gone with the wind".

Also, only Hermes (the god of electronics) knows what these companies will do with, to, and for your files.

So am I correct that any RAW editing one wants to do must be done immediately in LR Classic, before transfer to the hard drive?

Absolutely not!

Your workflow should be to safely transfer the RAW files directly from SD card to external HDD (and preferrably to a secondary backup) before Lightroom even gets a sniff of them.

Then you can work on the files in lightroom or whatever editor; safe in the knowledge that your camera-original files can remain intact and untouched.

Absolutely not!

Your workflow should be to safely transfer the RAW files directly from SD card to external HDD (and preferrably to a secondary backup) before Lightroom even gets a sniff of them.

 

Then you can work on the files in lightroom or whatever editor; safe in the knowledge that your camera-original files can remain intact and untouched.

Excellent! As clarifying as a fine Port. This really simplifies it all. Thanks RJ _1

Sorry, I didn't mean to sound snide.

I exported a RAW photo from Lightroom directly to the SSD drive,

Ah. I think that's the simple answer.

Exporting isn't copying. Exporting is Lightroom terminology for creating a new file, kind of like "save as". Simply copy your files from one drive to another outside of lightroom and then import them. Alternatively, if you have imported files from one drive, you can move them to another within Lightroom.

I don't think I am clinically paranoid, but I don't trust remote, connected sites as much as I do the machines sitting on my own table.

 

When you use "cloud" storage, I always suspect that someday I would wake up and find my files "gone with the wind".

 

Also, only Hermes (the god of electronics) knows what these companies will do with, to, and for your files.

Likely not clinically paranoid, but ...

I generally find that issues such as the OP is having are user driven. As we’ve seen in the progression of this thread, a little education goes a long way in helping one properly deal with software and the cloud. The cloud is likely as safe as the vaccine. Paranoia doesn’t have to be clinical in order to be problematic.

That being said, there’s good reason, other than paranoia about cloud gremlins converting raw files to jpgs, to have both a cloud and an external backup.

Sorry, I didn't mean to sound snide.

 

 

 

Ah. I think that's the simple answer.

 

Exporting isn't copying. Exporting is Lightroom terminology for creating a new file, kind of like "save as". Simply copy your files from one drive to another outside of lightroom and then import them. Alternatively, if you have imported files from one drive, you can move them to another within Lightroom.

Not at all paddler4, didn't take it that way a bit. Thanks for this; much clearer now, learning Adobe's language is certainly a big part of using their programs.

Thanks -

learning Adobe's language is certainly a big part of using their programs.

Absolutely. It doesn't help that some of the language is inconsistent from one Adobe app to the next. In fact, even though Adobe Camera Raw and Lightroom are almost identical functionally, some of the functions are labeled differently in the two programs. In time, it becomes clearer.

The key concept in this thread is the difference between "exporting" from Lightroom, which always creates new files, and copying or synchronizing files and directories, which preserves the existing data structure. Copying is an OS function, often implemented with copy and paste operations. Synchronization is a software driven process which uses OS functions to move or duplicate large amounts of data without limits imposed on copy (or cut) and paste. Synchronization software can do much more than copy. It can flag "duplicate" files for replacement or renaming, which can cause problems if you don't know what you're doing. For what the OP is doing, it's best to copy to a clean drive or directory.

I have been using a file sync program called "File Synchronization" for several years. Based on my computer-smart kids, I'm trying another program called "BeyondCompare." Both have a simple interface and a wide variety of options. Even with fast drives and processing, it can take several hours to move one TB of data.

I organize my images based on rules for a relational data base, first expounded by and IBM employee, Cobb, in 1970 (then largely ignored by IBM). In simple terms, a relational data base is a grid or rows and columns. Rows are called "records" and the columns hold data related to that record. Each record is unique from all the other records, identified by a key, which can be one of the data columns, or an artificial "hash" code. Each record must have the same data structure (but not all columns must be filled). Records are accessed using "queries", which specify which data you wish to retrieve. The underlying principle is "Write Once, Read Many". This expands on what others, particularly samstevens described above.

• My images are organized by directories named with a sortable date code (yymmdd) and a brief descriptor of the subject or event
  
• Inside each director are sub-directories, such as RAW, TIFF, JPEG and (naturally) PNET (sized to 1000 pixels on the long side).
  
• There is no need to rename files. The directory path and file name constitute a unique name or key. (I'm not creative enough to name 200,000 images without duplication).
  
• As suggested by the directory structure above, my "exported" derivative files are stored under the "key" directory, usually without renaming.
   
  
• Each file is stored only once (but may be duplicated, with the data structure, for archival purposes).
  
• Images may be associated by Lightroom into "Collections", which are by reference only, not copies. In database terms, a Collection is a structured query, which is more or less permanent for convenience. If the original file is altered, the new version automatically appears in the Collection.
  

I am very rigid about storing RAW files in key directory, but I'm flexible about the number and names of subdirectories, and not all records have the same number and names. Simple names like TIFF and JPEG are full-sized files. PNET is special because I use this file size a lot. If a client doesn't need full-sized images, I may export a set of 8"x12" @ 300 ppi image for delivery. (Not everyone has 30+ TB of storage).

In order to preserve this structure and location, I always save the RAW files first, then instruct Lightroom to find and keep then in their original locations. The few times I've let Lightroom import and organize images, it is consistently wrong by my standards.

The key concept in this thread is the difference between "exporting" from Lightroom, which always creates new files, and copying or synchronizing files and directories, which preserves the existing data structure.

+1. Exactly.

The word 'Export' should always be treated with suspicion in an image-editing context. It usually implies a change in file type, colour space, bit depth or some such alteration. Occasionally all three.

However, the 'Export' or 'Export As...' option is usually followed by a dialogue box that informs you exactly how the file is going to be exported. It's unwise to just click through such a dialogue without checking what option(s) is/are selected by default. Not until you're sure that's what you want to happen.

Can't offer any suggestions on Adobe stuff (Linux user), but if the external HDD is for working or short term storage, do yourself a favour and use a SSD in an enclosure rather than 'spinning rust'.

For longer term storage, classic HDD are far cheaper and you don't care so much about the performance.

As to using magnetic storage for archival purposes, I'm not going to make any statement on that other than that the topic of long term data backup is complex, full of personal opinions, light on experience and has no good single answer.

Laser engraving on to platinum, perhaps, as a millennial option?

For long term storage, redundant storage like a RAID 2 (or higher) is essential. The magnetic tracks on a disk tend to weaken and bleed if the disk is idle. When running, these tracks are continually refreshed. I have nearly 50 hard drives I keep for archival purposes, up to 10 years old. So far only one has been unreadable due to mechanical issues. If they eventually fade with age, it probably doesn't matter.

Instead of RAIDs, I use DROBO units with 5 drive slots. One drive can fail without data loss, and can be hot-replaced and automatically regenerated without shutting down and re-striping. I use SSDs for speed (and shock-proof operation), realizing that they have a limited number of read/write cycles in their lifetime. A network drive could be located remotely and synchronized via the internet, for optimum safety in the event of a disaster (but risking hacking).

Thanks Ed for the post on this, but it'll be Monday before I get a chance to study it -

I'm somewhat anal about data structures, but structure makes it easier to back up, and locate particular images or groups of images. I've used a number of data base systems, including Oracle, which have powerful query tools. That proved to involve too much programming for my needs. In the end, I organize images manually, one download at a time.

If you want to test the mettle of a database manager, suggest a change to the basic data structure (fields) within the database. The response falls between a simple "We don't do it that way," to a string of expletives. On several occasions in my career, the client has responded by starting a new IP department willing to do it "our way."

Correct me if I'm wrong.

You're wrong.

Time has nothing to do with that. The number of times a JPEG (or any format that involves lossy compression) is opened, altered and saved again does.

RAW converted to TIFF may be RAW or any other format (even JPG). The arrangement of bits and what those bits represent has nothing to do with it either.

As far as storage is concerned, the cheapest and most stable way to get a lot of storage is on spinning hard drives. They last a long time, are cheap, and available in many sizes. You can carry them with you. Have them fixed in your computer. Put them in network attached storage devices that are or are not remotely accessible. Have as many duplicates as you want.

CDs and DVDs "rot" sooner than hard drives fail. Solid State Drives are expensive and have a rather limited (in comparison) number of times they can be written to before they fail. Tape is worse than all of them, being slow and not very durable. What else is there?

"You're wrong.

Time has nothing to do with that. The number of times a JPEG (or any format that involves lossy compression) is opened, altered and saved again does.

RAW converted to TIFF may be RAW or any other format (even JPG). The arrangement of bits and what those bits represent has nothing to do with it either" Q.G

Interesting, but.

A thought.

My understanding is, TIFF images are difficult to alter making them ideal for protecting information and archiving., TIFF is a clean file format which contains no links or hidden data and therefore can never have a virus embedded within it.

"TIFF is what’s known as a loss-less format, what you see is what you get. Formats like JPEG are compressed to create smaller file sizes which reduces file quality. While larger in size, saving files in this format ensures that images will produce and maintain the highest quality for future use"

Loss loss format

Maintain the highest quality for future use.