Is RAID the way to go?

<p>I primarily shoot sports and events as a part time job, and over the past few years have developed a massive collection of RAW images, and I need to figure out the best way to either archive or protect them from loss. My current 1.5 TB hard drive is nearing capacity, so I was thinking of going with a RAID setup with two 3 TB drives for redundancy, but wanted to see what other people do.</p>

<p>I use Lightroom 5 currently for all my storage and processing, and as of January 1st I started creating new catalogs for all my new photographs and have been splitting my one huge catalog into smaller ones to make it easier to manage. </p>

<p>I've also read people recommending to back up the entire catalog to a BD-R, but generally my sports catalogs exceed 45 GB, so I am not sure about that one. I also like the ability of having everything at my fingertips.</p>

<p>Thoughts / recommendations are appreciated!</p>

<p>Whatever you use to backup the images is how and where you should backup the catalog. <br>

I'd go big drive as you suggest, put only images, your catalog and LR preset's and such on that one drive. Makes it real easy to clone to other drives too. </p>

<p>I got a commercial grade ReadyNAS from Newegg:<br>

http://www.newegg.com/Product/Product.aspx?Item=N82E16822122160</p>

<p>They're selling now for $2k, I think I paid $1500 for it last year. Very easy to set up and plenty expandable. <br>

I would go that way if I were you, plus a cloud backup.</p>

<p>Optical burning is a great idea. But I don't understand how can you have a 45gb catalog with only a 1.5TB hard drive of images?</p>

<p>I love my nas and suggest them all the time now. Although I'm only a half a year into using one, I wish I had started it sooner. I'd perhaps look at a 3 bay nas made by Qnap or Synology and then put 3 x 4TB western digital red drives in it and run them raid 5. Raid 5 with 3 x 3Tb will give you just under 6TB of space but also allow for the comforting ability that if one drive goes bad and dies, you just replace the bad drive and the nas rebuilds itself without losing any data.</p>

<p>Another thing to consider is MS Office 365. With their $9/mnth subscription, you get unlimited cloud storage in their OneDrive. I'm currently in the midst of switching to it from the popular Crashplan for cloud backup. I've confirmed with MS and they will take my 10TB of data.</p>

<p>I use SyncBack software to mirror/synchronize my drives. When I export from Lr to my computer, a mirror copy of the file is sent to the nas as well as MS OneDrive. That's really handy when I open other computers/tabs/phones and have my Lr work instantly available on them. The OneDrive app for Android is surprisingly nice as well.</p>

<p>A Class 2 or higher RAID (with redundancy) is essential for capacity and security. Do not even think of a RAID 0, which trades redundancy for speed.</p>

<p>A good alternative, one which I've adopted, is a DROBO drive. It has RAID-like redundancy, so the failure of one drive is easily recovered. Unlike a RAID, you can expand it or swap drives (cold) at any time, and it will automatically reconfigure itself to the change. DROBO drives are available with USB3 or NAS interfaces.</p>

<p>On the road, I use one or more (always a backup) 2TB WD Passport drives, which are powered from an USB3 cable. I have found it's cheaper and safer to simply buy a new drive when one is full, rather than erase and recycle.</p>

<p>I use about 8 TB a year for audio/video production, plus long-term storage. With 8 or more drives on line at any time, I experience about one drive failure a year. My practice is to back up each project on Blu-Ray discs as I go. That has saved my bacon on several occasions.</p>

<blockquote>

<p>I primarily shoot sports and events as a part time job, and over the past few years have developed a massive collection of RAW images, and I need to figure out the best way to either archive or protect them from loss.</p>

</blockquote>

<p>It's an obvious question: but are you required to keep all these images ad infinitum? You could delete pics after a set period (5, 7 or 10 years). Otherwise you are paying to keep images of zero value to you. I suppose if these are legendary players then some may be worth keeping, but if they are just high school/college teams, why is it necessary to treasure them? Of course you still need to back up images that have not expired. I don't keep images of plays/dance/teams for more than 2 years myself.</p>

Hardware RAID 1 with two disks is the only RAID worth doing with a home setup.

 

It is a safe file system and is supported by most motherboard chipsets.

 

When one drive fails, it gets replaced and the RAID volume is rebuilt from the other drive's contents.

 

A rebuild takes a while (hours or days depending on drive size and your hardware), but you are back to a safe condition.

 

Read times are better than a single drive because of parallelism, content can be pulled from either drive in array.

 

Write times slower because of needing to write to both drives in the array.

 

Sure there are plenty of other flavors of RAID, but you are not running a data center that needs 24/7 up time. And you are limited by the number hard drive ports on the motherboard.

<p>the intent of raid, is to doublke saved files so the chance of data loss is reduced or minmimumized</p>

<p> some raid setups do this.<br>

it er configuraqtions copy saved data to two hard drves simultaneousdly.<br>

However in some instancves, if onbe of the two drioves fail the data cannot be recovered from the remaining drive.<br>

a safer method is to periodically back up the data to another drive that can be unplugged anbd safely stored.</p>

<p>cd/dvd backup has another set of problerms.<br>

data on a cd/dvd is usually safe for several yeaqrs.<br>

but some of these disks will evenbtually fade and either have gaps or become unreadable.<br>

do not trust or depend on a well known branbd-any brand.<br>

it is the actual manufacturer of these disks that determines the useful life of a cd/dvd media disk.<br>

cddvd date shuld be renewed = copied and newer generations of disks copied beore the first generaTION FADES WILL WORK.<br>

i HAVE NOTED THAT SOME DIDKS SEEM TO RETAIN DFATA AND OTHER BRAND NAME DISLKS DO NOT.<br>

some older data is sec ure, some newer data is not.</p>

<p>ll digital media is somewhat insecure.<br>

I saw a pictuire or President lincolns inaguration. still secure after more than a centuiry and a half.</p>

<p>properly stored negaivs and prints land lifetimes.<br>

i am suire there will eb correcponding imprvements in the life of digital information.<br>

the best answer I can give with raid is perhaps.</p>

<p>You might have a problem: You collected 1.5GB during<em> a couple of years</em>.<br>

If you set up a RAID (5?) now, with 4 3TB disks of which one serves as backup, your 1st disk might die before you cluttered the 9TB you have. <br>

I suppose a RAID makes sense if you 'll have it overflowing after 2 years? - Stuff another with further data if or once you have that much. Also figure out how to be sure you'll be able to access data on RAID disks in case the disks outlive their controler.<br>

I have no clue / opinion about "just mirroring" your data drive. - yes, I did that before myself, but that machine became obsolete in between. - The issue I currently have with such a setup: The redundancy drive drains energy all the time and if ransomware (or whatever else Murphy's Law is hiding in it's sleeves) gets hold of my machine the backup drive is lost too. A mirrored drive adds a bit of convenience but doesn't substitute other kinds of backup. - Is it worth it or paranoid to have 2 external backups + 2 internal copies? - Only you can tell. <br>

I decided to be too cheap for RAIDs without a space benefit - i.e. at least 3 disks involved.</p>

I agree with Richard Hermann: RAID 1 with two disks (mirroring) is the sensible solution.<br>Put two 4TB disks in it, and you're good for a while. And if a two disk RAID starts filling up, it's not an occassion to regret not having started with a bigger RAID, but instead not bad at all to then just add another, separate two disk RAID. If one RAID fails catastrophically, and you don't have a usable backup (see later), you will not have lost all your data. And whether you have one NAS, two, or seventeen, they are all equally accessible over your (home) network.<br><br>However (!) a RAID is not a substitue for backups. You do still need more, unattached disks for backups plus a regular backup regime.

<blockquote>

<p>However (!) a RAID is not a substitue for backups. You do still need more, unattached disks for backups plus a regular backup regime.</p>

</blockquote>

<p>And that is worth repeating!</p>

<p>RAID1 is a viable option in a larger strategy which ideally includes off-site backups. One of the main things to be careful about with any type of RAID is how easy it is to get data from one of its disks when it is connected to a completely different controller. I do not know for recent NAS, but in the past, this was a true headache. And since anything can break, I just always want to make sure that my hard disks are not formatted in such a way that they are "married" to a specific controller (which may no longer be in the market when the problems occur). It's basically what keeps me with syncing tools for now, but as said, I've got little experience with recent NAS systems so it could be a thing of the past.</p>

Redundant Array of Inexpensive Disks (RAID) comes in many flavors. The RAID number has nothing to do with how many drives it needs to create.

 

The array itself can be a safe file system RAID 1, or unsafe RAID 0 which is fast but doubles the apparent capacity of the underlying drives.

 

These are the simplest types only requiring two physical drives.

 

The higher RAID types are more elaborate that require a minimum of three or more physical drives to create.

 

RAID 10 or RAID 01 are built combining a hierarchy of drives and their underlying data layout (stripe set versus mirroring) into the single RAID volume.

 

Mainstream motherboards don't support higher level RAID volumes in hardware.

 

RAID in software is a waste of your time, don't do it.

Good advice regarding the underlying drives being tied to the controller hardware.

 

Say you bought a new motherboard with different chipsets

 

Consider moving from a Marvell to the Intel chipset with your RAID drives. They likely will not just plug in, tell BIOS the RAID type, and just work.

 

If however you stay within a chipset family, the underlying drives should just mount. They try not to break you with their upgrades.

 

I've been lucky because I have used the Intel RAID contained in most if not all MB's. I've moved my RAID 1 volume through a few MB upgrades. YMMV

<blockquote>

<p>A good alternative, one which I've adopted, is a DROBO drive. </p>

</blockquote>

<p><br />A drive from a DROBO can't be read on any other system. DROBO fails, you are stuck. Maybe you can mail your drives to the company. If they go out of business and DROBO fails, kiss all your files goodbye.</p>

<p>You may be interested in this article:</p>

<p>http://www.smallnetbuilder.com/nas/nas-basics/32626-how-to-buy-a-nas-2015-edition</p>

<p>As the article states and as both Q.G. and Wouter have written, RAID is <strong>not</strong> a substitute for good backups - note the plural, more than one backup.</p>

One final note if you go with a hardware RAID supported by the MB.

 

Any action that resets the BIOS back to factory settings, like updating the BIOS or pressing the reset button/switch/jumper/remove battery, will likely unhook your RAID volumes.

 

Just make a note to check drive mode and settings before booting in the OS after a crash or upgrade.

 

Usually switching from the default drive mode (like IDE or AHCI) to RAID is all that is needed.

I don't think RAID inside a PC is the way to do this. Not even if all the PC does is act as a RAID storage device.<br>A (dedicated) NAS will have it's own system that will not be disturbed by anything you do on your PC (like updating the OS). A NAS too will get soft- and firmware updates, but ones provided by the NAS's producer, which are aimed at nothing else but making it work better and safer as the thing you use it for. No risk of updates meant to supercharge your PC messing up your RAID then.<br>So my advice: RAID 1, two disks 2TB or 4TB each (for a total of 2 or 4TB storage), in a two bay NAS. And two external USB 2TB or 4TB (depending on what you put in the NAS) disks for a backup rotation.

Network Attached Storage is OK for an external secondary backup. Offsite in the cloud is even better in case of disaster. Networked solutions are painfully slow compared to a hardware solution.

 

If I was in the market for an external storage solution, I'd get a Thunderbolt or a USB 3.1 based RAID enclosure. These are the bee's knees in current tech for bandwidth.

 

RAID 1 on main machine, some local external safe storage solution, and finally the cloud for tripled protection.

 

None of this helps if you don't plan and execute a sound backup strategy.

Agree with QG above. I wouldn't put raid 1 (or raid 5) inside a computer.

 

I also feel a two bay nas is pointless and you may as well just stick with external drives in

enclosures. Raid 5 in a three bay nas makes more sense to me although it is a lot slower than raid 1

or raid 10. But you don't use a nas for speed. The differences in raid 1 or 5 should be researched

before jumping in though.

 

http://www.tomshardware.com/forum/244377-32-raid-raid-raid

 

I don't like Drobo or any of the smaller nas companies for the reasons Jeff mentions. Also with nas,

most run a variant of Unix/Linux and is another safety net.

Using a laptop as a main workstation, external local storage makes perfect sense. Its the only option

 

Consider a tower case with gobs of space for drives combined with a modern MB with drive ports to spare. The recent X99 boards come with ten SATA ports, some with even more if you look at the high end WS grade offerings.

 

Why not have automatic near real time safety with a hardware RAID 1 volume?

Considering the setting (we are not running a large company in which huge masses of data are shuffled back and forth daily across a large network, that can't tolerate a 10 seconds interruption due to a drive failing) anything more than RAID 1 is over doing things. Not needed. Too expensive.<br>RAID 1 gives an instant backup of dynamic data, so all data is immediately safe, not lost when a drive fails. (If there is a identical copy, there is no need to reconstruct the lost copy doing calculations. Besides, modern drives already have error correction routines baked in). Data can get lost in other ways in a dynamic work flow, which is why backups - left untouched for the duration of the backup rotation - are also needed.

The drawback with using a regular computer with mainboard supported RAID and loads of channels and drive bays is that it is run by an OS that is not tailored to what the machine is going to be used for, and the risk that the OS and its constant upgrades to enhance all sorts of performance issues totally irrelevant for what the machine is used for will interfere is present. Dedicated NAS boxes are much safer.

An OS does not know and does not care about the underlying workings of a hardware RAID volume.

 

An OS sees a volume to read/write and nothing else.

 

The OS drivers working in tandem with the RAID hardware handle all the actual grunt work.

 

A RAID 1 volume on a work machine is the first layer of defense against catastrophic failure.

 

second layer of defense, on-site backups via your chosen method: NAS, other internal drives, external HDs, optical media, physical film archives, whatever floats your boat.

 

Finally the third layer: offsite repository like a safe deposit box, converted salt mine, or the cloud for the chance of fire/theft/meteor strike.

Not everyone can drop $2K on a robust NAS solution which serves your needs well.

 

How many of us have a few extra drives/cables lying around, unused ports on the MB, room in the computer case, and the MB manual to show the steps?

 

All is needed is a little push towards taking the first steps towards safer data with a RAID 1 volume.

 

Are you willing to loose whatever changes occurred since the last backup if the work drive goes away?

 

Layers of defense

We mustn't exagerate neither problem nor the solution. We do not need an autosafe button that activates the moment the program we are using detects a mouse move or keyboard touch. We know we can lose the very latest thing we did since the last time we saved our working file if, say, the power supply would fail. Annoying, but rarely a great, unrecoverable loss.<br>Hence we also mustn't exagerate the solution we need to keep a rather comfortable level of data safety. A robust NAS, RAID 1, two bays, that does the job well does not cost multiple thousands of dollars. And together with a good backup routine provides all the layers of defense ("defense"?) needed.<br>And i you'd ask me, a RAID running on your work machine, balancing the need for a good enough work machine against the need for fail safe storage, is asking for troubles for the reasons given earlier.<br>Keep it simple. Keep it easy. Keep it cheap. Keep it safe. It really works. ;-)