Hitachi 1TB Hard Drive (and RAID discussion)

I'm swapping out Seagate and WD drives and moving stuff around so I'm having to do multiple SuperDuper operations. Mine probably won't finish until tomorrow. I've also got to replace the drive that's been slowly failing. I've got a plan...

I've decided that it's not a good idea to mount a file image over the network and doubly bad to do it over wireless. I've switched to sparse images, and that would probably help, but I still think my corruption problem is caused by network latencies and timeouts. It's pretty clear that my problems only start when I open a 20+GB file image over wireless. Takes forever too. :-(

I've tried the spotlight exclusion trick but I'll try it again. Sometimes I've told Spotlight to ignore a whole drive and it just ignores me instead. That's why I went to the unmount trick. If that works then I'll get rid of the disk images except for the encrypted one and leave my backup volume mounted. :-)

Thanks.

P.S. I can't imagine doing this on Windows. :-p

Craig

Try PGP. I use it for work files. I block off an entire folder with all my work in it. It's mandatory for me now.

OK, this may be a little off topic, but hopefully someone can answer this. I have an Iomega NAS200d in my network, which has two 160GB Hitachi drives, and would love to upgrade to larger drives like the 1TB Hitachi mentioned in this thread, but the manual says this is not possible. Is there a way to do this? Iomega will say no, since they want to sell me a larger NAS. Anyone have an idea what would happen if I try this? Any thoughts and suggestions would be appreciated, since the current drives are full, and I'm always having to delete old files, or move them to off network backups.

Thanks,

Vic

Texas diver,

The answer is likely "no", not easily anyway. The problem is that you would need a way to partition/format/install the Iomega system with the new hard drives, and I don't believe the Iomega unit supports this.

I have an Infrant produced unit that you can actually take 1 drive out, put in a bigger 1...wait for "RAID" rebuild to complete and then do the next drive. After replacing all of the drives you would have the larger capacity. Likewise, I could just take all of the drives out and install larger ones and setup the system from scratch.

The downfall of almost all small/entry NAS systems that any of us would consider is performance. The Infrant I have maxes out at 23MB/s on a good day, if you have parallel/concurrent writes the performance drops...and this is over GigE.

The home office/media center storage market is still not addressed. There are no affordable, reliable and decent performing NAS systems on the market. Its ridiculous, since any reasonable PC hardware can run Linux with software RAID and such to do it. I'm just too lazy, I want someone else to set it up for me. I want something that just works, which we are a Mac household (other than work required VMs with Windows). I will likely go the OpenFiler path, unless I can find a bargain on a StoreVault or some other entry business class NAS.

The other option would be to look at RAID based USB/FireWire enclosures. There are a few out there, prices vary greatly...most are RAID1 or RAID0 with 2 drives only, there are a few that offer RAID5 or RAID1+0 but these go up in price significantly.

The only thing about RAID0/1/5 is power surge. Power surge within your computer can fry all your HDs. SO it is still prudent to backup to DVD or an external RAID Array.

Um. did we lose everyone else? i'll bet most people are saying... "WHAAAAAAAAAAAAAAAAAAAAAA? I just want to take pictures!"

This RAID stuff is way over my head, and I suspect, way too complex for my needs.

I have a Mac Pro with two 500GB drives, each about half filled, and presently using the second drive for my Time Machine Backup. Critical files in my house are Aperture Vault, iPhoto Libraries, and some MYOB and Quicken files. Contacts, Calendar etc are backed up automatically to .Mac.

Any reason I cannot install a 1TB Ethernet or wireless drive and just target the Time Machine for all family computers in the household to this drive. Sounds simple and fairly reliable for my needs. Anyone disagree?

That sounds like a good idea Jeff. It could take a long time to backup over gigabit ethernet, so schedule your backups for night time :-)

It also couldn't hurt to store that backup at your office - i.e. in a different location from your main machine.

Cheers

James

This RAID stuff is way over my head, and I suspect, way too complex for my needs.

I have a Mac Pro with two 500GB drives, each about half filled, and presently using the second drive for my Time Machine Backup. Critical files in my house are Aperture Vault, iPhoto Libraries, and some MYOB and Quicken files. Contacts, Calendar etc are backed up automatically to .Mac.

Any reason I cannot install a 1TB Ethernet or wireless drive and just target the Time Machine for all family computers in the household to this drive. Sounds simple and fairly reliable for my needs. Anyone disagree?

Here is a great site to show how raid works in Pictures...

http://www.acnc.com/04_01_00.html

The most USEFUL RAID Array for photogs are RAID 0, RAID 1 or RAID 5. You can see RAID in action by choosing the # on top of the diagram and clicking the diagram to start the animation. Enjoy!

The purpose of RAID (ignoring RAID 0) is to reduce or eliminate downtime. It can also provide performance advantages, though no RAID configuration can outperform RAID 0. Keep in mind that RAID 0 is not actually a RAID level and offers no redundancy.

The problem is that people mistakenly believe that RAID can take the place of backups. It cannot. ChrisJ points out that there are problems that effect the data on multiple drives simultaneously and those are not limited to power spikes. The simplest examples are OS failures and user errors. There's a whole category of problems which RAID is vulnerable to. Being and ex-RAID developer I can tell a lot of stories.

I believe that most photographers, of which I count myself as one, do not need reduced downtime due to drive failure. Some do. For that reason, most photographers don't need RAID at all. All photographers need a true backup system.

My backup system consists of a separate set of disk drives residing on independent controllers and implemented using asyncronous duplication. That eliminates almost all possibility of corruption of both stores simultaneously. If my data were more critical I would use remote backups like James suggests and some here implement. I do not use RAID. I do use data striping (RAID 0). If file systems were smarter, RAID 0 wouldn't be attractive either.

Here is a great site to show how raid works in Pictures...

 

http://www.acnc.com/04_01_00.html

 

The most USEFUL RAID Array for photogs are RAID 0, RAID 1 or RAID 5. You can see RAID in action by choosing the # on top of the diagram and clicking the diagram to start the animation. Enjoy!

Very helpful; also explains why (as Craig says) I do not need it. I think Time Machine with it's ongoing incremental backups of my whole machine on a 'remote' hard drive, and in addition backups to my second drive on my MacPro of Aperture vault should work well resulting in 2 backups of my photographs on separate drives.

For those of you thinking of setting up fancy RAID setups, Time Machine may not work for you. On my Mac Pro, Time Machine freezes when it starts looking for volumes, and then crashes.

I have been using CrashPlan, which is excellent. It's like Time Machine, but it only works across the network (local network when you're at home, and remotely when you're away).

One good thing about RAID 0, is it speeds up I/O out of your Harddrive.

For Videophiles, it is great for encoding/editing/burning videos.

I know this thread seems to be concentrating on disk drives both JBOD and various flavors of RAID but...

I get the sense from reading tis that these drives are being used for archive as much as "backup".

It is important to have your data secured but the various flavors of RAID all have significant limits. My personal favorite is RAID 1+0

From wikipedia:

RAID 1+0: mirrored sets in a striped set (minimum 4 disks; even number of disks) provides fault tolerance and improved performance but increases complexity. The key difference from RAID 0+1 is that RAID 1+0 creates a striped set from a series of mirrored drives. In a failed disk situation RAID 1+0 performs better and is more fault tolerant than RAID 0+1. The array can sustain multiple drive losses as long as no two drives lost comprise a single pair of one mirror.

However off-site tape backup is just as important.

I may be preaching to the choir here, but what if your place burns to the ground? Are you sure your data safe?

Is anyone else out there using LTO tape drives? I've got a Quantum LTO2 drive that stores 200Gb native at up to 123 GB/hr. Not as giant as a 1TB disk, but the tapes are easily stored at the bank in a safe Deposit box...

A little history lesson regarding RAID 1, RAID 1+0, and RAID 0+1.

In the beginning there was the Patterson RAID paper. Prior to that, there was data striping and simple mirroring. There were no compound RAID levels. There was only RAID 1 through 5. All RAID levels implicitly included data striping. Without it, there could be no "redundant array of inexpensive disks" to replace a single, large, expensive one. RAID 1 encompassed RAID 0+1 and 1+0 from the beginning.

When the first RAID products were introduced, there were smart engineering teams and dumb ones. The dumb ones implemented simple mirroring, sometimes on top of data stripes and sometimes only in drive pairs. The smart teams (that would include my team) implemented redundancy underneath data striping because that was the right way to do it. We did RAID 1 right before doing it was cool.

When the dumb people realized that they were stupid and their competitors were smart, they didn't simply fix their own shoddy products (ever), they decided instead to "invent" new RAID levels. After all, one of them had already "invented" a non-standard RAID 6 that encompassed no more than heterogeneous hardware in a RAID 5 configuration. It was stylish to claim that you had invented new RAID levels, so it was no big deal to claim that RAID 0+1 and 1+0 were new levels because it allowed the idiots to claim that their product fixes were really new inventions. Never mind that Patterson's RAID 1 already encompassed all that... On the bright side, we got to add new RAID levels to our marketing sheet without changing anything in our product.

There is no RAID 1+0 and 0+1, there is only RAID 1. Any company who claims otherwise, and worse yet allows you to choose, you should run from. Sadly, all companies are forced to claim now that they support these ridiculous RAID levels. The ones that offer configurations that make absolutely no sense don't deserve your business. RAID implementations are surprisingly difficult to get right, so it's important to not use configurations more complicated than you need. Ruling out providers that indicate through their marketing that they don't understand RAID is the first step in that process.

Regarding TimeMachine, I purposefully haven't upgraded my main machine to Leopard because I suspected that would be true. We already knew that SuperDuper was broken and I didn't trust Apple to get TimeMachine right at first ship. Leopard is pretty much a fluff upgrade as it stands now. TimeMachine is the feature I want.

Mark, you can store disk drives in a safety deposit box too

Mark, you can store disk drives in a safety deposit box too

I do also, but most don't

Nice thing about removable mirrored drived is you can grab one, take it offsite and pop in another drive and rebuild the set.

I won't dispute your RAID information but if you want to run level 10 you better get the right controller card.

I would be interested to hear how many folks actually move their data off-site, how and how often.

I won't dispute your RAID information but if you want to run level 10 you better get the right controller card.

I agree. In my years of developing RAID products I wrote a lot of testing software and evaluated a lot of competitive products. Virtually none of them could complete one of our stress tests and none survived our data integrity testing. These weren't all no-name products either---they included Compaq and IBM among many others. As years went by after my change in career, my friends continued with RAID and told me repeatedly that things never got better. I don't trust RAID cards because I know how bad they are and how hard they are to get right. Now that they are commodities I trust them even less.

That said, RAID 1 is a whole lot easier to do than any parity scheme. Anyone who trusts their data to RAID 5 had better have a very well-reasoned justification for doing so. Disk space is just too cheap these days to continue avoiding mirroring and on-line backups. I fully support the use of RAID 1 when you need it. I groan when people recommend RAID 5.

The proper way to do what RAID provides is to integrate it into the filesystem. Novell did this in Netware 3 and it was impossible to outrun Netware with RAID instead of JBOD. I know because we tried really hard. ZFS is a step in the right direction and I'm pleased that Apple has moved to integrate it into OS X. Still, ZFS has made some disappointing restrictions.

RAID1 is da-bomb!

I recently did an "old school" Oracle implementation using 7 spindled mirrored.

This is the absolute best compromise in my opinion albeit at the expense of space.

Check out the Dell MD1000 enclosures. 15 drives to a max of 1Tb each.

This looks really nice. If it works as advertised with Time Machine then nothing could be simpler. I wonder if one will be able to add more drives?

http://www.apple.com/timecapsule/backup.html

This looks really nice. If it works as advertised with Time Machine then nothing could be simpler. I wonder if one will be able to add more drives?

http://www.apple.com/timecapsule/backup.html

If you check the tech specs page it appears the answer is yes:

"USB port for connecting a USB printer or USB external hard drive"

I have one on pre-order right now. I'm accepting that performance on the drive isn't going to be outrageous and it will be really just for backup, as right now my Time Machine drive isn't large enough to protect my internal drive and primary FW800 portable drive.

I have given up on performance on "home" NAS and can't justify the price of NAS that performs, so I am settling for simplicity.