When I stuffed those terabyte Seagate drives into the VM server I just knew that configuring for RAID would be a good idea. After all – that’s a healthy amount of storage and the likelihood that something would go tits-up eventually was too great. At first I simply used the RAID controller on the motherboard but I grew uncomfortable with the software drivers required. They were proprietary, of course. So I opted for a dedicated controller from Silicon Image.

Recently those decisions paid me back.

I never noticed anything amiss. The problem was revealed by a log entry noticed as part of regular maintenance. The RAID configuration console subsequently reported that one drive had become ill and gone offline. This array was configured as a simple mirror, so now the volume was flying without a net.

These are common drives and the local Best Buy has ’em in stock for under $90. Off I went, in horrible rain and cold, and soon I had the spare.

Swapping it in took almost as long as the short trip. The latest machine builds have been in mid- or full-tower cases but this server, nope, tight quarters in the mini-ATX case. But soon it was done. I restarted all the VMs and set the array to rebuilding.

It took a while – a couple of days. Yes, that’s excessive, I couldn’t agree more. But all things considered… It’s not a particularly powerful box; it’s busy, no less than a half-dozen VMs dedicated to their individual tasks, plus the host OS; the spare drive was added right out of the box – unpartitioned, unformatted. A sector-by-sector block-by-block reconstruction, so it’s no small wonder it took a while. I certainly could have helped it along.

Meanwhile, I’ve been running some tests on the failed drive. Yup, it’s failed alright. It’s also under warranty.

So just as soon as I scrub the data it’s going back to Seagate for replacement.

A happy ending, must be the Spirit of Christmas. There’s a lesson here, too. As we approach 2010, if you don’t already, resolve to apply some discipline to your data protection practices. Make maintenance and recovery plans. Use appropriate technology. Review your logs.

Happy New Year!

Added 1-Jan-2010…

Wouldn’t you know it? Following the secure deletion of all data from the failed disk I decided to run the Seagate diagnostics. It passed all tests.

I’ve repartitioned the thing and it’s formatting now. I’ll use it as scratch workspace for video and other big projects and with luck it’ll fail again soon. Something bothers me about using it for anything important.