Last week, I swapped a new set of four 2-terabyte drives into one of the Drobo arrays. I formatted the drive as an 8 GB single partition, created the top level directories, and set Vice Versa to work backing up the server on line files. After about two days, all the data had been transferred, but an anomaly had developed. The Drobo dashboard showed that 800 GB of storage have been used, while Windows Explorer said that 2.4 terabytes had been used. I decided to ignore the discrepancy, figuring that it was just a bit of unimportant weirdness.
Today, after a server reboot occasioned by operating system updates, the Drobo dashboard indicated a drive failure in the array with the new disks. I pulled out the drive with the red light next to it and installed a fresh 2 terabyte drive in that slot. After 15 or 20 seconds, a green light came on next to the slot. Then I looked at the Drobo dashboard. It indicated that all drives were ready, and said that I could add capacity at any time by replacing a drive with a drive of larger capacity. This gave me some pause, because there had been nowhere near enough time for the Drobo to rebuild the array.
I talked to the Data Robotics support people about the issue. When I went to the Drobo dashboard, I notice that my support agreement had expired. I went to the website to renew it, and after taking all my credit card and other information, Drobo gave me a screen that said “DroboCare Out of Warranty is no longer being sold”.
That left me somewhat confused. I called Drobo tech support. The tech helped me without asking about the service agreement. At first he said this immediate green light business was normal operation, and that the Drobo will have sufficient redundancy even without the new disk. I told him that, even though there was little enough data for the Drobo to achieve redundancy with only three disks, it wouldn’t have had a chance to do that after the disk failure, since I replace the failed disk with a new one only a few minutes after the failure.
Then he said the thing to do was to shut down the computer, power down the Drobo, reboot the computer, and power up the Drobo. I did that, and the rebuild did not start. The tech and said after doing the power cycle I should probably update the Drobo firmware. I tried to do that, and got a message that said: “The Drobo firmware update was unsuccessful. Please contact Drobo customer support.”
So I did. The (new) tech said that, although the automatic update procedure built into the Drobo dashboard updates the device firmware before it updates the dashboard itself, the dashboard needed to be updated before the firmware update would take. I temporarily disabled all the Vice Versa scripts. Under the direction of the tech, I put the Drobos in standby, unplugged the USB cables, unplugged power cables, uninstalled the Drobo dashboard, downloaded the latest version, installed it, plugged the USB cables back in, and plugged the power cables back in. Then I let the dashboard make another attempt at upgrading the firmware.
The update appeared to go OK until it was time to reboot the Drobo. The dashboard was unable to shut it down, and told me to shut down the operating system, power cycle the unit, and restart the operating system. I did so, but in a slightly different order and recommended by the dashboard. I use the order recommended by the tech previously: power down the computer, unplug the AC cable, reboot the computer, and plug in the AC cable.
The system came back up with the right firmware installed, and still no indication of a rebuild taking place on the Drobo with the recently-replaced drive.
This is a mystery to me. I can imagine how it is possible, when the Drobo has data occupying less than half of its capacity, for the disk controller to construct two separate mirrors, rather than using RAID 5. It could also construct a RAID 6 array, and scale it back to a RAID 5 upon failure of one of the disks. Both of these are pretty sophisticated strategies, and don’t seem consistent with the overall approach of the basic Drobo boxes. The Data Robotics web site isn’t forthcoming with what’s going on behind the curtain, and the techs, while genuinely trying to be helpful, don’t seem to know.
I’ve spent a great deal of the day messing with computers instead of making photographs. Time to get back to that.
Brooks – Very sorry to read about your experience, but glad to hear there was a happy ending and all your data is safe and protected (which is really what Drobo is all about). I’m not sure what the issue might have been regarding the capacity discrepancy of Drobo Dashboard and Windows Explorer, but maybe I can shed some light on what appeared to be an unrealistically fast array rebuild.
A lot of folks don’t realize that BeyondRAID is “data aware” and knows how much capacity on the drives actually contain data. This is what enables Drobo to know how much capacity is being used and communicate via the blue LED capacity indicators and also notify (via yellow or red drive LED) when a new, larger drive is required.
This data awareness also allows Drobo to detect large data transfers (like you transferred the initial 2.4TB of data), analyze how much capacity it has available, and use more efficient mirroring techniques (as opposed to striping with parity) to give you the best performance possible during the transfer. Once the data is on the drives, Drobo will periodically re-layout mirrored data in a striping with parity scheme as this provides the best combination of redundancy and usable capacity (assuming the system has more than two drives which is the case for your system).
What I think occurred in this instance is that Drobo had a majority of the data on three of the drives since there was plenty of capacity available, and it just so happened that the fourth drive (which contained little-to-no data) is the one that failed before the Drobo needed to re-layout any data to free up space. Hence the almost instant rebuild that occurred.
Sorry again about the issue, but hopefully this gives you some insight into how your experience – while not possible with a traditional RAID array – could be expected with a Drobo and BeyondRAID.
Regards,
Jim Sherhart
Data Robotics
Thanks, Jim. That makes me feel better. BTW, in the dim, distant past, I did some work on RAID algorithms. Check out this paper: http://portal.acm.org/citation.cfm?id=163521
Jim
Hi, I just 3TB on my Drobo 4-bay and it show a red light on my drive, how do I firmware update on my Drobo. Thank you
This is the kind of thing with which the support part of the Drobo website is really good (how’s that for a Churchillian construction?). You don’t specify which Drobo you’re using, and how many drives you replaced, and why you think a firmware update will fix things, so you don’t give me a lot to go on.
Most firmware updates can be accomplished from the dashboard.
Jim
Hello, I wanted to share my recent Drobo experience so other users could find more help like I did.
First, I have had a 4 Bay Drobo Gen 1 for a few years and use Mac OS X Yosemite. I bought it used on eBay. I loaded it full of 4 – 2TB drives and used it with Carbon Copy Cloner to make regular copies of all my drives. I never had a problem till recently when it started to unmount sporadically. Sure enough, one the drives finally went bad. One of the drive bay lights was blinking red like it should. Here’s what I did.
1) Be very patient and don’t give up.
It all looked pretty bleak the night before I got it all to work.
2) Update the Drobo Dashboard app.
I had to do this twice to get to the latest version. This will allow you to get to step 3.
3) Update the firmware.
I was on the oldest version. V 1.3.8 was the version that worked for my Gen 1. The Dashboard is what allows you to update the firmware under Tools. Follow the troubleshooting tips on one the links below. I had to do some quirky Mac OS stuff but it all worked after a few tries. Again, be patient and stay positive.
4) If you’re still unsuccessful, try replacing the bad drive with your new one. That’s what I did sort of by accident, but everything worked out eventually.
Below are some the pages that were helpful.
http://dashboardhelp.drobo.com/guide/220/en/ALL_Manually_Updating_Firmware_from_the_Website.htm
http://support.drobo.com/app/answers/detail/a_id/110/~/how-do-i-automatically-or-manually-update-my-firmware-to-a-new-version%3F-can-i
http://support.drobo.com/app/answers/detail/a_id/31/~/what-are-the-latest-firmware-and-software-releases-and-revisions-for-drobo
I updated the dashboard, and then updated the firmware on my Drobo 5N. It’s loaded with 3x4T+1x2T, single-failure redundancy.
The Drobo indicated failed drive as soon as the firmware update completed; seemed odd timing.
So I bought a new disk, unplugged the old one, and installed the new one. The Drobo went to yellow/green flash on all disks, and the dashboard indicated that it was rebuilding the redundant array. Eventually all lights were green, but it indicated 20% used (should be 40%), so I went to check out what I had. The dashboard said “unable to mount”, and the Drobo was not visible as a network device. Was it unable to mount one of it’s drives, or was it unable to mount itself as a network disk? Hmmmm..
I selected Drobo repair. A couple hours later and it’s the same. Hmmmm.. I’m beginning to think this Drobo NAS thing isn’t all it’s supposed to be; it appears to have died as a result of a disk failure, the one thing that a redundant NAS should never, ever do. Hmmmm..
I decided to restart the Drobo. After restart, and an hour or two, I’m happy to report that I once again have all green and 40% consumed, as I expected. The Drobo appears as my Z: on my system, and I can access the files on the Drobo from Windows. All expected backups are on the Drobo and it appears to be running well.
There were a few scary moments there, but all’s well now.
Now the $64K question is: is the “failed” disk really bad, or was that an artifact of the firmware update? Think I’ll format it, check it thoroughly, and re-install it if it’s good.
I have a 5N, it’s been a solid workhorse for years apart from the mistake of using Seagate drives (slowly switching over to WD Red drives, anyhow.. yesterday it dais that the last drive (WD Red) had failed. I took the drive back to where I bought it, they refused to honor the three year warranty, I had a little rant at them and then went elsewhere and bought a new 4Tb WD Red (exact same model that ‘died’.)
I replaced the drive and the Drobo started to rebuild, then this morning it suddenly decided that there was no drive in Bay 1, there is and it’s a WD Red.
I powered off, removed and reset the drives, same, issue. I am pretty much resigned to the fact I have lost everything more or less. My question is, if i get another 5N, no drives and slot the existing drives in will (if it is the Drobo that’s dying) it carry on and rebuild or won’t that work?, failing that can i get a 5N2 and do the same? Hoping to get some positive feedback, it’s my birthday, I’ve lost a massive filling in my tooth and my Drobo is dying along with 5 years of data, movies, music and photos – not a happy bunny today.
In my experience, pretty much every time I move my Drobo 5N, one drive says it’s done – normally the top or bottom ones.
In the past, I have formatted the ‘faulty drive’ and reinserted it with success.
I also know that as long as you keep the drive order the same, they can be put into a new housing – I was considering this so I could get a faster housing for speedier backups.
I currently have a unit that won’t accept a new drive at all. Old drive out, new one in stays red! Maybe it’s time for me to ditch Drobo…
Thank you for this article. Drobo support is the biggest con out there. Their knowledge base is one of the worst, it cannot answer simple questions quickly, for example changing a failed drive. I would urge people not to buy Drobo, although a good product but their support and knowledge based is just crap that does not tell you anything. Keep away from them!