Saturday, March 02, 2002

I wanted to know what to do, as I was bored, yes even though I am working, parts of this are boring as hell, so I went to look, and I found this list. This right bastard has some damn lame suggestions, Everything up through 37 I cannot even do, not that I would want to do even half of them, then we get to 38. Sing, SING??? What have I got to sing about, you give me a list, and the first 37 things on it I cannot do, the first one I can do is SING? You got to be bloody well kidding me. Okay lets get past that one.... More stuff I cannot do, and we get to 42, make a list of 50 things to do when you are bored. Ummm does this idiot realize thats the reason we went to his stupid site, was to get a list of things to do because we are bored. If he is doing it why should we????? Uhum, 45 Surf the web, how does he think we found his site? Surely we did not just want to go there, perhaps maybe we were SURFING... Finally, read an article about things to do when you are bore, well John thats what I did, and you know what, you just served to irritate the hell out of me....
Hmmm what do you do when you have run out of things to do when you are waiting for the raid array on your server to rebuild for the umpteenth time? I have looked at Slashdot about 5 times, Geeknews about 3 (its not like they update their content enough to warrant more viewing), WinInformant once (he does not update all weekend) once, Activewin once, and I even checked out Fugly.com, which I had not been to in a while... Looks like Morpheus is going down the Gnutella path, too bad, I rather liked the FastTrack system they were using. And my experience with Gnutella was that well, as soon as people started using it it started sucking and you could never find anything on it... Hopefully its changed.
Some things you might want to know if you go with a roll your own raid system:
  • Check your Raid controller bios REVs. Make sure they are compatible with your drives!
  • Check your Drive's firmware REVs. Make sure they are compaible with your Raid controller!
  • DO NOT MIX. Keep all the drives on a channel, perferably on the system with the same Firmware REV.
  • Quality drives are WORTH THE MONEY. We got IBM drives, they are quality, but we still had issues.
  • You are on your own. The drive company will point at the controller company, the controller company will point at the drive company. Just like with software it happens with hardware.
  • Make sure you have the newest/oldest drivers. Do not be their guinnea pig, but if somethings been out for 5 months, it SHOULD be good.
  • The above goes for firmware REVs too. ALWAYS make sure you can back out of firmware REVs.
  • No matter what HAVE backups. And if you do notice a problem, make sure you have REALLY GOOD backups.
  • Well in the spirit of the name of this place, you all are probably wondering where the F&*K I have been. Well yet again its been a busy week, and I am actually writing to you after being at work since 3am. Needless to say I am pretty tired, but rather than start at the end and end at the beginning why don't I start before I finish. The week starts off with Sunday, you know that, I know that, heck even our major bucks hosting provider knows that. So its Sunday afternoon, and for a change I do not have the on call phone. Which is nice, I usually have it, I usually feel a bit of a responsibility to have it. I setup the systems, so it kind of makes sense that I maintain them. We do not usually get pages but if something does come up hey makes sense for me to handle it... Well anyway as I was saying, I did not have the on call phone, but apparently our extremely expensive hosting provider, could not reach the on call phone so paged me. Apparently our EXTREMELY PRICEY hosting provider, needed to do some emergency work on one of our back end network switches. Okay I tell them, whats the impact to us I ask. Silence, this person did not know, they tell me to hold on. They come back and say they are going to forward me to their manager. I say okay. At this point I know we are in for some fun.... The supervisor, at our EXTREMELY PRICEY hosting provider, apparently does not know either, and he asks if I would like the technician who is going to do the work to call me right before he does it. I explain to him that I happen to be responsible for the servers there and if they happen to go down it would happen to be nice for me to let people know more than 1 hour before it happens if at all possible... He understands and says he will get in touch with someone. So about an hour later I get a call from another person, and he tells me what he thinks the impact will be. I say GREAT, are you sure of that. He says ummmmm, he will have someone verify. Wonderful! I am happy, the world is right.... He calls me back, he says he has verified this, and that impact will be X. I say awesome, please have your guy call me when he gets done....He says sure thing, I say wonderful. So.... Have I mentioned our EXTREMELY PRICEY hosting provider? Well the impact X he listed, well he was wrong, it was X and Y and Z. He missed to other boxes. Oh and instead of starting when they said, they started an hour later. And called me abou 45 minutes after they were done... So I was up at 4:30 cause of their mistakes for the day. Oh and of course the baby was sick and up from 12:30 till 2:30.... So we go on to find that our EXTREMELY PRICEY hosting provider messed up and in fact it affected more than they had thought and they had really no clue about it at the time, though looking at it afterwards they should have seen it they said. So they schedule for Tuesday AM... They actually did it, and did it correctly I was amazed I can tell you... So now we move onto Wednesday, apparently our Database server at our reasonable prices hosting facility goes down. And through someone accidentially rebooting the wrong box I have to go down, and notice there is a yellow light on the raid. I should have run at that point. I replace the drive, unfortunately it does not bind. So I write down I need to schedule some time on Friday for a reboot to get into the bios an see why it did not recognize the new drive. There were three raid systems in there, 1 dual channel controller and 1 single channel controller. One of the raid setups on the dual channel controller died. Luckily not the database one... So here I am, been here since 3am and its now almost 3pm. I just ran into more issues... Ahhh I love hardware failures....