Saturday, January 15, 2005

Murphy's Law

Tuesday night was awful. I got really sick. I don't know if it was the flu or what, but I was nauseated, alternated between being brutally hot, then I'd break out in a cold sweat, then get the chills, and then do it all over again, all night long. I also had back spasms and shooting pains down my legs all night. I was afraid to take any ibuprofen, cerrtain that I'd vomit it up. I didn't sleep more than 15 minutes at a time. It was terrible. I got up to do the whole mocha/lunch thing for my husband and lasted about 5 minutes before dizziness and nausea sent me right back to bed. At 6AM I called and left a message at work that I would be home sick. I finally fell asleep at about 6:30AM (I got up the nerve to take the ibuprofen, which knocked out the fever and back spasms, allowing me to sleep). At 8:30 I was jangled awake by my cell phone. It was work. "I'm so sorry to call you but....." was how the call began. Naturally, the day I'm not coming in, the servers decided to break. Apparently, our webserver wasn't working. To be more specific, once I had the caller go to the server closet to check on the offending machine, the hard drive seemed to have crashed! Why!?!? Why when I am SICK is it doing this to me? I could just say "oh well, no web server or online catalog today. I'm sick." But no, I feel compelled to go in. So, I throw on some clothes, brush my teeth and hair, and head to town. I was figuring I'd be rebuilding the server, which is a huge undertaking, so planned to be there all day. Fortunately, the SCSI controller for the hard drive has a nice utility that verifies the hard drive and moves data from bad sectors to good ones. After a few minutes of trying some other things, I ran the utility. It did indeed find bad sectors and remapped them to good ones. Voila! The server rebooted and all was well. Elapsed time - about 90 minutes. Back I head to my bed. Unfortunately, sleep eludes me. I channel surf for a bit, until the mind numbing garbage on the tube puts me in the mood to sleep once again. I head to bed. About an hour into the nap my phone rings again. No way. This can't be! "I am so sorry to call again...." Yeah, right. If you were sorry, you wouldn't call!!! This time, both the web server and the proxy server are down, meaning no internet access, no online catalogs, no web site. The caller tells me that there are no lights on the two servers and the monitor won't come on. Hmmm... no power, obviously, but how can that be? They are plugged into a battery backup and the power is on at the building. The battery backup must have died for some reason. So, I attempt to walk the caller through hooking up a brand new backup I have by the servers. Easy. Just open the box, plug it in, turn it on, then plug the servers into it. Why is this difficult? Three people get involved. They can't seem to figure out how to trace the power cord from the back of the server to where it's plugged into the old backup so they unplug the correct power cord and plug it into the new backup. I am pulling my hair out on the other end of the phone. I can't figure out how to explain this more simply. Finally, they claim to have them plugged in to the new backup, but still no power. I ask them if the new backup is on - are there lights on it? They say there are, it is. I am at my wit's end. I head to work. I go to the server room.... the backup is not turned on. I turn it on, all is well. Elapsed time, 45 minutes including drive to and from. Unbelievable.

Why is it I can work day in and day out, 9-10 hours a day and the servers are fine. I take a day off, call in sick, whatever, something goes down? This happens almost without exception. Murphy's Law.

I went to work on Thursday at 10, was tired, weak, so I went home at 4. A record short day for me.... I'm well now.