Overtime

Thes past seven days have been long. Collectively, and individually.

At work, our department has been preparing for this week, because this is the week of a goodly number of annual meetings. That means all TMHU employees need to be on site. That means all the employees that work from home and all the field travelers bring their computers on site. That means we get thirty users who have been holding off on getting computer problems fixed all at the same time. To make it all the more memorable, we perform a full backup of all the off-site computers because for most of them, this is the only time their data is backed up at all.

So, most of these people are in meetings from 08:00 until 18:00, then they have dinner meetings. Then they go back to their hotels, and since they still need to get their normal jobs done, they work from the hotels. Now somewhere in there, we need their computer for somewhere in
the 2-5 hour range. So I started at 06:00 everyday this week, and I worked until about 05:30 each day. Busy.

Plus, we had normal service calls to go on as well. I say again, busy.

Yesterday (28 Jan), Saturday, is normally a time that I look forward to sleeping in, spending some time with my lovely wife, pursuing some leisure activities, and volunteering to direct video for my church if it is my weekend on.

I did one of those items yesterday. Instead of sleeping in, spending some time with my lovely wife, and pursuing some leisure activities. I went to work. We had scheduled this weekend work for last weekend, but we had to reschedule for this weekend due to schedule conflicts. So I was at work at 09:00. We were bent upon upgrading our CallManager servers but we wanted to make sure we had good backups of the servers themselves. Since we are running a RAID 1 on those servers, we broke the mirrors and rebuilt the array with another physical disk.

For my non-technical readers, on each of our two servers, we had two physical hard drives. We configured each server to act like it only had one hard drive and just clone (mirror) any information onto both disks. So at any given moment, the computer would have two drives with identical information. What we did was pull of the drives out. Once the computer detected that one of the drives was missing, we told it to ignore it and keep on going. Than we put in a new, blank drive. When the computer detected the new one, it asked if it should ignore it or rebuilt it with the information on the first disk. We chose the second option, and ten minutes later, the server is exactly as it was, but now we are holding a hard drive with all the information that is on the server. If
something catastrophic were to happen (like if we were to royally screw up this upgrade we were attempting) was can just drop in the drive we just took out, and the server will revert instantly.

Breaking the mirror took longer than expected, about two hours. After we were sure we had good backups, we started the upgrade, which begins with a data backup. Unfortunately, this is another king of backup from Cisco, and we had to do it. So we started the backup and went to lunch.

Came back from lunch to see the backup still running. After another hour or so, we got a message that the backup had failed. So we tried again. It finished, but it was 15:00 by this time and I needed to be at church by 16:00. We reverted the servers and I was off to church.

This was the first weekend with our new video guy on staff, Shane. This guy is awesome. He knows his stuff, he’s super creative, and he’s a fun guy. There is one thing. He’s a Mac guy. Along with
Curtis, and Brian these guys are taking over! They are in the process of moving out our old Video Toaster machine and moving in a G5.

I must say, the video looks good, and the Mac is easier to use than the Video Toaster.

So now I look forward to next week with a new sense of dismay, and anticipation. I begin my last semester of undergraduate education tomorrow. Wish me luck.

4 Replies to “Overtime”

  1. Was able to follow most of that! 🙂 Love you and we are praying for you last semester of school. Love watching you reach you goals.

    Mom

  2. I thought that week was awesome! I live for those weeks when all the field guys are in! It’s better than Christmas! You and Lae did great.

  3. What version of Callmanager are you currently on?

    We decided to do our upgrade in-house… our thought is that rather than try to upgrade in one fell swoop, we’ll implement a new installation on new servers, and then migrate over.

Leave a Reply

Your email address will not be published. Required fields are marked *