How about we nuke it?

[Long posting alert, you might want to get yourself a nice drink or go to the bathroom before starting]

So here I am, wandering in to work a bit late (for the reason, see the entry below) and what do I find, but a user, her name is Summer, who can't do any work because, they can't get into an important piece of software on our computer system. So, being the good Sys Admin that I am, I try to help her... and I try... and I try...

This piece of software is written in-house and is a pretty nice piece of software. It seems to work on everyone else's computer and they can all do their work. Eventually, I narrow it down to a localized phenomenon (<== long, but cool word, just say it out loud) and my supervisor, (who was telecommuting, but decided he needed to come in to work on this problem as well) decided that it had to be something with the Summer's machine. I informed him that the app worked correctly for anyone else who logged onto the machine so that wasn't the case, further, the application had been uninstalled and re-installed 8 ways to Thursday since my arrival at a little past 8am (I didn't tell him about the little past 8 part. :). Did I mention that Summer worked in Accounting? You know, the department that pays our wages by taking it from the sucker- er, customers who pay for our service. :) Anyway, this became our first priority of the day, to solve Summer's problem.

We eventually decided that it was more insideous than just her account. It was her account on that one machine as she was able to work on any other machine in the building, neither she, nor he (my supervisor) wanted to kill her AD account, so I dropped a baby nuke on her local account and allowed windows to rebuild it on the fly on that machine. This didn't work. (FYI: the other stuff I'm supposed to do on a daily basis is beginning to pile up :) My supervisor is really stubborn and would not admit defeat. Instead, he codes like a mad man while I research the problem on the Interweb (thanks go to my friend, Marc, for the term :). I begin to connect the dots and realize that the only other person to have a problem similar to this used to be in the same departmnt as Summer and had the same software installed on her computer. We also just created a new account for her because we suspected there was something wrong with her account since it was acting screwey 2 days earlier.

Things began to click into place at this point and we realized that the big change to the software that precipitated the problems with both accounts happened on Wednesday afternoon, just after Summer left early. The other user, Melanie, began having this problem at that time, and we thought we had corrected it (we were wrong), then we suspected that there was something wrong with her AD account as it was behaving funny anyway, so I created her a new one, and all was right in Melanie's world again.

Yesterday, Summer, the user who left early on Wednesday, took the day off and didn't user her computer for us to discover her problem. We had a day where there weren't any problems with the software, so we figured it was fixed. We were wrong! Finally late in the day, we realized that both these users had a particular piece of software on their machines when their accounts were created on them the first time and they were also gived admin user rights to the machines BEFORE their accounts were created. This combination corrupted the accounts (yes I know this post is boring, keep slogging through it and you'll see where this leads :) and made it so the software wouldn't work.

Finally, at 4:15pm, my boss admits defeat and tells me to nuke her account, then create a new one for her. I do so, and it all works great. Did I mention that as a possoble solution, I suggested we rebuild Summer's entire account about 10 minutes after I arrived this morning?

Well, I fixed the problem, but it took all day and sucked the brain out of both my boss and me. He wanted to fix the code, and I wanted to fix the configuration without nuking the account. We were both shown up by my gut instinct that said to nuke the account within 10 minutes of my arrival. FYI: It took me about 45 minutes to save all her data, nuke her account, create a new one, transfer all configuration and data to it, and restore her access. Who knew?

I could have saved myself, my boss, and the user a large headache by nuking the account at the beginning (which I considered, by the way), but they wouldn't have appreciated the effort it would have saved. So, I burned an entire day so that the next time something bad happens and I suggest
we take drastic measures, they'll all listen.

Hee Hee, I am evil!!! :)