I was enjoying Cuban cigars and ice-cold vodka with an old friend of mine, who I’ll call Joe, one afternoon when he received a call from one of his clients, Petulance Corp., begging for a miraculous cure to the sudden and comprehensive demise of their mail server.

Now I know this company and they are cheap. I don’t mean spending their money to get the “biggest bang for their buck” cheap. I mean “just because the whole company depends on this one piece of gear and if it fails the business will stop doesn’t mean we need a backup” cheap.

Joe listened to the panicked voice at the other end of the phone for a couple of minutes and said, “I’ll see what I can do but don’t count on any miracles.”

He asked me, “Do you know anyone who can get me a replacement for a mail server that’s melted down into a small puddle of metal and plastic slag by close of business?”

We both drained our glasses and I said, “Not at four o clock on a Friday afternoon.”

Joe sighed and dialled the number for the CEO of Petulance Corp., he put the call on speaker. After telling the CEO that I was also here he said, “I’m sorry, but, I don’t think we can have your email back up and running until Tuesday at the earliest”.

Now I won’t repeat the invective that was then hurled towards Joe. Let’s just say the CEO wasn’t happy. Then he followed up with, “We just can’t live without mail for that long. It’s the most critical thing we need. Our entire business depends on mail. People are contacting me all the time by mail. You’re responsible for this mess, you need to fix it!”

Calmly, Joe said, “we could do something for you over the weekend by getting one of our servers to act as a mail server for your domain until we get a replacement in for you.”

Any sane and rational person would think to themselves that this was a valid, if not ideal, option. Not the CEO of Petulance. He said, “That doesn’t work for me. Our email contains very sensitive, confidential information that no one but me can see. You need to get me a server now.”

Now I first met Joe when he and I worked together on a job for the Navy. My take on this was that if the Navy trusted him with a clearance and access to stuff that mere mortals weren’t able to see, then surely Petulance Corp. could trust him.

Joe poured himself another drink and settled into his chair and said, slowly, “It’s late on a Friday afternoon. It’s going to be hard for me to get any hardware from anyone at this time of the day.”

The response was priceless. “You tell them that the CEO of Petulance Corporation demands that they provide us with hardware, otherwise we won’t do business with them again.”

Now Joe and I looked at each other, nonplussed and struggling to not laugh.

“You really want me to ring up my suppliers and tell them that if they don’t get you a server today you’ll never do business with them again?”

“Yes.”

Poor Joe just sat there and said to him, “I’m sorry, but that’s just not going to happen.”

The silence on the end of the phone was deafening. For any of you who have kids, it was that silent pause you get between saying no to whatever it is your toddler wants and the instant that the tears and screaming starts. Then the screaming started.

Joe and I poured ourselves a drink while we waited for the rant on the other end of the phone to stop. It went longer than either of us expected.

In the end, Joe said quietly, “You need to make a choice. Either you get mail back this weekend by getting it on our server or wait until Tuesday to get your own server.”

Finally, the CEO said he’d wait until Tuesday, but if it wasn’t up by then he’d call in the lawyers. Then Joe asked, “Can you have your backup tapes ready so we can restore all the settings and user accounts?”

“Backups?”

Joe and I looked at each other, shocked, stunned but, deep down, not really surprised.

“You have been doing backups, haven’t you?” Joe asked.

“We do whatever you told us to do,” came the cagey reply

“When I installed the server for you, I quoted for a backup system. You said it was too expensive and told me that you’d come up with a more realistic solution,” Joe said.

Now let me say, Joe is no shrinking violet. He’s one of the best in the business, but don’t treat him like a fool. It won’t fly. That’s when the backpedalling started.

After a lot more back and forth they finally agreed on a strategy. Joe hung up and looked at me. “Wanna help?”

I should have known better, but I agreed.

With the clock ticking towards 5pm, Joe hit the phones and got a server organised for pickup first thing Monday morning. Then he called the poor, harried IT support guy who spent his days at Petulance being kicked from pillory to post, toiling away at a truly thankless, soul-destroying job.

Once Joe found out that the server software and licence keys were in fact somewhere he could access, he relaxed a little. He said to me, “You know there’s no backups, don’t you?”

“You owe me a bottle of 42 Below for this,” I said.

~~~

After picking up the server we drove out to Petulance Corp., which is in the wilds of Ingleburn. I was once told that more dead bodies were found in Ingleburn than any other suburb in Sydney. True? Who knows? But it makes for a great story.

We were greeted like an attacking horde of undead zombie brain-eaters. This is nothing new for this Petulance – technology is a necessary evil, provided by troglodytes with no business acumen.

As we walked towards the grubby corner where they kept the server, we could hear the mutterings about email made just loud enough so we could hear.

The server corner was hot, badly ventilated, poorly lit and really somewhere that you wouldn’t put critical business equipment. After we put the server in place we installed the software and did all the other techno-wonkery that really impresses people who know nothing about technology.

Joe went looking for the backups.

After a while he came back, not happy, I mean really not happy. They’d been doing the backup on an old tape, had never verified it actually had backed up and did the backups once a week, over-writing the tape each time.

“It’ll be a miracle of we get anything other than oxide deposits off this tape,” he said, somewhat resigned.

Then the CEO walked in.

“So how long before my email is back?” he asked.

Joe held up the tape and said, “If this tape doesn’t work you’ll have lost mail and it’ll take us the rest of the day to get things working.”

That was it, the CEO hit the roof and turned a dark shade of purple. I actually thought he was going to have a stroke or a heart attack on the spot.

“If that backup doesn’t work, I’ll sue you!”

Then there was that terrible pregnant silence, you know the one that you get in every western when the stranger walks into the saloon, the piano player stops and everyone in the bar turns to look at who’s just walked in.

“When I set this system up for you”, Joe said, “I said that you need to have a reliable backup system. I quoted one for you and you said it was too expensive and that you’d get a backup system that cost less because the solution I specified was a Rolls-Royce solution when a Falcon would do.”

Joe held the tape out and said, “Here’s your Falcon.”

“So you’re saying it’s my fault.”

“I’m saying it was your choice and now we roll the dice and see if it works.”

The CEO skulked away.

It took a while and almost all the data was restored.

Joe asked everyone to log in and enough of the data seemed to be there to keep people from forming a lynch mob.

The CEO seemed happy enough until Joe popped an invoice on his desk for the work. “I thought this was all part of the service.”

Joe shook his head. “Remember when I offered you support on the system and you said you weren’t willing to pay support fees because they were a rip off and your insurance would cover the costs?”

“But my premiums will go up.”

Joe shrugged and said, “Ever since you made me wait three months for my previous payment I advised you that you were COD. Can I get the funds transferred please?”

To cut a long story short, he eventually transferred the cash after a lot of complaining and arguing about the fact that every other supplier accepted 60 days terms from him.

When we were in the car leaving beautiful downtown Ingleburn, Joe said, “The problem with him is that he thinks he’s the smartest guy in the room. And when it comes to his business, he may be. But when it comes to technology, he’s got no idea and his ego won’t let him admit it.”

The moral of the story is that, no matter how tight your budget is, always make sure that you have allowed enough to cover for the inevitable system crash. Like death and taxes, your server will fail. It’s just a question of when. And I guarantee you that it will happen at the most inopportune moment.

A couple of days later, Joe advised Petulance that he didn’t want to support them anymore and that they had 60 days to find another support outlet.

The guys that took over from Joe do support for another small organisation I know. When they told me that they’d picked up Petulance as a customer, I asked them how they had found dealing with them. They said it was difficult but that they’d get them under control.

The fallout from that is a story for another day.

The 1% Spend ^[1] is written by a prominent Australian I.T. consultant who is choosing to remain anonymous (and candid).

Photo: Jen Jen ^[2]