The Forklift

It was never fun getting a call at 2am saying the servers are down. Not “the website is slow” down. Everything down.

I couldn’t connect to the network at all. No remote access, nothing. And this wasn’t some quiet office system,this was a warehouse during peak picking time. Forklift drivers were trying to get pallets onto trucks ready for the supermarkets, and without the system they were completely stuck. So there was a lot of urgency to figure out what was going on.

I headed on down to site, fully expecting to restart a router and go back to bed. What a waste of time this was going to be.

To get to the server room you had to go through an external door into one of the warehouse cold rooms. Inside this cold room was a smaller room with all the server equipment,switches, patch panels, UPS, the lot.

I opened the door and found everything on the floor. The rack, the switches, the patch panels, all of it. And the wall had moved about half a metre.

Figuring out what happened could wait. Right now we had a warehouse full of forklift drivers standing around and trucks that needed loading.

I got someone to help me push the wall back into place. Then I picked everything up off the floor, reconnected the patch panels, plugged it all back in.

Hit the power. Nothing.

After a lot of searching I found the UPS was dead. Removed the UPS from the chain, connected everything straight to mains power. It all came back up. Pickers were picking, stock was moving, trucks were loading.

Crisis sorted.

The next day someone pulled the security camera footage. A forklift driver had ploughed straight into the wall with a full pallet. Hit it hard enough to move the wall half a metre and knock everything off the rack.

And then just… kept going. Apparently didn’t even notice.