In my day job, part of my colleague’s responsibilities is to maintain a farm of servers for off-island businesses.  I won’t bore you with why it is critical to maintain uptime on these servers - suffice to say that is is.

Being concerned and responsible, an adequate generator system was installed and adequate UPS support is provided to allow for switch over to the generator following failure of the utility system.

Thus, today, when the grid went down we were unconcerned.  A quick dash down 4 flights of stairs in the dark, a change of one switch and then a turn of the key on the generator and all was well..well, sort of.  The server room burbled along as happy as could be with no problems.  However, connectivity to the Internet was gone thus making it impossible for remote to the servers.

Question - what did we forget?  ..we neglected to include the data converters (at the end of the fibre) in our generator circuit.  Pop the data converter into a fully charged UPS and all is well again.

A good test of our system and one we learned from.