Reflections from Nick Aschenbach, Head of Engineering at VECKTA
On a long enough timescale and despite our best efforts, inevitably our production systems go down. The best thing we can do is control how we respond to the situation. Today, we had a production outage and I am proud of how my teammates reacted. There was no finger pointing, no blaming and no egos. We banded together, got right to work and solved the issue as a team. This is exactly the culture we are trying to build at VECKTA.
It’s not until we are challenged that we know that we are capable. The philosopher Epictetus was no stranger to adversity. Born into slavery in ancient Greece and evolving into a teacher of Stoicism later in life, said, “What would have become of Hercules, do you think, if there had been no lion, hydra, stag or boar – and no savage criminals to rid the world of? What would he have done in the absence of such challenges? Obviously he would have just rolled over in bed and gone back to sleep. So by snoring his life away in luxury and comfort he never would have developed into the mighty Hercules.”
It’s the challenges that define us and provide us with an opportunity to grow and be better versions of ourselves. We cannot change external events, but we can change how we respond to them.
There’s a phrase common in Stoic philosophy, Premeditatio Malorum. The premeditation of the evils and troubles that might lie ahead.
Famous Roman philosopher King Marcus Aurelius said, “Just as nature takes every obstacle, every impediment, and works around it—turns it to its purposes, incorporates it into itself—so, too, a rational being can turn each setback into raw material and use it to achieve its goal.”
This event got the team thinking about enhancing our high availability disaster recovery (HADR) plan. While we were able to restore the production database from a backup on AWS, we are putting in a procedure to reduce our downtime in the future.
More generally in businesses we are increasingly impacted by outages driven by severe weather events as we saw in Florida with Hurricane Ian this fall or the power crisis in Texas in early 2021. Here in California, the investor-owned utilities SDG&E, PG&E and SCE regularly shut off power during fire season to protect public safety. It’s been estimated that the cost of outages has grown to $150 billion in the United States each year.
If you are thinking ahead and starting to plan for the worst when it comes to energy, please feel free to reach out to our team at VECKTA. We help organizations in energy transition by lowering costs, reducing greenhouse gas emissions and mitigating risk from disruptions to the energy grid.