Lessons on IT Resilience During Hurricane Season
Besides battening down the hatches in the face of a hurricane or any other disaster, organizations need to worry about the day after – getting back online if they experience an outage. Here are some ideas on how to accomplish that.
It’s hurricane season once again – and like the inevitability of the big storms, there are other inevitable features of the season that you can bank on as well. For example, the media will trot out its stock footage of the crowds at Walmart grabbing everything they can off the shelves to stock up before the worst. Scenes of the crushing inland-bound traffic will be broadcast from coast to coast as people seek to escape the storm. And there will always be the holdouts who refuse to leave their homes despite dire warnings from authorities that they need to evacuate.
And another inevitability of hurricane season is the outages that will ensue. Power plants, ISPs, cloud service providers, banks, insurance companies, and airlines – are all potential victims of a destructive storm that could flood their facilities, short out their electricity, or create a personal vacuum, with workers unable to reach their offices to maintain facilities.
Big storms almost always cause inconveniences, losses – and suffering – for victims, whether they are residents of affected areas or companies in the path of the storm. For individuals or families, those losses could be in the form of a house that needs major repairs due to flooding or wind damage – while for companies, the losses could amount to millions of dollars.
According to Calyptix Security and ITIC research, infrastructure failure costs 90% of companies as much as $300,000 per hour. At the same time, some large enterprises could lose between one to five million dollars each hour.
Lack of Preparation Is Common
Question: If those losses and suffering are inevitable – as in, we know they will happen – why aren’t we better prepared for hurricane season? Why do companies risk losing millions of dollars when they know a storm is coming – or will inevitably come at some point? What should they be doing to better prepare for an IT emergency that is all but guaranteed to hit?
Why is a question we can’t really answer; no doubt the reason is different for each organization. We know most companies invest in preparing business continuity plans and various disaster recovery technologies. The question, of course, is how effective those plans are and what they entail. Many issues must be considered during hurricane season, and anticipating them all in advance is a science unto itself.
Failure to Plan for Hurricane Season Is Planning to Fail
“Prepping” for disaster recovery means different things, depending on the situation.
To prepare for the possibility of a power failure, organizations might want to invest in:
- Generators to keep their core operations going
- Off-site backups and data replication
- A good disaster recovery (DR) plan to restore data
- Replicating services off-site
- IT resilience validation tools to automatically determine points of failure
Implementing these stopgap measures will indeed cost money and require the dedication of time and personnel. Still, the money spent will be a drop in the bucket in the event – probably an inevitable one – that an organization is off-line for hours or even days.
While many outage-causing disasters can’t be avoided, some can. Organizations should implement tools that automate resilience to avoid configuration errors that can cause outages. These tools can parse through systems to determine where a point of failure might exist.
Automation tools should include support for a wide set of IT layers and technology stacks. They should connect to existing IT service management (ITSM) and configuration management database (CMDB) tools. The best tools will also provide business awareness and feature built-in libraries of industry best practices.
Fortunately, some of teh most sophisticated IT firms in the world have already addressed these concerns. Look for a system that provides:
-
- A realistic resilience strategy
- Clear KPIs for everything related to resilience and data protection
- An automated quality check process
Taking a holistic approach that combines resilience and quality will ensure that your organization can survive the next hurricane – or any other disaster.
MORE ON BUSINESS RESILIENCE
