In early March we saw a number of articles talking about the risk of the Northwest to an earthquake similar to the one that devastated Chile. We’ve heard about the fault off our west coast that has been dormant for several hundred years, and that it in fact might wake up some day — nothing new right? During recent computer simulations of a “hypothetical” 9.0 quake it was determined that the shaking could last as long as up to 5 minutes. That would rattle the Seattle, Portland and Vancouver areas significantly. It’s no secret that a quake of that magnitude would severely affect the infrastructure in those major cities. Business as we know it would be disrupted as well. Buildings that were constructed in the Seattle prior to 1994 are expected to collapse due to the lesser building codes under which they were built.
Disaster Managers throughout the region are working to strengthen the infrastructure to withstand a major seismic jolt, but they won’t be able to touch everything. Even if your structure withstands the rocking and rolling, it’s highly likely your business will still be disrupted due to the damage to the surrounding infrastructure. How long can you afford to be away from your business? What are the expectations of your local clients? They may understand for a period time because they too will be unable to get around. Do you provide products and services to those outside the area? What are their expectations? This may put a different spin on things.
In the interest of the survival of your business you should consider geographical diversity. Not necessarily a “second place of business” but at a minimum, from a business continuity perspective. You should develop a plan that allows you to respond to the event, to resume your critical business functions and processes, and to recover your business over time – you should consider an alternative outside the region and away from the seismic risk. For example, a few years back Forbes Magazine published a list of the safest cities in the US. Four out of the top five were in the Inland Northwest. They were Boise, Yakima, Spokane and the Tri-Cities. Here at IT-Lifeline, as a provider of business continuity services, we not only take pride in our location, but also in the technology environment that we’ve put together in support of an organization that experiences any major business disruption – including that 9.0 shaker.
It’s your choice – wait and assume the risk or take action and put a viable business continuity solution in place. The success of your business will depend on it. Remember, scientists cannot predict when a quake will occur, but they are certain that one will happen.
Most of you that read this post periodically know that I’ve been a business continuity planner for quite some time, in fact – 35 years. One of my issues with the “industry” over the years has been the constant shift in terminology. When I started, the process was called disaster recovery, it then became crisis management, the process then became business continuity and today the last time I checked, it was still called business continuity management planning. So over the years I have been a disaster recovery planner, a crisis management planner, a business continuity planner and now a business continuity management planner.
Now we are starting to see a change in some of the terminology that has been associated with the components of the business continuity professional best practices. I say, “Leave them alone” – many of us have spent years educating our organizations / clients on these terms.
The most recent change I’ve come across during some recent reading is something called “undetected configuration drift”. Have you ever seen it used? I’ve seen it used in a number of fashions and they all make sense from a business continuity perspective, but from a terminology perspective –a bit much, in my opinion. It sounds like some type of disease or illness. The term used to be called “gap”. To me it’s a little easier to say and a lot easier to explain. What we’re talking about here is the “disparity” between your disaster recovery (technology assets) environment you’ve defined / set aside and the actual technology required at time of event to recover your critical business functions following a major business disruption. Doesn’t “gap” make more sense?
Let’s commit to the “keep it simple” process! No need to make things difficult. “Gap” works for me!
It’s a pretty well known fact that small to medium size businesses (SMBs) are the backbone of our economy. Depending on what article you are reading an SMB is defined as a business that employs less than 500 employees. In the city of Spokane, in 2007 99.8% of our local businesses fell in to that category. 85% (11,058) of our businesses fell into the small business (<20 employees) category. Every locale is different — what doesi t look like in your corner of the world?
Another well known fact, to those of us that work in the business continuity arena, is that SMBs often run lean and go without basic protections against operational risks, due to lack of time, budget and staff resources. That includes disaster recovery and business continuity planning. Published figures indicate that 60 to 65 percent of SMBs do not plan and those that do, don’t ever test their DR plans. That’s not good!
A survey conducted by Symantec in September of 2009 with businesses that actually do plan for disasters produced some interesting statistics – Symantec 2009 SMB Disaster Preparedness Survey. Check out the North American Data.
The survey centered on the SMBs disaster recovery planning efforts – the recovery of their technology (systems / data) following a major disruption. The survey identified a significant difference on how the respondents to the survey perceived their level of disaster preparedness versus their actual level of preparedness.
The majority (>80%) of the SMBs surveyed were “quite confident” in their DR plans and the level of protection. In excess of 60% believed that their customers would wait patiently for them to recover or call to get what they could and then wait patiently for recovery. 38 percent felt their customers would “evaluate other options that included looking at competitors.” Wait till you see what the customers had to say!
Symantec’s survey indicated that the confidence level was “misplaced.” In the end Symantec reports that in reality SMBs are “remarkably unprepared” based on their response to the survey. Several issues were pointed out. Here is what I observed:
- 50 – 59% of the respondents indicated that the percentage of company / customer data backed up was only 27%. Overall it was only 40% of the company data – that is scary
- Only 16% of the respondents indicated that they back up their computer systems and information daily, and 11% weekly – that could result in a lot of lost data and scary as well
- 63% indicated they would lose 40 percent of their company data should they experience a major fire. Not good at all from a business continuance perspective – it’s probably not going to happen
In addition, what I found most interesting was that Symantec solicited input from the SMBs customers. Almost 40% of those customers surveyed indicated that they have actually switched vendors in the past due to unreliable technology and the impact it had on their business. The bottom line is this – if you are an SMB and have a high reliance on company / client data can you afford to lose 40% of your customers following a major business disruption?
As pointed out in the Symantec survey, it is recommended that SMBs follow best practices:
- Determine your needs: — Identify what’s critical and what’s not. Establish a priority. Monitor risk and prevent threats that the organization might face
- Engage trusted advisors – Look to a solution provider to help create plans, implement solutions and monitor trends and threats
- Automate where you can – Automate the backup process, ensure the timely and regular backup of your systems and data
- Test annually – ensure your return on investment and plan viability
I encourage you to read the Symantec article / survey. If you don’t have a plan I encourage you develop one. If you have an existing plan, take the time to ensure it is viable and would meet the needs of your organization in order to ensure business continuity following a major business disruption.
So Mr. or Ms. Business Continuity Planner what’s your New Year’s Resolution for 2010? Here are some suggestions:
- If you haven’t started a plan – develop one! Make sure you familiarize yourself with the industries suggested best practices. It will make your life easier and allow you to build a viable plan that would indeed work should your organization experience a major disruption to its business operations.
- If you have a plan – review it to ensure it is indeed viable.
- Review your risk – Natural, Technological and Human caused events. Knowledge of your local risk is critical – know what could affect your business operations and plan accordingly. Make sure you have planned for the worst case scenario.
- Revisit your business impact analysis. Make sure that nothing has changed in your organization that needs to be addressed from a business continuity perspective. You don’t want to find out at time of event that a new product or service is critical to the organizations survival that is unaccounted for from a resumption or recovery point of view.
- Revisit your recovery solution. Make sure the resources you’ve set aside match the output from your BIA review. Avoid any gap that could delay the resumption or recovery of a critical business function or process. Make sure you have the appropriate “pieces” in place – technology and people resources. Don’t forget to revisit your data backup strategies to ensure they too would support the continuity effort following an event.
- Review your planning documentation. Ensure yourself that the documentation reflects the current organization. Make sure you have addressed the three key elements of business continuity planning – Crisis Management (Response), Business Continuity (Resumption & Recovery) and last but not least Disaster Recovery – the recovery of the IT infrastructure required to support the critical and essential business processes / functions.
- Commit yourself to a defined schedule for maintaining, testing and exercising your plan. Pick some dates and stick to them. Regarding IT testing – remember almost 80% if 1st time technology tests FAIL. Something you need to discover during a test and NOT at time of event.
- Schedule a 3rd party review of your planning process and efforts. A third set of eyes is always better. Have that 3rd party audit and certify the condition of your business continuity plan. Identify the good, the bad and the ugly! Prepare a plan to address the bad and the ugly. Present the findings to your executives and board if necessary.
Whatever your New Year’s Resolution, may 2010 be a productive year for you all. Here’s to hoping you never have to implement your plan, but if you do, here’s to wishing your planning efforts prove to be successful in every way. Happy New Year to you all!
Whether you are starting to develop your business continuity plan or have a mature business continuity plan one thing is certain – in order to facilitate a viable and cost effective business continuity capability you must have an understanding of the business as the executive level envisions it. Without that specific knowledge, the probability of ensuring the preferred recovery from a major business disruption is questionable at best – thus impacting the business even further.
Too often I see that the recovery objectives are based on a functional area manager’s or the planner’s individual perception. Although the manager’s or planner’s perception may be correct, it needs to be verified and blessed at the executive level. Only then can one put together a viable and cost effective solution for ensuring the desired results – the resumption of critical business functions and the recovery of the business over time. Only then can you align the required recovery sequence with the required resources – technology (systems, applications & data) and people (alternate workspace & their other needs).
If your organization is currently experiencing mass change – people, products & services, facilities, and business strategies, you must ensure that your business continuity strategy keeps pace in order to meet the expectations of the business, even when time and personnel resources are limited. One can not overlook the continuous need to maintain a thorough knowledge of the requirements for resuming business operations following a disruptive event. You can ill afford the gap that can be created by a lack of attention to business continuity at time of change. You may find yourself spending too much or not enough in terms of real dollars to protect the business when the time comes. It is imperative that the business continuity plan, and thus the recovery time objectives match the expectations of the business and your clients.
Change is an often overlooked issue when it comes to business continuity planning. Things that change on the business front need to be analyzed to determine if the change impacts the organizations business continuity plan. Once thorough change management is implemented, you can make use of the change management procedures to improve your ability to keep the business continuity plan updated by noting significant changes in the infrastructure that are vital to keeping the plan effective. A properly implemented change management program plays a vital role in reducing infrastructure instability and improving operational availability following a business disruption.
What type of change deserves oversight? It’s essential to pay attention to changes in management, staff, business strategies (products & services), business processes, and the technology environment utilized by the organization. It is also imperative to look outside the business and factor in changes that involve critical vendors and key services providers.
In order to ensure plan viability, it is a MUST that the change management processes consider the impact of change on the organizations business continuity plan. At time of event is NOT the time to find out that a change or a series of changes were not factored in to the business continuity plan and thus the organization cannot fully recover their critical business functions and processes.
Where do you stand? Are you one of those organizations that have or are considering deferring your business continuity planning efforts? In the past, when I have had a conversation with those that fall in the above category their thought process was usually – “those concerned about our business continuity plan, the regulators or external auditors, will understand given the economy and the other issues facing business and industry these days.” My response to them is this – “Maybe they will or maybe they won’t.” It’s a gamble you may not want to take.
I typically follow with – “How about the other side of that coin? Will your customers, your clients and others understand?” If you are providing a product or service they depend on to sustain their existence the answer is probably not. Is this a gamble you really want to take?
I agree it is important to focus on the bottom line, but it is also important to remember that should you experience a major business disruption, it’s been proven that many of those that depend on that product or service will abandon you, and THAT will definitely impact your bottom line.
During a recent workshop on business continuity and crisis communications those in attendance agreed – without a plan and the ability to communicate, those that depend on their product or service would not “wait and see” to see how things transpired following a major event. They could not be away from their customers / clients for an extended period of time.
Planning is essential. You need to understand the risk your organization faces, develop an impact scenario, develop and implement a solution for recovering your technology, to include your voice environment, identify a recovery location, and most importantly document your plan. In addition, once the plan is developed it needs to be tested and / or exercised on a regular basis.
As the local business and industry group, the Alliance for Business Continuity & Disaster Preparedness www.preparespokane.com, continues to promote – “Every business should have a plan! Plan to stay in business!”
You Must Ensure the Recoverability of Your Critical IT Infrastructure!
If it were possible I would ask for a show of hands, but it’s not, so let’s pretend. How many of you have a disaster recovery plan? By a disaster recovery plan I mean a formal documented plan that would allow you to recover your critical technology assets. I mean a specific plan to recover the systems, the applications and the data? Oh yeah, don’t forget ALL the platforms, the desktops and the telecommunications environment – voice and data. How many of you test the DR plan? Do you bring the entire critical Information Technology environment up to ensure it would work if need be – from the ground up? How many of you test more than once a year or when technology changes?
If you raised your hand to all of the above, congratulations – you are definitely in the minority. Depending on what survey you read, very few organizations that have a plan test their plan – especially the small / medium businesses. Fewer yet, regardless of size, actually identify measurable test goals and objectives and/or track the results.
When you factor in Symantec’s 2009 disaster recovery statistics – 25% of all disaster recovery tests fail, and only 15% of those surveyed indicated they have never had a test fail. If you have never tested – your odds of full recovery are not good, especially in a timely fashion.
When you consider an event that requires the actual restoration of businesses information technology that supports the critical business functions; not having tested the restoration process is a scary thought. Are you the one who would have to explain to senior management that you were unable to restore the technology environment to the level the business units and clients were expecting?
Testing is the ONLY way to ensure the recoverability of your critical infrastructure! IT-Lifeline clients that test prove that every day. They are ensuring the recoverability of the technology it would require to support their organization and thus their customers, should they experience a business disruption that affects their technology assets. Can you say the same?
If you had access to our local paper on Saturday you may have come across an article entitled “Nonprofit agency loses data to overnight theft of computers”. It seems that four computers were stolen from this organization’s location that contained “irreplaceable data”. The computers contained critical information that included grant information, donor lists, e-mail addresses, supporter information and promotional material. The local paper reported that an agency spokesman indicated that they had insurance, the equipment needed to be replaced anyway, but that “it’s the data that’s irreplaceable”. The agency spokesman went on to say from now on “we are going to make sure we back up the data.”
Although I am sorry for their loss, the above is a lesson that is often learned the hard way. All too often someone loses their computer or computers either through theft, fire, water or simply computer hardware failure only to find out that the data that resided on the system was “irreplaceable”. Data backup today is simple, easy and often times can be done in a hands-off type environment. In other words – automatically, on a regularly scheduled basis, and better yet it is relatively inexpensive. If you compare the cost of the backup, to the expense of recreating what was lost, if that’s even possible, it can truly be cost effective.
As a business continuity planner, I am still amazed that data backup doesn’t get the attention it requires, not only from small businesses, but medium and large businesses as well. I constantly see no backup or limited backup. In addition, some of those that do backup there data don’t give much thought to where that data is stored once it is backed up. Some leave on it on site (Hum!), some take it home and others hand it off to someone who doesn’t have a clue about protecting or securing magnetic media.
Where do you stand on this issue? Is your data backed-up in a timely fashion? How often? Where is it kept? Do you know? If you don’t and might be the one charged with recreating the lost data, if you can, you might just want to ask the “someone” charged with ensuring that the process is performed in a timely fashion and that the media is stored in a safe and secure location.
As the winter weather worsens in the Pacific Northwest, I am continually reminded of the need for PREPAREDNESS.
A series of heavy snowfall since mid-December has left Spokane buried. We broke the 24-hour, 48-hour, 72-hour, 7-day, and 1 month records for most snowfall in a given period just to give you an idea if you reside outside of this area (Mayor Mary Verner spoke to this issue and others in a press conference on 1/7/09). We experienced roof collapses all over the city – 28 buildings so far – and then came the bad news – we would be getting 6-10 more inches of snow followed by rain. Why was that such bad news, you may ask? The added weight that the rain would add to the existing snow on structures all over the area would be a major concern for buildings and structures with flat or low-pitched roofs – we would be approaching 30lbs per square ft, dangerously high. They closed all Spokane School District #81 schools, along with numerous others 4 days already this week because of safety concerns regarding the structural soundness of the buildings. On top of those issues, flooding will start to become a real threat as the rain melts the snow buildup causing transportation, drainage, and safety issues all over the state.
That is a lot of information to take in all at once so I will get back to my main point in writing this post – you must be prepared at all times, for best to worse case scenarios! We were smack in the middle of the biggest snowfall this city has ever experienced, a state of emergency on its own, and then came the next event (more snow and rain leading to heavier snow), and the next (expected flooding all over Washington state)….
As the City of Spokane tried to raise awareness of these events through Press Releases and media, they also spoke to the theme of preparedness. They wanted people/ businesses to be aware of the possible dangers and who they could turn to/ where they could go for assistance should they need it.
Take the time to plan … plan for the next event! Figure out NOW what you need to do before you find yourself scrambling to figure out what your next move will be, should you experience a business disruption as the result of that event – whatever it might be; this article speaks to that point. The rest of this winter should be interesting!