Business Continuity in Lean Times

Business Continuity in Lean Times

January 15, 2013 2:00 pm

(Save to cal)

Online Tech

Join Online Tech's Systems Support Manager Steve Aiello as he leads a three-part webinar series on the topic of disaster recovery.

Title:Business Continuity in Lean Times
When: January 15th @ 2PM ET.
Description: Businesses have a responsibility to their stakeholders to think about their long-term viability - Steve provides an overview of disaster recovery and business continuity with real company examples and the benefits of developing a business continuity plan.

 

 

 

View slides (PDF).


April: Hi, everyone. Thanks for joining us today. Thanks for your patience. We are working on our DR Plan for Office 2013. We have enjoyed a very short recovery time objective there. Here we go. We’re going to get started.

Today I am happy to bring you Steven Aiello. Steven is our Support Systems Manager at Online Tech. Steven has a lot of background in information security and is also a certified incident responder. He is a CISSP and is certified in VMware, Cisco and a very long list of other technologies that I’ll let him go into further detail about.

We’re kicking off the first of a three-part webinar series today. In the series, we’ll talk about business continuity and disaster recovery. For our first one today, we’re going to focus on business continuity and exploring what businesses are doing to achieve business continuity in lean times. With no further ado, let me turn it over to Steven Aiello.

Steve: Okay. Thank you very much for joining us. I very much enjoy this topic. It’s something that I'm very passionate about. A lot of disasters and fires and earthquakes and hurricanes get all the press, but it’s really what you do after that and all of the hard work that you put in before a disaster that really makes or breaks a scenario. Thank you very much for coming and sharing the time with us.

A little bit about me, I got my career started in a small venue out in Livonia, Michigan, basic IT work. The lessons that I learned there in the business really helped shaped my mindset in what I appreciated about business.

A lot of small to medium-sized businesses face very unique challenges. We’ll talk a little about that and I’ll give you some specific examples of how small to medium-sized businesses really should be thinking about business continuity even if they're not really thinking about disaster recovery. It’s really important for the business itself.

Once I moved from on there, I worked for a company out in Dearborn. We did a lot of debt collection specifically around the healthcare space. It was a lot of HIPAA compliance, back when HIPAA was still brand new. I was really fortunate that the owners of the company took it very seriously. Right from the very beginning, I was exposed to a lot of training in the HIPAA space. I was fortunate enough they would send me to a lot of seminars so I could really understand what was going on and how that would affect your business. I'm really thankful for that.

One of my first real enterprise jobs, I spent about eight and a half years at ADP, which is the world’s largest payroll firm. The last quarterly meeting I attended there, in some way shape and form ADP touches 74% of the world’s payroll.

I spent eight and a half years at ADP in a really, really large environment. They have data centers globally, they're doing payroll for the military, they're doing payroll for some of the largest financial firms on the planet. Every single compliance initiative that you could think of, SAS70, SOX, PCI, ADP … If you call actually the bank on your credit card, a lot of times you’ll get ADP, not actually a bank.

Really a phenomenal learning opportunity for me, and I spent the first four and half years there writing software to aid in our compliance. Specifically I was working very heavily with SOX, Sarbanes Oxley, and SAS70, really understanding the mindset that the auditors had when they came in, how they were looking at our business processes and how that translates into functional IP systems. It was a really eye-opening experience for me and something that I've carried through ever since then.

Then here, I've come to Online Tech. It’s a wonderful corporate culture. I'm much happier. We have a lot of really great technology here and we have a lot of really great customers. I see some of you out there. I’m going to say hello to Larry. I see you there. It’s nice to see that you joined us. It’s a wonderful place to work in technology. We have a lot of really cool things that we’re doing here as well as dealing with a large amount of compliance here.

My actual undergrad was in technology management. It actually was not really in IT. It was all about how organizations and systems work. I think sometimes that this particular learning path has really helped me when talking about business continuity and disaster recovery because what's really important and what we’re going to talk about especially in this particular, this week’s scenarios, is we are going to try to stay as far away from IT as possible.

We’re really going to focus on trying to talk about the business, trying to understand input and output, trying to understand processes and where money is coming from and where money is going to. As an IT person, I definitely love tech. I have my home lab for that. When it comes to a business, I'm sure all of you business owners are very much interested in making profit and keeping critical systems that maintain your business online with the highest level of availability as possible.

Just a little bit about me. I'm continuing my education. I did my CISSP a while ago, and that was a good eye-opening experience. I did get my ISACA certification. I just took the SANS class and … Thank you, Ken, I see you out there. Hopefully, I’ll be taking my certification test for that probably in January or February. I passed my VMware certification.

As we’ll see, VMware and the technologies around virtualization can help businesses accomplish a lot of things. Everybody wants to learn how to be more efficient, how to consolidate data center resources and at the same time improve their security posture, which can include availability, include their business continuity and disaster recovery plan. Virtualization, whether you're VMware or not, can really help you immensely with that.

Long, long ago in a galaxy far, far away I actually did … I started off as a network guy. The network guys here always make fun of me because I list it so I’m putting my disclaimer, I did it back in 2003. Yes. If you talk to the current people with their CISCO certs, I think I'm just going to go re-up it so they’ll stop hassling me.

That’s my technical background. Then my master’s which I’m almost done with is in information assurance, which really, really honed in on this love of business continuity, disaster recovery and looking at the business holistically. All right, enough about me.


Steve: Whenever I start off with a presentation I like to have a quote. This is one that I like to use frequently, especially in IT, because I think many, many IT guys have this utopian vision of what their environment should look like. A lot of business people have the concept of five, six, nine and no downtime ever but on very limited budget. I think that this quote is very apt, right? “Have no fear of perfection. You will never reach it.” I think it’s wise.

I think when you talk about business continuity disaster recovery, not everybody’s going to be happy. Maybe your IT guy will not have the exchange server online quite as fast as you would like. Maybe the training department has found out that they're not quite as critical as the payroll department and their systems won't be back online as quick as they would have thought. It’s all a trade-off and you're going to compromise something in order to be effective with the budget that you have. I think this is a really good quote, and Salvador Dali there sets us off on the right direction. Just keep that in mind as we’re going through that.

The reason why I bring up my first place of employment, even though it was a very, very small shop, is I had a personal encounter. I have a really good relationship with all my previous employers. There's a gentleman whose name is Mr. Roma, and I went … I just drove out and I saw him last year, him and his wife. I just wanted to stop by and talk to them and see how they were doing.

When I came in and we were sitting down and we were having lunch, he said, “You know, Steve. I don’t know what I'm going to do.” He said, “My kids don’t want to take over this business. It’s not profitable enough for me to sell it and live on for the rest of my life. I couldn’t really sell the assets that I have and be comfortable for the rest of my life. I'm almost 70 years old. What can I do? I don’t want to work until the day I die. I don’t know how to get out of this situation.”

It really struck a chord with me because this is a personal friend that gave me a start in my career. Maybe some of you can think fondly of those people that gave you a break. Even though Mr. Roma didn’t need a disaster recovery plan, he absolutely needed the business continuity plan. He needed to understand that as time goes by, things in his business will shift either very, very quickly and very rapidly or very, very slowly.

We in the IT field and people in the business field need to be able to anticipate these dangers. A lot of times we know things will come but we don’t think about it or we don’t see them as pressing. Or maybe we think we’ll live forever, right? When I sat and I talked to him and I looked at this really wonderful person in the face, it struck me that every single person needs a business continuity plan. It’s not just about disaster recovery. It’s about the business.

That’s one of the main driving factors which make me passionate about this area. If it’s not you getting older, it’s if a business goes under, it’s people that are out of work, it’s people that are unemployed, it’s people that aren’t getting paychecks. It’s a business that’s no longer in the community. From these kind of humble beginnings, I can definitely see the value in people spending time on this. It’s not a small effort.


April: Let's take a second, Steve, and just clarify. I know that you're still close to it, closer than most of us are to the concept. Business continuity and disaster recovery are related?

Steve: They are, yes. Within business continuity you have subsets. That’s how the evolution of this conversation will be. If you folks have any questions, please pop them up. I like things to be interactive.

Business continuity is kind of the overarching umbrella. It’s the whole system and process of looking at the business for the sake of it being a business. Even if you're a small business owner, you have to step back and look at it outside of yourself. You have to look at it, if you're a small business owner, “How does this business continue if I were hit by a bus tomorrow or if I could no longer work, if I fell ill? How do I protect the business as an entity if I'm not there or if this key piece is not there? What are our plans to mitigate those types of dangers?”

Within the business continuity plan there are certain things that you do. You do a business impact analysis, which we’ll talk about, we’ll do some risk assessment. Then you start to identify different systems and pieces that you need to bring back online. Where will your employees work from? What IT systems are critical? What processes are critical?

You start to dissect in the business almost like a high school biology product, right? Maybe you have the frog or guinea pig on the table and you start taking it apart piece by piece and looking at what the most fundamental systems are to keep the blood going within your organization. Business continuity is the overall strategy, that’s the whole business body, per se, at how you could … It’s a holistic thing. Then you start digging in deeper and deeper and start using these specific tools within the business continuity framework.


April: Okay. Disaster recovery, how do we recover from a specific incident when a failure happens and business continuity is kind of the bigger rap or how does the business continue over years, decades, generations?

Steve: Yes, absolutely. It could be a fire, how do you keep the critical systems online, how do you maintain that, whereas disaster recovery really could be more localized to IT.

April: Okay, great. Thanks.

Steve: No problem. All right, so the big question is why are we here? I will ship someone a six-pack if you know what this is. Does anybody know what this is without Googling it?

April: We need some image searching.

Steve: Yes, some image search.

Steve: Yes, that is correct. What this is is the NOAA image of the tsunami that hit Northeast Japan. Then here is what happened in the more practical application a year or two ago when we had the tsunami off the Northeast coast of Japan, just tons and tons of damage. I did a whole other presentation for a supply chain management organization in how … the differences in Toyota and Honda and how they organized their operational structures. Honda lost 77% of their net revenue that year. Whereas Honda, because they had a more decentralized business model, same general location, only lost 33% of their net revenue because they had structured their organization differently.

April: It’s different?

Steve: Yes. These things … In the next webinar we’re going to get … we’re going to talk about successes as we lead in specifically to disaster recovery. How you plan these things out and how you actually develop your business around these things is really important.

This is an image of the floods in Thailand that drove hard drive prices 300%. This is last year, 18 months ago. This is a little bit more recent, Sandy. We heard of people that were literally carrying buckets of diesel fuel off steps to keep diesel generators running, keep the lights on. These are things that happened that are very, very real, right? Or maybe this is the reason why we’re here.

This is Section 1291. If you're worried about PCI, you have to have some sort of business recovery and continuity procedures. Whether it’s a disaster, whether you're meeting PCI requirements, whether … This is a little easier to identify, this says HIPAA right at the bottom, whether you're under HIPAA compliance. Anything like that, you are forced to or you’re compelled to, when you want to do business in this space, have these types of measures in place.

Maybe you just want to be more effective. You want to consolidate IP infrastructure. You want to improve business continuity disaster recovery posture and upgrade the security environment of your environment. These things are not necessarily diametrically opposed, right? Virtualization can help you consolidate your IP infrastructure. You can consolidate a data center down into one or two racks.

At that same time, you can significantly improve your ability to fail over to a secondary site with a lot of the technologies that are available now in a very short period of time. Just because we have these as action items doesn’t mean they're opposed. In fact, there's a lot in terms of benefit that you can get from virtualization disaster recovery planning and business continuity planning.

April: We’re going to talk about that in the third seminar?

Steve: Absolutely, yes. I just want to start off, whenever we discuss a topic that people may not be quite familiar with, let's get on the same page as far as the language that we’re using. Some of these may be familiar and one of these may be not. Recovery time objective, this is very, very… Can anybody else? Okay.

Some of these may be very familiar. Recovery time objective, what's the maximum amount of time that an application can be done. Recovery point objective, how much data can we lose? At what point is it critical that our business is able to retain a timespan of data? Maybe if you are a company that does orders and shipping once a week, maybe a day, maybe it’s a matter of minutes if you're a bank, if you're a financial trading firm. There’s different tolerances and there will be different levels of accessibility for every different business.

The last thing that most people don’t really think about is this concept of backlog or a backlog trap. The reason why I think in IT that we may have or in some businesses, service businesses may have a hard time figuring out about this is because things are intangible. Seeing people do work is sometimes very hard to measure.

Think about it however from the prospect of a manufacturing plant where if you have a machine that can only chunk out 10 bumpers per minute and you’re down for 10 minutes, you’re down 100 bumpers. If you are already running your production at full capacity, you can never recover those 100 bumpers that you lost in that 10 minutes.

When you look at business continuity and you look at this recovery planning, you have to understand that if you ever do experience a disaster it is essential that especially in manufacturing but even in a service industry, you build in a certain amount of slack time in order to recover the services, the processes that you were supposed to deliver to those customers during the time that you were down.

This is something that a lot of people don’t take into consideration when they’re doing their business continuity and disaster recovery planning. If everybody’s budget shrinks, they want as close to 100% productivity as you can get out of every single employee. That’s great from a management perspective and from a cost perspective, but what happens to your brand, what happens to your customer loyalty when you fail to deliver on time?

These are all things that are not IT decisions, these are business decisions. This specifically is how disaster recovery could be different from business continuity. Now that we have that common language, let’s move forward there. Any questions on that, on backlog traps specifically?

April: Actually, I have one.

Steve: Sure.

April: You may be getting to this later, but is there a guideline for what the optimum percentage of productivity is or am I jumping ahead here?

Steve: I don’t have any hard statistical figures on this but there have been reports in Europe where people effectively are less productive, get less productive after 32 hours of work. I wouldn’t say that you would, “Hey, let’s have everybody work 32 hours,” I don’t think people would be very happy about that, but if you build in certain types of structures like companies like Google has done or you build in time for training within your organization and you allocate maybe 30 minutes a day for self-improvement training or some other things where you can safely remove that off a person’s plate and then put that extra bit of work that was generated by this backlog trap, it will help you fulfill those obligations to customers.

Generally the larger – and this is the really phenomenal thing – the larger the organization is, as long as it’s organized well, the more effective that you can run it. Think about, let’s say, if you have five stores or five locations and plants. If you have five plants, you can run each one of those plants at 80%. If any one of those plants fails, you can divvy that 80% off into your other four plants.

Whereas if you only have two plants and you wanted to be redundant, you could only run each one of those plants at 50%. So you actually gain efficiency the larger you scale out, but this is fundamentally a juxtaposition of what we’re seeing in larger super corporations. They want to consolidate everything instead of distributing knowledge and work and building in this concept of what’s called organic resiliency.

We’ll talk about these concepts of resiliency and efficiency as we get more into the next presentation, into the DR and how we can structure things. That’s a great question.

All right, so this is not a short-term process. I took this right out of … for those of you who are familiar with (ISC)2, I took this right out of the book, the CISSP book.

You are looking at 12 months or greater to develop a very solid business continuity and disaster recovery plan. Inventorying everybody in your organization, if you’re a larger organization, it could take you a month, knowing what they all do, identifying the actual data that’s important.

I remember I was talking to somebody and they said, “Oh, we need to make sure that we have our SMTP relay server in our BC/DR plan, because that’s critical,” and I said, “Well, why? How long does it take to stand up an SMTP relay server?” There’s no company data that is residing on the SMTP server and you can bring an intern in here and get one stood up in probably about 45 minutes.

If you only have so much money to spend on your infrastructure and to protect your data, it’s very, very important to understand what data you have that; A, is really important and that’s really making you money and B, spending your money wisely to protect the most valuable data. Doing that analyzing what records, what data you want to back up and recover, that could be a six-month process. That is really important for businesses if not everything, you don’t have enough money to back up everything and restore everything with the highest priority.

In the same timeline business impact analysis, which we’ll talk about specifically in this conversation, understanding how the business works and where the business would be greatly impacted and what could have the greatest impact on the business is something that most businesses forget. That’s a huge, huge mistake. It’s a huge mistake to skip the business impact analysis, because if you don’t do a business impact analysis or a risk analysis you’re not going to have an accurate baseline to know what you should be recovering first.

After this first six months then the next quarter of your business you may be developing strategy to recover these pieces of data that you have, selecting alternate sites that maybe you’re going to outsource or you’re going to put something in a colo, you can do disaster recovery with a provider like Online Tech. Then, developing the contingency plans. It should be all around the one-year mark.

This is expensive, it’s time consuming but it’s absolutely critical for businesses that need to stay online. Then testing and maintenance is an ongoing effort, preferably quarterly, at least semi-annually.

April: Thanks, Chris.


Steve: Oh, thank you, guys. All right, so I am a very black and white person. When I talk about business continuity, there is a standard. Nick develops a lot of them. When you look at business continuity, special publication 800-34, that’s where you want to go. If this is where … Let’s see, don’t you feel like you could lose some of the input … vital records better. I’m not sure if I totally understand that question. Are we going to have an opportunity for people to ask questions audibly at the end?

April: It’s tough to do that without getting a lot of feedback but we’ll follow up on that question, we’ll get you directly in touch with Steve, so that we can get to any answers.

Steve: Oh, I think I understand. I think I understand what you’re saying. If you’re saying that you run those in parallel you wouldn’t have the information from the business impact analysis to identify the vital records, is that what you’re saying? Okay, so the question was is if you’re doing a business impact analysis and trying to identify vital records that you want to back up and recover, at the same time would you be losing information about the business to identify the vital records?

In theory, yes, you could. That’s a very good observation. What we did, especially at a larger company with ADP, we would have people going in and we would be identifying potential impacts for segments of the business. There’s a product, it’s a well-known product in China. I’ve worked a lot with the eTIME product at ADP, and it was for a very, very large organization. Companies like Hilton, the Army, the Marines use it.

What we would do is we would go through and we would go into those specific organizations. It’s essentially an interview process. It’s an interview process and you would watch what they were doing and you would talk to the business owners. By talking to the business owners, you’re able to look and see what processes they’re using. You can say, “Hey, I’m seeing these people use this workflow or this set of documents or this database every single day,” and then you can say, “Where is your revenue coming from?” If it’s from this particular section of the business or if it’s a smaller business, you can tie that together.

If you were to look at it as a segmented process, yes, that is an astute observation. If you’re to look at it as a holistic process and pull out documents or pull out databases or pull out systems that you’re seeing people use frequently, you can very much generate an inventory as you’re going through and then at the end of the five out of the six-month process you could sit down with the business owners and say, “Based on the interviews that we’ve done, based on the observations we’ve done this is what we’ve seen you use on a day-to-day basis to generate revenue.” Then it would be the business owner’s responsibility to quantify and match the income with those process. Does that make sense? Okay, all right. Thank you.

April: Great question.

Steve: That’s a phenomenal question, yes. Getting into this, Nick, and if you want to read more about this, this is open; it’s special publications 800-34. If you Google it it’ll be a PDF, but essentially there are seven steps in this process. What we’re going to do is we’re going to get into the first four today, very, very lightly. We’re going to do the first few. We’re not going to touch an IT contingency plan today and hopefully, you all get a little chuckle out of the company that I chose. I tried to pick the thing that was the furthest from IT that I could, because I want to stress the importance that the initial phases are all about the business, it’s not about IT.

Just going down the list, developing the contingency plan and policy statement, conducting the business impact analysis, which is huge, identifying preventative controls and recovery strategies. That’s where we’re going to stop today. Then in the next one that’s where we’ll transition into develop the IT contingency plan in the next webinar, and how we can talk about that theoretically. Then the third webinar will be very, very specific technologies that even small businesses can use to develop these types of strategies. Every testing and training exercises, that’s a function of exercising your business continuity and disaster recovery planning and the plan maintenance.

All right, some very critical questions to ask, how do we know what to recover, how do we know … How do we make these wise decisions and how much money and time do you spend on recovery and what’s the single most important thing you have to have? The support of your executive body. If you can’t get funding for this, it will be nearly impossible to do this successfully.

That’s why very specifically in the next webinar I want to give you examples of companies that did an outstanding job in their business continuity and disaster recovery planning and saved public image and saved money because they did their due diligence. That will be coming in the next webinar. These are really very important points to ask yourself as you’re going through this process.

All right, so a lot of people when we talk about risk – we’re going to start getting right into the meat of it now – it’s about managing risk. A lot of people talk about risk mitigation. You can’t always mitigate risk. You can manage risk.

There’s a couple of different things that you can do to manage risk. You can avoid it and you can say, “Maybe I’m in one line of business and I see this opportunity for me, but this opportunity opened me up to a lot of vulnerability.” Maybe I could get a manufacturing plant in China but I’m not comfortable with the political unrest there and what I’ve seen going on in the Foxconn plant. You could avoid a certain line of business because of risk.

You can mitigate the risk, which is what most people are familiar with. It could be something like a fire suppression system mitigates the risk of fire. You could transfer risk, and many of us are familiar with that. Auto insurance, it’s transferring risk. Then, if you look at your options, you can accept risk. This is a very viable option if you’ve thought about it, if you weighed and you’ve done a cost benefit analysis. If you have $100,000 asset and it costs you $75,000 to protect that asset, you might just choose to accept the risk because it’s too expensive to actually protect the asset. It’s probably the least favored option, but with good rationale. It is an option.

This is going to be our little case study that we’re going to use, and this is an actual company. I’ve never been here. I saw them on the Food Network. What it’s called is the Sourdough Sunrise Bed and Breakfast. It’s a small bed and breakfast.

They have a structure that was built in 2005. It’s not anything historical. I don’t really know how many rooms they have but we’ll just say 10 for easy math. The rooms rent out at 150 bucks a night. You can actually go to their website if you’re visiting Alaska. Again, I have never been there. I have no affiliations with this company. I just couldn’t think of anything else that was less technical than sourdough bread.


April: You were probably hungry at the time.

Steve: Yes, so this is the small business that we’re going to use. All right, you may be experiencing network connection difficulties.

All right, let’s talk about some of the asset or the features of this company. They have the oldest sourdough starter in Alaska. This is true. Again, I saw them on the Food Network. Their sourdough start is 105 years old. I didn’t know sourdough could last that long, but this is their claim to fame. They have this sourdough starter that has been around for 105 years old. They’re fairly famous so they have brand recognition in the sourdough community, I guess, if you want to talk about that.

The proprietors, these are the actual names of the proprietors, Rich and Sue, and the property is located to several major attractions around Alaska. This is our non-technical business that we’re going to start talking about business continuity with.

When we’re getting ready to look at a business, doing the business impact analysis, this is really where we’re going to start dissecting the organization and where we’re going to dissect how the organization runs, what the dependencies are in the organization and even dependencies outside of the organization.

If we were to go in and we were to look at this bed and breakfast, the cabin is the physical structure. Maybe it’s your facility, maybe it’s your office, maybe it’s whatever. Their physical facility has a dependency on external factors. They are dependent on geographic location. They are dependent on their environment around them.

You could very easily draw this tie in to the BP oil spill down in the Gulf. There were fishing industries and tourist industries that are absolutely dependent on the geographic regions around them. Your business may be affected by that as well. They also have external factors, the SeaLife Center, hiking and fishing in a national park, which they have … they don’t have direct control over, so how did they mitigate and plan this.

They have relationships that they’ve built with their customers, and a lot of those relationships are built on story, the rapport, the history of the sourdough start. You can see we have some interdependencies here and some models that we need to look at as we’re going through and looking at this organization.

Maybe if you’re a small business maybe yours looks something like this. You have facilities, you have IT, you have sales and delivery, you’ve got production, finance, office operations, management of course sits over all of this, and you have compliance suppliers and critical customers that are external factors for you. Maybe this is for a small business. This is maybe what your model would look like.

It may be very helpful as you’re doing a business impact analysis to literally sit down and draw the interdependencies if possible within your organization. If you’re a really large organization you’re going to find this very hard. The harder you find it to make one of these diagrams, the harder it’s going to be to develop a successful business continuity and disaster recovery plan. Because in order to fully understand the business, you need to understand the relationships that the business departments have within one another and you need to understand who is supporting who.

If you sit down and you do this and you find this is very difficult, that may be a sign that you need to look at your organizational structure and try to align business processes or workflows more neatly. This could be an indication that you could actually drive operational efficiencies by realigning certain strategic tasks or objectives so you can get a better understanding of what’s actually going on with the business.

What we’re going to do is we’re going to use this sheet. This is something that you can see down at the bottom, it’s ready.gov. This is something that FEMA puts out. What you would do is you would start to use these sheets to analyze different sections of your business, or if your business is larger, different processes within different subsections of your business, and you would look at them in terms of time. As you can see in the bottom left, here you have what would be the impact of less than an hour, an hour to eight hours, 8-24, and you would really start to breakdown the type of impact that certain outages would have for you and your organization.

You can see there’s three columns. Maybe, like you see in the second column, it’s not initially a financial impact but what it could be is customer dissatisfaction or defection, which by its nature will cause financial loss. Or it could cause you to delay a strategic initiative that you have in your business, which means long-term loss. Then in the final column you would try to quantify what those long-term losses were with actual dollar values.

As we move forward, we’re going to take a look. This national park is something that was critical to driving customers to this bed and breakfast. If the national park had a bear scare, there’s a grizzly bear running around the park and they had to shut it down for an hour, the bed and breakfast is probably not going to see any significant impact. There’s really nothing that they need to do. Even at a day, if the park had to close down for the day because of the other things that the customers and the visitors of Alaska are doing, the impact may be minimal.

People maybe a little bit frustrated but maybe they’ll choose to go hiking or go kayaking or something like that in order to compensate. If you start to stretch out to a week and it becomes public, maybe people will stop booking trips to the bed and breakfast at that point. Based on the revenue models and things like that for a week, 10 rooms and whatever, 10%, that would be $840 that this company would lose per week. If it goes off to a month, the park is going to be closed for a length of time, that could be $16,000.

How you start adding these things up based on what your customers are telling you of why they’re doing business with you, what they like about you; you can very specifically start to calculate how much money you’ll lose based on how much downtime.

If this park is going to be down for a month or it’s going to be closed permanently, if this business was depending on that park for 10% of the reason why people are coming to their bed and breakfast, not only would they be cutting 10% off the top but the lack of the national park there could start to impact other local vendors and decrease the overall ability of the geographic location to be desirable for people to come to this location, if that makes sense.

It’s all about understanding the short to long-term impact and being able to try to put some dollars around this impact. As you see down in red, we can kind of go green, yellow, red. When you start hitting loss of sales and also being able to identify if it’s hindering you from executing future business objectives, that’s when you really start to run into serious problems. Looking outside of your organization, long term, that’s really, really critical.

April: This will help inform the prioritization.

Steve: Absolutely, absolutely. This might be a little easier to understand. So we got a cabin fire. If there’s a small fire in the kitchen and you put it out and there’s going to be some customer dissatisfaction, maybe there will be some small financial impact but it may be very hard to quantify. If you have to shut down this location for a whole day based on the calculations and … I can put them out there, you would lose $1200 for this particular business per day in the peak season and 225 when off season.

When you’re doing your business impact analysis, it’s really important for the company to know, “Hey, these are my peak operating times.” Maybe it’s time of day, maybe if you’re a retailer maybe it’s time of season. If you’re a tourist industry, maybe it’s during the summertime.

You need to take these things into account and say, “In peak travelling season I would lose $1200 a day if I had to shut my bed and breakfast down. Off season, it would be $225 a day because of the booking rate.” If we’re down for a week, let’s say there was a major … some plumbing issue or something like that, you’re looking at $8400 per week at peak, $1575 off.

Then again, if you’re extending out to a month, this is when you start to see real serious pain. If you’re looking at taking this organization out for a season, again you start looking at loss of sales, income. If they’re looking at, “Hey we got a spot on Food Network. We got a lot of referrals from previous visitors.” Again you get into the area of people that are going to stop referring business, they’re going to maybe say, “Yes, we tried. We couldn’t book. They were unavailable,” and it hurts your business image if that make sense.

This maybe a more traditional … fire to a facility may be more what people are thinking about traditionally, but thinking about external factors here is also really important because external factors definitely affect the business. Any questions around that area?


April: Nick wants to see where the timing really impacts the level of risk and assuming what you’d expect to do about it?

Steve: Absolutely. I think in my next slide or one of the other upcoming slides, we talked about backlog traps and how this fuels in. If you go down in a peak operating time you may never ever … you may never be able to reclaim that revenue. What we’re looking at, how do we want to do certain things? This all goes back to the backlog trap that we were talking about earlier.

What I wanted to show is when we start hitting those the lines of business where it starts impacting our future plans to do business, this is where we can see…

So in the orange, we see our profits going, our revenues going up. Then in blue, this is where we start experiencing downtime. If we’re going up we’re conducting business, we’re generating revenue and then we have an outage where we see that blue block start. Let’s say this outage is for a day, a week, a month, a year. It could be very different, depending on your business, and then you start to come up.

If your outage affects your business brand, if it starts affecting the long-term viability of your business, you can’t execute business plans, then we’re not going to grow at this same kick. You can see that the projection of the blue line, it’s … when you recover it’s not parallel. When you start getting into these situations where you’re affecting your brand image or things like that, you actually lose the trajectory that you had to generate revenue, to generate profit, to generate and execute those plans.

This is something that’s really important to think about. Mostly people say, “Oh, well if we do $52,000 a year in business and we’re down for a week, we’ve lost $1000.” That’s really not how you look at it. As we talk about calculating loss, there’s some very specific formulas that we can use. These you can take and you can look and you can plug these right into your model.

We have something that’s called annualized loss expectancy, ALE. What ALE is, it’s the annualized rate of occurrence. Let’s say the likelihood that your building would catch on fire. Maybe you could get those statistics from a fire department, from your insurance agency, and then the single loss expectancy. The single loss expectancy, the data we need to calculate that is we need to understand the asset value.

For example, for this bed and breakfast, I just did a quick Google search on homes or cabins in Alaska. Let’s say a 10-bedroom cabins is $280,000. We estimate that if there was a fire that was more than just a little kitchen fire occurred, we would lose 65% of the cabin. It’s what we call exposure factor. You would multiply that $280,000 times 65% of the cabin and you would lose $180,000 of value. That’s how you can calculate what your single loss expectancy would be.

Now you return to that annualized loss expectancy formula and you say, “Okay, well we think that there’s a 3% – and I didn’t call up the insurance company on this – there’s a 3% chance that we may experience a fire this year.” We’re going to multiply the single loss expectancy of $182,000 times 3%, which gives us $5460.

This should be our target number to make sure that we’re covered in this area. We would want to make sure that to cover this value, we hopefully are spending less than $5400 in insurance, right? Because if we’re spending more than this, we’re really over spending for what our actual rate and cost of loss could be. Does that make sense to everybody?


April: Something a lot of business owners would want to know before they buy insurance or create a second site or invest.

Steve: Absolutely, absolutely. A ton of money, right? These are very simple formulas. I am not an accountant in any way, shape or form, I’m not even a great mathematician, but I can do these formulas so I’m sure you can all do them, too.

What we want to talk about now, we understand what’s going on and we can’t fully avoid a cabin fire. We can’t fully avoid it. You don’t want to accept it, so what are we going to do is we’re going to look at mitigating it and we’re going to look at transferring some of that risk in order to manage the risk that we’re facing. What we’re looking at now is we say, “Okay, well, we’ll do $280,000 in property insurance.” People might have heard, especially with Sandy going around, something called business interruption insurance.

If we calculate that there’s X amount of months that it would take to rebuild this structure and maybe it’s going to take $280,000, we need to calculate … let’s say if it takes us five months, the five months of peak season for our business to rebuild a cabin, the worst that we would be negatively impacted would be $195,000 of our annual business. You need to add your $195,000 of business interruption insurance to the actual $280,000 let’s say if the cabin was completely wiped out in property insurance to get you about $475,000 of total insurance annually.

Now I will say this, I’m not an insurance agent but as I was doing a lot of research on business interruption insurance, it’s very, very important that you ask specifically how much money you would get. The reason why I’m bringing this up is because there’s a lot of people that had business interruption insurance with Sandy and they found that what they actually got after their copays and after all these other lovely things that insurance folks like to write into their policy is they got significantly less than what they actually did.

If you are looking at business interruption insurance, make sure that you make it clear, this is the dollar that I need to get after all of my premium and my copays and all of the fees and all of the other jazz that you’re going to get before to make sure that your business is covered.

Mitigation, we got our transfer, now we want to mitigate. We want to make sure that our smoke detectors are up-to-date; we put new batteries in them every year. We have a smoke free property. Don’t smoke on the premises. We supply and maintain all of the fire extinguisher on all the floors. These are very simple mitigating things that people can do and that are probably very common in your environment.

What’s the practical application of this; the first and the largest thing that I can drive home, I’m an IT guy but I would say maybe don’t bring your IT guys in the room when you first start talking about this. We get really excited and we start thinking in terms of servers and circuits and my beautiful VMware cluster that I don’t want to go down, but you really need to think in terms of the business, the processes of the business, the revenues that you generated by the business. We need to identify the sources of income that are being generated within your organization because the biggest things that you need to protect are your sources of income.

Obviously I’m just going to say this, that comes second to human life. You want it always to make sure that you protect your employees. But from a business perspective, no. When you’re talking about business continuity and disaster recovery, you want to make sure that you have a DR plan for your most critical business assets in your business process. If you’re a large organization, simplify this process by breaking it out by vision, by organizational unit, by geographic location. Start off with one and see how it works there and break these things out into small manageable chunks because if you do that, it will make the whole piece easier.

Definitely you want to map out the interdependencies within the organization. If everybody feels that they rely on IT but IT really relies on facility, you need to understand how that trickles throughout your whole environment, which gets back to the interdependency map that we showed earlier. We want to map income to department, services and processes, and we need to identify what systems are most time-sensitive. Maybe your payroll department, they only process payroll once a week.

The very first place that I ever worked, getting back to Mr. Roma, we got all of our credit cards on Friday, so we had them all … hey if the credit card processing system was down Monday through Thursday or Saturday through Thursday, for us it wasn’t really a big deal. You may have important systems in your organization that aren’t time sensitive, and understanding the differences between those two things is very, very key. Whereas emails may be very time sensitive, WebEx may be very time sensitive, but not mission critical for the business. So learning to think in those two different mindsets are very key.

Next week we’re going to transition a little bit from the theory and the thought process, do some practical application. I really … especially all of you who are trying to make a case for a business continuity and a DR plan, I want to give you some ammunition that you can go back and talk to people and say, “Hey, we’ll talk about a big case study that happened at LaSalle Bank in 2004, the second largest high-rise fire in Chicago and LaSalle Bank did a phenomenal job in their business continuity and disaster recovery planning.”

We’re going to talk about that. We’re going to talk about staffing and facilities resilience a little bit. We did that macro-efficiency and micro-resiliency concept that I mentioned earlier, process design, an organizational design, facility design that allow you to recover quickly and then we’re going to start talking about IP tools to develop and to deliver all of these services that you’re looking for. Any questions, April? Any questions from folks out there?

April: I don’t have any more at the moment but let me encourage everyone to feel free to submit questions. We will send an email to everyone who registered and we will let you know how to send a question. We will also be posting a recording of this webinar and with Steve’s permission the slides.

Steve: Sure.

April: We look forward to meeting all of you back here in two weeks, Tuesday at two and we will have part two of our three part series on business continuity and disaster recovery. Awesome, thank you Steve.

Steve: Thank you all for joining us.

April: Thanks everyone, we’ll see you in a couple of weeks.


About Steven Aiello

Steven Aiello is a Systems Support Manager with Online Tech, the Midwest’s premier managed data center operator. His certifications include CISSP (Certified Information System Security Professional), ISACA CISA, VMware VCP ( VMware Certified Professional), Cisco CCNA ( Cisco Certified Network Associate), Comptia Security+, and Certified Incident Responder (New Mexico Tech).



Webinars    |    Online Tech


Get started now. Exceptional service awaits.