Podcast: Avoiding the Most Common Errors and Mistakes that Can Have a Debilitating Effect on IT Infrastructure Performance and Budgets, Part 1

Trouble ahead?

Nothing can lead to the updating of resumes and the decline of an organization’s fortunes faster than difficult-to-correct mistakes and errors in their IT infrastructure. In over a decade of consulting with large organizations on their data center, cloud, and IT infrastructure strategies, Jeff Gilmer of Excipio Consulting has helped organizations clean up a lot of mistakes, as well as avoid future errors that can inhibit the achievement of business goals.

Does your organization have the right answers to the following questions?

  • Are the infrastructure goals and strategy (assuming they exist) mapped to the overall goals of the organization?
  • Is everyone in the company (finance, IT, operations, legal) contributing to the process and have they reached alignment in their thinking?
  • Is your organization falling into the trap of letting the provider dictate the solution, settling for a company’s product offering even though it is not tight fit for your IT infrastructure needs?
  • Do you have the proper resources for a migration, and are you properly staffed for efficient operations following that migration?

This is just a sampling of the issues that are addressed in this discussion. So much damage can be done by not making the right decisions with your IT infrastructure, and Jeff does a good job laying out common problems that need to be avoided. In fact, Jeff had so much to share on this that we had to break it into two segments, with part 2 going up later in the week.

You can listen to the conversation with Jeff in the player, and/or you can read the full transcript beneath the player.




Kevin O’Neill, Data Center Spotlight: This is Kevin O’Neill with Data Center Spotlight, and today we’re talking about common mistakes organizations make regarding their data centers, their cloud computing, and their overall IT infrastructure, and we’re talking today to Jeff Gilmer. Jeff, thanks for being back with us again today.

Jeff Gilmer, Excipio Consulting: Yeah, thank you, Kevin, it’s always a pleasure to speak with you.

Data Center Spotlight: Well, it’s great to have you with us, and Jeff is a senior partner at Excipio Consulting, and Jeff, I would describe you as an IT infrastructure and data center and cloud strategist. How do you describe yourself at Excipio Consulting, and what you do there?

Jeff Gilmer: Well, at Excipio, what we are is a third-party advisor, so we are completely unbiased and we help organizations understand their current environment. We provide some comparative analysis of benchmarking for them, so they can see how other people and other organizations are performing the functions that they are within technologies and compare that to their organization, and then we do a lot of work with evaluating future potential solutions. In the data center world, that could be a cloud computing solution, it could be a colocation, it could be a managed colo, it could be an outsourced migration, disaster recovery, there’s endless things of different issues happening within the data center environment today. We seem to get our fingers involved in a lot of those, and help our clients understand the right direction and the proper direction before they move forward.

Data Center Spotlight: Well, Jeff, when it comes to data center and cloud and IT infrastructure errors and mistakes, and strategy that doesn’t quite work out, there’s a lot of meat on the bone to chew on, so why don’t we just get right to it, and why don’t you tell us what’s the first common mistake or error companies in all forms of organizations, I know you do a lot of governmental work – what’s the first common mistake they make regarding their data center, cloud, and IT infrastructure that you’d like to talk about today, Jeff?

Jeff Gilmer: Well, there are lots we could put at number one, I’ll just kind of grab a couple to talk about that I think we see more frequently today.

One of the more common ones is, the organization has grown up, and you’ve got people that are evaluating the storage, and the compute, and the facility side, and some are looking at cloud solutions and applications, and they really don’t have a solid strategy specifically for the organization, and within that strategy from an IT strategy, they don’t have a specific data center strategy. So you’ve got multiple people looking to do multiple things, going different directions and providing different areas. You may be familiar with that, Kevin, just take a look at people that are looking to go to a provider, and you’ve got the compute side looking at one provider, and maybe you’ve got somebody looking at a cloud provider, maybe you’ve got somebody looking at a facility provider. How do you select the right provider is a common area one.

Data Center Spotlight: That happens a lot, and not only how do you select the right provider, but a lot of the providers are trying to send you in different directions, some of the people involved in the decision are from different sides of the table. We’ve talked in the past about sort of the divide between the operations, and the IT operations staff, and the IT staff, and I would imagine that the entire decision-making complex as far as what you even need gets pretty complex.

Jeff Gilmer: Yeah, it does. So let’s take some of the highlights around why you need a data center strategy, right? Let’s just talk about what you’re going to do, so that you don’t get into an inaccurate contract, a contract that doesn’t meet your needs, or you select the wrong provider, and now you’re stuck for five years, or you made a migration for that data center provider, and now you have to move again. You definitely do not want to move a data center due to the risks of it, right?

So common issues, they don’t understand from a business side compliancy issues, or regulatory issues, migration issues, and just day to day operations and the impact of those operations, and will this new location, or will this new provider, or will this new solution meet all of our requirements from a compliancy standpoint? Even within compliancy, depending on whether you’re public sector or private sector, you’re going to have things such as PCI compliance. You might have FISMA issues, you might have CJIS if you’re handling any sort of data around criminal justice information, or working with anybody within that type, which could be attorneys, it could be examiners, it could be obviously public sector with police, fire, those areas. You could have HIPAA compliance issues, and on and on, and then there’s regulatory issues, as you start to get into the financial and the insurance world, or utility companies, anybody handling or working with utility companies, or working with the government. There are regulatory issues you may need to understand.

And there are providers that yes, they may need some of that, and some of those do fall back onto the client and how they handle the data, but that’s just one area within that. There’s several others as well.

Data Center Spotlight: Now, Jeff, let me ask you a question. Do you run into within these organizations, are there people within the organizations that are involved in the decision-making process who are sort of entranced by the shiny new toy, as opposed to a proven solution, which I would imagine is a big part of what you guys are trying to advise people on to deal with solutions that are proven, that have worked in the past, that you know can solve their problems, and position them well for the future?

Jeff Gilmer: Yeah, I mean, it’s of course a lot of the providers there have what we call their hook, but it is kind of their hook because they want you to come out and see their facility, right? And you go out and you look at that facility, but if I was to ask the majority of people that go and tour the facility, and they look at what is in that facility, do they really know the quality of it? So, I’ll give you some different examples. I was in one facility, it was a brand new data center, and I happened to go there with the client, and they gave us a tour, and it was beautiful. They a secure gate before you could enter, they had the mantraps so that you couldn’t go through, they had biometrics to get into certain areas. They had their UPS system compartmentalized off, and we walked through the power system, and we saw the generator, and we saw the other aspects of it, and we went back up front, and in their marketing materials it stated that they were a Tier 3 data center with multiple redundancies, all the way throughout the facility. And the client saw this brand-new building, and everything looked fabulous, and it was pretty, and shiny, and had work areas, and saw some of the compartmentalization of their areas, and thought, yep, Tier 3, we want to come here.

Well, there were multiple things I didn’t bring up. When we left that building, first of all, it was only a single generator. Second of all, it didn’t have an automatic transfer switch on the generator, it was a manual switch. So right away, there’s two things that move that to Tier 1, and this happened to be a significantly financial client, who was going to have to recover things within minutes. You can’t have where oops, it goes down, I got to run back and manually switch the generator off, as an example. It didn’t have redundant cooling units within the facility. So, if the cooling went down, there was no secondary or backup redundant cooling with it, yet they were portraying in their marketing that they were a Tier 3 data center. They were not even close to that from an Uptime Institute standard, where you start to compare redundancy.

So, my point behind this is, when you start to put your data center strategy together, there are things around how long should the contract be? Where should we spend capital? Where should we spend expense? How do we size this appropriately? What are the different requirements that we should have in that contract? What’s important in that contract? What about uptime? Can the facility actually maintain the uptime that they’re committing within the contract? We find that to be a significant difference for uptime standards versus what they really are capable of doing, and it really comes down to that most organizations, I don’t mean this the wrong way, but they have a lack of understanding of what their true IT strategy is, and what are the impacts of that strategy before they pursue a data center strategy in its entirety. You need to be educated, and you need to bring those requirements to the forefront in your strategy before you can get onto the next step, which is then starting to go look at those providers.

Data Center Spotlight: Jeff, it sounds like we’re talking about a lack of strategy here, and a lack of understanding as far as what their requirements are, and without having an understanding of their requirements, that means they really can’t get a valid strategy together.

Jeff Gilmer: Yeah, that’s exactly correct, and it’s easy to do, because you can go research on the Internet today, and you can find numerous data center providers, and of course it’s their marketing, and some of it is factual, some of it may be slightly stretched, some of it may not be completely accurate for all their data center locations, and you really start to make decisions by what you read and what you gather off the Internet, and just so you know, everything on the Internet is true, right? You start to hear they’re making something as complex as a data center decision, you really need to work with people in the industry who understand what the providers can and cannot provide, and build your requirements, build your strategy, build it around what your organization needs, and then take that out to the providers.

The reality is, if you have those requirements accurate, you’re going to find two or three providers that meet your requirements, and can meet your needs. Don’t just limit to who you like, who has the nicest building, who has the prettiest marketing, where it’s at. Some of the best data centers have been running for 15 years, and all the bugs have been worked out of them. That might be your best choice than the brand new one down the road that’s 12 months old, that maybe only has two or three clients in it, and doesn’t know what some of the issues are yet.

Data Center Spotlight: Sounds like the data center version of you don’t want to buy a new car in its first model year.

Jeff Gilmer: Very similar, very similar. Let a couple of other clients test it out and make sure everything works before you jump in with both feet, right?

Data Center Spotlight: That makes sense. All right, so we’re dealing with an incomplete strategy that is not based on the facts and the requirements provided, and maybe just even a little bit of sloppiness here, Jeff.

Jeff Gilmer: Yeah, again, it’s due diligence. It’s great to gather that information, trust what’s there, but then second guess it. Whenever you get into a contact, you always want to look at the worst-case scenarios, right? Your attorneys are going to tell you that. Well, look at the worst-case scenarios of, does this data center really meet the requirements that we have for our organization?

Data Center Spotlight: All right, so the first mistake that we’re talking about is a lack of a strategy and/or a strategy that is pursued or implemented without doing due diligence as far as the providers and the solutions are going to be used. That’s our first mistake.

Jeff Gilmer: Right.

Data Center Spotlight: All right, what’s next, Jeff?

Jeff Gilmer: Well, let’s kind of elaborate on that. So, then they now have to find requirements from all this marketing material, or maybe a couple of tours to data center providers, and talking to some of them, and now they go up and they rely on those data center providers to provide their solutions for them, and they pick it based on the data center provider’s solution, rather than picking the provider based on the requirements they’ve defined in the strategy of number one, right? So, number one leads to a problem with number two. Number two is, because you have a poor understanding of your own requirements, you have a tendency to lean towards that nice facility, or lean to somebody who maybe has a client that you know, or a competitor of yours, or someone else within that facility, or this provider, boy, they have a nice flowchart drawing of how they’re going to do our migration plan. There are a lot of different things within that, but you really have to go through those requirements and properly identify them, and we can talk about each one of these in a little bit more detail if you’d like, Kevin.

Data Center Spotlight: Yeah, let’s do that.

Jeff Gilmer: Okay, let’s go back and talk about understanding all the requirements, and really getting a feel for those requirements. So, instead of looking at just that nice facility, or that solution, or how they go about things, stop and take a look at it from your perspective. Do we have the ability from our resources, from the understanding of our technology, from an application inventory, from mapping those applications to their dependent applications, to looking at the compute infrastructure requirements. Do we have all of those things in place where we can go and talk intelligently on our end, and really know if this data center solution will meet our requirements at all? And you break that into just compartments, those really three parts, when you start to look at that from a data center provider, right?

You’ve got the, will the solution be a good data center solution for us, number one? Number two, how solid is this provider at building a complete migration plan, and reality is, at the client side, the client needs to really have somebody checking and building their migration plan themselves. So the provider may provide their parts of the plan, you have to have your own parts as a client, and somebody’s got to pull all that together. And then the third part is how solid are they from a redundancy, recoverability, and from a disaster recovery standpoint, meaning, we may have to go to a second provider for DR. We may have to use our own facilities for disaster recovery. We may have to work with them with a second facility within that old provider for disaster recovery, and it doesn’t do you a lot of good to pick a great production facility, if you have no recoverability in your final solution for your strategy, which all goes back to number one that we’ve just talked about, right? You’ve got to have that whole strategy, so you go there with a solid understanding and look at those three providers.

Long term, you’ve got to run production here, is it a good facility? Is it a good scenario? Can I get monitoring, can I get management, can I understand what’s going on? Number two, how well are they going to migrate me in there? Can they do it solidly, can they do it with very little downtime, with very little failures, in the timeframe that I need to do it, because you want to move a data center quickly. You don’t want to move over 12 months or 15 months, because you’re going to be running into duplicate application issues. And number three, can they provide the recoverability that we need through some sort of solution that meets our recovery requirements?

Data Center Spotlight: Okay, you know what you’re talking about here, Jeff, you do a great job of not giving a hard sell for Excipio when we have these discussions, but I have seen some of the evaluations that you do. I am aware of the process that you go through, and how intricate it is, and I think what we’re talking about here is not only a lack of understanding, but just a lack of process, and I think that’s where you guys bring a lot of value to the table, because you have seen a lot of mistakes made. You have seen people not have a complete process, and the results of that lack of completeness, and lack of thoroughness causing problems for people, and you’re really talking about companies that don’t have the proper process and an established process in place, and that leading to problems, aren’t you?

Jeff Gilmer: Yeah, so without getting too much into an Excipio promotion here, because there’s other people besides Excipio that perform this work as well, but here’s the reality of it. You’re going to go into a data center contract, and even a small data center contract over a five year period is going to probably be in the millions of dollars, by the time you add in the network connectivity, by the time you add in the compute infrastructure, by the time you add in the facility, the monitoring, the management, and all the other factors that go along with it, security, and on and on and on, right? Well, taking a look at what it would take to do this strategy, and having an organization like ours, or another come in and help you with the strategy, you’re not going to spend even 10% on a million to do the strategy, and the reality of it is, we’re going to find probably more than that 10% that that client spends to build the strategy in savings and other issues and contract negotiations, and picking the right provider, and doing the right terms, and strengthening that organization through lifecycle management issues, and asset management issues before they make that migration to the new facility, you’re going to recover that and more by really utilizing what’s out there, the people that have done this, the other people that are really in an unbiased situation, that are acting on the client’s behalf to really make sure that this migration happens effectively.

And a big issue is without failures and downtime, because you don’t want that first migration wave, you’re going to move to the new data center to fail, and now your company’s down, it’s not going to make you look good or anyone within the company look good, and there’s probably lots of revenue, could be clients, could be other issues. It’s definitely a resume generating situation for you if that happens. So, spending the time to really look at it from not necessarily the provider, but somebody who’s going to be unbiased on your side is real critical when you start to make a data center decision.

Data Center Spotlight: The downside of not doing this properly is obviously immense. I mean, you talk about being a resume generating situation, but infrastructure issues can more or less bring down entire companies, and companies that are larger, I mean, Delta just announced fairly recently in the past month or so that their data center outage cost them $150 million. The downside of making mistakes in this realm and having an incomplete process and not understanding your requirements and going into it in a haphazard way in any way, shape, or form, that downside is really staggering.

Jeff Gilmer: It is unbelievable, and the reality of it is, it’s interesting to understand what some of those effects are, we can talk about those at another time, but the majority of them are what you brought up earlier. Their lack of process or uneducated people performing the migration, who have not performed a migration from a resource standpoint, or from looking at it overall, not having the documentation, not having the ability to recover, not having the fallback procedures in place, on and on and on, all of these things that fall under that kind of process and procedures that you brought up a few minutes ago.

Data Center Spotlight: Well, I also imagine communication is an issue, because as we’ve talked about before, we touched on earlier in this conversation, there can be a divide between the operations and facilities group and the IT group, and additionally, I’m sure you’re seeing the CFO involved in a lot more of these discussions. So you’ve got the financial area of the company, the operations area of the company, the IT area of the company, all probably have somewhat different goals, all probably question what some of the motivations are, and do we need to be spending this money on this? Do we have to put so much focus on this? Have we properly looked at that? I would imagine that communication can be quite an issue as well, Jeff.

Jeff Gilmer: Yeah, absolutely, and let’s take that to kind of what I would call probably our third issue here, right? Our third issue is now is active resources, and lack of resources, and not so much lack of resources, as lack of understanding. You just mentioned several resources, you mentioned facility, you mentioned compute and technology, you mentioned financial. Well, let’s just step back and look at that for a minute.

A lot of organizations expect their resources to do the migration to the new data center, but let’s wait a minute here. Those resources are running your facility today, they’re running your compute infrastructure within that data center today. How are you going to pull them off of what they’re doing in their day-to-day activities to now just go and develop a migration strategy, test that strategy, pilot that strategy, and do the migration wave at the same time they’re running your business?

It just doesn’t make sense, and then, let’s just think about this. If you take them away from the daily support issues, and a server goes down, and now you don’t have them on server because they’re migrating to recover that server, or they miss an upgrade, or they miss a security patch, or they miss X, Y, and Z along the way of what they should be doing because their resource demand is over there, now you’ve compounded it. So, I’ve got Bob over here, my server admin, and we’re expecting him to run the day-to-day, complete all the support, complete the upgrades, do the security patches, make sure that you make the migration effectively, from a virtualization or transfer over or physical or whatever it is, and then if we have a failure that Bob’s expected to recover that at the same time he’s trying to do all the other work, it just isn’t feasible. You’re just setting yourself up for risk. When you take a look at what the cost really is for those resources to help you make that migration, it’s really miniscule, but it is insignificant based on the risk you just brought up of a client example of Delta Airlines, and what really can happen to them if you’re not truly prepared. Spend the time; spend the energy to get resources that can help you meet that work effort.

Data Center Spotlight: Okay, well, Jeff, I know you and I had a discussion, and I took some notes about what we’re going to talk about today, and you know we like to leave these things to something usually right around a half hour, not much longer than that. I think this has turned into a two-part podcast, would you agree?

Jeff Gilmer: Certainly, if you’d like to get into more, there are plenty of other opportunities we can discuss.

Data Center Spotlight: Yeah, let’s have this be part one of a part two, but why don’t you maybe take us into the, we still have a couple of minutes, Jeff, why don’t you maybe take us into the next thing, or maybe put a bow on this and set us up for next time?

Jeff Gilmer: Yeah, so I think, if you start to look at other issues out there, and things that would be good for us to talk about, Kevin, one is, let’s talk about what really is the appropriate time to develop a data center migration plan, if you’re going to move to one of these providers, and how long should it take you to make a migration? Should it be 3 months, or should be 15 months, and what are the pros and cons of that? We talked a little bit about disaster recovery, when you start to select a new data center, and that’s one of the biggest issues we see as people forget to take into account the ability to recover, and how do you structure that? We can talk a little bit about that, and then let’s throw the new one out there that everybody hears day in and day out is cloud. How does cloud come into play, and can I close my data center, and move everything to the cloud? I can’t tell you how many times in a week we get asked that question. I just want to close my data center and move everything to the cloud. I would say, very little conceptual understanding of what that means, and what that would take, and can it actually be accomplished today? We do have some clients that have been able to accomplish, we have far more that it’s not even a consideration; it’s not even close when you start to look at their strategic timeframe of being considered. So, those are three other topics we can get into in more detail if you’d like.

Data Center Spotlight: Okay. Well, Jeff, I appreciate it. What are a couple of things that you’d really like people to take away from this conversation we’ve had today?

Jeff Gilmer: Migrating a data center is not something that you do once a year, it’s not something you typically do once every five years, once every ten years, and it’s also the heartbeat of the technology, and the technology are the arms and legs and everything that makes your business run. You can’t kill the heart and expect the business to run, so when you go to make a data center or move a migration, do the planning. Make sure you have a proper design. Make sure you understand the requirements. Make sure you understand the ability, if you have a failure, what your fallback procedures are going to be. Make sure you have the resources to properly build a strategy, and additional resources that can properly help you with that migration. A data center failure is not what anybody wants to have today, and people just don’t realize it. I mean, think of it this way. How many of us today, if our phone quit working for 30 minutes, we’d be going crazy? We can’t see email, we can’t talk to anybody, we can’t send texts, we can’t check the weather, I can’t get directions to my next appointment, and on and on and on. Your data center is running that phone, think of it that way. If your phone completely shut down then you have to go backwards. Now, take that standpoint of your business shutting down. That’s really what we’re talking about here. Planning, scheduling, designing, proper resources that are all critical things that you need to consider to help avoid some of those three major issues we talked about today.

Data Center Spotlight: Well, Jeff, this is some good content, and I look forward to continuing this, because I know there’s a lot more mistakes that people make that you see on a regular basis.

Jeff Gilmer: That sounds great.

Data Center Spotlight: All right. Thanks Jeff, appreciate your time as always, looking forward to part two of this.

Jeff Gilmer: Thank you, Kevin.




Image © Copyright michael ely and licensed for reuse under this Creative Commons License.