IT infrastructure success cannot be achieved without accurately assessing future needs. But as fast as infrastructure options, virtualization, and server capacity are changing, how can you calculate your future data center needs? What are the mistakes made by others that you need to avoid?
Getting this wrong can hamstring your organization for years, either by underestimating your requirements, leading to a lack of capacity, or overbuilding and ending up with a white elephant data center burdening you with sunk cost. In part 3 of a series, Jeff Gilmer of Excipio Consulting, once again provides illuminating analysis on the factors to consider when making these critical decisions. He’ll give real-world examples and even some basic formulas that will put you on the right track to accurately projecting your future data center capacity requirement.
You can listen to the full conversation, or read the transcript below the player.
Kevin O’Neill, Data Center Spotlight: This is Kevin O’Neill with Data Center Spotlight. I have with me today Jeff Gilmer of Excipio Consulting, and we’re doing a series with Jeff on a number of data center planning and cloud and cloud sizing-type topics, and today’s topic is how you use technology to calculate future data center capacity requirements. Jeff, thank you again for joining us today.
Jeff Gilmer, Excipio Consulting: Yeah, thank you, Kevin, it’s always a pleasure to be here.
Data Center Spotlight: On this call, we are going to define what capacity is. We are going to help you determine the longevity of your data center for your planning purposes. We are going to talk about how IT impacts your data center capacity planning, and how you decide to size a data center. We’re also going to talk about, really, just the definition of data center capacity, and we’ll talk about a number of issues surrounding those things, so if you, within your organization, have some questions about what your future data center needs are going to be, there’s a very good chance the next half hour or so is going to help you out. Does that sound like a good agenda for us today, Jeff?
Jeff Gilmer: Sure, that sounds great. Just let me know which of those topics you’d like to discuss, and we’ll get into it.
Data Center Spotlight: Well, I think we’re going to try to hit a lot of them, but before we do, Jeff, can you tell me a little about you and Excipio, just for folks who maybe haven’t heard us before, about what you and your organization do?
Jeff Gilmer: Sure, so Excipio Consulting, we provide advisory services in a multitude of different areas related to technology. We have really six key solution suites, and within those six different solution suites, data center lifecycle management is one of those areas, and within data center lifecycle management, we help clients understand their overall data center strategy, whether that be an internal facility, whether it be the data center operations, so those functions that operate within the data center, the servers, the storage, the network as an example, all the way through from production, to testing, to disaster recovery, to cloud solutions, to external service providers, and anywhere in between. So, it’s really looking and helping clients develop a full data center strategy from today going into the future.
Data Center Spotlight: Terrific. Well, we’re happy to have you here, because we know this is a top-grade interest, we frequently hear people in organizations talking about their data center capacity requirements, and really the uncertainty surrounding what those future needs are going to be, and when someone is evaluating their data center capacity, Jeff, what are they referring to, and how do they know when they are going to run out of capacity? What are the factors that contribute to all of that, and that entire decision-making process?
Jeff Gilmer: Yeah, so when we look at data center capacity, we really boil it down to three key areas. Those areas are physical space within the data center, power capabilities, and cooling, how much can you really cool, related to the equipment that you have in the facility? So, in looking at an overall strategy, and looking at things such as the age of your current data center, how long is your data center going to last?
How much capacity do we need in our future data center requirements? We go back to the compute technology, which is really the basis for determining how much space, how much power, how much cooling we’re going to need from a capacity standpoint.
Data Center Spotlight: Okay, well, terrific. I was in a conversation a few months ago with the CIO of a very large company, and one of their biggest issues is determining how long their current data center will last. How long will it survive? What are the factors, and Jeff, how do you and the folks at Excipio, with your clients, determine what the timeframe is, and really the useful life of a data center?
Jeff Gilmer: Yeah, so again, we go back to those three capacity requirements, physical space, power, and cooling, and the reality today is, the majority of devices are shrinking in size, and becoming much more efficient. So, physical space is commonly not the determining factor for running out of capacity. It’s normally going to be either, they’re going to exceed their power capacity, or they’re not going to be able to cool the environment, depending how many devices and how dense their environment is. So, let’s just take the cooling one as an example for our discussion today.
You really need to start by understanding what your cooling capacity is, and that’s gathered through your facilities people, by looking at the total tons of cooling, based on what type of cooling units you have, what type of what’s commonly called the CRAC units are within the facility, or it might be in-row cooling, or it might be modular cooling, but there will be a certain cooling capacity based on the total tonnage in cooling terms, or BTUs is another way that people might look at it, and you need to calculate what is your capacity.
Then the second step that you need to do is you need to have a good understanding of your IT devices that are within that data center that are going to impact your cooling, so that would be things such as your servers. It would be your storage equipment, it would be your backup equipment, it might be your telecommunications. It might be a mainframe. It could be WAN devices, or LAN devices. It could be video. There’s a multitude of things, and each one of those has a higher impact or lower impact on cooling. Obviously, servers and storage probably have a higher cooling demand density, whereas things such as video or some of the LAN equipment, or WAN equipment for your network probably require less cooling requirements, so not generate as much heat.
By taking those devices, and looking at the watts per device, and then there are formulas you can use to take and calculate based on the average wattage used. You can then determine, what is the cooling requirement to cool all of your IT devices within that data center? So, the first number you want to achieve is by running the calculation of the watts per device, and again, it’s the watts that it’s actually drawing, not the watts that are on the name plate, and we can explain that in a minute or two here, and you determine what is required for cooling of the IT.
The second component to that is there’s an efficiency loss, so you’re going to take a look, and there are tools that can measure this, or you can just do a generic type of assumption, but look at your raised floor. Do you have cabling under the raised floor? Are you running water piping under the raised floor? Is there power being run under the floor? What is in there that might be impacting the efficiency of the cooling?You might have a loss anywhere from 10% to 40% of your estimated efficiency lost, just by how ineffective the under-floor cooling really is. So, you need to factor that component in as well.
And then what happens is, you need to look at your peak and your average demand, because there will be times, everybody comes in at 9 o’clock in the morning in the office, they all log into email, and all of a sudden, you have a huge compute demand. They’re logging into their applications, they’re logging into their information, or you may be running certain jobs in the evenings that might demand that. So in the end, you end up with a minimum cooling requirement, and a maximum cooling requirement, and you now compare that back to the tonnage that you have, and you can determine what your capacity is.
So, let’s just take an example of a client. We had a client, they had 700 server and mainframe devices within that data center. They were going to consolidate other data centers, and through that process of looking at their cooling capacity, we were able to determine that based on the maximum cooling capability of their data center at 1012 devices, based on their average watts per device, amongst those 700 servers they have today, that they were going to exceed their maximum cooling capacity. So, we then worked with them on how they could consolidate in another 300 servers, but at that point in time, they were either going to have to increase cooling capacity, or stop consolidating and bringing more equipment into that current data center.
Data Center Spotlight: Now, you touched on an interesting topic when you mentioned the watts that are actually being drawn, versus the wattage stated by the devices. Can you talk a little bit about that, between what actual usage is, versus what equipment is capable of handling?
Jeff Gilmer: Yeah, exactly. One of the, I guess I’d call it an error, one of the mistakes that we see people make when they start to go through this process, is they’ll take their physical servers, and they’ll total up their physical servers, and they’ll take the maximum wattage that is listed on the back of that server, and they’ll calculate that maximum wattage, times their number of servers, and say, here’s what we need for power, and here’s what we need for cooling. Well, that is going to be extremely conservative, or a very, generally what we would call an overuse of the power consumption, or over-determination of the cooling requirements.
The reality is, it’s based on the demand of that server. So, if we go back a few years, when people are running one application per server, that application was probably, could only, in a utilization standpoint, of maybe 3% to 5% of the demands of that server. Well, you can’t take the total wattage, you need to take a percentage of those total watts. Today, most people are virtualized, anywhere between, let’s just say 40% and 70% is a common range, there’s obviously people that are higher than that, but as we start to go through that calculation, if somebody is virtualized between 40% and 60%, their real power demand is probably between 40% and 60% of that power demand, of what they’re using on that server.
So, you look at the virtualization scheme, you look at the demand on that, and then you calculate the power from there. So, taking an example, if that server is drawing 1000 watts, but yet you’re at 40% virtualized, you’re probably only drawing 400 to 450, or maybe on peak demands, up to 600 watts at 60% range. That’s the figure you want to use when you start to calculate the power demands. You don’t want to use 1000 watts, you want to use a percentage of that based on your current virtualization strategies, and your current utilization on the server, and that holds true for your backup and your storage and your network devices as well, so you’re really coming at a percentage. In general, if you don’t know how to calculate that, but if you are commonly virtualized, what most clients are that we see, you probably want to use a range between 45% and 60% of that total number, when you start to do your power calculation.
Data Center Spotlight: Now Jeff, I’ve heard you use the phrase compute power demand, are you getting into some of the formulas as far as how you calculate the compute power demand?
Jeff Gilmer: Yeah, exactly, and that’s, again, I’ll just run through it at a high level here, we won’t get into the specifics on this call, Kevin, but what you get into is, you really need to make sure that you total up all of those devices that you have within that environment. So, you start looking at your total device count, and you also need to understand your growth. You’re going to have a growth over a period of time. So when you’re looking at calculating your future data center requirements, most data centers have a lifespan of around 15 years. That’s what you’re going to base it on. Just from a depreciation standpoint, a capitalization standpoint, and the financials based on the age of the equipment, when you’re going to commonly have to refresh the bulk of your cooling infrastructure, or your power infrastructure, 15 years is a common age denominator that is used within the industry.
So, by taking your device count, let’s say that you’ve got, whatever, 200, or 250 devices, you’re going to need to understand your growth. Now the reality today is, physical servers, physical storage, physical network gear are what we’re talking about. So if you’re virtualizing, and you’re adding another 100 virtual sessions, but they’re still on the same 200 servers that you have, that isn’t going to increase, necessarily, a significant amount of your power and cooling demand. What you’re really looking at is physical growth, and most physical growth today, in reality, is coming from storage, it’s not coming from the servers.
So you start to take into account, what is that growth going to be? It might be 2%, 3%, 5% devices over the time period on an annual basis of that data center. So you calculate that device growth, and you can get an idea as you begin to go through it. Maybe you’ve got 200 today, but maybe 15 years from now, you’re going to have 400 devices. It may grow in that type of dimension.
So then what you do, is you take the power calculations that we just talked about, and you can base it on the 45% to 60%, or 65% range if you’re in the common virtualization like most clients, and you can calculate your average watts per device, and then you can use a formula, and it’s actually taking your watts per device, times 3.412 to get your BTUs, and then you can divide that out and calculate your cooling tons. So now you’ve got your power consumption, your real power consumption. You’ve got your real cooling requirements, and then based on the number of the devices, depending on the power per rack that you’re going to be at, and commonly today data centers need a capacity average of 10 kW per rack. You can determine how many devices you can put into that rack based on that power consumption.
So, let’s put this into some real terms. Let’s take a client. The client that I’m going to talk about had 240 different devices. Of those devices, if you took the power supply, it was close to 300,000 watts. In reality, they were consuming closer to 130,000 watts. Their cooling requirements calculated out to 37 tons, and then there at 10 kW per rack, we were getting enough devices in that rack that we needed 14 racks for those 200 devices, and at 32 square feet a rack, now you can use 25 square feet, or you could use, 28, 30, 35, even on the high end of an efficient data center, but we used 32 for this client. They needed 450 square feet, and they needed to purchase 20 racks to begin with. So, we went through and worked with the company, and the CFO wanted to know, what’s going to be my capital spend as I start to do this future data center?
So you start with those 240 devices, and you roll out your growth, and he wanted to know, how much am I going to have to spend in year one? How much capital in year five? How much capital in year ten? And then, at the end of 15 years, where do I end up? Going through that process, as we talked about before, initially the client was purchasing 20 racks. In year five, they were going to have to purchase an additional six racks, and in year ten, an additional 12 racks. The bulk of these racks were to support storage now, not necessarily server growth, but based on this, the CFO can determine the capital requirements up front, there’s also capital requirements each five-year period, and make a financial analysis on this data center, and does it make sense for us to build this data center? Does it make sense for us to take these requirements that we’ve now defined, based on the power, cooling, number of racks, and square footage you’re going to need, and go out to an external colocation provider?
Does it make sense to take the financials based on this, and compare it to a cloud solution? There’s a multitude of ways you can go once you complete this type of analysis.
So, in summary, take your devices that you have in inventory, calculate the actual power consumed, calculate your cooling ton requirements, calculate the number of racks, and from there, you can calculate the square footage you’re going to need, and now you have the underlying requirements over your growth over the next 15 years, five years is probably going to be pretty accurate. Once you get beyond that, it might get a little gray in IT due to technology changes, but you have a solid basis to understand your financial cost for building a data center, upgrading a data center, utilizing a colocation, or moving to a private cloud, and you’re going to have an equal comparison, because you’re using the same standards across the board.
Data Center Spotlight: What’s interesting to me Jeff, as you talk about this, is the practice and what you folks do at Excipio is so specialized, and someone who went through a similar process five years ago, the service offerings available to them, when you mentioned public cloud, and private cloud, and what you’re going to keep the servers in, your sort of traditional servers you’re going to keep in your data center, and the percentage of virtualization you’re going to have overall throughout your organization. It seems as though it’s an entirely different world in doing this sort of analysis than it was even just four or five years ago. Is that an accurate interpretation?
Jeff Gilmer: Absolutely. I think back just a few years ago, whatever, four, five, six, seven years ago, there wasn’t virtualization. People were not virtualizing their servers. From a storage standpoint, you were getting one terabyte in a rack. Today, you’re getting, 10 to even 100 terabytes of storage in a rack. The devices have drastically been reduced in physical size.
We have clients today, as a matter of fact, we have one client that we converted 74 racks into one rack, and through the virtualization and the type of blade servers we incorporated, we were able to now get 1200 virtual servers within one rack, the power consumption has dropped drastically, think of 5 kW at 74 racks, and now you’re running 10 kW in one rack. So there’s a lot of significant changes in technology that are driving those data center decisions, and what you really need to think about is, who’s to say, take a look at your phone, Kevin. Your phone has more memory and more compute power in it than a server probably had five, six, seven years ago. Who’s to say servers aren’t going to be the size of your phone five years from now?
So, all of these variables come into play that you really need to understand your strategy, and you really need to understand where you’re going with this type of process, and granted we’ve taken, whatever, 15, 20 minutes here today, or a little longer, to talk about this at a high level.
We do put on workshops, and there’s several workshops that we’ve put on around the country, and these are two, three hour workshops where we invite people to come in, and go through and really understand all the different variables there are in the data center environment today. Take what we just talked about, and really break it down so people have a process, and a methodology to back to their organization and understand their data center demands and requirements.
Data Center Spotlight: In addition to the workshops you do, Jeff, I know you’ve shared with me, when we were talking about this, and planning for this interview, you shared some numbers with me, a little more detail that just don’t do as well when you’re in a conversation like we are. If someone wanted to find out your schedule of workshops, and some industry events you’re presenting at, or they just wanted to engage with you directly to get some more information regarding what we just discussed, what would be the best way for them to get in touch with you?
Jeff Gilmer: Well, the easiest way would be to go to our website, excipio.net. We have on there educational areas, we have seminars on there, we have examples of what we just talked about today that people can view, there’s a multitude of resources available at or through the website.
Data Center Spotlight: And also, if folks want to search here on datacenterspotlight.com and look for your name, Gilmer, there’s some of the other conversations we’ve had about these issues should prove pretty illuminating as well I would think, because you’ve done a nice job in outlining all these issues for us. You seem to speak every year in Charlotte at the, what’s the name of that event.
Jeff Gilmer: Critical Facilities Summit, it’s in Charlotte, it is the first week in October this year, and I will be speaking about two topics. One, what we’ve talked about today, we’ll actually be performing a workshop there on site, I believe it’s going to be on Monday afternoon, and people who attend, there’s no charge to attend the workshop are welcome to join, and then I’ll also be speaking about data center disaster recovery, and what does disaster recovery really mean, talking about the differences between an emergency management plan, between business continuity, and between disaster recovery, and helping people understand, what is really needed for disaster recovery, and what are the steps and processes you need to go through to identify how to recover in the case of a data center incident.
Data Center Spotlight: Good, well, I wanted to get out there maybe a little bit more than we have in our previous conversations about how people can access more information from you, because obviously, there are some leverage points within the analysis and work that you do that can just completely change the financial future of a company’s IT and operations, based on how you can help them with their capacity planning. You help them not get boxed in by not having enough capacity, and your analysis can help them from overspending and just having stranded capacity. I’ve enjoyed these conversations, Jeff, was there anything we talked about today that we haven’t touched on?
Jeff Gilmer: Well, unless you want to spend a couple more hours, but I think we’ve covered enough for today, Jeff.
Data Center Spotlight: [LAUGH] I think we’ve done enough to let people know that there’s a lot more work to be done for anyone who is dealing with this stuff in their job title. Jeff, as always, I enjoyed the conversation, I appreciate you being with us today.
Jeff Gilmer: Yeah, thank you, Kevin.