An Expert's Guide to Optimizing Virtualization Management

Introduction

Many organizations today have virtualized their environments in order to reduce costs and gain efficiencies. In fact, Gartner reports that at least 70 percent of all x86 servers are now virtualized.

However, for many organizations, realizing the benefits they expected from virtualization has proven elusive. Because provisioning virtual machines (VMs) is so easy, they find themselves suffering from VM sprawl — there are so many VMs that administrators cannot manage them effectively. After all, VMs have the same licensing, support, security and compliance issues that physical machines do. With hundreds or thousands of VMs running on dozens or hundreds of physical machines, many organizations are finding their operational expenses (OpEx) and capital expenditures (CapEx) actually growing rather than shrinking.

Quest Software is the world's largest provider of virtualization solutions, and we've talked with organizations around the world about their virtualization challenges. Top of mind for many of them is controlling costs by optimizing their virtualization management. This e-book explains five key ways you can control VM sprawl, optimize your virtualized environment and finally reap the benefits that led you to choose virtualization in the first place.

Strategy #1: Increase your VM density

What Is VM Density, And Why Does IT Matter?

The first strategy for optimizing your virtual environment is to measure and improve your VM density. VM density is easy to calculate, and it provides a quick way to evaluate how effectively you are using virtualization:

VM density = the number of VMs divided by the number of physical servers

Why does VM density matter? Here's an example to illustrate. Suppose you have 50 host machines running 500 VMs, which yields a VM density of 10. Increasing that VM density to 11 could have a big impact on both your OpEx and your CapEx because you could fit more virtual machines on the servers you already have. In this example, you'd be making space for the next 50 VMs you need — without having to buy and maintain another physical server.

90 percent of organizations over-allocate virtual resources, thereby unnecessarily limiting the number of virtual machines per physical machine.

HOW DOES YOUR VM DENSITY COMPARE TO SIMILAR VIRTUALIZED ENVIRONMENTS?

As a point of reference, Table 1 lists the average VM densities in Quest customer environments. Interestingly, we found that density increases with the size of the environment — but only up to a point. The largest VM environments actually see a drop in VM density.

Number of servers

Average VM density

Fewer than 200 VMs

10.5 VMs/server

200–1,000 VMs

17 VMs/server

1,001–2,000 VMs

18.5 VMs/server

2,001–3,000 VMs

26 VMs/server

3,001–4,000 VMs

38 VMs/server

More than 4,000 VMs

30 VMs/server

Table 1. Average VM density for small to large virtualized environments

By identifying zombie VMs and removing them from your environment, you can reduce OpEx and CapEx significantly.

Optimizing Your VM Density

How can you improve your VM density? Our experience is that 90 percent of organizations over-allocate virtual resources, thereby unnecessarily limiting the number of virtual machines per physical machine. In fact, nearly all organizations we've worked with have a single standard template for VM provisioning, so every VM gets the same resource allocation regardless of actual needs. Moreover, since they lack a way to assess actual resource requirements, if a user or DevOps team demands seemingly excessive VM resource allocations, they have no way to refute the request and therefore, again over-allocate resources.

By allocating resources more carefully, you can fit more VMs on each physical machine and thereby reduce costs. The three virtual resources particularly important to pay attention to are:

  • CPU — If you over-allocate vCPU in ESX, for example, the CPU scheduler in vSphere has to work harder to find available resources.
  • Memory — Over-allocating memory increases overhead because the VM swap file will have more activity.
  • Storage — Storage is the most frequently over-allocated resource and obviously limits VM density.

Finding the Right Solution to Help

Solutions are available to help you gain insight into your VM density. For example, Foglight for Virtualization provides an optimization feature that calculates average and peak VM CPU, memory and storage utilization over a time period that's meaningful for you. The default period is 30 days, but you can, for example, extend that to 90 days to account for quarter-end processing.

But the Quest solution goes further — it offers a customized recommendation for optimizing your VM resource allocations based on the activity in your unique environment. In practice, customers have been able to use this optimization feature to increase their VM density by one to two VMs per host. The more hosts you have, the greater the savings.

One small organization, for instance, was able to fit more than 200 additional VMs on the same hardware, while larger organizations have made room for even more.

Strategy #2: Reclaim your wasted disk space

Storage may be less expensive than ever, but it is far from free. By identifying the wasted disk space in your environment, you can make the most of the storage you already have, minimizing future expenditures. One of the most common misperceptions is that if you thin-provision disks within a storage array, there is wasted space you can reclaim.

Based on our experience with a broad range of virtual environments, we have compiled a list of the most common causes of wasted disk space:

  • Over-allocated virtual files — As discussed earlier, most organizations provision VMs using a standard template. This strategy requires giving every VM the maximum storage allocation that might be required by any VM, often 20 or 25 gigabytes. Since most VMs require far less, the result is a great deal of wasted disk space.
  • Abandoned VMs — Most organizations use vCenter to manage their VMware vSphere environments. But most do not realize that if they use vCenter to remove a VM from inventory rather than delete it, the VM is only removed from the vCenter inventory — the associated files are not actually deleted. Therefore, they have virtual machines taking up space they cannot even see in vCenter.
  • Unused templates — Organizations often have a large set of templates for deploying VMs. Templates that no one is using any longer take up space that could be put to better use.
  • Powered-off VMs — VMs take up space even when powered off. If a VM hasn't been powered on for a period of time — say a year or more — you can reclaim disk space by archiving or deleting it.
  • Snapshots that are left open — A snapshot that is left open will grow and grow and grow, which not only wastes space but degrades performance as well. We even see orphaned snapshots — if you are deleting snapshots in vCenter and a snapshot fails to merge with the base disk, the snapshot still exists, taking up space, but you cannot see or manage it in vCenter.

Finding the Right Solution to Help

Solutions are available to help you identify and reclaim the wasted disk space in your virtualized environment. Look for a solution that can not only find all the sources listed above, but help you reclaim the space. Foglight for Virtualization has found and reclaimed tens of thousands of terabytes of wasted disk space for organizations worldwide. For example, one customer in the education sector with 1,000 VMs was able to free up dozens of terabytes of storage, eliminating the need for additional storage expenditures for quite some time.

Strategy #3: Pull the plug on your zombie VMs

Reducing VM Sprawl and Associated Costs

So far, we've focused on optimizing the allocation of physical resources by increasing VM density and freeing up wasted disk space. Now let's turn our attention to reducing VM sprawl. Most organizations have dozens or hundreds of VMs that no one is using — in fact, our experience as shown that zombie VMs comprise, on average, a full 10 percent of an organization's VMs. This isn't really surprising, since VMs are easy to create and tend to live on even after projects are completed and staff move on to other priorities or even other jobs.

However, as noted earlier, these zombie VMs have the same licensing, support, security and compliance requirements as the VMs and physical machines that are actually delivering value to the organization. Therefore, by identifying and removing them from your environment, you can reduce OpEx and CapEx significantly.

Finding the Right Solution to Help

Of course, you need to be careful when decommissioning VMs; in particular, you want a tool that will help you be sure that no one is using a VM before you remove it. Foglight for Virtualization looks at resource utilization over a period of time — 30 days, 60 days, 90 days or whatever is comfortable for you — and identifies zombie VMs. Then you can either power them off and wait for the phone to ring, or be bold and simply delete them right away.

Foglight for Virtualization has enabled organizations to find and remove thousands of zombie VMs. The savings add up quickly. For example, if you have 500 VMs, you likely have 50 VMs that no one is using but that continue to consume resources. Removing them will give you space for 50 new, useful VMs — without any new expenditures.

Strategy #4: Enable each admin to manage more VMs — while still meeting your SLAs

Thanks to the combined forces of VM sprawl, tighter budgets and better management tools, most administrators are being asked to manage more and more VMs. Enterprise Management Associates (EMA) reports that the average admin-to-VM ratio is 1:77. Therefore, based on average salary figures, each additional VM adds an average of $881 in administrative staffing costs.

In the best performing organizations, however, EMA found that each administrator manages an average of 469 VMs — cutting the overhead added by each new VM to just $452. Top organizations have a ratio as high as 1:1,800, which reduces the staffing cost of a new VM to just $37.

The savings can be surprising. If you add 100 new VMs per year, increasing your VM-to-admin ratio from average (1:77) to best (1:469) will save you $42,900 a year in administrative overhead.

Finding the Right Solution to Help

Of course, you cannot sacrifice performance and availability when seeking these savings. But you don't have to. With the right tools, your admins can manage more VMs while continuing to meet your service level agreements (SLAs). Instead of managing by exception, look for advanced management solutions that enable you to proactively manage your environment and pinpoint emerging problems within seconds. You'll also want change and configuration management, capacity planning and inventory management capabilities.

Foglight for Virtualization does all that and more. The de facto standard for midsize to large virtual infrastructures, Foglight for Virtualization enables a single administrator to effectively manage hundreds of VMs, or more, with intelligent analytics and automation. Features include:

  • An intuitive graphical interface that highlights emerging issues
  • The ability to not only look at data in real time, but granularly go back in time to see how the environment was performing
  • Proactive on-screen, email and text alerts
  • Flexible automation of issue remediation, custom workflows and more

Moreover, the Foglight suite of solutions includes plug-ins (called cartridges) for Microsoft SQL Server, Exchange and Active Directory — three of the most virtualized applications — to deliver an enterprise-wide view through a single pane of glass. You'll be able to easily determine if you're having a virtualization problem or a problem with the application itself, so you can quickly pinpoint and resolve the root problem and maintain your SLAs — even if you're managing hundreds or thousands of VMs by yourself.

With the right tools, your admins can manage more VMs while continuing to meet your SLAs.

Strategy #5: Improve your capacity planning.

In non-virtualized environments, there is often a general understanding that requests for new services can take time to fulfill. For example, the IT shop of a major airline would plan for new applications months or even a year in advance in order to allow time for budget approvals, hardware acquisition and deployment, and so on.

Most virtualized environments play by a completely different set of rules. Employees expect the new VMs they need to be deployed almost immediately. Fulfilling these requests quickly helps the organization stay agile and competitive. But it puts new pressure on IT to improve capacity planning. You need to be able to keep on top of the answers to questions like the following:

  • How many additional VMs can we add to our environment with our current hardware?
  • What is the gating factor to growth? CPU? Memory? Storage?
  • What is the potential impact of adding a requested application? How can we prevent issues? For example, do we need to add more storage arrays?
  • How do we know when a host is running out of resources before it happens?

Finding the Right Solution to Help

With the right tools, you can plan effectively for future server and storage acquisitions, and know days or even weeks before you hit a CPU, memory or storage limitation. Foglight for Virtualization delivers intelligent analysis and automation capabilities to help you predict future hardware resource requirements, reserve capacity for planned VM deployments and automatically deploy VMs into reserved slots. Plus, it provides dynamic capacity planning to help you determine how to accommodate future workloads, as well as what-if scenario modeling to accurately forecast CapEx requirements.

Foglight for Virtualization

Thousands of organizations — including Quest itself — are already using Foglight for Virtualization to optimize their virtual environments and reap CapEx and OpEx savings. Let's recap a few of its key features:

Real-time and historical analysis

Foglight for Virtualization delivers enterprise-wide virtual monitoring from a single pane of glass. You can monitor the environment in real time or perform historical analysis of performance, resource utilization and more (see Figure 1).

Figure 1. Easily monitor your virtual environment in real time with Foglight for Virtualization.

Proactive, actionable insights

Today, administrators must manage more VMs, and most must manage VMs from multiple vendors. With Foglight for Virtualization, you don't need to be a master of both Hyper-V and VMware, for instance. The solution not only identifies emerging issues and issues appropriate alerts, it also provides expert advice to help you resolve problems quickly and proactively prevent future issues (see Figure 2).

Figure 2. Foglight for Virtualization issues alarms to alert you to issues in real time and provides expert advice to help you resolve them quickly.

Analyze and forecast capacity trends

Put away your Excel spreadsheets. With Foglight for Virtualization, you can view current resource consumption in real time, forecast future resource requirements and even model what-if scenarios to understand the potential impact of possible changes to the environment (see Figure 3).

Figure 3. What-if scenario modeling helps you understand the potential impact of changes to the environment.

Optimize and reduce data center total cost of ownership

Foglight for Virtualization helps you increase VM density and control OpEx by right-sizing CPU, memory and storage; reclaiming wasted disk space from abandoned VMs, unused templates, powered-off VMs and unneeded snapshots; and identifying and deleting zombie VMs.

Foglight for Virtualization helps you increase VM density and control OpEx. Foglight for Virtualization not only identifies emerging issues and issues appropriate alerts, it also provides expert advice to help you resolve problems quickly and proactively prevent future issues.

By heeding the five guidelines detailed in this e-book and choosing the right tools, you can become a virtualization management expert.

Automation You Can Trust

Foglight for Virtualization enables you to automate common tasks, automate remediation for particular alarms, proactively optimize virtual resources, easily create custom workflows and more.

Conclusion

An optimized, cost-effective virtualized environment is within your grasp. By heeding the five guidelines detailed in this e-book and choosing the right tools, you can become a virtualization management expert.

For more information about Foglight for Virtualization, please visit quest.com/solutions/virtualization-management.

About Quest

Quest helps our customers reduce tedious administration tasks so they can focus on the innovation necessary for their businesses to grow. Quest® solutions are scalable, affordable and simple-to-use, and they deliver unmatched efficiency and productivity. Combined with Quest's invitation to the global community to be a part of its innovation, as well as our firm commitment to ensuring customer satisfaction, Quest will continue to accelerate the delivery of the most comprehensive solutions for Azure cloud management, SaaS, security, workforce mobility and data-driven insight.