Virtualization has become a mainstream technology that allows efficient and safe resource sharing in data centers. In this paper, we present a large scale workload characterization study of 90K virtual machines hosted on 8K physical servers, across several geographically distributed corporate data centers of a major service provider. The study focuses on 19 days of operation and focuses on the state of the practice, i.e., how virtual machines are deployed across different physical resources with an emphasis on processors and memory, focusing on resource sharing and usage of physical resources, virtual machine life cycles, and migration patterns and their frequencies. This paper illustrates that indeed there is a huge tendency in over-provisioning CPU and memory resources while certain virtualization features (e.g., migration and collocation) are used rather conservatively, showing that there is significant room for the development of policies that aim to reduce operational costs in data centers.@en

Concurrent execution of multiple applications leads to varying partial utilization of shared resources. Understanding system behavior in these conditions is essential for making concurrent execution efficient. Unfortunately, anticipating behavior of shared resources at partial utilization in complex systems is difficult, realistic experiments that reproduce and examine such behavior are therefore needed. To facilitate experiments at partial utilization, we present a tool that accurately controls the processor utilization of arbitrary concurrent workloads, either establishing constant partial load or replaying a variable load trace. We validate the ability of the tool to enforce the configured partial utilization on multiple platforms, and use the tool to collect novel information on system behavior at partial utilization levels. In detail, our experiments show how to examine the complex relationship between utilization and throughput, useful for tasks such as performance debugging or system dimensioning, and we show this relationship for the DaCapo benchmarks. Further, we show that CPU pinning (a technique used to improve workload isolation) can benefit from dynamic response to system utilization, improving system efficiency with partial utilization. Finally, we show that the overhead of virtualization also changes with partial utilization and CPU allocation.@en