Invisible GPUs are Coming

With NVIDIA's new Kepler architecture, graphics processing units move from the physical to the virtual world.

With NVIDIA's new Kepler architecture, graphics processing units move from the physical to the virtual world.

By Kenneth Wong

If you’re a system administrator or IT manager, you have undoubtedly encountered invisible machines. Invisible machines, or virtual machines, have become a standard part of enterprise IT architecture. With hardware virtualization, a single workstation could become the host for several guests, or virtual machines. Each guest appears as an individual machine in the network;  each performs and executes the designated user’s commands as if it were an autonomous machine, running its own operating system. Spawned from the processing power, memory and storage capacity of a single machine, these pseudo-machines are, in reality, clusters of designated computing resources pretending to be machines.

NVIDIA CEO Jen-Hsun Huang at the GPU Technology Conference 2012, getting ready to demonstrate features in NVIDIA’s Kepler architecture.

With improvements in chip architecture from Intel and AMD, it became possible to virtualize the CPU about six years ago. This feature in modern CPUs allows several virtual machines to share the host machine’s processing power, representing a giant leap in hardware virtualization.

But the graphics processing unit (GPU)  remained locked out of the virtual world—until now.

NVIDIA CEO Jen-Hsun Huang broke the news to the crowd at the company’s annual GPU Technology Conference (GTC). “I want to announce a GPU that we can all simultaneously share. Today, we’re going to take the GPU into the cloud,” he said. “For the first time, we’ve virtualized the GPU.”

Made Possible by Kepler

By Huang’s own admission, the GPU virtualization project took a good five years. Eliot Eshelman, senior technical account manager at Microway, explains the difficulty in making the GPU invisible: “Many typical office desktop systems have already been moved to the datacenter using virtualized desktop infrastructure (VDI). However, intensive GPU hardware acceleration used a driver model that required the OS to monopolize access to the device. It meant that, at best, one virtual machine had access the GPU hardware acceleration; in many cases, GPU hardware acceleration wasn’t available to the virtual machines at all.”

With NVIDIA’s next-generation GPU architecture “Kepler,” the NVIDIA VGX GPU Hypervisor (when integrated into the virtual machine manager) has a way to share a GPU among several users. According to NVIDIA, “In the case of VDI, it enables access to a single GPU from many virtual machines running on a single server, while also ensuring protected access of critical resources.”

Huang declared, “Every virtual machine now gets to have a GPU to itself, just as it does a CPU to itself.”

The other Kepler advantage, its energy efficiency, is also expected to appeal to data center operators.

Virtualization’s Promise


Intel’s Phi Signals New Phase in HPC

Intel’s next-generation chip based on the company’s many-integrated core (MIC) architecture is about to become reality. Expected to ship by the end of 2012, Intel Xeon Phi is what the company has up to this point referred to only by the codename Knights Corner. With Phi, Intel leaps from multiple cores (two, four, six, or eight cores) to more than 50 cores in a single product. Marketed as a coprocessor, Phi is a departure from Intel’s previous products, which are meant to be central or primary processing units. Phi and other MIC products will, in Intel’s words, “complement the existing Intel Xeon processor E5-2600/4600 product families.” As a coprocessor, Phi will take on highly parallel computing jobs, the territory of high performance computing (HPC).Intel
Phi is outfitted to do battle where graphics accelerators (GPUs) have been aggressively campaigning for dominance. But the CPU maker’s product may have one advantage its rivals don’t—the x86 programming environment that’s ready to accept codes written in C, C++, Fortran, and other common languages. Rival HPC products from NVIDIA, for instance, require mastering the GPU-specific CUDA programming environment. In a subtle jab at competing coprocessors, Intel declares its MIC architecture offers “more flexibility when implementing cluster solutions that are not available with alternative graphics accelerator-based technologies.”
Some engineering software, such as ANSYS’s simulation packages, are designed to take advantage of HPC clusters equipped with many processing cores, but adapting programs originally developed for the CPU to run on the GPU is not without challenges. A coprocessor with a simpler, easier programming environment can significantly lower the barrier to HPC and open up new possibilities.
The first Phi-powered test cluster is up and running and is delivering 118 TFLOPs of performance, according to Intel. But the first public evaluation of Phi’s performance will be Stampede, a Petascale supercomputer in Texas Advanced Computing Center that’s expected to go online in early 2013. For more on Intel Xeon Phi, read “Products based on Intel Many Integrated Core architecture scheduled for release this year” and “Intel Xeon Phi Enters Parallel Quest for HPC Dominance”).

Virtualization is driven by, among other things, convenience and cost efficiency. With fewer physical machines to acquire and manage, network administrators can control cost and streamline IT operations. With the proliferation of mobile devices in enterprise IT—described by some as the “bring-your-own-device,” or BYOD movement—virtualization offers new promises. You could, for example, use an iPad to remotely interact with a virtual machine to run GPU-accelerated applications.

Sumit Dhawan, Citrix’s general manager and vice president, attributes the increasing demand for VDI to, among other things, “the need for people to work and play from anywhere. Desktop virtualization is the perfect solution. People can bring in any device, and have the flexibility to work from anywhere.”

One of the features of Kepler,  according to NVIDIA, is Hyper-Q—the ability to communicate with multiple CPU cores simultaneously. The NVIDIA Kepler GPU family includes the GeForce GTX 680 GPU, targeted at media and entertainment, and the Tesla K10 and K20 accelerators, for HPC.

“A single Tesla K10 accelerator board features two GK104 Kepler GPUs that deliver an aggregate performance of 4.58 teraflops of peak single-precision floating point and 320GB per second memory bandwidth,” according to NVIDIA. “]K20]  delivers three times more double precision compared to Fermi architecture-based Tesla GPUs, and it supports the Hyper-Q and dynamic parallelism capabilities.” Dynamic parallelism in Kepler lets the GPU dynamically spawn new work threads on its own without going back to the CPU.

The Performance Lynchpin

With virtualized IT setups, the speed of interconnects—the physical connectors or wireless connectors that allow client devices to communicate with the host machine—plays an important role in delivering acceptable or exceptional performance.

The difference between a virtual GPU’s performance and a dedicated GPU’s performance may also affect the type of applications you can choose to run. For instance, will running computer fluid dynamics (CFD) software on a virtual GPU from a remote client deliver the same performance as running it on a dedicated physical GPU in a workstation? It’s a question that’s difficult to answer until early adopters have gathered enough data.

Kenneth Wong has been a regular contributor to the CAD industry press since 2000—first as an editor, later as a columnist and freelance writer for various publications. During his nine-year tenure, he has closely followed the migration from 2D to 3D, the growth of PLM, and the impact of globalization on manufacturing. Email him at [email protected]  or visit

More Info

Advanced Micro Devices

Citrix Systems




Share This Article

Subscribe to our FREE magazine, FREE email newsletters or both!

Join over 90,000 engineering professionals who get fresh engineering news as soon as it is published.

About the Author

Kenneth Wong's avatar
Kenneth Wong

Kenneth Wong is Digital Engineering’s resident blogger and senior editor. Email him at [email protected] or share your thoughts on this article at

      Follow DE