DE · Topics ·

Getting the Bigger Performance Picture

Professional benchmark software only tells part of a workstation’s performance story.

Professional benchmark software only tells part of a workstation’s performance story.

Holomark 2 is created by Rhino 3D users. The Cooper Mini model in the benchmark has been modified in hundreds of ways by Rhino users. Image courtesy of Robert McNeel and Associates.

Professional engineering workstations provide a high level of power and performance for engineering, simulation and design software. The trick is to make sure any workstation you consider for purchase is a good fit for your specific requirements.

A variety of benchmarking software tools are available to evaluate workstations. They test specific performance issues, but they don’t tell the whole story of how any model will run in all use cases. It is generally a good idea to use a combination of different benchmarking tools to get a comprehensive understanding of a workstation’s performance. There is a list of the most popular and valuable workstation benchmarks at the end of this article.

The workstation vendors we spoke with recommend these standard benchmarks but are quick to point out they can’t tell the whole story.

“Even the SPECapc benchmarks only test a subset of the available features in each application,” says Brian Bothwell, advisory performance engineer at Lenovo. “While a good approximation of performance can be made, I don’t think any premade benchmark will be able to cover entire workflows, even specific to one application.”

The various benchmark programs from SPEC are used by all major workstation vendors and thousands of customers. Image courtesy of SPEC.

There are other issues to consider beyond how a computer runs under any particular benchmark. “There are two main concerns for most customers,” says Scott Hamilton, a senior consultant for Precision workstations at Dell. “First, is the workstation configuration tested and certified? Second, what is the best configuration for the application and workflow used?”

Hamilton says Dell maintains a website to look up certification for the most common applications ( Software vendors provide similar offerings.

There are additional ways to test how a particular workstation will run using existing software. “If the user is already using the application, they can leverage their own testing by using Windows Performance Monitor to see what is happening while using different workflows within the app itself,” says Bill Martin-Otto, senior workstation performance manager at Lenovo. “This will allow them to determine if they are running out of memory or maxing out the CPU or [graphics processing unit (GPU)]. It can also tell them how much bandwidth they are using on the storage device.”

Fit for Multipurpose

Most workstations often serve more than one purpose, yet software engineers have differing requirements.

“Experienced users doing CAD modeling with complicated models understand the application processes are mostly single threaded, and if it is rendering, then it is a multi-threaded CPU process,” says Pradeep Ramineni, senior performance engineer at Lenovo.

CAE tools for simulation and engineering present a different set of challenges.

“It really depends on how complicated the user models are and what solvers are used,” Ramineni says. “Typically, the CAE application licenses are more expensive than the workstation hardware used to run these applications. Though CAE applications are heavily multithreaded processes and memory bandwidth/capacity dependent, a system configured for CAD would be good enough for simple models. However, users need to better understand their application licensing and models’ complexity and use a combination of [independent software vendor (ISV)] support, benchmark data and white papers in the public domain to configure a workstation that fits their workflow requirements.”

Most hardware vendors are members of the Standards Performance Evaluation Corporation (SPEC). It is a nonprofit that publishes and promotes the use of standardized benchmark testing to evaluate compute performance and energy efficiency. Membership provides funding for SPEC, but there are other reasons vendors participate.

“We gain a deeper understanding of workstation class workloads, including financial services, CPU and GPU rendering, compute, interactive graphics, and others,” Lenovo’s Martin-Otto says. “Our participation in SPEC also keeps us more up to date on changes with various applications and workloads that are used in the SPEC/Graphics and Workstation Performance Group benchmarks and allows us to provide better guidance for our customers. Since the benchmarks are developed with our competitors it provides results, which are vendor agnostic.”

SolidWorks 2020 is one of several SPEC benchmark routines that tests for one specific product. Images courtesy of SPEC.


Testing the Broader Workflow

Existing benchmark tools test specific performance issues inside the workstation. But no one works in isolation these days. Workstations are on local networks, and often work with remote cloud services. Now, there is no equivalent to a downloadable benchmark to test for such system-wide performance.

“Most benchmarks test one subsection of an entire workflow,” says Pete Seidel, senior performance engineer at Lenovo. “The best way to evaluate an entire workflow is to run it on a few different hardware configurations to determine which one performs the best for a given price range. This method would be difficult and expensive for a customer to do on their own. Lenovo offers customers the ability to try their specific workflows in our performance lab on numerous hardware configurations.” Other large vendors have similar performance labs.

Dell’s Hamilton offers an idea on how to use benchmark software to examine broader performance issues.

“Users can find a benchmark that will measure the specific application workload like those provided by SPEC, then test all the applications to determine maximum requirements and identify bottlenecks so they can specify a system that meets all of their needs,” Hamilton says. “Also, if they are running some applications simultaneously then they should run simultaneous benchmarks as well to represent their true working conditions most accurately.”

Special Consideration for Mobile Workstations

Mobile workstations have a unique set of performance settings and considerations due to power and thermal design limitations.

“Buyers need to understand more about how their mobile workstation is designed than they would when running a desktop,” Lenovo’s Martin-Otto says.

“When running benchmarks on mobile workstations, you need to make sure you are running it in the highest performance mode possible—which is also the loudest acoustics,” Martin-Otto says. Just make sure the workstation is plugged in, as operating on battery power will result in much lower performance.

Graphics is another area where mobile workstations pose issues. For example, some mobile workstations use hybrid graphics, mixing integrated and discrete graphics processors for power efficiency. In some situations, using an external monitor instead of the built-in display may force the dedicated GPU to render everything on the screen.

More Dell Coverage

Accelerating and Advancing CAE
New ebook from Jon Peddie Research (JPR) explains how GPU acceleration is improving CAE workflows.
NVIDIA Powers Generative AI Advances
NVIDIA is teaming with Microsoft and Dell to accelerate generative AI initiatives in the enterprise.
Justifying Engineering Workstation Upgrades
New workstations sporting the latest NVIDIA RTX™ GPUs provide a strong return on investment.
Configuring a Workstation for Simulation
Advanced GPUs and new solvers that take advantage of GPU acceleration are enabling faster and more accurate simulation.
NVIDIA GTC 2023: GPUs Enable New Advanced Workflows
AI workloads set to become part of design and simulation thanks to advanced GPUs
Dell Debuts Powerful Next-Generation Workstations
New Dell Precision tower and mobile workstations harness the power of the latest generation of NVIDIA RTX™ Ada Generation GPUs and 13th-generation Intel processors.
Dell Company Profile

More Lenovo Coverage

Making the Case for Mobile Engineering Workstations
This new Making the Case paper explains how mobile engineering workstations from Lenovo can improve performance for engineers on the go.
Lenovo ThinkReality VRX Is Now Available
Virtual reality headset comes in a compact form factor with mixed reality capabilities for immersive training, collaboration and 3D design.
Editor’s Picks: March 9-15, 2023
One of the picks will be chosen as DE’s Editor’s Pick of the Week.
Lenovo Powers Up ThinkStation Line 
ThinkStation PX, P7 and P5 workstations feature high-end chips from Intel, NVIDIA Ada GPUs and a revamped chassis designed by Aston Martin.
Editor’s Picks: January 6-12, 2023
One of the picks will be chosen as DE’s Editor’s Pick of the Week.
ThinkPad X1, Lenovo Go Power Hybrid Working
Lenovo unveils an array of product offerings at CES 2023, including the updated ThinkPad X1 portfolio.
Lenovo Company Profile

Share This Article

Subscribe to our FREE magazine, FREE email newsletters or both!

Join over 90,000 engineering professionals who get fresh engineering news as soon as it is published.

About the Author

Randall  Newton's avatar
Randall Newton

Randall S. Newton is principal analyst at Consilia Vektor, covering engineering technology. He has been part of the computer graphics industry in a variety of roles since 1985.

  Follow DE